2024-04-16T05:11:02.560 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-16T05:11:02.561 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901 --name mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi --owner scheduled_mchangir@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --verbose -- /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/orig.config.yaml 2024-04-16T05:11:02.622 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T05:11:02.709 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901 branch: qa-debug-resource-temporarily-unavailable-issue description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: mchangir@redhat.com first_in_suite: false job_id: '7657901' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: qa-debug-resource-temporarily-unavailable-issue ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a87fcff6340baeedd43af126e793f3241b6400ea subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: a87fcff6340baeedd43af126e793f3241b6400ea kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: qa-debug-resource-temporarily-unavailable-issue sha1: a87fcff6340baeedd43af126e793f3241b6400ea owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2991 sha1: a87fcff6340baeedd43af126e793f3241b6400ea sleep_before_teardown: 0 suite: fs suite_branch: qa-debug-resource-temporarily-unavailable-issue suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a87fcff6340baeedd43af126e793f3241b6400ea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBELp0o3cUKATNtjsL3TDTiIaIDukpq5xX+mE9ecn5vDiuGVcnvKpXedNoveEOYxOZM33r8t26wnPfue9IWwUo9s= smithi194.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZ5w4slM01MMsoRhPYMQmhAnQjuEpb16pq6SzGNOgkPZ7hZZSOHRzGZhijmx9rovC3iPgWBQWNYHsivIjNC82E= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCxmS4ZcQk2bk5TqIuT7tppLPsGHAV4W2UyGB3dV1fKQzndrg4QxYtmHeAYV4Zyk5/b1tKcx9J9jcWbX+nPHFRQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-16_05:02:59 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-16T05:11:02.709 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa; will attempt to use it 2024-04-16T05:11:02.710 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks 2024-04-16T05:11:02.710 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-16T05:11:02.711 INFO:teuthology.task.internal:Checking packages... 2024-04-16T05:11:02.737 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a87fcff6340baeedd43af126e793f3241b6400ea' 2024-04-16T05:11:02.738 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-16T05:11:02.738 INFO:teuthology.packaging:ref: None 2024-04-16T05:11:02.738 INFO:teuthology.packaging:tag: None 2024-04-16T05:11:02.738 INFO:teuthology.packaging:branch: qa-debug-resource-temporarily-unavailable-issue 2024-04-16T05:11:02.738 INFO:teuthology.packaging:sha1: a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:11:02.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=qa-debug-resource-temporarily-unavailable-issue 2024-04-16T05:11:02.881 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3101.ga87fcff6 2024-04-16T05:11:02.883 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-16T05:11:02.889 INFO:teuthology.task.internal:no buildpackages task found 2024-04-16T05:11:02.889 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-16T05:11:02.947 INFO:teuthology.task.internal:Saving configuration 2024-04-16T05:11:02.968 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-16T05:11:02.974 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T05:11:03.000 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:48.255039', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBELp0o3cUKATNtjsL3TDTiIaIDukpq5xX+mE9ecn5vDiuGVcnvKpXedNoveEOYxOZM33r8t26wnPfue9IWwUo9s='} 2024-04-16T05:11:03.025 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:48.256297', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZ5w4slM01MMsoRhPYMQmhAnQjuEpb16pq6SzGNOgkPZ7hZZSOHRzGZhijmx9rovC3iPgWBQWNYHsivIjNC82E='} 2024-04-16T05:11:03.048 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:48.253682', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCxmS4ZcQk2bk5TqIuT7tppLPsGHAV4W2UyGB3dV1fKQzndrg4QxYtmHeAYV4Zyk5/b1tKcx9J9jcWbX+nPHFRQ='} 2024-04-16T05:11:03.048 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-16T05:11:03.054 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-16T05:11:03.054 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-16T05:11:03.055 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-16T05:11:03.055 INFO:teuthology.run_tasks:Running task console_log... 2024-04-16T05:11:03.228 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3323f570d0>, signals=[15]) 2024-04-16T05:11:03.229 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-16T05:11:03.235 INFO:teuthology.task.internal:Opening connections... 2024-04-16T05:11:03.235 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:11:03.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:11:03.329 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2024-04-16T05:11:03.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:11:03.417 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-04-16T05:11:03.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:11:03.503 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-16T05:11:03.510 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-04-16T05:11:03.528 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-04-16T05:11:03.528 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-04-16T05:11:03.589 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T05:11:03.590 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-04-16T05:11:03.590 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T05:11:03.590 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T05:11:03.590 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T05:11:03.590 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-04-16T05:11:03.609 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2024-04-16T05:11:03.627 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2024-04-16T05:11:03.627 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-04-16T05:11:03.686 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="CentOS Stream" 2024-04-16T05:11:03.686 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="9" 2024-04-16T05:11:03.686 INFO:teuthology.orchestra.run.smithi194.stdout:ID="centos" 2024-04-16T05:11:03.686 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="rhel fedora" 2024-04-16T05:11:03.686 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="9" 2024-04-16T05:11:03.686 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el9" 2024-04-16T05:11:03.686 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T05:11:03.687 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2024-04-16T05:11:03.687 INFO:teuthology.orchestra.run.smithi194.stdout:LOGO="fedora-logo-icon" 2024-04-16T05:11:03.687 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T05:11:03.687 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://centos.org/" 2024-04-16T05:11:03.687 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T05:11:03.687 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T05:11:03.687 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T05:11:03.687 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-04-16T05:11:03.709 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-16T05:11:03.728 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-16T05:11:03.728 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-16T05:11:03.785 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-04-16T05:11:03.785 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-04-16T05:11:03.785 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-04-16T05:11:03.785 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-04-16T05:11:03.785 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-04-16T05:11:03.785 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-04-16T05:11:03.786 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T05:11:03.786 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-16T05:11:03.786 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-04-16T05:11:03.786 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T05:11:03.786 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-04-16T05:11:03.786 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T05:11:03.786 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T05:11:03.786 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T05:11:03.786 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-16T05:11:03.809 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-16T05:11:03.820 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-16T05:11:03.841 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-16T05:11:03.841 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-04-16T05:11:03.844 DEBUG:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2024-04-16T05:11:03.846 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-04-16T05:11:03.861 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-16T05:11:03.980 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-16T05:11:03.981 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-04-16T05:11:03.983 DEBUG:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2024-04-16T05:11:03.985 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-04-16T05:11:04.090 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-16T05:11:04.161 INFO:teuthology.run_tasks:Running task kernel... 2024-04-16T05:11:04.177 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-16T05:11:04.177 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-16T05:11:04.177 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-16T05:11:04.178 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T05:11:04.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:11:04.195 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-16T05:11:04.251 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-361.el9.x86_64 2024-04-16T05:11:04.251 INFO:teuthology.task.kernel:Running kernel on smithi086: 5.14.0-361.el9.x86_64 2024-04-16T05:11:04.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum install -y kernel 2024-04-16T05:11:06.322 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-16T05:11:08.448 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - BaseOS 2.0 MB/s | 2.0 MB 00:01 2024-04-16T05:11:10.155 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.0 MB 00:01 2024-04-16T05:11:12.333 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-16T05:11:16.323 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-16T05:11:17.974 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-16T05:11:19.494 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-16T05:11:24.408 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-16T05:11:26.765 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-16T05:11:26.765 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-16T05:11:26.864 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-16T05:11:26.870 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-16T05:11:26.870 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-04-16T05:11:26.870 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout:Install 4 Packages 2024-04-16T05:11:26.871 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:11:26.872 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 77 M 2024-04-16T05:11:26.872 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 123 M 2024-04-16T05:11:26.872 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-04-16T05:11:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 1.5 MB/s | 277 kB 00:00 2024-04-16T05:11:27.669 INFO:teuthology.orchestra.run.smithi086.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 19 MB/s | 15 MB 00:00 2024-04-16T05:11:28.185 INFO:teuthology.orchestra.run.smithi086.stdout:(3/4): kernel-modules-core-5.14.0-438.el9.x86_6 24 MB/s | 28 MB 00:01 2024-04-16T05:11:28.527 INFO:teuthology.orchestra.run.smithi086.stdout:(4/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 20 MB/s | 34 MB 00:01 2024-04-16T05:11:28.530 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:11:28.531 INFO:teuthology.orchestra.run.smithi086.stdout:Total 46 MB/s | 77 MB 00:01 2024-04-16T05:11:28.531 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-04-16T05:11:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-04-16T05:11:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-04-16T05:11:29.732 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-04-16T05:11:29.732 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-04-16T05:11:32.059 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-04-16T05:11:32.780 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-16T05:11:32.869 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T05:11:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T05:11:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T05:11:40.468 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T05:11:40.502 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:11:45.288 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:14.780 INFO:teuthology.orchestra.run.smithi086.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-16T05:12:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:12:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:15.226 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:15.226 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-16T05:12:15.226 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T05:12:15.226 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T05:12:15.638 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:15.639 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:12:15.639 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-04-16T05:12:15.639 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-16T05:12:15.639 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-16T05:12:15.639 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-16T05:12:15.639 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-16T05:12:15.639 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:12:15.639 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-16T05:12:15.970 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T05:12:16.076 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T05:12:16.076 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T05:12:16.077 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-16T05:12:16.077 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T05:12:16.077 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T05:12:16.077 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-16T05:12:16.077 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T05:12:16.077 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T05:12:16.156 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T05:12:16.157 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T05:12:16.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-04-16T05:12:16.191 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-04-16T05:12:16.191 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-04-16T05:12:16.191 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.06 2024-04-16T05:12:16.191 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 68.el9 2024-04-16T05:12:16.191 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Group : Unspecified 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 8264254 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : builder@centos.org 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : CentOS 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-04-16T05:12:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:12:16.193 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-16T05:12:16.193 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-16T05:12:16.193 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-16T05:12:16.193 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-04-16T05:12:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:12:16.193 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-04-16T05:12:16.194 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-16T05:12:16.194 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 5.14.0-438.el9.x86_64 2024-04-16T05:12:16.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-16T05:12:16.779 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-16T05:12:18.282 INFO:teuthology.orchestra.run.smithi086.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-16T05:12:18.332 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-16T05:12:18.334 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-16T05:12:18.351 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.HXuUmyW9LP 2024-04-16T05:12:18.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HXuUmyW9LP 2024-04-16T05:12:18.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.HXuUmyW9LP 2024-04-16T05:12:18.554 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.HXuUmyW9LP is 6KB 2024-04-16T05:12:18.602 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.HXuUmyW9LP 2024-04-16T05:12:18.616 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-04-16T05:12:18.681 INFO:teuthology.orchestra.run.smithi086.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-16T05:12:18.681 INFO:teuthology.orchestra.run.smithi086.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-16T05:12:18.681 INFO:teuthology.orchestra.run.smithi086.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-16T05:12:18.681 INFO:teuthology.orchestra.run.smithi086.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-16T05:12:18.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-16T05:12:18.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-04-16T05:12:48.840 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-16T05:12:48.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-16T05:12:48.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:13:07.359 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-16T05:13:16.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-16T05:13:16.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:13:19.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-16T05:13:31.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-16T05:13:31.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:14:03.272 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-16T05:14:03.723 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-16T05:14:03.724 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-438.el9.x86_64"... 2024-04-16T05:14:03.724 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-16T05:14:03.739 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-438.el9.x86_64 2024-04-16T05:14:03.739 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-16T05:14:03.739 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T05:14:03.739 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-16T05:14:04.740 DEBUG:teuthology.parallel:result is None 2024-04-16T05:14:04.740 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-16T05:14:04.748 INFO:teuthology.task.internal:Creating test directory... 2024-04-16T05:14:04.748 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T05:14:04.751 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T05:14:04.754 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T05:14:04.772 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-16T05:14:04.780 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-16T05:14:04.808 INFO:teuthology.task.internal:Creating archive directory... 2024-04-16T05:14:04.808 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T05:14:04.811 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T05:14:04.821 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T05:14:04.933 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-16T05:14:04.939 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-16T05:14:04.940 DEBUG:teuthology.orchestra.run.smithi194:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T05:14:04.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:14:04.959 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T05:14:04.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:14:04.979 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T05:14:04.985 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T05:14:05.002 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T05:14:05.046 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:14:05.053 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:14:05.080 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:14:05.092 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:14:05.116 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:14:05.474 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:14:05.477 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-16T05:14:05.484 INFO:teuthology.task.internal:Configuring sudo... 2024-04-16T05:14:05.484 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T05:14:05.486 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T05:14:05.490 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T05:14:05.534 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-16T05:14:05.549 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-16T05:14:05.549 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T05:14:05.554 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T05:14:05.563 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T05:14:05.581 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:14:05.670 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:14:05.737 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:14:05.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:14:05.850 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:14:05.850 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T05:14:05.896 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:14:05.945 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:14:06.008 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:14:06.061 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:14:06.100 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:14:06.100 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T05:14:06.171 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:14:06.227 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:14:06.280 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:14:06.329 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:14:06.367 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:14:06.368 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T05:14:06.436 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-04-16T05:14:06.439 DEBUG:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2024-04-16T05:14:06.441 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-04-16T05:14:06.482 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:14:06.515 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:14:06.566 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:14:07.096 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-16T05:14:07.106 INFO:teuthology.task.internal:Starting timer... 2024-04-16T05:14:07.107 INFO:teuthology.run_tasks:Running task pcp... 2024-04-16T05:14:07.146 INFO:teuthology.run_tasks:Running task selinux... 2024-04-16T05:14:07.179 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-16T05:14:07.180 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-04-16T05:14:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-04-16T05:14:07.230 DEBUG:teuthology.orchestra.run.smithi194:> sudo service auditd rotate 2024-04-16T05:14:07.282 INFO:teuthology.orchestra.run.smithi194.stdout:Rotating logs: 2024-04-16T05:14:07.285 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-04-16T05:14:07.346 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-04-16T05:14:07.348 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-16T05:14:07.348 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-04-16T05:14:07.376 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-04-16T05:14:07.377 DEBUG:teuthology.orchestra.run.smithi194:> /usr/sbin/getenforce 2024-04-16T05:14:07.401 INFO:teuthology.orchestra.run.smithi194.stdout:Permissive 2024-04-16T05:14:07.401 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-04-16T05:14:07.431 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-04-16T05:14:07.431 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi194.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-04-16T05:14:07.431 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:14:07.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:14:07.461 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:14:07.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:14:07.490 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:14:07.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:14:07.519 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-16T05:14:07.519 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-04-16T05:14:07.571 DEBUG:teuthology.orchestra.run.smithi194:> sudo /usr/sbin/setenforce permissive 2024-04-16T05:14:07.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-04-16T05:14:07.641 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-16T05:14:07.651 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-16T05:14:07.661 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-16T05:14:07.836 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-16T05:14:07.853 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-16T05:14:07.855 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi194.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-16T05:19:12.392 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-04-16T05:19:12.393 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-16T05:19:12.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:19:12.486 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-16T05:19:12.564 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-16T05:19:12.565 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-16T05:19:12.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:19:12.656 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-04-16T05:19:12.725 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-16T05:19:12.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-16T05:19:12.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:19:12.812 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-16T05:19:12.882 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-16T05:19:12.882 INFO:teuthology.run_tasks:Running task clock... 2024-04-16T05:19:12.893 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-16T05:19:12.893 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T05:19:12.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:19:12.897 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T05:19:12.897 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:19:12.900 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T05:19:12.900 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:19:12.928 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T05:19:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T05:19:12.945 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T05:19:12.946 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T05:19:12.953 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T05:19:12.970 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T05:19:12.975 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: ntpd: command not found 2024-04-16T05:19:12.977 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-04-16T05:19:12.988 INFO:teuthology.orchestra.run.smithi194.stdout:506 Cannot talk to daemon 2024-04-16T05:19:12.992 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-04-16T05:19:13.000 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-04-16T05:19:13.005 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T05:19:13.009 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T05:19:13.014 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-04-16T05:19:13.021 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T05:19:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T05:19:13.030 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T05:19:13.047 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T05:19:13.065 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:19:13.068 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:19:13.068 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-04-16T05:19:13.068 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.068 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.068 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.068 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.080 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:19:13.084 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:19:13.085 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-16T05:19:13.085 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.085 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.085 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.085 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.090 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:19:13.094 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:19:13.094 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-04-16T05:19:13.094 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.094 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.094 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.094 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:13.095 INFO:teuthology.run_tasks:Running task install... 2024-04-16T05:19:13.104 DEBUG:teuthology.task.install:project ceph 2024-04-16T05:19:13.104 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea'}} 2024-04-16T05:19:13.104 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea'} 2024-04-16T05:19:13.104 INFO:teuthology.task.install:Using flavor: default 2024-04-16T05:19:13.110 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-16T05:19:13.110 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-16T05:19:13.110 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'tag': None, 'wait_for_package': False} 2024-04-16T05:19:13.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:19:13.113 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'tag': None, 'wait_for_package': False} 2024-04-16T05:19:13.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:19:13.114 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'tag': None, 'wait_for_package': False} 2024-04-16T05:19:13.114 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:19:13.260 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/ 2024-04-16T05:19:13.260 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.ga87fcff6 2024-04-16T05:19:13.264 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/ 2024-04-16T05:19:13.264 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.ga87fcff6 2024-04-16T05:19:13.380 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T05:19:13.381 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:19:13.381 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T05:19:13.382 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/ 2024-04-16T05:19:13.382 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.ga87fcff6 2024-04-16T05:19:13.414 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T05:19:13.414 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a87fcff6340baeedd43af126e793f3241b6400ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T05:19:13.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T05:19:13.491 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T05:19:13.491 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:19:13.491 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T05:19:13.500 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T05:19:13.500 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:19:13.500 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T05:19:13.518 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T05:19:13.518 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a87fcff6340baeedd43af126e793f3241b6400ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T05:19:13.528 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T05:19:13.528 DEBUG:teuthology.orchestra.run.smithi194:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a87fcff6340baeedd43af126e793f3241b6400ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T05:19:13.565 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T05:19:13.585 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T05:19:13.597 DEBUG:teuthology.orchestra.run.smithi194:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T05:19:13.633 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-04-16T05:19:13.635 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-04-16T05:19:13.664 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T05:19:13.677 DEBUG:teuthology.orchestra.run.smithi194:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T05:19:13.731 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-04-16T05:19:13.733 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-04-16T05:19:13.743 INFO:teuthology.orchestra.run.smithi194.stdout:check_obsoletes = 1 2024-04-16T05:19:13.745 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum clean all 2024-04-16T05:19:13.892 INFO:teuthology.orchestra.run.smithi086.stdout:74 files removed 2024-04-16T05:19:13.919 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T05:19:13.967 INFO:teuthology.orchestra.run.smithi196.stdout:74 files removed 2024-04-16T05:19:13.980 INFO:teuthology.orchestra.run.smithi194.stdout:74 files removed 2024-04-16T05:19:13.993 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T05:19:14.007 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T05:19:14.670 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 217 kB/s | 83 kB 00:00 2024-04-16T05:19:14.719 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 169 kB/s | 83 kB 00:00 2024-04-16T05:19:14.769 INFO:teuthology.orchestra.run.smithi194.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-04-16T05:19:15.121 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-16T05:19:15.180 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-16T05:19:15.211 INFO:teuthology.orchestra.run.smithi194.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-16T05:19:15.571 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-16T05:19:15.612 INFO:teuthology.orchestra.run.smithi194.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-16T05:19:15.689 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-04-16T05:19:16.040 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-04-16T05:19:16.040 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-16T05:19:16.108 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-04-16T05:19:17.742 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-04-16T05:19:17.816 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-04-16T05:19:17.831 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - BaseOS 2.9 MB/s | 2.0 MB 00:00 2024-04-16T05:19:18.574 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-16T05:19:18.672 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-16T05:19:18.998 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.0 MB 00:00 2024-04-16T05:19:21.883 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-16T05:19:25.833 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-16T05:19:27.442 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-16T05:19:28.969 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-16T05:19:34.019 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-16T05:19:36.665 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:19:36.666 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:19:36.680 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T05:19:36.681 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T05:19:36.682 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T05:19:36.683 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T05:19:36.685 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T05:19:36.688 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T05:19:36.689 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T05:19:36.769 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-04-16T05:19:36.785 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================================ 2024-04-16T05:19:36.785 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-04-16T05:19:36.785 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================================ 2024-04-16T05:19:36.785 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 6.6 k 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 890 k 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 145 k 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 1.5 M 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 142 k 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 3.5 M 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 7.4 M 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 49 k 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 11 M 2024-04-16T05:19:36.786 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 49 M 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 273 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 762 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 224 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 27 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 18 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 32 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 746 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 127 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T05:19:36.787 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 161 k 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 321 k 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 298 k 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 100 k 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 85 k 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 170 k 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T05:19:36.788 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T05:19:36.789 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.4 M 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-04-16T05:19:36.790 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 21 M 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 34 k 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 2.4 M 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 249 k 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 4.7 M 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 17 M 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 16 k 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 25 k 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T05:19:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 163 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T05:19:36.792 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 501 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T05:19:36.793 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 45 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 135 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T05:19:36.794 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T05:19:36.795 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T05:19:36.796 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T05:19:36.797 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T05:19:36.798 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================================ 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout:Install 150 Packages 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 24 Packages 2024-04-16T05:19:36.799 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:19:36.800 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 229 M 2024-04-16T05:19:36.800 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-04-16T05:19:38.495 INFO:teuthology.orchestra.run.smithi196.stdout:(1/174): ceph-19.0.0-3101.ga87fcff6.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-04-16T05:19:39.470 INFO:teuthology.orchestra.run.smithi196.stdout:(2/174): ceph-fuse-19.0.0-3101.ga87fcff6.el9.x8 913 kB/s | 890 kB 00:00 2024-04-16T05:19:39.653 INFO:teuthology.orchestra.run.smithi196.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 792 kB/s | 145 kB 00:00 2024-04-16T05:19:41.045 INFO:teuthology.orchestra.run.smithi196.stdout:(4/174): ceph-mds-19.0.0-3101.ga87fcff6.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-16T05:19:41.128 INFO:teuthology.orchestra.run.smithi196.stdout:(5/174): ceph-base-19.0.0-3101.ga87fcff6.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-04-16T05:19:41.820 INFO:teuthology.orchestra.run.smithi196.stdout:(6/174): ceph-mgr-19.0.0-3101.ga87fcff6.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-04-16T05:19:41.837 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - AppStream 877 kB/s | 19 MB 00:22 2024-04-16T05:19:43.173 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 824 kB/s | 19 MB 00:23 2024-04-16T05:19:43.812 INFO:teuthology.orchestra.run.smithi196.stdout:(7/174): ceph-mon-19.0.0-3101.ga87fcff6.el9.x86 1.8 MB/s | 4.7 MB 00:02 2024-04-16T05:19:46.562 INFO:teuthology.orchestra.run.smithi196.stdout:(8/174): ceph-common-19.0.0-3101.ga87fcff6.el9. 2.4 MB/s | 21 MB 00:08 2024-04-16T05:19:46.653 INFO:teuthology.orchestra.run.smithi196.stdout:(9/174): ceph-selinux-19.0.0-3101.ga87fcff6.el9 275 kB/s | 25 kB 00:00 2024-04-16T05:19:47.145 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-04-16T05:19:48.588 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-04-16T05:19:48.945 INFO:teuthology.orchestra.run.smithi196.stdout:(10/174): ceph-osd-19.0.0-3101.ga87fcff6.el9.x8 2.4 MB/s | 17 MB 00:07 2024-04-16T05:19:49.512 INFO:teuthology.orchestra.run.smithi196.stdout:(11/174): cephfs-mirror-19.0.0-3101.ga87fcff6.e 396 kB/s | 224 kB 00:00 2024-04-16T05:19:49.612 INFO:teuthology.orchestra.run.smithi196.stdout:(12/174): cephfs-shell-19.0.0-3101.ga87fcff6.el 270 kB/s | 27 kB 00:00 2024-04-16T05:19:49.720 INFO:teuthology.orchestra.run.smithi196.stdout:(13/174): libcephfs-devel-19.0.0-3101.ga87fcff6 293 kB/s | 32 kB 00:00 2024-04-16T05:19:50.171 INFO:teuthology.orchestra.run.smithi196.stdout:(14/174): libcephfs2-19.0.0-3101.ga87fcff6.el9. 1.6 MB/s | 746 kB 00:00 2024-04-16T05:19:50.354 INFO:teuthology.orchestra.run.smithi196.stdout:(15/174): libcephsqlite-19.0.0-3101.ga87fcff6.e 891 kB/s | 163 kB 00:00 2024-04-16T05:19:50.364 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-16T05:19:50.455 INFO:teuthology.orchestra.run.smithi196.stdout:(16/174): librados-devel-19.0.0-3101.ga87fcff6. 1.2 MB/s | 127 kB 00:00 2024-04-16T05:19:50.763 INFO:teuthology.orchestra.run.smithi196.stdout:(17/174): libradosstriper1-19.0.0-3101.ga87fcff 1.6 MB/s | 501 kB 00:00 2024-04-16T05:19:50.972 INFO:teuthology.orchestra.run.smithi196.stdout:(18/174): ceph-radosgw-19.0.0-3101.ga87fcff6.el 1.5 MB/s | 11 MB 00:07 2024-04-16T05:19:50.990 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.2 MB 00:04 2024-04-16T05:19:51.047 INFO:teuthology.orchestra.run.smithi196.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.ga8 605 kB/s | 45 kB 00:00 2024-04-16T05:19:51.214 INFO:teuthology.orchestra.run.smithi196.stdout:(20/174): python3-ceph-common-19.0.0-3101.ga87f 809 kB/s | 135 kB 00:00 2024-04-16T05:19:51.381 INFO:teuthology.orchestra.run.smithi196.stdout:(21/174): python3-cephfs-19.0.0-3101.ga87fcff6. 970 kB/s | 161 kB 00:00 2024-04-16T05:19:51.640 INFO:teuthology.orchestra.run.smithi196.stdout:(22/174): python3-rados-19.0.0-3101.ga87fcff6.e 1.2 MB/s | 321 kB 00:00 2024-04-16T05:19:51.898 INFO:teuthology.orchestra.run.smithi196.stdout:(23/174): python3-rbd-19.0.0-3101.ga87fcff6.el9 1.1 MB/s | 298 kB 00:00 2024-04-16T05:19:51.998 INFO:teuthology.orchestra.run.smithi196.stdout:(24/174): python3-rgw-19.0.0-3101.ga87fcff6.el9 998 kB/s | 100 kB 00:00 2024-04-16T05:19:52.082 INFO:teuthology.orchestra.run.smithi196.stdout:(25/174): rbd-fuse-19.0.0-3101.ga87fcff6.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-16T05:19:53.291 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-04-16T05:19:53.465 INFO:teuthology.orchestra.run.smithi196.stdout:(26/174): librgw2-19.0.0-3101.ga87fcff6.el9.x86 2.0 MB/s | 5.4 MB 00:02 2024-04-16T05:19:53.624 INFO:teuthology.orchestra.run.smithi196.stdout:(27/174): rbd-nbd-19.0.0-3101.ga87fcff6.el9.x86 1.1 MB/s | 170 kB 00:00 2024-04-16T05:19:53.699 INFO:teuthology.orchestra.run.smithi196.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 456 kB/s | 34 kB 00:00 2024-04-16T05:19:53.758 INFO:teuthology.orchestra.run.smithi196.stdout:(29/174): rbd-mirror-19.0.0-3101.ga87fcff6.el9. 1.8 MB/s | 3.1 MB 00:01 2024-04-16T05:19:53.841 INFO:teuthology.orchestra.run.smithi196.stdout:(30/174): ceph-mgr-cephadm-19.0.0-3101.ga87fcff 1.0 MB/s | 142 kB 00:00 2024-04-16T05:19:55.331 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-04-16T05:19:55.908 INFO:teuthology.orchestra.run.smithi196.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3101.ga87fc 1.6 MB/s | 3.5 MB 00:02 2024-04-16T05:19:56.133 INFO:teuthology.orchestra.run.smithi196.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.ga8 1.1 MB/s | 249 kB 00:00 2024-04-16T05:19:56.234 INFO:teuthology.orchestra.run.smithi196.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.ga87fcff6.e 493 kB/s | 49 kB 00:00 2024-04-16T05:19:56.351 INFO:teuthology.orchestra.run.smithi196.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.ga 141 kB/s | 16 kB 00:00 2024-04-16T05:19:56.668 INFO:teuthology.orchestra.run.smithi196.stdout:(35/174): ceph-volume-19.0.0-3101.ga87fcff6.el9 862 kB/s | 273 kB 00:00 2024-04-16T05:19:57.384 INFO:teuthology.orchestra.run.smithi196.stdout:(36/174): cephadm-19.0.0-3101.ga87fcff6.el9.noa 1.0 MB/s | 762 kB 00:00 2024-04-16T05:19:57.536 INFO:teuthology.orchestra.run.smithi196.stdout:(37/174): cephfs-top-19.0.0-3101.ga87fcff6.el9. 120 kB/s | 18 kB 00:00 2024-04-16T05:19:57.618 INFO:teuthology.orchestra.run.smithi196.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 2.0 MB/s | 7.4 MB 00:03 2024-04-16T05:19:57.768 INFO:teuthology.orchestra.run.smithi196.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 662 kB 00:00 2024-04-16T05:19:57.827 INFO:teuthology.orchestra.run.smithi196.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 920 kB 00:00 2024-04-16T05:19:57.852 INFO:teuthology.orchestra.run.smithi196.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 106 kB 00:00 2024-04-16T05:19:57.877 INFO:teuthology.orchestra.run.smithi196.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-16T05:19:57.903 INFO:teuthology.orchestra.run.smithi196.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-04-16T05:19:57.987 INFO:teuthology.orchestra.run.smithi194.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:19:57.988 INFO:teuthology.orchestra.run.smithi194.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:19:58.002 INFO:teuthology.orchestra.run.smithi194.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T05:19:58.003 INFO:teuthology.orchestra.run.smithi194.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T05:19:58.004 INFO:teuthology.orchestra.run.smithi194.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T05:19:58.005 INFO:teuthology.orchestra.run.smithi194.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T05:19:58.008 INFO:teuthology.orchestra.run.smithi196.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2024-04-16T05:19:58.008 INFO:teuthology.orchestra.run.smithi194.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T05:19:58.010 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T05:19:58.011 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T05:19:58.028 INFO:teuthology.orchestra.run.smithi196.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 139 kB/s | 17 kB 00:00 2024-04-16T05:19:58.062 INFO:teuthology.orchestra.run.smithi196.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-04-16T05:19:58.087 INFO:teuthology.orchestra.run.smithi196.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 171 kB/s | 10 kB 00:00 2024-04-16T05:19:58.090 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-16T05:19:58.112 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-04-16T05:19:58.112 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-04-16T05:19:58.112 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-04-16T05:19:58.112 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-16T05:19:58.112 INFO:teuthology.orchestra.run.smithi194.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T05:19:58.112 INFO:teuthology.orchestra.run.smithi194.stdout: ceph x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 6.6 k 2024-04-16T05:19:58.112 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 890 k 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 145 k 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 1.5 M 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 142 k 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 3.5 M 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 7.4 M 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 49 k 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 11 M 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 49 M 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 273 k 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 762 k 2024-04-16T05:19:58.113 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 224 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 27 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 18 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 32 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 746 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 127 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T05:19:58.114 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 161 k 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 321 k 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 298 k 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 100 k 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 85 k 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:19:58.115 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 170 k 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T05:19:58.116 INFO:teuthology.orchestra.run.smithi194.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: librados2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.4 M 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T05:19:58.117 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 21 M 2024-04-16T05:19:58.118 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 34 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 2.4 M 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 249 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 4.7 M 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 17 M 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 16 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 25 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T05:19:58.119 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 163 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 501 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T05:19:58.120 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T05:19:58.121 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 45 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 135 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T05:19:58.122 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T05:19:58.123 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T05:19:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T05:19:58.125 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T05:19:58.126 INFO:teuthology.orchestra.run.smithi194.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout:Installing weak dependencies: 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout:Install 150 Packages 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 24 Packages 2024-04-16T05:19:58.127 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:19:58.128 INFO:teuthology.orchestra.run.smithi196.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 876 kB/s | 44 kB 00:00 2024-04-16T05:19:58.128 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 229 M 2024-04-16T05:19:58.128 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-16T05:19:58.138 INFO:teuthology.orchestra.run.smithi196.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-16T05:19:58.171 INFO:teuthology.orchestra.run.smithi196.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-04-16T05:19:58.197 INFO:teuthology.orchestra.run.smithi196.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 804 kB/s | 46 kB 00:00 2024-04-16T05:19:58.222 INFO:teuthology.orchestra.run.smithi196.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-16T05:19:58.247 INFO:teuthology.orchestra.run.smithi196.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-16T05:19:58.272 INFO:teuthology.orchestra.run.smithi196.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-04-16T05:19:58.293 INFO:teuthology.orchestra.run.smithi196.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 765 kB/s | 34 kB 00:00 2024-04-16T05:19:58.410 INFO:teuthology.orchestra.run.smithi196.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-04-16T05:19:58.452 INFO:teuthology.orchestra.run.smithi196.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 9.7 MB/s | 1.5 MB 00:00 2024-04-16T05:19:58.485 INFO:teuthology.orchestra.run.smithi196.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 817 kB 00:00 2024-04-16T05:19:58.511 INFO:teuthology.orchestra.run.smithi196.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 132 kB 00:00 2024-04-16T05:19:58.661 INFO:teuthology.orchestra.run.smithi196.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-04-16T05:19:58.695 INFO:teuthology.orchestra.run.smithi196.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.6 MB/s | 1.0 MB 00:00 2024-04-16T05:19:58.720 INFO:teuthology.orchestra.run.smithi196.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-16T05:19:58.932 INFO:teuthology.orchestra.run.smithi194.stdout:(1/174): ceph-19.0.0-3101.ga87fcff6.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-16T05:19:58.945 INFO:teuthology.orchestra.run.smithi196.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-04-16T05:19:58.971 INFO:teuthology.orchestra.run.smithi196.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 97 kB/s | 24 kB 00:00 2024-04-16T05:19:58.996 INFO:teuthology.orchestra.run.smithi196.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-04-16T05:19:59.021 INFO:teuthology.orchestra.run.smithi196.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-16T05:19:59.046 INFO:teuthology.orchestra.run.smithi196.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-04-16T05:19:59.238 INFO:teuthology.orchestra.run.smithi196.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2024-04-16T05:19:59.263 INFO:teuthology.orchestra.run.smithi196.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-04-16T05:19:59.297 INFO:teuthology.orchestra.run.smithi196.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 174 kB 00:00 2024-04-16T05:19:59.322 INFO:teuthology.orchestra.run.smithi196.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 69 kB 00:00 2024-04-16T05:19:59.348 INFO:teuthology.orchestra.run.smithi196.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-04-16T05:19:59.373 INFO:teuthology.orchestra.run.smithi196.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-16T05:19:59.398 INFO:teuthology.orchestra.run.smithi196.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-16T05:19:59.424 INFO:teuthology.orchestra.run.smithi196.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-16T05:19:59.449 INFO:teuthology.orchestra.run.smithi196.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 943 kB/s | 47 kB 00:00 2024-04-16T05:19:59.474 INFO:teuthology.orchestra.run.smithi196.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 853 kB/s | 43 kB 00:00 2024-04-16T05:19:59.749 INFO:teuthology.orchestra.run.smithi194.stdout:(2/174): ceph-fuse-19.0.0-3101.ga87fcff6.el9.x8 1.1 MB/s | 890 kB 00:00 2024-04-16T05:19:59.750 INFO:teuthology.orchestra.run.smithi196.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 142 kB/s | 39 kB 00:00 2024-04-16T05:19:59.909 INFO:teuthology.orchestra.run.smithi196.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-16T05:19:59.915 INFO:teuthology.orchestra.run.smithi194.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 870 kB/s | 145 kB 00:00 2024-04-16T05:19:59.934 INFO:teuthology.orchestra.run.smithi196.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 223 kB/s | 41 kB 00:00 2024-04-16T05:19:59.959 INFO:teuthology.orchestra.run.smithi196.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-04-16T05:19:59.984 INFO:teuthology.orchestra.run.smithi196.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 700 kB/s | 35 kB 00:00 2024-04-16T05:20:00.009 INFO:teuthology.orchestra.run.smithi196.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 928 kB/s | 46 kB 00:00 2024-04-16T05:20:00.035 INFO:teuthology.orchestra.run.smithi196.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-04-16T05:20:00.060 INFO:teuthology.orchestra.run.smithi196.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-16T05:20:00.085 INFO:teuthology.orchestra.run.smithi196.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-16T05:20:00.110 INFO:teuthology.orchestra.run.smithi196.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-04-16T05:20:00.136 INFO:teuthology.orchestra.run.smithi196.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-16T05:20:00.161 INFO:teuthology.orchestra.run.smithi196.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-16T05:20:00.211 INFO:teuthology.orchestra.run.smithi196.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-04-16T05:20:00.237 INFO:teuthology.orchestra.run.smithi196.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-04-16T05:20:00.270 INFO:teuthology.orchestra.run.smithi196.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-04-16T05:20:00.296 INFO:teuthology.orchestra.run.smithi196.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-04-16T05:20:01.146 INFO:teuthology.orchestra.run.smithi196.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 214 kB/s | 187 kB 00:00 2024-04-16T05:20:01.666 INFO:teuthology.orchestra.run.smithi194.stdout:(4/174): ceph-mds-19.0.0-3101.ga87fcff6.el9.x86 1.3 MB/s | 2.4 MB 00:01 2024-04-16T05:20:01.954 INFO:teuthology.orchestra.run.smithi196.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 484 kB/s | 803 kB 00:01 2024-04-16T05:20:02.055 INFO:teuthology.orchestra.run.smithi196.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 215 kB/s | 195 kB 00:00 2024-04-16T05:20:02.163 INFO:teuthology.orchestra.run.smithi196.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-04-16T05:20:02.255 INFO:teuthology.orchestra.run.smithi196.stdout:(98/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 278 kB/s | 25 kB 00:00 2024-04-16T05:20:02.530 INFO:teuthology.orchestra.run.smithi196.stdout:(99/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 15 MB/s | 4.1 MB 00:00 2024-04-16T05:20:02.566 INFO:teuthology.orchestra.run.smithi194.stdout:(5/174): ceph-mgr-19.0.0-3101.ga87fcff6.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-04-16T05:20:02.672 INFO:teuthology.orchestra.run.smithi196.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.7 MB/s | 245 kB 00:00 2024-04-16T05:20:02.739 INFO:teuthology.orchestra.run.smithi196.stdout:(101/174): python3-requests-2.25.1-8.el9.noarch 159 kB/s | 125 kB 00:00 2024-04-16T05:20:02.764 INFO:teuthology.orchestra.run.smithi196.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.6 MB/s | 248 kB 00:00 2024-04-16T05:20:03.006 INFO:teuthology.orchestra.run.smithi196.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 351 kB/s | 94 kB 00:00 2024-04-16T05:20:03.098 INFO:teuthology.orchestra.run.smithi196.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-04-16T05:20:03.299 INFO:teuthology.orchestra.run.smithi196.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.7 MB/s | 548 kB 00:00 2024-04-16T05:20:03.358 INFO:teuthology.orchestra.run.smithi196.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-04-16T05:20:03.410 INFO:teuthology.orchestra.run.smithi196.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 373 kB/s | 19 kB 00:00 2024-04-16T05:20:03.569 INFO:teuthology.orchestra.run.smithi196.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-04-16T05:20:03.619 INFO:teuthology.orchestra.run.smithi196.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-04-16T05:20:03.669 INFO:teuthology.orchestra.run.smithi196.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-04-16T05:20:03.720 INFO:teuthology.orchestra.run.smithi196.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-16T05:20:03.770 INFO:teuthology.orchestra.run.smithi196.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-16T05:20:03.829 INFO:teuthology.orchestra.run.smithi196.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2024-04-16T05:20:03.887 INFO:teuthology.orchestra.run.smithi196.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-04-16T05:20:03.929 INFO:teuthology.orchestra.run.smithi196.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-16T05:20:03.963 INFO:teuthology.orchestra.run.smithi196.stdout:(116/174): protobuf-compiler-3.14.0-13.el9.x86_ 903 kB/s | 863 kB 00:00 2024-04-16T05:20:03.980 INFO:teuthology.orchestra.run.smithi196.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-04-16T05:20:04.030 INFO:teuthology.orchestra.run.smithi196.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-16T05:20:04.047 INFO:teuthology.orchestra.run.smithi196.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-16T05:20:04.080 INFO:teuthology.orchestra.run.smithi196.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-16T05:20:04.139 INFO:teuthology.orchestra.run.smithi196.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 604 kB/s | 35 kB 00:00 2024-04-16T05:20:04.189 INFO:teuthology.orchestra.run.smithi196.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-04-16T05:20:04.215 INFO:teuthology.orchestra.run.smithi196.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-04-16T05:20:04.265 INFO:teuthology.orchestra.run.smithi196.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-04-16T05:20:04.290 INFO:teuthology.orchestra.run.smithi196.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-16T05:20:04.315 INFO:teuthology.orchestra.run.smithi196.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-16T05:20:04.341 INFO:teuthology.orchestra.run.smithi196.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-16T05:20:04.366 INFO:teuthology.orchestra.run.smithi196.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-16T05:20:04.391 INFO:teuthology.orchestra.run.smithi196.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-16T05:20:04.417 INFO:teuthology.orchestra.run.smithi196.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-16T05:20:04.450 INFO:teuthology.orchestra.run.smithi196.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-04-16T05:20:04.475 INFO:teuthology.orchestra.run.smithi196.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-04-16T05:20:04.501 INFO:teuthology.orchestra.run.smithi196.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-16T05:20:04.526 INFO:teuthology.orchestra.run.smithi196.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-16T05:20:04.551 INFO:teuthology.orchestra.run.smithi196.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-16T05:20:04.577 INFO:teuthology.orchestra.run.smithi196.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 325 kB/s | 16 kB 00:00 2024-04-16T05:20:04.594 INFO:teuthology.orchestra.run.smithi196.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-04-16T05:20:04.627 INFO:teuthology.orchestra.run.smithi196.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-04-16T05:20:04.878 INFO:teuthology.orchestra.run.smithi196.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 108 kB/s | 31 kB 00:00 2024-04-16T05:20:04.912 INFO:teuthology.orchestra.run.smithi196.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 665 kB/s | 188 kB 00:00 2024-04-16T05:20:04.937 INFO:teuthology.orchestra.run.smithi196.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-16T05:20:04.962 INFO:teuthology.orchestra.run.smithi196.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-04-16T05:20:04.987 INFO:teuthology.orchestra.run.smithi196.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-16T05:20:05.021 INFO:teuthology.orchestra.run.smithi196.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-16T05:20:05.046 INFO:teuthology.orchestra.run.smithi196.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-16T05:20:05.096 INFO:teuthology.orchestra.run.smithi196.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-16T05:20:05.122 INFO:teuthology.orchestra.run.smithi196.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-16T05:20:05.147 INFO:teuthology.orchestra.run.smithi196.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-16T05:20:05.189 INFO:teuthology.orchestra.run.smithi196.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-16T05:20:06.516 INFO:teuthology.orchestra.run.smithi194.stdout:(6/174): ceph-mon-19.0.0-3101.ga87fcff6.el9.x86 1.2 MB/s | 4.7 MB 00:03 2024-04-16T05:20:07.381 INFO:teuthology.orchestra.run.smithi196.stdout:(150/174): librados2-19.0.0-3101.ga87fcff6.el9. 1.5 MB/s | 3.4 MB 00:02 2024-04-16T05:20:07.472 INFO:teuthology.orchestra.run.smithi196.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-04-16T05:20:07.556 INFO:teuthology.orchestra.run.smithi196.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-04-16T05:20:07.590 INFO:teuthology.orchestra.run.smithi196.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-16T05:20:07.632 INFO:teuthology.orchestra.run.smithi196.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-04-16T05:20:07.674 INFO:teuthology.orchestra.run.smithi196.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-16T05:20:08.382 INFO:teuthology.orchestra.run.smithi196.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 101 kB/s | 71 kB 00:00 2024-04-16T05:20:08.924 INFO:teuthology.orchestra.run.smithi196.stdout:(157/174): librbd1-19.0.0-3101.ga87fcff6.el9.x8 862 kB/s | 3.1 MB 00:03 2024-04-16T05:20:09.074 INFO:teuthology.orchestra.run.smithi196.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 53 kB/s | 36 kB 00:00 2024-04-16T05:20:09.933 INFO:teuthology.orchestra.run.smithi196.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 191 kB/s | 192 kB 00:01 2024-04-16T05:20:10.150 INFO:teuthology.orchestra.run.smithi196.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 238 kB/s | 256 kB 00:01 2024-04-16T05:20:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 39 kB/s | 23 kB 00:00 2024-04-16T05:20:10.917 INFO:teuthology.orchestra.run.smithi196.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 141 kB/s | 108 kB 00:00 2024-04-16T05:20:11.450 INFO:teuthology.orchestra.run.smithi196.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 166 kB/s | 155 kB 00:00 2024-04-16T05:20:11.734 INFO:teuthology.orchestra.run.smithi196.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 166 kB/s | 136 kB 00:00 2024-04-16T05:20:12.118 INFO:teuthology.orchestra.run.smithi196.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 96 kB/s | 64 kB 00:00 2024-04-16T05:20:12.318 INFO:teuthology.orchestra.run.smithi196.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 49 kB/s | 29 kB 00:00 2024-04-16T05:20:12.957 INFO:teuthology.orchestra.run.smithi194.stdout:(7/174): ceph-base-19.0.0-3101.ga87fcff6.el9.x8 383 kB/s | 5.4 MB 00:14 2024-04-16T05:20:13.794 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-04-16T05:20:14.193 INFO:teuthology.orchestra.run.smithi196.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 597 kB/s | 1.2 MB 00:02 2024-04-16T05:20:15.643 INFO:teuthology.orchestra.run.smithi196.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 664 kB/s | 2.2 MB 00:03 2024-04-16T05:20:16.877 INFO:teuthology.orchestra.run.smithi196.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 377 kB/s | 465 kB 00:01 2024-04-16T05:20:17.477 INFO:teuthology.orchestra.run.smithi196.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 52 kB/s | 31 kB 00:00 2024-04-16T05:20:17.821 INFO:teuthology.orchestra.run.smithi196.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 645 kB/s | 2.3 MB 00:03 2024-04-16T05:20:17.944 INFO:teuthology.orchestra.run.smithi196.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 602 kB/s | 74 kB 00:00 2024-04-16T05:20:18.824 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-16T05:20:19.469 INFO:teuthology.orchestra.run.smithi196.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 552 kB/s | 1.1 MB 00:01 2024-04-16T05:20:21.546 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:20:21.547 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:20:21.560 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T05:20:21.561 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T05:20:21.562 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T05:20:21.563 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T05:20:21.565 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T05:20:21.568 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T05:20:21.569 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T05:20:21.650 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================================ 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================================ 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 6.6 k 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 890 k 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 145 k 2024-04-16T05:20:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 1.5 M 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 142 k 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 3.5 M 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 7.4 M 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 49 k 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 11 M 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 49 M 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 273 k 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 762 k 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 224 k 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 27 k 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 18 k 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T05:20:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 32 k 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 746 k 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 127 k 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T05:20:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 161 k 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 321 k 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 298 k 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 100 k 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 85 k 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 170 k 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T05:20:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.4 M 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T05:20:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 21 M 2024-04-16T05:20:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 34 k 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 2.4 M 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 249 k 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 4.7 M 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 17 M 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 16 k 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 25 k 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T05:20:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 163 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 501 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T05:20:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T05:20:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 45 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 135 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T05:20:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T05:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T05:20:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T05:20:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T05:20:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================================ 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout:Install 150 Packages 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 24 Packages 2024-04-16T05:20:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:20:21.687 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 229 M 2024-04-16T05:20:21.687 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-04-16T05:20:21.802 INFO:teuthology.orchestra.run.smithi196.stdout:(174/174): ceph-test-19.0.0-3101.ga87fcff6.el9. 1.4 MB/s | 49 MB 00:35 2024-04-16T05:20:21.807 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:20:21.807 INFO:teuthology.orchestra.run.smithi196.stdout:Total 5.1 MB/s | 229 MB 00:45 2024-04-16T05:20:22.170 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-04-16T05:20:22.398 INFO:teuthology.orchestra.run.smithi194.stdout:(8/174): ceph-common-19.0.0-3101.ga87fcff6.el9. 910 kB/s | 21 MB 00:23 2024-04-16T05:20:22.440 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-04-16T05:20:22.440 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-04-16T05:20:22.489 INFO:teuthology.orchestra.run.smithi194.stdout:(9/174): ceph-selinux-19.0.0-3101.ga87fcff6.el9 275 kB/s | 25 kB 00:00 2024-04-16T05:20:22.558 INFO:teuthology.orchestra.run.smithi086.stdout:(1/174): ceph-19.0.0-3101.ga87fcff6.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-16T05:20:24.178 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-04-16T05:20:24.178 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-04-16T05:20:24.242 INFO:teuthology.orchestra.run.smithi086.stdout:(2/174): ceph-fuse-19.0.0-3101.ga87fcff6.el9.x8 529 kB/s | 890 kB 00:01 2024-04-16T05:20:24.542 INFO:teuthology.orchestra.run.smithi086.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 483 kB/s | 145 kB 00:00 2024-04-16T05:20:24.940 INFO:teuthology.orchestra.run.smithi194.stdout:(10/174): ceph-radosgw-19.0.0-3101.ga87fcff6.el 905 kB/s | 11 MB 00:11 2024-04-16T05:20:25.281 INFO:teuthology.orchestra.run.smithi194.stdout:(11/174): cephfs-mirror-19.0.0-3101.ga87fcff6.e 658 kB/s | 224 kB 00:00 2024-04-16T05:20:25.398 INFO:teuthology.orchestra.run.smithi194.stdout:(12/174): cephfs-shell-19.0.0-3101.ga87fcff6.el 231 kB/s | 27 kB 00:00 2024-04-16T05:20:25.515 INFO:teuthology.orchestra.run.smithi194.stdout:(13/174): libcephfs-devel-19.0.0-3101.ga87fcff6 272 kB/s | 32 kB 00:00 2024-04-16T05:20:26.265 INFO:teuthology.orchestra.run.smithi194.stdout:(14/174): libcephfs2-19.0.0-3101.ga87fcff6.el9. 996 kB/s | 746 kB 00:00 2024-04-16T05:20:26.474 INFO:teuthology.orchestra.run.smithi194.stdout:(15/174): libcephsqlite-19.0.0-3101.ga87fcff6.e 785 kB/s | 163 kB 00:00 2024-04-16T05:20:26.666 INFO:teuthology.orchestra.run.smithi194.stdout:(16/174): librados-devel-19.0.0-3101.ga87fcff6. 661 kB/s | 127 kB 00:00 2024-04-16T05:20:27.141 INFO:teuthology.orchestra.run.smithi194.stdout:(17/174): libradosstriper1-19.0.0-3101.ga87fcff 1.0 MB/s | 501 kB 00:00 2024-04-16T05:20:27.700 INFO:teuthology.orchestra.run.smithi086.stdout:(4/174): ceph-mds-19.0.0-3101.ga87fcff6.el9.x86 765 kB/s | 2.4 MB 00:03 2024-04-16T05:20:30.925 INFO:teuthology.orchestra.run.smithi086.stdout:(5/174): ceph-base-19.0.0-3101.ga87fcff6.el9.x8 625 kB/s | 5.4 MB 00:08 2024-04-16T05:20:30.959 INFO:teuthology.orchestra.run.smithi086.stdout:(6/174): ceph-mgr-19.0.0-3101.ga87fcff6.el9.x86 470 kB/s | 1.5 MB 00:03 2024-04-16T05:20:31.018 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-04-16T05:20:31.116 INFO:teuthology.orchestra.run.smithi194.stdout:(18/174): librgw2-19.0.0-3101.ga87fcff6.el9.x86 1.4 MB/s | 5.4 MB 00:03 2024-04-16T05:20:31.130 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T05:20:31.156 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T05:20:31.204 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:20:31.213 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:20:31.224 INFO:teuthology.orchestra.run.smithi194.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.ga8 418 kB/s | 45 kB 00:00 2024-04-16T05:20:31.253 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T05:20:31.306 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T05:20:31.349 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T05:20:31.383 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T05:20:31.425 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T05:20:31.608 INFO:teuthology.orchestra.run.smithi194.stdout:(20/174): python3-ceph-common-19.0.0-3101.ga87f 351 kB/s | 135 kB 00:00 2024-04-16T05:20:31.745 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T05:20:31.800 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:20:31.808 INFO:teuthology.orchestra.run.smithi194.stdout:(21/174): python3-cephfs-19.0.0-3101.ga87fcff6. 808 kB/s | 161 kB 00:00 2024-04-16T05:20:31.921 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:20:31.960 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:20:32.051 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:20:32.080 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:20:32.118 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T05:20:32.297 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T05:20:32.309 INFO:teuthology.orchestra.run.smithi194.stdout:(22/174): python3-rados-19.0.0-3101.ga87fcff6.e 641 kB/s | 321 kB 00:00 2024-04-16T05:20:32.353 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T05:20:32.373 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:20:32.460 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:20:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:20:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:20:32.595 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T05:20:32.618 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:20:32.632 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:20:32.784 INFO:teuthology.orchestra.run.smithi194.stdout:(23/174): python3-rbd-19.0.0-3101.ga87fcff6.el9 627 kB/s | 298 kB 00:00 2024-04-16T05:20:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:20:32.865 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:20:32.897 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T05:20:33.001 INFO:teuthology.orchestra.run.smithi194.stdout:(24/174): python3-rgw-19.0.0-3101.ga87fcff6.el9 459 kB/s | 100 kB 00:00 2024-04-16T05:20:33.002 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T05:20:33.108 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:20:33.151 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:20:33.209 INFO:teuthology.orchestra.run.smithi194.stdout:(25/174): rbd-fuse-19.0.0-3101.ga87fcff6.el9.x8 410 kB/s | 85 kB 00:00 2024-04-16T05:20:33.210 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:20:33.294 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:20:33.502 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:20:33.607 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T05:20:33.702 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T05:20:33.741 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T05:20:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T05:20:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T05:20:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T05:20:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T05:20:33.969 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T05:20:34.042 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T05:20:34.077 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T05:20:34.120 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T05:20:34.153 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T05:20:34.182 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T05:20:34.251 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T05:20:34.286 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 39/198 2024-04-16T05:20:34.329 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 40/198 2024-04-16T05:20:34.358 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 41/198 2024-04-16T05:20:34.427 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T05:20:34.481 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T05:20:34.555 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T05:20:34.610 INFO:teuthology.orchestra.run.smithi194.stdout:(26/174): ceph-osd-19.0.0-3101.ga87fcff6.el9.x8 613 kB/s | 17 MB 00:28 2024-04-16T05:20:34.697 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T05:20:34.751 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T05:20:34.926 INFO:teuthology.orchestra.run.smithi194.stdout:(27/174): rbd-nbd-19.0.0-3101.ga87fcff6.el9.x86 539 kB/s | 170 kB 00:00 2024-04-16T05:20:35.027 INFO:teuthology.orchestra.run.smithi194.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 342 kB/s | 34 kB 00:00 2024-04-16T05:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T05:20:35.285 INFO:teuthology.orchestra.run.smithi194.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.ga87fcff 550 kB/s | 142 kB 00:00 2024-04-16T05:20:35.354 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T05:20:35.424 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T05:20:35.474 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:20:35.483 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:20:35.514 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T05:20:35.554 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T05:20:35.592 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 53/198 2024-04-16T05:20:35.622 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T05:20:35.848 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T05:20:36.024 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T05:20:36.108 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T05:20:36.151 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T05:20:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T05:20:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T05:20:36.393 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T05:20:36.466 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T05:20:36.499 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T05:20:36.536 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T05:20:36.583 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T05:20:36.618 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T05:20:36.662 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T05:20:36.704 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T05:20:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T05:20:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T05:20:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T05:20:36.956 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T05:20:37.050 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T05:20:37.090 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T05:20:37.129 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T05:20:37.164 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T05:20:37.188 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T05:20:37.262 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T05:20:37.389 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T05:20:37.434 INFO:teuthology.orchestra.run.smithi086.stdout:(7/174): ceph-mon-19.0.0-3101.ga87fcff6.el9.x86 739 kB/s | 4.7 MB 00:06 2024-04-16T05:20:37.477 INFO:teuthology.orchestra.run.smithi194.stdout:(30/174): rbd-mirror-19.0.0-3101.ga87fcff6.el9. 741 kB/s | 3.1 MB 00:04 2024-04-16T05:20:37.488 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T05:20:37.538 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T05:20:37.587 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T05:20:37.642 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T05:20:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T05:20:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T05:20:38.004 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T05:20:38.040 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T05:20:38.072 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T05:20:38.103 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:20:38.132 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T05:20:38.166 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T05:20:38.205 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T05:20:38.237 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T05:20:38.266 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T05:20:38.343 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T05:20:38.377 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T05:20:38.410 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T05:20:38.465 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T05:20:38.494 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T05:20:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T05:20:38.665 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T05:20:38.709 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T05:20:38.741 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T05:20:38.787 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T05:20:38.885 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T05:20:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T05:20:39.669 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T05:20:39.722 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T05:20:39.756 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T05:20:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T05:20:39.901 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T05:20:39.949 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T05:20:40.126 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T05:20:40.163 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T05:20:40.241 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T05:20:40.560 INFO:teuthology.orchestra.run.smithi194.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3101.ga87fc 687 kB/s | 3.5 MB 00:05 2024-04-16T05:20:40.879 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T05:20:40.977 INFO:teuthology.orchestra.run.smithi194.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.ga8 598 kB/s | 249 kB 00:00 2024-04-16T05:20:40.986 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T05:20:41.085 INFO:teuthology.orchestra.run.smithi194.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.ga87fcff6.e 456 kB/s | 49 kB 00:00 2024-04-16T05:20:41.185 INFO:teuthology.orchestra.run.smithi194.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.ga 165 kB/s | 16 kB 00:00 2024-04-16T05:20:41.661 INFO:teuthology.orchestra.run.smithi194.stdout:(35/174): ceph-volume-19.0.0-3101.ga87fcff6.el9 575 kB/s | 273 kB 00:00 2024-04-16T05:20:42.196 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T05:20:42.272 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T05:20:42.303 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T05:20:42.334 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T05:20:42.362 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T05:20:42.594 INFO:teuthology.orchestra.run.smithi194.stdout:(36/174): cephadm-19.0.0-3101.ga87fcff6.el9.noa 817 kB/s | 762 kB 00:00 2024-04-16T05:20:42.676 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T05:20:42.678 INFO:teuthology.orchestra.run.smithi194.stdout:(37/174): cephfs-top-19.0.0-3101.ga87fcff6.el9. 219 kB/s | 18 kB 00:00 2024-04-16T05:20:42.811 INFO:teuthology.orchestra.run.smithi194.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-04-16T05:20:42.862 INFO:teuthology.orchestra.run.smithi194.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-04-16T05:20:42.896 INFO:teuthology.orchestra.run.smithi194.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-16T05:20:42.930 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:20:42.937 INFO:teuthology.orchestra.run.smithi194.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-04-16T05:20:42.962 INFO:teuthology.orchestra.run.smithi194.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 32 kB 00:00 2024-04-16T05:20:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:20:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 125/198 2024-04-16T05:20:43.063 INFO:teuthology.orchestra.run.smithi194.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-16T05:20:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T05:20:43.088 INFO:teuthology.orchestra.run.smithi194.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 685 kB/s | 17 kB 00:00 2024-04-16T05:20:43.113 INFO:teuthology.orchestra.run.smithi194.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 405 kB/s | 10 kB 00:00 2024-04-16T05:20:43.139 INFO:teuthology.orchestra.run.smithi194.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.2 MB/s | 159 kB 00:00 2024-04-16T05:20:43.164 INFO:teuthology.orchestra.run.smithi194.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.7 MB/s | 44 kB 00:00 2024-04-16T05:20:43.206 INFO:teuthology.orchestra.run.smithi194.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-04-16T05:20:43.248 INFO:teuthology.orchestra.run.smithi194.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-04-16T05:20:43.273 INFO:teuthology.orchestra.run.smithi194.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.8 MB/s | 46 kB 00:00 2024-04-16T05:20:43.307 INFO:teuthology.orchestra.run.smithi194.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 8.3 MB/s | 282 kB 00:00 2024-04-16T05:20:43.350 INFO:teuthology.orchestra.run.smithi194.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.3 MB/s | 187 kB 00:00 2024-04-16T05:20:43.383 INFO:teuthology.orchestra.run.smithi194.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 6.4 MB/s | 217 kB 00:00 2024-04-16T05:20:43.417 INFO:teuthology.orchestra.run.smithi194.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-04-16T05:20:43.542 INFO:teuthology.orchestra.run.smithi194.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-16T05:20:43.560 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T05:20:43.595 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:20:43.609 INFO:teuthology.orchestra.run.smithi194.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 23 MB/s | 1.5 MB 00:00 2024-04-16T05:20:43.649 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:20:43.651 INFO:teuthology.orchestra.run.smithi194.stdout:(57/174): postgresql-contrib-13.14-1.el9.x86_64 19 MB/s | 817 kB 00:00 2024-04-16T05:20:43.671 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 129/198 2024-04-16T05:20:43.685 INFO:teuthology.orchestra.run.smithi194.stdout:(58/174): postgresql-private-libs-13.14-1.el9.x 3.9 MB/s | 132 kB 00:00 2024-04-16T05:20:43.844 INFO:teuthology.orchestra.run.smithi194.stdout:(59/174): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-04-16T05:20:43.902 INFO:teuthology.orchestra.run.smithi194.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-04-16T05:20:43.936 INFO:teuthology.orchestra.run.smithi194.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-04-16T05:20:44.087 INFO:teuthology.orchestra.run.smithi194.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 5.8 MB 00:00 2024-04-16T05:20:44.120 INFO:teuthology.orchestra.run.smithi194.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 734 kB/s | 24 kB 00:00 2024-04-16T05:20:44.153 INFO:teuthology.orchestra.run.smithi194.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 543 kB/s | 18 kB 00:00 2024-04-16T05:20:44.187 INFO:teuthology.orchestra.run.smithi194.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 4.2 MB/s | 144 kB 00:00 2024-04-16T05:20:44.212 INFO:teuthology.orchestra.run.smithi194.stdout:(66/174): python3-markupsafe-1.1.1-12.el9.x86_6 1.2 MB/s | 32 kB 00:00 2024-04-16T05:20:44.338 INFO:teuthology.orchestra.run.smithi194.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-04-16T05:20:44.371 INFO:teuthology.orchestra.run.smithi194.stdout:(68/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 6.4 MB/s | 217 kB 00:00 2024-04-16T05:20:44.405 INFO:teuthology.orchestra.run.smithi194.stdout:(69/174): python3-oauthlib-3.1.1-5.el9.noarch.r 5.1 MB/s | 174 kB 00:00 2024-04-16T05:20:44.455 INFO:teuthology.orchestra.run.smithi194.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-16T05:20:44.480 INFO:teuthology.orchestra.run.smithi194.stdout:(71/174): python3-prettytable-0.7.2-27.el9.noar 1.6 MB/s | 41 kB 00:00 2024-04-16T05:20:44.522 INFO:teuthology.orchestra.run.smithi194.stdout:(72/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-04-16T05:20:44.556 INFO:teuthology.orchestra.run.smithi194.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.9 MB/s | 132 kB 00:00 2024-04-16T05:20:44.589 INFO:teuthology.orchestra.run.smithi194.stdout:(74/174): python3-pyasn1-modules-0.4.8-6.el9.no 6.2 MB/s | 211 kB 00:00 2024-04-16T05:20:44.618 INFO:teuthology.orchestra.run.smithi194.stdout:(75/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.8 MB/s | 47 kB 00:00 2024-04-16T05:20:44.640 INFO:teuthology.orchestra.run.smithi194.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 1.7 MB/s | 43 kB 00:00 2024-04-16T05:20:44.982 INFO:teuthology.orchestra.run.smithi194.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-04-16T05:20:45.007 INFO:teuthology.orchestra.run.smithi194.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 39 kB 00:00 2024-04-16T05:20:45.033 INFO:teuthology.orchestra.run.smithi194.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.6 MB/s | 41 kB 00:00 2024-04-16T05:20:45.074 INFO:teuthology.orchestra.run.smithi194.stdout:(80/174): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2024-04-16T05:20:45.100 INFO:teuthology.orchestra.run.smithi194.stdout:(81/174): qatlib-service-24.02.0-1.el9.x86_64.r 1.4 MB/s | 35 kB 00:00 2024-04-16T05:20:45.133 INFO:teuthology.orchestra.run.smithi194.stdout:(82/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-16T05:20:45.184 INFO:teuthology.orchestra.run.smithi194.stdout:(83/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-04-16T05:20:45.217 INFO:teuthology.orchestra.run.smithi194.stdout:(84/174): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2024-04-16T05:20:45.251 INFO:teuthology.orchestra.run.smithi194.stdout:(85/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-04-16T05:20:45.285 INFO:teuthology.orchestra.run.smithi194.stdout:(86/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-16T05:20:45.319 INFO:teuthology.orchestra.run.smithi194.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-04-16T05:20:45.352 INFO:teuthology.orchestra.run.smithi194.stdout:(88/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 7.1 MB/s | 241 kB 00:00 2024-04-16T05:20:45.411 INFO:teuthology.orchestra.run.smithi194.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-04-16T05:20:45.444 INFO:teuthology.orchestra.run.smithi194.stdout:(90/174): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-04-16T05:20:45.478 INFO:teuthology.orchestra.run.smithi194.stdout:(91/174): python3-pycparser-2.20-6.el9.noarch.r 3.6 MB/s | 124 kB 00:00 2024-04-16T05:20:45.511 INFO:teuthology.orchestra.run.smithi194.stdout:(92/174): python3-urllib3-1.26.5-5.el9.noarch.r 5.5 MB/s | 187 kB 00:00 2024-04-16T05:20:45.645 INFO:teuthology.orchestra.run.smithi194.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-04-16T05:20:45.737 INFO:teuthology.orchestra.run.smithi194.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-16T05:20:45.779 INFO:teuthology.orchestra.run.smithi194.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-16T05:20:45.821 INFO:teuthology.orchestra.run.smithi194.stdout:(96/174): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-16T05:20:45.871 INFO:teuthology.orchestra.run.smithi194.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-16T05:20:45.918 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:20:45.961 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:20:45.971 INFO:teuthology.orchestra.run.smithi194.stdout:(98/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-04-16T05:20:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:20:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:20:46.619 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:20:46.619 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T05:20:46.619 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T05:20:46.619 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:20:46.651 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:20:47.171 INFO:teuthology.orchestra.run.smithi194.stdout:(99/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.4 MB/s | 4.1 MB 00:01 2024-04-16T05:20:47.305 INFO:teuthology.orchestra.run.smithi194.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.8 MB/s | 245 kB 00:00 2024-04-16T05:20:47.481 INFO:teuthology.orchestra.run.smithi194.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.4 MB/s | 248 kB 00:00 2024-04-16T05:20:47.564 INFO:teuthology.orchestra.run.smithi194.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-04-16T05:20:48.139 INFO:teuthology.orchestra.run.smithi194.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-04-16T05:20:49.864 INFO:teuthology.orchestra.run.smithi194.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 501 kB/s | 863 kB 00:01 2024-04-16T05:20:50.223 INFO:teuthology.orchestra.run.smithi194.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-16T05:20:50.290 INFO:teuthology.orchestra.run.smithi194.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-04-16T05:20:50.340 INFO:teuthology.orchestra.run.smithi194.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-04-16T05:20:50.557 INFO:teuthology.orchestra.run.smithi194.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-16T05:20:50.616 INFO:teuthology.orchestra.run.smithi194.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-04-16T05:20:50.683 INFO:teuthology.orchestra.run.smithi194.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 729 kB/s | 49 kB 00:00 2024-04-16T05:20:50.766 INFO:teuthology.orchestra.run.smithi194.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 810 kB/s | 67 kB 00:00 2024-04-16T05:20:50.849 INFO:teuthology.orchestra.run.smithi194.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-16T05:20:50.933 INFO:teuthology.orchestra.run.smithi194.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-16T05:20:51.009 INFO:teuthology.orchestra.run.smithi194.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-16T05:20:51.058 INFO:teuthology.orchestra.run.smithi194.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 874 kB/s | 43 kB 00:00 2024-04-16T05:20:51.126 INFO:teuthology.orchestra.run.smithi194.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-04-16T05:20:51.184 INFO:teuthology.orchestra.run.smithi194.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-04-16T05:20:51.260 INFO:teuthology.orchestra.run.smithi194.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-16T05:20:51.335 INFO:teuthology.orchestra.run.smithi194.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-16T05:20:51.382 INFO:teuthology.orchestra.run.smithi086.stdout:(8/174): ceph-radosgw-19.0.0-3101.ga87fcff6.el9 778 kB/s | 11 MB 00:13 2024-04-16T05:20:51.402 INFO:teuthology.orchestra.run.smithi194.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-04-16T05:20:51.461 INFO:teuthology.orchestra.run.smithi194.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 607 kB/s | 35 kB 00:00 2024-04-16T05:20:51.508 INFO:teuthology.orchestra.run.smithi086.stdout:(9/174): ceph-selinux-19.0.0-3101.ga87fcff6.el9 201 kB/s | 25 kB 00:00 2024-04-16T05:20:51.528 INFO:teuthology.orchestra.run.smithi194.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-04-16T05:20:51.645 INFO:teuthology.orchestra.run.smithi194.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-16T05:20:51.720 INFO:teuthology.orchestra.run.smithi194.stdout:(124/174): ceph-mgr-diskprediction-local-19.0.0 533 kB/s | 7.4 MB 00:14 2024-04-16T05:20:51.745 INFO:teuthology.orchestra.run.smithi194.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-16T05:20:51.787 INFO:teuthology.orchestra.run.smithi194.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-16T05:20:51.812 INFO:teuthology.orchestra.run.smithi194.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-16T05:20:51.846 INFO:teuthology.orchestra.run.smithi194.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 397 kB/s | 23 kB 00:00 2024-04-16T05:20:51.880 INFO:teuthology.orchestra.run.smithi194.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-04-16T05:20:51.913 INFO:teuthology.orchestra.run.smithi194.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-04-16T05:20:51.997 INFO:teuthology.orchestra.run.smithi194.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-16T05:20:52.022 INFO:teuthology.orchestra.run.smithi194.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-04-16T05:20:52.081 INFO:teuthology.orchestra.run.smithi194.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-04-16T05:20:52.097 INFO:teuthology.orchestra.run.smithi194.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-04-16T05:20:52.182 INFO:teuthology.orchestra.run.smithi194.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-16T05:20:52.198 INFO:teuthology.orchestra.run.smithi194.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-16T05:20:52.282 INFO:teuthology.orchestra.run.smithi194.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 899 kB/s | 90 kB 00:00 2024-04-16T05:20:52.298 INFO:teuthology.orchestra.run.smithi194.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-16T05:20:52.357 INFO:teuthology.orchestra.run.smithi194.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-16T05:20:52.399 INFO:teuthology.orchestra.run.smithi194.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-16T05:20:52.424 INFO:teuthology.orchestra.run.smithi194.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-04-16T05:20:52.458 INFO:teuthology.orchestra.run.smithi194.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-04-16T05:20:52.483 INFO:teuthology.orchestra.run.smithi194.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 871 kB/s | 51 kB 00:00 2024-04-16T05:20:52.559 INFO:teuthology.orchestra.run.smithi194.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-16T05:20:52.584 INFO:teuthology.orchestra.run.smithi194.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-04-16T05:20:52.626 INFO:teuthology.orchestra.run.smithi194.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-04-16T05:20:52.653 INFO:teuthology.orchestra.run.smithi194.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 289 kB/s | 20 kB 00:00 2024-04-16T05:20:52.687 INFO:teuthology.orchestra.run.smithi194.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.1 MB/s | 192 kB 00:00 2024-04-16T05:20:52.753 INFO:teuthology.orchestra.run.smithi194.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-16T05:20:55.954 INFO:teuthology.orchestra.run.smithi194.stdout:(150/174): librados2-19.0.0-3101.ga87fcff6.el9. 1.0 MB/s | 3.4 MB 00:03 2024-04-16T05:20:56.062 INFO:teuthology.orchestra.run.smithi194.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 21 MB/s | 2.3 MB 00:00 2024-04-16T05:20:56.271 INFO:teuthology.orchestra.run.smithi194.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 8.7 MB/s | 1.8 MB 00:00 2024-04-16T05:20:56.312 INFO:teuthology.orchestra.run.smithi194.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 986 kB/s | 41 kB 00:00 2024-04-16T05:20:56.355 INFO:teuthology.orchestra.run.smithi194.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-04-16T05:20:56.413 INFO:teuthology.orchestra.run.smithi194.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-04-16T05:20:56.588 INFO:teuthology.orchestra.run.smithi194.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 408 kB/s | 71 kB 00:00 2024-04-16T05:20:56.630 INFO:teuthology.orchestra.run.smithi194.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 876 kB/s | 36 kB 00:00 2024-04-16T05:20:56.756 INFO:teuthology.orchestra.run.smithi194.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 192 kB 00:00 2024-04-16T05:20:56.831 INFO:teuthology.orchestra.run.smithi194.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 256 kB 00:00 2024-04-16T05:20:56.915 INFO:teuthology.orchestra.run.smithi194.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 277 kB/s | 23 kB 00:00 2024-04-16T05:20:56.990 INFO:teuthology.orchestra.run.smithi194.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-04-16T05:20:57.057 INFO:teuthology.orchestra.run.smithi194.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-04-16T05:20:57.083 INFO:teuthology.orchestra.run.smithi086.stdout:(10/174): ceph-common-19.0.0-3101.ga87fcff6.el9 619 kB/s | 21 MB 00:34 2024-04-16T05:20:57.124 INFO:teuthology.orchestra.run.smithi194.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-04-16T05:20:57.182 INFO:teuthology.orchestra.run.smithi194.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-04-16T05:20:57.224 INFO:teuthology.orchestra.run.smithi194.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 690 kB/s | 29 kB 00:00 2024-04-16T05:20:57.366 INFO:teuthology.orchestra.run.smithi194.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 8.5 MB/s | 1.2 MB 00:00 2024-04-16T05:20:57.483 INFO:teuthology.orchestra.run.smithi194.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-04-16T05:20:57.592 INFO:teuthology.orchestra.run.smithi194.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 21 MB/s | 2.3 MB 00:00 2024-04-16T05:20:57.624 INFO:teuthology.orchestra.run.smithi086.stdout:(11/174): cephfs-mirror-19.0.0-3101.ga87fcff6.e 414 kB/s | 224 kB 00:00 2024-04-16T05:20:57.642 INFO:teuthology.orchestra.run.smithi194.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 9.1 MB/s | 465 kB 00:00 2024-04-16T05:20:57.701 INFO:teuthology.orchestra.run.smithi194.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 535 kB/s | 31 kB 00:00 2024-04-16T05:20:57.767 INFO:teuthology.orchestra.run.smithi086.stdout:(12/174): cephfs-shell-19.0.0-3101.ga87fcff6.el 190 kB/s | 27 kB 00:00 2024-04-16T05:20:57.859 INFO:teuthology.orchestra.run.smithi194.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-04-16T05:20:57.883 INFO:teuthology.orchestra.run.smithi086.stdout:(13/174): libcephfs-devel-19.0.0-3101.ga87fcff6 272 kB/s | 32 kB 00:00 2024-04-16T05:20:57.934 INFO:teuthology.orchestra.run.smithi194.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 992 kB/s | 74 kB 00:00 2024-04-16T05:20:58.389 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:20:58.389 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-04-16T05:20:58.389 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-04-16T05:20:58.389 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-04-16T05:20:58.389 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-04-16T05:20:58.389 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-04-16T05:20:58.390 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-04-16T05:20:58.390 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-04-16T05:20:58.390 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:20:58.554 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:20:58.569 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:20:58.569 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:20:58.570 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T05:20:58.570 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:20:59.234 INFO:teuthology.orchestra.run.smithi086.stdout:(14/174): libcephfs2-19.0.0-3101.ga87fcff6.el9. 553 kB/s | 746 kB 00:01 2024-04-16T05:20:59.567 INFO:teuthology.orchestra.run.smithi086.stdout:(15/174): libcephsqlite-19.0.0-3101.ga87fcff6.e 490 kB/s | 163 kB 00:00 2024-04-16T05:20:59.859 INFO:teuthology.orchestra.run.smithi086.stdout:(16/174): librados-devel-19.0.0-3101.ga87fcff6. 434 kB/s | 127 kB 00:00 2024-04-16T05:21:00.074 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:21:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:21:00.099 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:21:00.099 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T05:21:00.099 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:21:00.100 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:21:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:00.325 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:21:00.347 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:21:00.347 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:21:00.347 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T05:21:00.347 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:21:00.347 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:21:00.347 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:00.651 INFO:teuthology.orchestra.run.smithi086.stdout:(17/174): libradosstriper1-19.0.0-3101.ga87fcff 633 kB/s | 501 kB 00:00 2024-04-16T05:21:00.792 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:21:00.814 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:21:00.814 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:21:00.814 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T05:21:00.815 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:21:00.815 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:21:00.815 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:00.902 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T05:21:01.312 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:21:01.342 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:21:01.375 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 139/198 2024-04-16T05:21:01.435 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 140/198 2024-04-16T05:21:01.465 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:21:02.101 INFO:teuthology.orchestra.run.smithi194.stdout:(173/174): librbd1-19.0.0-3101.ga87fcff6.el9.x8 344 kB/s | 3.1 MB 00:09 2024-04-16T05:21:02.499 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:21:02.533 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:21:03.488 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:21:03.509 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:21:03.607 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:21:03.771 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 144/198 2024-04-16T05:21:03.794 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:21:03.817 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:21:03.817 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:21:03.817 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T05:21:03.817 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:21:03.817 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:21:03.817 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:03.862 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:21:03.872 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:21:03.917 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 147/198 2024-04-16T05:21:04.134 INFO:teuthology.orchestra.run.smithi086.stdout:(18/174): ceph-osd-19.0.0-3101.ga87fcff6.el9.x8 519 kB/s | 17 MB 00:33 2024-04-16T05:21:04.518 INFO:teuthology.orchestra.run.smithi086.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.ga8 118 kB/s | 45 kB 00:00 2024-04-16T05:21:04.843 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T05:21:04.872 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:21:04.895 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:21:04.895 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:21:04.895 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T05:21:04.895 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:21:04.895 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:21:04.895 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:21:04.943 INFO:teuthology.orchestra.run.smithi086.stdout:(20/174): python3-ceph-common-19.0.0-3101.ga87f 317 kB/s | 135 kB 00:00 2024-04-16T05:21:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:21:04.961 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:21:04.961 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T05:21:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:21:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:21:05.028 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:21:05.029 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T05:21:05.029 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:21:05.029 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:21:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:05.333 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:21:05.355 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:21:05.355 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:21:05.355 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T05:21:05.355 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:21:05.355 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:21:05.355 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:05.435 INFO:teuthology.orchestra.run.smithi086.stdout:(21/174): python3-cephfs-19.0.0-3101.ga87fcff6. 329 kB/s | 161 kB 00:00 2024-04-16T05:21:06.293 INFO:teuthology.orchestra.run.smithi086.stdout:(22/174): python3-rados-19.0.0-3101.ga87fcff6.e 374 kB/s | 321 kB 00:00 2024-04-16T05:21:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:(23/174): python3-rbd-19.0.0-3101.ga87fcff6.el9 441 kB/s | 298 kB 00:00 2024-04-16T05:21:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:(24/174): python3-rgw-19.0.0-3101.ga87fcff6.el9 323 kB/s | 100 kB 00:00 2024-04-16T05:21:07.502 INFO:teuthology.orchestra.run.smithi086.stdout:(25/174): rbd-fuse-19.0.0-3101.ga87fcff6.el9.x8 379 kB/s | 85 kB 00:00 2024-04-16T05:21:09.727 INFO:teuthology.orchestra.run.smithi086.stdout:(26/174): librgw2-19.0.0-3101.ga87fcff6.el9.x86 606 kB/s | 5.4 MB 00:09 2024-04-16T05:21:09.929 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:21:09.970 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 154/198 2024-04-16T05:21:10.000 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 155/198 2024-04-16T05:21:10.019 INFO:teuthology.orchestra.run.smithi086.stdout:(27/174): rbd-nbd-19.0.0-3101.ga87fcff6.el9.x86 585 kB/s | 170 kB 00:00 2024-04-16T05:21:10.027 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T05:21:10.119 INFO:teuthology.orchestra.run.smithi086.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 341 kB/s | 34 kB 00:00 2024-04-16T05:21:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 157/198 2024-04-16T05:21:10.318 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T05:21:10.367 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 159/198 2024-04-16T05:21:10.406 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T05:21:10.420 INFO:teuthology.orchestra.run.smithi086.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.ga87fcff 474 kB/s | 142 kB 00:00 2024-04-16T05:21:10.459 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T05:21:10.502 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T05:21:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 163/198 2024-04-16T05:21:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T05:21:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T05:21:10.634 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T05:21:10.694 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T05:21:10.728 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T05:21:11.305 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:21:11.343 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:21:11.363 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:21:12.181 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 170/198 2024-04-16T05:21:12.211 INFO:teuthology.orchestra.run.smithi086.stdout:(30/174): rbd-mirror-19.0.0-3101.ga87fcff6.el9. 672 kB/s | 3.1 MB 00:04 2024-04-16T05:21:12.264 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T05:21:12.304 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T05:21:12.385 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T05:21:12.434 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T05:21:12.459 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T05:21:12.484 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T05:21:12.511 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T05:21:12.511 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:21:12.599 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:21:12.629 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T05:21:12.629 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:21:12.722 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:21:12.748 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T05:21:12.781 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T05:21:12.806 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T05:21:12.831 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T05:21:12.864 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T05:21:12.904 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T05:21:12.936 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T05:21:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T05:21:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T05:21:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:21:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T05:21:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:21:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:21:13.223 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:21:13.224 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:21:13.307 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:21:13.338 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:21:13.365 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T05:21:13.390 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T05:21:13.415 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T05:21:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:21:15.178 INFO:teuthology.orchestra.run.smithi086.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3101.ga87fc 762 kB/s | 3.5 MB 00:04 2024-04-16T05:21:15.462 INFO:teuthology.orchestra.run.smithi086.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.ga8 878 kB/s | 249 kB 00:00 2024-04-16T05:21:15.562 INFO:teuthology.orchestra.run.smithi086.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.ga87fcff6.e 493 kB/s | 49 kB 00:00 2024-04-16T05:21:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.ga 152 kB/s | 16 kB 00:00 2024-04-16T05:21:16.004 INFO:teuthology.orchestra.run.smithi086.stdout:(35/174): ceph-volume-19.0.0-3101.ga87fcff6.el9 819 kB/s | 273 kB 00:00 2024-04-16T05:21:16.787 INFO:teuthology.orchestra.run.smithi086.stdout:(36/174): cephadm-19.0.0-3101.ga87fcff6.el9.noa 973 kB/s | 762 kB 00:00 2024-04-16T05:21:16.913 INFO:teuthology.orchestra.run.smithi086.stdout:(37/174): cephfs-top-19.0.0-3101.ga87fcff6.el9. 146 kB/s | 18 kB 00:00 2024-04-16T05:21:17.054 INFO:teuthology.orchestra.run.smithi086.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-04-16T05:21:17.113 INFO:teuthology.orchestra.run.smithi086.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-04-16T05:21:17.147 INFO:teuthology.orchestra.run.smithi086.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-16T05:21:17.189 INFO:teuthology.orchestra.run.smithi086.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-04-16T05:21:17.222 INFO:teuthology.orchestra.run.smithi086.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 954 kB/s | 32 kB 00:00 2024-04-16T05:21:17.331 INFO:teuthology.orchestra.run.smithi086.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-16T05:21:17.365 INFO:teuthology.orchestra.run.smithi086.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 514 kB/s | 17 kB 00:00 2024-04-16T05:21:17.407 INFO:teuthology.orchestra.run.smithi086.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 241 kB/s | 10 kB 00:00 2024-04-16T05:21:17.448 INFO:teuthology.orchestra.run.smithi086.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-04-16T05:21:17.482 INFO:teuthology.orchestra.run.smithi086.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-16T05:21:17.549 INFO:teuthology.orchestra.run.smithi086.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-04-16T05:21:17.608 INFO:teuthology.orchestra.run.smithi086.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 578 kB 00:00 2024-04-16T05:21:17.641 INFO:teuthology.orchestra.run.smithi086.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-16T05:21:17.700 INFO:teuthology.orchestra.run.smithi086.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-04-16T05:21:17.734 INFO:teuthology.orchestra.run.smithi086.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2024-04-16T05:21:17.767 INFO:teuthology.orchestra.run.smithi086.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 6.4 MB/s | 217 kB 00:00 2024-04-16T05:21:17.792 INFO:teuthology.orchestra.run.smithi086.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2024-04-16T05:21:17.934 INFO:teuthology.orchestra.run.smithi086.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-16T05:21:18.035 INFO:teuthology.orchestra.run.smithi086.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.5 MB 00:00 2024-04-16T05:21:18.085 INFO:teuthology.orchestra.run.smithi086.stdout:(57/174): postgresql-contrib-13.14-1.el9.x86_64 16 MB/s | 817 kB 00:00 2024-04-16T05:21:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:(58/174): postgresql-private-libs-13.14-1.el9.x 3.9 MB/s | 132 kB 00:00 2024-04-16T05:21:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:(59/174): postgresql-server-13.14-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-04-16T05:21:18.327 INFO:teuthology.orchestra.run.smithi086.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-16T05:21:18.361 INFO:teuthology.orchestra.run.smithi086.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-04-16T05:21:18.512 INFO:teuthology.orchestra.run.smithi086.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 39 MB/s | 5.8 MB 00:00 2024-04-16T05:21:18.537 INFO:teuthology.orchestra.run.smithi086.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 978 kB/s | 24 kB 00:00 2024-04-16T05:21:18.570 INFO:teuthology.orchestra.run.smithi086.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 546 kB/s | 18 kB 00:00 2024-04-16T05:21:18.604 INFO:teuthology.orchestra.run.smithi086.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 4.2 MB/s | 144 kB 00:00 2024-04-16T05:21:18.637 INFO:teuthology.orchestra.run.smithi086.stdout:(66/174): python3-markupsafe-1.1.1-12.el9.x86_6 948 kB/s | 32 kB 00:00 2024-04-16T05:21:18.788 INFO:teuthology.orchestra.run.smithi086.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-04-16T05:21:18.821 INFO:teuthology.orchestra.run.smithi086.stdout:(68/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 6.4 MB/s | 217 kB 00:00 2024-04-16T05:21:18.859 INFO:teuthology.orchestra.run.smithi086.stdout:(69/174): python3-oauthlib-3.1.1-5.el9.noarch.r 5.1 MB/s | 174 kB 00:00 2024-04-16T05:21:18.877 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 1/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 3/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 4/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 5/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 6/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 7/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 8/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 9/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:21:18.878 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 13/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 14/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 15/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 17/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 18/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 19/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 20/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 21/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 22/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 23/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 24/198 2024-04-16T05:21:18.879 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 25/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 26/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 27/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 28/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 29/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 30/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 31/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 32/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 33/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 34/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 35/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 36/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 37/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 38/198 2024-04-16T05:21:18.880 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 39/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T05:21:18.881 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T05:21:18.882 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T05:21:18.883 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:21:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T05:21:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T05:21:18.886 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T05:21:18.887 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T05:21:18.888 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T05:21:18.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T05:21:18.890 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T05:21:18.891 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T05:21:18.892 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T05:21:18.893 INFO:teuthology.orchestra.run.smithi086.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 2.0 MB/s | 69 kB 00:00 2024-04-16T05:21:18.922 INFO:teuthology.orchestra.run.smithi086.stdout:(71/174): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-04-16T05:21:18.964 INFO:teuthology.orchestra.run.smithi086.stdout:(72/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-04-16T05:21:18.998 INFO:teuthology.orchestra.run.smithi086.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.9 MB/s | 132 kB 00:00 2024-04-16T05:21:19.040 INFO:teuthology.orchestra.run.smithi086.stdout:(74/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 211 kB 00:00 2024-04-16T05:21:19.082 INFO:teuthology.orchestra.run.smithi086.stdout:(75/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-04-16T05:21:19.132 INFO:teuthology.orchestra.run.smithi086.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 855 kB/s | 43 kB 00:00 2024-04-16T05:21:19.583 INFO:teuthology.orchestra.run.smithi086.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-16T05:21:19.624 INFO:teuthology.orchestra.run.smithi086.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 931 kB/s | 39 kB 00:00 2024-04-16T05:21:19.658 INFO:teuthology.orchestra.run.smithi086.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.2 MB/s | 41 kB 00:00 2024-04-16T05:21:19.717 INFO:teuthology.orchestra.run.smithi086.stdout:(80/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-04-16T05:21:19.775 INFO:teuthology.orchestra.run.smithi086.stdout:(81/174): qatlib-service-24.02.0-1.el9.x86_64.r 597 kB/s | 35 kB 00:00 2024-04-16T05:21:19.842 INFO:teuthology.orchestra.run.smithi086.stdout:(82/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 693 kB/s | 46 kB 00:00 2024-04-16T05:21:19.926 INFO:teuthology.orchestra.run.smithi086.stdout:(83/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-04-16T05:21:19.993 INFO:teuthology.orchestra.run.smithi086.stdout:(84/174): uuid-1.6.2-55.el9.x86_64.rpm 846 kB/s | 56 kB 00:00 2024-04-16T05:21:20.051 INFO:teuthology.orchestra.run.smithi086.stdout:(85/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-04-16T05:21:20.110 INFO:teuthology.orchestra.run.smithi086.stdout:(86/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 754 kB/s | 44 kB 00:00 2024-04-16T05:21:20.144 INFO:teuthology.orchestra.run.smithi086.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-04-16T05:21:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:(88/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-04-16T05:21:20.253 INFO:teuthology.orchestra.run.smithi086.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-04-16T05:21:20.303 INFO:teuthology.orchestra.run.smithi086.stdout:(90/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-04-16T05:21:20.345 INFO:teuthology.orchestra.run.smithi086.stdout:(91/174): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-04-16T05:21:20.387 INFO:teuthology.orchestra.run.smithi086.stdout:(92/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.4 MB/s | 187 kB 00:00 2024-04-16T05:21:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 774 kB/s | 187 kB 00:00 2024-04-16T05:21:20.954 INFO:teuthology.orchestra.run.smithi086.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-04-16T05:21:21.054 INFO:teuthology.orchestra.run.smithi086.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-16T05:21:21.113 INFO:teuthology.orchestra.run.smithi086.stdout:(96/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-16T05:21:21.172 INFO:teuthology.orchestra.run.smithi086.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 369 kB/s | 21 kB 00:00 2024-04-16T05:21:21.230 INFO:teuthology.orchestra.run.smithi086.stdout:(98/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 438 kB/s | 25 kB 00:00 2024-04-16T05:21:22.048 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T05:21:22.049 INFO:teuthology.orchestra.run.smithi196.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T05:21:22.050 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.051 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.052 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T05:21:22.053 INFO:teuthology.orchestra.run.smithi196.stdout: jq-1.6-16.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.054 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T05:21:22.055 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T05:21:22.056 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T05:21:22.057 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T05:21:22.058 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T05:21:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T05:21:22.060 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.061 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:21:22.062 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T05:21:22.062 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T05:21:22.062 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T05:21:22.062 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T05:21:22.062 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T05:21:22.062 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T05:21:22.062 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T05:21:22.062 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:21:22.063 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-04-16T05:21:22.206 INFO:teuthology.orchestra.run.smithi086.stdout:(99/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 4.2 MB/s | 4.1 MB 00:00 2024-04-16T05:21:22.265 INFO:teuthology.orchestra.run.smithi086.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 4.1 MB/s | 245 kB 00:00 2024-04-16T05:21:22.323 INFO:teuthology.orchestra.run.smithi086.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 4.2 MB/s | 248 kB 00:00 2024-04-16T05:21:22.381 INFO:teuthology.orchestra.run.smithi086.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-04-16T05:21:22.457 INFO:teuthology.orchestra.run.smithi086.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-16T05:21:22.533 DEBUG:teuthology.parallel:result is None 2024-04-16T05:21:22.599 INFO:teuthology.orchestra.run.smithi086.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-04-16T05:21:22.941 INFO:teuthology.orchestra.run.smithi086.stdout:(105/174): ceph-mgr-diskprediction-local-19.0.0 707 kB/s | 7.4 MB 00:10 2024-04-16T05:21:22.966 INFO:teuthology.orchestra.run.smithi086.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-16T05:21:23.075 INFO:teuthology.orchestra.run.smithi086.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-04-16T05:21:23.342 INFO:teuthology.orchestra.run.smithi086.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-16T05:21:23.392 INFO:teuthology.orchestra.run.smithi086.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-04-16T05:21:23.442 INFO:teuthology.orchestra.run.smithi086.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 976 kB/s | 49 kB 00:00 2024-04-16T05:21:23.501 INFO:teuthology.orchestra.run.smithi086.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-16T05:21:23.559 INFO:teuthology.orchestra.run.smithi086.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-16T05:21:23.643 INFO:teuthology.orchestra.run.smithi086.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-16T05:21:23.710 INFO:teuthology.orchestra.run.smithi086.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-04-16T05:21:23.760 INFO:teuthology.orchestra.run.smithi086.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 869 kB/s | 43 kB 00:00 2024-04-16T05:21:23.802 INFO:teuthology.orchestra.run.smithi086.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 773 kB/s | 32 kB 00:00 2024-04-16T05:21:23.844 INFO:teuthology.orchestra.run.smithi086.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-04-16T05:21:23.911 INFO:teuthology.orchestra.run.smithi086.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-16T05:21:23.978 INFO:teuthology.orchestra.run.smithi086.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-16T05:21:24.078 INFO:teuthology.orchestra.run.smithi086.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-16T05:21:24.154 INFO:teuthology.orchestra.run.smithi086.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-04-16T05:21:24.212 INFO:teuthology.orchestra.run.smithi086.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-16T05:21:24.271 INFO:teuthology.orchestra.run.smithi086.stdout:(123/174): gperftools-libs-2.9.1-2.el9.x86_64.r 232 kB/s | 309 kB 00:01 2024-04-16T05:21:24.355 INFO:teuthology.orchestra.run.smithi086.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-16T05:21:24.379 INFO:teuthology.orchestra.run.smithi086.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-16T05:21:24.405 INFO:teuthology.orchestra.run.smithi086.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-16T05:21:24.430 INFO:teuthology.orchestra.run.smithi086.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-04-16T05:21:24.464 INFO:teuthology.orchestra.run.smithi086.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-16T05:21:24.489 INFO:teuthology.orchestra.run.smithi086.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-16T05:21:24.515 INFO:teuthology.orchestra.run.smithi086.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-16T05:21:24.598 INFO:teuthology.orchestra.run.smithi086.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 556 kB/s | 46 kB 00:00 2024-04-16T05:21:24.649 INFO:teuthology.orchestra.run.smithi086.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-04-16T05:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-16T05:21:24.699 INFO:teuthology.orchestra.run.smithi086.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-16T05:21:24.732 INFO:teuthology.orchestra.run.smithi086.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-16T05:21:24.758 INFO:teuthology.orchestra.run.smithi086.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-16T05:21:24.783 INFO:teuthology.orchestra.run.smithi086.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-16T05:21:24.825 INFO:teuthology.orchestra.run.smithi086.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-04-16T05:21:24.839 INFO:teuthology.orchestra.run.smithi194.stdout:(174/174): ceph-test-19.0.0-3101.ga87fcff6.el9. 806 kB/s | 49 MB 01:02 2024-04-16T05:21:24.845 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:21:24.845 INFO:teuthology.orchestra.run.smithi194.stdout:Total 2.6 MB/s | 229 MB 01:26 2024-04-16T05:21:24.850 INFO:teuthology.orchestra.run.smithi086.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-16T05:21:24.892 INFO:teuthology.orchestra.run.smithi086.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-04-16T05:21:24.917 INFO:teuthology.orchestra.run.smithi086.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 884 kB/s | 59 kB 00:00 2024-04-16T05:21:24.943 INFO:teuthology.orchestra.run.smithi086.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-04-16T05:21:24.968 INFO:teuthology.orchestra.run.smithi086.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-16T05:21:25.002 INFO:teuthology.orchestra.run.smithi086.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-16T05:21:25.027 INFO:teuthology.orchestra.run.smithi086.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-16T05:21:25.086 INFO:teuthology.orchestra.run.smithi086.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 343 kB/s | 20 kB 00:00 2024-04-16T05:21:25.128 INFO:teuthology.orchestra.run.smithi086.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-16T05:21:25.153 INFO:teuthology.orchestra.run.smithi086.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-16T05:21:25.202 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-16T05:21:25.228 INFO:teuthology.orchestra.run.smithi086.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-16T05:21:25.472 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-16T05:21:25.472 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-16T05:21:27.192 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-16T05:21:27.192 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-16T05:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout:(150/174): librbd1-19.0.0-3101.ga87fcff6.el9.x8 815 kB/s | 3.1 MB 00:03 2024-04-16T05:21:29.270 INFO:teuthology.orchestra.run.smithi086.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-04-16T05:21:29.353 INFO:teuthology.orchestra.run.smithi086.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-04-16T05:21:29.387 INFO:teuthology.orchestra.run.smithi086.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-16T05:21:29.429 INFO:teuthology.orchestra.run.smithi086.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-04-16T05:21:29.471 INFO:teuthology.orchestra.run.smithi086.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-16T05:21:29.638 INFO:teuthology.orchestra.run.smithi086.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 428 kB/s | 71 kB 00:00 2024-04-16T05:21:29.697 INFO:teuthology.orchestra.run.smithi086.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 625 kB/s | 36 kB 00:00 2024-04-16T05:21:29.814 INFO:teuthology.orchestra.run.smithi086.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-04-16T05:21:29.922 INFO:teuthology.orchestra.run.smithi086.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-04-16T05:21:29.989 INFO:teuthology.orchestra.run.smithi086.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 345 kB/s | 23 kB 00:00 2024-04-16T05:21:30.073 INFO:teuthology.orchestra.run.smithi086.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-04-16T05:21:30.148 INFO:teuthology.orchestra.run.smithi086.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-04-16T05:21:30.207 INFO:teuthology.orchestra.run.smithi086.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-04-16T05:21:30.274 INFO:teuthology.orchestra.run.smithi086.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 960 kB/s | 64 kB 00:00 2024-04-16T05:21:30.341 INFO:teuthology.orchestra.run.smithi086.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 430 kB/s | 29 kB 00:00 2024-04-16T05:21:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 4.0 MB/s | 1.2 MB 00:00 2024-04-16T05:21:30.983 INFO:teuthology.orchestra.run.smithi086.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.3 MB/s | 2.2 MB 00:00 2024-04-16T05:21:31.300 INFO:teuthology.orchestra.run.smithi086.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.2 MB/s | 2.3 MB 00:00 2024-04-16T05:21:31.350 INFO:teuthology.orchestra.run.smithi086.stdout:(169/174): librados2-19.0.0-3101.ga87fcff6.el9. 561 kB/s | 3.4 MB 00:06 2024-04-16T05:21:31.379 INFO:teuthology.orchestra.run.smithi086.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 5.8 MB/s | 465 kB 00:00 2024-04-16T05:21:31.496 INFO:teuthology.orchestra.run.smithi086.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 214 kB/s | 31 kB 00:00 2024-04-16T05:21:31.538 INFO:teuthology.orchestra.run.smithi086.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-04-16T05:21:31.638 INFO:teuthology.orchestra.run.smithi086.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 523 kB/s | 74 kB 00:00 2024-04-16T05:21:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-16T05:21:34.176 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T05:21:34.210 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T05:21:34.241 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:21:34.249 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:21:34.290 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T05:21:34.344 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T05:21:34.377 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T05:21:34.403 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T05:21:34.445 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T05:21:34.766 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T05:21:34.829 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:21:34.950 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:21:34.980 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:21:35.071 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:21:35.100 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:21:35.142 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T05:21:35.315 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T05:21:35.390 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T05:21:35.418 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:21:35.521 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:21:35.544 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:21:35.591 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:21:35.631 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T05:21:35.653 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:21:35.669 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:21:35.862 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:21:35.893 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:21:35.925 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T05:21:36.024 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T05:21:36.114 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:21:36.163 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:21:36.250 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:36.305 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:36.535 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:36.646 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T05:21:36.739 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T05:21:36.768 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T05:21:36.794 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T05:21:36.830 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T05:21:36.904 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T05:21:36.958 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T05:21:36.997 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T05:21:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T05:21:37.103 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T05:21:37.147 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T05:21:37.179 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T05:21:37.207 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T05:21:37.276 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T05:21:37.304 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 39/198 2024-04-16T05:21:37.339 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 40/198 2024-04-16T05:21:37.368 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 41/198 2024-04-16T05:21:37.436 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T05:21:37.490 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T05:21:37.564 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T05:21:37.697 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T05:21:37.750 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T05:21:38.264 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T05:21:38.346 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T05:21:38.430 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T05:21:38.466 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:21:38.475 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:21:38.506 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T05:21:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T05:21:38.593 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 53/198 2024-04-16T05:21:38.622 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T05:21:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T05:21:38.989 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T05:21:39.057 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T05:21:39.101 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T05:21:39.226 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T05:21:39.287 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T05:21:39.340 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T05:21:39.413 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T05:21:39.444 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T05:21:39.481 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T05:21:39.528 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T05:21:39.563 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T05:21:39.606 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T05:21:39.648 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T05:21:39.782 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T05:21:39.828 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T05:21:39.859 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T05:21:39.900 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T05:21:39.993 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T05:21:40.025 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T05:21:40.065 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T05:21:40.091 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T05:21:40.115 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T05:21:40.187 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T05:21:40.315 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T05:21:40.423 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T05:21:40.465 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T05:21:40.513 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T05:21:40.569 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T05:21:40.759 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T05:21:40.802 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T05:21:40.930 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T05:21:40.984 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T05:21:41.016 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T05:21:41.047 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:21:41.077 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T05:21:41.109 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T05:21:41.148 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T05:21:41.179 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T05:21:41.209 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T05:21:41.284 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T05:21:41.319 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T05:21:41.353 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T05:21:41.394 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T05:21:41.429 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T05:21:41.517 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T05:21:41.599 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T05:21:41.643 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T05:21:41.674 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T05:21:41.722 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T05:21:41.819 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T05:21:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T05:21:42.592 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T05:21:42.639 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T05:21:42.673 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T05:21:42.721 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T05:21:42.799 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T05:21:42.849 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T05:21:43.027 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T05:21:43.055 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T05:21:43.132 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T05:21:43.756 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T05:21:43.859 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T05:21:45.058 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T05:21:45.137 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T05:21:45.168 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T05:21:45.199 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T05:21:45.227 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T05:21:45.538 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T05:21:45.565 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:21:45.639 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:21:45.662 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 125/198 2024-04-16T05:21:45.697 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T05:21:46.174 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T05:21:46.202 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:21:46.264 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:21:46.286 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 129/198 2024-04-16T05:21:48.589 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:21:48.632 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:21:49.167 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:21:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:21:49.266 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:21:49.267 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T05:21:49.267 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T05:21:49.267 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:21:49.295 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:22:01.220 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:22:01.220 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /sys 2024-04-16T05:22:01.220 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /proc 2024-04-16T05:22:01.220 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /mnt 2024-04-16T05:22:01.220 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /var/tmp 2024-04-16T05:22:01.220 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /home 2024-04-16T05:22:01.220 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /root 2024-04-16T05:22:01.220 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /tmp 2024-04-16T05:22:01.220 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:01.394 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:01.395 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T05:22:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:02.899 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:02.899 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:02.899 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T05:22:02.900 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:02.900 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:02.900 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:03.150 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:03.173 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:03.173 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T05:22:03.173 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:03.173 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:03.627 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:03.650 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:03.650 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:03.650 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T05:22:03.650 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:03.650 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:03.650 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T05:22:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:04.168 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:04.200 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 139/198 2024-04-16T05:22:04.261 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 140/198 2024-04-16T05:22:04.291 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:05.331 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:05.360 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:06.323 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:06.343 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:06.443 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:06.597 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 144/198 2024-04-16T05:22:06.637 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:06.659 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:06.659 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:06.660 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T05:22:06.660 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:06.660 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:06.660 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:06.704 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:06.715 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:06.761 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 147/198 2024-04-16T05:22:07.690 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T05:22:07.723 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:07.745 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:07.745 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:07.746 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T05:22:07.746 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:07.746 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:07.746 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:07.792 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:07.814 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:07.814 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:07.814 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T05:22:07.814 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:07.859 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:07.881 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:07.881 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T05:22:07.881 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:07.881 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:07.977 INFO:teuthology.orchestra.run.smithi086.stdout:(174/174): ceph-test-19.0.0-3101.ga87fcff6.el9. 657 kB/s | 49 MB 01:16 2024-04-16T05:22:07.981 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:22:07.981 INFO:teuthology.orchestra.run.smithi086.stdout:Total 2.2 MB/s | 229 MB 01:46 2024-04-16T05:22:08.193 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:08.216 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:08.216 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:08.216 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T05:22:08.216 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:08.216 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:08.216 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:08.342 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-04-16T05:22:08.615 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-04-16T05:22:08.616 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-04-16T05:22:10.710 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-04-16T05:22:10.711 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-04-16T05:22:12.772 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:22:12.813 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 154/198 2024-04-16T05:22:12.852 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 155/198 2024-04-16T05:22:12.879 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T05:22:13.064 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 157/198 2024-04-16T05:22:13.178 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T05:22:13.218 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 159/198 2024-04-16T05:22:13.257 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T05:22:13.310 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T05:22:13.353 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T05:22:13.378 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 163/198 2024-04-16T05:22:13.410 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T05:22:13.440 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T05:22:13.494 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T05:22:13.537 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T05:22:13.562 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T05:22:14.157 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:14.210 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:14.230 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:15.045 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 170/198 2024-04-16T05:22:15.130 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T05:22:15.170 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T05:22:15.252 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T05:22:15.300 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T05:22:15.325 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T05:22:15.350 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T05:22:15.376 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T05:22:15.377 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:15.464 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:15.494 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T05:22:15.494 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:15.579 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:15.605 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T05:22:15.646 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T05:22:15.672 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T05:22:15.697 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T05:22:15.730 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T05:22:15.769 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T05:22:15.801 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T05:22:15.827 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T05:22:15.852 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T05:22:15.877 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:22:15.902 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T05:22:15.908 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:15.908 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:16.088 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:16.088 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:16.155 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:16.186 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:22:16.238 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T05:22:16.264 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T05:22:16.297 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T05:22:16.361 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:22:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-04-16T05:22:17.715 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T05:22:17.749 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T05:22:17.779 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:22:17.789 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:22:17.837 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T05:22:17.897 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T05:22:17.932 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T05:22:17.958 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T05:22:18.002 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T05:22:18.328 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T05:22:18.375 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:22:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:22:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:22:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:22:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:22:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T05:22:18.900 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T05:22:18.955 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T05:22:18.983 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:22:19.086 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:22:19.109 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:22:19.173 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:22:19.221 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T05:22:19.243 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:22:19.259 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:22:19.462 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:22:19.492 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:22:19.515 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T05:22:19.697 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T05:22:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:22:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:22:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:22:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:22:20.257 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:22:20.368 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T05:22:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T05:22:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T05:22:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T05:22:20.587 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T05:22:20.660 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T05:22:20.723 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T05:22:20.761 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T05:22:20.825 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T05:22:20.861 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T05:22:20.904 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T05:22:20.937 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T05:22:20.965 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T05:22:21.034 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T05:22:21.078 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 39/198 2024-04-16T05:22:21.114 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 40/198 2024-04-16T05:22:21.142 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 41/198 2024-04-16T05:22:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T05:22:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T05:22:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T05:22:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T05:22:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T05:22:21.645 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:22:21.645 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 1/198 2024-04-16T05:22:21.646 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2/198 2024-04-16T05:22:21.646 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 3/198 2024-04-16T05:22:21.646 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 4/198 2024-04-16T05:22:21.646 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 5/198 2024-04-16T05:22:21.646 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 6/198 2024-04-16T05:22:21.646 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 7/198 2024-04-16T05:22:21.646 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 8/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 9/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 13/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 14/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 15/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 17/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 18/198 2024-04-16T05:22:21.647 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 19/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 20/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 21/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 22/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 23/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 24/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 25/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 26/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 27/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 28/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 29/198 2024-04-16T05:22:21.648 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 30/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 31/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 32/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 33/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 34/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 35/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 36/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 37/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 38/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 39/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T05:22:21.649 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T05:22:21.650 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T05:22:21.652 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T05:22:21.653 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T05:22:21.653 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T05:22:21.653 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T05:22:21.653 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T05:22:21.653 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T05:22:21.653 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T05:22:21.653 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T05:22:21.653 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T05:22:21.653 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T05:22:21.654 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T05:22:21.655 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T05:22:21.656 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T05:22:21.656 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T05:22:21.656 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T05:22:21.656 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T05:22:21.656 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T05:22:21.656 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T05:22:21.656 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T05:22:21.656 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T05:22:21.656 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T05:22:21.657 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T05:22:21.658 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T05:22:21.659 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T05:22:21.660 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T05:22:21.661 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T05:22:21.662 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T05:22:21.663 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T05:22:21.664 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T05:22:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T05:22:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T05:22:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T05:22:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:22:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:22:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T05:22:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T05:22:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 53/198 2024-04-16T05:22:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T05:22:22.697 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T05:22:22.878 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T05:22:22.960 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T05:22:23.003 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T05:22:23.154 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T05:22:23.198 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T05:22:23.253 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T05:22:23.325 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T05:22:23.357 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T05:22:23.385 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T05:22:23.432 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T05:22:23.468 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T05:22:23.510 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T05:22:23.553 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T05:22:23.686 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T05:22:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T05:22:23.763 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T05:22:23.804 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T05:22:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T05:22:23.938 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T05:22:23.978 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T05:22:24.004 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T05:22:24.028 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T05:22:24.127 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T05:22:24.252 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T05:22:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T05:22:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T05:22:24.443 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T05:22:24.499 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T05:22:24.682 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T05:22:24.688 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T05:22:24.688 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:24.688 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2024-04-16T05:22:24.688 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:24.688 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:24.688 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T05:22:24.689 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T05:22:24.690 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.691 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T05:22:24.692 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: jq-1.6-16.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T05:22:24.693 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T05:22:24.694 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T05:22:24.695 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T05:22:24.697 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T05:22:24.698 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T05:22:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-16T05:22:24.701 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-16T05:22:24.748 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T05:22:24.875 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T05:22:24.913 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T05:22:24.945 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T05:22:24.976 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:22:25.004 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T05:22:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T05:22:25.077 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T05:22:25.109 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T05:22:25.129 DEBUG:teuthology.parallel:result is None 2024-04-16T05:22:25.139 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T05:22:25.216 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T05:22:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T05:22:25.291 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T05:22:25.331 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T05:22:25.366 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T05:22:25.455 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T05:22:25.537 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T05:22:25.589 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T05:22:25.617 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T05:22:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T05:22:25.763 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T05:22:25.810 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T05:22:26.526 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T05:22:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T05:22:26.608 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T05:22:26.656 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T05:22:26.745 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T05:22:26.775 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T05:22:26.950 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T05:22:26.973 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T05:22:27.050 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T05:22:27.691 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T05:22:27.803 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T05:22:29.005 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T05:22:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T05:22:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T05:22:29.142 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T05:22:29.170 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T05:22:29.480 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T05:22:29.508 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:22:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:22:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 125/198 2024-04-16T05:22:29.666 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T05:22:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T05:22:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:22:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:22:30.271 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 129/198 2024-04-16T05:22:32.496 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:22:32.525 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:22:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:22:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:22:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:22:33.196 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T05:22:33.196 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T05:22:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:22:44.926 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:22:44.926 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-04-16T05:22:44.926 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-04-16T05:22:44.926 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-04-16T05:22:44.926 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-04-16T05:22:44.926 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-04-16T05:22:44.926 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-04-16T05:22:44.926 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-04-16T05:22:44.926 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:45.106 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:45.122 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:45.122 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:45.122 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T05:22:45.122 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:46.619 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:46.645 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:46.645 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T05:22:46.645 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:46.646 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:46.892 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:46.892 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:46.892 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T05:22:46.892 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:46.893 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:46.893 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:47.328 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:47.351 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:47.351 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T05:22:47.351 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:47.351 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T05:22:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:47.902 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 139/198 2024-04-16T05:22:47.963 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 140/198 2024-04-16T05:22:47.993 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:49.023 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:49.053 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:50.008 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:50.036 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:50.137 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:50.291 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 144/198 2024-04-16T05:22:50.313 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:50.337 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:50.337 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:50.337 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T05:22:50.337 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:50.337 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:50.337 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:50.381 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:50.392 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 147/198 2024-04-16T05:22:51.367 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T05:22:51.399 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:51.422 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:51.422 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:51.422 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T05:22:51.422 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:51.423 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:51.423 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:51.466 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:51.489 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:51.489 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:51.489 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T05:22:51.489 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:51.534 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:51.557 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:51.557 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:51.557 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T05:22:51.557 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:51.557 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:51.558 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:51.859 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:51.883 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:51.883 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:51.883 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T05:22:51.883 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:51.883 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:51.883 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:22:56.464 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:22:56.505 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 154/198 2024-04-16T05:22:56.535 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 155/198 2024-04-16T05:22:56.562 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T05:22:56.735 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 157/198 2024-04-16T05:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T05:22:56.885 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 159/198 2024-04-16T05:22:56.937 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T05:22:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T05:22:57.045 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T05:22:57.077 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 163/198 2024-04-16T05:22:57.118 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T05:22:57.147 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T05:22:57.215 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T05:22:57.282 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T05:22:57.312 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T05:22:57.867 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:57.910 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:58.750 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 170/198 2024-04-16T05:22:58.838 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T05:22:58.879 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T05:22:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T05:22:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T05:22:59.016 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T05:22:59.041 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T05:22:59.068 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T05:22:59.068 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:59.155 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:59.185 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T05:22:59.185 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:59.270 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:59.296 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T05:22:59.329 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T05:22:59.354 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T05:22:59.379 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T05:22:59.413 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T05:22:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T05:22:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T05:22:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T05:22:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T05:22:59.568 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:22:59.593 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T05:22:59.600 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:59.600 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:59.788 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:59.788 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:59.862 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:59.894 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:22:59.920 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T05:22:59.947 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T05:22:59.971 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T05:23:00.037 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:23:05.450 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:23:05.450 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 1/198 2024-04-16T05:23:05.450 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2/198 2024-04-16T05:23:05.450 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 3/198 2024-04-16T05:23:05.450 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 4/198 2024-04-16T05:23:05.450 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 5/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 6/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 7/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 8/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 9/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 13/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 14/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 15/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 17/198 2024-04-16T05:23:05.451 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 18/198 2024-04-16T05:23:05.452 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 19/198 2024-04-16T05:23:05.452 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 20/198 2024-04-16T05:23:05.452 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 21/198 2024-04-16T05:23:05.452 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 22/198 2024-04-16T05:23:05.452 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 23/198 2024-04-16T05:23:05.452 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 24/198 2024-04-16T05:23:05.452 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 25/198 2024-04-16T05:23:05.452 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 26/198 2024-04-16T05:23:05.468 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 27/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 28/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 29/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 30/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 31/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 32/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 33/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 34/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 35/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 36/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 37/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 38/198 2024-04-16T05:23:05.469 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 39/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T05:23:05.470 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T05:23:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T05:23:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T05:23:05.473 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T05:23:05.474 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T05:23:05.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T05:23:05.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T05:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T05:23:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T05:23:05.479 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T05:23:05.480 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T05:23:05.481 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:23:05.482 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T05:23:05.482 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T05:23:05.482 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T05:23:08.294 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T05:23:08.294 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:23:08.294 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-04-16T05:23:08.294 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T05:23:08.294 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T05:23:08.294 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.295 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T05:23:08.296 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.297 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T05:23:08.298 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: jq-1.6-16.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.299 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T05:23:08.300 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T05:23:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T05:23:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T05:23:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T05:23:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T05:23:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T05:23:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T05:23:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T05:23:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T05:23:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:23:08.308 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-16T05:23:08.756 DEBUG:teuthology.parallel:result is None 2024-04-16T05:23:08.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:23:08.982 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T05:23:09.005 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-3101.ga87fcff6.el9 2024-04-16T05:23:09.006 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.ga87fcff6.el9 2024-04-16T05:23:09.006 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.ga87fcff6 is installed. 2024-04-16T05:23:09.008 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:23:09.234 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T05:23:09.260 INFO:teuthology.orchestra.run.smithi194.stdout:19.0.0-3101.ga87fcff6.el9 2024-04-16T05:23:09.260 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.ga87fcff6.el9 2024-04-16T05:23:09.260 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.ga87fcff6 is installed. 2024-04-16T05:23:09.261 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:23:09.490 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T05:23:09.517 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-3101.ga87fcff6.el9 2024-04-16T05:23:09.517 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.ga87fcff6.el9 2024-04-16T05:23:09.517 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.ga87fcff6 is installed. 2024-04-16T05:23:09.519 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-16T05:23:09.519 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:23:09.519 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T05:23:09.551 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:23:09.551 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T05:23:09.586 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:23:09.586 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T05:23:09.620 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-16T05:23:09.620 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:23:09.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T05:23:09.649 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T05:23:09.717 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:23:09.717 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T05:23:09.749 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T05:23:09.819 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:23:09.819 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T05:23:09.851 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T05:23:09.921 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-16T05:23:09.921 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:23:09.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T05:23:09.950 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T05:23:10.018 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:23:10.018 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T05:23:10.049 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T05:23:10.119 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:23:10.119 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T05:23:10.149 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T05:23:10.218 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-16T05:23:10.218 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:23:10.219 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T05:23:10.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T05:23:10.314 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:23:10.314 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T05:23:10.345 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T05:23:10.415 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:23:10.415 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T05:23:10.445 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T05:23:10.514 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-16T05:23:10.608 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-16T05:23:10.608 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:23:10.609 INFO:tasks.cephadm:Cluster fsid is 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:23:10.609 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-16T05:23:10.609 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.86', 'mon.b': '172.21.15.194', 'mon.c': '172.21.15.196'} 2024-04-16T05:23:10.609 INFO:tasks.cephadm:First mon is mon.a on smithi086 2024-04-16T05:23:10.609 INFO:tasks.cephadm:First mgr is x 2024-04-16T05:23:10.609 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-16T05:23:10.609 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-04-16T05:23:10.638 DEBUG:teuthology.orchestra.run.smithi194:> sudo hostname $(hostname -s) 2024-04-16T05:23:10.668 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-04-16T05:23:10.698 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-16T05:23:10.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:23:10.842 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78606/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3101-ga87fcff6', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3101.ga87fcff6'}, 'url': 'https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 04:15:21.905790', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'qa-debug-resource-temporarily-unavailable-issue', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-16T05:23:11.067 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref qa-debug-resource-temporarily-unavailable-issue, sha1 a87fcff6340baeedd43af126e793f3241b6400ea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:23:11.069 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/x86_64/flavors/default/cephadm 2024-04-16T05:23:11.069 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T05:23:13.679 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r--. 1 ubuntu ubuntu 781558 Apr 16 05:23 /home/ubuntu/cephtest/cephadm 2024-04-16T05:23:13.680 DEBUG:teuthology.orchestra.run.smithi194:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T05:23:16.136 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r--. 1 ubuntu ubuntu 781558 Apr 16 05:23 /home/ubuntu/cephtest/cephadm 2024-04-16T05:23:16.137 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T05:23:18.366 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 781558 Apr 16 05:23 /home/ubuntu/cephtest/cephadm 2024-04-16T05:23:18.367 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T05:23:18.387 DEBUG:teuthology.orchestra.run.smithi194:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T05:23:18.406 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T05:23:18.440 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea on all hosts... 2024-04-16T05:23:18.440 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea pull 2024-04-16T05:23:18.443 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea pull 2024-04-16T05:23:18.448 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea pull 2024-04-16T05:23:19.202 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:23:19.230 INFO:teuthology.orchestra.run.smithi194.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:23:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:24:25.447 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-16T05:24:25.447 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev)", 2024-04-16T05:24:25.447 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317", 2024-04-16T05:24:25.447 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-04-16T05:24:25.447 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1" 2024-04-16T05:24:25.447 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-16T05:24:25.447 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-16T05:24:43.685 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:24:43.686 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev)", 2024-04-16T05:24:43.686 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317", 2024-04-16T05:24:43.686 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-04-16T05:24:43.686 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1" 2024-04-16T05:24:43.686 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:24:43.686 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:24:45.702 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-16T05:24:45.702 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_version": "ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev)", 2024-04-16T05:24:45.702 INFO:teuthology.orchestra.run.smithi194.stdout: "image_id": "3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317", 2024-04-16T05:24:45.702 INFO:teuthology.orchestra.run.smithi194.stdout: "repo_digests": [ 2024-04-16T05:24:45.702 INFO:teuthology.orchestra.run.smithi194.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1" 2024-04-16T05:24:45.702 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2024-04-16T05:24:45.702 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-16T05:24:45.727 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-04-16T05:24:45.764 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph 2024-04-16T05:24:45.793 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-04-16T05:24:45.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-04-16T05:24:45.859 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 777 /etc/ceph 2024-04-16T05:24:45.885 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-04-16T05:24:45.915 INFO:tasks.cephadm:Writing seed config... 2024-04-16T05:24:45.916 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-16T05:24:45.916 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-16T05:24:45.917 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-16T05:24:45.918 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-16T05:24:45.919 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-16T05:24:45.920 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-16T05:24:45.920 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-16T05:24:45.920 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-16T05:24:45.920 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-16T05:24:45.920 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-16T05:24:45.920 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-16T05:24:45.921 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:24:45.921 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-16T05:24:45.939 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-16T05:24:45.939 DEBUG:teuthology.orchestra.run.smithi086:mon.a> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.a.service 2024-04-16T05:24:45.982 DEBUG:teuthology.orchestra.run.smithi086:mgr.x> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service 2024-04-16T05:24:46.025 INFO:tasks.cephadm:Bootstrapping... 2024-04-16T05:24:46.025 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea -v bootstrap --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.86 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:24:46.223 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:24:46.223 INFO:teuthology.orchestra.run.smithi086.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea', '-v', 'bootstrap', '--fsid', '6a4a23f8-fbb1-11ee-bc8f-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.86', '--skip-admin-label'] 2024-04-16T05:24:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-16T05:24:46.224 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying podman|docker is present... 2024-04-16T05:24:46.250 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-16T05:24:46.250 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying lvm2 is present... 2024-04-16T05:24:46.250 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying time synchronization is in place... 2024-04-16T05:24:46.257 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-16T05:24:46.258 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-16T05:24:46.264 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-16T05:24:46.264 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-16T05:24:46.270 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-04-16T05:24:46.277 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-04-16T05:24:46.277 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-04-16T05:24:46.277 INFO:teuthology.orchestra.run.smithi086.stdout:Repeating the final host check... 2024-04-16T05:24:46.300 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-16T05:24:46.301 INFO:teuthology.orchestra.run.smithi086.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-16T05:24:46.301 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl is present 2024-04-16T05:24:46.301 INFO:teuthology.orchestra.run.smithi086.stdout:lvcreate is present 2024-04-16T05:24:46.307 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-16T05:24:46.307 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-16T05:24:46.313 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-16T05:24:46.313 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-16T05:24:46.319 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-04-16T05:24:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-04-16T05:24:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-04-16T05:24:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:Host looks OK 2024-04-16T05:24:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:Cluster fsid: 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:Acquiring lock 140668763632544 on /run/cephadm/6a4a23f8-fbb1-11ee-bc8f-c7b262605968.lock 2024-04-16T05:24:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:Lock 140668763632544 acquired on /run/cephadm/6a4a23f8-fbb1-11ee-bc8f-c7b262605968.lock 2024-04-16T05:24:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 3300 ... 2024-04-16T05:24:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 6789 ... 2024-04-16T05:24:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:Base mon IP(s) is [172.21.15.86:3300, 172.21.15.86:6789], mon addrv is [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-04-16T05:24:46.330 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.86 metric 100 2024-04-16T05:24:46.330 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.86 metric 100 2024-04-16T05:24:46.333 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-16T05:24:46.333 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-16T05:24:46.333 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1668sec hoplimit 64 pref medium 2024-04-16T05:24:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-16T05:24:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-16T05:24:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-16T05:24:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-16T05:24:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-16T05:24:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-16T05:24:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-04-16T05:24:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-04-16T05:24:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-16T05:24:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-16T05:24:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:24:46.778 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 2024-04-16T05:24:46.778 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:24:46.778 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Getting image source signatures 2024-04-16T05:24:46.778 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:831530ac7ca651ac084d96199c7d2480907501d88600b7dc48bade85dc1cca4f 2024-04-16T05:24:46.778 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:9793b5b5582798bde394f4383d142e9f248a25e4ebfd72ee05ea9ec6251a877d 2024-04-16T05:24:46.779 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying config sha256:3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 2024-04-16T05:24:46.779 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-16T05:24:47.733 INFO:teuthology.orchestra.run.smithi086.stdout:ceph: stdout ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev) 2024-04-16T05:24:47.733 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph version: ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev) 2024-04-16T05:24:47.733 INFO:teuthology.orchestra.run.smithi086.stdout:Extracting ceph user uid/gid from container image... 2024-04-16T05:24:48.713 INFO:teuthology.orchestra.run.smithi086.stdout:stat: stdout 167 167 2024-04-16T05:24:48.714 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial keys... 2024-04-16T05:24:49.610 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQChCx5mpN7qARAA24Kfez0WiDIJ52P2ddv8wg== 2024-04-16T05:24:50.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQChCx5mpytkNhAAcWllg5XLgDR2KLYz/i/QDw== 2024-04-16T05:24:51.688 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQCiCx5mYptfMxAArTmQl6kARMuuTyn5HFL6lg== 2024-04-16T05:24:51.689 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial monmap... 2024-04-16T05:24:52.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-16T05:24:52.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-16T05:24:52.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:52.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-16T05:24:52.527 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool for a [v2:172.21.15.86:3300,v1:172.21.15.86:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-16T05:24:52.528 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = quincy 2024-04-16T05:24:52.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: set fsid to 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:52.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-16T05:24:52.528 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:24:52.528 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mon... 2024-04-16T05:24:53.466 INFO:teuthology.orchestra.run.smithi086.stdout:create mon.a on 2024-04-16T05:24:53.675 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-16T05:24:53.823 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-16T05:24:53.990 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968.target → /etc/systemd/system/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968.target. 2024-04-16T05:24:53.990 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968.target → /etc/systemd/system/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968.target. 2024-04-16T05:24:54.242 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.a 2024-04-16T05:24:54.242 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.a.service: Unit ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.a.service not loaded. 2024-04-16T05:24:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968.target.wants/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.a.service → /etc/systemd/system/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@.service. 2024-04-16T05:24:54.482 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:54 smithi086 systemd[1]: Starting Ceph mon.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:24:54.791 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:54 smithi086 podman[31622]: 2024-04-16 05:24:54.659326996 +0000 UTC m=+0.100699959 container create 12ab1f1b2a90f3e057cc85c1389a804edae65ca36504a15c031b2a4e00c4deb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:24:54.791 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:54 smithi086 podman[31622]: 2024-04-16 05:24:54.576498904 +0000 UTC m=+0.017871880 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:24:54.974 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T05:24:54.974 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-16T05:24:54.981 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T05:24:54.981 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-16T05:24:54.982 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-16T05:24:54.982 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-04-16T05:24:54.982 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon to start... 2024-04-16T05:24:54.982 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon... 2024-04-16T05:24:55.102 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:54 smithi086 podman[31622]: 2024-04-16 05:24:54.903316412 +0000 UTC m=+0.344689378 container init 12ab1f1b2a90f3e057cc85c1389a804edae65ca36504a15c031b2a4e00c4deb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-16T05:24:55.102 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:54 smithi086 podman[31622]: 2024-04-16 05:24:54.906606628 +0000 UTC m=+0.347979592 container start 12ab1f1b2a90f3e057cc85c1389a804edae65ca36504a15c031b2a4e00c4deb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:24:55.102 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:54 smithi086 ceph-mon[31651]: mkfs 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:55.102 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:54 smithi086 bash[31622]: 12ab1f1b2a90f3e057cc85c1389a804edae65ca36504a15c031b2a4e00c4deb0 2024-04-16T05:24:55.102 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:54 smithi086 ceph-mon[31651]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T05:24:55.102 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:54 smithi086 systemd[1]: Started Ceph mon.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:24:56.041 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T05:24:56.041 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: monmap epoch 1 2024-04-16T05:24:56.041 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:56.041 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: last_changed 2024-04-16T05:24:52.023563+0000 2024-04-16T05:24:56.041 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: created 2024-04-16T05:24:52.023563+0000 2024-04-16T05:24:56.041 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: min_mon_release 19 (squid) 2024-04-16T05:24:56.041 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: election_strategy: 1 2024-04-16T05:24:56.041 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-16T05:24:56.042 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: fsmap 2024-04-16T05:24:56.042 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: osdmap e1: 0 total, 0 up, 0 in 2024-04-16T05:24:56.042 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: mgrmap e1: no daemons active 2024-04-16T05:24:56.042 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:55 smithi086 ceph-mon[31651]: from='client.? 172.21.15.86:0/542233289' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-16T05:24:56.043 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout cluster: 2024-04-16T05:24:56.049 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout id: 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout services: 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.514054s) 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout data: 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-16T05:24:56.050 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pgs: 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.388+0000 7f2052636700 1 Processor -- start 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.388+0000 7f2052636700 1 -- start start 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.388+0000 7f2052636700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c107120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.388+0000 7f2052636700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1076f0 con 0x7f204c106d00 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.388+0000 7f204bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.388+0000 7f204bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47482/0 (socket says 172.21.15.86:47482) 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.388+0000 7f204bfff700 1 -- 172.21.15.86:0/3735529741 learned_addr learned my addr 172.21.15.86:0/3735529741 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.389+0000 7f204bfff700 1 -- 172.21.15.86:0/3735529741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c107f00 con 0x7f204c106d00 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.389+0000 7f204bfff700 1 --2- 172.21.15.86:0/3735529741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c107120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f203c0098a0 tx=0x7f203c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e509f7b0502c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.389+0000 7f204affd700 1 -- 172.21.15.86:0/3735529741 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f203c004160 con 0x7f204c106d00 2024-04-16T05:24:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.389+0000 7f204affd700 1 -- 172.21.15.86:0/3735529741 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203c004300 con 0x7f204c106d00 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.389+0000 7f204affd700 1 -- 172.21.15.86:0/3735529741 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f203c0036a0 con 0x7f204c106d00 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.389+0000 7f2052636700 1 -- 172.21.15.86:0/3735529741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 msgr2=0x7f204c107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.389+0000 7f2052636700 1 --2- 172.21.15.86:0/3735529741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c107120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f203c0098a0 tx=0x7f203c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.390+0000 7f2052636700 1 -- 172.21.15.86:0/3735529741 shutdown_connections 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.390+0000 7f2052636700 1 --2- 172.21.15.86:0/3735529741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c107120 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.390+0000 7f2052636700 1 -- 172.21.15.86:0/3735529741 >> 172.21.15.86:0/3735529741 conn(0x7f204c102360 msgr2=0x7f204c104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.390+0000 7f2052636700 1 -- 172.21.15.86:0/3735529741 shutdown_connections 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.390+0000 7f2052636700 1 -- 172.21.15.86:0/3735529741 wait complete. 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.390+0000 7f2052636700 1 Processor -- start 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.390+0000 7f2052636700 1 -- start start 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f2052636700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:56.052 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f2052636700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f204c068680 con 0x7f204c106d00 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f204bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f204bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47490/0 (socket says 172.21.15.86:47490) 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f204bfff700 1 -- 172.21.15.86:0/542233289 learned_addr learned my addr 172.21.15.86:0/542233289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f204bfff700 1 -- 172.21.15.86:0/542233289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c19c3d0 con 0x7f204c106d00 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f204bfff700 1 --2- 172.21.15.86:0/542233289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c19be90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f203c000c00 tx=0x7f203c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f20497fa700 1 -- 172.21.15.86:0/542233289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f203c003db0 con 0x7f204c106d00 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f20497fa700 1 -- 172.21.15.86:0/542233289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203c011630 con 0x7f204c106d00 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f2052636700 1 -- 172.21.15.86:0/542233289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c19c660 con 0x7f204c106d00 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f20497fa700 1 -- 172.21.15.86:0/542233289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f203c011950 con 0x7f204c106d00 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.391+0000 7f2052636700 1 -- 172.21.15.86:0/542233289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c19f350 con 0x7f204c106d00 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.392+0000 7f20497fa700 1 -- 172.21.15.86:0/542233289 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f203c01a460 con 0x7f204c106d00 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.392+0000 7f20497fa700 1 -- 172.21.15.86:0/542233289 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f203c01b780 con 0x7f204c106d00 2024-04-16T05:24:56.053 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.392+0000 7f2052636700 1 -- 172.21.15.86:0/542233289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f204c064150 con 0x7f204c106d00 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.395+0000 7f20497fa700 1 -- 172.21.15.86:0/542233289 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f203c01b980 con 0x7f204c106d00 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.468+0000 7f2052636700 1 -- 172.21.15.86:0/542233289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f204c02e170 con 0x7f204c106d00 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.468+0000 7f20497fa700 1 -- 172.21.15.86:0/542233289 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f203c01bb60 con 0x7f204c106d00 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.469+0000 7f2052636700 1 -- 172.21.15.86:0/542233289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 msgr2=0x7f204c19be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.469+0000 7f2052636700 1 --2- 172.21.15.86:0/542233289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c19be90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f203c000c00 tx=0x7f203c004750 comp rx=0 tx=0).stop 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.469+0000 7f2052636700 1 -- 172.21.15.86:0/542233289 shutdown_connections 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.469+0000 7f2052636700 1 --2- 172.21.15.86:0/542233289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f204c106d00 0x7f204c19be90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.469+0000 7f2052636700 1 -- 172.21.15.86:0/542233289 >> 172.21.15.86:0/542233289 conn(0x7f204c102360 msgr2=0x7f204c10b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.469+0000 7f2052636700 1 -- 172.21.15.86:0/542233289 shutdown_connections 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.469+0000 7f2052636700 1 -- 172.21.15.86:0/542233289 wait complete. 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:mon is available 2024-04-16T05:24:56.054 INFO:teuthology.orchestra.run.smithi086.stdout:Assimilating anything we can from ceph.conf... 2024-04-16T05:24:57.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-16T05:24:57.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.491+0000 7f7cce0f6700 1 Processor -- start 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.491+0000 7f7cce0f6700 1 -- start start 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.491+0000 7f7cce0f6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc8107390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.491+0000 7f7cce0f6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc807be30 con 0x7f7cc8106f70 2024-04-16T05:24:57.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.492+0000 7f7cc77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc8107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.492+0000 7f7cc77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc8107390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47504/0 (socket says 172.21.15.86:47504) 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.492+0000 7f7cc77fe700 1 -- 172.21.15.86:0/3797071923 learned_addr learned my addr 172.21.15.86:0/3797071923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.492+0000 7f7cc77fe700 1 -- 172.21.15.86:0/3797071923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc81078d0 con 0x7f7cc8106f70 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.492+0000 7f7cc77fe700 1 --2- 172.21.15.86:0/3797071923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc8107390 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009e80 tx=0x7f7cc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6178a014e75ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.492+0000 7f7cc67fc700 1 -- 172.21.15.86:0/3797071923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0017070 con 0x7f7cc8106f70 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.492+0000 7f7cc67fc700 1 -- 172.21.15.86:0/3797071923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc0003a20 con 0x7f7cc8106f70 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.492+0000 7f7cc67fc700 1 -- 172.21.15.86:0/3797071923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0003d20 con 0x7f7cc8106f70 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.493+0000 7f7cce0f6700 1 -- 172.21.15.86:0/3797071923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 msgr2=0x7f7cc8107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.493+0000 7f7cce0f6700 1 --2- 172.21.15.86:0/3797071923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc8107390 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009e80 tx=0x7f7cc000b330 comp rx=0 tx=0).stop 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.493+0000 7f7cce0f6700 1 -- 172.21.15.86:0/3797071923 shutdown_connections 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.493+0000 7f7cce0f6700 1 --2- 172.21.15.86:0/3797071923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc8107390 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.493+0000 7f7cce0f6700 1 -- 172.21.15.86:0/3797071923 >> 172.21.15.86:0/3797071923 conn(0x7f7cc8102bb0 msgr2=0x7f7cc8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:57.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.493+0000 7f7cce0f6700 1 -- 172.21.15.86:0/3797071923 shutdown_connections 2024-04-16T05:24:57.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.493+0000 7f7cce0f6700 1 -- 172.21.15.86:0/3797071923 wait complete. 2024-04-16T05:24:57.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.494+0000 7f7cce0f6700 1 Processor -- start 2024-04-16T05:24:57.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.494+0000 7f7cce0f6700 1 -- start start 2024-04-16T05:24:57.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.494+0000 7f7cce0f6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:57.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.494+0000 7f7cce0f6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc8107e00 con 0x7f7cc8106f70 2024-04-16T05:24:57.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.494+0000 7f7cc77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:57.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.494+0000 7f7cc77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47520/0 (socket says 172.21.15.86:47520) 2024-04-16T05:24:57.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.494+0000 7f7cc77fe700 1 -- 172.21.15.86:0/2620093636 learned_addr learned my addr 172.21.15.86:0/2620093636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:24:57.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.494+0000 7f7cc77fe700 1 -- 172.21.15.86:0/2620093636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc819e4d0 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.495+0000 7f7cc77fe700 1 --2- 172.21.15.86:0/2620093636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc819df90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7cc807b270 tx=0x7f7cc0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.495+0000 7f7cc4ff9700 1 -- 172.21.15.86:0/2620093636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0022020 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.495+0000 7f7cc4ff9700 1 -- 172.21.15.86:0/2620093636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc0018040 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.495+0000 7f7cce0f6700 1 -- 172.21.15.86:0/2620093636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc819e760 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.495+0000 7f7cc4ff9700 1 -- 172.21.15.86:0/2620093636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0017070 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.495+0000 7f7cce0f6700 1 -- 172.21.15.86:0/2620093636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc81a1450 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.495+0000 7f7cc4ff9700 1 -- 172.21.15.86:0/2620093636 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7cc0025020 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.495+0000 7f7cc4ff9700 1 -- 172.21.15.86:0/2620093636 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7cc000f050 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.496+0000 7f7cce0f6700 1 -- 172.21.15.86:0/2620093636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc8064150 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.498+0000 7f7cc4ff9700 1 -- 172.21.15.86:0/2620093636 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7cc002c030 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.564+0000 7f7cce0f6700 1 -- 172.21.15.86:0/2620093636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7cc81a2a30 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.567+0000 7f7cc4ff9700 1 -- 172.21.15.86:0/2620093636 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f7cc0012660 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.568+0000 7f7cc4ff9700 1 -- 172.21.15.86:0/2620093636 <== mon.0 v2:172.21.15.86:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cc0012d00 con 0x7f7cc8106f70 2024-04-16T05:24:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.569+0000 7f7cce0f6700 1 -- 172.21.15.86:0/2620093636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 msgr2=0x7f7cc819df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:57.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.569+0000 7f7cce0f6700 1 --2- 172.21.15.86:0/2620093636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc819df90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7cc807b270 tx=0x7f7cc0004ad0 comp rx=0 tx=0).stop 2024-04-16T05:24:57.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.569+0000 7f7cce0f6700 1 -- 172.21.15.86:0/2620093636 shutdown_connections 2024-04-16T05:24:57.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.569+0000 7f7cce0f6700 1 --2- 172.21.15.86:0/2620093636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8106f70 0x7f7cc819df90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:57.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.569+0000 7f7cce0f6700 1 -- 172.21.15.86:0/2620093636 >> 172.21.15.86:0/2620093636 conn(0x7f7cc8102bb0 msgr2=0x7f7cc81945b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:57.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.569+0000 7f7cce0f6700 1 -- 172.21.15.86:0/2620093636 shutdown_connections 2024-04-16T05:24:57.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.569+0000 7f7cce0f6700 1 -- 172.21.15.86:0/2620093636 wait complete. 2024-04-16T05:24:57.136 INFO:teuthology.orchestra.run.smithi086.stdout:Generating new minimal ceph.conf... 2024-04-16T05:24:57.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:56 smithi086 ceph-mon[31651]: from='client.? 172.21.15.86:0/2620093636' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-16T05:24:57.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:56 smithi086 ceph-mon[31651]: from='client.? 172.21.15.86:0/2620093636' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-16T05:24:58.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:57 smithi086 ceph-mon[31651]: from='client.? 172.21.15.86:0/4293767209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:24:58.169 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.522+0000 7f397c931700 1 Processor -- start 2024-04-16T05:24:58.169 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.522+0000 7f397c931700 1 -- start start 2024-04-16T05:24:58.169 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.522+0000 7f397c931700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f3978108b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:58.170 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.522+0000 7f397c931700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3978109120 con 0x7f3978108730 2024-04-16T05:24:58.170 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.522+0000 7f397659c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f3978108b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:58.170 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.523+0000 7f397659c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f3978108b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47522/0 (socket says 172.21.15.86:47522) 2024-04-16T05:24:58.170 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.523+0000 7f397659c700 1 -- 172.21.15.86:0/4025519495 learned_addr learned my addr 172.21.15.86:0/4025519495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:24:58.170 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.523+0000 7f397659c700 1 -- 172.21.15.86:0/4025519495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781098e0 con 0x7f3978108730 2024-04-16T05:24:58.170 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.523+0000 7f397659c700 1 --2- 172.21.15.86:0/4025519495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f3978108b50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f39680098a0 tx=0x7f3968009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c8a263451a9dca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:58.170 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.523+0000 7f397559a700 1 -- 172.21.15.86:0/4025519495 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3968004160 con 0x7f3978108730 2024-04-16T05:24:58.170 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.523+0000 7f397559a700 1 -- 172.21.15.86:0/4025519495 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3968004300 con 0x7f3978108730 2024-04-16T05:24:58.170 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.523+0000 7f397559a700 1 -- 172.21.15.86:0/4025519495 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3968011b50 con 0x7f3978108730 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.524+0000 7f397c931700 1 -- 172.21.15.86:0/4025519495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 msgr2=0x7f3978108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.524+0000 7f397c931700 1 --2- 172.21.15.86:0/4025519495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f3978108b50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f39680098a0 tx=0x7f3968009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.524+0000 7f397c931700 1 -- 172.21.15.86:0/4025519495 shutdown_connections 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.524+0000 7f397c931700 1 --2- 172.21.15.86:0/4025519495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f3978108b50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.524+0000 7f397c931700 1 -- 172.21.15.86:0/4025519495 >> 172.21.15.86:0/4025519495 conn(0x7f3978103dd0 msgr2=0x7f39781061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.524+0000 7f397c931700 1 -- 172.21.15.86:0/4025519495 shutdown_connections 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.524+0000 7f397c931700 1 -- 172.21.15.86:0/4025519495 wait complete. 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f397c931700 1 Processor -- start 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f397c931700 1 -- start start 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f397c931700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f397807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f397c931700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f397810cc90 con 0x7f3978108730 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f397659c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f397807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:58.171 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f397659c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f397807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47538/0 (socket says 172.21.15.86:47538) 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f397659c700 1 -- 172.21.15.86:0/4293767209 learned_addr learned my addr 172.21.15.86:0/4293767209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f397659c700 1 -- 172.21.15.86:0/4293767209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397807e0d0 con 0x7f3978108730 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f397659c700 1 --2- 172.21.15.86:0/4293767209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f397807db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3968011670 tx=0x7f3968011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.525+0000 7f39677fe700 1 -- 172.21.15.86:0/4293767209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39680039c0 con 0x7f3978108730 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.526+0000 7f39677fe700 1 -- 172.21.15.86:0/4293767209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3968011e50 con 0x7f3978108730 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.526+0000 7f397c931700 1 -- 172.21.15.86:0/4293767209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397807c2c0 con 0x7f3978108730 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.526+0000 7f39677fe700 1 -- 172.21.15.86:0/4293767209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f396801a5c0 con 0x7f3978108730 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.526+0000 7f397c931700 1 -- 172.21.15.86:0/4293767209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397807c800 con 0x7f3978108730 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.526+0000 7f39677fe700 1 -- 172.21.15.86:0/4293767209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f396801ac00 con 0x7f3978108730 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.526+0000 7f39677fe700 1 -- 172.21.15.86:0/4293767209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3968019700 con 0x7f3978108730 2024-04-16T05:24:58.172 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.527+0000 7f397c931700 1 -- 172.21.15.86:0/4293767209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3978064150 con 0x7f3978108730 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.529+0000 7f39677fe700 1 -- 172.21.15.86:0/4293767209 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3968003b60 con 0x7f3978108730 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.597+0000 7f397c931700 1 -- 172.21.15.86:0/4293767209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f397807cd30 con 0x7f3978108730 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.598+0000 7f39677fe700 1 -- 172.21.15.86:0/4293767209 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f39680199a0 con 0x7f3978108730 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.598+0000 7f397c931700 1 -- 172.21.15.86:0/4293767209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 msgr2=0x7f397807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.598+0000 7f397c931700 1 --2- 172.21.15.86:0/4293767209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f397807db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3968011670 tx=0x7f3968011b60 comp rx=0 tx=0).stop 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.598+0000 7f397c931700 1 -- 172.21.15.86:0/4293767209 shutdown_connections 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.598+0000 7f397c931700 1 --2- 172.21.15.86:0/4293767209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3978108730 0x7f397807db90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.598+0000 7f397c931700 1 -- 172.21.15.86:0/4293767209 >> 172.21.15.86:0/4293767209 conn(0x7f3978103dd0 msgr2=0x7f3978104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.599+0000 7f397c931700 1 -- 172.21.15.86:0/4293767209 shutdown_connections 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.599+0000 7f397c931700 1 -- 172.21.15.86:0/4293767209 wait complete. 2024-04-16T05:24:58.173 INFO:teuthology.orchestra.run.smithi086.stdout:Restarting the monitor... 2024-04-16T05:24:58.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:58 smithi086 systemd[1]: Stopping Ceph mon.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:24:58.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:58 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a[31645]: 2024-04-16T05:24:58.423+0000 7fbe45509700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:24:58.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:58 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a[31645]: 2024-04-16T05:24:58.423+0000 7fbe45509700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-16T05:24:58.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:58 smithi086 podman[31899]: 2024-04-16 05:24:58.568463675 +0000 UTC m=+0.236268289 container died 12ab1f1b2a90f3e057cc85c1389a804edae65ca36504a15c031b2a4e00c4deb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-16T05:24:59.183 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:58 smithi086 podman[31899]: 2024-04-16 05:24:58.943427293 +0000 UTC m=+0.611231945 container cleanup 12ab1f1b2a90f3e057cc85c1389a804edae65ca36504a15c031b2a4e00c4deb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=) 2024-04-16T05:24:59.183 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:58 smithi086 bash[31899]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a 2024-04-16T05:24:59.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 podman[31912]: 2024-04-16 05:24:59.185843465 +0000 UTC m=+0.617526633 container remove 12ab1f1b2a90f3e057cc85c1389a804edae65ca36504a15c031b2a4e00c4deb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-16T05:24:59.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 systemd[1]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-16T05:24:59.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 systemd[1]: Stopped Ceph mon.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:24:59.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 systemd[1]: Starting Ceph mon.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:24:59.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 podman[32009]: 2024-04-16 05:24:59.427373962 +0000 UTC m=+0.017985852 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 podman[32009]: 2024-04-16 05:24:59.528457539 +0000 UTC m=+0.119069423 container create 29c249e4574d08a74b86554571b36f204251270b0b54e05dc5ac05a644e48123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 podman[32009]: 2024-04-16 05:24:59.713975355 +0000 UTC m=+0.304587241 container init 29c249e4574d08a74b86554571b36f204251270b0b54e05dc5ac05a644e48123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 podman[32009]: 2024-04-16 05:24:59.71697881 +0000 UTC m=+0.307590694 container start 29c249e4574d08a74b86554571b36f204251270b0b54e05dc5ac05a644e48123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: set uid:gid to 167:167 (ceph:ceph) 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev), process ceph-mon, pid 2 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: pidfile_write: ignore empty --pid-file 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: load: jerasure load: lrc 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: RocksDB version: 7.9.2 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Git sha 0 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Compile date 2024-04-16 03:54:27 2024-04-16T05:24:59.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: DB SUMMARY 2024-04-16T05:24:59.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: DB Session ID: 0OXH6F8FOG9JZRWSMQ4O 2024-04-16T05:24:59.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: CURRENT file: CURRENT 2024-04-16T05:24:59.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: IDENTITY file: IDENTITY 2024-04-16T05:24:59.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-16T05:24:59.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-16T05:24:59.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105672 ; 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.error_if_exists: 0 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.create_if_missing: 0 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.paranoid_checks: 1 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.env: 0x55b2d9894cc0 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.fs: PosixFileSystem 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.info_log: 0x55b2dbc8bc20 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_file_opening_threads: 16 2024-04-16T05:24:59.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.statistics: (nil) 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.use_fsync: 0 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_log_file_size: 0 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.keep_log_file_num: 1000 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.recycle_log_file_num: 0 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.allow_fallocate: 1 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.allow_mmap_reads: 0 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.allow_mmap_writes: 0 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.use_direct_reads: 0 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.create_missing_column_families: 0 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.db_log_dir: 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.wal_dir: 2024-04-16T05:24:59.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.advise_random_on_open: 1 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.db_write_buffer_size: 0 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.write_buffer_manager: 0x55b2dbc5a1e0 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.rate_limiter: (nil) 2024-04-16T05:24:59.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.wal_recovery_mode: 2 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.enable_thread_tracking: 0 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.enable_pipelined_write: 0 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.unordered_write: 0 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.row_cache: None 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.wal_filter: None 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.allow_ingest_behind: 0 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.two_write_queues: 0 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.manual_wal_flush: 0 2024-04-16T05:24:59.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.wal_compression: 0 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.atomic_flush: 0 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.log_readahead_size: 0 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.best_efforts_recovery: 0 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-16T05:24:59.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.allow_data_in_errors: 0 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.db_host_id: __hostname__ 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_background_jobs: 2 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_background_compactions: -1 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_subcompactions: 1 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_total_wal_size: 0 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-16T05:24:59.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_open_files: -1 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bytes_per_sync: 0 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_readahead_size: 0 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_background_flushes: -1 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Compression algorithms supported: 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: kZSTD supported: 0 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: kXpressCompression supported: 0 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: kLZ4HCCompression supported: 1 2024-04-16T05:24:59.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: kZlibCompression supported: 1 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: kSnappyCompression supported: 1 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: kLZ4Compression supported: 1 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: kBZip2Compression supported: 0 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.merge_operator: 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_filter: None 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_filter_factory: None 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.sst_partitioner_factory: None 2024-04-16T05:24:59.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b2dbc8bde0) 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_type: 0 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_index_type: 0 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_shortening: 1 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: checksum: 4 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: no_block_cache: 0 2024-04-16T05:24:59.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache: 0x55b2dbc8d090 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_options: 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: capacity : 536870912 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_shard_bits : 4 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: strict_capacity_limit : 0 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_compressed: (nil) 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: persistent_cache: (nil) 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size: 4096 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size_deviation: 10 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_restart_interval: 16 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_block_restart_interval: 1 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: metadata_block_size: 4096 2024-04-16T05:24:59.791 INFO:journalctl@ceph.mon.a.smithi086.stdout: partition_filters: 0 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: use_delta_encoding: 1 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: filter_policy: bloomfilter 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: whole_key_filtering: 1 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: verify_compression: 0 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: format_version: 5 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_index_compression: 1 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_align: 0 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: max_auto_readahead_size: 262144 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: prepopulate_block_cache: 0 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: initial_auto_readahead_size: 8192 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.write_buffer_size: 33554432 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_write_buffer_number: 2 2024-04-16T05:24:59.792 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression: NoCompression 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression: Disabled 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.prefix_extractor: nullptr 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.num_levels: 7 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-16T05:24:59.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression_opts.level: 32767 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression_opts.strategy: 0 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression_opts.enabled: false 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-16T05:24:59.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.target_file_size_base: 67108864 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-16T05:24:59.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.arena_block_size: 1048576 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.disable_auto_compactions: 0 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-16T05:24:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.inplace_update_support: 0 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.bloom_locality: 0 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.max_successive_merges: 0 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-16T05:24:59.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.paranoid_file_checks: 0 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.force_consistency_checks: 1 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.report_bg_io_stats: 0 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.ttl: 2592000 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.enable_blob_files: false 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.min_blob_size: 0 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.blob_file_size: 268435456 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-16T05:24:59.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.blob_file_starting_level: 0 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 925ad3fa-e772-4526-836b-b15e8d916bff 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713245099746446, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713245099747524, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100741, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98635, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14306, "raw_average_key_size": 49, "raw_value_size": 90881, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713245099, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "925ad3fa-e772-4526-836b-b15e8d916bff", "db_session_id": "0OXH6F8FOG9JZRWSMQ4O", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713245099747623, "job": 1, "event": "recovery_finished"} 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b2dbd7a000 2024-04-16T05:24:59.799 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: DB pointer 0x55b2dbd64000 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** DB Stats ** 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-04-16T05:24:59.800 INFO:journalctl@ceph.mon.a.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: L0 2/0 100.24 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: Sum 2/0 100.24 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 132.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-16T05:24:59.801 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative compaction: 0.00 GB write, 30.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval compaction: 0.00 GB write, 30.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache BinnedLRUCache@0x55b2dbc8d090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: starting mon.a rank 0 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:59.802 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mon.a@-1(???) e1 preinit fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mon.a@-1(???).mds e1 new map 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mon.a@-1(???).mds e1 print_map 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout: e1 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout: legacy client fscid: -1 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout: No filesystems configured 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-16T05:24:59.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 bash[32009]: 29c249e4574d08a74b86554571b36f204251270b0b54e05dc5ac05a644e48123 2024-04-16T05:24:59.804 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 systemd[1]: Started Ceph mon.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:24:59.804 INFO:teuthology.orchestra.run.smithi086.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-16T05:25:00.033 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: monmap epoch 1 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: last_changed 2024-04-16T05:24:52.023563+0000 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: created 2024-04-16T05:24:52.023563+0000 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: min_mon_release 19 (squid) 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: election_strategy: 1 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: fsmap 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: osdmap e1: 0 total, 0 up, 0 in 2024-04-16T05:25:00.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:24:59 smithi086 ceph-mon[32028]: mgrmap e1: no daemons active 2024-04-16T05:25:00.737 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.226+0000 7f4158cc9700 1 Processor -- start 2024-04-16T05:25:00.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.226+0000 7f4158cc9700 1 -- start start 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.226+0000 7f4158cc9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f4154106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.226+0000 7f4158cc9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4154107390 con 0x7f41541069a0 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.227+0000 7f415259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f4154106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.227+0000 7f415259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f4154106dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47554/0 (socket says 172.21.15.86:47554) 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.227+0000 7f415259c700 1 -- 172.21.15.86:0/45838522 learned_addr learned my addr 172.21.15.86:0/45838522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.227+0000 7f415259c700 1 -- 172.21.15.86:0/45838522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4154107510 con 0x7f41541069a0 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.227+0000 7f415259c700 1 --2- 172.21.15.86:0/45838522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f4154106dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f41440098a0 tx=0x7f4144009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6c0895d3a000b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.227+0000 7f4151d9b700 1 -- 172.21.15.86:0/45838522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4144004160 con 0x7f41541069a0 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.227+0000 7f4151d9b700 1 -- 172.21.15.86:0/45838522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4144004300 con 0x7f41541069a0 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.227+0000 7f4151d9b700 1 -- 172.21.15.86:0/45838522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4144011b50 con 0x7f41541069a0 2024-04-16T05:25:00.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.228+0000 7f4158cc9700 1 -- 172.21.15.86:0/45838522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 msgr2=0x7f4154106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.228+0000 7f4158cc9700 1 --2- 172.21.15.86:0/45838522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f4154106dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f41440098a0 tx=0x7f4144009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.228+0000 7f4158cc9700 1 -- 172.21.15.86:0/45838522 shutdown_connections 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.228+0000 7f4158cc9700 1 --2- 172.21.15.86:0/45838522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f4154106dc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.228+0000 7f4158cc9700 1 -- 172.21.15.86:0/45838522 >> 172.21.15.86:0/45838522 conn(0x7f41541023e0 msgr2=0x7f4154104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.228+0000 7f4158cc9700 1 -- 172.21.15.86:0/45838522 shutdown_connections 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.228+0000 7f4158cc9700 1 -- 172.21.15.86:0/45838522 wait complete. 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.229+0000 7f4158cc9700 1 Processor -- start 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.229+0000 7f4158cc9700 1 -- start start 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.229+0000 7f4158cc9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f415419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.229+0000 7f4158cc9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f415410a860 con 0x7f41541069a0 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.229+0000 7f415259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f415419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.229+0000 7f415259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f415419c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47560/0 (socket says 172.21.15.86:47560) 2024-04-16T05:25:00.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.229+0000 7f415259c700 1 -- 172.21.15.86:0/777141971 learned_addr learned my addr 172.21.15.86:0/777141971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.230+0000 7f415259c700 1 -- 172.21.15.86:0/777141971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415419cbc0 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.230+0000 7f415259c700 1 --2- 172.21.15.86:0/777141971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f415419c680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4144009fb0 tx=0x7f4144011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.230+0000 7f414b7fe700 1 -- 172.21.15.86:0/777141971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41440038b0 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.230+0000 7f4158cc9700 1 -- 172.21.15.86:0/777141971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415419ceb0 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.230+0000 7f414b7fe700 1 -- 172.21.15.86:0/777141971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4144011e50 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.230+0000 7f414b7fe700 1 -- 172.21.15.86:0/777141971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4144019550 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.230+0000 7f4158cc9700 1 -- 172.21.15.86:0/777141971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415419fb40 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.230+0000 7f414b7fe700 1 -- 172.21.15.86:0/777141971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4144019be0 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.230+0000 7f414b7fe700 1 -- 172.21.15.86:0/777141971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4144024730 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.231+0000 7f4158cc9700 1 -- 172.21.15.86:0/777141971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4154064150 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.233+0000 7f414b7fe700 1 -- 172.21.15.86:0/777141971 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4144043e50 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.299+0000 7f4158cc9700 1 -- 172.21.15.86:0/777141971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f415419fd10 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.301+0000 7f414b7fe700 1 -- 172.21.15.86:0/777141971 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f4144032020 con 0x7f41541069a0 2024-04-16T05:25:00.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.302+0000 7f4158cc9700 1 -- 172.21.15.86:0/777141971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 msgr2=0x7f415419c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:00.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.302+0000 7f4158cc9700 1 --2- 172.21.15.86:0/777141971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f415419c680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4144009fb0 tx=0x7f4144011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:00.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.302+0000 7f4158cc9700 1 -- 172.21.15.86:0/777141971 shutdown_connections 2024-04-16T05:25:00.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.302+0000 7f4158cc9700 1 --2- 172.21.15.86:0/777141971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41541069a0 0x7f415419c680 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:00.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.302+0000 7f4158cc9700 1 -- 172.21.15.86:0/777141971 >> 172.21.15.86:0/777141971 conn(0x7f41541023e0 msgr2=0x7f4154102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:00.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.302+0000 7f4158cc9700 1 -- 172.21.15.86:0/777141971 shutdown_connections 2024-04-16T05:25:00.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.302+0000 7f4158cc9700 1 -- 172.21.15.86:0/777141971 wait complete. 2024-04-16T05:25:00.745 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-16T05:25:00.759 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:25:00.759 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mgr... 2024-04-16T05:25:00.760 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-16T05:25:00.760 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-16T05:25:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.x 2024-04-16T05:25:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service: Unit ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service not loaded. 2024-04-16T05:25:01.140 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968.target.wants/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@.service. 2024-04-16T05:25:01.300 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:01 smithi086 systemd[1]: Starting Ceph mgr.x for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:25:01.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:01 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/777141971' entity='client.admin' 2024-04-16T05:25:01.718 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:01 smithi086 podman[32243]: 2024-04-16 05:25:01.279702982 +0000 UTC m=+0.018140602 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:25:01.718 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:01 smithi086 podman[32243]: 2024-04-16 05:25:01.48080448 +0000 UTC m=+0.219242082 container create 3de8902fe01512238291b91705b3da5a0ca03707565220b688c14cc7ce76d42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:25:03.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:03 smithi086 podman[32243]: 2024-04-16 05:25:03.601940775 +0000 UTC m=+2.340378377 container init 3de8902fe01512238291b91705b3da5a0ca03707565220b688c14cc7ce76d42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-16T05:25:03.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:03 smithi086 podman[32243]: 2024-04-16 05:25:03.605453161 +0000 UTC m=+2.343890768 container start 3de8902fe01512238291b91705b3da5a0ca03707565220b688c14cc7ce76d42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-16T05:25:04.450 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:04 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:04.156+0000 7fa618fb9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:25:04.450 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:04 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:04.250+0000 7fa618fb9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:25:04.464 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T05:25:04.465 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-16T05:25:04.472 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T05:25:04.472 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-16T05:25:04.472 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-16T05:25:04.473 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-04-16T05:25:04.480 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T05:25:04.481 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-16T05:25:04.488 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T05:25:04.488 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-16T05:25:04.488 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-16T05:25:04.488 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-16T05:25:04.488 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr to start... 2024-04-16T05:25:04.489 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr... 2024-04-16T05:25:04.718 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:04 smithi086 bash[32243]: 3de8902fe01512238291b91705b3da5a0ca03707565220b688c14cc7ce76d42a 2024-04-16T05:25:04.718 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:04 smithi086 systemd[1]: Started Ceph mgr.x for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:25:05.217 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:04 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:04.918+0000 7fa618fb9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:25:05.217 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:05 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:05.019+0000 7fa618fb9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:25:05.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:05 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:05.160+0000 7fa618fb9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:25:05.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:05 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:05.257+0000 7fa618fb9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:25:05.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:05 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:05.339+0000 7fa618fb9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:25:05.899 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:05 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:05.609+0000 7fa618fb9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:25:05.899 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:05 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:05.718+0000 7fa618fb9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:25:05.899 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:05 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:05.805+0000 7fa618fb9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:25:05.899 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:05 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:05.900+0000 7fa618fb9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:25:06.217 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:06 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:06.078+0000 7fa618fb9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:25:07.343 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:07 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:07.059+0000 7fa618fb9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:25:07.343 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:07 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:07.191+0000 7fa618fb9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:25:07.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:07 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2321811001' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T05:25:07.626 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:07 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:07.532+0000 7fa618fb9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:25:07.627 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:07 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:07.627+0000 7fa618fb9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:25:07.967 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:07 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:07.826+0000 7fa618fb9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:08.301 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:08 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:08.013+0000 7fa618fb9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:25:08.561 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:08 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:08.302+0000 7fa618fb9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:08.561 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:08 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:08.562+0000 7fa618fb9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:25:08.835 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:08 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:08.803+0000 7fa618fb9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:25:09.967 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:09 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:09.615+0000 7fa618fb9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:25:10.467 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:10.188+0000 7fa618fb9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:25:10.467 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:10.286+0000 7fa618fb9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:25:10.823 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:10.504+0000 7fa618fb9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: Activating manager daemon x 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: mgrmap e2: x(active, starting, since 0.00290503s) 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: Manager daemon x is now available 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:25:10.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:25:10.825 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' 2024-04-16T05:25:10.825 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' 2024-04-16T05:25:10.825 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:10 smithi086 ceph-mon[32028]: from='mgr.14100 172.21.15.86:0/1253983970' entity='mgr.x' 2024-04-16T05:25:12.635 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:25:12.635 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T05:25:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T05:25:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T05:25:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T05:25:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T05:24:54.956652+0000", 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.450+0000 7fb9165af700 1 Processor -- start 2024-04-16T05:25:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.450+0000 7fb9165af700 1 -- start start 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.450+0000 7fb9165af700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91010b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.450+0000 7fb9165af700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb91007be50 con 0x7fb910109190 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.450+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91010b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.450+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91010b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39978/0 (socket says 172.21.15.86:39978) 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.450+0000 7fb90ffff700 1 -- 172.21.15.86:0/357765262 learned_addr learned my addr 172.21.15.86:0/357765262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.451+0000 7fb90ffff700 1 -- 172.21.15.86:0/357765262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91010bac0 con 0x7fb910109190 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.451+0000 7fb90ffff700 1 --2- 172.21.15.86:0/357765262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91010b580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb9000098a0 tx=0x7fb900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9257bc24b2a571b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.451+0000 7fb90effd700 1 -- 172.21.15.86:0/357765262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb900004160 con 0x7fb910109190 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.451+0000 7fb90effd700 1 -- 172.21.15.86:0/357765262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb900004300 con 0x7fb910109190 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.451+0000 7fb90effd700 1 -- 172.21.15.86:0/357765262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb900011b50 con 0x7fb910109190 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.452+0000 7fb9165af700 1 -- 172.21.15.86:0/357765262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 msgr2=0x7fb91010b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:12.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.452+0000 7fb9165af700 1 --2- 172.21.15.86:0/357765262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91010b580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb9000098a0 tx=0x7fb900009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.452+0000 7fb9165af700 1 -- 172.21.15.86:0/357765262 shutdown_connections 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.452+0000 7fb9165af700 1 --2- 172.21.15.86:0/357765262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91010b580 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.452+0000 7fb9165af700 1 -- 172.21.15.86:0/357765262 >> 172.21.15.86:0/357765262 conn(0x7fb910102c40 msgr2=0x7fb910105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.452+0000 7fb9165af700 1 -- 172.21.15.86:0/357765262 shutdown_connections 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.452+0000 7fb9165af700 1 -- 172.21.15.86:0/357765262 wait complete. 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.452+0000 7fb9165af700 1 Processor -- start 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.452+0000 7fb9165af700 1 -- start start 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb9165af700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb9165af700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb91010ca80 con 0x7fb910109190 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91019c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39980/0 (socket says 172.21.15.86:39980) 2024-04-16T05:25:12.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb90ffff700 1 -- 172.21.15.86:0/2321811001 learned_addr learned my addr 172.21.15.86:0/2321811001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb90ffff700 1 -- 172.21.15.86:0/2321811001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91019cbc0 con 0x7fb910109190 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb90ffff700 1 --2- 172.21.15.86:0/2321811001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91019c680 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb900009fb0 tx=0x7fb900011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb90d7fa700 1 -- 172.21.15.86:0/2321811001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9000038b0 con 0x7fb910109190 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb90d7fa700 1 -- 172.21.15.86:0/2321811001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb900011e50 con 0x7fb910109190 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb9165af700 1 -- 172.21.15.86:0/2321811001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91019ce50 con 0x7fb910109190 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb90d7fa700 1 -- 172.21.15.86:0/2321811001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb900019420 con 0x7fb910109190 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.453+0000 7fb9165af700 1 -- 172.21.15.86:0/2321811001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91019fb40 con 0x7fb910109190 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.454+0000 7fb90d7fa700 1 -- 172.21.15.86:0/2321811001 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb900019ab0 con 0x7fb910109190 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.454+0000 7fb90d7fa700 1 -- 172.21.15.86:0/2321811001 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb900024730 con 0x7fb910109190 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.454+0000 7fb9165af700 1 -- 172.21.15.86:0/2321811001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb910064150 con 0x7fb910109190 2024-04-16T05:25:12.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.457+0000 7fb90d7fa700 1 -- 172.21.15.86:0/2321811001 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb900043e50 con 0x7fb910109190 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.530+0000 7fb9165af700 1 -- 172.21.15.86:0/2321811001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb91019ffe0 con 0x7fb910109190 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.530+0000 7fb90d7fa700 1 -- 172.21.15.86:0/2321811001 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb900032030 con 0x7fb910109190 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.531+0000 7fb9165af700 1 -- 172.21.15.86:0/2321811001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 msgr2=0x7fb91019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.531+0000 7fb9165af700 1 --2- 172.21.15.86:0/2321811001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91019c680 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb900009fb0 tx=0x7fb900011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.531+0000 7fb9165af700 1 -- 172.21.15.86:0/2321811001 shutdown_connections 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.531+0000 7fb9165af700 1 --2- 172.21.15.86:0/2321811001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910109190 0x7fb91019c680 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.531+0000 7fb9165af700 1 -- 172.21.15.86:0/2321811001 >> 172.21.15.86:0/2321811001 conn(0x7fb910102c40 msgr2=0x7fb910103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.531+0000 7fb9165af700 1 -- 172.21.15.86:0/2321811001 shutdown_connections 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.531+0000 7fb9165af700 1 -- 172.21.15.86:0/2321811001 wait complete. 2024-04-16T05:25:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (1/15)... 2024-04-16T05:25:12.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:12 smithi086 ceph-mon[32028]: mgrmap e3: x(active, since 1.00733s) 2024-04-16T05:25:13.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:13 smithi086 ceph-mon[32028]: mgrmap e4: x(active, since 2s) 2024-04-16T05:25:15.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:15 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1239510517' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T05:25:15.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:25:15.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:25:15.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:25:15.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T05:25:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T05:25:15.951 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T05:25:15.951 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:15.952 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:25:15.953 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T05:24:54.956652+0000", 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.041+0000 7ff63381b700 1 Processor -- start 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.041+0000 7ff63381b700 1 -- start start 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.041+0000 7ff63381b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c107340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.041+0000 7ff63381b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c07be90 con 0x7ff62c106f20 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.042+0000 7ff6315b7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.042+0000 7ff6315b7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c107340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60320/0 (socket says 172.21.15.86:60320) 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.042+0000 7ff6315b7700 1 -- 172.21.15.86:0/3405662 learned_addr learned my addr 172.21.15.86:0/3405662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.042+0000 7ff6315b7700 1 -- 172.21.15.86:0/3405662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c107880 con 0x7ff62c106f20 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.042+0000 7ff6315b7700 1 --2- 172.21.15.86:0/3405662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c107340 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6280098a0 tx=0x7ff628009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a3fdcc4e475763 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.042+0000 7ff623fff700 1 -- 172.21.15.86:0/3405662 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff628004160 con 0x7ff62c106f20 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.042+0000 7ff623fff700 1 -- 172.21.15.86:0/3405662 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff628004300 con 0x7ff62c106f20 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.042+0000 7ff623fff700 1 -- 172.21.15.86:0/3405662 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff628011b50 con 0x7ff62c106f20 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.043+0000 7ff63381b700 1 -- 172.21.15.86:0/3405662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 msgr2=0x7ff62c107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.043+0000 7ff63381b700 1 --2- 172.21.15.86:0/3405662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c107340 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6280098a0 tx=0x7ff628009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.043+0000 7ff63381b700 1 -- 172.21.15.86:0/3405662 shutdown_connections 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.043+0000 7ff63381b700 1 --2- 172.21.15.86:0/3405662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c107340 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.043+0000 7ff63381b700 1 -- 172.21.15.86:0/3405662 >> 172.21.15.86:0/3405662 conn(0x7ff62c102b80 msgr2=0x7ff62c104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.043+0000 7ff63381b700 1 -- 172.21.15.86:0/3405662 shutdown_connections 2024-04-16T05:25:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.043+0000 7ff63381b700 1 -- 172.21.15.86:0/3405662 wait complete. 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.044+0000 7ff63381b700 1 Processor -- start 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.044+0000 7ff63381b700 1 -- start start 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.044+0000 7ff63381b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.044+0000 7ff63381b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c108910 con 0x7ff62c106f20 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.044+0000 7ff6315b7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.044+0000 7ff6315b7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c19e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60328/0 (socket says 172.21.15.86:60328) 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.044+0000 7ff6315b7700 1 -- 172.21.15.86:0/1239510517 learned_addr learned my addr 172.21.15.86:0/1239510517 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.044+0000 7ff6315b7700 1 -- 172.21.15.86:0/1239510517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c19ede0 con 0x7ff62c106f20 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.045+0000 7ff6315b7700 1 --2- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c19e8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff628011b60 tx=0x7ff628011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.045+0000 7ff6227fc700 1 -- 172.21.15.86:0/1239510517 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff628003970 con 0x7ff62c106f20 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.045+0000 7ff6227fc700 1 -- 172.21.15.86:0/1239510517 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff628003b10 con 0x7ff62c106f20 2024-04-16T05:25:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.045+0000 7ff6227fc700 1 -- 172.21.15.86:0/1239510517 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff628024610 con 0x7ff62c106f20 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.045+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c19f070 con 0x7ff62c106f20 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.045+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c1a1d60 con 0x7ff62c106f20 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.046+0000 7ff6227fc700 1 -- 172.21.15.86:0/1239510517 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff62801a430 con 0x7ff62c106f20 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.046+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62c064150 con 0x7ff62c106f20 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.046+0000 7ff6227fc700 1 --2- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7ff6180395b0 0x7ff61803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.046+0000 7ff6227fc700 1 -- 172.21.15.86:0/1239510517 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff628011e00 con 0x7ff62c106f20 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.046+0000 7ff630db6700 1 --2- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7ff6180395b0 0x7ff61803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.046+0000 7ff630db6700 1 --2- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7ff6180395b0 0x7ff61803ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff61c006fd0 tx=0x7ff61c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.051+0000 7ff6227fc700 1 -- 172.21.15.86:0/1239510517 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff628016260 con 0x7ff62c106f20 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.333+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff62c02e250 con 0x7ff62c106f20 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.334+0000 7ff6227fc700 1 -- 172.21.15.86:0/1239510517 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff62804d240 con 0x7ff62c106f20 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7ff6180395b0 msgr2=0x7ff61803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 --2- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7ff6180395b0 0x7ff61803ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff61c006fd0 tx=0x7ff61c006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:15.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 msgr2=0x7ff62c19e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:15.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 --2- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c19e8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff628011b60 tx=0x7ff628011c40 comp rx=0 tx=0).stop 2024-04-16T05:25:15.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 shutdown_connections 2024-04-16T05:25:15.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 --2- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7ff6180395b0 0x7ff61803ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:15.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 --2- 172.21.15.86:0/1239510517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c106f20 0x7ff62c19e8a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:15.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 >> 172.21.15.86:0/1239510517 conn(0x7ff62c102b80 msgr2=0x7ff62c1954c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:15.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 shutdown_connections 2024-04-16T05:25:15.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:15.336+0000 7ff63381b700 1 -- 172.21.15.86:0/1239510517 wait complete. 2024-04-16T05:25:15.958 INFO:teuthology.orchestra.run.smithi086.stdout:mgr is available 2024-04-16T05:25:16.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:16 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3484664908' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-16T05:25:17.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:25:17.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-04-16T05:25:17.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-16T05:25:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.307+0000 7f1885080700 1 Processor -- start 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.307+0000 7f1885080700 1 -- start start 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.307+0000 7f1885080700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f1880108b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.307+0000 7f1885080700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1880109120 con 0x7f1880108730 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.307+0000 7f187ed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f1880108b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.307+0000 7f187ed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f1880108b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60332/0 (socket says 172.21.15.86:60332) 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.307+0000 7f187ed9d700 1 -- 172.21.15.86:0/90881965 learned_addr learned my addr 172.21.15.86:0/90881965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f187ed9d700 1 -- 172.21.15.86:0/90881965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18801098e0 con 0x7f1880108730 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f187ed9d700 1 --2- 172.21.15.86:0/90881965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f1880108b50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f18700098a0 tx=0x7f1870009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab7cdd2e7762db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f187dd9b700 1 -- 172.21.15.86:0/90881965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1870004160 con 0x7f1880108730 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f187dd9b700 1 -- 172.21.15.86:0/90881965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1870004300 con 0x7f1880108730 2024-04-16T05:25:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f187dd9b700 1 -- 172.21.15.86:0/90881965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1870011b50 con 0x7f1880108730 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f1885080700 1 -- 172.21.15.86:0/90881965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 msgr2=0x7f1880108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f1885080700 1 --2- 172.21.15.86:0/90881965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f1880108b50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f18700098a0 tx=0x7f1870009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f1885080700 1 -- 172.21.15.86:0/90881965 shutdown_connections 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f1885080700 1 --2- 172.21.15.86:0/90881965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f1880108b50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.308+0000 7f1885080700 1 -- 172.21.15.86:0/90881965 >> 172.21.15.86:0/90881965 conn(0x7f1880103dd0 msgr2=0x7f18801061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.309+0000 7f1885080700 1 -- 172.21.15.86:0/90881965 shutdown_connections 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.309+0000 7f1885080700 1 -- 172.21.15.86:0/90881965 wait complete. 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.309+0000 7f1885080700 1 Processor -- start 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.309+0000 7f1885080700 1 -- start start 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f1885080700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f188007db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f1885080700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f188010cc90 con 0x7f1880108730 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f187ed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f188007db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f187ed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f188007db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60340/0 (socket says 172.21.15.86:60340) 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f187ed9d700 1 -- 172.21.15.86:0/3484664908 learned_addr learned my addr 172.21.15.86:0/3484664908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f187ed9d700 1 -- 172.21.15.86:0/3484664908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188007e0d0 con 0x7f1880108730 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f187ed9d700 1 --2- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f188007db90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1870011670 tx=0x7f1870011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f1867fff700 1 -- 172.21.15.86:0/3484664908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18700039c0 con 0x7f1880108730 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f1867fff700 1 -- 172.21.15.86:0/3484664908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1870011e50 con 0x7f1880108730 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188007c2c0 con 0x7f1880108730 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f1867fff700 1 -- 172.21.15.86:0/3484664908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f187001a5c0 con 0x7f1880108730 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.310+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188007c800 con 0x7f1880108730 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.311+0000 7f1867fff700 1 -- 172.21.15.86:0/3484664908 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f187001a8c0 con 0x7f1880108730 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.311+0000 7f1867fff700 1 --2- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1868039510 0x7f186803b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.311+0000 7f1867fff700 1 -- 172.21.15.86:0/3484664908 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f187004daa0 con 0x7f1880108730 2024-04-16T05:25:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.311+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1880064150 con 0x7f1880108730 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.311+0000 7f187e59c700 1 --2- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1868039510 0x7f186803b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.312+0000 7f187e59c700 1 --2- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1868039510 0x7f186803b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1874006fd0 tx=0x7f1874006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.316+0000 7f1867fff700 1 -- 172.21.15.86:0/3484664908 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1870019c70 con 0x7f1880108730 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.514+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f188007ca80 con 0x7f1880108730 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.515+0000 7f1867fff700 1 -- 172.21.15.86:0/3484664908 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f187002cc00 con 0x7f1880108730 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1868039510 msgr2=0x7f186803b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 --2- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1868039510 0x7f186803b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1874006fd0 tx=0x7f1874006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 msgr2=0x7f188007db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 --2- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f188007db90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1870011670 tx=0x7f1870011b60 comp rx=0 tx=0).stop 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 shutdown_connections 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 --2- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1868039510 0x7f186803b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 --2- 172.21.15.86:0/3484664908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1880108730 0x7f188007db90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:17.091 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 >> 172.21.15.86:0/3484664908 conn(0x7f1880103dd0 msgr2=0x7f1880104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:17.092 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 shutdown_connections 2024-04-16T05:25:17.092 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.517+0000 7f1885080700 1 -- 172.21.15.86:0/3484664908 wait complete. 2024-04-16T05:25:17.092 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling cephadm module... 2024-04-16T05:25:17.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:17 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2740594688' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-16T05:25:17.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: ignoring --setuser ceph since I am not root 2024-04-16T05:25:17.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: ignoring --setgroup ceph since I am not root 2024-04-16T05:25:18.717 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:18 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:18.329+0000 7f4c151eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:25:18.717 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:18 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:18.424+0000 7f4c151eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:25:18.978 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:18 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2740594688' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-16T05:25:18.978 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:18 smithi086 ceph-mon[32028]: mgrmap e5: x(active, since 7s) 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.455+0000 7f7d6746c700 1 Processor -- start 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.455+0000 7f7d6746c700 1 -- start start 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.455+0000 7f7d6746c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d60107360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.455+0000 7f7d6746c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6007be90 con 0x7f7d60106f40 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.455+0000 7f7d65208700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d60107360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.455+0000 7f7d65208700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d60107360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60350/0 (socket says 172.21.15.86:60350) 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.455+0000 7f7d65208700 1 -- 172.21.15.86:0/2264346875 learned_addr learned my addr 172.21.15.86:0/2264346875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.456+0000 7f7d65208700 1 -- 172.21.15.86:0/2264346875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d601078a0 con 0x7f7d60106f40 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.456+0000 7f7d65208700 1 --2- 172.21.15.86:0/2264346875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d60107360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009e80 tx=0x7f7d5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe44ee8c46472a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.456+0000 7f7d5ffff700 1 -- 172.21.15.86:0/2264346875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d58017070 con 0x7f7d60106f40 2024-04-16T05:25:18.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.456+0000 7f7d5ffff700 1 -- 172.21.15.86:0/2264346875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7d5800b8c0 con 0x7f7d60106f40 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.456+0000 7f7d5ffff700 1 -- 172.21.15.86:0/2264346875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d580045c0 con 0x7f7d60106f40 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.457+0000 7f7d6746c700 1 -- 172.21.15.86:0/2264346875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 msgr2=0x7f7d60107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.457+0000 7f7d6746c700 1 --2- 172.21.15.86:0/2264346875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d60107360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009e80 tx=0x7f7d5800b330 comp rx=0 tx=0).stop 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.457+0000 7f7d6746c700 1 -- 172.21.15.86:0/2264346875 shutdown_connections 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.457+0000 7f7d6746c700 1 --2- 172.21.15.86:0/2264346875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d60107360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.457+0000 7f7d6746c700 1 -- 172.21.15.86:0/2264346875 >> 172.21.15.86:0/2264346875 conn(0x7f7d60102c40 msgr2=0x7f7d60105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.457+0000 7f7d6746c700 1 -- 172.21.15.86:0/2264346875 shutdown_connections 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.457+0000 7f7d6746c700 1 -- 172.21.15.86:0/2264346875 wait complete. 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.457+0000 7f7d6746c700 1 Processor -- start 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.457+0000 7f7d6746c700 1 -- start start 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d6746c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d6019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d6746c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60108930 con 0x7f7d60106f40 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d65208700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d6019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:18.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d65208700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d6019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60366/0 (socket says 172.21.15.86:60366) 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d65208700 1 -- 172.21.15.86:0/2740594688 learned_addr learned my addr 172.21.15.86:0/2740594688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d65208700 1 -- 172.21.15.86:0/2740594688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6019ed40 con 0x7f7d60106f40 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d65208700 1 --2- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d6019e800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009530 tx=0x7f7d5800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d5e7fc700 1 -- 172.21.15.86:0/2740594688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d58020020 con 0x7f7d60106f40 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6019efd0 con 0x7f7d60106f40 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d601a1cc0 con 0x7f7d60106f40 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d5e7fc700 1 -- 172.21.15.86:0/2740594688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7d58003b50 con 0x7f7d60106f40 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.458+0000 7f7d5e7fc700 1 -- 172.21.15.86:0/2740594688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d58017040 con 0x7f7d60106f40 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.459+0000 7f7d5e7fc700 1 -- 172.21.15.86:0/2740594688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f7d58023020 con 0x7f7d60106f40 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.459+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d4c005360 con 0x7f7d60106f40 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.459+0000 7f7d5e7fc700 1 --2- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f7d48039560 0x7f7d4803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.460+0000 7f7d5e7fc700 1 -- 172.21.15.86:0/2740594688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d5800f050 con 0x7f7d60106f40 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.460+0000 7f7d64a07700 1 --2- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f7d48039560 0x7f7d4803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:18.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.460+0000 7f7d64a07700 1 --2- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f7d48039560 0x7f7d4803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006fd0 tx=0x7f7d54006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.464+0000 7f7d5e7fc700 1 -- 172.21.15.86:0/2740594688 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d5802a020 con 0x7f7d60106f40 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.718+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f7d4c005650 con 0x7f7d60106f40 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.769+0000 7f7d5e7fc700 1 -- 172.21.15.86:0/2740594688 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f7d5804cfa0 con 0x7f7d60106f40 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.771+0000 7f7d5e7fc700 1 -- 172.21.15.86:0/2740594688 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f7d58025400 con 0x7f7d60106f40 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f7d48039560 msgr2=0x7f7d4803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 --2- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f7d48039560 0x7f7d4803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006fd0 tx=0x7f7d54006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 msgr2=0x7f7d6019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 --2- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d6019e800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009530 tx=0x7f7d5800bd50 comp rx=0 tx=0).stop 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 shutdown_connections 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 --2- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f7d48039560 0x7f7d4803ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 --2- 172.21.15.86:0/2740594688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d60106f40 0x7f7d6019e800 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 >> 172.21.15.86:0/2740594688 conn(0x7f7d60102c40 msgr2=0x7f7d601954a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:18.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 shutdown_connections 2024-04-16T05:25:18.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.772+0000 7f7d6746c700 1 -- 172.21.15.86:0/2740594688 wait complete. 2024-04-16T05:25:19.289 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:19.078+0000 7f4c151eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:25:19.289 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:19.173+0000 7f4c151eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:25:19.595 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:19.321+0000 7f4c151eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:25:19.596 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:19.424+0000 7f4c151eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:25:19.596 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:19.510+0000 7f4c151eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:25:19.891 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:19 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3548033153' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-16T05:25:19.892 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:19.782+0000 7f4c151eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:25:19.892 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:19.893+0000 7f4c151eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:25:20.217 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:19.981+0000 7f4c151eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:25:20.217 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:20 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:20.077+0000 7f4c151eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:25:20.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:25:20.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-16T05:25:20.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T05:25:20.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-16T05:25:20.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-16T05:25:20.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:25:20.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.391+0000 7f1861ca1700 1 Processor -- start 2024-04-16T05:25:20.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.391+0000 7f1861ca1700 1 -- start start 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f1861ca1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c10b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f1861ca1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f185c07be90 con 0x7f185c109150 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f185b7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c10b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f185b7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c10b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60384/0 (socket says 172.21.15.86:60384) 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f185b7fe700 1 -- 172.21.15.86:0/407555645 learned_addr learned my addr 172.21.15.86:0/407555645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f185b7fe700 1 -- 172.21.15.86:0/407555645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c10ba80 con 0x7f185c109150 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f185b7fe700 1 --2- 172.21.15.86:0/407555645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c10b540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f184c0098a0 tx=0x7f184c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862061f1577cf7c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f185a7fc700 1 -- 172.21.15.86:0/407555645 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f184c004160 con 0x7f185c109150 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f185a7fc700 1 -- 172.21.15.86:0/407555645 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f184c004300 con 0x7f185c109150 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.392+0000 7f185a7fc700 1 -- 172.21.15.86:0/407555645 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f184c011b50 con 0x7f185c109150 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.393+0000 7f1861ca1700 1 -- 172.21.15.86:0/407555645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 msgr2=0x7f185c10b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:20.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.393+0000 7f1861ca1700 1 --2- 172.21.15.86:0/407555645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c10b540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f184c0098a0 tx=0x7f184c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.393+0000 7f1861ca1700 1 -- 172.21.15.86:0/407555645 shutdown_connections 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.393+0000 7f1861ca1700 1 --2- 172.21.15.86:0/407555645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c10b540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.393+0000 7f1861ca1700 1 -- 172.21.15.86:0/407555645 >> 172.21.15.86:0/407555645 conn(0x7f185c102b40 msgr2=0x7f185c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.393+0000 7f1861ca1700 1 -- 172.21.15.86:0/407555645 shutdown_connections 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.393+0000 7f1861ca1700 1 -- 172.21.15.86:0/407555645 wait complete. 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.394+0000 7f1861ca1700 1 Processor -- start 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.394+0000 7f1861ca1700 1 -- start start 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.394+0000 7f1861ca1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.394+0000 7f1861ca1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f185c10caa0 con 0x7f185c109150 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.394+0000 7f185b7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.394+0000 7f185b7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60394/0 (socket says 172.21.15.86:60394) 2024-04-16T05:25:20.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.394+0000 7f185b7fe700 1 -- 172.21.15.86:0/3548033153 learned_addr learned my addr 172.21.15.86:0/3548033153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.394+0000 7f185b7fe700 1 -- 172.21.15.86:0/3548033153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c19cc10 con 0x7f185c109150 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.395+0000 7f185b7fe700 1 --2- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c19c6d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f184c009fb0 tx=0x7f184c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.395+0000 7f1858ff9700 1 -- 172.21.15.86:0/3548033153 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f184c0038b0 con 0x7f185c109150 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.395+0000 7f1858ff9700 1 -- 172.21.15.86:0/3548033153 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f184c011e50 con 0x7f185c109150 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.395+0000 7f1858ff9700 1 -- 172.21.15.86:0/3548033153 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f184c019420 con 0x7f185c109150 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.395+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c19cea0 con 0x7f185c109150 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.395+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c19fb90 con 0x7f185c109150 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.396+0000 7f1858ff9700 1 -- 172.21.15.86:0/3548033153 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f184c019720 con 0x7f185c109150 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.396+0000 7f1858ff9700 1 --2- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1844039560 0x7f184403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.396+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c064150 con 0x7f185c109150 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.396+0000 7f185affd700 1 -- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1844039560 msgr2=0x7f184403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1904754808 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.396+0000 7f1858ff9700 1 -- 172.21.15.86:0/3548033153 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f184c04e9a0 con 0x7f185c109150 2024-04-16T05:25:20.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.396+0000 7f185affd700 1 --2- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1844039560 0x7f184403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.405+0000 7f1858ff9700 1 -- 172.21.15.86:0/3548033153 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f184c020240 con 0x7f185c109150 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.597+0000 7f185affd700 1 -- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1844039560 msgr2=0x7f184403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1904754808 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.597+0000 7f185affd700 1 --2- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1844039560 0x7f184403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.671+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f185c19fd60 con 0x7f185c109150 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.672+0000 7f1858ff9700 1 -- 172.21.15.86:0/3548033153 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f184c0204a0 con 0x7f185c109150 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1844039560 msgr2=0x7f184403ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 --2- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1844039560 0x7f184403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 msgr2=0x7f185c19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 --2- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c19c6d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f184c009fb0 tx=0x7f184c011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 shutdown_connections 2024-04-16T05:25:20.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 --2- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7f1844039560 0x7f184403ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:20.323 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 --2- 172.21.15.86:0/3548033153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c109150 0x7f185c19c6d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:20.323 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 >> 172.21.15.86:0/3548033153 conn(0x7f185c102b40 msgr2=0x7f185c103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:20.323 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 shutdown_connections 2024-04-16T05:25:20.323 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:19.674+0000 7f1861ca1700 1 -- 172.21.15.86:0/3548033153 wait complete. 2024-04-16T05:25:20.323 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-04-16T05:25:20.323 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 5... 2024-04-16T05:25:20.535 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:20 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:20.255+0000 7f4c151eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:25:21.717 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:21 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:21.245+0000 7f4c151eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:25:21.718 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:21 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:21.376+0000 7f4c151eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:25:22.005 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:21 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:21.722+0000 7f4c151eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:25:22.006 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:21 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:21.811+0000 7f4c151eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:25:22.006 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:22 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:22.007+0000 7f4c151eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:22.467 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:22 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:22.192+0000 7f4c151eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:25:22.734 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:22 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:22.479+0000 7f4c151eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:22.734 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:22 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:22.735+0000 7f4c151eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:25:23.217 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:22 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:22.973+0000 7f4c151eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:25:24.217 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:23 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:23.768+0000 7f4c151eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:25:24.650 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:24.334+0000 7f4c151eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:25:24.650 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:24.432+0000 7f4c151eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:25:24.651 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:24.651+0000 7f4c151eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:25:24.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: Active manager daemon x restarted 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: Activating manager daemon x 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: osdmap e2: 0 total, 0 up, 0 in 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: mgrmap e6: x(active, starting, since 0.00361949s) 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: Manager daemon x is now available 2024-04-16T05:25:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:24.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:24.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:24.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:24.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:24 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:25:25.966 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:25 smithi086 ceph-mon[32028]: Found migration_current of "None". Setting to last migration. 2024-04-16T05:25:25.966 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:25 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:25:25.966 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:25 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:25.966 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:25 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:25.966 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:25 smithi086 ceph-mon[32028]: mgrmap e7: x(active, since 1.00804s) 2024-04-16T05:25:26.276 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:25:26.276 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-16T05:25:26.276 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-16T05:25:26.276 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:25:26.276 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.738+0000 7fcc9e120700 1 Processor -- start 2024-04-16T05:25:26.276 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.738+0000 7fcc9e120700 1 -- start start 2024-04-16T05:25:26.276 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.738+0000 7fcc9e120700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc9810b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.738+0000 7fcc9e120700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9807bdf0 con 0x7fcc981091d0 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.738+0000 7fcc977fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc9810b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.738+0000 7fcc977fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc9810b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48708/0 (socket says 172.21.15.86:48708) 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.738+0000 7fcc977fe700 1 -- 172.21.15.86:0/1368197830 learned_addr learned my addr 172.21.15.86:0/1368197830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.739+0000 7fcc977fe700 1 -- 172.21.15.86:0/1368197830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9810bb00 con 0x7fcc981091d0 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.739+0000 7fcc977fe700 1 --2- 172.21.15.86:0/1368197830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc9810b5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0098a0 tx=0x7fcc8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3cd0fe08882b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.739+0000 7fcc967fc700 1 -- 172.21.15.86:0/1368197830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc8c004160 con 0x7fcc981091d0 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.739+0000 7fcc967fc700 1 -- 172.21.15.86:0/1368197830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc8c004300 con 0x7fcc981091d0 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.739+0000 7fcc967fc700 1 -- 172.21.15.86:0/1368197830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc8c011b50 con 0x7fcc981091d0 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.740+0000 7fcc9e120700 1 -- 172.21.15.86:0/1368197830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 msgr2=0x7fcc9810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.740+0000 7fcc9e120700 1 --2- 172.21.15.86:0/1368197830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc9810b5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0098a0 tx=0x7fcc8c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.740+0000 7fcc9e120700 1 -- 172.21.15.86:0/1368197830 shutdown_connections 2024-04-16T05:25:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.740+0000 7fcc9e120700 1 --2- 172.21.15.86:0/1368197830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc9810b5c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.740+0000 7fcc9e120700 1 -- 172.21.15.86:0/1368197830 >> 172.21.15.86:0/1368197830 conn(0x7fcc98102b80 msgr2=0x7fcc98104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.740+0000 7fcc9e120700 1 -- 172.21.15.86:0/1368197830 shutdown_connections 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.740+0000 7fcc9e120700 1 -- 172.21.15.86:0/1368197830 wait complete. 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.741+0000 7fcc9e120700 1 Processor -- start 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.741+0000 7fcc9e120700 1 -- start start 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.741+0000 7fcc9e120700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc98108a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.741+0000 7fcc9e120700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9810ca60 con 0x7fcc981091d0 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.741+0000 7fcc977fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc98108a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.741+0000 7fcc977fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc98108a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48712/0 (socket says 172.21.15.86:48712) 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.741+0000 7fcc977fe700 1 -- 172.21.15.86:0/1304901429 learned_addr learned my addr 172.21.15.86:0/1304901429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.741+0000 7fcc977fe700 1 -- 172.21.15.86:0/1304901429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc98108f90 con 0x7fcc981091d0 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.741+0000 7fcc977fe700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc98108a50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009fb0 tx=0x7fcc8c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:26.278 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.742+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc8c0038b0 con 0x7fcc981091d0 2024-04-16T05:25:26.279 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.742+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc98107180 con 0x7fcc981091d0 2024-04-16T05:25:26.279 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.742+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc8c011e50 con 0x7fcc981091d0 2024-04-16T05:25:26.279 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.742+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc8c019420 con 0x7fcc981091d0 2024-04-16T05:25:26.279 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.742+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc981076c0 con 0x7fcc981091d0 2024-04-16T05:25:26.279 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.742+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fcc8c019740 con 0x7fcc981091d0 2024-04-16T05:25:26.279 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.743+0000 7fcc94ff9700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 0x7fcc8003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:26.279 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.743+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 --> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcc98064150 con 0x7fcc80039560 2024-04-16T05:25:26.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.743+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc8c04e9e0 con 0x7fcc981091d0 2024-04-16T05:25:26.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.743+0000 7fcc96ffd700 1 -- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 msgr2=0x7fcc8003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1904754808 2024-04-16T05:25:26.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.743+0000 7fcc96ffd700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 0x7fcc8003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:25:26.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.944+0000 7fcc96ffd700 1 -- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 msgr2=0x7fcc8003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1904754808 2024-04-16T05:25:26.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.944+0000 7fcc96ffd700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 0x7fcc8003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:25:26.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.345+0000 7fcc96ffd700 1 -- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 msgr2=0x7fcc8003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1904754808 2024-04-16T05:25:26.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.345+0000 7fcc96ffd700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 0x7fcc8003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T05:25:26.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:22.146+0000 7fcc96ffd700 1 -- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 msgr2=0x7fcc8003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1904754808 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:22.146+0000 7fcc96ffd700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 0x7fcc8003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.747+0000 7fcc96ffd700 1 -- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 msgr2=0x7fcc8003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1904754808 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.747+0000 7fcc96ffd700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 0x7fcc8003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.655+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fcc8c003a50 con 0x7fcc981091d0 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.655+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 msgr2=0x7fcc8003ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.655+0000 7fcc94ff9700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 0x7fcc8003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.659+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fcc8c018350 con 0x7fcc981091d0 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.659+0000 7fcc94ff9700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fcc8003cc60 0x7fcc8003f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.659+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcc8c012ea0 con 0x7fcc8003cc60 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.659+0000 7fcc96ffd700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fcc8003cc60 0x7fcc8003f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.660+0000 7fcc96ffd700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fcc8003cc60 0x7fcc8003f050 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc88003a10 tx=0x7fcc88009290 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.660+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 <== mgr.14116 v2:172.21.15.86:6800/1420225001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fcc8c012ea0 con 0x7fcc8003cc60 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fcc98103e70 con 0x7fcc8003cc60 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc94ff9700 1 -- 172.21.15.86:0/1304901429 <== mgr.14116 v2:172.21.15.86:6800/1420225001 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fcc98103e70 con 0x7fcc8003cc60 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fcc8003cc60 msgr2=0x7fcc8003f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc9e120700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fcc8003cc60 0x7fcc8003f050 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc88003a10 tx=0x7fcc88009290 comp rx=0 tx=0).stop 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 msgr2=0x7fcc98108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc9e120700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc98108a50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009fb0 tx=0x7fcc8c011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 shutdown_connections 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc9e120700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fcc8003cc60 0x7fcc8003f050 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc9e120700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc981091d0 0x7fcc98108a50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.667+0000 7fcc9e120700 1 --2- 172.21.15.86:0/1304901429 >> [v2:172.21.15.86:6800/1904754808,v1:172.21.15.86:6801/1904754808] conn(0x7fcc80039560 0x7fcc8003ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.668+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 >> 172.21.15.86:0/1304901429 conn(0x7fcc98102b80 msgr2=0x7fcc98103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.668+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 shutdown_connections 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.668+0000 7fcc9e120700 1 -- 172.21.15.86:0/1304901429 wait complete. 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 5 is available 2024-04-16T05:25:26.283 INFO:teuthology.orchestra.run.smithi086.stdout:Setting orchestrator backend to cephadm... 2024-04-16T05:25:26.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:26 smithi086 ceph-mon[32028]: [16/Apr/2024:05:25:25] ENGINE Bus STARTING 2024-04-16T05:25:26.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:26 smithi086 ceph-mon[32028]: [16/Apr/2024:05:25:25] ENGINE Serving on http://172.21.15.86:8765 2024-04-16T05:25:26.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:26 smithi086 ceph-mon[32028]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-16T05:25:26.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:26 smithi086 ceph-mon[32028]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-16T05:25:26.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:26 smithi086 ceph-mon[32028]: [16/Apr/2024:05:25:25] ENGINE Serving on https://172.21.15.86:7150 2024-04-16T05:25:26.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:26 smithi086 ceph-mon[32028]: [16/Apr/2024:05:25:25] ENGINE Bus STARTED 2024-04-16T05:25:26.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:26 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:27.481 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.701+0000 7f67539c8700 1 Processor -- start 2024-04-16T05:25:27.481 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.701+0000 7f67539c8700 1 -- start start 2024-04-16T05:25:27.481 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.701+0000 7f67539c8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:27.481 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.701+0000 7f67539c8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f674c1090b0 con 0x7f674c1086c0 2024-04-16T05:25:27.481 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.701+0000 7f6751764700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:27.481 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.701+0000 7f6751764700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48796/0 (socket says 172.21.15.86:48796) 2024-04-16T05:25:27.481 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.701+0000 7f6751764700 1 -- 172.21.15.86:0/866144521 learned_addr learned my addr 172.21.15.86:0/866144521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.701+0000 7f6751764700 1 -- 172.21.15.86:0/866144521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c109820 con 0x7f674c1086c0 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f6751764700 1 --2- 172.21.15.86:0/866144521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c108ae0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f67480098a0 tx=0x7f6748009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2101ec8186bf13d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f6743fff700 1 -- 172.21.15.86:0/866144521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6748004160 con 0x7f674c1086c0 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f6743fff700 1 -- 172.21.15.86:0/866144521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6748004300 con 0x7f674c1086c0 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f6743fff700 1 -- 172.21.15.86:0/866144521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6748011b50 con 0x7f674c1086c0 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f67539c8700 1 -- 172.21.15.86:0/866144521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 msgr2=0x7f674c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f67539c8700 1 --2- 172.21.15.86:0/866144521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c108ae0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f67480098a0 tx=0x7f6748009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f67539c8700 1 -- 172.21.15.86:0/866144521 shutdown_connections 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f67539c8700 1 --2- 172.21.15.86:0/866144521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c108ae0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f67539c8700 1 -- 172.21.15.86:0/866144521 >> 172.21.15.86:0/866144521 conn(0x7f674c103ce0 msgr2=0x7f674c106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.702+0000 7f67539c8700 1 -- 172.21.15.86:0/866144521 shutdown_connections 2024-04-16T05:25:27.482 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.703+0000 7f67539c8700 1 -- 172.21.15.86:0/866144521 wait complete. 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.703+0000 7f67539c8700 1 Processor -- start 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.703+0000 7f67539c8700 1 -- start start 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.703+0000 7f67539c8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.703+0000 7f67539c8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f674c10cbd0 con 0x7f674c1086c0 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.703+0000 7f6751764700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.703+0000 7f6751764700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48802/0 (socket says 172.21.15.86:48802) 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.703+0000 7f6751764700 1 -- 172.21.15.86:0/695861565 learned_addr learned my addr 172.21.15.86:0/695861565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.704+0000 7f6751764700 1 -- 172.21.15.86:0/695861565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c19cb80 con 0x7f674c1086c0 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.704+0000 7f6751764700 1 --2- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c19c640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6748000c00 tx=0x7f6748011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.704+0000 7f67427fc700 1 -- 172.21.15.86:0/695861565 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6748003b20 con 0x7f674c1086c0 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.704+0000 7f67427fc700 1 -- 172.21.15.86:0/695861565 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6748003cc0 con 0x7f674c1086c0 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.704+0000 7f67427fc700 1 -- 172.21.15.86:0/695861565 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6748024910 con 0x7f674c1086c0 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.704+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674c19ce10 con 0x7f674c1086c0 2024-04-16T05:25:27.483 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.704+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f674c19fb00 con 0x7f674c1086c0 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.705+0000 7f67427fc700 1 -- 172.21.15.86:0/695861565 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f6748022020 con 0x7f674c1086c0 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.705+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674c064150 con 0x7f674c1086c0 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.705+0000 7f67427fc700 1 --2- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6738039550 0x7f673803ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.705+0000 7f67427fc700 1 -- 172.21.15.86:0/695861565 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6748011e00 con 0x7f674c1086c0 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.705+0000 7f6750f63700 1 --2- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6738039550 0x7f673803ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.706+0000 7f6750f63700 1 --2- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6738039550 0x7f673803ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f673c006fd0 tx=0x7f673c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.711+0000 7f67427fc700 1 -- 172.21.15.86:0/695861565 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f674801ad80 con 0x7f674c1086c0 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.712+0000 7f67427fc700 1 -- 172.21.15.86:0/695861565 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f674801a950 con 0x7f674c1086c0 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.922+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f674c104f80 con 0x7f6738039550 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.928+0000 7f67427fc700 1 -- 172.21.15.86:0/695861565 <== mgr.14116 v2:172.21.15.86:6800/1420225001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f674c104f80 con 0x7f6738039550 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.930+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6738039550 msgr2=0x7f673803ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.930+0000 7f67539c8700 1 --2- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6738039550 0x7f673803ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f673c006fd0 tx=0x7f673c006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:27.484 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.930+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 msgr2=0x7f674c19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:27.485 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.930+0000 7f67539c8700 1 --2- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c19c640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6748000c00 tx=0x7f6748011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:27.485 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.931+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 shutdown_connections 2024-04-16T05:25:27.485 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.931+0000 7f67539c8700 1 --2- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6738039550 0x7f673803ba10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.485 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.931+0000 7f67539c8700 1 --2- 172.21.15.86:0/695861565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f674c1086c0 0x7f674c19c640 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.485 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.931+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 >> 172.21.15.86:0/695861565 conn(0x7f674c103ce0 msgr2=0x7f674c104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:27.485 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.931+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 shutdown_connections 2024-04-16T05:25:27.485 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.931+0000 7f67539c8700 1 -- 172.21.15.86:0/695861565 wait complete. 2024-04-16T05:25:27.808 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:27 smithi086 ceph-mon[32028]: mgrmap e8: x(active, since 2s) 2024-04-16T05:25:27.809 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:27 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:27.809 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:27 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:28.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-16T05:25:28.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.916+0000 7f6b9a32e700 1 Processor -- start 2024-04-16T05:25:28.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.916+0000 7f6b9a32e700 1 -- start start 2024-04-16T05:25:28.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.916+0000 7f6b9a32e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9410b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:28.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.916+0000 7f6b9a32e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9407be10 con 0x7f6b941091d0 2024-04-16T05:25:28.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.916+0000 7f6b93fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9410b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:28.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.916+0000 7f6b93fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9410b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48816/0 (socket says 172.21.15.86:48816) 2024-04-16T05:25:28.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.916+0000 7f6b93fff700 1 -- 172.21.15.86:0/2594117457 learned_addr learned my addr 172.21.15.86:0/2594117457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.916+0000 7f6b93fff700 1 -- 172.21.15.86:0/2594117457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9410bb00 con 0x7f6b941091d0 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.916+0000 7f6b93fff700 1 --2- 172.21.15.86:0/2594117457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9410b5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6b840098a0 tx=0x7f6b84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ba7c54758afc1e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b92ffd700 1 -- 172.21.15.86:0/2594117457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b84004160 con 0x7f6b941091d0 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b92ffd700 1 -- 172.21.15.86:0/2594117457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b84004300 con 0x7f6b941091d0 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b92ffd700 1 -- 172.21.15.86:0/2594117457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b84011b50 con 0x7f6b941091d0 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b9a32e700 1 -- 172.21.15.86:0/2594117457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 msgr2=0x7f6b9410b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b9a32e700 1 --2- 172.21.15.86:0/2594117457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9410b5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6b840098a0 tx=0x7f6b84009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b9a32e700 1 -- 172.21.15.86:0/2594117457 shutdown_connections 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b9a32e700 1 --2- 172.21.15.86:0/2594117457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9410b5c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b9a32e700 1 -- 172.21.15.86:0/2594117457 >> 172.21.15.86:0/2594117457 conn(0x7f6b94102c40 msgr2=0x7f6b94105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b9a32e700 1 -- 172.21.15.86:0/2594117457 shutdown_connections 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.917+0000 7f6b9a32e700 1 -- 172.21.15.86:0/2594117457 wait complete. 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.918+0000 7f6b9a32e700 1 Processor -- start 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.918+0000 7f6b9a32e700 1 -- start start 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.918+0000 7f6b9a32e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.918+0000 7f6b9a32e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9410ca70 con 0x7f6b941091d0 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.918+0000 7f6b93fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.918+0000 7f6b93fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48824/0 (socket says 172.21.15.86:48824) 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.918+0000 7f6b93fff700 1 -- 172.21.15.86:0/3380494509 learned_addr learned my addr 172.21.15.86:0/3380494509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.919+0000 7f6b93fff700 1 -- 172.21.15.86:0/3380494509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9419cbb0 con 0x7f6b941091d0 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.919+0000 7f6b93fff700 1 --2- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9419c670 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009fb0 tx=0x7f6b84011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.919+0000 7f6b917fa700 1 -- 172.21.15.86:0/3380494509 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b840038b0 con 0x7f6b941091d0 2024-04-16T05:25:28.655 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.919+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9419ce40 con 0x7f6b941091d0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.919+0000 7f6b917fa700 1 -- 172.21.15.86:0/3380494509 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b84011e50 con 0x7f6b941091d0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.919+0000 7f6b917fa700 1 -- 172.21.15.86:0/3380494509 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b84019420 con 0x7f6b941091d0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.919+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9419fb30 con 0x7f6b941091d0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.920+0000 7f6b917fa700 1 -- 172.21.15.86:0/3380494509 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6b84019720 con 0x7f6b941091d0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.920+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b94064150 con 0x7f6b941091d0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.920+0000 7f6b917fa700 1 --2- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6b7c039560 0x7f6b7c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.920+0000 7f6b917fa700 1 -- 172.21.15.86:0/3380494509 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b8404d9d0 con 0x7f6b941091d0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.920+0000 7f6b937fe700 1 --2- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6b7c039560 0x7f6b7c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.921+0000 7f6b937fe700 1 --2- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6b7c039560 0x7f6b7c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b88006fd0 tx=0x7f6b88006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.925+0000 7f6b917fa700 1 -- 172.21.15.86:0/3380494509 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b8400bc70 con 0x7f6b941091d0 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.126+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6b94103e80 con 0x7f6b7c039560 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.127+0000 7f6b917fa700 1 -- 172.21.15.86:0/3380494509 <== mgr.14116 v2:172.21.15.86:6800/1420225001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6b94103e80 con 0x7f6b7c039560 2024-04-16T05:25:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.129+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6b7c039560 msgr2=0x7f6b7c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.129+0000 7f6b9a32e700 1 --2- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6b7c039560 0x7f6b7c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b88006fd0 tx=0x7f6b88006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.129+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 msgr2=0x7f6b9419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.129+0000 7f6b9a32e700 1 --2- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9419c670 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009fb0 tx=0x7f6b84011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.129+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 shutdown_connections 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.129+0000 7f6b9a32e700 1 --2- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f6b7c039560 0x7f6b7c03ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.129+0000 7f6b9a32e700 1 --2- 172.21.15.86:0/3380494509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b941091d0 0x7f6b9419c670 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.129+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 >> 172.21.15.86:0/3380494509 conn(0x7f6b94102c40 msgr2=0x7f6b94103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.130+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 shutdown_connections 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.130+0000 7f6b9a32e700 1 -- 172.21.15.86:0/3380494509 wait complete. 2024-04-16T05:25:28.657 INFO:teuthology.orchestra.run.smithi086.stdout:Generating ssh key... 2024-04-16T05:25:28.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:28 smithi086 ceph-mon[32028]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:29.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-mon[32028]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:29.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:29.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:29.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: Generating public/private rsa key pair. 2024-04-16T05:25:29.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: Your identification has been saved in /tmp/tmp9d8e_qy8/key. 2024-04-16T05:25:29.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: Your public key has been saved in /tmp/tmp9d8e_qy8/key.pub. 2024-04-16T05:25:29.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: The key fingerprint is: 2024-04-16T05:25:29.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: SHA256:iVtsGXD17sGQ8xZG3E9we2ce671apzwFVc/EyXh/VHM ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:25:29.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: The key's randomart image is: 2024-04-16T05:25:29.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: +---[RSA 3072]----+ 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: | . ......++E| 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: | o +...BX| 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: | . + + .BB| 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: | o + B ..oO| 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: | . S * oo| 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: | + o .. o| 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: | . . +o| 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: | .o.o| 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: | .+o | 2024-04-16T05:25:29.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: +----[SHA256]-----+ 2024-04-16T05:25:30.268 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.076+0000 7f3ae5e20700 1 Processor -- start 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.076+0000 7f3ae5e20700 1 -- start start 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.076+0000 7f3ae5e20700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae01079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.076+0000 7f3ae5e20700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0107f70 con 0x7f3ae0107580 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.076+0000 7f3adf7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae01079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.076+0000 7f3adf7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae01079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48828/0 (socket says 172.21.15.86:48828) 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.076+0000 7f3adf7fe700 1 -- 172.21.15.86:0/3772444723 learned_addr learned my addr 172.21.15.86:0/3772444723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.076+0000 7f3adf7fe700 1 -- 172.21.15.86:0/3772444723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae0108780 con 0x7f3ae0107580 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.076+0000 7f3adf7fe700 1 --2- 172.21.15.86:0/3772444723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae01079a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00098a0 tx=0x7f3ad0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2eff0a1c1b9c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3adeffd700 1 -- 172.21.15.86:0/3772444723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ad0004160 con 0x7f3ae0107580 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3adeffd700 1 -- 172.21.15.86:0/3772444723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3ad0004300 con 0x7f3ae0107580 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3adeffd700 1 -- 172.21.15.86:0/3772444723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ad0011b50 con 0x7f3ae0107580 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3ae5e20700 1 -- 172.21.15.86:0/3772444723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 msgr2=0x7f3ae01079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3ae5e20700 1 --2- 172.21.15.86:0/3772444723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae01079a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00098a0 tx=0x7f3ad0009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:30.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3ae5e20700 1 -- 172.21.15.86:0/3772444723 shutdown_connections 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3ae5e20700 1 --2- 172.21.15.86:0/3772444723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae01079a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3ae5e20700 1 -- 172.21.15.86:0/3772444723 >> 172.21.15.86:0/3772444723 conn(0x7f3ae0102b80 msgr2=0x7f3ae0104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3ae5e20700 1 -- 172.21.15.86:0/3772444723 shutdown_connections 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.077+0000 7f3ae5e20700 1 -- 172.21.15.86:0/3772444723 wait complete. 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.078+0000 7f3ae5e20700 1 Processor -- start 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.078+0000 7f3ae5e20700 1 -- start start 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.078+0000 7f3ae5e20700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.078+0000 7f3ae5e20700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae007aff0 con 0x7f3ae0107580 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.078+0000 7f3adf7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.078+0000 7f3adf7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48836/0 (socket says 172.21.15.86:48836) 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.078+0000 7f3adf7fe700 1 -- 172.21.15.86:0/2402071946 learned_addr learned my addr 172.21.15.86:0/2402071946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:30.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.078+0000 7f3adf7fe700 1 -- 172.21.15.86:0/2402071946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae019a9f0 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.079+0000 7f3adf7fe700 1 --2- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae019a4b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009fb0 tx=0x7f3ad0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.079+0000 7f3add7fa700 1 -- 172.21.15.86:0/2402071946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ad00038b0 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.079+0000 7f3add7fa700 1 -- 172.21.15.86:0/2402071946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3ad0011e50 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.079+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae019ac80 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.079+0000 7f3add7fa700 1 -- 172.21.15.86:0/2402071946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ad0019550 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.079+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae019d970 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.080+0000 7f3add7fa700 1 -- 172.21.15.86:0/2402071946 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3ad0019850 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.080+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae0064150 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.080+0000 7f3add7fa700 1 --2- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f3acc03b770 0x7f3acc03dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.080+0000 7f3add7fa700 1 -- 172.21.15.86:0/2402071946 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3ad004dae0 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.080+0000 7f3ad7fff700 1 --2- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f3acc03b770 0x7f3acc03dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.080+0000 7f3ad7fff700 1 --2- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f3acc03b770 0x7f3acc03dc30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8006fd0 tx=0x7f3ac8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.085+0000 7f3add7fa700 1 -- 172.21.15.86:0/2402071946 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ad0018dc0 con 0x7f3ae0107580 2024-04-16T05:25:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.284+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f3ae0062f30 con 0x7f3acc03b770 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.628+0000 7f3add7fa700 1 -- 172.21.15.86:0/2402071946 <== mgr.14116 v2:172.21.15.86:6800/1420225001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ae0062f30 con 0x7f3acc03b770 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f3acc03b770 msgr2=0x7f3acc03dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 --2- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f3acc03b770 0x7f3acc03dc30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8006fd0 tx=0x7f3ac8006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 msgr2=0x7f3ae019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 --2- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae019a4b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009fb0 tx=0x7f3ad0011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 shutdown_connections 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 --2- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f3acc03b770 0x7f3acc03dc30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 --2- 172.21.15.86:0/2402071946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ae0107580 0x7f3ae019a4b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 >> 172.21.15.86:0/2402071946 conn(0x7f3ae0102b80 msgr2=0x7f3ae0190b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 shutdown_connections 2024-04-16T05:25:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.630+0000 7f3ae5e20700 1 -- 172.21.15.86:0/2402071946 wait complete. 2024-04-16T05:25:30.895 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:30 smithi086 ceph-mon[32028]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:30.895 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:30 smithi086 ceph-mon[32028]: Generating ssh key... 2024-04-16T05:25:31.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOOITO0oATfHLwKhPKtob4+MLIWNV7p2616NsdGtWk6lwh6dpw0zzat/knIeArU/sZ0my/9yl0xtmPjI9CETax+KDuW4FPUtsxTieI8Y6vOjRUGN7PNQqAyTPakyUUw7qijuGD31AqL6zzfr1k9rmeS3FRPMy+7H0Z+vTB7Lj7sfNSVRh0YbSMewXdtSSPPBq57SVjLI90ZkCb4q3ttuhcKuhafHpJKN33in+liohPQ8ns6MJbTBJen+PwECO401MkTi7sSmo8K3tiTOSImETpRDZYTYgUgowmNVIq+Chp4h50FRhDGjC5y98HLHde1HKEXODe6wUUvl3+0ibu1NG8GXlZKymsAxrF7MisIWwZXby1pHw8PaC7BtBUL5sEcz3l3IMFF8bG11VlfY7ug0xH9wzh6r23uweExnLXzBSyS+4M6Ts7wTL9T/TDP0YIawHeWZYy0fZbifV+8bZlqvvgVrR1LHD+xK7Tn0pj4z4jV0Sf6UPRnP+HxEtE0+VlzXc= ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:25:31.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.689+0000 7fd1781d6700 1 Processor -- start 2024-04-16T05:25:31.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.689+0000 7fd1781d6700 1 -- start start 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.689+0000 7fd1781d6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170106da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.689+0000 7fd1781d6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd170107370 con 0x7fd170106980 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.689+0000 7fd175f72700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.689+0000 7fd175f72700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44672/0 (socket says 172.21.15.86:44672) 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.689+0000 7fd175f72700 1 -- 172.21.15.86:0/3718279062 learned_addr learned my addr 172.21.15.86:0/3718279062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.689+0000 7fd175f72700 1 -- 172.21.15.86:0/3718279062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1701074f0 con 0x7fd170106980 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.690+0000 7fd175f72700 1 --2- 172.21.15.86:0/3718279062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170106da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0098a0 tx=0x7fd16c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339ccad7271f5f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.690+0000 7fd174f70700 1 -- 172.21.15.86:0/3718279062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd16c004160 con 0x7fd170106980 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.690+0000 7fd174f70700 1 -- 172.21.15.86:0/3718279062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd16c004300 con 0x7fd170106980 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.690+0000 7fd174f70700 1 -- 172.21.15.86:0/3718279062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd16c011b50 con 0x7fd170106980 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.690+0000 7fd1781d6700 1 -- 172.21.15.86:0/3718279062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 msgr2=0x7fd170106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.690+0000 7fd1781d6700 1 --2- 172.21.15.86:0/3718279062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170106da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0098a0 tx=0x7fd16c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:31.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.690+0000 7fd1781d6700 1 -- 172.21.15.86:0/3718279062 shutdown_connections 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.690+0000 7fd1781d6700 1 --2- 172.21.15.86:0/3718279062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170106da0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.690+0000 7fd1781d6700 1 -- 172.21.15.86:0/3718279062 >> 172.21.15.86:0/3718279062 conn(0x7fd1701023e0 msgr2=0x7fd170104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.691+0000 7fd1781d6700 1 -- 172.21.15.86:0/3718279062 shutdown_connections 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.691+0000 7fd1781d6700 1 -- 172.21.15.86:0/3718279062 wait complete. 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.691+0000 7fd1781d6700 1 Processor -- start 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.691+0000 7fd1781d6700 1 -- start start 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.691+0000 7fd1781d6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170109280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.691+0000 7fd1781d6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd17010cc30 con 0x7fd170106980 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.691+0000 7fd175f72700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.691+0000 7fd175f72700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44688/0 (socket says 172.21.15.86:44688) 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.691+0000 7fd175f72700 1 -- 172.21.15.86:0/2929852057 learned_addr learned my addr 172.21.15.86:0/2929852057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.692+0000 7fd175f72700 1 -- 172.21.15.86:0/2929852057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1701097c0 con 0x7fd170106980 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.692+0000 7fd175f72700 1 --2- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170109280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd16c011850 tx=0x7fd16c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.692+0000 7fd166ffd700 1 -- 172.21.15.86:0/2929852057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd16c003af0 con 0x7fd170106980 2024-04-16T05:25:31.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.692+0000 7fd166ffd700 1 -- 172.21.15.86:0/2929852057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd16c020070 con 0x7fd170106980 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.692+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1701079b0 con 0x7fd170106980 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.692+0000 7fd166ffd700 1 -- 172.21.15.86:0/2929852057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd16c0195a0 con 0x7fd170106980 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.692+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd170107ef0 con 0x7fd170106980 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.693+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd170064150 con 0x7fd170106980 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.693+0000 7fd166ffd700 1 -- 172.21.15.86:0/2929852057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd16c019850 con 0x7fd170106980 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.693+0000 7fd166ffd700 1 --2- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fd15c039600 0x7fd15c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.693+0000 7fd166ffd700 1 -- 172.21.15.86:0/2929852057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd16c011e00 con 0x7fd170106980 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.694+0000 7fd175771700 1 --2- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fd15c039600 0x7fd15c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.694+0000 7fd175771700 1 --2- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fd15c039600 0x7fd15c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd160006fd0 tx=0x7fd160006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.698+0000 7fd166ffd700 1 -- 172.21.15.86:0/2929852057 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd16c019b40 con 0x7fd170106980 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.896+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd170103680 con 0x7fd15c039600 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.897+0000 7fd166ffd700 1 -- 172.21.15.86:0/2929852057 <== mgr.14116 v2:172.21.15.86:6800/1420225001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd170103680 con 0x7fd15c039600 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.899+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fd15c039600 msgr2=0x7fd15c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.899+0000 7fd1781d6700 1 --2- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fd15c039600 0x7fd15c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd160006fd0 tx=0x7fd160006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.899+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 msgr2=0x7fd170109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.899+0000 7fd1781d6700 1 --2- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170109280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd16c011850 tx=0x7fd16c011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.899+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 shutdown_connections 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.899+0000 7fd1781d6700 1 --2- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fd15c039600 0x7fd15c03bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.899+0000 7fd1781d6700 1 --2- 172.21.15.86:0/2929852057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd170106980 0x7fd170109280 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.899+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 >> 172.21.15.86:0/2929852057 conn(0x7fd1701023e0 msgr2=0x7fd170102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.900+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 shutdown_connections 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:30.900+0000 7fd1781d6700 1 -- 172.21.15.86:0/2929852057 wait complete. 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:Adding key to root@localhost authorized_keys... 2024-04-16T05:25:31.562 INFO:teuthology.orchestra.run.smithi086.stdout:Adding host smithi086... 2024-04-16T05:25:32.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:31 smithi086 ceph-mon[32028]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:33.113 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:32 smithi086 ceph-mon[32028]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:33.114 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:32 smithi086 ceph-mon[32028]: Deploying cephadm binary to smithi086 2024-04-16T05:25:37.016 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Added host 'smithi086' with addr '172.21.15.86' 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.942+0000 7f29de143700 1 Processor -- start 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.943+0000 7f29de143700 1 -- start start 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.943+0000 7f29de143700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d8106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.943+0000 7f29de143700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29d8107330 con 0x7f29d8106940 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.943+0000 7f29d77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d8106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.943+0000 7f29d77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d8106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44698/0 (socket says 172.21.15.86:44698) 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.943+0000 7f29d77fe700 1 -- 172.21.15.86:0/601256303 learned_addr learned my addr 172.21.15.86:0/601256303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.943+0000 7f29d77fe700 1 -- 172.21.15.86:0/601256303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d8109920 con 0x7f29d8106940 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.943+0000 7f29d77fe700 1 --2- 172.21.15.86:0/601256303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d8106d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29c80098a0 tx=0x7f29c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52d48496b17fd50 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29d6ffd700 1 -- 172.21.15.86:0/601256303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29c8004160 con 0x7f29d8106940 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29d6ffd700 1 -- 172.21.15.86:0/601256303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29c8004300 con 0x7f29d8106940 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29d6ffd700 1 -- 172.21.15.86:0/601256303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29c8011b50 con 0x7f29d8106940 2024-04-16T05:25:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29de143700 1 -- 172.21.15.86:0/601256303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 msgr2=0x7f29d8106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29de143700 1 --2- 172.21.15.86:0/601256303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d8106d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29c80098a0 tx=0x7f29c8009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29de143700 1 -- 172.21.15.86:0/601256303 shutdown_connections 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29de143700 1 --2- 172.21.15.86:0/601256303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d8106d60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29de143700 1 -- 172.21.15.86:0/601256303 >> 172.21.15.86:0/601256303 conn(0x7f29d8102360 msgr2=0x7f29d8104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29de143700 1 -- 172.21.15.86:0/601256303 shutdown_connections 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.944+0000 7f29de143700 1 -- 172.21.15.86:0/601256303 wait complete. 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.945+0000 7f29de143700 1 Processor -- start 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.945+0000 7f29de143700 1 -- start start 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.945+0000 7f29de143700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.945+0000 7f29de143700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29d810ccd0 con 0x7f29d8106940 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.945+0000 7f29d77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.945+0000 7f29d77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44708/0 (socket says 172.21.15.86:44708) 2024-04-16T05:25:37.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.945+0000 7f29d77fe700 1 -- 172.21.15.86:0/1658035431 learned_addr learned my addr 172.21.15.86:0/1658035431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.945+0000 7f29d77fe700 1 -- 172.21.15.86:0/1658035431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d819cc10 con 0x7f29d8106940 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.946+0000 7f29d77fe700 1 --2- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d819c6d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009fb0 tx=0x7f29c8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.946+0000 7f29d57fa700 1 -- 172.21.15.86:0/1658035431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29c80038b0 con 0x7f29d8106940 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.946+0000 7f29d57fa700 1 -- 172.21.15.86:0/1658035431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29c8011e50 con 0x7f29d8106940 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.946+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d819cea0 con 0x7f29d8106940 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.946+0000 7f29d57fa700 1 -- 172.21.15.86:0/1658035431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29c8019550 con 0x7f29d8106940 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.946+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d819fb90 con 0x7f29d8106940 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.947+0000 7f29d57fa700 1 -- 172.21.15.86:0/1658035431 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f29c8019850 con 0x7f29d8106940 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.947+0000 7f29d57fa700 1 --2- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f29c4039560 0x7f29c403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.947+0000 7f29d57fa700 1 -- 172.21.15.86:0/1658035431 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f29c804d9b0 con 0x7f29d8106940 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.947+0000 7f29cffff700 1 --2- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f29c4039560 0x7f29c403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.947+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d8064150 con 0x7f29d8106940 2024-04-16T05:25:37.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.947+0000 7f29cffff700 1 --2- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f29c4039560 0x7f29c403ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006fd0 tx=0x7f29c0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.952+0000 7f29d57fa700 1 -- 172.21.15.86:0/1658035431 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29c8020240 con 0x7f29d8106940 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:32.152+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}) -- 0x7f29d8103690 con 0x7f29c4039560 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.412+0000 7f29d57fa700 1 -- 172.21.15.86:0/1658035431 <== mgr.14116 v2:172.21.15.86:6800/1420225001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f29d8103690 con 0x7f29c4039560 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.415+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f29c4039560 msgr2=0x7f29c403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.415+0000 7f29de143700 1 --2- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f29c4039560 0x7f29c403ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006fd0 tx=0x7f29c0006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.415+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 msgr2=0x7f29d819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.415+0000 7f29de143700 1 --2- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d819c6d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009fb0 tx=0x7f29c8011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.416+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 shutdown_connections 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.416+0000 7f29de143700 1 --2- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f29c4039560 0x7f29c403ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:37.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.416+0000 7f29de143700 1 --2- 172.21.15.86:0/1658035431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d8106940 0x7f29d819c6d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:37.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.416+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 >> 172.21.15.86:0/1658035431 conn(0x7f29d8102360 msgr2=0x7f29d8102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:37.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.416+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 shutdown_connections 2024-04-16T05:25:37.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.416+0000 7f29de143700 1 -- 172.21.15.86:0/1658035431 wait complete. 2024-04-16T05:25:37.021 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mon service... 2024-04-16T05:25:37.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:37 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:37.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:37 smithi086 ceph-mon[32028]: Added host smithi086 2024-04-16T05:25:37.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:37 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:38.434 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-16T05:25:38.434 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.441+0000 7fe5f965b700 1 Processor -- start 2024-04-16T05:25:38.434 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.441+0000 7fe5f965b700 1 -- start start 2024-04-16T05:25:38.434 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.441+0000 7fe5f965b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f4108af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.442+0000 7fe5f965b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41090c0 con 0x7fe5f41086d0 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.442+0000 7fe5f2ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f4108af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.442+0000 7fe5f2ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f4108af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44712/0 (socket says 172.21.15.86:44712) 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.442+0000 7fe5f2ffd700 1 -- 172.21.15.86:0/2473515642 learned_addr learned my addr 172.21.15.86:0/2473515642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.442+0000 7fe5f2ffd700 1 -- 172.21.15.86:0/2473515642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f4109830 con 0x7fe5f41086d0 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.442+0000 7fe5f2ffd700 1 --2- 172.21.15.86:0/2473515642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f4108af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40098a0 tx=0x7fe5e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb25640edb28da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.442+0000 7fe5f27fc700 1 -- 172.21.15.86:0/2473515642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e4004160 con 0x7fe5f41086d0 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.442+0000 7fe5f27fc700 1 -- 172.21.15.86:0/2473515642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe5e4004300 con 0x7fe5f41086d0 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.442+0000 7fe5f27fc700 1 -- 172.21.15.86:0/2473515642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e4011b50 con 0x7fe5f41086d0 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.443+0000 7fe5f965b700 1 -- 172.21.15.86:0/2473515642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 msgr2=0x7fe5f4108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.443+0000 7fe5f965b700 1 --2- 172.21.15.86:0/2473515642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f4108af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40098a0 tx=0x7fe5e4009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.443+0000 7fe5f965b700 1 -- 172.21.15.86:0/2473515642 shutdown_connections 2024-04-16T05:25:38.435 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.443+0000 7fe5f965b700 1 --2- 172.21.15.86:0/2473515642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f4108af0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.443+0000 7fe5f965b700 1 -- 172.21.15.86:0/2473515642 >> 172.21.15.86:0/2473515642 conn(0x7fe5f4103d10 msgr2=0x7fe5f4106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.443+0000 7fe5f965b700 1 -- 172.21.15.86:0/2473515642 shutdown_connections 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.443+0000 7fe5f965b700 1 -- 172.21.15.86:0/2473515642 wait complete. 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.444+0000 7fe5f965b700 1 Processor -- start 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.444+0000 7fe5f965b700 1 -- start start 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.444+0000 7fe5f965b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.444+0000 7fe5f965b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f410cbe0 con 0x7fe5f41086d0 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.444+0000 7fe5f2ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.444+0000 7fe5f2ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f419c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44718/0 (socket says 172.21.15.86:44718) 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.444+0000 7fe5f2ffd700 1 -- 172.21.15.86:0/2349846465 learned_addr learned my addr 172.21.15.86:0/2349846465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.444+0000 7fe5f2ffd700 1 -- 172.21.15.86:0/2349846465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f419cb40 con 0x7fe5f41086d0 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.444+0000 7fe5f2ffd700 1 --2- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f419c600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4011670 tx=0x7fe5e4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:38.436 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.445+0000 7fe5f0ff9700 1 -- 172.21.15.86:0/2349846465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e40039c0 con 0x7fe5f41086d0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.445+0000 7fe5f0ff9700 1 -- 172.21.15.86:0/2349846465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe5e4011e50 con 0x7fe5f41086d0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.445+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f419cdd0 con 0x7fe5f41086d0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.445+0000 7fe5f0ff9700 1 -- 172.21.15.86:0/2349846465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e401a5c0 con 0x7fe5f41086d0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.445+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f419fac0 con 0x7fe5f41086d0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.445+0000 7fe5f0ff9700 1 -- 172.21.15.86:0/2349846465 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe5e401a8c0 con 0x7fe5f41086d0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.446+0000 7fe5f0ff9700 1 --2- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fe5e0039560 0x7fe5e003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.446+0000 7fe5f0ff9700 1 -- 172.21.15.86:0/2349846465 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe5e4003b60 con 0x7fe5f41086d0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.446+0000 7fe5ebfff700 1 --2- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fe5e0039560 0x7fe5e003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.446+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f4064150 con 0x7fe5f41086d0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.446+0000 7fe5ebfff700 1 --2- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fe5e0039560 0x7fe5e003ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc006fd0 tx=0x7fe5dc006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.451+0000 7fe5f0ff9700 1 -- 172.21.15.86:0/2349846465 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5e4019c70 con 0x7fe5f41086d0 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.691+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe5f4104f80 con 0x7fe5e0039560 2024-04-16T05:25:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.695+0000 7fe5f0ff9700 1 -- 172.21.15.86:0/2349846465 <== mgr.14116 v2:172.21.15.86:6800/1420225001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe5f4104f80 con 0x7fe5e0039560 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.697+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fe5e0039560 msgr2=0x7fe5e003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.697+0000 7fe5f965b700 1 --2- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fe5e0039560 0x7fe5e003ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc006fd0 tx=0x7fe5dc006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.697+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 msgr2=0x7fe5f419c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.697+0000 7fe5f965b700 1 --2- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f419c600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4011670 tx=0x7fe5e4011b60 comp rx=0 tx=0).stop 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.697+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 shutdown_connections 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.697+0000 7fe5f965b700 1 --2- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fe5e0039560 0x7fe5e003ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.697+0000 7fe5f965b700 1 --2- 172.21.15.86:0/2349846465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5f41086d0 0x7fe5f419c600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.697+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 >> 172.21.15.86:0/2349846465 conn(0x7fe5f4103d10 msgr2=0x7fe5f4104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.698+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 shutdown_connections 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.698+0000 7fe5f965b700 1 -- 172.21.15.86:0/2349846465 wait complete. 2024-04-16T05:25:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mgr service... 2024-04-16T05:25:38.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:38 smithi086 ceph-mon[32028]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:38.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:38 smithi086 ceph-mon[32028]: Saving service mon spec with placement count:5 2024-04-16T05:25:38.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:38 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:40.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:40 smithi086 ceph-mon[32028]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:40 smithi086 ceph-mon[32028]: Saving service mgr spec with placement count:2 2024-04-16T05:25:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:40 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:40 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:40.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-16T05:25:40.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.886+0000 7ff89e6e6700 1 Processor -- start 2024-04-16T05:25:40.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.886+0000 7ff89e6e6700 1 -- start start 2024-04-16T05:25:40.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.887+0000 7ff89e6e6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff898108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:40.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.887+0000 7ff89e6e6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8981090b0 con 0x7ff8981086c0 2024-04-16T05:25:40.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.887+0000 7ff897fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff898108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:40.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.887+0000 7ff897fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff898108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44734/0 (socket says 172.21.15.86:44734) 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.887+0000 7ff897fff700 1 -- 172.21.15.86:0/2927503995 learned_addr learned my addr 172.21.15.86:0/2927503995 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.887+0000 7ff897fff700 1 -- 172.21.15.86:0/2927503995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff898109870 con 0x7ff8981086c0 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.887+0000 7ff897fff700 1 --2- 172.21.15.86:0/2927503995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff898108ae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8880098a0 tx=0x7ff888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab00a35ac1da72c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.888+0000 7ff896ffd700 1 -- 172.21.15.86:0/2927503995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff888004160 con 0x7ff8981086c0 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.888+0000 7ff896ffd700 1 -- 172.21.15.86:0/2927503995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff888004300 con 0x7ff8981086c0 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.888+0000 7ff896ffd700 1 -- 172.21.15.86:0/2927503995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff888011b50 con 0x7ff8981086c0 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.888+0000 7ff89e6e6700 1 -- 172.21.15.86:0/2927503995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 msgr2=0x7ff898108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.888+0000 7ff89e6e6700 1 --2- 172.21.15.86:0/2927503995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff898108ae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8880098a0 tx=0x7ff888009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.888+0000 7ff89e6e6700 1 -- 172.21.15.86:0/2927503995 shutdown_connections 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.888+0000 7ff89e6e6700 1 --2- 172.21.15.86:0/2927503995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff898108ae0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.888+0000 7ff89e6e6700 1 -- 172.21.15.86:0/2927503995 >> 172.21.15.86:0/2927503995 conn(0x7ff898103d40 msgr2=0x7ff898106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.889+0000 7ff89e6e6700 1 -- 172.21.15.86:0/2927503995 shutdown_connections 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.889+0000 7ff89e6e6700 1 -- 172.21.15.86:0/2927503995 wait complete. 2024-04-16T05:25:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.889+0000 7ff89e6e6700 1 Processor -- start 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.889+0000 7ff89e6e6700 1 -- start start 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff89e6e6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff89807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff89e6e6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff89810cc20 con 0x7ff8981086c0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff897fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff89807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff897fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff89807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44738/0 (socket says 172.21.15.86:44738) 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff897fff700 1 -- 172.21.15.86:0/648450042 learned_addr learned my addr 172.21.15.86:0/648450042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff897fff700 1 -- 172.21.15.86:0/648450042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89807e0d0 con 0x7ff8981086c0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff897fff700 1 --2- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff89807db90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff888009fb0 tx=0x7ff888011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff8957fa700 1 -- 172.21.15.86:0/648450042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8880038b0 con 0x7ff8981086c0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff8957fa700 1 -- 172.21.15.86:0/648450042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff888011e50 con 0x7ff8981086c0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89807c2c0 con 0x7ff8981086c0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.890+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89807c800 con 0x7ff8981086c0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.891+0000 7ff8957fa700 1 -- 172.21.15.86:0/648450042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff888019550 con 0x7ff8981086c0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.892+0000 7ff8957fa700 1 -- 172.21.15.86:0/648450042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff888019850 con 0x7ff8981086c0 2024-04-16T05:25:40.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.892+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff884005360 con 0x7ff8981086c0 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.892+0000 7ff8957fa700 1 --2- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff880039560 0x7ff88003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.892+0000 7ff8957fa700 1 -- 172.21.15.86:0/648450042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff88804d9b0 con 0x7ff8981086c0 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.892+0000 7ff8977fe700 1 --2- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff880039560 0x7ff88003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.892+0000 7ff8977fe700 1 --2- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff880039560 0x7ff88003ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff88c006fd0 tx=0x7ff88c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.897+0000 7ff8957fa700 1 -- 172.21.15.86:0/648450042 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff88800fda0 con 0x7ff8981086c0 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.116+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff884000bf0 con 0x7ff880039560 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.121+0000 7ff8957fa700 1 -- 172.21.15.86:0/648450042 <== mgr.14116 v2:172.21.15.86:6800/1420225001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff884000bf0 con 0x7ff880039560 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff880039560 msgr2=0x7ff88003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 --2- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff880039560 0x7ff88003ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff88c006fd0 tx=0x7ff88c006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 msgr2=0x7ff89807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 --2- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff89807db90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff888009fb0 tx=0x7ff888011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 shutdown_connections 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 --2- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff880039560 0x7ff88003ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:40.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 --2- 172.21.15.86:0/648450042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8981086c0 0x7ff89807db90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:40.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 >> 172.21.15.86:0/648450042 conn(0x7ff898103d40 msgr2=0x7ff8981048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:40.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 shutdown_connections 2024-04-16T05:25:40.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:39.123+0000 7ff89e6e6700 1 -- 172.21.15.86:0/648450042 wait complete. 2024-04-16T05:25:41.854 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:41 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:41.855 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:41 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2936736766' entity='client.admin' 2024-04-16T05:25:41.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.976+0000 7f4e67b6d700 1 Processor -- start 2024-04-16T05:25:41.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.976+0000 7f4e67b6d700 1 -- start start 2024-04-16T05:25:41.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e67b6d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e60108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:41.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e67b6d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60109070 con 0x7f4e60108680 2024-04-16T05:25:41.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e65909700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e60108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e65909700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e60108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53122/0 (socket says 172.21.15.86:53122) 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e65909700 1 -- 172.21.15.86:0/2307883497 learned_addr learned my addr 172.21.15.86:0/2307883497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e65909700 1 -- 172.21.15.86:0/2307883497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e60109830 con 0x7f4e60108680 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e65909700 1 --2- 172.21.15.86:0/2307883497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e60108aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0098a0 tx=0x7f4e5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8668e5e4896d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e64907700 1 -- 172.21.15.86:0/2307883497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e5c004160 con 0x7f4e60108680 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e64907700 1 -- 172.21.15.86:0/2307883497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5c004300 con 0x7f4e60108680 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.977+0000 7f4e64907700 1 -- 172.21.15.86:0/2307883497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e5c011b20 con 0x7f4e60108680 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.978+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2307883497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 msgr2=0x7f4e60108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.978+0000 7f4e67b6d700 1 --2- 172.21.15.86:0/2307883497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e60108aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0098a0 tx=0x7f4e5c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.978+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2307883497 shutdown_connections 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.978+0000 7f4e67b6d700 1 --2- 172.21.15.86:0/2307883497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e60108aa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.978+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2307883497 >> 172.21.15.86:0/2307883497 conn(0x7f4e60103d40 msgr2=0x7f4e60106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:41.862 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.978+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2307883497 shutdown_connections 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.978+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2307883497 wait complete. 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.979+0000 7f4e67b6d700 1 Processor -- start 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.979+0000 7f4e67b6d700 1 -- start start 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.979+0000 7f4e67b6d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e6019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.979+0000 7f4e67b6d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6010cbe0 con 0x7f4e60108680 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e65909700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e6019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e65909700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e6019c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53136/0 (socket says 172.21.15.86:53136) 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e65909700 1 -- 172.21.15.86:0/2936736766 learned_addr learned my addr 172.21.15.86:0/2936736766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e65909700 1 -- 172.21.15.86:0/2936736766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6019cb20 con 0x7f4e60108680 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e65909700 1 --2- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e6019c5e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0116a0 tx=0x7f4e5c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e56ffd700 1 -- 172.21.15.86:0/2936736766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e5c003a40 con 0x7f4e60108680 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e56ffd700 1 -- 172.21.15.86:0/2936736766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5c003be0 con 0x7f4e60108680 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6019cdb0 con 0x7f4e60108680 2024-04-16T05:25:41.863 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e56ffd700 1 -- 172.21.15.86:0/2936736766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e5c01a720 con 0x7f4e60108680 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.980+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6019d210 con 0x7f4e60108680 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.981+0000 7f4e56ffd700 1 -- 172.21.15.86:0/2936736766 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4e5c01aa60 con 0x7f4e60108680 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.981+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e60064150 con 0x7f4e60108680 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.981+0000 7f4e56ffd700 1 --2- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f4e4c03d970 0x7f4e4c03fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.981+0000 7f4e56ffd700 1 -- 172.21.15.86:0/2936736766 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4e5c04ec90 con 0x7f4e60108680 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.982+0000 7f4e65108700 1 --2- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f4e4c03d970 0x7f4e4c03fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.982+0000 7f4e65108700 1 --2- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f4e4c03d970 0x7f4e4c03fe30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4e50006fd0 tx=0x7f4e50006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.987+0000 7f4e56ffd700 1 -- 172.21.15.86:0/2936736766 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e5c011e20 con 0x7f4e60108680 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.199+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4e6007c100 con 0x7f4e60108680 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.201+0000 7f4e56ffd700 1 -- 172.21.15.86:0/2936736766 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4e5c016df0 con 0x7f4e60108680 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.203+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f4e4c03d970 msgr2=0x7f4e4c03fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:41.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.203+0000 7f4e67b6d700 1 --2- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f4e4c03d970 0x7f4e4c03fe30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4e50006fd0 tx=0x7f4e50006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:41.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.204+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 msgr2=0x7f4e6019c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:41.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.204+0000 7f4e67b6d700 1 --2- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e6019c5e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0116a0 tx=0x7f4e5c011ba0 comp rx=0 tx=0).stop 2024-04-16T05:25:41.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.204+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 shutdown_connections 2024-04-16T05:25:41.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.204+0000 7f4e67b6d700 1 --2- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f4e4c03d970 0x7f4e4c03fe30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:41.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.204+0000 7f4e67b6d700 1 --2- 172.21.15.86:0/2936736766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60108680 0x7f4e6019c5e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:41.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.204+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 >> 172.21.15.86:0/2936736766 conn(0x7f4e60103d40 msgr2=0x7f4e60104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:41.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.204+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 shutdown_connections 2024-04-16T05:25:41.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.204+0000 7f4e67b6d700 1 -- 172.21.15.86:0/2936736766 wait complete. 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.276+0000 7ff78d3a1700 1 Processor -- start 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.276+0000 7ff78d3a1700 1 -- start start 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.276+0000 7ff78d3a1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff788108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.276+0000 7ff78d3a1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff788109070 con 0x7ff788108680 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.276+0000 7ff786ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff788108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.277+0000 7ff786ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff788108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53150/0 (socket says 172.21.15.86:53150) 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.277+0000 7ff786ffd700 1 -- 172.21.15.86:0/2885359950 learned_addr learned my addr 172.21.15.86:0/2885359950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.277+0000 7ff786ffd700 1 -- 172.21.15.86:0/2885359950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff788109830 con 0x7ff788108680 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.277+0000 7ff786ffd700 1 --2- 172.21.15.86:0/2885359950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff788108aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7780098a0 tx=0x7ff778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a908a4ae48e20a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.277+0000 7ff7867fc700 1 -- 172.21.15.86:0/2885359950 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff778004160 con 0x7ff788108680 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.277+0000 7ff7867fc700 1 -- 172.21.15.86:0/2885359950 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff778004300 con 0x7ff788108680 2024-04-16T05:25:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.277+0000 7ff7867fc700 1 -- 172.21.15.86:0/2885359950 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff778011b20 con 0x7ff788108680 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.278+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2885359950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 msgr2=0x7ff788108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.278+0000 7ff78d3a1700 1 --2- 172.21.15.86:0/2885359950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff788108aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7780098a0 tx=0x7ff778009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.278+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2885359950 shutdown_connections 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.278+0000 7ff78d3a1700 1 --2- 172.21.15.86:0/2885359950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff788108aa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.278+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2885359950 >> 172.21.15.86:0/2885359950 conn(0x7ff788103d40 msgr2=0x7ff788106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.278+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2885359950 shutdown_connections 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.278+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2885359950 wait complete. 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.278+0000 7ff78d3a1700 1 Processor -- start 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.278+0000 7ff78d3a1700 1 -- start start 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff78d3a1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff78807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff78d3a1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff78810cbe0 con 0x7ff788108680 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff786ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff78807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff786ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff78807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53166/0 (socket says 172.21.15.86:53166) 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff786ffd700 1 -- 172.21.15.86:0/2913569275 learned_addr learned my addr 172.21.15.86:0/2913569275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:43.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff786ffd700 1 -- 172.21.15.86:0/2913569275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78807e0d0 con 0x7ff788108680 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff786ffd700 1 --2- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff78807db90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff778000c00 tx=0x7ff778011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff784ff9700 1 -- 172.21.15.86:0/2913569275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff778003b60 con 0x7ff788108680 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78807c2c0 con 0x7ff788108680 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff784ff9700 1 -- 172.21.15.86:0/2913569275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff77801a910 con 0x7ff788108680 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff784ff9700 1 -- 172.21.15.86:0/2913569275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff778024930 con 0x7ff788108680 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.279+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78807c780 con 0x7ff788108680 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.280+0000 7ff784ff9700 1 -- 172.21.15.86:0/2913569275 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff778022020 con 0x7ff788108680 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.280+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff788064150 con 0x7ff788108680 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.280+0000 7ff784ff9700 1 --2- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff7740395b0 0x7ff77403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.281+0000 7ff784ff9700 1 -- 172.21.15.86:0/2913569275 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff77804d900 con 0x7ff788108680 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.281+0000 7ff77e7fc700 1 --2- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff7740395b0 0x7ff77403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:43.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.281+0000 7ff77e7fc700 1 --2- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff7740395b0 0x7ff77403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff770006fd0 tx=0x7ff770006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.285+0000 7ff784ff9700 1 -- 172.21.15.86:0/2913569275 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff77800f2e0 con 0x7ff788108680 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.488+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff78807ca00 con 0x7ff788108680 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.491+0000 7ff784ff9700 1 -- 172.21.15.86:0/2913569275 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff77802d3f0 con 0x7ff788108680 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff7740395b0 msgr2=0x7ff77403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 --2- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff7740395b0 0x7ff77403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff770006fd0 tx=0x7ff770006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 msgr2=0x7ff78807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 --2- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff78807db90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff778000c00 tx=0x7ff778011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 shutdown_connections 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 --2- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7ff7740395b0 0x7ff77403ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 --2- 172.21.15.86:0/2913569275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff788108680 0x7ff78807db90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 >> 172.21.15.86:0/2913569275 conn(0x7ff788103d40 msgr2=0x7ff788104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 shutdown_connections 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:42.494+0000 7ff78d3a1700 1 -- 172.21.15.86:0/2913569275 wait complete. 2024-04-16T05:25:43.583 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling the dashboard module... 2024-04-16T05:25:43.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:43 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2913569275' entity='client.admin' 2024-04-16T05:25:43.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:43 smithi086 ceph-mon[32028]: from='mgr.14116 172.21.15.86:0/1140244079' entity='mgr.x' 2024-04-16T05:25:44.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:44 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3230712383' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-16T05:25:44.786 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:44 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: ignoring --setuser ceph since I am not root 2024-04-16T05:25:44.786 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:44 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: ignoring --setgroup ceph since I am not root 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.037+0000 7f8365e1e700 1 Processor -- start 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.037+0000 7f8365e1e700 1 -- start start 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f8365e1e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836010b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f8365e1e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f836007beb0 con 0x7f8360109110 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f835f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836010b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f835f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836010b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53172/0 (socket says 172.21.15.86:53172) 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f835f7fe700 1 -- 172.21.15.86:0/77488646 learned_addr learned my addr 172.21.15.86:0/77488646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f835f7fe700 1 -- 172.21.15.86:0/77488646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836010ba40 con 0x7f8360109110 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f835f7fe700 1 --2- 172.21.15.86:0/77488646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836010b500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83500098a0 tx=0x7f8350009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a2427bb4f0156b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f835e7fc700 1 -- 172.21.15.86:0/77488646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8350004160 con 0x7f8360109110 2024-04-16T05:25:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f835e7fc700 1 -- 172.21.15.86:0/77488646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8350004300 con 0x7f8360109110 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.038+0000 7f835e7fc700 1 -- 172.21.15.86:0/77488646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8350011b20 con 0x7f8360109110 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.039+0000 7f8365e1e700 1 -- 172.21.15.86:0/77488646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 msgr2=0x7f836010b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.039+0000 7f8365e1e700 1 --2- 172.21.15.86:0/77488646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836010b500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83500098a0 tx=0x7f8350009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.039+0000 7f8365e1e700 1 -- 172.21.15.86:0/77488646 shutdown_connections 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.039+0000 7f8365e1e700 1 --2- 172.21.15.86:0/77488646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836010b500 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.039+0000 7f8365e1e700 1 -- 172.21.15.86:0/77488646 >> 172.21.15.86:0/77488646 conn(0x7f8360102b80 msgr2=0x7f8360104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.039+0000 7f8365e1e700 1 -- 172.21.15.86:0/77488646 shutdown_connections 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.039+0000 7f8365e1e700 1 -- 172.21.15.86:0/77488646 wait complete. 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.040+0000 7f8365e1e700 1 Processor -- start 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.040+0000 7f8365e1e700 1 -- start start 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.040+0000 7f8365e1e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.040+0000 7f8365e1e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f836010ca40 con 0x7f8360109110 2024-04-16T05:25:45.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.040+0000 7f835f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.040+0000 7f835f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836019c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53186/0 (socket says 172.21.15.86:53186) 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.040+0000 7f835f7fe700 1 -- 172.21.15.86:0/3230712383 learned_addr learned my addr 172.21.15.86:0/3230712383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.041+0000 7f835f7fe700 1 -- 172.21.15.86:0/3230712383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836019cb10 con 0x7f8360109110 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.041+0000 7f835f7fe700 1 --2- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836019c5d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8350011b60 tx=0x7f8350011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.041+0000 7f835cff9700 1 -- 172.21.15.86:0/3230712383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8350003b60 con 0x7f8360109110 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.041+0000 7f835cff9700 1 -- 172.21.15.86:0/3230712383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f835001a910 con 0x7f8360109110 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.041+0000 7f835cff9700 1 -- 172.21.15.86:0/3230712383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8350024930 con 0x7f8360109110 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.041+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836019ce00 con 0x7f8360109110 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.041+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836019fa90 con 0x7f8360109110 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.042+0000 7f835cff9700 1 -- 172.21.15.86:0/3230712383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8350022020 con 0x7f8360109110 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.042+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8360064150 con 0x7f8360109110 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.042+0000 7f835cff9700 1 --2- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f8348039600 0x7f834803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:45.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.042+0000 7f835cff9700 1 -- 172.21.15.86:0/3230712383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f835004d840 con 0x7f8360109110 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.042+0000 7f835effd700 1 --2- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f8348039600 0x7f834803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.043+0000 7f835effd700 1 --2- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f8348039600 0x7f834803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8354006fd0 tx=0x7f8354006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.047+0000 7f835cff9700 1 -- 172.21.15.86:0/3230712383 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f835000b3e0 con 0x7f8360109110 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.323+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f836019fc60 con 0x7f8360109110 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.494+0000 7f835cff9700 1 -- 172.21.15.86:0/3230712383 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8350019de0 con 0x7f8360109110 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.496+0000 7f835cff9700 1 -- 172.21.15.86:0/3230712383 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f8350024ad0 con 0x7f8360109110 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f8348039600 msgr2=0x7f834803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 --2- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f8348039600 0x7f834803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8354006fd0 tx=0x7f8354006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 msgr2=0x7f836019c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 --2- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836019c5d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8350011b60 tx=0x7f8350011c40 comp rx=0 tx=0).stop 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 shutdown_connections 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 --2- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f8348039600 0x7f834803bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:45.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 --2- 172.21.15.86:0/3230712383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8360109110 0x7f836019c5d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:45.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 >> 172.21.15.86:0/3230712383 conn(0x7f8360102b80 msgr2=0x7f83601036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:45.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 shutdown_connections 2024-04-16T05:25:45.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:44.498+0000 7f8365e1e700 1 -- 172.21.15.86:0/3230712383 wait complete. 2024-04-16T05:25:45.354 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:45 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:45.109+0000 7ffb8f22f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:25:45.355 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:45 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:45.204+0000 7ffb8f22f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:25:45.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:45 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3230712383' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-16T05:25:45.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:45 smithi086 ceph-mon[32028]: mgrmap e9: x(active, since 19s) 2024-04-16T05:25:45.926 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:45 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:45.873+0000 7ffb8f22f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:25:46.211 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:45 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:45.972+0000 7ffb8f22f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:25:46.212 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:46.113+0000 7ffb8f22f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:25:46.212 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:46.212+0000 7ffb8f22f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:25:46.468 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:46.296+0000 7ffb8f22f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:25:46.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.653+0000 7fabdb917700 1 Processor -- start 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.653+0000 7fabdb917700 1 -- start start 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabdb917700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd4106d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabdb917700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4107350 con 0x7fabd4106960 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabd96b3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd4106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabd96b3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd4106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53212/0 (socket says 172.21.15.86:53212) 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabd96b3700 1 -- 172.21.15.86:0/3557450556 learned_addr learned my addr 172.21.15.86:0/3557450556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabd96b3700 1 -- 172.21.15.86:0/3557450556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd41074d0 con 0x7fabd4106960 2024-04-16T05:25:46.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabd96b3700 1 --2- 172.21.15.86:0/3557450556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd4106d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fabd00098a0 tx=0x7fabd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2fe968604d6307 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabcbfff700 1 -- 172.21.15.86:0/3557450556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabd0004160 con 0x7fabd4106960 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabcbfff700 1 -- 172.21.15.86:0/3557450556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd0004300 con 0x7fabd4106960 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.654+0000 7fabcbfff700 1 -- 172.21.15.86:0/3557450556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabd0011b20 con 0x7fabd4106960 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.655+0000 7fabdb917700 1 -- 172.21.15.86:0/3557450556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 msgr2=0x7fabd4106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.655+0000 7fabdb917700 1 --2- 172.21.15.86:0/3557450556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd4106d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fabd00098a0 tx=0x7fabd0009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.655+0000 7fabdb917700 1 -- 172.21.15.86:0/3557450556 shutdown_connections 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.655+0000 7fabdb917700 1 --2- 172.21.15.86:0/3557450556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd4106d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.655+0000 7fabdb917700 1 -- 172.21.15.86:0/3557450556 >> 172.21.15.86:0/3557450556 conn(0x7fabd4102360 msgr2=0x7fabd4104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.655+0000 7fabdb917700 1 -- 172.21.15.86:0/3557450556 shutdown_connections 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.655+0000 7fabdb917700 1 -- 172.21.15.86:0/3557450556 wait complete. 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.656+0000 7fabdb917700 1 Processor -- start 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.656+0000 7fabdb917700 1 -- start start 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.656+0000 7fabdb917700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:46.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.656+0000 7fabdb917700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabd410cc10 con 0x7fabd4106960 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.656+0000 7fabd96b3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.656+0000 7fabd96b3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53224/0 (socket says 172.21.15.86:53224) 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.656+0000 7fabd96b3700 1 -- 172.21.15.86:0/531763241 learned_addr learned my addr 172.21.15.86:0/531763241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.656+0000 7fabd96b3700 1 -- 172.21.15.86:0/531763241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd419cb80 con 0x7fabd4106960 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.657+0000 7fabd96b3700 1 --2- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd419c640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fabd0000c00 tx=0x7fabd0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.657+0000 7fabca7fc700 1 -- 172.21.15.86:0/531763241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabd0003b60 con 0x7fabd4106960 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.657+0000 7fabca7fc700 1 -- 172.21.15.86:0/531763241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd001a910 con 0x7fabd4106960 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.657+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd419ce70 con 0x7fabd4106960 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.657+0000 7fabca7fc700 1 -- 172.21.15.86:0/531763241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabd0024930 con 0x7fabd4106960 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.657+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd419fb00 con 0x7fabd4106960 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.658+0000 7fabca7fc700 1 -- 172.21.15.86:0/531763241 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fabd0022020 con 0x7fabd4106960 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.658+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd4064150 con 0x7fabd4106960 2024-04-16T05:25:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.658+0000 7fabca7fc700 1 --2- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fabc0039660 0x7fabc003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.658+0000 7fabca7fc700 1 -- 172.21.15.86:0/531763241 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fabd004e820 con 0x7fabd4106960 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.658+0000 7fabd8eb2700 1 -- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fabc0039660 msgr2=0x7fabc003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1420225001 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.658+0000 7fabd8eb2700 1 --2- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fabc0039660 0x7fabc003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.663+0000 7fabca7fc700 1 -- 172.21.15.86:0/531763241 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabd002ad60 con 0x7fabd4106960 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.859+0000 7fabd8eb2700 1 -- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fabc0039660 msgr2=0x7fabc003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1420225001 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.859+0000 7fabd8eb2700 1 --2- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fabc0039660 0x7fabc003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.926+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fabd419fcd0 con 0x7fabd4106960 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.927+0000 7fabca7fc700 1 -- 172.21.15.86:0/531763241 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fabd002dc90 con 0x7fabd4106960 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fabc0039660 msgr2=0x7fabc003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 --2- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fabc0039660 0x7fabc003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 msgr2=0x7fabd419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 --2- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd419c640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fabd0000c00 tx=0x7fabd0011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:46.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 shutdown_connections 2024-04-16T05:25:46.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 --2- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7fabc0039660 0x7fabc003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:46.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 --2- 172.21.15.86:0/531763241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4106960 0x7fabd419c640 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:46.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 >> 172.21.15.86:0/531763241 conn(0x7fabd4102360 msgr2=0x7fabd4102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:46.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 shutdown_connections 2024-04-16T05:25:46.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:45.929+0000 7fabdb917700 1 -- 172.21.15.86:0/531763241 wait complete. 2024-04-16T05:25:46.548 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-04-16T05:25:46.548 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 9... 2024-04-16T05:25:46.765 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:46 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/531763241' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-16T05:25:46.765 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:46.565+0000 7ffb8f22f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:25:46.765 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:46.678+0000 7ffb8f22f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:25:46.765 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:46.765+0000 7ffb8f22f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:25:47.048 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:46.864+0000 7ffb8f22f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:25:47.467 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:47 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:47.048+0000 7ffb8f22f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:25:48.393 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:48.004+0000 7ffb8f22f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:25:48.393 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:48.132+0000 7ffb8f22f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:25:48.717 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:48.462+0000 7ffb8f22f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:25:48.717 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:48.549+0000 7ffb8f22f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:25:49.210 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:48.743+0000 7ffb8f22f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:49.210 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:48.926+0000 7ffb8f22f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:25:49.467 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:49 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:49.211+0000 7ffb8f22f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:49.967 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:49 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:49.470+0000 7ffb8f22f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:25:49.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:49 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:49.708+0000 7ffb8f22f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:25:50.967 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:50.513+0000 7ffb8f22f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:25:51.391 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:51.077+0000 7ffb8f22f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:25:51.391 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:51.175+0000 7ffb8f22f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:25:51.391 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:25:51.392+0000 7ffb8f22f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:25:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: Active manager daemon x restarted 2024-04-16T05:25:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: Activating manager daemon x 2024-04-16T05:25:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: osdmap e3: 0 total, 0 up, 0 in 2024-04-16T05:25:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: mgrmap e10: x(active, starting, since 0.00334009s) 2024-04-16T05:25:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:25:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:25:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:25:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:25:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:25:51.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: Manager daemon x is now available 2024-04-16T05:25:51.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:51.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:25:52.569 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:25:52.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:52 smithi086 ceph-mon[32028]: mgrmap e11: x(active, since 1.00582s) 2024-04-16T05:25:53.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.985+0000 7f7c4f660700 1 Processor -- start 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.985+0000 7f7c4f660700 1 -- start start 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.985+0000 7f7c4f660700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c4810b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.986+0000 7f7c4f660700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4807be50 con 0x7f7c48109170 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.986+0000 7f7c4d3fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c4810b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.986+0000 7f7c4d3fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c4810b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53236/0 (socket says 172.21.15.86:53236) 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.986+0000 7f7c4d3fc700 1 -- 172.21.15.86:0/3379622129 learned_addr learned my addr 172.21.15.86:0/3379622129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.986+0000 7f7c4d3fc700 1 -- 172.21.15.86:0/3379622129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4810baa0 con 0x7f7c48109170 2024-04-16T05:25:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.986+0000 7f7c4d3fc700 1 --2- 172.21.15.86:0/3379622129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c4810b560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c380098a0 tx=0x7f7c38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78f852a40a4ab63 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.986+0000 7f7c3ffff700 1 -- 172.21.15.86:0/3379622129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c38004160 con 0x7f7c48109170 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.986+0000 7f7c3ffff700 1 -- 172.21.15.86:0/3379622129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c38004300 con 0x7f7c48109170 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.986+0000 7f7c3ffff700 1 -- 172.21.15.86:0/3379622129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c38011b20 con 0x7f7c48109170 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.987+0000 7f7c4f660700 1 -- 172.21.15.86:0/3379622129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 msgr2=0x7f7c4810b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.987+0000 7f7c4f660700 1 --2- 172.21.15.86:0/3379622129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c4810b560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c380098a0 tx=0x7f7c38009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.987+0000 7f7c4f660700 1 -- 172.21.15.86:0/3379622129 shutdown_connections 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.987+0000 7f7c4f660700 1 --2- 172.21.15.86:0/3379622129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c4810b560 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.987+0000 7f7c4f660700 1 -- 172.21.15.86:0/3379622129 >> 172.21.15.86:0/3379622129 conn(0x7f7c48102b80 msgr2=0x7f7c48104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.987+0000 7f7c4f660700 1 -- 172.21.15.86:0/3379622129 shutdown_connections 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.987+0000 7f7c4f660700 1 -- 172.21.15.86:0/3379622129 wait complete. 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.988+0000 7f7c4f660700 1 Processor -- start 2024-04-16T05:25:53.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.988+0000 7f7c4f660700 1 -- start start 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.988+0000 7f7c4f660700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c481089f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.988+0000 7f7c4f660700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4810caa0 con 0x7f7c48109170 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.988+0000 7f7c4d3fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c481089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.988+0000 7f7c4d3fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c481089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53242/0 (socket says 172.21.15.86:53242) 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.988+0000 7f7c4d3fc700 1 -- 172.21.15.86:0/743951300 learned_addr learned my addr 172.21.15.86:0/743951300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.989+0000 7f7c4d3fc700 1 -- 172.21.15.86:0/743951300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c48108f30 con 0x7f7c48109170 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.989+0000 7f7c4d3fc700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c481089f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c380116a0 tx=0x7f7c38011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.989+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c38003a40 con 0x7f7c48109170 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.989+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c38003be0 con 0x7f7c48109170 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.989+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c48107120 con 0x7f7c48109170 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.989+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c3801a750 con 0x7f7c48109170 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.989+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c481075e0 con 0x7f7c48109170 2024-04-16T05:25:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.990+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7c3801aa70 con 0x7f7c48109170 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.990+0000 7f7c3e7fc700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 0x7f7c3403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.990+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 --> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c48064150 con 0x7f7c340395b0 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.990+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c3804ed10 con 0x7f7c48109170 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.990+0000 7f7c4cbfb700 1 -- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 msgr2=0x7f7c3403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1420225001 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:46.990+0000 7f7c4cbfb700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 0x7f7c3403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:47.191+0000 7f7c4cbfb700 1 -- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 msgr2=0x7f7c3403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1420225001 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:47.192+0000 7f7c4cbfb700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 0x7f7c3403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:47.593+0000 7f7c4cbfb700 1 -- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 msgr2=0x7f7c3403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1420225001 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:47.593+0000 7f7c4cbfb700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 0x7f7c3403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:48.394+0000 7f7c4cbfb700 1 -- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 msgr2=0x7f7c3403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1420225001 2024-04-16T05:25:53.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:48.394+0000 7f7c4cbfb700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 0x7f7c3403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:49.995+0000 7f7c4cbfb700 1 -- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 msgr2=0x7f7c3403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1420225001 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:49.995+0000 7f7c4cbfb700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 0x7f7c3403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:51.395+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f7c3802d910 con 0x7f7c48109170 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:51.396+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 msgr2=0x7f7c3403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:51.396+0000 7f7c3e7fc700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 0x7f7c3403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.397+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f7c3804fc10 con 0x7f7c48109170 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.397+0000 7f7c3e7fc700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7c3403ce40 0x7f7c3403f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.397+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c3802ee50 con 0x7f7c3403ce40 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.398+0000 7f7c4cbfb700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7c3403ce40 0x7f7c3403f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.398+0000 7f7c4cbfb700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7c3403ce40 0x7f7c3403f230 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c44003a10 tx=0x7f7c440092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.399+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7c3802ee50 con 0x7f7c3403ce40 2024-04-16T05:25:53.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.405+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7c48103d90 con 0x7f7c3403ce40 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.405+0000 7f7c3e7fc700 1 -- 172.21.15.86:0/743951300 <== mgr.14148 v2:172.21.15.86:6800/3194681946 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f7c48103d90 con 0x7f7c3403ce40 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7c3403ce40 msgr2=0x7f7c3403f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7c3403ce40 0x7f7c3403f230 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c44003a10 tx=0x7f7c440092b0 comp rx=0 tx=0).stop 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 msgr2=0x7f7c481089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c481089f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c380116a0 tx=0x7f7c38011ba0 comp rx=0 tx=0).stop 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 shutdown_connections 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c48109170 0x7f7c481089f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7c3403ce40 0x7f7c3403f230 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 --2- 172.21.15.86:0/743951300 >> [v2:172.21.15.86:6800/1420225001,v1:172.21.15.86:6801/1420225001] conn(0x7f7c340395b0 0x7f7c3403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 >> 172.21.15.86:0/743951300 conn(0x7f7c48102b80 msgr2=0x7f7c48103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 shutdown_connections 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:52.406+0000 7f7c4f660700 1 -- 172.21.15.86:0/743951300 wait complete. 2024-04-16T05:25:53.244 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 9 is available 2024-04-16T05:25:53.245 INFO:teuthology.orchestra.run.smithi086.stdout:Generating a dashboard self-signed certificate... 2024-04-16T05:25:53.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:53 smithi086 ceph-mon[32028]: [16/Apr/2024:05:25:51] ENGINE Bus STARTING 2024-04-16T05:25:53.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:53 smithi086 ceph-mon[32028]: [16/Apr/2024:05:25:51] ENGINE Serving on https://172.21.15.86:7150 2024-04-16T05:25:53.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:53 smithi086 ceph-mon[32028]: [16/Apr/2024:05:25:51] ENGINE Serving on http://172.21.15.86:8765 2024-04-16T05:25:53.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:53 smithi086 ceph-mon[32028]: [16/Apr/2024:05:25:51] ENGINE Bus STARTED 2024-04-16T05:25:53.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:53 smithi086 ceph-mon[32028]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-16T05:25:53.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:53 smithi086 ceph-mon[32028]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-16T05:25:54.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:54 smithi086 ceph-mon[32028]: mgrmap e12: x(active, since 2s) 2024-04-16T05:25:54.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:25:54.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:25:54.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:25:54.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-16T05:25:54.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.653+0000 7fb4ea126700 1 Processor -- start 2024-04-16T05:25:54.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.653+0000 7fb4ea126700 1 -- start start 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.653+0000 7fb4ea126700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e4108b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.653+0000 7fb4ea126700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e4109120 con 0x7fb4e4108730 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.653+0000 7fb4e37fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e4108b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.653+0000 7fb4e37fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e4108b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54158/0 (socket says 172.21.15.86:54158) 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.653+0000 7fb4e37fe700 1 -- 172.21.15.86:0/705596729 learned_addr learned my addr 172.21.15.86:0/705596729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4e37fe700 1 -- 172.21.15.86:0/705596729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e41098e0 con 0x7fb4e4108730 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4e37fe700 1 --2- 172.21.15.86:0/705596729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e4108b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40098a0 tx=0x7fb4d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d745a550dde4a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4e27fc700 1 -- 172.21.15.86:0/705596729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d4004160 con 0x7fb4e4108730 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4e27fc700 1 -- 172.21.15.86:0/705596729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d4004300 con 0x7fb4e4108730 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4e27fc700 1 -- 172.21.15.86:0/705596729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d4011b20 con 0x7fb4e4108730 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4ea126700 1 -- 172.21.15.86:0/705596729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 msgr2=0x7fb4e4108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:54.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4ea126700 1 --2- 172.21.15.86:0/705596729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e4108b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40098a0 tx=0x7fb4d4009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4ea126700 1 -- 172.21.15.86:0/705596729 shutdown_connections 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4ea126700 1 --2- 172.21.15.86:0/705596729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e4108b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.654+0000 7fb4ea126700 1 -- 172.21.15.86:0/705596729 >> 172.21.15.86:0/705596729 conn(0x7fb4e4103dd0 msgr2=0x7fb4e41061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.655+0000 7fb4ea126700 1 -- 172.21.15.86:0/705596729 shutdown_connections 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.655+0000 7fb4ea126700 1 -- 172.21.15.86:0/705596729 wait complete. 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.655+0000 7fb4ea126700 1 Processor -- start 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.655+0000 7fb4ea126700 1 -- start start 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.655+0000 7fb4ea126700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.655+0000 7fb4ea126700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e410cc90 con 0x7fb4e4108730 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4e37fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4e37fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e419c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54174/0 (socket says 172.21.15.86:54174) 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4e37fe700 1 -- 172.21.15.86:0/2528301293 learned_addr learned my addr 172.21.15.86:0/2528301293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4e37fe700 1 -- 172.21.15.86:0/2528301293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e419cb10 con 0x7fb4e4108730 2024-04-16T05:25:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4e37fe700 1 --2- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e419c5d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40117e0 tx=0x7fb4d4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4e0ff9700 1 -- 172.21.15.86:0/2528301293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d4003bf0 con 0x7fb4e4108730 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e419cda0 con 0x7fb4e4108730 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4e0ff9700 1 -- 172.21.15.86:0/2528301293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d401a910 con 0x7fb4e4108730 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4e0ff9700 1 -- 172.21.15.86:0/2528301293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d40249a0 con 0x7fb4e4108730 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.656+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e419d200 con 0x7fb4e4108730 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.657+0000 7fb4e0ff9700 1 -- 172.21.15.86:0/2528301293 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fb4d4022020 con 0x7fb4e4108730 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.657+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e4064150 con 0x7fb4e4108730 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.657+0000 7fb4e0ff9700 1 --2- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb4cc0395b0 0x7fb4cc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.657+0000 7fb4e0ff9700 1 -- 172.21.15.86:0/2528301293 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb4d404dcc0 con 0x7fb4e4108730 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.657+0000 7fb4e2ffd700 1 --2- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb4cc0395b0 0x7fb4cc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.658+0000 7fb4e2ffd700 1 --2- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb4cc0395b0 0x7fb4cc03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8006fd0 tx=0x7fb4d8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.662+0000 7fb4e0ff9700 1 -- 172.21.15.86:0/2528301293 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4d4011e20 con 0x7fb4e4108730 2024-04-16T05:25:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.887+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb4e4104f80 con 0x7fb4cc0395b0 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.975+0000 7fb4e0ff9700 1 -- 172.21.15.86:0/2528301293 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb4e4104f80 con 0x7fb4cc0395b0 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb4cc0395b0 msgr2=0x7fb4cc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 --2- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb4cc0395b0 0x7fb4cc03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8006fd0 tx=0x7fb4d8006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 msgr2=0x7fb4e419c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 --2- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e419c5d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40117e0 tx=0x7fb4d4011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 shutdown_connections 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 --2- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb4cc0395b0 0x7fb4cc03ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 --2- 172.21.15.86:0/2528301293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4108730 0x7fb4e419c5d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 >> 172.21.15.86:0/2528301293 conn(0x7fb4e4103dd0 msgr2=0x7fb4e4104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 shutdown_connections 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:53.978+0000 7fb4ea126700 1 -- 172.21.15.86:0/2528301293 wait complete. 2024-04-16T05:25:54.646 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial admin user... 2024-04-16T05:25:55.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:55 smithi086 ceph-mon[32028]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:55.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:55 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$7C8KGjPOvA29jQ4Qxm.LKevJMoFkLaP/1uf5Ye1kFkcxpV5GDAq0i", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713245155, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.113+0000 7f1cc7a08700 1 Processor -- start 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.113+0000 7f1cc7a08700 1 -- start start 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.113+0000 7f1cc7a08700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.113+0000 7f1cc7a08700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc007c860 con 0x7f1cc007de20 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.113+0000 7f1cc57a4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.113+0000 7f1cc57a4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54180/0 (socket says 172.21.15.86:54180) 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.113+0000 7f1cc57a4700 1 -- 172.21.15.86:0/4152776271 learned_addr learned my addr 172.21.15.86:0/4152776271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.114+0000 7f1cc57a4700 1 -- 172.21.15.86:0/4152776271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc007c9e0 con 0x7f1cc007de20 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.114+0000 7f1cc57a4700 1 --2- 172.21.15.86:0/4152776271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc007c320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0098a0 tx=0x7f1cbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdf6fb204bf0fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.114+0000 7f1cb7fff700 1 -- 172.21.15.86:0/4152776271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cbc004160 con 0x7f1cc007de20 2024-04-16T05:25:56.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.114+0000 7f1cb7fff700 1 -- 172.21.15.86:0/4152776271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc004300 con 0x7f1cc007de20 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.114+0000 7f1cb7fff700 1 -- 172.21.15.86:0/4152776271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cbc011b20 con 0x7f1cc007de20 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.115+0000 7f1cc7a08700 1 -- 172.21.15.86:0/4152776271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 msgr2=0x7f1cc007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.115+0000 7f1cc7a08700 1 --2- 172.21.15.86:0/4152776271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc007c320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0098a0 tx=0x7f1cbc009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.115+0000 7f1cc7a08700 1 -- 172.21.15.86:0/4152776271 shutdown_connections 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.115+0000 7f1cc7a08700 1 --2- 172.21.15.86:0/4152776271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc007c320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.115+0000 7f1cc7a08700 1 -- 172.21.15.86:0/4152776271 >> 172.21.15.86:0/4152776271 conn(0x7f1cc0103d40 msgr2=0x7f1cc0106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.115+0000 7f1cc7a08700 1 -- 172.21.15.86:0/4152776271 shutdown_connections 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.115+0000 7f1cc7a08700 1 -- 172.21.15.86:0/4152776271 wait complete. 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.115+0000 7f1cc7a08700 1 Processor -- start 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.115+0000 7f1cc7a08700 1 -- start start 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cc7a08700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cc7a08700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010ec80 con 0x7f1cc007de20 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cc57a4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:56.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cc57a4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54194/0 (socket says 172.21.15.86:54194) 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cc57a4700 1 -- 172.21.15.86:0/1918085314 learned_addr learned my addr 172.21.15.86:0/1918085314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cc57a4700 1 -- 172.21.15.86:0/1918085314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc019ed00 con 0x7f1cc007de20 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cc57a4700 1 --2- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc019e7c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0117e0 tx=0x7f1cbc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cb67fc700 1 -- 172.21.15.86:0/1918085314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cbc003bf0 con 0x7f1cc007de20 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cb67fc700 1 -- 172.21.15.86:0/1918085314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc01a910 con 0x7f1cc007de20 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc019ef90 con 0x7f1cc007de20 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cb67fc700 1 -- 172.21.15.86:0/1918085314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cbc024970 con 0x7f1cc007de20 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.116+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc019f3f0 con 0x7f1cc007de20 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.117+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc0064150 con 0x7f1cc007de20 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.118+0000 7f1cb67fc700 1 -- 172.21.15.86:0/1918085314 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f1cbc022020 con 0x7f1cc007de20 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.118+0000 7f1cb67fc700 1 --2- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1cac039610 0x7f1cac03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.118+0000 7f1cb67fc700 1 -- 172.21.15.86:0/1918085314 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1cbc04db30 con 0x7f1cc007de20 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.118+0000 7f1cc4fa3700 1 --2- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1cac039610 0x7f1cac03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:56.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.118+0000 7f1cc4fa3700 1 --2- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1cac039610 0x7f1cac03bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.123+0000 7f1cb67fc700 1 -- 172.21.15.86:0/1918085314 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1cbc011e20 con 0x7f1cc007de20 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.346+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1cc01a1da0 con 0x7f1cac039610 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.572+0000 7f1cb67fc700 1 -- 172.21.15.86:0/1918085314 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1cc01a1da0 con 0x7f1cac039610 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.574+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1cac039610 msgr2=0x7f1cac03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.574+0000 7f1cc7a08700 1 --2- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1cac039610 0x7f1cac03bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.574+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 msgr2=0x7f1cc019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.574+0000 7f1cc7a08700 1 --2- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc019e7c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0117e0 tx=0x7f1cbc011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.574+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 shutdown_connections 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.574+0000 7f1cc7a08700 1 --2- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1cac039610 0x7f1cac03bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.574+0000 7f1cc7a08700 1 --2- 172.21.15.86:0/1918085314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cc007de20 0x7f1cc019e7c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.574+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 >> 172.21.15.86:0/1918085314 conn(0x7f1cc0103d40 msgr2=0x7f1cc0104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.575+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 shutdown_connections 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:55.575+0000 7f1cc7a08700 1 -- 172.21.15.86:0/1918085314 wait complete. 2024-04-16T05:25:56.205 INFO:teuthology.orchestra.run.smithi086.stdout:Fetching dashboard port number... 2024-04-16T05:25:56.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:56 smithi086 ceph-mon[32028]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:56.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:25:58.221 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 8443 2024-04-16T05:25:58.221 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.423+0000 7fa3b7eb7700 1 Processor -- start 2024-04-16T05:25:58.221 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.423+0000 7fa3b7eb7700 1 -- start start 2024-04-16T05:25:58.221 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.423+0000 7fa3b7eb7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:58.221 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.423+0000 7fa3b7eb7700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b007c910 con 0x7fa3b007de90 2024-04-16T05:25:58.221 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.424+0000 7fa3b5c53700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.424+0000 7fa3b5c53700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54208/0 (socket says 172.21.15.86:54208) 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.424+0000 7fa3b5c53700 1 -- 172.21.15.86:0/2213413693 learned_addr learned my addr 172.21.15.86:0/2213413693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.424+0000 7fa3b5c53700 1 -- 172.21.15.86:0/2213413693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b007ca90 con 0x7fa3b007de90 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.424+0000 7fa3b5c53700 1 --2- 172.21.15.86:0/2213413693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b007c340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0098a0 tx=0x7fa3ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c98fe93aeb82ce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.424+0000 7fa3b4c51700 1 -- 172.21.15.86:0/2213413693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3ac004160 con 0x7fa3b007de90 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.424+0000 7fa3b4c51700 1 -- 172.21.15.86:0/2213413693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3ac004300 con 0x7fa3b007de90 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.424+0000 7fa3b4c51700 1 -- 172.21.15.86:0/2213413693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3ac011b20 con 0x7fa3b007de90 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.425+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2213413693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 msgr2=0x7fa3b007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.425+0000 7fa3b7eb7700 1 --2- 172.21.15.86:0/2213413693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b007c340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0098a0 tx=0x7fa3ac009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.425+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2213413693 shutdown_connections 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.425+0000 7fa3b7eb7700 1 --2- 172.21.15.86:0/2213413693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b007c340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.425+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2213413693 >> 172.21.15.86:0/2213413693 conn(0x7fa3b0103dd0 msgr2=0x7fa3b01061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:58.222 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.425+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2213413693 shutdown_connections 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.425+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2213413693 wait complete. 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.425+0000 7fa3b7eb7700 1 Processor -- start 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.425+0000 7fa3b7eb7700 1 -- start start 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3b7eb7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3b7eb7700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b010ed10 con 0x7fa3b007de90 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3b5c53700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3b5c53700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54216/0 (socket says 172.21.15.86:54216) 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3b5c53700 1 -- 172.21.15.86:0/2826267862 learned_addr learned my addr 172.21.15.86:0/2826267862 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3b5c53700 1 -- 172.21.15.86:0/2826267862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b019ece0 con 0x7fa3b007de90 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3b5c53700 1 --2- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b019e7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0117e0 tx=0x7fa3ac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3a6ffd700 1 -- 172.21.15.86:0/2826267862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3ac003ba0 con 0x7fa3b007de90 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3a6ffd700 1 -- 172.21.15.86:0/2826267862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3ac01a910 con 0x7fa3b007de90 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b019ef70 con 0x7fa3b007de90 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3a6ffd700 1 -- 172.21.15.86:0/2826267862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3ac0248f0 con 0x7fa3b007de90 2024-04-16T05:25:58.223 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.426+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b019f3d0 con 0x7fa3b007de90 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.427+0000 7fa3a6ffd700 1 -- 172.21.15.86:0/2826267862 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa3ac022020 con 0x7fa3b007de90 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.427+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b0064150 con 0x7fa3b007de90 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.427+0000 7fa3a6ffd700 1 --2- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa39c039660 0x7fa39c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.427+0000 7fa3a6ffd700 1 -- 172.21.15.86:0/2826267862 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa3ac04dd70 con 0x7fa3b007de90 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.428+0000 7fa3b5452700 1 --2- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa39c039660 0x7fa39c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.428+0000 7fa3b5452700 1 --2- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa39c039660 0x7fa39c03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006fd0 tx=0x7fa3a0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.433+0000 7fa3a6ffd700 1 -- 172.21.15.86:0/2826267862 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3ac052070 con 0x7fa3b007de90 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.462+0000 7fa3a6ffd700 1 -- 172.21.15.86:0/2826267862 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa3ac02dc40 con 0x7fa3b007de90 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.630+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa3b01a1da0 con 0x7fa3b007de90 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.631+0000 7fa3a6ffd700 1 -- 172.21.15.86:0/2826267862 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa3ac00b3e0 con 0x7fa3b007de90 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa39c039660 msgr2=0x7fa39c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 --2- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa39c039660 0x7fa39c03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006fd0 tx=0x7fa3a0006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 msgr2=0x7fa3b019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:58.224 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 --2- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b019e7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0117e0 tx=0x7fa3ac011ba0 comp rx=0 tx=0).stop 2024-04-16T05:25:58.225 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 shutdown_connections 2024-04-16T05:25:58.225 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 --2- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa39c039660 0x7fa39c03bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:58.225 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 --2- 172.21.15.86:0/2826267862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b007de90 0x7fa3b019e7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:58.225 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 >> 172.21.15.86:0/2826267862 conn(0x7fa3b0103dd0 msgr2=0x7fa3b0104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:58.225 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 shutdown_connections 2024-04-16T05:25:58.225 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:57.633+0000 7fa3b7eb7700 1 -- 172.21.15.86:0/2826267862 wait complete. 2024-04-16T05:25:58.229 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T05:25:58.229 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-16T05:25:58.237 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T05:25:58.238 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-16T05:25:58.238 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-16T05:25:58.238 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-16T05:25:58.240 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph Dashboard is now available at: 2024-04-16T05:25:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:25:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: URL: https://smithi086.front.sepia.ceph.com:8443/ 2024-04-16T05:25:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: User: admin 2024-04-16T05:25:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: Password: oapeyoo2yw 2024-04-16T05:25:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:25:58.240 INFO:teuthology.orchestra.run.smithi086.stdout:Saving cluster configuration to /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config directory 2024-04-16T05:25:58.241 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling autotune for osd_memory_target 2024-04-16T05:25:58.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:58 smithi086 ceph-mon[32028]: mgrmap e13: x(active, since 6s) 2024-04-16T05:25:58.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:25:58 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2826267862' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-16T05:25:59.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.662+0000 7fc6ad100700 1 Processor -- start 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.662+0000 7fc6ad100700 1 -- start start 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.662+0000 7fc6ad100700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a81079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.662+0000 7fc6ad100700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8107f90 con 0x7fc6a81075a0 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.662+0000 7fc6a6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a81079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.662+0000 7fc6a6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a81079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54226/0 (socket says 172.21.15.86:54226) 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.662+0000 7fc6a6d9d700 1 -- 172.21.15.86:0/246202623 learned_addr learned my addr 172.21.15.86:0/246202623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.662+0000 7fc6a6d9d700 1 -- 172.21.15.86:0/246202623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a81087a0 con 0x7fc6a81075a0 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.663+0000 7fc6a6d9d700 1 --2- 172.21.15.86:0/246202623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a81079c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098a0 tx=0x7fc698009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0161eeed844c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.663+0000 7fc6a5d9b700 1 -- 172.21.15.86:0/246202623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc698004160 con 0x7fc6a81075a0 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.663+0000 7fc6a5d9b700 1 -- 172.21.15.86:0/246202623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc698004300 con 0x7fc6a81075a0 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.663+0000 7fc6a5d9b700 1 -- 172.21.15.86:0/246202623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc698011b20 con 0x7fc6a81075a0 2024-04-16T05:25:59.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.664+0000 7fc6ad100700 1 -- 172.21.15.86:0/246202623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 msgr2=0x7fc6a81079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.664+0000 7fc6ad100700 1 --2- 172.21.15.86:0/246202623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a81079c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098a0 tx=0x7fc698009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.664+0000 7fc6ad100700 1 -- 172.21.15.86:0/246202623 shutdown_connections 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.664+0000 7fc6ad100700 1 --2- 172.21.15.86:0/246202623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a81079c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.664+0000 7fc6ad100700 1 -- 172.21.15.86:0/246202623 >> 172.21.15.86:0/246202623 conn(0x7fc6a8102c40 msgr2=0x7fc6a8105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.664+0000 7fc6ad100700 1 -- 172.21.15.86:0/246202623 shutdown_connections 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.664+0000 7fc6ad100700 1 -- 172.21.15.86:0/246202623 wait complete. 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.664+0000 7fc6ad100700 1 Processor -- start 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.665+0000 7fc6ad100700 1 -- start start 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.665+0000 7fc6ad100700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a807ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.665+0000 7fc6ad100700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a810bb50 con 0x7fc6a81075a0 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.665+0000 7fc6a6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a807ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.665+0000 7fc6a6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a807ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54232/0 (socket says 172.21.15.86:54232) 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.665+0000 7fc6a6d9d700 1 -- 172.21.15.86:0/1644900870 learned_addr learned my addr 172.21.15.86:0/1644900870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:25:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.665+0000 7fc6a6d9d700 1 -- 172.21.15.86:0/1644900870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a819f3c0 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.665+0000 7fc6a6d9d700 1 --2- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a807ba70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6980116a0 tx=0x7fc698011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.666+0000 7fc68ffff700 1 -- 172.21.15.86:0/1644900870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc698003a40 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.666+0000 7fc68ffff700 1 -- 172.21.15.86:0/1644900870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc698003be0 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.666+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a819f590 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.666+0000 7fc68ffff700 1 -- 172.21.15.86:0/1644900870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc69801a720 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.666+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a819f930 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.667+0000 7fc68ffff700 1 -- 172.21.15.86:0/1644900870 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc69801aa20 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.667+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a8064150 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.667+0000 7fc68ffff700 1 --2- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc690039650 0x7fc69003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.667+0000 7fc68ffff700 1 -- 172.21.15.86:0/1644900870 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc69804df20 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.667+0000 7fc6a659c700 1 --2- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc690039650 0x7fc69003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.668+0000 7fc6a659c700 1 --2- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc690039650 0x7fc69003bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc69c006fd0 tx=0x7fc69c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.672+0000 7fc68ffff700 1 -- 172.21.15.86:0/1644900870 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc698011e20 con 0x7fc6a81075a0 2024-04-16T05:25:59.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.868+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fc6a819fc00 con 0x7fc6a81075a0 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.869+0000 7fc68ffff700 1 -- 172.21.15.86:0/1644900870 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fc69800bc40 con 0x7fc6a81075a0 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc690039650 msgr2=0x7fc69003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 --2- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc690039650 0x7fc69003bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc69c006fd0 tx=0x7fc69c006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 msgr2=0x7fc6a807ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 --2- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a807ba70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6980116a0 tx=0x7fc698011ba0 comp rx=0 tx=0).stop 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 shutdown_connections 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 --2- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc690039650 0x7fc69003bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 --2- 172.21.15.86:0/1644900870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a81075a0 0x7fc6a807ba70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 >> 172.21.15.86:0/1644900870 conn(0x7fc6a8102c40 msgr2=0x7fc6a81037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 shutdown_connections 2024-04-16T05:25:59.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:58.871+0000 7fc6ad100700 1 -- 172.21.15.86:0/1644900870 wait complete. 2024-04-16T05:26:00.834 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.915+0000 7f15885bb700 1 Processor -- start 2024-04-16T05:26:00.834 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.915+0000 7f15885bb700 1 -- start start 2024-04-16T05:26:00.834 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f15885bb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158010b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:00.834 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f15885bb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f158007be70 con 0x7f1580109150 2024-04-16T05:26:00.834 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f1586357700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158010b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:00.834 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f1586357700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158010b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54240/0 (socket says 172.21.15.86:54240) 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f1586357700 1 -- 172.21.15.86:0/1411627765 learned_addr learned my addr 172.21.15.86:0/1411627765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f1586357700 1 -- 172.21.15.86:0/1411627765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158010ba80 con 0x7f1580109150 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f1586357700 1 --2- 172.21.15.86:0/1411627765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158010b540 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f15740098a0 tx=0x7f1574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34059212e35cab27 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f1585355700 1 -- 172.21.15.86:0/1411627765 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1574004160 con 0x7f1580109150 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f1585355700 1 -- 172.21.15.86:0/1411627765 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1574004300 con 0x7f1580109150 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.916+0000 7f1585355700 1 -- 172.21.15.86:0/1411627765 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1574011b20 con 0x7f1580109150 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.917+0000 7f15885bb700 1 -- 172.21.15.86:0/1411627765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 msgr2=0x7f158010b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.917+0000 7f15885bb700 1 --2- 172.21.15.86:0/1411627765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158010b540 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f15740098a0 tx=0x7f1574009bb0 comp rx=0 tx=0).stop 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.917+0000 7f15885bb700 1 -- 172.21.15.86:0/1411627765 shutdown_connections 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.917+0000 7f15885bb700 1 --2- 172.21.15.86:0/1411627765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158010b540 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.917+0000 7f15885bb700 1 -- 172.21.15.86:0/1411627765 >> 172.21.15.86:0/1411627765 conn(0x7f1580102b80 msgr2=0x7f1580104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.917+0000 7f15885bb700 1 -- 172.21.15.86:0/1411627765 shutdown_connections 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.917+0000 7f15885bb700 1 -- 172.21.15.86:0/1411627765 wait complete. 2024-04-16T05:26:00.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f15885bb700 1 Processor -- start 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f15885bb700 1 -- start start 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f15885bb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f15885bb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f158010ca90 con 0x7f1580109150 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f1586357700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f1586357700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54248/0 (socket says 172.21.15.86:54248) 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f1586357700 1 -- 172.21.15.86:0/889601634 learned_addr learned my addr 172.21.15.86:0/889601634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f1586357700 1 -- 172.21.15.86:0/889601634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158019cb60 con 0x7f1580109150 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f1586357700 1 --2- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158019c620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f15740116a0 tx=0x7f1574011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f15737fe700 1 -- 172.21.15.86:0/889601634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1574003a40 con 0x7f1580109150 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f15737fe700 1 -- 172.21.15.86:0/889601634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1574003be0 con 0x7f1580109150 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f15737fe700 1 -- 172.21.15.86:0/889601634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f157401a750 con 0x7f1580109150 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.918+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158019cdf0 con 0x7f1580109150 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.919+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158019d250 con 0x7f1580109150 2024-04-16T05:26:00.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.919+0000 7f15737fe700 1 -- 172.21.15.86:0/889601634 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1574024aa0 con 0x7f1580109150 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.919+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1580064150 con 0x7f1580109150 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.920+0000 7f15737fe700 1 --2- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f156c039650 0x7f156c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.920+0000 7f15737fe700 1 -- 172.21.15.86:0/889601634 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f157404dfc0 con 0x7f1580109150 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.920+0000 7f1585b56700 1 --2- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f156c039650 0x7f156c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.920+0000 7f1585b56700 1 --2- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f156c039650 0x7f156c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f157c006fd0 tx=0x7f157c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:59.925+0000 7f15737fe700 1 -- 172.21.15.86:0/889601634 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1574011e20 con 0x7f1580109150 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.233+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f158019fc20 con 0x7f1580109150 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.236+0000 7f15737fe700 1 -- 172.21.15.86:0/889601634 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f1574029de0 con 0x7f1580109150 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f156c039650 msgr2=0x7f156c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 --2- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f156c039650 0x7f156c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f157c006fd0 tx=0x7f157c006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 msgr2=0x7f158019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 --2- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158019c620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f15740116a0 tx=0x7f1574011ba0 comp rx=0 tx=0).stop 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 shutdown_connections 2024-04-16T05:26:00.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 --2- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f156c039650 0x7f156c03bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 --2- 172.21.15.86:0/889601634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1580109150 0x7f158019c620 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 >> 172.21.15.86:0/889601634 conn(0x7f1580102b80 msgr2=0x7f15801036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 shutdown_connections 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-16T05:26:00.238+0000 7f15885bb700 1 -- 172.21.15.86:0/889601634 wait complete. 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout:Or, if you are only running a single cluster on this host: 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:00.838 INFO:teuthology.orchestra.run.smithi086.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-16T05:26:00.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:00.839 INFO:teuthology.orchestra.run.smithi086.stdout: ceph telemetry on 2024-04-16T05:26:00.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:00.839 INFO:teuthology.orchestra.run.smithi086.stdout:For more information see: 2024-04-16T05:26:00.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:00.839 INFO:teuthology.orchestra.run.smithi086.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-16T05:26:00.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:00.839 INFO:teuthology.orchestra.run.smithi086.stdout:Bootstrap complete. 2024-04-16T05:26:00.878 INFO:tasks.cephadm:Fetching config... 2024-04-16T05:26:00.879 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:26:00.879 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-16T05:26:00.896 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-16T05:26:00.896 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:26:00.896 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-16T05:26:00.952 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-16T05:26:00.952 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:26:00.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-16T05:26:01.022 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-16T05:26:01.022 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:26:01.022 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-16T05:26:01.077 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-16T05:26:01.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOOITO0oATfHLwKhPKtob4+MLIWNV7p2616NsdGtWk6lwh6dpw0zzat/knIeArU/sZ0my/9yl0xtmPjI9CETax+KDuW4FPUtsxTieI8Y6vOjRUGN7PNQqAyTPakyUUw7qijuGD31AqL6zzfr1k9rmeS3FRPMy+7H0Z+vTB7Lj7sfNSVRh0YbSMewXdtSSPPBq57SVjLI90ZkCb4q3ttuhcKuhafHpJKN33in+liohPQ8ns6MJbTBJen+PwECO401MkTi7sSmo8K3tiTOSImETpRDZYTYgUgowmNVIq+Chp4h50FRhDGjC5y98HLHde1HKEXODe6wUUvl3+0ibu1NG8GXlZKymsAxrF7MisIWwZXby1pHw8PaC7BtBUL5sEcz3l3IMFF8bG11VlfY7ug0xH9wzh6r23uweExnLXzBSyS+4M6Ts7wTL9T/TDP0YIawHeWZYy0fZbifV+8bZlqvvgVrR1LHD+xK7Tn0pj4z4jV0Sf6UPRnP+HxEtE0+VlzXc= ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T05:26:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOOITO0oATfHLwKhPKtob4+MLIWNV7p2616NsdGtWk6lwh6dpw0zzat/knIeArU/sZ0my/9yl0xtmPjI9CETax+KDuW4FPUtsxTieI8Y6vOjRUGN7PNQqAyTPakyUUw7qijuGD31AqL6zzfr1k9rmeS3FRPMy+7H0Z+vTB7Lj7sfNSVRh0YbSMewXdtSSPPBq57SVjLI90ZkCb4q3ttuhcKuhafHpJKN33in+liohPQ8ns6MJbTBJen+PwECO401MkTi7sSmo8K3tiTOSImETpRDZYTYgUgowmNVIq+Chp4h50FRhDGjC5y98HLHde1HKEXODe6wUUvl3+0ibu1NG8GXlZKymsAxrF7MisIWwZXby1pHw8PaC7BtBUL5sEcz3l3IMFF8bG11VlfY7ug0xH9wzh6r23uweExnLXzBSyS+4M6Ts7wTL9T/TDP0YIawHeWZYy0fZbifV+8bZlqvvgVrR1LHD+xK7Tn0pj4z4jV0Sf6UPRnP+HxEtE0+VlzXc= ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:01.177 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOOITO0oATfHLwKhPKtob4+MLIWNV7p2616NsdGtWk6lwh6dpw0zzat/knIeArU/sZ0my/9yl0xtmPjI9CETax+KDuW4FPUtsxTieI8Y6vOjRUGN7PNQqAyTPakyUUw7qijuGD31AqL6zzfr1k9rmeS3FRPMy+7H0Z+vTB7Lj7sfNSVRh0YbSMewXdtSSPPBq57SVjLI90ZkCb4q3ttuhcKuhafHpJKN33in+liohPQ8ns6MJbTBJen+PwECO401MkTi7sSmo8K3tiTOSImETpRDZYTYgUgowmNVIq+Chp4h50FRhDGjC5y98HLHde1HKEXODe6wUUvl3+0ibu1NG8GXlZKymsAxrF7MisIWwZXby1pHw8PaC7BtBUL5sEcz3l3IMFF8bG11VlfY7ug0xH9wzh6r23uweExnLXzBSyS+4M6Ts7wTL9T/TDP0YIawHeWZYy0fZbifV+8bZlqvvgVrR1LHD+xK7Tn0pj4z4jV0Sf6UPRnP+HxEtE0+VlzXc= ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T05:26:01.225 INFO:teuthology.orchestra.run.smithi194.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOOITO0oATfHLwKhPKtob4+MLIWNV7p2616NsdGtWk6lwh6dpw0zzat/knIeArU/sZ0my/9yl0xtmPjI9CETax+KDuW4FPUtsxTieI8Y6vOjRUGN7PNQqAyTPakyUUw7qijuGD31AqL6zzfr1k9rmeS3FRPMy+7H0Z+vTB7Lj7sfNSVRh0YbSMewXdtSSPPBq57SVjLI90ZkCb4q3ttuhcKuhafHpJKN33in+liohPQ8ns6MJbTBJen+PwECO401MkTi7sSmo8K3tiTOSImETpRDZYTYgUgowmNVIq+Chp4h50FRhDGjC5y98HLHde1HKEXODe6wUUvl3+0ibu1NG8GXlZKymsAxrF7MisIWwZXby1pHw8PaC7BtBUL5sEcz3l3IMFF8bG11VlfY7ug0xH9wzh6r23uweExnLXzBSyS+4M6Ts7wTL9T/TDP0YIawHeWZYy0fZbifV+8bZlqvvgVrR1LHD+xK7Tn0pj4z4jV0Sf6UPRnP+HxEtE0+VlzXc= ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:01.240 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOOITO0oATfHLwKhPKtob4+MLIWNV7p2616NsdGtWk6lwh6dpw0zzat/knIeArU/sZ0my/9yl0xtmPjI9CETax+KDuW4FPUtsxTieI8Y6vOjRUGN7PNQqAyTPakyUUw7qijuGD31AqL6zzfr1k9rmeS3FRPMy+7H0Z+vTB7Lj7sfNSVRh0YbSMewXdtSSPPBq57SVjLI90ZkCb4q3ttuhcKuhafHpJKN33in+liohPQ8ns6MJbTBJen+PwECO401MkTi7sSmo8K3tiTOSImETpRDZYTYgUgowmNVIq+Chp4h50FRhDGjC5y98HLHde1HKEXODe6wUUvl3+0ibu1NG8GXlZKymsAxrF7MisIWwZXby1pHw8PaC7BtBUL5sEcz3l3IMFF8bG11VlfY7ug0xH9wzh6r23uweExnLXzBSyS+4M6Ts7wTL9T/TDP0YIawHeWZYy0fZbifV+8bZlqvvgVrR1LHD+xK7Tn0pj4z4jV0Sf6UPRnP+HxEtE0+VlzXc= ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T05:26:01.287 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOOITO0oATfHLwKhPKtob4+MLIWNV7p2616NsdGtWk6lwh6dpw0zzat/knIeArU/sZ0my/9yl0xtmPjI9CETax+KDuW4FPUtsxTieI8Y6vOjRUGN7PNQqAyTPakyUUw7qijuGD31AqL6zzfr1k9rmeS3FRPMy+7H0Z+vTB7Lj7sfNSVRh0YbSMewXdtSSPPBq57SVjLI90ZkCb4q3ttuhcKuhafHpJKN33in+liohPQ8ns6MJbTBJen+PwECO401MkTi7sSmo8K3tiTOSImETpRDZYTYgUgowmNVIq+Chp4h50FRhDGjC5y98HLHde1HKEXODe6wUUvl3+0ibu1NG8GXlZKymsAxrF7MisIWwZXby1pHw8PaC7BtBUL5sEcz3l3IMFF8bG11VlfY7ug0xH9wzh6r23uweExnLXzBSyS+4M6Ts7wTL9T/TDP0YIawHeWZYy0fZbifV+8bZlqvvgVrR1LHD+xK7Tn0pj4z4jV0Sf6UPRnP+HxEtE0+VlzXc= ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:01.302 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-16T05:26:01.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:01 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/889601634' entity='client.admin' 2024-04-16T05:26:01.545 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:03.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.775+0000 7f9b395c3700 1 -- 172.21.15.86:0/2017943798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 msgr2=0x7f9b34105450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.775+0000 7f9b395c3700 1 --2- 172.21.15.86:0/2017943798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 0x7f9b34105450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009910 tx=0x7f9b24009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.775+0000 7f9b395c3700 1 -- 172.21.15.86:0/2017943798 shutdown_connections 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.775+0000 7f9b395c3700 1 --2- 172.21.15.86:0/2017943798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 0x7f9b34105450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.775+0000 7f9b395c3700 1 -- 172.21.15.86:0/2017943798 >> 172.21.15.86:0/2017943798 conn(0x7f9b340fc9f0 msgr2=0x7f9b340fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.775+0000 7f9b395c3700 1 -- 172.21.15.86:0/2017943798 shutdown_connections 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.775+0000 7f9b395c3700 1 -- 172.21.15.86:0/2017943798 wait complete. 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.776+0000 7f9b395c3700 1 Processor -- start 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.776+0000 7f9b395c3700 1 -- start start 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.776+0000 7f9b395c3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 0x7f9b34197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.776+0000 7f9b395c3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b341085e0 con 0x7f9b34103060 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.776+0000 7f9b32ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 0x7f9b34197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.776+0000 7f9b32ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 0x7f9b34197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36296/0 (socket says 172.21.15.86:36296) 2024-04-16T05:26:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.776+0000 7f9b32ffd700 1 -- 172.21.15.86:0/2845800702 learned_addr learned my addr 172.21.15.86:0/2845800702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.777+0000 7f9b32ffd700 1 -- 172.21.15.86:0/2845800702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b34198130 con 0x7f9b34103060 2024-04-16T05:26:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.777+0000 7f9b32ffd700 1 --2- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 0x7f9b34197bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b24004fa0 tx=0x7f9b24005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.777+0000 7f9b30ff9700 1 -- 172.21.15.86:0/2845800702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b24027070 con 0x7f9b34103060 2024-04-16T05:26:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.777+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b341983c0 con 0x7f9b34103060 2024-04-16T05:26:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.777+0000 7f9b30ff9700 1 -- 172.21.15.86:0/2845800702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2400be40 con 0x7f9b34103060 2024-04-16T05:26:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.777+0000 7f9b30ff9700 1 -- 172.21.15.86:0/2845800702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b2400f460 con 0x7f9b34103060 2024-04-16T05:26:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.777+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3419b0b0 con 0x7f9b34103060 2024-04-16T05:26:03.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.778+0000 7f9b30ff9700 1 -- 172.21.15.86:0/2845800702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9b2402a070 con 0x7f9b34103060 2024-04-16T05:26:03.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.778+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b34108d60 con 0x7f9b34103060 2024-04-16T05:26:03.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.778+0000 7f9b30ff9700 1 --2- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9b20039650 0x7f9b2003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:03.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.778+0000 7f9b30ff9700 1 -- 172.21.15.86:0/2845800702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9b2401e030 con 0x7f9b34103060 2024-04-16T05:26:03.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.778+0000 7f9b2a7fc700 1 --2- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9b20039650 0x7f9b2003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:03.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.778+0000 7f9b2a7fc700 1 --2- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9b20039650 0x7f9b2003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006fd0 tx=0x7f9b1c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.783+0000 7f9b30ff9700 1 -- 172.21.15.86:0/2845800702 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b24013070 con 0x7f9b34103060 2024-04-16T05:26:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.984+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9b3402e0f0 con 0x7f9b34103060 2024-04-16T05:26:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.986+0000 7f9b30ff9700 1 -- 172.21.15.86:0/2845800702 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9b240292c0 con 0x7f9b34103060 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9b20039650 msgr2=0x7f9b2003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 --2- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9b20039650 0x7f9b2003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006fd0 tx=0x7f9b1c006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 msgr2=0x7f9b34197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 --2- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 0x7f9b34197bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b24004fa0 tx=0x7f9b24005080 comp rx=0 tx=0).stop 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 shutdown_connections 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 --2- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9b20039650 0x7f9b2003bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 --2- 172.21.15.86:0/2845800702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34103060 0x7f9b34197bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 >> 172.21.15.86:0/2845800702 conn(0x7f9b340fc9f0 msgr2=0x7f9b340fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 shutdown_connections 2024-04-16T05:26:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:03.989+0000 7f9b395c3700 1 -- 172.21.15.86:0/2845800702 wait complete. 2024-04-16T05:26:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:03 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:03 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:03 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:26:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:03 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:03 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:03 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:04.591 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-16T05:26:04.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-16T05:26:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:05.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:04 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2845800702' entity='client.admin' 2024-04-16T05:26:05.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:05.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:06.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.337+0000 7ffa3a254700 1 -- 172.21.15.86:0/957189558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 msgr2=0x7ffa34101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.337+0000 7ffa3a254700 1 --2- 172.21.15.86:0/957189558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 0x7ffa34101870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009910 tx=0x7ffa24009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.337+0000 7ffa3a254700 1 -- 172.21.15.86:0/957189558 shutdown_connections 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.337+0000 7ffa3a254700 1 --2- 172.21.15.86:0/957189558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 0x7ffa34101870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.337+0000 7ffa3a254700 1 -- 172.21.15.86:0/957189558 >> 172.21.15.86:0/957189558 conn(0x7ffa340fc9f0 msgr2=0x7ffa340fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.337+0000 7ffa3a254700 1 -- 172.21.15.86:0/957189558 shutdown_connections 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.337+0000 7ffa3a254700 1 -- 172.21.15.86:0/957189558 wait complete. 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.338+0000 7ffa3a254700 1 Processor -- start 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.338+0000 7ffa3a254700 1 -- start start 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.338+0000 7ffa3a254700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 0x7ffa34196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:06.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.338+0000 7ffa3a254700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341074e0 con 0x7ffa34101450 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.338+0000 7ffa337fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 0x7ffa34196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.338+0000 7ffa337fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 0x7ffa34196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36310/0 (socket says 172.21.15.86:36310) 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.338+0000 7ffa337fe700 1 -- 172.21.15.86:0/3511126549 learned_addr learned my addr 172.21.15.86:0/3511126549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.338+0000 7ffa337fe700 1 -- 172.21.15.86:0/3511126549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa341970a0 con 0x7ffa34101450 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.339+0000 7ffa337fe700 1 --2- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 0x7ffa34196b60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa24006010 tx=0x7ffa24005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.339+0000 7ffa30ff9700 1 -- 172.21.15.86:0/3511126549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa24027070 con 0x7ffa34101450 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.339+0000 7ffa30ff9700 1 -- 172.21.15.86:0/3511126549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2400be40 con 0x7ffa34101450 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.339+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa34197330 con 0x7ffa34101450 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.339+0000 7ffa30ff9700 1 -- 172.21.15.86:0/3511126549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa2400f460 con 0x7ffa34101450 2024-04-16T05:26:06.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.339+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3419a020 con 0x7ffa34101450 2024-04-16T05:26:06.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.340+0000 7ffa30ff9700 1 -- 172.21.15.86:0/3511126549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ffa2402a070 con 0x7ffa34101450 2024-04-16T05:26:06.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.340+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa34107c60 con 0x7ffa34101450 2024-04-16T05:26:06.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.340+0000 7ffa30ff9700 1 --2- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa1c039650 0x7ffa1c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:06.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.340+0000 7ffa30ff9700 1 -- 172.21.15.86:0/3511126549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa2401e030 con 0x7ffa34101450 2024-04-16T05:26:06.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.340+0000 7ffa32ffd700 1 --2- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa1c039650 0x7ffa1c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:06.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.340+0000 7ffa32ffd700 1 --2- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa1c039650 0x7ffa1c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffa28006fd0 tx=0x7ffa28006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.345+0000 7ffa30ff9700 1 -- 172.21.15.86:0/3511126549 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa24013070 con 0x7ffa34101450 2024-04-16T05:26:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.554+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ffa3402e0f0 con 0x7ffa1c039650 2024-04-16T05:26:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.557+0000 7ffa30ff9700 1 -- 172.21.15.86:0/3511126549 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa3402e0f0 con 0x7ffa1c039650 2024-04-16T05:26:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.559+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa1c039650 msgr2=0x7ffa1c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.559+0000 7ffa3a254700 1 --2- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa1c039650 0x7ffa1c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffa28006fd0 tx=0x7ffa28006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.559+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 msgr2=0x7ffa34196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.559+0000 7ffa3a254700 1 --2- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 0x7ffa34196b60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa24006010 tx=0x7ffa24005050 comp rx=0 tx=0).stop 2024-04-16T05:26:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.559+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 shutdown_connections 2024-04-16T05:26:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.559+0000 7ffa3a254700 1 --2- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa1c039650 0x7ffa1c03bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.560+0000 7ffa3a254700 1 --2- 172.21.15.86:0/3511126549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101450 0x7ffa34196b60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.560+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 >> 172.21.15.86:0/3511126549 conn(0x7ffa340fc9f0 msgr2=0x7ffa340fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.560+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 shutdown_connections 2024-04-16T05:26:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:06.560+0000 7ffa3a254700 1 -- 172.21.15.86:0/3511126549 wait complete. 2024-04-16T05:26:07.370 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi194 2024-04-16T05:26:07.370 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:26:07.370 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.conf 2024-04-16T05:26:07.388 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:26:07.388 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:07.445 INFO:tasks.cephadm:Adding host smithi194 to orchestrator... 2024-04-16T05:26:07.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch host add smithi194 2024-04-16T05:26:07.682 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:07.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:07 smithi086 ceph-mon[32028]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:07.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:07.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:07.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:07.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:07.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:07.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:07.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:08.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:08 smithi086 ceph-mon[32028]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-16T05:26:08.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:08 smithi086 ceph-mon[32028]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:08.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:08 smithi086 ceph-mon[32028]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:08.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:08 smithi086 ceph-mon[32028]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:26:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.252+0000 7f6783b65700 1 -- 172.21.15.86:0/2704164030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 msgr2=0x7f677c1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.252+0000 7f6783b65700 1 --2- 172.21.15.86:0/2704164030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 0x7f677c1089b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6778009910 tx=0x7f6778009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.252+0000 7f6783b65700 1 -- 172.21.15.86:0/2704164030 shutdown_connections 2024-04-16T05:26:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.252+0000 7f6783b65700 1 --2- 172.21.15.86:0/2704164030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 0x7f677c1089b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.252+0000 7f6783b65700 1 -- 172.21.15.86:0/2704164030 >> 172.21.15.86:0/2704164030 conn(0x7f677c0fe030 msgr2=0x7f677c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.252+0000 7f6783b65700 1 -- 172.21.15.86:0/2704164030 shutdown_connections 2024-04-16T05:26:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.252+0000 7f6783b65700 1 -- 172.21.15.86:0/2704164030 wait complete. 2024-04-16T05:26:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.253+0000 7f6783b65700 1 Processor -- start 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.253+0000 7f6783b65700 1 -- start start 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.253+0000 7f6783b65700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 0x7f677c074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.253+0000 7f6783b65700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10cb90 con 0x7f677c1085d0 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.253+0000 7f6781901700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 0x7f677c074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.253+0000 7f6781901700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 0x7f677c074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36320/0 (socket says 172.21.15.86:36320) 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.253+0000 7f6781901700 1 -- 172.21.15.86:0/1862152320 learned_addr learned my addr 172.21.15.86:0/1862152320 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.253+0000 7f6781901700 1 -- 172.21.15.86:0/1862152320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c076a30 con 0x7f677c1085d0 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.254+0000 7f6781901700 1 --2- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 0x7f677c074df0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6778005850 tx=0x7f6778005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.254+0000 7f676effd700 1 -- 172.21.15.86:0/1862152320 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6778026070 con 0x7f677c1085d0 2024-04-16T05:26:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.254+0000 7f676effd700 1 -- 172.21.15.86:0/1862152320 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f677800bc30 con 0x7f677c1085d0 2024-04-16T05:26:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.254+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c076cc0 con 0x7f677c1085d0 2024-04-16T05:26:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.254+0000 7f676effd700 1 -- 172.21.15.86:0/1862152320 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6778003a20 con 0x7f677c1085d0 2024-04-16T05:26:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.254+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c075500 con 0x7f677c1085d0 2024-04-16T05:26:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.255+0000 7f676effd700 1 -- 172.21.15.86:0/1862152320 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6778029070 con 0x7f677c1085d0 2024-04-16T05:26:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.255+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c103c40 con 0x7f677c1085d0 2024-04-16T05:26:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.255+0000 7f676effd700 1 --2- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67680396b0 0x7f676803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.255+0000 7f676effd700 1 -- 172.21.15.86:0/1862152320 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f677801e070 con 0x7f677c1085d0 2024-04-16T05:26:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.255+0000 7f6781100700 1 --2- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67680396b0 0x7f676803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.255+0000 7f6781100700 1 --2- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67680396b0 0x7f676803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.260+0000 7f676effd700 1 -- 172.21.15.86:0/1862152320 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6778013070 con 0x7f677c1085d0 2024-04-16T05:26:09.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:09.463+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}) -- 0x7f677c076320 con 0x7f67680396b0 2024-04-16T05:26:09.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:09 smithi086 ceph-mon[32028]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:10.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:10 smithi086 ceph-mon[32028]: Deploying cephadm binary to smithi194 2024-04-16T05:26:12.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:12 smithi086 ceph-mon[32028]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:13.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.684+0000 7f676effd700 1 -- 172.21.15.86:0/1862152320 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f677c076320 con 0x7f67680396b0 2024-04-16T05:26:13.683 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi194' with addr '172.21.15.194' 2024-04-16T05:26:13.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.686+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67680396b0 msgr2=0x7f676803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:13.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.686+0000 7f6783b65700 1 --2- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67680396b0 0x7f676803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:13.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.686+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 msgr2=0x7f677c074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:13.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.686+0000 7f6783b65700 1 --2- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 0x7f677c074df0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6778005850 tx=0x7f6778005030 comp rx=0 tx=0).stop 2024-04-16T05:26:13.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.686+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 shutdown_connections 2024-04-16T05:26:13.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.686+0000 7f6783b65700 1 --2- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67680396b0 0x7f676803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:13.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.686+0000 7f6783b65700 1 --2- 172.21.15.86:0/1862152320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c1085d0 0x7f677c074df0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:13.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.686+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 >> 172.21.15.86:0/1862152320 conn(0x7f677c0fe030 msgr2=0x7f677c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:13.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.687+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 shutdown_connections 2024-04-16T05:26:13.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:13.687+0000 7f6783b65700 1 -- 172.21.15.86:0/1862152320 wait complete. 2024-04-16T05:26:14.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch host ls --format=json 2024-04-16T05:26:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:14.623 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:14 smithi086 ceph-mon[32028]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:14.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:14.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:14.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:15 smithi086 ceph-mon[32028]: Added host smithi194 2024-04-16T05:26:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:16.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.024+0000 7fc730bbf700 1 -- 172.21.15.86:0/3342108523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 msgr2=0x7fc72c1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:16.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.024+0000 7fc730bbf700 1 --2- 172.21.15.86:0/3342108523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 0x7fc72c1089b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009910 tx=0x7fc71c009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:16.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.024+0000 7fc730bbf700 1 -- 172.21.15.86:0/3342108523 shutdown_connections 2024-04-16T05:26:16.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.024+0000 7fc730bbf700 1 --2- 172.21.15.86:0/3342108523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 0x7fc72c1089b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:16.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.024+0000 7fc730bbf700 1 -- 172.21.15.86:0/3342108523 >> 172.21.15.86:0/3342108523 conn(0x7fc72c0fe030 msgr2=0x7fc72c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:16.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.024+0000 7fc730bbf700 1 -- 172.21.15.86:0/3342108523 shutdown_connections 2024-04-16T05:26:16.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.024+0000 7fc730bbf700 1 -- 172.21.15.86:0/3342108523 wait complete. 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.024+0000 7fc730bbf700 1 Processor -- start 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.025+0000 7fc730bbf700 1 -- start start 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.025+0000 7fc730bbf700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 0x7fc72c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.025+0000 7fc730bbf700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c10cb90 con 0x7fc72c1085d0 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.025+0000 7fc72a59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 0x7fc72c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.025+0000 7fc72a59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 0x7fc72c197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55830/0 (socket says 172.21.15.86:55830) 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.025+0000 7fc72a59c700 1 -- 172.21.15.86:0/2239705963 learned_addr learned my addr 172.21.15.86:0/2239705963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.025+0000 7fc72a59c700 1 -- 172.21.15.86:0/2239705963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c1984d0 con 0x7fc72c1085d0 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.025+0000 7fc72a59c700 1 --2- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 0x7fc72c197f90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc71c005850 tx=0x7fc71c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.025+0000 7fc723fff700 1 -- 172.21.15.86:0/2239705963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc71c026070 con 0x7fc72c1085d0 2024-04-16T05:26:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.026+0000 7fc723fff700 1 -- 172.21.15.86:0/2239705963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71c00bc30 con 0x7fc72c1085d0 2024-04-16T05:26:16.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.026+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72c198760 con 0x7fc72c1085d0 2024-04-16T05:26:16.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.026+0000 7fc723fff700 1 -- 172.21.15.86:0/2239705963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc71c003a20 con 0x7fc72c1085d0 2024-04-16T05:26:16.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.026+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c19c860 con 0x7fc72c1085d0 2024-04-16T05:26:16.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.026+0000 7fc723fff700 1 -- 172.21.15.86:0/2239705963 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc71c029070 con 0x7fc72c1085d0 2024-04-16T05:26:16.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.026+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72c103c40 con 0x7fc72c1085d0 2024-04-16T05:26:16.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.027+0000 7fc723fff700 1 --2- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc718039650 0x7fc71803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:16.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.027+0000 7fc723fff700 1 -- 172.21.15.86:0/2239705963 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc71c01e070 con 0x7fc72c1085d0 2024-04-16T05:26:16.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.027+0000 7fc721d9b700 1 --2- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc718039650 0x7fc71803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:16.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.027+0000 7fc721d9b700 1 --2- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc718039650 0x7fc71803bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc714006fd0 tx=0x7fc714006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:16.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.032+0000 7fc723fff700 1 -- 172.21.15.86:0/2239705963 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc71c013070 con 0x7fc72c1085d0 2024-04-16T05:26:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.237+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc72c1992f0 con 0x7fc718039650 2024-04-16T05:26:16.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.238+0000 7fc723fff700 1 -- 172.21.15.86:0/2239705963 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fc72c1992f0 con 0x7fc718039650 2024-04-16T05:26:16.237 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:16.237 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.194", "hostname": "smithi194", "labels": [], "status": ""}] 2024-04-16T05:26:16.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.240+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc718039650 msgr2=0x7fc71803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:16.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.240+0000 7fc730bbf700 1 --2- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc718039650 0x7fc71803bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc714006fd0 tx=0x7fc714006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:16.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.240+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 msgr2=0x7fc72c197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:16.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.240+0000 7fc730bbf700 1 --2- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 0x7fc72c197f90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc71c005850 tx=0x7fc71c005030 comp rx=0 tx=0).stop 2024-04-16T05:26:16.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.240+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 shutdown_connections 2024-04-16T05:26:16.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.240+0000 7fc730bbf700 1 --2- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc718039650 0x7fc71803bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:16.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.240+0000 7fc730bbf700 1 --2- 172.21.15.86:0/2239705963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c1085d0 0x7fc72c197f90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:16.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.240+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 >> 172.21.15.86:0/2239705963 conn(0x7fc72c0fe030 msgr2=0x7fc72c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:16.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.241+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 shutdown_connections 2024-04-16T05:26:16.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:16.241+0000 7fc730bbf700 1 -- 172.21.15.86:0/2239705963 wait complete. 2024-04-16T05:26:16.530 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:16 smithi086 ceph-mon[32028]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:16.531 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:16 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:16.816 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-04-16T05:26:16.816 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:26:16.816 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-04-16T05:26:16.834 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:26:16.835 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:16.891 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-04-16T05:26:16.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch host add smithi196 2024-04-16T05:26:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:17.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:17 smithi086 ceph-mon[32028]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:26:18.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.620+0000 7f10185ad700 1 -- 172.21.15.86:0/4232297425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 msgr2=0x7f10101089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:18.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.620+0000 7f10185ad700 1 --2- 172.21.15.86:0/4232297425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 0x7f10101089b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f100c009910 tx=0x7f100c009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:18.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.620+0000 7f10185ad700 1 -- 172.21.15.86:0/4232297425 shutdown_connections 2024-04-16T05:26:18.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.620+0000 7f10185ad700 1 --2- 172.21.15.86:0/4232297425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 0x7f10101089b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:18.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.620+0000 7f10185ad700 1 -- 172.21.15.86:0/4232297425 >> 172.21.15.86:0/4232297425 conn(0x7f10100fe030 msgr2=0x7f1010100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:18.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.620+0000 7f10185ad700 1 -- 172.21.15.86:0/4232297425 shutdown_connections 2024-04-16T05:26:18.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.620+0000 7f10185ad700 1 -- 172.21.15.86:0/4232297425 wait complete. 2024-04-16T05:26:18.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.620+0000 7f10185ad700 1 Processor -- start 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.620+0000 7f10185ad700 1 -- start start 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f10185ad700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 0x7f1010074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f10185ad700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f101010cb90 con 0x7f10101085d0 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f1016349700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 0x7f1010074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f1016349700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 0x7f1010074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55840/0 (socket says 172.21.15.86:55840) 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f1016349700 1 -- 172.21.15.86:0/805458006 learned_addr learned my addr 172.21.15.86:0/805458006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f1016349700 1 -- 172.21.15.86:0/805458006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1010076ce0 con 0x7f10101085d0 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f1016349700 1 --2- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 0x7f1010074df0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f100c005850 tx=0x7f100c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f10077fe700 1 -- 172.21.15.86:0/805458006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f100c026070 con 0x7f10101085d0 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f10077fe700 1 -- 172.21.15.86:0/805458006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f100c00bc30 con 0x7f10101085d0 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1010075330 con 0x7f10101085d0 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.621+0000 7f10077fe700 1 -- 172.21.15.86:0/805458006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f100c003a20 con 0x7f10101085d0 2024-04-16T05:26:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.622+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10100757f0 con 0x7f10101085d0 2024-04-16T05:26:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.622+0000 7f10077fe700 1 -- 172.21.15.86:0/805458006 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f100c029070 con 0x7f10101085d0 2024-04-16T05:26:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.623+0000 7f10077fe700 1 --2- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0ffc039650 0x7f0ffc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.623+0000 7f10077fe700 1 -- 172.21.15.86:0/805458006 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f100c01e070 con 0x7f10101085d0 2024-04-16T05:26:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.623+0000 7f1015b48700 1 --2- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0ffc039650 0x7f0ffc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.623+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1010103c40 con 0x7f10101085d0 2024-04-16T05:26:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.623+0000 7f1015b48700 1 --2- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0ffc039650 0x7f0ffc03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1000006fd0 tx=0x7f1000006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:18.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.628+0000 7f10077fe700 1 -- 172.21.15.86:0/805458006 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f100c030020 con 0x7f10101085d0 2024-04-16T05:26:18.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:18 smithi086 ceph-mon[32028]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:18.830+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7f1010076320 con 0x7f0ffc039650 2024-04-16T05:26:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:20 smithi086 ceph-mon[32028]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:20 smithi086 ceph-mon[32028]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:26:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:21.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:21 smithi086 ceph-mon[32028]: Deploying cephadm binary to smithi196 2024-04-16T05:26:21.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:21 smithi086 ceph-mon[32028]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-16T05:26:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:21 smithi086 ceph-mon[32028]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:21 smithi086 ceph-mon[32028]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:22.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:22 smithi086 ceph-mon[32028]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:23.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.237+0000 7f10077fe700 1 -- 172.21.15.86:0/805458006 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f1010076320 con 0x7f0ffc039650 2024-04-16T05:26:23.236 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-04-16T05:26:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0ffc039650 msgr2=0x7f0ffc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 --2- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0ffc039650 0x7f0ffc03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1000006fd0 tx=0x7f1000006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 msgr2=0x7f1010074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 --2- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 0x7f1010074df0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f100c005850 tx=0x7f100c005030 comp rx=0 tx=0).stop 2024-04-16T05:26:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 shutdown_connections 2024-04-16T05:26:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 --2- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0ffc039650 0x7f0ffc03bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 --2- 172.21.15.86:0/805458006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10101085d0 0x7f1010074df0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 >> 172.21.15.86:0/805458006 conn(0x7f10100fe030 msgr2=0x7f10100ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 shutdown_connections 2024-04-16T05:26:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:23.239+0000 7f10185ad700 1 -- 172.21.15.86:0/805458006 wait complete. 2024-04-16T05:26:23.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch host ls --format=json 2024-04-16T05:26:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:24.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:23 smithi086 ceph-mon[32028]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:26:24.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:24.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:24.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:24.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:24.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:23 smithi086 ceph-mon[32028]: Added host smithi196 2024-04-16T05:26:24.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:24.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:23 smithi086 ceph-mon[32028]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:24.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:25.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:25 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.671+0000 7fba7a085700 1 -- 172.21.15.86:0/4171244002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 msgr2=0x7fba741016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.671+0000 7fba7a085700 1 --2- 172.21.15.86:0/4171244002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 0x7fba741016b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba6400b0a0 tx=0x7fba6400b3b0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.671+0000 7fba7a085700 1 -- 172.21.15.86:0/4171244002 shutdown_connections 2024-04-16T05:26:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.671+0000 7fba7a085700 1 --2- 172.21.15.86:0/4171244002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 0x7fba741016b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.671+0000 7fba7a085700 1 -- 172.21.15.86:0/4171244002 >> 172.21.15.86:0/4171244002 conn(0x7fba740fcf10 msgr2=0x7fba740ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.671+0000 7fba7a085700 1 -- 172.21.15.86:0/4171244002 shutdown_connections 2024-04-16T05:26:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.671+0000 7fba7a085700 1 -- 172.21.15.86:0/4171244002 wait complete. 2024-04-16T05:26:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.671+0000 7fba7a085700 1 Processor -- start 2024-04-16T05:26:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba7a085700 1 -- start start 2024-04-16T05:26:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba7a085700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 0x7fba7419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba7a085700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba7410ed90 con 0x7fba741012d0 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba737fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 0x7fba7419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba737fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 0x7fba7419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48626/0 (socket says 172.21.15.86:48626) 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba737fe700 1 -- 172.21.15.86:0/1661403476 learned_addr learned my addr 172.21.15.86:0/1661403476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba737fe700 1 -- 172.21.15.86:0/1661403476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7419a720 con 0x7fba741012d0 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba737fe700 1 --2- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 0x7fba7419a1e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba640062a0 tx=0x7fba64003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba70ff9700 1 -- 172.21.15.86:0/1661403476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba64017040 con 0x7fba741012d0 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7419a9b0 con 0x7fba741012d0 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba70ff9700 1 -- 172.21.15.86:0/1661403476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba64022070 con 0x7fba741012d0 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba70ff9700 1 -- 172.21.15.86:0/1661403476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6401d440 con 0x7fba741012d0 2024-04-16T05:26:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.672+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7419ae10 con 0x7fba741012d0 2024-04-16T05:26:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.673+0000 7fba70ff9700 1 -- 172.21.15.86:0/1661403476 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fba6401d700 con 0x7fba741012d0 2024-04-16T05:26:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.673+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7406b1e0 con 0x7fba741012d0 2024-04-16T05:26:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.673+0000 7fba70ff9700 1 --2- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba5c039650 0x7fba5c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.674+0000 7fba70ff9700 1 -- 172.21.15.86:0/1661403476 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fba6400f050 con 0x7fba741012d0 2024-04-16T05:26:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.674+0000 7fba72ffd700 1 --2- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba5c039650 0x7fba5c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.674+0000 7fba72ffd700 1 --2- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba5c039650 0x7fba5c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba68006fd0 tx=0x7fba68006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.679+0000 7fba70ff9700 1 -- 172.21.15.86:0/1661403476 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba64007e20 con 0x7fba741012d0 2024-04-16T05:26:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.894+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba7419b5b0 con 0x7fba5c039650 2024-04-16T05:26:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.895+0000 7fba70ff9700 1 -- 172.21.15.86:0/1661403476 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fba7419b5b0 con 0x7fba5c039650 2024-04-16T05:26:25.894 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:26:25.894 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.194", "hostname": "smithi194", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-04-16T05:26:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.898+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba5c039650 msgr2=0x7fba5c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.898+0000 7fba7a085700 1 --2- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba5c039650 0x7fba5c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba68006fd0 tx=0x7fba68006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.898+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 msgr2=0x7fba7419a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.898+0000 7fba7a085700 1 --2- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 0x7fba7419a1e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba640062a0 tx=0x7fba64003ce0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.898+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 shutdown_connections 2024-04-16T05:26:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.898+0000 7fba7a085700 1 --2- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba5c039650 0x7fba5c03bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.898+0000 7fba7a085700 1 --2- 172.21.15.86:0/1661403476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba741012d0 0x7fba7419a1e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.898+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 >> 172.21.15.86:0/1661403476 conn(0x7fba740fcf10 msgr2=0x7fba740fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.899+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 shutdown_connections 2024-04-16T05:26:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:25.899+0000 7fba7a085700 1 -- 172.21.15.86:0/1661403476 wait complete. 2024-04-16T05:26:26.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:26 smithi086 ceph-mon[32028]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:26.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:26.524 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-16T05:26:26.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd crush tunables default 2024-04-16T05:26:26.761 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:27.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:27 smithi086 ceph-mon[32028]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:26:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.319+0000 7fa73d731700 1 -- 172.21.15.86:0/1204481831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 msgr2=0x7fa7381089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:28.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.319+0000 7fa73d731700 1 --2- 172.21.15.86:0/1204481831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 0x7fa7381089e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa728009910 tx=0x7fa728009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:28.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.320+0000 7fa73d731700 1 -- 172.21.15.86:0/1204481831 shutdown_connections 2024-04-16T05:26:28.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.320+0000 7fa73d731700 1 --2- 172.21.15.86:0/1204481831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 0x7fa7381089e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:28.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.320+0000 7fa73d731700 1 -- 172.21.15.86:0/1204481831 >> 172.21.15.86:0/1204481831 conn(0x7fa738077b70 msgr2=0x7fa738077f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:28.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.320+0000 7fa73d731700 1 -- 172.21.15.86:0/1204481831 shutdown_connections 2024-04-16T05:26:28.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.320+0000 7fa73d731700 1 -- 172.21.15.86:0/1204481831 wait complete. 2024-04-16T05:26:28.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.320+0000 7fa73d731700 1 Processor -- start 2024-04-16T05:26:28.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.320+0000 7fa73d731700 1 -- start start 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa73d731700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 0x7fa738101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa73d731700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810cbc0 con 0x7fa738108600 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa736ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 0x7fa738101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa736ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 0x7fa738101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48634/0 (socket says 172.21.15.86:48634) 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa736ffd700 1 -- 172.21.15.86:0/885502105 learned_addr learned my addr 172.21.15.86:0/885502105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa736ffd700 1 -- 172.21.15.86:0/885502105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738103b50 con 0x7fa738108600 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa736ffd700 1 --2- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 0x7fa738101c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa728005850 tx=0x7fa728005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa734ff9700 1 -- 172.21.15.86:0/885502105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa728026070 con 0x7fa738108600 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa734ff9700 1 -- 172.21.15.86:0/885502105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa72800bc30 con 0x7fa738108600 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7381021a0 con 0x7fa738108600 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa734ff9700 1 -- 172.21.15.86:0/885502105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa728003a20 con 0x7fa738108600 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.321+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa738102660 con 0x7fa738108600 2024-04-16T05:26:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.322+0000 7fa734ff9700 1 -- 172.21.15.86:0/885502105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa728029070 con 0x7fa738108600 2024-04-16T05:26:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.322+0000 7fa734ff9700 1 --2- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa7240396b0 0x7fa72403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.322+0000 7fa734ff9700 1 -- 172.21.15.86:0/885502105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa72801e070 con 0x7fa738108600 2024-04-16T05:26:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.322+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa738064150 con 0x7fa738108600 2024-04-16T05:26:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.323+0000 7fa72e7fc700 1 --2- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa7240396b0 0x7fa72403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.323+0000 7fa72e7fc700 1 --2- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa7240396b0 0x7fa72403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa720006fd0 tx=0x7fa720006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.327+0000 7fa734ff9700 1 -- 172.21.15.86:0/885502105 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa728030070 con 0x7fa738108600 2024-04-16T05:26:28.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:28 smithi086 ceph-mon[32028]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:28.529+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa738102890 con 0x7fa738108600 2024-04-16T05:26:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.297+0000 7fa734ff9700 1 -- 172.21.15.86:0/885502105 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fa728013070 con 0x7fa738108600 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa7240396b0 msgr2=0x7fa72403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 --2- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa7240396b0 0x7fa72403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa720006fd0 tx=0x7fa720006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 msgr2=0x7fa738101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 --2- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 0x7fa738101c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa728005850 tx=0x7fa728005030 comp rx=0 tx=0).stop 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 shutdown_connections 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 --2- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa7240396b0 0x7fa72403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 --2- 172.21.15.86:0/885502105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa738108600 0x7fa738101c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 >> 172.21.15.86:0/885502105 conn(0x7fa738077b70 msgr2=0x7fa7380fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 shutdown_connections 2024-04-16T05:26:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:29.300+0000 7fa73d731700 1 -- 172.21.15.86:0/885502105 wait complete. 2024-04-16T05:26:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-04-16T05:26:29.602 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:29 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/885502105' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-16T05:26:29.602 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:29.602 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:29.602 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:29.602 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:29.602 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:26:29.602 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:29.602 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:29.928 INFO:tasks.cephadm:Adding mon.a on smithi086 2024-04-16T05:26:29.929 INFO:tasks.cephadm:Adding mon.b on smithi194 2024-04-16T05:26:29.929 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-04-16T05:26:29.929 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch apply mon '3;smithi086:172.21.15.86=a;smithi194:172.21.15.194=b;smithi196:172.21.15.196=c' 2024-04-16T05:26:30.153 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:30.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-16T05:26:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/885502105' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-16T05:26:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:26:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:31.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.490+0000 7fae6250e700 1 -- 172.21.15.196:0/1972778493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 msgr2=0x7fae5c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:31.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.490+0000 7fae6250e700 1 --2- 172.21.15.196:0/1972778493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 0x7fae5c1011b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009910 tx=0x7fae4c009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:31.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.490+0000 7fae6250e700 1 -- 172.21.15.196:0/1972778493 shutdown_connections 2024-04-16T05:26:31.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.490+0000 7fae6250e700 1 --2- 172.21.15.196:0/1972778493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 0x7fae5c1011b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:31.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.491+0000 7fae6250e700 1 -- 172.21.15.196:0/1972778493 >> 172.21.15.196:0/1972778493 conn(0x7fae5c077b70 msgr2=0x7fae5c077f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:31.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.491+0000 7fae6250e700 1 -- 172.21.15.196:0/1972778493 shutdown_connections 2024-04-16T05:26:31.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.491+0000 7fae6250e700 1 -- 172.21.15.196:0/1972778493 wait complete. 2024-04-16T05:26:31.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.491+0000 7fae6250e700 1 Processor -- start 2024-04-16T05:26:31.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.491+0000 7fae6250e700 1 -- start start 2024-04-16T05:26:31.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.492+0000 7fae6250e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 0x7fae5c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:31.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.492+0000 7fae6250e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c10ba30 con 0x7fae5c100dd0 2024-04-16T05:26:31.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.492+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 0x7fae5c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:31.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.492+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 0x7fae5c196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.196:50522/0 (socket says 172.21.15.196:50522) 2024-04-16T05:26:31.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.492+0000 7fae5bfff700 1 -- 172.21.15.196:0/607761693 learned_addr learned my addr 172.21.15.196:0/607761693 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-16T05:26:31.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.492+0000 7fae5bfff700 1 -- 172.21.15.196:0/607761693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5c10a860 con 0x7fae5c100dd0 2024-04-16T05:26:31.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.493+0000 7fae5bfff700 1 --2- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 0x7fae5c196c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae4c005850 tx=0x7fae4c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:31.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.493+0000 7fae597fa700 1 -- 172.21.15.196:0/607761693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae4c026070 con 0x7fae5c100dd0 2024-04-16T05:26:31.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.493+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5c197190 con 0x7fae5c100dd0 2024-04-16T05:26:31.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.493+0000 7fae597fa700 1 -- 172.21.15.196:0/607761693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c00bbd0 con 0x7fae5c100dd0 2024-04-16T05:26:31.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.493+0000 7fae597fa700 1 -- 172.21.15.196:0/607761693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae4c003970 con 0x7fae5c100dd0 2024-04-16T05:26:31.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.493+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae5c197650 con 0x7fae5c100dd0 2024-04-16T05:26:31.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.494+0000 7fae597fa700 1 -- 172.21.15.196:0/607761693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fae4c029070 con 0x7fae5c100dd0 2024-04-16T05:26:31.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.494+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae5c1022b0 con 0x7fae5c100dd0 2024-04-16T05:26:31.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.494+0000 7fae597fa700 1 --2- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae44039650 0x7fae4403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:31.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.494+0000 7fae597fa700 1 -- 172.21.15.196:0/607761693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fae4c01e070 con 0x7fae5c100dd0 2024-04-16T05:26:31.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.494+0000 7fae5b7fe700 1 --2- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae44039650 0x7fae4403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:31.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.495+0000 7fae5b7fe700 1 --2- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae44039650 0x7fae4403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fae50006fd0 tx=0x7fae50006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:31.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.499+0000 7fae597fa700 1 -- 172.21.15.196:0/607761693 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae4c013070 con 0x7fae5c100dd0 2024-04-16T05:26:31.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.707+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi194:172.21.15.194=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7fae5c10a050 con 0x7fae44039650 2024-04-16T05:26:31.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.715+0000 7fae597fa700 1 -- 172.21.15.196:0/607761693 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fae5c10a050 con 0x7fae44039650 2024-04-16T05:26:31.714 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-04-16T05:26:31.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae44039650 msgr2=0x7fae4403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:31.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 --2- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae44039650 0x7fae4403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fae50006fd0 tx=0x7fae50006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:31.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 msgr2=0x7fae5c196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:31.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 --2- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 0x7fae5c196c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae4c005850 tx=0x7fae4c005030 comp rx=0 tx=0).stop 2024-04-16T05:26:31.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 shutdown_connections 2024-04-16T05:26:31.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 --2- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae44039650 0x7fae4403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:31.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 --2- 172.21.15.196:0/607761693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae5c100dd0 0x7fae5c196c50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:31.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 >> 172.21.15.196:0/607761693 conn(0x7fae5c077b70 msgr2=0x7fae5c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:31.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 shutdown_connections 2024-04-16T05:26:31.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:31.717+0000 7fae6250e700 1 -- 172.21.15.196:0/607761693 wait complete. 2024-04-16T05:26:32.253 DEBUG:teuthology.orchestra.run.smithi194:mon.b> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.b.service 2024-04-16T05:26:32.256 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.c.service 2024-04-16T05:26:32.261 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T05:26:32.261 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph mon dump -f json 2024-04-16T05:26:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:32 smithi086 ceph-mon[32028]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:33.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:33 smithi086 ceph-mon[32028]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi194:172.21.15.194=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:33.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:33 smithi086 ceph-mon[32028]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi194:172.21.15.194=b;smithi196:172.21.15.196=c;count:3 2024-04-16T05:26:33.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:33 smithi086 ceph-mon[32028]: Deploying daemon mon.c on smithi196 2024-04-16T05:26:34.287 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:26:34.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:34 smithi086 ceph-mon[32028]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 podman[36344]: 2024-04-16 05:26:36.024941637 +0000 UTC m=+0.332473654 container create 2a64957c51d304e5f4f946e56fd1499e2e453bda7e90c3351a68f87040fc7f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:26:36.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 podman[36344]: 2024-04-16 05:26:36.22762886 +0000 UTC m=+0.535160877 container init 2a64957c51d304e5f4f946e56fd1499e2e453bda7e90c3351a68f87040fc7f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-c, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:26:36.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 podman[36344]: 2024-04-16 05:26:36.23067216 +0000 UTC m=+0.538204178 container start 2a64957c51d304e5f4f946e56fd1499e2e453bda7e90c3351a68f87040fc7f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5) 2024-04-16T05:26:36.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: set uid:gid to 167:167 (ceph:ceph) 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev), process ceph-mon, pid 2 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pidfile_write: ignore empty --pid-file 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: load: jerasure load: lrc 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: RocksDB version: 7.9.2 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Git sha 0 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Compile date 2024-04-16 03:54:27 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: DB SUMMARY 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: DB Session ID: KH2ZUTMBWCD10ZBCA3B7 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: CURRENT file: CURRENT 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: IDENTITY file: IDENTITY 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-16T05:26:36.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.error_if_exists: 0 2024-04-16T05:26:36.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.create_if_missing: 0 2024-04-16T05:26:36.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.paranoid_checks: 1 2024-04-16T05:26:36.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-16T05:26:36.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-16T05:26:36.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-16T05:26:36.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.env: 0x5611e0e2ecc0 2024-04-16T05:26:36.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.fs: PosixFileSystem 2024-04-16T05:26:36.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.info_log: 0x5611e1b13c20 2024-04-16T05:26:36.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_file_opening_threads: 16 2024-04-16T05:26:36.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.statistics: (nil) 2024-04-16T05:26:36.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.use_fsync: 0 2024-04-16T05:26:36.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_log_file_size: 0 2024-04-16T05:26:36.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-16T05:26:36.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-16T05:26:36.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.keep_log_file_num: 1000 2024-04-16T05:26:36.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.recycle_log_file_num: 0 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.allow_fallocate: 1 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.allow_mmap_reads: 0 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.allow_mmap_writes: 0 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.use_direct_reads: 0 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.create_missing_column_families: 0 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.db_log_dir: 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.wal_dir: 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-16T05:26:36.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.advise_random_on_open: 1 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.db_write_buffer_size: 0 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.write_buffer_manager: 0x5611e1ae21e0 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.rate_limiter: (nil) 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-16T05:26:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.wal_recovery_mode: 2 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.enable_thread_tracking: 0 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.enable_pipelined_write: 0 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.unordered_write: 0 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.row_cache: None 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.wal_filter: None 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.allow_ingest_behind: 0 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.two_write_queues: 0 2024-04-16T05:26:36.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.manual_wal_flush: 0 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.wal_compression: 0 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.atomic_flush: 0 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.log_readahead_size: 0 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.best_efforts_recovery: 0 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.allow_data_in_errors: 0 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.db_host_id: __hostname__ 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-16T05:26:36.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_background_jobs: 2 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_background_compactions: -1 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_subcompactions: 1 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_total_wal_size: 0 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_open_files: -1 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bytes_per_sync: 0 2024-04-16T05:26:36.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_readahead_size: 0 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_background_flushes: -1 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Compression algorithms supported: 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: kZSTD supported: 0 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: kXpressCompression supported: 0 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: kLZ4HCCompression supported: 1 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: kZlibCompression supported: 1 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: kSnappyCompression supported: 1 2024-04-16T05:26:36.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: kLZ4Compression supported: 1 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: kBZip2Compression supported: 0 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.merge_operator: 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_filter: None 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_filter_factory: None 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.sst_partitioner_factory: None 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-16T05:26:36.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5611e1b13de0) 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_type: 0 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_index_type: 0 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_shortening: 1 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: checksum: 4 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: no_block_cache: 0 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache: 0x5611e1b15090 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_options: 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: capacity : 536870912 2024-04-16T05:26:36.464 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_shard_bits : 4 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: strict_capacity_limit : 0 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_compressed: (nil) 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: persistent_cache: (nil) 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size: 4096 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size_deviation: 10 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_restart_interval: 16 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_block_restart_interval: 1 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: metadata_block_size: 4096 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: partition_filters: 0 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: use_delta_encoding: 1 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: filter_policy: bloomfilter 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: whole_key_filtering: 1 2024-04-16T05:26:36.465 INFO:journalctl@ceph.mon.c.smithi196.stdout: verify_compression: 0 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout: format_version: 5 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_index_compression: 1 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_align: 0 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout: max_auto_readahead_size: 262144 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout: prepopulate_block_cache: 0 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout: initial_auto_readahead_size: 8192 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.write_buffer_size: 33554432 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_write_buffer_number: 2 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression: NoCompression 2024-04-16T05:26:36.466 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression: Disabled 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.prefix_extractor: nullptr 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.num_levels: 7 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-16T05:26:36.467 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression_opts.level: 32767 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression_opts.strategy: 0 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression_opts.enabled: false 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.target_file_size_base: 67108864 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-16T05:26:36.468 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.arena_block_size: 1048576 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-16T05:26:36.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.disable_auto_compactions: 0 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-16T05:26:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.inplace_update_support: 0 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.bloom_locality: 0 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.max_successive_merges: 0 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.paranoid_file_checks: 0 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.force_consistency_checks: 1 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.report_bg_io_stats: 0 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.ttl: 2592000 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-16T05:26:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.enable_blob_files: false 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.min_blob_size: 0 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.blob_file_size: 268435456 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.blob_file_starting_level: 0 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 52399562-f78a-4f60-b740-d136b10d13bd 2024-04-16T05:26:36.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713245196260339, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713245196260757, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713245196, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "52399562-f78a-4f60-b740-d136b10d13bd", "db_session_id": "KH2ZUTMBWCD10ZBCA3B7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713245196260836, "job": 1, "event": "recovery_finished"} 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5611e1c02000 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: DB pointer 0x5611e1bec000 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** DB Stats ** 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-16T05:26:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-04-16T05:26:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T05:26:36.475 INFO:journalctl@ceph.mon.c.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache BinnedLRUCache@0x5611e1b15090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: starting mon.c rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(???) e0 preinit fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).mds e1 new map 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: e1 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: legacy client fscid: -1 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-16T05:26:36.476 INFO:journalctl@ceph.mon.c.smithi196.stdout: No filesystems configured 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:36.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Deploying cephadm binary to smithi196 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.478 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Added host smithi196 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.479 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/885502105' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/885502105' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:26:36.480 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi194:172.21.15.194=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi194:172.21.15.194=b;smithi196:172.21.15.196=c;count:3 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: Deploying daemon mon.c on smithi196 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 ceph-mon[36373]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-16T05:26:36.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 bash[36344]: 2a64957c51d304e5f4f946e56fd1499e2e453bda7e90c3351a68f87040fc7f49 2024-04-16T05:26:36.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:36 smithi196 systemd[1]: Started Ceph mon.c for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:26:41.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:26:41.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: mon.a calling monitor election 2024-04-16T05:26:41.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:41.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: mon.c calling monitor election 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: monmap epoch 2 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: last_changed 2024-04-16T05:26:36.283369+0000 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: created 2024-04-16T05:24:52.023563+0000 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: min_mon_release 19 (squid) 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: election_strategy: 1 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-16T05:26:41.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: fsmap 2024-04-16T05:26:41.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:41.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: mgrmap e13: x(active, since 49s) 2024-04-16T05:26:41.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: overall HEALTH_OK 2024-04-16T05:26:41.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:41.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:41.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:41.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:41.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:41.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:26:41.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: mon.a calling monitor election 2024-04-16T05:26:41.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:41.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: mon.c calling monitor election 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: monmap epoch 2 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: last_changed 2024-04-16T05:26:36.283369+0000 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: created 2024-04-16T05:24:52.023563+0000 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: min_mon_release 19 (squid) 2024-04-16T05:26:41.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: election_strategy: 1 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: fsmap 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: mgrmap e13: x(active, since 49s) 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: overall HEALTH_OK 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:41.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:42.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.555+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2721755398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 msgr2=0x7f53dc008700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:42.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.555+0000 7f53fa2e7700 1 --2- 172.21.15.196:0/2721755398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 0x7f53dc008700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53f4101d30 tx=0x7f53e400fe90 comp rx=0 tx=0).stop 2024-04-16T05:26:42.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.555+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2721755398 shutdown_connections 2024-04-16T05:26:42.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.555+0000 7f53fa2e7700 1 --2- 172.21.15.196:0/2721755398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 0x7f53dc008700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:42.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.555+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2721755398 >> 172.21.15.196:0/2721755398 conn(0x7f53f4077970 msgr2=0x7f53f4077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:42.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.555+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2721755398 shutdown_connections 2024-04-16T05:26:42.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.555+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2721755398 wait complete. 2024-04-16T05:26:42.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.556+0000 7f53fa2e7700 1 Processor -- start 2024-04-16T05:26:42.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.556+0000 7f53fa2e7700 1 -- start start 2024-04-16T05:26:42.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.556+0000 7f53fa2e7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 0x7f53f4101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:42.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.556+0000 7f53fa2e7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 0x7f53f4102860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.556+0000 7f53fa2e7700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410a670 con 0x7f53dc006290 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.556+0000 7f53fa2e7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410a4f0 con 0x7f53f41023e0 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 0x7f53f4102860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 0x7f53f4101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 0x7f53f4101ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.196:43582/0 (socket says 172.21.15.196:43582) 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 0x7f53f4102860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58030/0 (socket says 172.21.15.196:58030) 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f3fff700 1 -- 172.21.15.196:0/2427091081 learned_addr learned my addr 172.21.15.196:0/2427091081 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f37fe700 1 -- 172.21.15.196:0/2427091081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 msgr2=0x7f53f4102860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f37fe700 1 -- 172.21.15.196:0/2427091081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 msgr2=0x7f53f4102860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f37fe700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 0x7f53f4102860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f37fe700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 0x7f53f4102860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f3fff700 1 -- 172.21.15.196:0/2427091081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 msgr2=0x7f53f4102860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:26:42.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f3fff700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 0x7f53f4102860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:42.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f3fff700 1 -- 172.21.15.196:0/2427091081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f4103d70 con 0x7f53dc006290 2024-04-16T05:26:42.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.557+0000 7f53f3fff700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 0x7f53f4101ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f53e401d9b0 tx=0x7f53e401da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:42.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.558+0000 7f53f17fa700 1 -- 172.21.15.196:0/2427091081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f53e4005400 con 0x7f53dc006290 2024-04-16T05:26:42.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.558+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f41a4520 con 0x7f53dc006290 2024-04-16T05:26:42.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.558+0000 7f53f17fa700 1 -- 172.21.15.196:0/2427091081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e40055a0 con 0x7f53dc006290 2024-04-16T05:26:42.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.558+0000 7f53f17fa700 1 -- 172.21.15.196:0/2427091081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f53e4011850 con 0x7f53dc006290 2024-04-16T05:26:42.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.558+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f41a4a80 con 0x7f53dc006290 2024-04-16T05:26:42.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.559+0000 7f53f17fa700 1 -- 172.21.15.196:0/2427091081 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f53e4011b50 con 0x7f53dc006290 2024-04-16T05:26:42.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.559+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f406a1c0 con 0x7f53dc006290 2024-04-16T05:26:42.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.559+0000 7f53f17fa700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f53e00393f0 0x7f53e003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:42.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.559+0000 7f53f17fa700 1 -- 172.21.15.196:0/2427091081 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f53e404f740 con 0x7f53dc006290 2024-04-16T05:26:42.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.559+0000 7f53f37fe700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f53e00393f0 0x7f53e003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:42.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.560+0000 7f53f37fe700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f53e00393f0 0x7f53e003b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009850 tx=0x7f53e8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:42.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.565+0000 7f53f17fa700 1 -- 172.21.15.196:0/2427091081 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53e4025380 con 0x7f53dc006290 2024-04-16T05:26:42.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:42 smithi086 ceph-mon[32028]: Deploying daemon mon.b on smithi194 2024-04-16T05:26:42.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:42 smithi086 ceph-mon[32028]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:42.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:42 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:42.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:42 smithi196 ceph-mon[36373]: Deploying daemon mon.b on smithi194 2024-04-16T05:26:42.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:42 smithi196 ceph-mon[36373]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:42.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:42 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:42.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.847+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f53f402dfe0 con 0x7f53dc006290 2024-04-16T05:26:42.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.847+0000 7f53f17fa700 1 -- 172.21.15.196:0/2427091081 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f53e4011e40 con 0x7f53dc006290 2024-04-16T05:26:42.846 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:26:42.846 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","modified":"2024-04-16T05:26:36.283369Z","created":"2024-04-16T05:24:52.023563Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-16T05:26:42.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.849+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f53e00393f0 msgr2=0x7f53e003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:42.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.849+0000 7f53fa2e7700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f53e00393f0 0x7f53e003b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009850 tx=0x7f53e8008040 comp rx=0 tx=0).stop 2024-04-16T05:26:42.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.850+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 msgr2=0x7f53f4101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:42.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.850+0000 7f53fa2e7700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 0x7f53f4101ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f53e401d9b0 tx=0x7f53e401da90 comp rx=0 tx=0).stop 2024-04-16T05:26:42.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.850+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 shutdown_connections 2024-04-16T05:26:42.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.850+0000 7f53fa2e7700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53dc006290 0x7f53f4101ea0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:42.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.850+0000 7f53fa2e7700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f53e00393f0 0x7f53e003b8b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:42.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.850+0000 7f53fa2e7700 1 --2- 172.21.15.196:0/2427091081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53f41023e0 0x7f53f4102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:42.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.850+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 >> 172.21.15.196:0/2427091081 conn(0x7f53f4077970 msgr2=0x7f53f41000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:42.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.850+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 shutdown_connections 2024-04-16T05:26:42.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:42.850+0000 7f53fa2e7700 1 -- 172.21.15.196:0/2427091081 wait complete. 2024-04-16T05:26:42.849 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-04-16T05:26:43.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:43 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/2427091081' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:43.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:43 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/2427091081' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:44.323 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T05:26:44.323 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph mon dump -f json 2024-04-16T05:26:44.568 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:26:44.689 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:44 smithi196 ceph-mon[36373]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:44.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:44 smithi086 ceph-mon[32028]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:45.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:45 smithi194 systemd[1]: Started Ceph mon.b for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:26:50.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.361+0000 7fba3e110700 1 -- 172.21.15.196:0/4145774271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 msgr2=0x7fba20005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.361+0000 7fba3e110700 1 --2- 172.21.15.196:0/4145774271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 0x7fba20005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba38102540 tx=0x7fba2800fe10 comp rx=0 tx=0).stop 2024-04-16T05:26:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.361+0000 7fba3e110700 1 -- 172.21.15.196:0/4145774271 shutdown_connections 2024-04-16T05:26:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.361+0000 7fba3e110700 1 --2- 172.21.15.196:0/4145774271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 0x7fba20005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.361+0000 7fba3e110700 1 -- 172.21.15.196:0/4145774271 >> 172.21.15.196:0/4145774271 conn(0x7fba380fe100 msgr2=0x7fba38100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.361+0000 7fba3e110700 1 -- 172.21.15.196:0/4145774271 shutdown_connections 2024-04-16T05:26:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.361+0000 7fba3e110700 1 -- 172.21.15.196:0/4145774271 wait complete. 2024-04-16T05:26:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.362+0000 7fba3e110700 1 Processor -- start 2024-04-16T05:26:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.362+0000 7fba3e110700 1 -- start start 2024-04-16T05:26:50.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: mon.a calling monitor election 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: mon.c calling monitor election 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: mon.b calling monitor election 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: monmap epoch 3 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: last_changed 2024-04-16T05:26:44.973763+0000 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: created 2024-04-16T05:24:52.023563+0000 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: min_mon_release 19 (squid) 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: election_strategy: 1 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: 2: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.b 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: fsmap 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: mgrmap e13: x(active, since 58s) 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: overall HEALTH_OK 2024-04-16T05:26:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:50.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:50.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.362+0000 7fba3e110700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 0x7fba38198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:50.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.362+0000 7fba3e110700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38198760 0x7fba3819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:50.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.362+0000 7fba3e110700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3819d030 0x7fba3819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:50.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba3e110700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba3810a680 con 0x7fba200035c0 2024-04-16T05:26:50.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba3e110700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba3810a500 con 0x7fba3819d030 2024-04-16T05:26:50.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba3e110700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba3810a800 con 0x7fba38198760 2024-04-16T05:26:50.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba377fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 0x7fba38198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:50.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba36ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38198760 0x7fba3819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:50.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba377fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 0x7fba38198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.196:43662/0 (socket says 172.21.15.196:43662) 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba377fe700 1 -- 172.21.15.196:0/3216064524 learned_addr learned my addr 172.21.15.196:0/3216064524 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba37fff700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3819d030 0x7fba3819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba36ffd700 1 -- 172.21.15.196:0/3216064524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38198760 msgr2=0x7fba3819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba36ffd700 1 -- 172.21.15.196:0/3216064524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38198760 msgr2=0x7fba3819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba36ffd700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38198760 0x7fba3819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba36ffd700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38198760 0x7fba3819caf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba37fff700 1 -- 172.21.15.196:0/3216064524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3819d030 msgr2=0x7fba3819d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba37fff700 1 -- 172.21.15.196:0/3216064524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3819d030 msgr2=0x7fba3819d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba37fff700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3819d030 0x7fba3819d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba37fff700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3819d030 0x7fba3819d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba377fe700 1 -- 172.21.15.196:0/3216064524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38198760 msgr2=0x7fba3819caf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba377fe700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38198760 0x7fba3819caf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba377fe700 1 -- 172.21.15.196:0/3216064524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3819d030 msgr2=0x7fba3819d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:26:50.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba377fe700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3819d030 0x7fba3819d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba377fe700 1 -- 172.21.15.196:0/3216064524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba381a4da0 con 0x7fba200035c0 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.363+0000 7fba377fe700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 0x7fba38198220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba28000c00 tx=0x7fba2801de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.364+0000 7fba34ff9700 1 -- 172.21.15.196:0/3216064524 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28022040 con 0x7fba200035c0 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.364+0000 7fba34ff9700 1 -- 172.21.15.196:0/3216064524 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba2802c070 con 0x7fba200035c0 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.364+0000 7fba34ff9700 1 -- 172.21.15.196:0/3216064524 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28027410 con 0x7fba200035c0 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.364+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba381a4f70 con 0x7fba200035c0 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.364+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba381a54a0 con 0x7fba200035c0 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.365+0000 7fba34ff9700 1 -- 172.21.15.196:0/3216064524 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fba280275b0 con 0x7fba200035c0 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.365+0000 7fba34ff9700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba24039ae0 0x7fba2403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.365+0000 7fba34ff9700 1 -- 172.21.15.196:0/3216064524 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fba280503a0 con 0x7fba200035c0 2024-04-16T05:26:50.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.365+0000 7fba36ffd700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba24039ae0 0x7fba2403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:50.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.365+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba38064150 con 0x7fba200035c0 2024-04-16T05:26:50.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.366+0000 7fba36ffd700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba24039ae0 0x7fba2403bfa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009850 tx=0x7fba2c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:50.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.371+0000 7fba34ff9700 1 -- 172.21.15.196:0/3216064524 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba2802e310 con 0x7fba200035c0 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: mon.a calling monitor election 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: mon.c calling monitor election 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: mon.b calling monitor election 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: monmap epoch 3 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: last_changed 2024-04-16T05:26:44.973763+0000 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: created 2024-04-16T05:24:52.023563+0000 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: min_mon_release 19 (squid) 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: election_strategy: 1 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: 2: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.b 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: fsmap 2024-04-16T05:26:50.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:50.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: mgrmap e13: x(active, since 58s) 2024-04-16T05:26:50.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: overall HEALTH_OK 2024-04-16T05:26:50.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:50.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: mon.a calling monitor election 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: mon.c calling monitor election 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: mon.b calling monitor election 2024-04-16T05:26:50.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: monmap epoch 3 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: last_changed 2024-04-16T05:26:44.973763+0000 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: created 2024-04-16T05:24:52.023563+0000 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: min_mon_release 19 (squid) 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: election_strategy: 1 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: 2: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.b 2024-04-16T05:26:50.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: fsmap 2024-04-16T05:26:50.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:50.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: mgrmap e13: x(active, since 58s) 2024-04-16T05:26:50.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: overall HEALTH_OK 2024-04-16T05:26:50.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:50.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:50.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.667+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fba3802dfe0 con 0x7fba200035c0 2024-04-16T05:26:50.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.667+0000 7fba34ff9700 1 -- 172.21.15.196:0/3216064524 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7fba2801d490 con 0x7fba200035c0 2024-04-16T05:26:50.667 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-16T05:26:50.667 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","modified":"2024-04-16T05:26:44.973763Z","created":"2024-04-16T05:24:52.023563Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-16T05:26:50.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba24039ae0 msgr2=0x7fba2403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:50.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba24039ae0 0x7fba2403bfa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009850 tx=0x7fba2c008040 comp rx=0 tx=0).stop 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 msgr2=0x7fba38198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 0x7fba38198220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba28000c00 tx=0x7fba2801de50 comp rx=0 tx=0).stop 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 shutdown_connections 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3819d030 0x7fba3819d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba200035c0 0x7fba38198220 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fba24039ae0 0x7fba2403bfa0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 --2- 172.21.15.196:0/3216064524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38198760 0x7fba3819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.671+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 >> 172.21.15.196:0/3216064524 conn(0x7fba380fe100 msgr2=0x7fba380fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.672+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 shutdown_connections 2024-04-16T05:26:50.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:50.672+0000 7fba3e110700 1 -- 172.21.15.196:0/3216064524 wait complete. 2024-04-16T05:26:50.673 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-04-16T05:26:51.233 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:51.233 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:51.233 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:51.233 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:51 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/3216064524' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:51.233 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:51.267 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-16T05:26:51.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph config generate-minimal-conf 2024-04-16T05:26:51.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:51.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:51.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:51.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:51 smithi194 ceph-mon[35865]: from='client.? 172.21.15.196:0/3216064524' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:51.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:51.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:51.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:51.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:51.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:51 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/3216064524' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:51.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:52.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:52.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.145+0000 7f2bddccb700 1 -- 172.21.15.86:0/2507775292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd8107480 msgr2=0x7f2bd8107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:53.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.145+0000 7f2bddccb700 1 --2- 172.21.15.86:0/2507775292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd8107480 0x7f2bd8107860 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b0a0 tx=0x7f2bc800b3b0 comp rx=0 tx=0).stop 2024-04-16T05:26:53.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.145+0000 7f2bddccb700 1 -- 172.21.15.86:0/2507775292 shutdown_connections 2024-04-16T05:26:53.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.145+0000 7f2bddccb700 1 --2- 172.21.15.86:0/2507775292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd8107480 0x7f2bd8107860 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:53.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.145+0000 7f2bddccb700 1 -- 172.21.15.86:0/2507775292 >> 172.21.15.86:0/2507775292 conn(0x7f2bd8076f50 msgr2=0x7f2bd8077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:53.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.145+0000 7f2bddccb700 1 -- 172.21.15.86:0/2507775292 shutdown_connections 2024-04-16T05:26:53.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.145+0000 7f2bddccb700 1 -- 172.21.15.86:0/2507775292 wait complete. 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bddccb700 1 Processor -- start 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bddccb700 1 -- start start 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bddccb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bd8107480 0x7f2bd8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bddccb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd81987a0 0x7f2bd819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bddccb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bd819d070 0x7f2bd819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bddccb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd810b9e0 con 0x7f2bd81987a0 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bddccb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd810b860 con 0x7f2bd819d070 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bddccb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd810bb60 con 0x7f2bd8107480 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bd6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd81987a0 0x7f2bd819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bd6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd81987a0 0x7f2bd819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52346/0 (socket says 172.21.15.86:52346) 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bd6ffd700 1 -- 172.21.15.86:0/1988417439 learned_addr learned my addr 172.21.15.86:0/1988417439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bd77fe700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bd8107480 0x7f2bd8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bd6ffd700 1 -- 172.21.15.86:0/1988417439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bd8107480 msgr2=0x7f2bd8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.146+0000 7f2bd6ffd700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bd8107480 0x7f2bd8198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.147+0000 7f2bd6ffd700 1 -- 172.21.15.86:0/1988417439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bd819d070 msgr2=0x7f2bd819d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.147+0000 7f2bd6ffd700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bd819d070 0x7f2bd819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.147+0000 7f2bd6ffd700 1 -- 172.21.15.86:0/1988417439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd81a4de0 con 0x7f2bd81987a0 2024-04-16T05:26:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.147+0000 7f2bd6ffd700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd81987a0 0x7f2bd819cb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000b780 tx=0x7f2bc000bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.147+0000 7f2bd4ff9700 1 -- 172.21.15.86:0/1988417439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0004230 con 0x7f2bd81987a0 2024-04-16T05:26:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.147+0000 7f2bd4ff9700 1 -- 172.21.15.86:0/1988417439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc00043d0 con 0x7f2bd81987a0 2024-04-16T05:26:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.147+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd81a4fb0 con 0x7f2bd81987a0 2024-04-16T05:26:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.147+0000 7f2bd4ff9700 1 -- 172.21.15.86:0/1988417439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0011c60 con 0x7f2bd81987a0 2024-04-16T05:26:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.147+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81a54f0 con 0x7f2bd81987a0 2024-04-16T05:26:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.148+0000 7f2bd4ff9700 1 -- 172.21.15.86:0/1988417439 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2bc00036a0 con 0x7f2bd81987a0 2024-04-16T05:26:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.148+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd8064150 con 0x7f2bd81987a0 2024-04-16T05:26:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.148+0000 7f2bd4ff9700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2bc4039ae0 0x7f2bc403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.148+0000 7f2bd4ff9700 1 -- 172.21.15.86:0/1988417439 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2bc004c4d0 con 0x7f2bd81987a0 2024-04-16T05:26:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.148+0000 7f2bd77fe700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2bc4039ae0 0x7f2bc403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.149+0000 7f2bd77fe700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2bc4039ae0 0x7f2bc403bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b830 tx=0x7f2bc8005040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.153+0000 7f2bd4ff9700 1 -- 172.21.15.86:0/1988417439 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bc0021e00 con 0x7f2bd81987a0 2024-04-16T05:26:53.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.362+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f2bd81a58a0 con 0x7f2bd81987a0 2024-04-16T05:26:53.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.363+0000 7f2bd4ff9700 1 -- 172.21.15.86:0/1988417439 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f2bc0003d30 con 0x7f2bd81987a0 2024-04-16T05:26:53.362 INFO:teuthology.orchestra.run.smithi086.stdout:# minimal ceph.conf for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:53.362 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-16T05:26:53.362 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:53.362 INFO:teuthology.orchestra.run.smithi086.stdout: mon_host = [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-04-16T05:26:53.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2bc4039ae0 msgr2=0x7f2bc403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2bc4039ae0 0x7f2bc403bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b830 tx=0x7f2bc8005040 comp rx=0 tx=0).stop 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd81987a0 msgr2=0x7f2bd819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd81987a0 0x7f2bd819cb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000b780 tx=0x7f2bc000bb40 comp rx=0 tx=0).stop 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 shutdown_connections 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bd819d070 0x7f2bd819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2bc4039ae0 0x7f2bc403bfa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bd8107480 0x7f2bd8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 --2- 172.21.15.86:0/1988417439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd81987a0 0x7f2bd819cb30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 >> 172.21.15.86:0/1988417439 conn(0x7f2bd8076f50 msgr2=0x7f2bd8069f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 shutdown_connections 2024-04-16T05:26:53.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:26:53.365+0000 7f2bddccb700 1 -- 172.21.15.86:0/1988417439 wait complete. 2024-04-16T05:26:53.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-16T05:26:53.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-16T05:26:53.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-16T05:26:53.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:53.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:53.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:53.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.902 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.902 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.902 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:53.902 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:53.902 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:53.902 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:53 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1988417439' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-16T05:26:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-16T05:26:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-16T05:26:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:53.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:53.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:53.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:53 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1988417439' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:53.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:53.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:53 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1988417439' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:54.515 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-16T05:26:54.516 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:26:54.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T05:26:54.548 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:26:54.548 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:54.617 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:26:54.617 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T05:26:54.652 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:26:54.652 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:54.722 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:26:54.722 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T05:26:54.757 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:26:54.757 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:54.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:54 smithi086 ceph-mon[32028]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T05:26:54.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:54 smithi086 ceph-mon[32028]: Reconfiguring daemon mon.a on smithi086 2024-04-16T05:26:54.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:54 smithi086 ceph-mon[32028]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:54.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:54 smithi196 ceph-mon[36373]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T05:26:54.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:54 smithi196 ceph-mon[36373]: Reconfiguring daemon mon.a on smithi086 2024-04-16T05:26:54.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:54 smithi196 ceph-mon[36373]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:54.828 INFO:tasks.cephadm:Adding mgr.x on smithi086 2024-04-16T05:26:54.828 INFO:tasks.cephadm:Adding mgr.y on smithi194 2024-04-16T05:26:54.828 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-04-16T05:26:54.828 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch apply mgr '3;smithi086=x;smithi194=y;smithi196=z' 2024-04-16T05:26:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:54 smithi194 ceph-mon[35865]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T05:26:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:54 smithi194 ceph-mon[35865]: Reconfiguring daemon mon.a on smithi086 2024-04-16T05:26:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:54 smithi194 ceph-mon[35865]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:55.102 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:26:56.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:56.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:56.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:56 smithi196 ceph-mon[36373]: Reconfiguring mon.b (monmap changed)... 2024-04-16T05:26:56.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:56.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:56.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:56.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:56 smithi196 ceph-mon[36373]: Reconfiguring daemon mon.b on smithi194 2024-04-16T05:26:56.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:56 smithi196 ceph-mon[36373]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:56.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:56.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:56.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:56 smithi194 ceph-mon[35865]: Reconfiguring mon.b (monmap changed)... 2024-04-16T05:26:56.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:56.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:56.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:56.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:56 smithi194 ceph-mon[35865]: Reconfiguring daemon mon.b on smithi194 2024-04-16T05:26:56.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:56 smithi194 ceph-mon[35865]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:56.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:56.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:56.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:56 smithi086 ceph-mon[32028]: Reconfiguring mon.b (monmap changed)... 2024-04-16T05:26:56.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:56.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:56.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:56.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:56 smithi086 ceph-mon[32028]: Reconfiguring daemon mon.b on smithi194 2024-04-16T05:26:56.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:56 smithi086 ceph-mon[32028]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:56.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.530+0000 7fee6af0d700 1 -- 172.21.15.196:0/1213860433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee64102660 msgr2=0x7fee64102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.530+0000 7fee6af0d700 1 --2- 172.21.15.196:0/1213860433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee64102660 0x7fee64102a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee5400b0a0 tx=0x7fee5400b3b0 comp rx=0 tx=0).stop 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.530+0000 7fee6af0d700 1 -- 172.21.15.196:0/1213860433 shutdown_connections 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.530+0000 7fee6af0d700 1 --2- 172.21.15.196:0/1213860433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee64102660 0x7fee64102a80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.530+0000 7fee6af0d700 1 -- 172.21.15.196:0/1213860433 >> 172.21.15.196:0/1213860433 conn(0x7fee640fdc20 msgr2=0x7fee64100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.530+0000 7fee6af0d700 1 -- 172.21.15.196:0/1213860433 shutdown_connections 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.530+0000 7fee6af0d700 1 -- 172.21.15.196:0/1213860433 wait complete. 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee6af0d700 1 Processor -- start 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee6af0d700 1 -- start start 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee6af0d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee64102660 0x7fee64197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee6af0d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641983c0 0x7fee6419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee6af0d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee64198840 0x7fee6419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee6af0d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee641086a0 con 0x7fee641983c0 2024-04-16T05:26:56.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee6af0d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fee64108520 con 0x7fee64102660 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee6af0d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee64108820 con 0x7fee64198840 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee63fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641983c0 0x7fee6419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee694aa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee64198840 0x7fee6419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee694aa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee64198840 0x7fee6419f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59366/0 (socket says 172.21.15.196:59366) 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee63fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641983c0 0x7fee6419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.196:48992/0 (socket says 172.21.15.196:48992) 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee694aa700 1 -- 172.21.15.196:0/3766468040 learned_addr learned my addr 172.21.15.196:0/3766468040 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.531+0000 7fee68ca9700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee64102660 0x7fee64197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee694aa700 1 -- 172.21.15.196:0/3766468040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee64102660 msgr2=0x7fee64197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee694aa700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee64102660 0x7fee64197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee694aa700 1 -- 172.21.15.196:0/3766468040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641983c0 msgr2=0x7fee6419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee694aa700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641983c0 0x7fee6419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee694aa700 1 -- 172.21.15.196:0/3766468040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee641a0290 con 0x7fee64198840 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee694aa700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee64198840 0x7fee6419f9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00b550 tx=0x7fee5c00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee61ffb700 1 -- 172.21.15.196:0/3766468040 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c015020 con 0x7fee64198840 2024-04-16T05:26:56.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee641a0580 con 0x7fee64198840 2024-04-16T05:26:56.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee61ffb700 1 -- 172.21.15.196:0/3766468040 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5c011d10 con 0x7fee64198840 2024-04-16T05:26:56.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee61ffb700 1 -- 172.21.15.196:0/3766468040 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c010dd0 con 0x7fee64198840 2024-04-16T05:26:56.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.532+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641a0b40 con 0x7fee64198840 2024-04-16T05:26:56.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.533+0000 7fee61ffb700 1 -- 172.21.15.196:0/3766468040 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fee5c018020 con 0x7fee64198840 2024-04-16T05:26:56.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.533+0000 7fee61ffb700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fee50039ae0 0x7fee5003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:56.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.533+0000 7fee61ffb700 1 -- 172.21.15.196:0/3766468040 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fee5c04c0f0 con 0x7fee64198840 2024-04-16T05:26:56.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.533+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee64064150 con 0x7fee64198840 2024-04-16T05:26:56.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.533+0000 7fee68ca9700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fee50039ae0 0x7fee5003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:56.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.534+0000 7fee68ca9700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fee50039ae0 0x7fee5003bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fee540076a0 tx=0x7fee54005040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:56.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.539+0000 7fee61ffb700 1 -- 172.21.15.196:0/3766468040 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee5c01f020 con 0x7fee64198840 2024-04-16T05:26:56.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.750+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi194=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7fee64062f30 con 0x7fee50039ae0 2024-04-16T05:26:56.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.758+0000 7fee61ffb700 1 -- 172.21.15.196:0/3766468040 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fee64062f30 con 0x7fee50039ae0 2024-04-16T05:26:56.757 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-04-16T05:26:56.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.760+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fee50039ae0 msgr2=0x7fee5003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:56.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.760+0000 7fee6af0d700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fee50039ae0 0x7fee5003bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fee540076a0 tx=0x7fee54005040 comp rx=0 tx=0).stop 2024-04-16T05:26:56.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.760+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee64198840 msgr2=0x7fee6419f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:56.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.760+0000 7fee6af0d700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee64198840 0x7fee6419f9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00b550 tx=0x7fee5c00b860 comp rx=0 tx=0).stop 2024-04-16T05:26:56.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.761+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 shutdown_connections 2024-04-16T05:26:56.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.761+0000 7fee6af0d700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fee50039ae0 0x7fee5003bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:56.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.761+0000 7fee6af0d700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee64198840 0x7fee6419f9c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:56.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.761+0000 7fee6af0d700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee64102660 0x7fee64197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:56.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.761+0000 7fee6af0d700 1 --2- 172.21.15.196:0/3766468040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641983c0 0x7fee6419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:56.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.761+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 >> 172.21.15.196:0/3766468040 conn(0x7fee640fdc20 msgr2=0x7fee640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:56.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.761+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 shutdown_connections 2024-04-16T05:26:56.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:26:56.761+0000 7fee6af0d700 1 -- 172.21.15.196:0/3766468040 wait complete. 2024-04-16T05:26:57.526 DEBUG:teuthology.orchestra.run.smithi194:mgr.y> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.y.service 2024-04-16T05:26:57.529 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.z.service 2024-04-16T05:26:57.531 INFO:tasks.cephadm:Deploying OSDs... 2024-04-16T05:26:57.531 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:26:57.531 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T05:26:57.551 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T05:26:57.551 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-04-16T05:26:57.610 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T05:26:57.610 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:57.610 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-16T05:26:57.610 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:57.610 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:57.610 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-16 05:26:01.842575578 +0000 2024-04-16T05:26:57.610 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-16 05:18:21.733302073 +0000 2024-04-16T05:26:57.610 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-16 05:18:21.733302073 +0000 2024-04-16T05:26:57.610 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-16 05:18:21.733302073 +0000 2024-04-16T05:26:57.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T05:26:57.681 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-16T05:26:57.681 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-16T05:26:57.681 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000229419 s, 2.2 MB/s 2024-04-16T05:26:57.682 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T05:26:57.740 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-04-16T05:26:57.797 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T05:26:57.797 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:57.797 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-16T05:26:57.797 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:57.797 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:57.797 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-16 05:26:01.870574973 +0000 2024-04-16T05:26:57.797 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-16 05:18:22.085294467 +0000 2024-04-16T05:26:57.797 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-16 05:18:22.085294467 +0000 2024-04-16T05:26:57.797 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-16 05:18:22.085294467 +0000 2024-04-16T05:26:57.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: Reconfiguring mon.c (monmap changed)... 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: Reconfiguring daemon mon.c on smithi196 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi194=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: Saving service mgr spec with placement smithi086=x;smithi194=y;smithi196=z;count:3 2024-04-16T05:26:57.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:57.811 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:57 smithi196 ceph-mon[36373]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-16T05:26:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-16T05:26:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000200945 s, 2.5 MB/s 2024-04-16T05:26:57.867 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: Reconfiguring mon.c (monmap changed)... 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: Reconfiguring daemon mon.c on smithi196 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi194=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: Saving service mgr spec with placement smithi086=x;smithi194=y;smithi196=z;count:3 2024-04-16T05:26:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:57.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:57 smithi194 ceph-mon[35865]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:57.924 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-04-16T05:26:57.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:57.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:57.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: Reconfiguring mon.c (monmap changed)... 2024-04-16T05:26:57.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:57.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: Reconfiguring daemon mon.c on smithi196 2024-04-16T05:26:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi194=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: Saving service mgr spec with placement smithi086=x;smithi194=y;smithi196=z;count:3 2024-04-16T05:26:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:57 smithi086 ceph-mon[32028]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T05:26:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:57.980 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-16T05:26:57.980 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:57.981 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:57.981 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-16 05:26:01.898574368 +0000 2024-04-16T05:26:57.981 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-16 05:18:22.421287207 +0000 2024-04-16T05:26:57.981 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-16 05:18:22.421287207 +0000 2024-04-16T05:26:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-16 05:18:22.421287207 +0000 2024-04-16T05:26:57.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T05:26:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-16T05:26:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-16T05:26:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000197333 s, 2.6 MB/s 2024-04-16T05:26:58.049 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T05:26:58.105 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-04-16T05:26:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T05:26:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:58.161 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-16T05:26:58.161 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:58.161 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:58.161 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-16 05:26:01.925573785 +0000 2024-04-16T05:26:58.161 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-16 05:18:22.771279644 +0000 2024-04-16T05:26:58.161 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-16 05:18:22.771279644 +0000 2024-04-16T05:26:58.161 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-16 05:18:22.771279644 +0000 2024-04-16T05:26:58.161 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T05:26:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-16T05:26:58.227 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-16T05:26:58.227 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000198174 s, 2.6 MB/s 2024-04-16T05:26:58.228 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T05:26:58.284 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-16T05:26:58.284 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T05:26:58.303 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T05:26:58.303 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 2024-04-16T05:26:58.361 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T05:26:58.361 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:58.361 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-04-16T05:26:58.361 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:58.361 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:58.361 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-16 05:26:44.415693163 +0000 2024-04-16T05:26:58.361 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-16 05:18:33.175473854 +0000 2024-04-16T05:26:58.361 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-16 05:18:33.175473854 +0000 2024-04-16T05:26:58.361 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-04-16 05:18:33.175473854 +0000 2024-04-16T05:26:58.362 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T05:26:58.430 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-16T05:26:58.430 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-16T05:26:58.430 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000235726 s, 2.2 MB/s 2024-04-16T05:26:58.431 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T05:26:58.489 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 2024-04-16T05:26:58.546 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T05:26:58.546 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:58.546 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-04-16T05:26:58.546 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:58.546 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:58.546 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-16 05:26:44.416693143 +0000 2024-04-16T05:26:58.546 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-16 05:18:33.532466446 +0000 2024-04-16T05:26:58.546 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-16 05:18:33.532466446 +0000 2024-04-16T05:26:58.546 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-04-16 05:18:33.532466446 +0000 2024-04-16T05:26:58.547 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T05:26:58.615 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-16T05:26:58.615 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-16T05:26:58.615 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000337084 s, 1.5 MB/s 2024-04-16T05:26:58.616 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T05:26:58.673 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 2024-04-16T05:26:58.730 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T05:26:58.730 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:58.730 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-04-16T05:26:58.730 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:58.730 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:58.730 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-16 05:26:44.416693143 +0000 2024-04-16T05:26:58.730 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-16 05:18:33.884459142 +0000 2024-04-16T05:26:58.730 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-16 05:18:33.884459142 +0000 2024-04-16T05:26:58.730 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-04-16 05:18:33.884459142 +0000 2024-04-16T05:26:58.731 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T05:26:58.798 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-16T05:26:58.798 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-16T05:26:58.798 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000215348 s, 2.4 MB/s 2024-04-16T05:26:58.799 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T05:26:58.855 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 2024-04-16T05:26:58.910 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T05:26:58.911 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:58.911 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-16T05:26:58.911 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:58.911 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:58.911 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-16 05:26:44.417693123 +0000 2024-04-16T05:26:58.911 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-16 05:18:34.246451631 +0000 2024-04-16T05:26:58.911 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-16 05:18:34.246451631 +0000 2024-04-16T05:26:58.911 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-04-16 05:18:34.246451631 +0000 2024-04-16T05:26:58.911 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T05:26:58.976 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-16T05:26:58.976 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-16T05:26:58.976 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000201709 s, 2.5 MB/s 2024-04-16T05:26:58.977 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T05:26:59.033 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-16T05:26:59.033 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T05:26:59.050 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T05:26:59.050 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-04-16T05:26:59.105 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T05:26:59.105 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:59.105 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-16T05:26:59.105 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:59.106 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:59.106 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-16 05:26:35.374303586 +0000 2024-04-16T05:26:59.106 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-16 05:18:48.595866922 +0000 2024-04-16T05:26:59.106 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-16 05:18:48.595866922 +0000 2024-04-16T05:26:59.106 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-16 05:18:48.595866922 +0000 2024-04-16T05:26:59.106 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T05:26:59.172 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-16T05:26:59.173 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-16T05:26:59.173 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000277745 s, 1.8 MB/s 2024-04-16T05:26:59.174 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T05:26:59.229 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-04-16T05:26:59.285 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T05:26:59.285 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:59.285 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-04-16T05:26:59.285 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:59.285 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:59.285 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-16 05:26:35.375303565 +0000 2024-04-16T05:26:59.285 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-16 05:18:48.927859642 +0000 2024-04-16T05:26:59.285 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-16 05:18:48.927859642 +0000 2024-04-16T05:26:59.285 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-16 05:18:48.927859642 +0000 2024-04-16T05:26:59.286 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T05:26:59.351 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-16T05:26:59.351 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-16T05:26:59.351 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000218348 s, 2.3 MB/s 2024-04-16T05:26:59.352 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T05:26:59.407 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-04-16T05:26:59.461 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T05:26:59.462 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-04-16T05:26:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-16 05:26:35.375303565 +0000 2024-04-16T05:26:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-16 05:18:49.275852011 +0000 2024-04-16T05:26:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-16 05:18:49.275852011 +0000 2024-04-16T05:26:59.462 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-16 05:18:49.275852011 +0000 2024-04-16T05:26:59.462 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T05:26:59.529 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-16T05:26:59.529 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-16T05:26:59.529 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000185717 s, 2.8 MB/s 2024-04-16T05:26:59.530 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T05:26:59.586 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-04-16T05:26:59.640 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T05:26:59.640 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:59.640 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-04-16T05:26:59.640 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:59.641 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:59.641 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-16 05:26:35.375303565 +0000 2024-04-16T05:26:59.641 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-16 05:18:49.613844599 +0000 2024-04-16T05:26:59.641 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-16 05:18:49.613844599 +0000 2024-04-16T05:26:59.641 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-16 05:18:49.613844599 +0000 2024-04-16T05:26:59.641 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T05:26:59.703 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:59.703 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:59.703 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:59.703 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:59.703 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:59.703 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:59.704 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:59.704 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:26:59.704 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:59.704 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:26:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:59.705 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-16T05:26:59.705 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-16T05:26:59.705 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000198256 s, 2.6 MB/s 2024-04-16T05:26:59.707 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T05:26:59.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:59.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:59.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:59.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:59.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:59.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:59.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:59.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:26:59.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:59.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:26:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:59.762 INFO:tasks.cephadm:Deploying osd.0 on smithi086 with /dev/vg_nvme/lv_4... 2024-04-16T05:26:59.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T05:26:59.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:26:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:26:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:26:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:27:00.716 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:00 smithi196 ceph-mon[36373]: Deploying daemon mgr.z on smithi196 2024-04-16T05:27:00.716 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:00 smithi196 ceph-mon[36373]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:00.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:00 smithi086 ceph-mon[32028]: Deploying daemon mgr.z on smithi196 2024-04-16T05:27:00.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:00 smithi086 ceph-mon[32028]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:00 smithi194 ceph-mon[35865]: Deploying daemon mgr.z on smithi196 2024-04-16T05:27:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:00 smithi194 ceph-mon[35865]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:00.985 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:00 smithi196 systemd[1]: Starting Ceph mgr.z for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:27:01.403 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:01 smithi196 podman[38105]: 2024-04-16 05:27:01.056483029 +0000 UTC m=+0.017875218 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:27:01.403 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:01 smithi196 podman[38105]: 2024-04-16 05:27:01.15666065 +0000 UTC m=+0.118052832 container create d3858bd2fb3031435adfb77641533fcae48a917ac72d3cb9ddd94cd4024d6093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-16T05:27:01.403 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:01 smithi196 podman[38105]: 2024-04-16 05:27:01.359102087 +0000 UTC m=+0.320494263 container init d3858bd2fb3031435adfb77641533fcae48a917ac72d3cb9ddd94cd4024d6093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:27:01.403 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:01 smithi196 podman[38105]: 2024-04-16 05:27:01.361654981 +0000 UTC m=+0.323047164 container start d3858bd2fb3031435adfb77641533fcae48a917ac72d3cb9ddd94cd4024d6093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-16T05:27:01.403 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:01 smithi196 ceph-mgr[38124]: -- 172.21.15.196:0/2163839858 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e39a6045a0 con 0x55e39a5d9000 2024-04-16T05:27:01.735 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:01 smithi196 bash[38105]: d3858bd2fb3031435adfb77641533fcae48a917ac72d3cb9ddd94cd4024d6093 2024-04-16T05:27:01.735 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:01 smithi196 systemd[1]: Started Ceph mgr.z for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:27:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T05:27:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T05:27:01.859 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-16T05:27:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-16T05:27:01.859 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114462 s, 916 MB/s 2024-04-16T05:27:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-16T05:27:02.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:01 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:01.905+0000 7f82d4fc3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:27:02.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:02.003+0000 7f82d4fc3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:27:02.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-04-16T05:27:02.668 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:02 smithi194 ceph-mon[35865]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:02.668 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:02.668 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:02.668 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:02.668 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:27:02.669 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:27:02.669 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:27:02.669 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:02.669 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:02 smithi194 ceph-mon[35865]: Deploying daemon mgr.y on smithi194 2024-04-16T05:27:02.810 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:27:02.871 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:02.641+0000 7f82d4fc3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:27:02.871 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:02.732+0000 7f82d4fc3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:27:02.871 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:02.873+0000 7f82d4fc3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:27:02.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-mon[36373]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:02.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:02.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:02.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:02.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:27:02.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:27:02.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:27:02.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:02.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-mon[36373]: Deploying daemon mgr.y on smithi194 2024-04-16T05:27:02.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:02 smithi086 ceph-mon[32028]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:02.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:02.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:02.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:02.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:27:02.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:27:02.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:27:02.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:02.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:02 smithi086 ceph-mon[32028]: Deploying daemon mgr.y on smithi194 2024-04-16T05:27:03.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:02.968+0000 7f82d4fc3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:27:03.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:03.053+0000 7f82d4fc3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:27:03.534 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:03.329+0000 7f82d4fc3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:27:03.534 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:03.445+0000 7f82d4fc3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:27:03.534 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:03.535+0000 7f82d4fc3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:27:03.814 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:03.632+0000 7f82d4fc3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:27:03.814 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:03.815+0000 7f82d4fc3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:27:03.824 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:03 smithi194 systemd[1]: Starting Ceph mgr.y for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:27:04.078 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:03 smithi194 podman[37247]: 2024-04-16 05:27:03.825885008 +0000 UTC m=+0.093003209 container create 28612d49764f63fe771079b87205e4e117e3070b29b116438a80c87b4f71a89f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-16T05:27:04.078 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:03 smithi194 podman[37247]: 2024-04-16 05:27:03.750334879 +0000 UTC m=+0.017453046 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:27:04.078 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:04 smithi194 podman[37247]: 2024-04-16 05:27:04.052946146 +0000 UTC m=+0.320064304 container init 28612d49764f63fe771079b87205e4e117e3070b29b116438a80c87b4f71a89f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-16T05:27:04.078 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:04 smithi194 podman[37247]: 2024-04-16 05:27:04.05545221 +0000 UTC m=+0.322570370 container start 28612d49764f63fe771079b87205e4e117e3070b29b116438a80c87b4f71a89f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:27:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.343+0000 7f63246ae700 1 -- 172.21.15.86:0/66575824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c105f90 msgr2=0x7f631c108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.343+0000 7f63246ae700 1 --2- 172.21.15.86:0/66575824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c105f90 0x7f631c108380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6310009990 tx=0x7f6310009ca0 comp rx=0 tx=0).stop 2024-04-16T05:27:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.343+0000 7f63246ae700 1 -- 172.21.15.86:0/66575824 shutdown_connections 2024-04-16T05:27:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.343+0000 7f63246ae700 1 --2- 172.21.15.86:0/66575824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c1088c0 0x7f631c10acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.343+0000 7f63246ae700 1 --2- 172.21.15.86:0/66575824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f631c103660 0x7f631c105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.343+0000 7f63246ae700 1 --2- 172.21.15.86:0/66575824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c105f90 0x7f631c108380 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.343+0000 7f63246ae700 1 -- 172.21.15.86:0/66575824 >> 172.21.15.86:0/66575824 conn(0x7f631c0fd050 msgr2=0x7f631c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.343+0000 7f63246ae700 1 -- 172.21.15.86:0/66575824 shutdown_connections 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.343+0000 7f63246ae700 1 -- 172.21.15.86:0/66575824 wait complete. 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f63246ae700 1 Processor -- start 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f63246ae700 1 -- start start 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f63246ae700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c103660 0x7f631c102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f63246ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c105f90 0x7f631c101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f63246ae700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f631c1088c0 0x7f631c101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f63246ae700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f631c10e840 con 0x7f631c103660 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f63246ae700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f631c10e6c0 con 0x7f631c1088c0 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f63246ae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f631c10e9c0 con 0x7f631c105f90 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f632244a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c103660 0x7f631c102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f632244a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c103660 0x7f631c102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35930/0 (socket says 172.21.15.86:35930) 2024-04-16T05:27:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f632244a700 1 -- 172.21.15.86:0/2455587277 learned_addr learned my addr 172.21.15.86:0/2455587277 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.344+0000 7f6321c49700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c105f90 0x7f631c101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f6322c4b700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f631c1088c0 0x7f631c101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f632244a700 1 -- 172.21.15.86:0/2455587277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c105f90 msgr2=0x7f631c101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f632244a700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c105f90 0x7f631c101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f632244a700 1 -- 172.21.15.86:0/2455587277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f631c1088c0 msgr2=0x7f631c101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f632244a700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f631c1088c0 0x7f631c101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f632244a700 1 -- 172.21.15.86:0/2455587277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c102400 con 0x7f631c103660 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f632244a700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c103660 0x7f631c102fc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f631800b530 tx=0x7f631800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f630f7fe700 1 -- 172.21.15.86:0/2455587277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318015020 con 0x7f631c103660 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f630f7fe700 1 -- 172.21.15.86:0/2455587277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6318011d10 con 0x7f631c103660 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c1a2fe0 con 0x7f631c103660 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f630f7fe700 1 -- 172.21.15.86:0/2455587277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631800f2e0 con 0x7f631c103660 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.345+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c1a34c0 con 0x7f631c103660 2024-04-16T05:27:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.346+0000 7f630f7fe700 1 -- 172.21.15.86:0/2455587277 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6318018020 con 0x7f631c103660 2024-04-16T05:27:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.346+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f631c0697e0 con 0x7f631c103660 2024-04-16T05:27:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.346+0000 7f630f7fe700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6308039890 0x7f630803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.346+0000 7f630f7fe700 1 -- 172.21.15.86:0/2455587277 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f631804c250 con 0x7f631c103660 2024-04-16T05:27:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.346+0000 7f6321c49700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6308039890 0x7f630803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.347+0000 7f6321c49700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6308039890 0x7f630803bd50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f631000b5c0 tx=0x7f6310005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.351+0000 7f630f7fe700 1 -- 172.21.15.86:0/2455587277 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f631801f020 con 0x7f631c103660 2024-04-16T05:27:04.405 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:04 smithi194 ceph-mgr[37266]: -- 172.21.15.194:0/2455184409 <== mon.2 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5f662a5a0 con 0x55f5f65fec00 2024-04-16T05:27:04.405 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:04 smithi194 bash[37247]: 28612d49764f63fe771079b87205e4e117e3070b29b116438a80c87b4f71a89f 2024-04-16T05:27:04.406 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:04 smithi194 systemd[1]: Started Ceph mgr.y for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:27:04.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:04.567+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f631c062f30 con 0x7f6308039890 2024-04-16T05:27:04.828 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:04 smithi196 ceph-mon[36373]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:04.828 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:04 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.828 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:04 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.828 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:04 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.828 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:04 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.828 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:04 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:04 smithi194 ceph-mon[35865]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:04 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:04 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:04 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:04 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:04 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:04.906 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:04 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:04.623+0000 7fd6684c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:27:04.906 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:04 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:04.718+0000 7fd6684c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:27:04.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:04 smithi086 ceph-mon[32028]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:04.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:04.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:05.112 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:04.828+0000 7f82d4fc3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:27:05.112 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:04.970+0000 7f82d4fc3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:27:05.402 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:05.315+0000 7f82d4fc3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:27:05.402 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:05.403+0000 7f82d4fc3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:27:05.650 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:05 smithi194 ceph-mon[35865]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:05.650 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:05.651 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:05.651 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:05.651 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:05 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:05.402+0000 7fd6684c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:27:05.651 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:05 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:05.494+0000 7fd6684c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:27:05.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:05 smithi086 ceph-mon[32028]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:05.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:05.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:05.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:05.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:05 smithi196 ceph-mon[36373]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:05.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:05.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:05.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:05.735 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:05.607+0000 7f82d4fc3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:27:05.905 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:05 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:05.651+0000 7fd6684c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:27:05.905 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:05 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:05.749+0000 7fd6684c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:27:05.905 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:05 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:05.833+0000 7fd6684c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:27:06.070 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:05.789+0000 7f82d4fc3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:27:06.328 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:06.072+0000 7f82d4fc3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:27:06.328 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:06.330+0000 7f82d4fc3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:27:06.374 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:06.094+0000 7fd6684c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:27:06.374 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:06.200+0000 7fd6684c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:27:06.374 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:06.284+0000 7fd6684c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:27:06.374 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:06.375+0000 7fd6684c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:27:06.655 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:06.550+0000 7fd6684c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:27:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:06.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.735 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:06.563+0000 7f82d4fc3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:27:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:06.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:06.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:27:06.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:27:06.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:06.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:07.735 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:07.349+0000 7f82d4fc3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:27:07.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:07 smithi196 ceph-mon[36373]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T05:27:07.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:07 smithi196 ceph-mon[36373]: Reconfiguring daemon mgr.x on smithi086 2024-04-16T05:27:07.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:07 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3619173978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "249ee95d-f444-4c1b-aeae-e88da2c920fc"}]: dispatch 2024-04-16T05:27:07.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:07 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3619173978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "249ee95d-f444-4c1b-aeae-e88da2c920fc"}]': finished 2024-04-16T05:27:07.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:07 smithi196 ceph-mon[36373]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T05:27:07.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:07 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:07.905 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:07 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:07.502+0000 7fd6684c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:27:07.906 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:07 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:07.632+0000 7fd6684c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:27:07.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:07 smithi194 ceph-mon[35865]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T05:27:07.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:07 smithi194 ceph-mon[35865]: Reconfiguring daemon mgr.x on smithi086 2024-04-16T05:27:07.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:07 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3619173978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "249ee95d-f444-4c1b-aeae-e88da2c920fc"}]: dispatch 2024-04-16T05:27:07.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:07 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3619173978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "249ee95d-f444-4c1b-aeae-e88da2c920fc"}]': finished 2024-04-16T05:27:07.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:07 smithi194 ceph-mon[35865]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T05:27:07.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:07 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:07.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:07 smithi086 ceph-mon[32028]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T05:27:07.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:07 smithi086 ceph-mon[32028]: Reconfiguring daemon mgr.x on smithi086 2024-04-16T05:27:07.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:07 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3619173978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "249ee95d-f444-4c1b-aeae-e88da2c920fc"}]: dispatch 2024-04-16T05:27:07.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:07 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3619173978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "249ee95d-f444-4c1b-aeae-e88da2c920fc"}]': finished 2024-04-16T05:27:07.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:07 smithi086 ceph-mon[32028]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T05:27:07.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:08.215 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:07.911+0000 7f82d4fc3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:27:08.215 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:08.005+0000 7f82d4fc3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:27:08.215 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:27:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:27:08.216+0000 7f82d4fc3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:27:08.239 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:07 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:07.960+0000 7fd6684c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:27:08.239 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:08.047+0000 7fd6684c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:27:08.239 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:08.240+0000 7fd6684c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:27:08.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:08.515+0000 7f630f7fe700 1 -- 172.21.15.86:0/2455587277 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f6318021c20 con 0x7f631c103660 2024-04-16T05:27:08.517 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:08.423+0000 7fd6684c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:27:08.905 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:08.707+0000 7fd6684c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:27:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-mon[35865]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3546327611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-mon[35865]: Standby manager daemon z started 2024-04-16T05:27:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:27:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:27:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:27:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:27:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:08 smithi086 ceph-mon[32028]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:08 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3546327611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:08 smithi086 ceph-mon[32028]: Standby manager daemon z started 2024-04-16T05:27:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:08 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:27:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:08 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:27:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:08 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:27:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:08 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:27:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:08 smithi196 ceph-mon[36373]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:08 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3546327611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:08 smithi196 ceph-mon[36373]: Standby manager daemon z started 2024-04-16T05:27:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:08 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:27:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:08 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:27:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:08 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:27:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:08 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.196:0/1751764655' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:27:09.405 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:08 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:08.968+0000 7fd6684c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:27:09.405 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:09 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:09.206+0000 7fd6684c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:27:09.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:09 smithi194 ceph-mon[35865]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-16T05:27:09.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:09 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:27:09.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:09 smithi086 ceph-mon[32028]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-16T05:27:09.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:09 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:27:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:09 smithi196 ceph-mon[36373]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-16T05:27:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:09 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:27:10.405 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:10 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:10.007+0000 7fd6684c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:27:10.874 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:10 smithi194 ceph-mon[35865]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:10.874 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:10 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:10.572+0000 7fd6684c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:27:10.874 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:10 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:10.666+0000 7fd6684c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:27:10.874 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:27:10 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:27:10.875+0000 7fd6684c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:27:10.947 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:10 smithi086 ceph-mon[32028]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:10 smithi196 ceph-mon[36373]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:11.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:11.539+0000 7f630f7fe700 1 -- 172.21.15.86:0/2455587277 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f631801a400 con 0x7f631c103660 2024-04-16T05:27:11.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:11 smithi086 ceph-mon[32028]: Standby manager daemon y started 2024-04-16T05:27:11.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:11 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:27:11.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:11 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:27:11.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:11 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:27:11.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:11 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:27:11.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:11 smithi194 ceph-mon[35865]: Standby manager daemon y started 2024-04-16T05:27:11.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:11 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:27:11.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:11 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:27:11.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:11 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:27:11.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:11 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:27:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:11 smithi196 ceph-mon[36373]: Standby manager daemon y started 2024-04-16T05:27:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:11 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:27:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:11 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:27:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:11 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:27:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:11 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.194:0/164470202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:27:12.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:12 smithi086 ceph-mon[32028]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:12.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:12 smithi086 ceph-mon[32028]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-16T05:27:12.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:12 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:27:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:12 smithi194 ceph-mon[35865]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:12 smithi194 ceph-mon[35865]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-16T05:27:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:12 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:27:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:12 smithi196 ceph-mon[36373]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:12 smithi196 ceph-mon[36373]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-16T05:27:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:12 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:27:13.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:13 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:13.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:13 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:13.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:13 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:13.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:13 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:13.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:13 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:13.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:13 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:13.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:13 smithi086 ceph-mon[32028]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:13.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:13 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:13.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:13 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:13.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:13 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:13.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:13 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:13.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:13 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:13.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:13 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:13.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:13 smithi196 ceph-mon[36373]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:14.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:13 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:14.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:13 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:14.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:13 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:14.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:13 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:14.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:13 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:14.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:13 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:14.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:13 smithi194 ceph-mon[35865]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:15.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T05:27:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T05:27:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:15.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T05:27:15.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:16.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:16 smithi086 ceph-mon[32028]: Deploying daemon osd.0 on smithi086 2024-04-16T05:27:16.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:16 smithi086 ceph-mon[32028]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:16 smithi196 ceph-mon[36373]: Deploying daemon osd.0 on smithi086 2024-04-16T05:27:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:16 smithi196 ceph-mon[36373]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:16.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:16 smithi194 ceph-mon[35865]: Deploying daemon osd.0 on smithi086 2024-04-16T05:27:16.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:16 smithi194 ceph-mon[35865]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:18.705 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:18 smithi086 ceph-mon[32028]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:18 smithi196 ceph-mon[36373]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:18.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:18 smithi194 ceph-mon[35865]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:20.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:20 smithi086 ceph-mon[32028]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:20.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:20 smithi196 ceph-mon[36373]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:20.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:20 smithi194 ceph-mon[35865]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:21.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:21.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:21.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:22.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:22 smithi194 ceph-mon[35865]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:22.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:22 smithi086 ceph-mon[32028]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:22 smithi196 ceph-mon[36373]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:23.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:23 smithi194 ceph-mon[35865]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:23.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:23 smithi086 ceph-mon[32028]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:23 smithi196 ceph-mon[36373]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:24.026 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 0 on host 'smithi086' 2024-04-16T05:27:24.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.026+0000 7f630f7fe700 1 -- 172.21.15.86:0/2455587277 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f631c062f30 con 0x7f6308039890 2024-04-16T05:27:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6308039890 msgr2=0x7f630803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6308039890 0x7f630803bd50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f631000b5c0 tx=0x7f6310005fd0 comp rx=0 tx=0).stop 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c103660 msgr2=0x7f631c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c103660 0x7f631c102fc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f631800b530 tx=0x7f631800b840 comp rx=0 tx=0).stop 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 shutdown_connections 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f631c1088c0 0x7f631c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631c103660 0x7f631c102fc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6308039890 0x7f630803bd50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 --2- 172.21.15.86:0/2455587277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c105f90 0x7f631c101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 >> 172.21.15.86:0/2455587277 conn(0x7f631c0fd050 msgr2=0x7f631c103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 shutdown_connections 2024-04-16T05:27:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:27:24.029+0000 7f63246ae700 1 -- 172.21.15.86:0/2455587277 wait complete. 2024-04-16T05:27:24.779 DEBUG:teuthology.orchestra.run.smithi086:osd.0> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.0.service 2024-04-16T05:27:24.781 INFO:tasks.cephadm:Deploying osd.1 on smithi194 with /dev/vg_nvme/lv_4... 2024-04-16T05:27:24.781 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T05:27:24.999 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:25 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:24.999 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:25 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:24.999 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:25 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:24.999 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:25 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:24.999 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:25 smithi086 ceph-mon[32028]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T05:27:25.027 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:27:25.148 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:25 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:25.148 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:25 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:25.148 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:25 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:25.148 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:25 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:25.148 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:25 smithi194 ceph-mon[35865]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T05:27:25.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:25 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:25.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:25 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:25.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:25 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:25.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:25 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:25.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:25 smithi196 ceph-mon[36373]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T05:27:26.267 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:26 smithi194 ceph-mon[35865]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T05:27:26.267 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:26 smithi194 ceph-mon[35865]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T05:27:26.267 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:26 smithi194 ceph-mon[35865]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:27:26.267 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:26.267 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:26 smithi194 ceph-mon[35865]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:26.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:26 smithi086 ceph-mon[32028]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T05:27:26.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:26 smithi086 ceph-mon[32028]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T05:27:26.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:26 smithi086 ceph-mon[32028]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:27:26.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:26.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:26 smithi086 ceph-mon[32028]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:26 smithi196 ceph-mon[36373]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T05:27:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:26 smithi196 ceph-mon[36373]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T05:27:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:26 smithi196 ceph-mon[36373]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:27:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:26 smithi196 ceph-mon[36373]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:26.718 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:27:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0[44192]: 2024-04-16T05:27:26.507+0000 7fd24d393700 -1 osd.0 0 waiting for initial osdmap 2024-04-16T05:27:26.718 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:27:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0[44192]: 2024-04-16T05:27:26.510+0000 7fd24517e700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:27:26.773 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T05:27:26.773 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T05:27:26.773 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-16T05:27:26.773 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-16T05:27:26.773 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113062 s, 927 MB/s 2024-04-16T05:27:26.774 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-16T05:27:27.302 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_4 2024-04-16T05:27:27.330 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:27 smithi194 ceph-mon[35865]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:27:27.330 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:27 smithi194 ceph-mon[35865]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T05:27:27.330 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:27 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:27.330 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:27 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:27.330 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:27 smithi194 ceph-mon[35865]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' 2024-04-16T05:27:27.342 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:27 smithi086 ceph-mon[32028]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:27:27.342 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:27 smithi086 ceph-mon[32028]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T05:27:27.342 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:27 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:27.342 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:27 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:27.342 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:27 smithi086 ceph-mon[32028]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' 2024-04-16T05:27:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:27 smithi196 ceph-mon[36373]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:27:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:27 smithi196 ceph-mon[36373]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T05:27:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:27 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:27 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:27 smithi196 ceph-mon[36373]: from='osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251]' entity='osd.0' 2024-04-16T05:27:27.552 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] boot 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:28.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:28.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:28.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] boot 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.336 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:28.337 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:28.337 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: osd.0 [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] boot 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:29.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 -- 172.21.15.194:0/1069898120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 msgr2=0x7fa280102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:29.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 --2- 172.21.15.194:0/1069898120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 0x7fa280102f90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa2700099c0 tx=0x7fa270009cd0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 -- 172.21.15.194:0/1069898120 shutdown_connections 2024-04-16T05:27:29.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 --2- 172.21.15.194:0/1069898120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa280104710 0x7fa28010afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 --2- 172.21.15.194:0/1069898120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 0x7fa280102f90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 --2- 172.21.15.194:0/1069898120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa280103d70 0x7fa2801041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 -- 172.21.15.194:0/1069898120 >> 172.21.15.194:0/1069898120 conn(0x7fa2800fe110 msgr2=0x7fa280100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 -- 172.21.15.194:0/1069898120 shutdown_connections 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 -- 172.21.15.194:0/1069898120 wait complete. 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.085+0000 7fa285c1c700 1 Processor -- start 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa285c1c700 1 -- start start 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa285c1c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 0x7fa280198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa285c1c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa280103d70 0x7fa2801992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa285c1c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa280104710 0x7fa2801a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:29.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa285c1c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010e7a0 con 0x7fa280103d70 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa285c1c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010e620 con 0x7fa280102b70 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa285c1c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010e920 con 0x7fa280104710 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27f7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 0x7fa280198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27f7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 0x7fa280198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:41312/0 (socket says 172.21.15.194:41312) 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27f7fe700 1 -- 172.21.15.194:0/2506649612 learned_addr learned my addr 172.21.15.194:0/2506649612 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27effd700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa280103d70 0x7fa2801992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27ffff700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa280104710 0x7fa2801a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27f7fe700 1 -- 172.21.15.194:0/2506649612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa280104710 msgr2=0x7fa2801a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27f7fe700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa280104710 0x7fa2801a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27f7fe700 1 -- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa280103d70 msgr2=0x7fa2801992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27f7fe700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa280103d70 0x7fa2801992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27f7fe700 1 -- 172.21.15.194:0/2506649612 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2801a0a10 con 0x7fa280102b70 2024-04-16T05:27:29.088 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.086+0000 7fa27f7fe700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 0x7fa280198db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098e0 tx=0x7fa270009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:29.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.087+0000 7fa27cff9700 1 -- 172.21.15.194:0/2506649612 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270027070 con 0x7fa280102b70 2024-04-16T05:27:29.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.087+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2801a0ca0 con 0x7fa280102b70 2024-04-16T05:27:29.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.087+0000 7fa27cff9700 1 -- 172.21.15.194:0/2506649612 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa27000fbc0 con 0x7fa280102b70 2024-04-16T05:27:29.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.087+0000 7fa27cff9700 1 -- 172.21.15.194:0/2506649612 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270017650 con 0x7fa280102b70 2024-04-16T05:27:29.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.087+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2801a10f0 con 0x7fa280102b70 2024-04-16T05:27:29.090 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.088+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2800697e0 con 0x7fa280102b70 2024-04-16T05:27:29.090 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.089+0000 7fa27cff9700 1 -- 172.21.15.194:0/2506649612 <== mon.2 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fa27002a070 con 0x7fa280102b70 2024-04-16T05:27:29.090 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.089+0000 7fa27cff9700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa2680a3bb0 0x7fa2680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:29.090 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.089+0000 7fa27cff9700 1 -- 172.21.15.194:0/2506649612 <== mon.2 v2:172.21.15.194:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa2700cf850 con 0x7fa280102b70 2024-04-16T05:27:29.090 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.089+0000 7fa27effd700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa2680a3bb0 0x7fa2680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:29.090 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.090+0000 7fa27effd700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa2680a3bb0 0x7fa2680a6070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa274009bf0 tx=0x7fa274009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:29.092 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.094+0000 7fa27cff9700 1 -- 172.21.15.194:0/2506649612 <== mon.2 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa27005eff0 con 0x7fa280102b70 2024-04-16T05:27:29.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:29 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi086 2024-04-16T05:27:29.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:29 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi086 to 17088M 2024-04-16T05:27:29.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:29 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:29.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:29 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:29.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:29 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:29.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:29 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:29.301 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:29.302+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa280062f30 con 0x7fa2680a3bb0 2024-04-16T05:27:29.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:29 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi086 2024-04-16T05:27:29.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:29 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi086 to 17088M 2024-04-16T05:27:29.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:29.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:29.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:29.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:29.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:29 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi086 2024-04-16T05:27:29.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:29 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi086 to 17088M 2024-04-16T05:27:29.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:29 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:29.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:29 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:29.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:29 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:29.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:29 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:30.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:30 smithi194 ceph-mon[35865]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T05:27:30.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:30 smithi194 ceph-mon[35865]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:30.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:30.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:30.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:30.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:30 smithi194 ceph-mon[35865]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:30.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:30 smithi086 ceph-mon[32028]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T05:27:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:30 smithi086 ceph-mon[32028]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:30 smithi086 ceph-mon[32028]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:30 smithi196 ceph-mon[36373]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T05:27:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:30 smithi196 ceph-mon[36373]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:30 smithi196 ceph-mon[36373]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:32 smithi086 ceph-mon[32028]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:32 smithi086 ceph-mon[32028]: from='client.? 172.21.15.194:0/3686405744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfd22f89-9efb-4095-82f4-20ceb80af165"}]: dispatch 2024-04-16T05:27:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:32 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfd22f89-9efb-4095-82f4-20ceb80af165"}]: dispatch 2024-04-16T05:27:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:32 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfd22f89-9efb-4095-82f4-20ceb80af165"}]': finished 2024-04-16T05:27:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:32 smithi086 ceph-mon[32028]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T05:27:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:32 smithi086 ceph-mon[32028]: from='client.? 172.21.15.194:0/1839409189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:32 smithi196 ceph-mon[36373]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:32 smithi196 ceph-mon[36373]: from='client.? 172.21.15.194:0/3686405744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfd22f89-9efb-4095-82f4-20ceb80af165"}]: dispatch 2024-04-16T05:27:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:32 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfd22f89-9efb-4095-82f4-20ceb80af165"}]: dispatch 2024-04-16T05:27:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:32 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfd22f89-9efb-4095-82f4-20ceb80af165"}]': finished 2024-04-16T05:27:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:32 smithi196 ceph-mon[36373]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T05:27:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:32 smithi196 ceph-mon[36373]: from='client.? 172.21.15.194:0/1839409189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:32 smithi194 ceph-mon[35865]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:32 smithi194 ceph-mon[35865]: from='client.? 172.21.15.194:0/3686405744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfd22f89-9efb-4095-82f4-20ceb80af165"}]: dispatch 2024-04-16T05:27:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:32 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfd22f89-9efb-4095-82f4-20ceb80af165"}]: dispatch 2024-04-16T05:27:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:32 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfd22f89-9efb-4095-82f4-20ceb80af165"}]': finished 2024-04-16T05:27:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:32 smithi194 ceph-mon[35865]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T05:27:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:32 smithi194 ceph-mon[35865]: from='client.? 172.21.15.194:0/1839409189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:34.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:34 smithi196 ceph-mon[36373]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:34.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:34 smithi194 ceph-mon[35865]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:34.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:34 smithi086 ceph-mon[32028]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:36 smithi196 ceph-mon[36373]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:36.756 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:36 smithi194 ceph-mon[35865]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:36.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:36 smithi086 ceph-mon[32028]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:38.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:38 smithi196 ceph-mon[36373]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:38.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:38 smithi194 ceph-mon[35865]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:38.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:38 smithi086 ceph-mon[32028]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:39.558 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T05:27:39.558 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:39.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T05:27:39.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:39.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T05:27:39.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:40.648 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:40 smithi194 ceph-mon[35865]: Deploying daemon osd.1 on smithi194 2024-04-16T05:27:40.648 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:40 smithi194 ceph-mon[35865]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:40.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:40 smithi086 ceph-mon[32028]: Deploying daemon osd.1 on smithi194 2024-04-16T05:27:40.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:40 smithi086 ceph-mon[32028]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:40 smithi196 ceph-mon[36373]: Deploying daemon osd.1 on smithi194 2024-04-16T05:27:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:40 smithi196 ceph-mon[36373]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:42.755 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:42 smithi194 ceph-mon[35865]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:42.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:42 smithi086 ceph-mon[32028]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:42.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:42 smithi196 ceph-mon[36373]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:44.730 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:44 smithi194 ceph-mon[35865]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:44.730 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:44.730 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:44.730 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:44.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:44 smithi086 ceph-mon[32028]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:44 smithi196 ceph-mon[36373]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:46.795 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:46 smithi194 ceph-mon[35865]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:46.795 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:46 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:46.796 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:46 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:46.796 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:46 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:46.796 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:46 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:46.796 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:46 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:46.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:46 smithi086 ceph-mon[32028]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:46 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:46 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:46 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:46 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:46 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:46 smithi196 ceph-mon[36373]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:46 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:46 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:46 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:46 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:46 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:47.462 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.462+0000 7fa27cff9700 1 -- 172.21.15.194:0/2506649612 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa280062f30 con 0x7fa2680a3bb0 2024-04-16T05:27:47.462 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 1 on host 'smithi194' 2024-04-16T05:27:47.464 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa2680a3bb0 msgr2=0x7fa2680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:47.464 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa2680a3bb0 0x7fa2680a6070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa274009bf0 tx=0x7fa274009260 comp rx=0 tx=0).stop 2024-04-16T05:27:47.464 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 msgr2=0x7fa280198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:47.464 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 0x7fa280198db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098e0 tx=0x7fa270009580 comp rx=0 tx=0).stop 2024-04-16T05:27:47.464 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 shutdown_connections 2024-04-16T05:27:47.464 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa2680a3bb0 0x7fa2680a6070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:47.464 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa280104710 0x7fa2801a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:47.464 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa280102b70 0x7fa280198db0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:47.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 --2- 172.21.15.194:0/2506649612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa280103d70 0x7fa2801992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:47.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 >> 172.21.15.194:0/2506649612 conn(0x7fa2800fe110 msgr2=0x7fa280107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:47.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 shutdown_connections 2024-04-16T05:27:47.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:27:47.465+0000 7fa285c1c700 1 -- 172.21.15.194:0/2506649612 wait complete. 2024-04-16T05:27:48.213 DEBUG:teuthology.orchestra.run.smithi194:osd.1> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.1.service 2024-04-16T05:27:48.256 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-04-16T05:27:48.256 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T05:27:48.466 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:48.466 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:48 smithi194 ceph-mon[35865]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:48.466 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:48.466 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:48.466 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:48 smithi194 ceph-mon[35865]: from='osd.1 [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:48.466 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:48 smithi194 ceph-mon[35865]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:48.501 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:27:48.623 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:48.623 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:48 smithi196 ceph-mon[36373]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:48.623 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:48.623 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:48.623 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:48 smithi196 ceph-mon[36373]: from='osd.1 [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:48.623 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:48 smithi196 ceph-mon[36373]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:48.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:48.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:48 smithi086 ceph-mon[32028]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:48.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:48.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:48.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:48 smithi086 ceph-mon[32028]: from='osd.1 [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:48.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:48 smithi086 ceph-mon[32028]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:49 smithi196 ceph-mon[36373]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T05:27:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:49 smithi196 ceph-mon[36373]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T05:27:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:49 smithi196 ceph-mon[36373]: from='osd.1 [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:27:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:49 smithi196 ceph-mon[36373]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:27:49.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:49 smithi086 ceph-mon[32028]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T05:27:49.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:49 smithi086 ceph-mon[32028]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T05:27:49.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:49.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:49 smithi086 ceph-mon[32028]: from='osd.1 [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:27:49.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:49 smithi086 ceph-mon[32028]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:27:49.755 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:49 smithi194 ceph-mon[35865]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T05:27:49.755 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:49 smithi194 ceph-mon[35865]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T05:27:49.755 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:49.755 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:49 smithi194 ceph-mon[35865]: from='osd.1 [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:27:49.755 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:49 smithi194 ceph-mon[35865]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:27:50.156 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:27:49 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1[40479]: 2024-04-16T05:27:49.871+0000 7fb2fd622700 -1 osd.1 0 waiting for initial osdmap 2024-04-16T05:27:50.157 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:27:49 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1[40479]: 2024-04-16T05:27:49.875+0000 7fb2f540d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:27:50.329 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T05:27:50.329 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T05:27:50.329 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-16T05:27:50.329 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-16T05:27:50.329 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134251 s, 781 MB/s 2024-04-16T05:27:50.329 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-16T05:27:50.511 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:50 smithi196 ceph-mon[36373]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:50.511 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:50 smithi196 ceph-mon[36373]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:27:50.511 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:50 smithi196 ceph-mon[36373]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T05:27:50.512 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:50.512 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:50.512 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:50 smithi196 ceph-mon[36373]: from='osd.1 ' entity='osd.1' 2024-04-16T05:27:50.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:50 smithi086 ceph-mon[32028]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:50 smithi086 ceph-mon[32028]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:27:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:50 smithi086 ceph-mon[32028]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T05:27:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:50 smithi086 ceph-mon[32028]: from='osd.1 ' entity='osd.1' 2024-04-16T05:27:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:50 smithi194 ceph-mon[35865]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:50 smithi194 ceph-mon[35865]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:27:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:50 smithi194 ceph-mon[35865]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T05:27:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:50.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:50 smithi194 ceph-mon[35865]: from='osd.1 ' entity='osd.1' 2024-04-16T05:27:50.928 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-04-16T05:27:51.170 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:27:51.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: osd.1 [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] boot 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:51.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: osd.1 [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] boot 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:51.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:51.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: osd.1 [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] boot 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:51.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:27:52.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:52 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi194 2024-04-16T05:27:52.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:52 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi194 to 17080M 2024-04-16T05:27:52.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:52 smithi196 ceph-mon[36373]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:52.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:52 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi194 2024-04-16T05:27:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:52 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi194 to 17080M 2024-04-16T05:27:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:52 smithi086 ceph-mon[32028]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:52.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.763+0000 7fe8f686a700 1 -- 172.21.15.196:0/492242683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 msgr2=0x7fe8f0101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:52.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.763+0000 7fe8f686a700 1 --2- 172.21.15.196:0/492242683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 0x7fe8f0101e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009990 tx=0x7fe8e0009ca0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.763+0000 7fe8f686a700 1 -- 172.21.15.196:0/492242683 shutdown_connections 2024-04-16T05:27:52.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.763+0000 7fe8f686a700 1 --2- 172.21.15.196:0/492242683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f01035f0 0x7fe8f0109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.763+0000 7fe8f686a700 1 --2- 172.21.15.196:0/492242683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 0x7fe8f0101e70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.763+0000 7fe8f686a700 1 --2- 172.21.15.196:0/492242683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f0102c50 0x7fe8f01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.763+0000 7fe8f686a700 1 -- 172.21.15.196:0/492242683 >> 172.21.15.196:0/492242683 conn(0x7fe8f00fcff0 msgr2=0x7fe8f00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:52.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.763+0000 7fe8f686a700 1 -- 172.21.15.196:0/492242683 shutdown_connections 2024-04-16T05:27:52.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.763+0000 7fe8f686a700 1 -- 172.21.15.196:0/492242683 wait complete. 2024-04-16T05:27:52.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8f686a700 1 Processor -- start 2024-04-16T05:27:52.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8f686a700 1 -- start start 2024-04-16T05:27:52.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8f686a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 0x7fe8f006b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:52.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8f686a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f0102c50 0x7fe8f019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:52.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8f686a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f01035f0 0x7fe8f01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8f686a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010d630 con 0x7fe8f01035f0 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8f686a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010d4b0 con 0x7fe8f0102c50 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8f686a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010d7b0 con 0x7fe8f0101a50 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8effff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 0x7fe8f006b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8effff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 0x7fe8f006b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54078/0 (socket says 172.21.15.196:54078) 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.764+0000 7fe8effff700 1 -- 172.21.15.196:0/662146506 learned_addr learned my addr 172.21.15.196:0/662146506 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8f4e07700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f01035f0 0x7fe8f01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8ef7fe700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f0102c50 0x7fe8f019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8effff700 1 -- 172.21.15.196:0/662146506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f0102c50 msgr2=0x7fe8f019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8effff700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f0102c50 0x7fe8f019db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8effff700 1 -- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f01035f0 msgr2=0x7fe8f01a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:52.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8effff700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f01035f0 0x7fe8f01a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8effff700 1 -- 172.21.15.196:0/662146506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f01a0a60 con 0x7fe8f0101a50 2024-04-16T05:27:52.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8f4e07700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f01035f0 0x7fe8f01a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:27:52.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8effff700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 0x7fe8f006b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00098e0 tx=0x7fe8e0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:52.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8ed7fa700 1 -- 172.21.15.196:0/662146506 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0027070 con 0x7fe8f0101a50 2024-04-16T05:27:52.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f01a0cf0 con 0x7fe8f0101a50 2024-04-16T05:27:52.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8ed7fa700 1 -- 172.21.15.196:0/662146506 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e000fb40 con 0x7fe8f0101a50 2024-04-16T05:27:52.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8ed7fa700 1 -- 172.21.15.196:0/662146506 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00176e0 con 0x7fe8f0101a50 2024-04-16T05:27:52.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.765+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a1140 con 0x7fe8f0101a50 2024-04-16T05:27:52.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.766+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f00697e0 con 0x7fe8f0101a50 2024-04-16T05:27:52.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.767+0000 7fe8ed7fa700 1 -- 172.21.15.196:0/662146506 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fe8e002a070 con 0x7fe8f0101a50 2024-04-16T05:27:52.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.768+0000 7fe8ed7fa700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3bb0 0x7fe8d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:52.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.768+0000 7fe8ed7fa700 1 -- 172.21.15.196:0/662146506 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fe8e00cfab0 con 0x7fe8f0101a50 2024-04-16T05:27:52.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.768+0000 7fe8ef7fe700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3bb0 0x7fe8d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:52.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.769+0000 7fe8ef7fe700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3bb0 0x7fe8d80a6070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009b30 tx=0x7fe8e4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:52.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.773+0000 7fe8ed7fa700 1 -- 172.21.15.196:0/662146506 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8e0013070 con 0x7fe8f0101a50 2024-04-16T05:27:52.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:52 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi194 2024-04-16T05:27:52.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:52 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi194 to 17080M 2024-04-16T05:27:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:52 smithi194 ceph-mon[35865]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:52.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:27:52.976+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe8f0062f30 con 0x7fe8d80a3bb0 2024-04-16T05:27:53.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:53 smithi196 ceph-mon[36373]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T05:27:53.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:53.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:53.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:53.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:53 smithi086 ceph-mon[32028]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T05:27:53.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:53.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:53.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:53.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:53 smithi194 ceph-mon[35865]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T05:27:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:54.633 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:54 smithi196 ceph-mon[36373]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:54.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:54 smithi196 ceph-mon[36373]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:54.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:54 smithi086 ceph-mon[32028]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:54.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:54 smithi086 ceph-mon[32028]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:54.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:54 smithi194 ceph-mon[35865]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:54 smithi194 ceph-mon[35865]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:56.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:56 smithi086 ceph-mon[32028]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:56.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:56 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/484312748' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913b98ff-30fe-4214-877a-7a8cc4765ad6"}]: dispatch 2024-04-16T05:27:56.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:56 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913b98ff-30fe-4214-877a-7a8cc4765ad6"}]: dispatch 2024-04-16T05:27:56.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:56 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "913b98ff-30fe-4214-877a-7a8cc4765ad6"}]': finished 2024-04-16T05:27:56.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:56 smithi086 ceph-mon[32028]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T05:27:56.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:56.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:56 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/1699164542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:56 smithi196 ceph-mon[36373]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:56 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/484312748' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913b98ff-30fe-4214-877a-7a8cc4765ad6"}]: dispatch 2024-04-16T05:27:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:56 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913b98ff-30fe-4214-877a-7a8cc4765ad6"}]: dispatch 2024-04-16T05:27:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:56 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "913b98ff-30fe-4214-877a-7a8cc4765ad6"}]': finished 2024-04-16T05:27:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:56 smithi196 ceph-mon[36373]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T05:27:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:56 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/1699164542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:56.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:56 smithi194 ceph-mon[35865]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:56.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:56 smithi194 ceph-mon[35865]: from='client.? 172.21.15.196:0/484312748' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913b98ff-30fe-4214-877a-7a8cc4765ad6"}]: dispatch 2024-04-16T05:27:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:56 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913b98ff-30fe-4214-877a-7a8cc4765ad6"}]: dispatch 2024-04-16T05:27:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:56 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "913b98ff-30fe-4214-877a-7a8cc4765ad6"}]': finished 2024-04-16T05:27:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:56 smithi194 ceph-mon[35865]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T05:27:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:56 smithi194 ceph-mon[35865]: from='client.? 172.21.15.196:0/1699164542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:58.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:27:58 smithi086 ceph-mon[32028]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:58.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:27:58 smithi196 ceph-mon[36373]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:58.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:27:58 smithi194 ceph-mon[35865]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:00.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:00 smithi086 ceph-mon[32028]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:00.718 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:00 smithi196 ceph-mon[36373]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:00 smithi194 ceph-mon[35865]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:02.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:02 smithi086 ceph-mon[32028]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:02.719 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:02 smithi194 ceph-mon[35865]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:02 smithi196 ceph-mon[36373]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:03.581 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:03 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T05:28:03.581 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:03 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:03.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:03 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T05:28:03.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:03 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:03.901 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:03 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T05:28:03.902 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:03 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:04.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:04 smithi196 ceph-mon[36373]: Deploying daemon osd.2 on smithi196 2024-04-16T05:28:04.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:04 smithi196 ceph-mon[36373]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:04.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:04 smithi194 ceph-mon[35865]: Deploying daemon osd.2 on smithi196 2024-04-16T05:28:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:04 smithi194 ceph-mon[35865]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:04.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:04 smithi086 ceph-mon[32028]: Deploying daemon osd.2 on smithi196 2024-04-16T05:28:04.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:04 smithi086 ceph-mon[32028]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:06 smithi196 ceph-mon[36373]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:06.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:06 smithi194 ceph-mon[35865]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:06.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:06 smithi086 ceph-mon[32028]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:08.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:08 smithi196 ceph-mon[36373]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:08.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:08 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:08.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:08 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:08.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:08 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:08.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:08 smithi194 ceph-mon[35865]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:08.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:08 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:08 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:08 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:08 smithi086 ceph-mon[32028]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:08 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:08 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:08 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:10.612 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:10 smithi196 ceph-mon[36373]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:10.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:10 smithi194 ceph-mon[35865]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:10.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:10 smithi086 ceph-mon[32028]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:11.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.292+0000 7fe8ed7fa700 1 -- 172.21.15.196:0/662146506 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe8f0062f30 con 0x7fe8d80a3bb0 2024-04-16T05:28:11.292 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-04-16T05:28:11.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.295+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3bb0 msgr2=0x7fe8d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:11.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.295+0000 7fe8f686a700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3bb0 0x7fe8d80a6070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009b30 tx=0x7fe8e4009260 comp rx=0 tx=0).stop 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.295+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 msgr2=0x7fe8f006b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.295+0000 7fe8f686a700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 0x7fe8f006b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00098e0 tx=0x7fe8e0009580 comp rx=0 tx=0).stop 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.296+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 shutdown_connections 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.296+0000 7fe8f686a700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f01035f0 0x7fe8f01a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.296+0000 7fe8f686a700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3bb0 0x7fe8d80a6070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.296+0000 7fe8f686a700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101a50 0x7fe8f006b390 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.296+0000 7fe8f686a700 1 --2- 172.21.15.196:0/662146506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f0102c50 0x7fe8f019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.296+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 >> 172.21.15.196:0/662146506 conn(0x7fe8f00fcff0 msgr2=0x7fe8f0105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.296+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 shutdown_connections 2024-04-16T05:28:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:28:11.296+0000 7fe8f686a700 1 -- 172.21.15.196:0/662146506 wait complete. 2024-04-16T05:28:11.949 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.2.service 2024-04-16T05:28:11.952 INFO:tasks.cephadm:Deploying osd.3 on smithi086 with /dev/vg_nvme/lv_3... 2024-04-16T05:28:11.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T05:28:12.204 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:28:12.226 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:12 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2[41391]: 2024-04-16T05:28:12.034+0000 7f8aef9097c0 -1 osd.2 0 log_to_monitors true 2024-04-16T05:28:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:12 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:12 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:12 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:12 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:12 smithi196 ceph-mon[36373]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:12 smithi196 ceph-mon[36373]: from='osd.2 [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:28:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:12 smithi196 ceph-mon[36373]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:28:12.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:12 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:12 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:12 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:12 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:12 smithi086 ceph-mon[32028]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:12.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:12 smithi086 ceph-mon[32028]: from='osd.2 [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:28:12.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:12 smithi086 ceph-mon[32028]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:28:12.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:12 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:12 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:12 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:12 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:12.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:12 smithi194 ceph-mon[35865]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:12.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:12 smithi194 ceph-mon[35865]: from='osd.2 [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:28:12.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:12 smithi194 ceph-mon[35865]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:28:13.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:13 smithi086 ceph-mon[32028]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T05:28:13.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:13 smithi086 ceph-mon[32028]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T05:28:13.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:13 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:13.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:13 smithi086 ceph-mon[32028]: from='osd.2 [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:28:13.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:13 smithi086 ceph-mon[32028]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:28:13.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:13 smithi194 ceph-mon[35865]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T05:28:13.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:13 smithi194 ceph-mon[35865]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T05:28:13.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:13 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:13.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:13 smithi194 ceph-mon[35865]: from='osd.2 [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:28:13.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:13 smithi194 ceph-mon[35865]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:28:13.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:13 smithi196 ceph-mon[36373]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T05:28:13.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:13 smithi196 ceph-mon[36373]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T05:28:13.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:13 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:13.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:13 smithi196 ceph-mon[36373]: from='osd.2 [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:28:13.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:13 smithi196 ceph-mon[36373]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:28:14.110 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2[41391]: 2024-04-16T05:28:13.804+0000 7f8ae4f6e700 -1 osd.2 0 waiting for initial osdmap 2024-04-16T05:28:14.110 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2[41391]: 2024-04-16T05:28:13.808+0000 7f8adcd59700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:28:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T05:28:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T05:28:14.211 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-16T05:28:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-16T05:28:14.212 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114102 s, 919 MB/s 2024-04-16T05:28:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-16T05:28:14.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:14 smithi086 ceph-mon[32028]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:28:14.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:14 smithi086 ceph-mon[32028]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T05:28:14.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:14.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:14.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:14 smithi086 ceph-mon[32028]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:14.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:14 smithi086 ceph-mon[32028]: from='osd.2 ' entity='osd.2' 2024-04-16T05:28:14.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:14 smithi194 ceph-mon[35865]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:28:14.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:14 smithi194 ceph-mon[35865]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T05:28:14.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:14 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:14.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:14 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:14.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:14 smithi194 ceph-mon[35865]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:14.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:14 smithi194 ceph-mon[35865]: from='osd.2 ' entity='osd.2' 2024-04-16T05:28:14.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:14 smithi196 ceph-mon[36373]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:28:14.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:14 smithi196 ceph-mon[36373]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T05:28:14.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:14 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:14.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:14 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:14.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:14 smithi196 ceph-mon[36373]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:28:14.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:14 smithi196 ceph-mon[36373]: from='osd.2 ' entity='osd.2' 2024-04-16T05:28:14.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-04-16T05:28:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:15.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: osd.2 [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] boot 2024-04-16T05:28:15.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T05:28:15.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:15.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:15.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:15.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:15.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: osd.2 [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] boot 2024-04-16T05:28:15.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T05:28:15.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:15.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:15.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:15.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:15.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:28:15.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:28:15.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: osd.2 [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] boot 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:15.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:16.566 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:16 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi196 2024-04-16T05:28:16.566 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:16 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi196 to 17080M 2024-04-16T05:28:16.566 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:16 smithi086 ceph-mon[32028]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:16.566 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:16 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:28:16.566 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:16 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T05:28:16.566 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:16 smithi086 ceph-mon[32028]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T05:28:16.566 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:16 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:28:16.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:16 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi196 2024-04-16T05:28:16.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:16 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi196 to 17080M 2024-04-16T05:28:16.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:16 smithi194 ceph-mon[35865]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:16.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:16 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:28:16.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:16 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T05:28:16.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:16 smithi194 ceph-mon[35865]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T05:28:16.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:16 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:28:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:16 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi196 2024-04-16T05:28:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:16 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi196 to 17080M 2024-04-16T05:28:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:16 smithi196 ceph-mon[36373]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:16 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:28:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:16 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T05:28:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:16 smithi196 ceph-mon[36373]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T05:28:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:16 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:28:16.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.784+0000 7f5b36f18700 1 -- 172.21.15.86:0/2763574445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 msgr2=0x7f5b30102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:16.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.784+0000 7f5b36f18700 1 --2- 172.21.15.86:0/2763574445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 0x7f5b30102860 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0099c0 tx=0x7f5b1c009cd0 comp rx=0 tx=0).stop 2024-04-16T05:28:16.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.784+0000 7f5b36f18700 1 -- 172.21.15.86:0/2763574445 shutdown_connections 2024-04-16T05:28:16.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.784+0000 7f5b36f18700 1 --2- 172.21.15.86:0/2763574445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b30102da0 0x7f5b30109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:16.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.784+0000 7f5b36f18700 1 --2- 172.21.15.86:0/2763574445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b30101200 0x7f5b30101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:16.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.784+0000 7f5b36f18700 1 --2- 172.21.15.86:0/2763574445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 0x7f5b30102860 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:16.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.784+0000 7f5b36f18700 1 -- 172.21.15.86:0/2763574445 >> 172.21.15.86:0/2763574445 conn(0x7f5b300fc7a0 msgr2=0x7f5b300febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:28:16.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.784+0000 7f5b36f18700 1 -- 172.21.15.86:0/2763574445 shutdown_connections 2024-04-16T05:28:16.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.784+0000 7f5b36f18700 1 -- 172.21.15.86:0/2763574445 wait complete. 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b36f18700 1 Processor -- start 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b36f18700 1 -- start start 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b36f18700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b30101200 0x7f5b30105190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b36f18700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 0x7f5b301037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b36f18700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b30102da0 0x7f5b30103d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b36f18700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010ce60 con 0x7f5b30102400 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b36f18700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010cce0 con 0x7f5b30101200 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b36f18700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010cfe0 con 0x7f5b30102da0 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b2ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 0x7f5b301037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b2ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 0x7f5b301037e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43110/0 (socket says 172.21.15.86:43110) 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.785+0000 7f5b2ffff700 1 -- 172.21.15.86:0/1315390837 learned_addr learned my addr 172.21.15.86:0/1315390837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:28:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b34cb4700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b30101200 0x7f5b30105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b354b5700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b30102da0 0x7f5b30103d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b2ffff700 1 -- 172.21.15.86:0/1315390837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b30102da0 msgr2=0x7f5b30103d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b2ffff700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b30102da0 0x7f5b30103d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b2ffff700 1 -- 172.21.15.86:0/1315390837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b30101200 msgr2=0x7f5b30105190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b2ffff700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b30101200 0x7f5b30105190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b2ffff700 1 -- 172.21.15.86:0/1315390837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b301045d0 con 0x7f5b30102400 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b354b5700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b30102da0 0x7f5b30103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b2ffff700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 0x7f5b301037e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c003890 tx=0x7f5b1c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b2dffb700 1 -- 172.21.15.86:0/1315390837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c027070 con 0x7f5b30102400 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b2dffb700 1 -- 172.21.15.86:0/1315390837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b1c0041d0 con 0x7f5b30102400 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b301a2f00 con 0x7f5b30102400 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b2dffb700 1 -- 172.21.15.86:0/1315390837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00f460 con 0x7f5b30102400 2024-04-16T05:28:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.786+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b301a3320 con 0x7f5b30102400 2024-04-16T05:28:16.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.787+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b300697e0 con 0x7f5b30102400 2024-04-16T05:28:16.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.789+0000 7f5b2dffb700 1 -- 172.21.15.86:0/1315390837 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f5b1c02a070 con 0x7f5b30102400 2024-04-16T05:28:16.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.790+0000 7f5b2dffb700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5b200a3c30 0x7f5b200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:16.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.790+0000 7f5b34cb4700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5b200a3c30 0x7f5b200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:16.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.790+0000 7f5b2dffb700 1 -- 172.21.15.86:0/1315390837 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f5b1c0d0390 con 0x7f5b30102400 2024-04-16T05:28:16.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.790+0000 7f5b34cb4700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5b200a3c30 0x7f5b200a60f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b30102260 tx=0x7f5b24009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:28:16.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:16.794+0000 7f5b2dffb700 1 -- 172.21.15.86:0/1315390837 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b1c013070 con 0x7f5b30102400 2024-04-16T05:28:17.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:17.009+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5b30108850 con 0x7f5b200a3c30 2024-04-16T05:28:17.273 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48972]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:28:17.273 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48972]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.273 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48972]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.273 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48972]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.274 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48977]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:28:17.274 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.274 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48977]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.274 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48977]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44653]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:28:17.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44653]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44653]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44653]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44656]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:28:17.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44656]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44656]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44656]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44659]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:28:17.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44659]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44659]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44659]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44662]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:28:17.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44662]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44662]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 sudo[44662]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.485 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45174]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:28:17.485 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45174]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.485 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45174]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.485 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45174]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.485 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45177]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:28:17.485 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45177]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.485 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45177]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.485 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45177]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45180]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:28:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45180]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45180]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45180]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45183]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:28:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45183]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45183]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 sudo[45183]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48980]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:28:17.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48980]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48980]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48980]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:17.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48983]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:28:17.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48983]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:28:17.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48983]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:28:17.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 sudo[48983]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:28:18.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:28:18.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:18.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:18.237 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:17 smithi196 ceph-mon[36373]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:18.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:18.275+0000 7f5b2dffb700 1 -- 172.21.15.86:0/1315390837 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b1c09a190 con 0x7f5b30102400 2024-04-16T05:28:18.274 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:28:18.274 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:18.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:18.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:18.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:18.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:18.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:18.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:18.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:18.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:18.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:17 smithi086 ceph-mon[32028]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:18.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:28:18.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T05:28:18.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:18.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:18.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:18.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:18.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:18.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:18.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:18.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:18.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:18.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:18.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:18.408 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:17 smithi194 ceph-mon[35865]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:19.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:18 smithi086 ceph-mon[32028]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T05:28:19.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:18 smithi086 ceph-mon[32028]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T05:28:19.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:18 smithi196 ceph-mon[36373]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T05:28:19.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:18 smithi196 ceph-mon[36373]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T05:28:19.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:18 smithi194 ceph-mon[35865]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T05:28:19.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:18 smithi194 ceph-mon[35865]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T05:28:20.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:19 smithi194 ceph-mon[35865]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:20.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:19 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2479125568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63b377ce-0b9e-48a5-85c1-6903db7c60bc"}]: dispatch 2024-04-16T05:28:20.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:19 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2479125568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63b377ce-0b9e-48a5-85c1-6903db7c60bc"}]': finished 2024-04-16T05:28:20.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:19 smithi194 ceph-mon[35865]: osdmap e22: 4 total, 3 up, 4 in 2024-04-16T05:28:20.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:19 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:20.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:19 smithi086 ceph-mon[32028]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:20.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:19 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2479125568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63b377ce-0b9e-48a5-85c1-6903db7c60bc"}]: dispatch 2024-04-16T05:28:20.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:19 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2479125568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63b377ce-0b9e-48a5-85c1-6903db7c60bc"}]': finished 2024-04-16T05:28:20.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:19 smithi086 ceph-mon[32028]: osdmap e22: 4 total, 3 up, 4 in 2024-04-16T05:28:20.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:19 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:19 smithi196 ceph-mon[36373]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:19 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2479125568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63b377ce-0b9e-48a5-85c1-6903db7c60bc"}]: dispatch 2024-04-16T05:28:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:19 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2479125568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63b377ce-0b9e-48a5-85c1-6903db7c60bc"}]': finished 2024-04-16T05:28:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:19 smithi196 ceph-mon[36373]: osdmap e22: 4 total, 3 up, 4 in 2024-04-16T05:28:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:19 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:21.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:20 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/845378230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:21.406 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:20 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/845378230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:21.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:20 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/845378230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:22.324 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:21 smithi196 ceph-mon[36373]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:22.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:21 smithi194 ceph-mon[35865]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:22.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:21 smithi086 ceph-mon[32028]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:24.730 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:24 smithi086 ceph-mon[32028]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:24.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:24 smithi196 ceph-mon[36373]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:24.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:24 smithi194 ceph-mon[35865]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:26 smithi086 ceph-mon[32028]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:26.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:26 smithi196 ceph-mon[36373]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:26.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:26 smithi194 ceph-mon[35865]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:27 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T05:28:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:27 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:27.844 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:27 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T05:28:27.844 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:27 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:27.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:27 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T05:28:27.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:27 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:28 smithi196 ceph-mon[36373]: Deploying daemon osd.3 on smithi086 2024-04-16T05:28:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:28 smithi196 ceph-mon[36373]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:28.809 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:28 smithi086 ceph-mon[32028]: Deploying daemon osd.3 on smithi086 2024-04-16T05:28:28.809 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:28 smithi086 ceph-mon[32028]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:28.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:28 smithi194 ceph-mon[35865]: Deploying daemon osd.3 on smithi086 2024-04-16T05:28:28.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:28 smithi194 ceph-mon[35865]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:30.600 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:30 smithi086 ceph-mon[32028]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:30.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:30 smithi194 ceph-mon[35865]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:30 smithi196 ceph-mon[36373]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:32 smithi086 ceph-mon[32028]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:32 smithi196 ceph-mon[36373]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:32 smithi194 ceph-mon[35865]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:34.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:34 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:34.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:34 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:34.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:34 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:34.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:34 smithi194 ceph-mon[35865]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:34.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:34 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:34.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:34 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:34.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:34 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:34.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:34 smithi086 ceph-mon[32028]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:34.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:34 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:34.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:34 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:34.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:34 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:34.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:34 smithi196 ceph-mon[36373]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:36.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:36 smithi086 ceph-mon[32028]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:36.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:36.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:36.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:36.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:36.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:36 smithi196 ceph-mon[36373]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:36.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:36 smithi194 ceph-mon[35865]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:36.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:36.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:36.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:36.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:36.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.182+0000 7f5b2dffb700 1 -- 172.21.15.86:0/1315390837 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b30108850 con 0x7f5b200a3c30 2024-04-16T05:28:37.180 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 3 on host 'smithi086' 2024-04-16T05:28:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5b200a3c30 msgr2=0x7f5b200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5b200a3c30 0x7f5b200a60f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b30102260 tx=0x7f5b24009190 comp rx=0 tx=0).stop 2024-04-16T05:28:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 msgr2=0x7f5b301037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 0x7f5b301037e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c003890 tx=0x7f5b1c0047b0 comp rx=0 tx=0).stop 2024-04-16T05:28:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 shutdown_connections 2024-04-16T05:28:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5b200a3c30 0x7f5b200a60f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b30102da0 0x7f5b30103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b30101200 0x7f5b30105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 --2- 172.21.15.86:0/1315390837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30102400 0x7f5b301037e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.184+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 >> 172.21.15.86:0/1315390837 conn(0x7f5b300fc7a0 msgr2=0x7f5b300fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:28:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.185+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 shutdown_connections 2024-04-16T05:28:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:28:37.185+0000 7f5b36f18700 1 -- 172.21.15.86:0/1315390837 wait complete. 2024-04-16T05:28:37.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:37 smithi196 ceph-mon[36373]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T05:28:37.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:37 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:37.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:37 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:37.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:37 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:37.876 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:37 smithi086 ceph-mon[32028]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T05:28:37.876 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:37 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:37.876 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:37 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:37.876 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:37 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:37.905 DEBUG:teuthology.orchestra.run.smithi086:osd.3> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.3.service 2024-04-16T05:28:37.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:37 smithi194 ceph-mon[35865]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T05:28:37.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:37 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:37.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:37 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:37.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:37 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:37.908 INFO:tasks.cephadm:Deploying osd.4 on smithi194 with /dev/vg_nvme/lv_3... 2024-04-16T05:28:37.908 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T05:28:38.152 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:28:38.551 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:38 smithi194 ceph-mon[35865]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:38.551 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:38 smithi194 ceph-mon[35865]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T05:28:38.551 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:38 smithi194 ceph-mon[35865]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T05:28:38.551 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:38 smithi194 ceph-mon[35865]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:28:38.551 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:38 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:38.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:38 smithi086 ceph-mon[32028]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:38.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:38 smithi086 ceph-mon[32028]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T05:28:38.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:38 smithi086 ceph-mon[32028]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T05:28:38.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:38 smithi086 ceph-mon[32028]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:28:38.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:38 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:38.718 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:28:38 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3[51156]: 2024-04-16T05:28:38.499+0000 7fa48e814700 -1 osd.3 0 waiting for initial osdmap 2024-04-16T05:28:38.718 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:28:38 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3[51156]: 2024-04-16T05:28:38.503+0000 7fa48a607700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:28:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:38 smithi196 ceph-mon[36373]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:38 smithi196 ceph-mon[36373]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T05:28:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:38 smithi196 ceph-mon[36373]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T05:28:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:38 smithi196 ceph-mon[36373]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:28:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:38 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:39.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:39 smithi086 ceph-mon[32028]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:28:39.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:39 smithi086 ceph-mon[32028]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T05:28:39.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:39.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:39.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:39 smithi194 ceph-mon[35865]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:28:39.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:39 smithi194 ceph-mon[35865]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T05:28:39.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:39.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:39 smithi196 ceph-mon[36373]: from='osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:28:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:39 smithi196 ceph-mon[36373]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T05:28:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:40.018 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T05:28:40.018 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T05:28:40.018 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-16T05:28:40.018 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-16T05:28:40.018 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113438 s, 924 MB/s 2024-04-16T05:28:40.018 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-16T05:28:40.620 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:40 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:28:40.620 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:40 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:28:40.620 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:40 smithi194 ceph-mon[35865]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:40.620 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:40 smithi194 ceph-mon[35865]: osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] boot 2024-04-16T05:28:40.620 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:40 smithi194 ceph-mon[35865]: osdmap e25: 4 total, 4 up, 4 in 2024-04-16T05:28:40.620 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:40 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:40.620 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:40 smithi194 ceph-mon[35865]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T05:28:40.653 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_3 2024-04-16T05:28:40.896 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:28:40.900 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:40 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:28:40.900 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:40 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:28:40.900 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:40 smithi086 ceph-mon[32028]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:40.900 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:40 smithi086 ceph-mon[32028]: osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] boot 2024-04-16T05:28:40.900 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:40 smithi086 ceph-mon[32028]: osdmap e25: 4 total, 4 up, 4 in 2024-04-16T05:28:40.900 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:40 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:40.900 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:40 smithi086 ceph-mon[32028]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T05:28:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:40 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:28:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:40 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:28:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:40 smithi196 ceph-mon[36373]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:40 smithi196 ceph-mon[36373]: osd.3 [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] boot 2024-04-16T05:28:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:40 smithi196 ceph-mon[36373]: osdmap e25: 4 total, 4 up, 4 in 2024-04-16T05:28:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:40 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:40 smithi196 ceph-mon[36373]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi086 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:42.027 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.028 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:41 smithi194 ceph-mon[35865]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:42.119 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi086 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.120 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:41 smithi086 ceph-mon[32028]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T05:28:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi086 2024-04-16T05:28:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-16T05:28:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:42.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:42.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:42.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:41 smithi196 ceph-mon[36373]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:42.515 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.516+0000 7f251cdc7700 1 -- 172.21.15.194:0/1132747394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 msgr2=0x7f2518101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:42.515 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.516+0000 7f251cdc7700 1 --2- 172.21.15.194:0/1132747394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 0x7f2518101e70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f25080099c0 tx=0x7f2508009cd0 comp rx=0 tx=0).stop 2024-04-16T05:28:42.515 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.516+0000 7f251cdc7700 1 -- 172.21.15.194:0/1132747394 shutdown_connections 2024-04-16T05:28:42.515 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.516+0000 7f251cdc7700 1 --2- 172.21.15.194:0/1132747394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25181035f0 0x7f2518109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:42.515 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.516+0000 7f251cdc7700 1 --2- 172.21.15.194:0/1132747394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 0x7f2518101e70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:42.516 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.516+0000 7f251cdc7700 1 --2- 172.21.15.194:0/1132747394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2518102c50 0x7f25181030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:42.516 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.516+0000 7f251cdc7700 1 -- 172.21.15.194:0/1132747394 >> 172.21.15.194:0/1132747394 conn(0x7f25180fcff0 msgr2=0x7f25180ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:28:42.516 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.516+0000 7f251cdc7700 1 -- 172.21.15.194:0/1132747394 shutdown_connections 2024-04-16T05:28:42.516 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.516+0000 7f251cdc7700 1 -- 172.21.15.194:0/1132747394 wait complete. 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251cdc7700 1 Processor -- start 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251cdc7700 1 -- start start 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251cdc7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 0x7f251806bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251cdc7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518102c50 0x7f251819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251cdc7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25181035f0 0x7f25181a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251cdc7700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f251810d630 con 0x7f25181035f0 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251cdc7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f251810d4b0 con 0x7f2518101a50 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251cdc7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f251810d7b0 con 0x7f2518102c50 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 0x7f251806bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 0x7f251806bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:57786/0 (socket says 172.21.15.194:57786) 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251659c700 1 -- 172.21.15.194:0/3980036516 learned_addr learned my addr 172.21.15.194:0/3980036516 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f2516d9d700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25181035f0 0x7f25181a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:42.518 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f2515d9b700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518102c50 0x7f251819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251659c700 1 -- 172.21.15.194:0/3980036516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518102c50 msgr2=0x7f251819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251659c700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518102c50 0x7f251819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251659c700 1 -- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25181035f0 msgr2=0x7f25181a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251659c700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25181035f0 0x7f25181a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f251659c700 1 -- 172.21.15.194:0/3980036516 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25181a09e0 con 0x7f2518101a50 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f2516d9d700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25181035f0 0x7f25181a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.517+0000 7f2515d9b700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518102c50 0x7f251819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.518+0000 7f251659c700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 0x7f251806bd80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2508003790 tx=0x7f2508009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.518+0000 7f25077fe700 1 -- 172.21.15.194:0/3980036516 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508009e60 con 0x7f2518101a50 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.518+0000 7f25077fe700 1 -- 172.21.15.194:0/3980036516 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2508003e40 con 0x7f2518101a50 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.518+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25181a0c70 con 0x7f2518101a50 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.518+0000 7f25077fe700 1 -- 172.21.15.194:0/3980036516 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25080173f0 con 0x7f2518101a50 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.518+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25181a11a0 con 0x7f2518101a50 2024-04-16T05:28:42.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.519+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25180697e0 con 0x7f2518101a50 2024-04-16T05:28:42.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.520+0000 7f25077fe700 1 -- 172.21.15.194:0/3980036516 <== mon.2 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2508003960 con 0x7f2518101a50 2024-04-16T05:28:42.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.520+0000 7f25077fe700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f25000a3cc0 0x7f25000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:42.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.520+0000 7f25077fe700 1 -- 172.21.15.194:0/3980036516 <== mon.2 v2:172.21.15.194:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f25080d0150 con 0x7f2518101a50 2024-04-16T05:28:42.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.520+0000 7f2515d9b700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f25000a3cc0 0x7f25000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:42.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.521+0000 7f2515d9b700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f25000a3cc0 0x7f25000a6180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f251806c300 tx=0x7f250c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:28:42.524 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.525+0000 7f25077fe700 1 -- 172.21.15.194:0/3980036516 <== mon.2 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2508013070 con 0x7f2518101a50 2024-04-16T05:28:42.731 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:28:42.732+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2518062ff0 con 0x7f25000a3cc0 2024-04-16T05:28:43.138 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:42 smithi086 ceph-mon[32028]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T05:28:43.138 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:42 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:43.139 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:42 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:43.139 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:42 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:43.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:42 smithi194 ceph-mon[35865]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T05:28:43.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:42 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:43.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:42 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:43.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:42 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:43.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:42 smithi196 ceph-mon[36373]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T05:28:43.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:42 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:43.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:42 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:43.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:42 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:43.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:43 smithi194 ceph-mon[35865]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:43.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:43 smithi194 ceph-mon[35865]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T05:28:43.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:43 smithi194 ceph-mon[35865]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-16T05:28:44.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:43 smithi086 ceph-mon[32028]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:44.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:43 smithi086 ceph-mon[32028]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T05:28:44.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:43 smithi086 ceph-mon[32028]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-16T05:28:44.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:43 smithi196 ceph-mon[36373]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:44.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:43 smithi196 ceph-mon[36373]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T05:28:44.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:43 smithi196 ceph-mon[36373]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-16T05:28:45.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:45 smithi086 ceph-mon[32028]: from='client.? 172.21.15.194:0/2514476396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48c2fbf7-a6c1-4405-92e9-494e58e34298"}]: dispatch 2024-04-16T05:28:45.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:45 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48c2fbf7-a6c1-4405-92e9-494e58e34298"}]: dispatch 2024-04-16T05:28:45.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:45 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48c2fbf7-a6c1-4405-92e9-494e58e34298"}]': finished 2024-04-16T05:28:45.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:45 smithi086 ceph-mon[32028]: osdmap e30: 5 total, 4 up, 5 in 2024-04-16T05:28:45.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:45 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:45.728 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:45 smithi194 ceph-mon[35865]: from='client.? 172.21.15.194:0/2514476396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48c2fbf7-a6c1-4405-92e9-494e58e34298"}]: dispatch 2024-04-16T05:28:45.728 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:45 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48c2fbf7-a6c1-4405-92e9-494e58e34298"}]: dispatch 2024-04-16T05:28:45.728 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:45 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48c2fbf7-a6c1-4405-92e9-494e58e34298"}]': finished 2024-04-16T05:28:45.728 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:45 smithi194 ceph-mon[35865]: osdmap e30: 5 total, 4 up, 5 in 2024-04-16T05:28:45.728 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:45 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:45.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:45 smithi196 ceph-mon[36373]: from='client.? 172.21.15.194:0/2514476396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48c2fbf7-a6c1-4405-92e9-494e58e34298"}]: dispatch 2024-04-16T05:28:45.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:45 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48c2fbf7-a6c1-4405-92e9-494e58e34298"}]: dispatch 2024-04-16T05:28:45.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:45 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48c2fbf7-a6c1-4405-92e9-494e58e34298"}]': finished 2024-04-16T05:28:45.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:45 smithi196 ceph-mon[36373]: osdmap e30: 5 total, 4 up, 5 in 2024-04-16T05:28:45.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:45 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:46.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:46 smithi086 ceph-mon[32028]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-16T05:28:46.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:46 smithi086 ceph-mon[32028]: from='client.? 172.21.15.194:0/639431412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:46.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:46 smithi196 ceph-mon[36373]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-16T05:28:46.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:46 smithi196 ceph-mon[36373]: from='client.? 172.21.15.194:0/639431412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:46.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:46 smithi194 ceph-mon[35865]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-16T05:28:46.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:46 smithi194 ceph-mon[35865]: from='client.? 172.21.15.194:0/639431412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:48.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:48 smithi086 ceph-mon[32028]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T05:28:48.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:48 smithi196 ceph-mon[36373]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T05:28:48.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:48 smithi194 ceph-mon[35865]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T05:28:50.613 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:50 smithi194 ceph-mon[35865]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:50.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:50 smithi086 ceph-mon[32028]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:50.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:50 smithi196 ceph-mon[36373]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:52.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:52 smithi194 ceph-mon[35865]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:52.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:52 smithi086 ceph-mon[32028]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:52 smithi196 ceph-mon[36373]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:53.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T05:28:53.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:53.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T05:28:53.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:53.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T05:28:53.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:54.505 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:54 smithi194 ceph-mon[35865]: Deploying daemon osd.4 on smithi194 2024-04-16T05:28:54.506 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:54 smithi194 ceph-mon[35865]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:54.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:54 smithi086 ceph-mon[32028]: Deploying daemon osd.4 on smithi194 2024-04-16T05:28:54.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:54 smithi086 ceph-mon[32028]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:54 smithi196 ceph-mon[36373]: Deploying daemon osd.4 on smithi194 2024-04-16T05:28:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:54 smithi196 ceph-mon[36373]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:56.625 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:56 smithi194 ceph-mon[35865]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:56.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:56 smithi086 ceph-mon[32028]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:56 smithi196 ceph-mon[36373]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:58.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:58 smithi086 ceph-mon[32028]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:58.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:58.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:58.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:28:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:58.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:58 smithi196 ceph-mon[36373]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:58.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:58.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:58.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:28:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:58.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:58 smithi194 ceph-mon[35865]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:58.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:58.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:28:58.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:28:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:00.558 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:00 smithi194 ceph-mon[35865]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:00.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:00 smithi086 ceph-mon[32028]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:00.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:00 smithi196 ceph-mon[36373]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:01.795 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.796+0000 7f25077fe700 1 -- 172.21.15.194:0/3980036516 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2518062ff0 con 0x7f25000a3cc0 2024-04-16T05:29:01.795 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 4 on host 'smithi194' 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f25000a3cc0 msgr2=0x7f25000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f25000a3cc0 0x7f25000a6180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f251806c300 tx=0x7f250c009260 comp rx=0 tx=0).stop 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 msgr2=0x7f251806bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 0x7f251806bd80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2508003790 tx=0x7f2508009580 comp rx=0 tx=0).stop 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 shutdown_connections 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25181035f0 0x7f25181a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2518101a50 0x7f251806bd80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f25000a3cc0 0x7f25000a6180 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 --2- 172.21.15.194:0/3980036516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518102c50 0x7f251819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 >> 172.21.15.194:0/3980036516 conn(0x7f25180fcff0 msgr2=0x7f2518106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 shutdown_connections 2024-04-16T05:29:01.798 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:01.799+0000 7f251cdc7700 1 -- 172.21.15.194:0/3980036516 wait complete. 2024-04-16T05:29:02.415 DEBUG:teuthology.orchestra.run.smithi194:osd.4> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.4.service 2024-04-16T05:29:02.417 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-04-16T05:29:02.417 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T05:29:02.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:02 smithi194 ceph-mon[35865]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:02.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:02 smithi194 ceph-mon[35865]: from='osd.4 [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:29:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:02 smithi194 ceph-mon[35865]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:29:02.663 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:29:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:02 smithi086 ceph-mon[32028]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:02 smithi086 ceph-mon[32028]: from='osd.4 [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:29:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:02 smithi086 ceph-mon[32028]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:29:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:02 smithi196 ceph-mon[36373]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:02 smithi196 ceph-mon[36373]: from='osd.4 [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:29:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:02 smithi196 ceph-mon[36373]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:29:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:03 smithi196 ceph-mon[36373]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T05:29:03.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:03 smithi196 ceph-mon[36373]: from='osd.4 [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:29:03.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:03 smithi196 ceph-mon[36373]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T05:29:03.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:03 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:03.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:03 smithi196 ceph-mon[36373]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:29:03.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:03 smithi196 ceph-mon[36373]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:04.122 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:03 smithi194 ceph-mon[35865]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T05:29:04.122 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:03 smithi194 ceph-mon[35865]: from='osd.4 [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:29:04.123 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:03 smithi194 ceph-mon[35865]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T05:29:04.123 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:03 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:04.123 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:03 smithi194 ceph-mon[35865]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:29:04.123 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:03 smithi194 ceph-mon[35865]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:04.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:03 smithi086 ceph-mon[32028]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T05:29:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:03 smithi086 ceph-mon[32028]: from='osd.4 [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:29:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:03 smithi086 ceph-mon[32028]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T05:29:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:03 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:03 smithi086 ceph-mon[32028]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:29:04.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:03 smithi086 ceph-mon[32028]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:04.540 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T05:29:04.541 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T05:29:04.541 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-16T05:29:04.541 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-16T05:29:04.541 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112982 s, 928 MB/s 2024-04-16T05:29:04.541 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-16T05:29:04.656 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:29:04 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4[47467]: 2024-04-16T05:29:04.396+0000 7f588751a700 -1 osd.4 0 waiting for initial osdmap 2024-04-16T05:29:04.656 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:29:04 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4[47467]: 2024-04-16T05:29:04.402+0000 7f587f305700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:29:05.042 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:04 smithi194 ceph-mon[35865]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:29:05.042 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:04 smithi194 ceph-mon[35865]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T05:29:05.043 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:04 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:05.043 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:04 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:05.043 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:04 smithi194 ceph-mon[35865]: from='osd.4 ' entity='osd.4' 2024-04-16T05:29:05.166 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:04 smithi196 ceph-mon[36373]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:29:05.167 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:04 smithi196 ceph-mon[36373]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T05:29:05.167 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:04 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:05.167 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:04 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:05.167 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:04 smithi196 ceph-mon[36373]: from='osd.4 ' entity='osd.4' 2024-04-16T05:29:05.195 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-04-16T05:29:05.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:04 smithi086 ceph-mon[32028]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:29:05.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:04 smithi086 ceph-mon[32028]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T05:29:05.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:05.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:05.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:04 smithi086 ceph-mon[32028]: from='osd.4 ' entity='osd.4' 2024-04-16T05:29:05.445 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: osd.4 [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] boot 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: osdmap e33: 5 total, 5 up, 5 in 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi194 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi194 to 8540M 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:06.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: osd.4 [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] boot 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: osdmap e33: 5 total, 5 up, 5 in 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi194 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi194 to 8540M 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.230 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:29:06.230 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:29:06.230 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:06.230 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: osd.4 [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] boot 2024-04-16T05:29:06.230 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: osdmap e33: 5 total, 5 up, 5 in 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi194 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi194 to 8540M 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:06.231 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:06.232 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:07.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.072+0000 7fe5e0a94700 1 -- 172.21.15.196:0/4074758833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc076aa0 msgr2=0x7fe5dc074f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.072+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/4074758833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc076aa0 0x7fe5dc074f00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009990 tx=0x7fe5cc009ca0 comp rx=0 tx=0).stop 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.072+0000 7fe5e0a94700 1 -- 172.21.15.196:0/4074758833 shutdown_connections 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.072+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/4074758833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5dc10aa80 0x7fe5dc10cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.072+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/4074758833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc076aa0 0x7fe5dc074f00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.072+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/4074758833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc0754d0 0x7fe5dc075950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.072+0000 7fe5e0a94700 1 -- 172.21.15.196:0/4074758833 >> 172.21.15.196:0/4074758833 conn(0x7fe5dc0fe140 msgr2=0x7fe5dc100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.072+0000 7fe5e0a94700 1 -- 172.21.15.196:0/4074758833 shutdown_connections 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.072+0000 7fe5e0a94700 1 -- 172.21.15.196:0/4074758833 wait complete. 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5e0a94700 1 Processor -- start 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5e0a94700 1 -- start start 2024-04-16T05:29:07.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5e0a94700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc0754d0 0x7fe5dc19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5e0a94700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5dc076aa0 0x7fe5dc19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5e0a94700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc10aa80 0x7fe5dc1a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5e0a94700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc110920 con 0x7fe5dc10aa80 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5e0a94700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1107a0 con 0x7fe5dc076aa0 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5e0a94700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc110aa0 con 0x7fe5dc0754d0 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5da59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc0754d0 0x7fe5dc19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5da59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc0754d0 0x7fe5dc19aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35444/0 (socket says 172.21.15.196:35444) 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5da59c700 1 -- 172.21.15.196:0/409530713 learned_addr learned my addr 172.21.15.196:0/409530713 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5dad9d700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc10aa80 0x7fe5dc1a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5d1d9b700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5dc076aa0 0x7fe5dc19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5da59c700 1 -- 172.21.15.196:0/409530713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5dc076aa0 msgr2=0x7fe5dc19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:07.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5da59c700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5dc076aa0 0x7fe5dc19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:07.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.073+0000 7fe5da59c700 1 -- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc10aa80 msgr2=0x7fe5dc1a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:07.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.074+0000 7fe5da59c700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc10aa80 0x7fe5dc1a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:07.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.074+0000 7fe5da59c700 1 -- 172.21.15.196:0/409530713 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc1a2b40 con 0x7fe5dc0754d0 2024-04-16T05:29:07.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.074+0000 7fe5da59c700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc0754d0 0x7fe5dc19aee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc004790 tx=0x7fe5cc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:07.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.074+0000 7fe5d3fff700 1 -- 172.21.15.196:0/409530713 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc027070 con 0x7fe5dc0754d0 2024-04-16T05:29:07.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.074+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc1a2dd0 con 0x7fe5dc0754d0 2024-04-16T05:29:07.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.074+0000 7fe5d3fff700 1 -- 172.21.15.196:0/409530713 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5cc004150 con 0x7fe5dc0754d0 2024-04-16T05:29:07.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.074+0000 7fe5d3fff700 1 -- 172.21.15.196:0/409530713 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc022e00 con 0x7fe5dc0754d0 2024-04-16T05:29:07.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.074+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1a3170 con 0x7fe5dc0754d0 2024-04-16T05:29:07.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.075+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5dc02dfe0 con 0x7fe5dc0754d0 2024-04-16T05:29:07.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.076+0000 7fe5d3fff700 1 -- 172.21.15.196:0/409530713 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe5cc02a070 con 0x7fe5dc0754d0 2024-04-16T05:29:07.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.076+0000 7fe5d3fff700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe5c80a3c40 0x7fe5c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:07.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.076+0000 7fe5d3fff700 1 -- 172.21.15.196:0/409530713 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fe5cc0d0010 con 0x7fe5dc0754d0 2024-04-16T05:29:07.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.077+0000 7fe5d1d9b700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe5c80a3c40 0x7fe5c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:07.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.077+0000 7fe5d1d9b700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe5c80a3c40 0x7fe5c80a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4005950 tx=0x7fe5c400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:07.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.081+0000 7fe5d3fff700 1 -- 172.21.15.196:0/409530713 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5cc013070 con 0x7fe5dc0754d0 2024-04-16T05:29:07.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:07.292+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe5dc062f30 con 0x7fe5c80a3c40 2024-04-16T05:29:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:07 smithi194 ceph-mon[35865]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T05:29:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:07 smithi194 ceph-mon[35865]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:07 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:07 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:07 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:07 smithi194 ceph-mon[35865]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:08.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:07 smithi196 ceph-mon[36373]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T05:29:08.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:07 smithi196 ceph-mon[36373]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:08.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:07 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:08.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:07 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:08.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:07 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:08.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:07 smithi196 ceph-mon[36373]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:08.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:07 smithi086 ceph-mon[32028]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T05:29:08.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:07 smithi086 ceph-mon[32028]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:08.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:08.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:08.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:07 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:08.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:07 smithi086 ceph-mon[32028]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:10 smithi196 ceph-mon[36373]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:10 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/3361071951' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e22bf80-acb0-4744-b7f5-94ae9032b5d8"}]: dispatch 2024-04-16T05:29:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:10 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e22bf80-acb0-4744-b7f5-94ae9032b5d8"}]: dispatch 2024-04-16T05:29:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:10 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e22bf80-acb0-4744-b7f5-94ae9032b5d8"}]': finished 2024-04-16T05:29:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:10 smithi196 ceph-mon[36373]: osdmap e35: 6 total, 5 up, 6 in 2024-04-16T05:29:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:10 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:10 smithi194 ceph-mon[35865]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:10 smithi194 ceph-mon[35865]: from='client.? 172.21.15.196:0/3361071951' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e22bf80-acb0-4744-b7f5-94ae9032b5d8"}]: dispatch 2024-04-16T05:29:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:10 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e22bf80-acb0-4744-b7f5-94ae9032b5d8"}]: dispatch 2024-04-16T05:29:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:10 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e22bf80-acb0-4744-b7f5-94ae9032b5d8"}]': finished 2024-04-16T05:29:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:10 smithi194 ceph-mon[35865]: osdmap e35: 6 total, 5 up, 6 in 2024-04-16T05:29:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:10 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:10.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:10 smithi086 ceph-mon[32028]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:10.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:10 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/3361071951' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e22bf80-acb0-4744-b7f5-94ae9032b5d8"}]: dispatch 2024-04-16T05:29:10.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:10 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e22bf80-acb0-4744-b7f5-94ae9032b5d8"}]: dispatch 2024-04-16T05:29:10.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:10 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e22bf80-acb0-4744-b7f5-94ae9032b5d8"}]': finished 2024-04-16T05:29:10.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:10 smithi086 ceph-mon[32028]: osdmap e35: 6 total, 5 up, 6 in 2024-04-16T05:29:10.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:10 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:11.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:11 smithi194 ceph-mon[35865]: from='client.? 172.21.15.196:0/1529003407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:11.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:11 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/1529003407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:11 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/1529003407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:12.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:12 smithi086 ceph-mon[32028]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:12 smithi194 ceph-mon[35865]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:12 smithi196 ceph-mon[36373]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:14.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:14 smithi196 ceph-mon[36373]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:14.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:14 smithi194 ceph-mon[35865]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:14.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:14 smithi086 ceph-mon[32028]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:16 smithi196 ceph-mon[36373]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:16.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:16 smithi194 ceph-mon[35865]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:16.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:16 smithi086 ceph-mon[32028]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:17.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T05:29:17.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:17 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:17.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T05:29:17.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:17 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:17.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T05:29:17.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:17 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:18 smithi196 ceph-mon[36373]: Deploying daemon osd.5 on smithi196 2024-04-16T05:29:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:18 smithi196 ceph-mon[36373]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:18.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:18 smithi194 ceph-mon[35865]: Deploying daemon osd.5 on smithi196 2024-04-16T05:29:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:18 smithi194 ceph-mon[35865]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:18.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:18 smithi086 ceph-mon[32028]: Deploying daemon osd.5 on smithi196 2024-04-16T05:29:18.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:18 smithi086 ceph-mon[32028]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:20.690 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:20 smithi196 ceph-mon[36373]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:20.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:20 smithi194 ceph-mon[35865]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:20.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:20 smithi086 ceph-mon[32028]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:22.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:22 smithi196 ceph-mon[36373]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:22.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:22 smithi194 ceph-mon[35865]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:22.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:22 smithi086 ceph-mon[32028]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:24.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:24 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:24.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:24 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:24.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:24 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:24.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:24 smithi194 ceph-mon[35865]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:24.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:24 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:24.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:24 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:24 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:24 smithi086 ceph-mon[32028]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:24 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:24 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:24 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:24 smithi196 ceph-mon[36373]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:26.506 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.507+0000 7fe5d3fff700 1 -- 172.21.15.196:0/409530713 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe5dc062f30 con 0x7fe5c80a3c40 2024-04-16T05:29:26.507 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-04-16T05:29:26.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.509+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe5c80a3c40 msgr2=0x7fe5c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:26.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.509+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe5c80a3c40 0x7fe5c80a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4005950 tx=0x7fe5c400b410 comp rx=0 tx=0).stop 2024-04-16T05:29:26.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.509+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc0754d0 msgr2=0x7fe5dc19aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.509+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc0754d0 0x7fe5dc19aee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc004790 tx=0x7fe5cc009580 comp rx=0 tx=0).stop 2024-04-16T05:29:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.509+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 shutdown_connections 2024-04-16T05:29:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.509+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc10aa80 0x7fe5dc1a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.509+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe5c80a3c40 0x7fe5c80a6100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.509+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5dc0754d0 0x7fe5dc19aee0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.509+0000 7fe5e0a94700 1 --2- 172.21.15.196:0/409530713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5dc076aa0 0x7fe5dc19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.510+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 >> 172.21.15.196:0/409530713 conn(0x7fe5dc0fe140 msgr2=0x7fe5dc1069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.510+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 shutdown_connections 2024-04-16T05:29:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:29:26.510+0000 7fe5e0a94700 1 -- 172.21.15.196:0/409530713 wait complete. 2024-04-16T05:29:26.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:26 smithi196 ceph-mon[36373]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:26.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:26.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:26 smithi194 ceph-mon[35865]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:26.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:26 smithi086 ceph-mon[32028]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:26.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:26.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:27.193 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.5.service 2024-04-16T05:29:27.196 INFO:tasks.cephadm:Deploying osd.6 on smithi086 with /dev/vg_nvme/lv_2... 2024-04-16T05:29:27.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T05:29:27.445 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:29:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:27 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:27 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:27 smithi196 ceph-mon[36373]: from='osd.5 [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:27 smithi196 ceph-mon[36373]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:27.858 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:27 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:27.858 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:27 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:27.858 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:27 smithi086 ceph-mon[32028]: from='osd.5 [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:27.858 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:27 smithi086 ceph-mon[32028]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:27.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:27 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:27 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:27 smithi194 ceph-mon[35865]: from='osd.5 [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:27 smithi194 ceph-mon[35865]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:28.882 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:28 smithi196 ceph-mon[36373]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:28.882 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:28 smithi196 ceph-mon[36373]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T05:29:28.882 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:28 smithi196 ceph-mon[36373]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T05:29:28.882 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:28.882 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:28 smithi196 ceph-mon[36373]: from='osd.5 [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:29:28.882 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:28 smithi196 ceph-mon[36373]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:29:28.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:28 smithi194 ceph-mon[35865]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:28.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:28 smithi194 ceph-mon[35865]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T05:29:28.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:28 smithi194 ceph-mon[35865]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T05:29:28.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:28.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:28 smithi194 ceph-mon[35865]: from='osd.5 [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:29:28.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:28 smithi194 ceph-mon[35865]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:29:28.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:28 smithi086 ceph-mon[32028]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:28.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:28 smithi086 ceph-mon[32028]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T05:29:28.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:28 smithi086 ceph-mon[32028]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T05:29:28.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:28.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:28 smithi086 ceph-mon[32028]: from='osd.5 [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:29:28.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:28 smithi086 ceph-mon[32028]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:29:29.236 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:29:29 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:29:29.090+0000 7f7151451700 -1 osd.5 0 waiting for initial osdmap 2024-04-16T05:29:29.236 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:29:29 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:29:29.096+0000 7f714923c700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:29:29.542 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T05:29:29.542 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T05:29:29.542 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-16T05:29:29.542 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-16T05:29:29.542 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133469 s, 786 MB/s 2024-04-16T05:29:29.542 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-16T05:29:29.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:29 smithi196 ceph-mon[36373]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:29:29.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:29 smithi196 ceph-mon[36373]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T05:29:29.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:29 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:29.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:29 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:29.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:29 smithi196 ceph-mon[36373]: from='osd.5 ' entity='osd.5' 2024-04-16T05:29:29.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:29 smithi196 ceph-mon[36373]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:29.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:29 smithi086 ceph-mon[32028]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:29:29.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:29 smithi086 ceph-mon[32028]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T05:29:29.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:29.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:29.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:29 smithi086 ceph-mon[32028]: from='osd.5 ' entity='osd.5' 2024-04-16T05:29:29.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:29 smithi086 ceph-mon[32028]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:29.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:29 smithi194 ceph-mon[35865]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:29:29.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:29 smithi194 ceph-mon[35865]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T05:29:29.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:29 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:29.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:29 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:29.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:29 smithi194 ceph-mon[35865]: from='osd.5 ' entity='osd.5' 2024-04-16T05:29:29.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:29 smithi194 ceph-mon[35865]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:30.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-04-16T05:29:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:29:30.730 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: osd.5 [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] boot 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: osdmap e38: 6 total, 6 up, 6 in 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi196 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:30.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi196 to 8540M 2024-04-16T05:29:30.732 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.732 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:30.732 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:30.732 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.732 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:30.732 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:30.732 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:30.733 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:29:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:29:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: osd.5 [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] boot 2024-04-16T05:29:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: osdmap e38: 6 total, 6 up, 6 in 2024-04-16T05:29:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T05:29:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi196 2024-04-16T05:29:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi196 to 8540M 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:30.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:29:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:29:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: osd.5 [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] boot 2024-04-16T05:29:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: osdmap e38: 6 total, 6 up, 6 in 2024-04-16T05:29:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi196 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi196 to 8540M 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:30.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:30.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:31.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:31 smithi086 ceph-mon[32028]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T05:29:31.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:31 smithi086 ceph-mon[32028]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:32.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:31 smithi194 ceph-mon[35865]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T05:29:32.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:31 smithi194 ceph-mon[35865]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.185+0000 7fd1cfa28700 1 -- 172.21.15.86:0/867923406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8103d60 msgr2=0x7fd1c81041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.185+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/867923406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8103d60 0x7fd1c81041c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80099c0 tx=0x7fd1b8009cd0 comp rx=0 tx=0).stop 2024-04-16T05:29:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.185+0000 7fd1cfa28700 1 -- 172.21.15.86:0/867923406 shutdown_connections 2024-04-16T05:29:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.185+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/867923406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8104700 0x7fd1c810afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.185+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/867923406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8102b60 0x7fd1c8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.185+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/867923406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8103d60 0x7fd1c81041c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.185+0000 7fd1cfa28700 1 -- 172.21.15.86:0/867923406 >> 172.21.15.86:0/867923406 conn(0x7fd1c80fe100 msgr2=0x7fd1c8100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.185+0000 7fd1cfa28700 1 -- 172.21.15.86:0/867923406 shutdown_connections 2024-04-16T05:29:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.185+0000 7fd1cfa28700 1 -- 172.21.15.86:0/867923406 wait complete. 2024-04-16T05:29:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cfa28700 1 Processor -- start 2024-04-16T05:29:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cfa28700 1 -- start start 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cfa28700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8102b60 0x7fd1c8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cfa28700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8103d60 0x7fd1c81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cfa28700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8104700 0x7fd1c81a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cfa28700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c810e7c0 con 0x7fd1c8102b60 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cfa28700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c810e640 con 0x7fd1c8103d60 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cfa28700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c810e940 con 0x7fd1c8104700 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cd7c4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8102b60 0x7fd1c8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cd7c4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8102b60 0x7fd1c8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50908/0 (socket says 172.21.15.86:50908) 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cd7c4700 1 -- 172.21.15.86:0/542811119 learned_addr learned my addr 172.21.15.86:0/542811119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1ccfc3700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8103d60 0x7fd1c81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cd7c4700 1 -- 172.21.15.86:0/542811119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8104700 msgr2=0x7fd1c81a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cdfc5700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8104700 0x7fd1c81a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cd7c4700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8104700 0x7fd1c81a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cd7c4700 1 -- 172.21.15.86:0/542811119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8103d60 msgr2=0x7fd1c81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cd7c4700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8103d60 0x7fd1c81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cd7c4700 1 -- 172.21.15.86:0/542811119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c81a0a10 con 0x7fd1c8102b60 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1ccfc3700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8103d60 0x7fd1c81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cdfc5700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8104700 0x7fd1c81a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.186+0000 7fd1cd7c4700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8102b60 0x7fd1c8198db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400b9f0 tx=0x7fd1c400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.187+0000 7fd1be7fc700 1 -- 172.21.15.86:0/542811119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4019070 con 0x7fd1c8102b60 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.187+0000 7fd1be7fc700 1 -- 172.21.15.86:0/542811119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c4014440 con 0x7fd1c8102b60 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.187+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c81a0d00 con 0x7fd1c8102b60 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.187+0000 7fd1be7fc700 1 -- 172.21.15.86:0/542811119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40125a0 con 0x7fd1c8102b60 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.187+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c81a1190 con 0x7fd1c8102b60 2024-04-16T05:29:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.188+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c802dfe0 con 0x7fd1c8102b60 2024-04-16T05:29:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.189+0000 7fd1be7fc700 1 -- 172.21.15.86:0/542811119 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd1c401c070 con 0x7fd1c8102b60 2024-04-16T05:29:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.189+0000 7fd1be7fc700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.190+0000 7fd1ccfc3700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.190+0000 7fd1be7fc700 1 -- 172.21.15.86:0/542811119 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4533+0+0 (secure 0 0 0) 0x7fd1c40ceeb0 con 0x7fd1c8102b60 2024-04-16T05:29:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.190+0000 7fd1ccfc3700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8005cd0 tx=0x7fd1b8005c20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.193+0000 7fd1be7fc700 1 -- 172.21.15.86:0/542811119 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1c40fdff0 con 0x7fd1c8102b60 2024-04-16T05:29:32.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:31 smithi196 ceph-mon[36373]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T05:29:32.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:31 smithi196 ceph-mon[36373]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:32.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:32.405+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd1c8062f30 con 0x7fd1b40a3c40 2024-04-16T05:29:33.070 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:32 smithi194 ceph-mon[35865]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T05:29:33.070 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:32 smithi194 ceph-mon[35865]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:33.070 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:33.070 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:33.070 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:33.173 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:32 smithi086 ceph-mon[32028]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T05:29:33.174 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:32 smithi086 ceph-mon[32028]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:33.174 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:33.174 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:33.174 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:33.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:32 smithi196 ceph-mon[36373]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T05:29:33.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:32 smithi196 ceph-mon[36373]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:33.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:33.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:33.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:34.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:33 smithi194 ceph-mon[35865]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T05:29:34.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:33 smithi194 ceph-mon[35865]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 536 GiB / 536 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-04-16T05:29:34.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:33 smithi086 ceph-mon[32028]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T05:29:34.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:33 smithi086 ceph-mon[32028]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 536 GiB / 536 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-04-16T05:29:34.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:33 smithi196 ceph-mon[36373]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T05:29:34.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:33 smithi196 ceph-mon[36373]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 536 GiB / 536 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-04-16T05:29:35.606 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:35 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2098479222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5af57df8-4de1-4092-ae4e-9fb9a9e8b60b"}]: dispatch 2024-04-16T05:29:35.607 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:35 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2098479222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5af57df8-4de1-4092-ae4e-9fb9a9e8b60b"}]': finished 2024-04-16T05:29:35.607 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:35 smithi086 ceph-mon[32028]: osdmap e43: 7 total, 6 up, 7 in 2024-04-16T05:29:35.607 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:35 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:35.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:35 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2098479222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5af57df8-4de1-4092-ae4e-9fb9a9e8b60b"}]: dispatch 2024-04-16T05:29:35.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:35 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2098479222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5af57df8-4de1-4092-ae4e-9fb9a9e8b60b"}]': finished 2024-04-16T05:29:35.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:35 smithi194 ceph-mon[35865]: osdmap e43: 7 total, 6 up, 7 in 2024-04-16T05:29:35.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:35 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:35.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:35 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2098479222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5af57df8-4de1-4092-ae4e-9fb9a9e8b60b"}]: dispatch 2024-04-16T05:29:35.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:35 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2098479222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5af57df8-4de1-4092-ae4e-9fb9a9e8b60b"}]': finished 2024-04-16T05:29:35.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:35 smithi196 ceph-mon[36373]: osdmap e43: 7 total, 6 up, 7 in 2024-04-16T05:29:35.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:35 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:36.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:36 smithi194 ceph-mon[35865]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-16T05:29:36.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:36 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1343530793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:36.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:36 smithi086 ceph-mon[32028]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-16T05:29:36.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:36 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1343530793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:36 smithi196 ceph-mon[36373]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-16T05:29:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:36 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1343530793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:38.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:38 smithi194 ceph-mon[35865]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T05:29:38.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:38 smithi086 ceph-mon[32028]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T05:29:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:38 smithi196 ceph-mon[36373]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T05:29:40.584 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:40 smithi086 ceph-mon[32028]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:40 smithi194 ceph-mon[35865]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:40 smithi196 ceph-mon[36373]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:42.846 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:42 smithi086 ceph-mon[32028]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:42.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:42 smithi194 ceph-mon[35865]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:42.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:42 smithi196 ceph-mon[36373]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:43.760 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:43 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T05:29:43.760 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:43 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:43.841 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:43 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T05:29:43.841 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:43 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:43.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:43 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T05:29:43.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:43 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:44.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:44 smithi194 ceph-mon[35865]: Deploying daemon osd.6 on smithi086 2024-04-16T05:29:44.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:44 smithi194 ceph-mon[35865]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:44.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:44 smithi086 ceph-mon[32028]: Deploying daemon osd.6 on smithi086 2024-04-16T05:29:44.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:44 smithi086 ceph-mon[32028]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:44 smithi196 ceph-mon[36373]: Deploying daemon osd.6 on smithi086 2024-04-16T05:29:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:44 smithi196 ceph-mon[36373]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:46.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:46 smithi086 ceph-mon[32028]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:46.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:46 smithi194 ceph-mon[35865]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:46 smithi196 ceph-mon[36373]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:48.596 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:48 smithi086 ceph-mon[32028]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:48.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:48 smithi194 ceph-mon[35865]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:48.984 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:48 smithi196 ceph-mon[36373]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:50.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:50.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:50.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:50.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:49 smithi086 ceph-mon[32028]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:50.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:50.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:49 smithi194 ceph-mon[35865]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:49 smithi196 ceph-mon[36373]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.473+0000 7fd1be7fc700 1 -- 172.21.15.86:0/542811119 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd1c8062f30 con 0x7fd1b40a3c40 2024-04-16T05:29:52.472 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 6 on host 'smithi086' 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 msgr2=0x7fd1b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8005cd0 tx=0x7fd1b8005c20 comp rx=0 tx=0).stop 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8102b60 msgr2=0x7fd1c8198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8102b60 0x7fd1c8198db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400b9f0 tx=0x7fd1c400bdb0 comp rx=0 tx=0).stop 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 shutdown_connections 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8104700 0x7fd1c81a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8102b60 0x7fd1c8198db0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 --2- 172.21.15.86:0/542811119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8103d60 0x7fd1c81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 >> 172.21.15.86:0/542811119 conn(0x7fd1c80fe100 msgr2=0x7fd1c8107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:52.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 shutdown_connections 2024-04-16T05:29:52.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:29:52.475+0000 7fd1cfa28700 1 -- 172.21.15.86:0/542811119 wait complete. 2024-04-16T05:29:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:52 smithi086 ceph-mon[32028]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:52 smithi194 ceph-mon[35865]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:52 smithi196 ceph-mon[36373]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:53.521 DEBUG:teuthology.orchestra.run.smithi086:osd.6> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.6.service 2024-04-16T05:29:53.523 INFO:tasks.cephadm:Deploying osd.7 on smithi194 with /dev/vg_nvme/lv_2... 2024-04-16T05:29:53.523 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T05:29:53.771 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:29:53.893 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:53 smithi194 ceph-mon[35865]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T05:29:53.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:53 smithi086 ceph-mon[32028]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T05:29:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:53 smithi196 ceph-mon[36373]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T05:29:54.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:54 smithi086 ceph-mon[32028]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:54.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:54 smithi086 ceph-mon[32028]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T05:29:54.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:54 smithi086 ceph-mon[32028]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T05:29:54.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:54 smithi086 ceph-mon[32028]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:29:54.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:54.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:54.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:54.542 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:29:54 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:29:54.505+0000 7f3310f46700 -1 osd.6 0 waiting for initial osdmap 2024-04-16T05:29:54.543 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:29:54 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:29:54.514+0000 7f330cd39700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:29:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:54 smithi194 ceph-mon[35865]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:54.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:54 smithi194 ceph-mon[35865]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T05:29:54.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:54 smithi194 ceph-mon[35865]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T05:29:54.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:54 smithi194 ceph-mon[35865]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:29:54.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:54.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:54.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:54 smithi196 ceph-mon[36373]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:54 smithi196 ceph-mon[36373]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T05:29:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:54 smithi196 ceph-mon[36373]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T05:29:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:54 smithi196 ceph-mon[36373]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:29:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:55.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:55 smithi086 ceph-mon[32028]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:29:55.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:55 smithi086 ceph-mon[32028]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T05:29:55.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:55 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:55.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:55 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:55.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:55 smithi194 ceph-mon[35865]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:29:55.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:55 smithi194 ceph-mon[35865]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T05:29:55.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:55 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:55.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:55 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:55.819 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T05:29:55.819 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T05:29:55.819 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-16T05:29:55.819 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-16T05:29:55.819 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129192 s, 812 MB/s 2024-04-16T05:29:55.819 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-16T05:29:55.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:55 smithi196 ceph-mon[36373]: from='osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:29:55.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:55 smithi196 ceph-mon[36373]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T05:29:55.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:55 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:55.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:55 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:56.467 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_2 2024-04-16T05:29:56.713 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:29:56.743 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:56 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:29:56.743 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:56 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:29:56.743 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:56 smithi194 ceph-mon[35865]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:56.743 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:56 smithi194 ceph-mon[35865]: osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] boot 2024-04-16T05:29:56.743 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:56 smithi194 ceph-mon[35865]: osdmap e46: 7 total, 7 up, 7 in 2024-04-16T05:29:56.743 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:56.953 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:56 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:29:56.953 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:56 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:29:56.953 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:56 smithi086 ceph-mon[32028]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:56.953 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:56 smithi086 ceph-mon[32028]: osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] boot 2024-04-16T05:29:56.953 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:56 smithi086 ceph-mon[32028]: osdmap e46: 7 total, 7 up, 7 in 2024-04-16T05:29:56.953 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:56 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:29:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:56 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:29:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:56 smithi196 ceph-mon[36373]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:56 smithi196 ceph-mon[36373]: osd.6 [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] boot 2024-04-16T05:29:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:56 smithi196 ceph-mon[36373]: osdmap e46: 7 total, 7 up, 7 in 2024-04-16T05:29:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:57.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T05:29:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:57 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.810 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T05:29:57.811 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.811 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.811 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:57.811 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:57.811 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:57.811 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.811 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:57.811 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:57.811 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:57 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T05:29:57.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:57.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:57 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:58.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.335+0000 7f0860d3b700 1 -- 172.21.15.194:0/618330690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c104720 msgr2=0x7f085c10afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:58.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.335+0000 7f0860d3b700 1 --2- 172.21.15.194:0/618330690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c104720 0x7f085c10afd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f085400b0d0 tx=0x7f085400b3e0 comp rx=0 tx=0).stop 2024-04-16T05:29:58.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.335+0000 7f0860d3b700 1 -- 172.21.15.194:0/618330690 shutdown_connections 2024-04-16T05:29:58.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.335+0000 7f0860d3b700 1 --2- 172.21.15.194:0/618330690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c104720 0x7f085c10afd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:58.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.335+0000 7f0860d3b700 1 --2- 172.21.15.194:0/618330690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c102b80 0x7f085c102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:58.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.335+0000 7f0860d3b700 1 --2- 172.21.15.194:0/618330690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f085c103d80 0x7f085c1041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:58.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.335+0000 7f0860d3b700 1 -- 172.21.15.194:0/618330690 >> 172.21.15.194:0/618330690 conn(0x7f085c0fe140 msgr2=0x7f085c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:58.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.335+0000 7f0860d3b700 1 -- 172.21.15.194:0/618330690 shutdown_connections 2024-04-16T05:29:58.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.335+0000 7f0860d3b700 1 -- 172.21.15.194:0/618330690 wait complete. 2024-04-16T05:29:58.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f0860d3b700 1 Processor -- start 2024-04-16T05:29:58.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f0860d3b700 1 -- start start 2024-04-16T05:29:58.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f0860d3b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c102b80 0x7f085c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:58.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f0860d3b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c103d80 0x7f085c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:58.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f0860d3b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f085c104720 0x7f085c1a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:58.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f0860d3b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f085c10e770 con 0x7f085c103d80 2024-04-16T05:29:58.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f0860d3b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f085c10e5f0 con 0x7f085c102b80 2024-04-16T05:29:58.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f0860d3b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f085c10e8f0 con 0x7f085c104720 2024-04-16T05:29:58.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c102b80 0x7f085c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c102b80 0x7f085c198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:35188/0 (socket says 172.21.15.194:35188) 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085a59c700 1 -- 172.21.15.194:0/3044397429 learned_addr learned my addr 172.21.15.194:0/3044397429 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f0853fff700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c103d80 0x7f085c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085ad9d700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f085c104720 0x7f085c1a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085a59c700 1 -- 172.21.15.194:0/3044397429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f085c104720 msgr2=0x7f085c1a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085a59c700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f085c104720 0x7f085c1a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085a59c700 1 -- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c103d80 msgr2=0x7f085c1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085a59c700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c103d80 0x7f085c1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085a59c700 1 -- 172.21.15.194:0/3044397429 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c1a08e0 con 0x7f085c102b80 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.336+0000 7f085a59c700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c102b80 0x7f085c198c60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f084c00c990 tx=0x7f084c00cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:58.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.337+0000 7f08537fe700 1 -- 172.21.15.194:0/3044397429 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c007b00 con 0x7f085c102b80 2024-04-16T05:29:58.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.337+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c1a0bd0 con 0x7f085c102b80 2024-04-16T05:29:58.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.337+0000 7f08537fe700 1 -- 172.21.15.194:0/3044397429 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c007ca0 con 0x7f085c102b80 2024-04-16T05:29:58.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.337+0000 7f08537fe700 1 -- 172.21.15.194:0/3044397429 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c018660 con 0x7f085c102b80 2024-04-16T05:29:58.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.337+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c1a1160 con 0x7f085c102b80 2024-04-16T05:29:58.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.338+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f085c0697e0 con 0x7f085c102b80 2024-04-16T05:29:58.340 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.339+0000 7f08537fe700 1 -- 172.21.15.194:0/3044397429 <== mon.2 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f084c00f450 con 0x7f085c102b80 2024-04-16T05:29:58.340 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.339+0000 7f08537fe700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08480a3c40 0x7f08480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:58.340 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.340+0000 7f08537fe700 1 -- 172.21.15.194:0/3044397429 <== mon.2 v2:172.21.15.194:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f084c0cf620 con 0x7f085c102b80 2024-04-16T05:29:58.340 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.340+0000 7f0853fff700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08480a3c40 0x7f08480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:58.340 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.340+0000 7f0853fff700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08480a3c40 0x7f08480a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f085c19a170 tx=0x7f084400b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:58.343 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.344+0000 7f08537fe700 1 -- 172.21.15.194:0/3044397429 <== mon.2 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f084c014030 con 0x7f085c102b80 2024-04-16T05:29:58.556 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:29:58.557+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f085c062f30 con 0x7f08480a3c40 2024-04-16T05:29:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:58 smithi194 ceph-mon[35865]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:58 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi086 2024-04-16T05:29:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:58 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-16T05:29:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:58.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:58 smithi086 ceph-mon[32028]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:58.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:58 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi086 2024-04-16T05:29:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:58 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-16T05:29:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:58 smithi196 ceph-mon[36373]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:58 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi086 2024-04-16T05:29:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:58 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-16T05:29:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:29:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:59 smithi194 ceph-mon[35865]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:29:59 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:59.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:59 smithi086 ceph-mon[32028]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:59.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:59.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:59.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:29:59 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:59.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:59 smithi196 ceph-mon[36373]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:59.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:59.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:59.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:29:59 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:00.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:00 smithi194 ceph-mon[35865]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:00.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:00 smithi194 ceph-mon[35865]: overall HEALTH_OK 2024-04-16T05:30:00.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:00 smithi086 ceph-mon[32028]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:00.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:00 smithi086 ceph-mon[32028]: overall HEALTH_OK 2024-04-16T05:30:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:00 smithi196 ceph-mon[36373]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:00 smithi196 ceph-mon[36373]: overall HEALTH_OK 2024-04-16T05:30:01.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:01 smithi194 ceph-mon[35865]: from='client.? 172.21.15.194:0/3070233537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a76013-993e-4ff1-976d-002d09209edf"}]: dispatch 2024-04-16T05:30:01.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:01 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a76013-993e-4ff1-976d-002d09209edf"}]: dispatch 2024-04-16T05:30:01.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:01 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5a76013-993e-4ff1-976d-002d09209edf"}]': finished 2024-04-16T05:30:01.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:01 smithi194 ceph-mon[35865]: osdmap e48: 8 total, 7 up, 8 in 2024-04-16T05:30:01.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:01 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:01.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:01 smithi086 ceph-mon[32028]: from='client.? 172.21.15.194:0/3070233537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a76013-993e-4ff1-976d-002d09209edf"}]: dispatch 2024-04-16T05:30:01.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:01 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a76013-993e-4ff1-976d-002d09209edf"}]: dispatch 2024-04-16T05:30:01.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:01 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5a76013-993e-4ff1-976d-002d09209edf"}]': finished 2024-04-16T05:30:01.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:01 smithi086 ceph-mon[32028]: osdmap e48: 8 total, 7 up, 8 in 2024-04-16T05:30:01.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:01 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:01 smithi196 ceph-mon[36373]: from='client.? 172.21.15.194:0/3070233537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a76013-993e-4ff1-976d-002d09209edf"}]: dispatch 2024-04-16T05:30:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:01 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a76013-993e-4ff1-976d-002d09209edf"}]: dispatch 2024-04-16T05:30:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:01 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5a76013-993e-4ff1-976d-002d09209edf"}]': finished 2024-04-16T05:30:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:01 smithi196 ceph-mon[36373]: osdmap e48: 8 total, 7 up, 8 in 2024-04-16T05:30:01.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:01 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:02 smithi194 ceph-mon[35865]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:02 smithi194 ceph-mon[35865]: from='client.? 172.21.15.194:0/2316831873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:02.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:02 smithi086 ceph-mon[32028]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:02.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:02 smithi086 ceph-mon[32028]: from='client.? 172.21.15.194:0/2316831873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:02 smithi196 ceph-mon[36373]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:02 smithi196 ceph-mon[36373]: from='client.? 172.21.15.194:0/2316831873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:04.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:04 smithi194 ceph-mon[35865]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:04.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:04 smithi086 ceph-mon[32028]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:04.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:04 smithi196 ceph-mon[36373]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:06.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:06 smithi194 ceph-mon[35865]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:06.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:06 smithi086 ceph-mon[32028]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:06 smithi196 ceph-mon[36373]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:08.836 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:08 smithi194 ceph-mon[35865]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:08.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:08 smithi086 ceph-mon[32028]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:08 smithi196 ceph-mon[36373]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:09.727 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:09 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T05:30:09.727 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:09 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:09.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:09 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T05:30:09.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:09 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:09 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T05:30:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:09 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:10.829 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:10 smithi194 ceph-mon[35865]: Deploying daemon osd.7 on smithi194 2024-04-16T05:30:10.829 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:10 smithi194 ceph-mon[35865]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:10.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:10 smithi086 ceph-mon[32028]: Deploying daemon osd.7 on smithi194 2024-04-16T05:30:10.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:10 smithi086 ceph-mon[32028]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:10 smithi196 ceph-mon[36373]: Deploying daemon osd.7 on smithi194 2024-04-16T05:30:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:10 smithi196 ceph-mon[36373]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:12.830 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:12 smithi194 ceph-mon[35865]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:12.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:12 smithi086 ceph-mon[32028]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:12 smithi196 ceph-mon[36373]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:14.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:14 smithi194 ceph-mon[35865]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:14.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:14 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:14.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:14 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:14.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:14 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:14.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:14 smithi086 ceph-mon[32028]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:14.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:14.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:14.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:14 smithi196 ceph-mon[36373]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:14 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:14 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:14 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:16.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:16 smithi194 ceph-mon[35865]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:16.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:16 smithi086 ceph-mon[32028]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:16 smithi196 ceph-mon[36373]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:18.221 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.222+0000 7f08537fe700 1 -- 172.21.15.194:0/3044397429 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f085c062f30 con 0x7f08480a3c40 2024-04-16T05:30:18.221 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 7 on host 'smithi194' 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.224+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08480a3c40 msgr2=0x7f08480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.224+0000 7f0860d3b700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08480a3c40 0x7f08480a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f085c19a170 tx=0x7f084400b3c0 comp rx=0 tx=0).stop 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.224+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c102b80 msgr2=0x7f085c198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.224+0000 7f0860d3b700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c102b80 0x7f085c198c60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f084c00c990 tx=0x7f084c00cca0 comp rx=0 tx=0).stop 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.225+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 shutdown_connections 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.225+0000 7f0860d3b700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08480a3c40 0x7f08480a6100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.225+0000 7f0860d3b700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f085c104720 0x7f085c1a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.225+0000 7f0860d3b700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085c102b80 0x7f085c198c60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.225+0000 7f0860d3b700 1 --2- 172.21.15.194:0/3044397429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c103d80 0x7f085c1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.225+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 >> 172.21.15.194:0/3044397429 conn(0x7f085c0fe140 msgr2=0x7f085c106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:30:18.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.225+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 shutdown_connections 2024-04-16T05:30:18.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:30:18.225+0000 7f0860d3b700 1 -- 172.21.15.194:0/3044397429 wait complete. 2024-04-16T05:30:18.516 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.516 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.516 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:18.516 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:18.516 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.517 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:18.517 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.517 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:18.517 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.517 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='osd.7 [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:18.517 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:18 smithi194 ceph-mon[35865]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:18.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='osd.7 [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:18.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:18 smithi086 ceph-mon[32028]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:18.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:18.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='osd.7 [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:18.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:18 smithi196 ceph-mon[36373]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:18.846 DEBUG:teuthology.orchestra.run.smithi194:osd.7> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.7.service 2024-04-16T05:30:18.848 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-04-16T05:30:18.848 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T05:30:19.099 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:30:20.238 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:20 smithi194 ceph-mon[35865]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T05:30:20.238 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:20 smithi194 ceph-mon[35865]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T05:30:20.238 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:20 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:20.238 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:20 smithi194 ceph-mon[35865]: from='osd.7 [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:30:20.238 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:20 smithi194 ceph-mon[35865]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:30:20.238 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:20 smithi194 ceph-mon[35865]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:20.238 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:30:20 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:30:20.230+0000 7f4765263700 -1 osd.7 0 waiting for initial osdmap 2024-04-16T05:30:20.239 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:30:20 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:30:20.238+0000 7f4761056700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:30:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:20 smithi196 ceph-mon[36373]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T05:30:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:20 smithi196 ceph-mon[36373]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T05:30:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:20 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:20 smithi196 ceph-mon[36373]: from='osd.7 [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:30:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:20 smithi196 ceph-mon[36373]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:30:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:20 smithi196 ceph-mon[36373]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:20.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:20 smithi086 ceph-mon[32028]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T05:30:20.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:20 smithi086 ceph-mon[32028]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T05:30:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:20 smithi086 ceph-mon[32028]: from='osd.7 [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:30:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:20 smithi086 ceph-mon[32028]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:30:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:20 smithi086 ceph-mon[32028]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T05:30:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T05:30:21.067 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-16T05:30:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-16T05:30:21.067 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112655 s, 931 MB/s 2024-04-16T05:30:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-16T05:30:21.374 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:21 smithi196 ceph-mon[36373]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:30:21.374 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:21 smithi196 ceph-mon[36373]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T05:30:21.374 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:21.374 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:21.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:21 smithi194 ceph-mon[35865]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:30:21.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:21 smithi194 ceph-mon[35865]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T05:30:21.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:21.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:21 smithi086 ceph-mon[32028]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:30:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:21 smithi086 ceph-mon[32028]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T05:30:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:21.763 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-04-16T05:30:22.004 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: osd.7 [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] boot 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: osdmap e51: 8 total, 8 up, 8 in 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:22.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:22.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:22.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:22.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:22 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:30:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:30:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: osd.7 [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] boot 2024-04-16T05:30:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: osdmap e51: 8 total, 8 up, 8 in 2024-04-16T05:30:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:22.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:22.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:22.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:22.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:22.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:22 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:30:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:30:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: osd.7 [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] boot 2024-04-16T05:30:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: osdmap e51: 8 total, 8 up, 8 in 2024-04-16T05:30:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:22.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:22.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:22.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:22.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:22.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:22.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:22 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:23 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi194 2024-04-16T05:30:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:23 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi194 to 5693M 2024-04-16T05:30:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:23 smithi196 ceph-mon[36373]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T05:30:23.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:23 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi194 2024-04-16T05:30:23.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:23 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi194 to 5693M 2024-04-16T05:30:23.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:23 smithi194 ceph-mon[35865]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.718+0000 7fdf0feb5700 1 -- 172.21.15.196:0/2808569928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf08103640 msgr2=0x7fdf08105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.718+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/2808569928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf08103640 0x7fdf08105a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009990 tx=0x7fdf04009ca0 comp rx=0 tx=0).stop 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.718+0000 7fdf0feb5700 1 -- 172.21.15.196:0/2808569928 shutdown_connections 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.718+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/2808569928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf081088a0 0x7fdf0810acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.718+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/2808569928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf08103640 0x7fdf08105a30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.718+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/2808569928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf08105f70 0x7fdf08108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.718+0000 7fdf0feb5700 1 -- 172.21.15.196:0/2808569928 >> 172.21.15.196:0/2808569928 conn(0x7fdf080fd010 msgr2=0x7fdf080ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.718+0000 7fdf0feb5700 1 -- 172.21.15.196:0/2808569928 shutdown_connections 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.718+0000 7fdf0feb5700 1 -- 172.21.15.196:0/2808569928 wait complete. 2024-04-16T05:30:23.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0feb5700 1 Processor -- start 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0feb5700 1 -- start start 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0feb5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf08103640 0x7fdf08102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0feb5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf08105f70 0x7fdf081015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0feb5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf081088a0 0x7fdf08101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0feb5700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810e7b0 con 0x7fdf08103640 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0feb5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810e630 con 0x7fdf08105f70 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0feb5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810e930 con 0x7fdf081088a0 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0e452700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf081088a0 0x7fdf08101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0e452700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf081088a0 0x7fdf08101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:45268/0 (socket says 172.21.15.196:45268) 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0e452700 1 -- 172.21.15.196:0/1712979096 learned_addr learned my addr 172.21.15.196:0/1712979096 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0d450700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf08105f70 0x7fdf081015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0dc51700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf08103640 0x7fdf08102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0e452700 1 -- 172.21.15.196:0/1712979096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf08105f70 msgr2=0x7fdf081015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:23.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0e452700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf08105f70 0x7fdf081015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:23.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0e452700 1 -- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf08103640 msgr2=0x7fdf08102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:23.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0e452700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf08103640 0x7fdf08102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:23.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.719+0000 7fdf0e452700 1 -- 172.21.15.196:0/1712979096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf081023e0 con 0x7fdf081088a0 2024-04-16T05:30:23.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.720+0000 7fdf0e452700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf081088a0 0x7fdf08101b30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdf000094f0 tx=0x7fdf0000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:23.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.720+0000 7fdefeffd700 1 -- 172.21.15.196:0/1712979096 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00004db0 con 0x7fdf081088a0 2024-04-16T05:30:23.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.720+0000 7fdefeffd700 1 -- 172.21.15.196:0/1712979096 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf00009e20 con 0x7fdf081088a0 2024-04-16T05:30:23.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.720+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf081a2f10 con 0x7fdf081088a0 2024-04-16T05:30:23.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.720+0000 7fdefeffd700 1 -- 172.21.15.196:0/1712979096 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00015400 con 0x7fdf081088a0 2024-04-16T05:30:23.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.720+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf081a33f0 con 0x7fdf081088a0 2024-04-16T05:30:23.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:23 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi194 2024-04-16T05:30:23.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:23 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi194 to 5693M 2024-04-16T05:30:23.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:23 smithi086 ceph-mon[32028]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T05:30:23.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.721+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf080697e0 con 0x7fdf081088a0 2024-04-16T05:30:23.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.722+0000 7fdefeffd700 1 -- 172.21.15.196:0/1712979096 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdf0000db20 con 0x7fdf081088a0 2024-04-16T05:30:23.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.722+0000 7fdefeffd700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdef40a3c40 0x7fdef40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:23.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.723+0000 7fdefeffd700 1 -- 172.21.15.196:0/1712979096 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7fdf000cfa60 con 0x7fdf081088a0 2024-04-16T05:30:23.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.723+0000 7fdf0dc51700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdef40a3c40 0x7fdef40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:23.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.723+0000 7fdf0dc51700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdef40a3c40 0x7fdef40a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b5c0 tx=0x7fdf04005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:23.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.726+0000 7fdefeffd700 1 -- 172.21.15.196:0/1712979096 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf00010070 con 0x7fdf081088a0 2024-04-16T05:30:23.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:23.932+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdf08062f30 con 0x7fdef40a3c40 2024-04-16T05:30:24.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:24 smithi086 ceph-mon[32028]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T05:30:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:24 smithi086 ceph-mon[32028]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:24 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:24 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:24 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:24.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:24 smithi196 ceph-mon[36373]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T05:30:24.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:24 smithi196 ceph-mon[36373]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:24.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:24 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:24.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:24 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:24.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:24 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:24.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:24 smithi194 ceph-mon[35865]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T05:30:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:24 smithi194 ceph-mon[35865]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:24 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:24 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:24 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:25.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:25 smithi196 ceph-mon[36373]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:25.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:25 smithi196 ceph-mon[36373]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T05:30:25.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:25 smithi194 ceph-mon[35865]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:25.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:25 smithi194 ceph-mon[35865]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T05:30:25.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:25 smithi086 ceph-mon[32028]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:25.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:25 smithi086 ceph-mon[32028]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T05:30:26.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:26 smithi194 ceph-mon[35865]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T05:30:26.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:26 smithi194 ceph-mon[35865]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:26 smithi086 ceph-mon[32028]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T05:30:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:26 smithi086 ceph-mon[32028]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:26.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:26 smithi196 ceph-mon[36373]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T05:30:26.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:26 smithi196 ceph-mon[36373]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:27 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/4023900547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3010ed-edc0-471d-b44b-e700733fe43b"}]: dispatch 2024-04-16T05:30:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:27 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3010ed-edc0-471d-b44b-e700733fe43b"}]: dispatch 2024-04-16T05:30:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:27 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b3010ed-edc0-471d-b44b-e700733fe43b"}]': finished 2024-04-16T05:30:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:27 smithi196 ceph-mon[36373]: osdmap e56: 9 total, 8 up, 9 in 2024-04-16T05:30:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:27 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:27 smithi194 ceph-mon[35865]: from='client.? 172.21.15.196:0/4023900547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3010ed-edc0-471d-b44b-e700733fe43b"}]: dispatch 2024-04-16T05:30:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:27 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3010ed-edc0-471d-b44b-e700733fe43b"}]: dispatch 2024-04-16T05:30:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:27 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b3010ed-edc0-471d-b44b-e700733fe43b"}]': finished 2024-04-16T05:30:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:27 smithi194 ceph-mon[35865]: osdmap e56: 9 total, 8 up, 9 in 2024-04-16T05:30:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:27 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:27.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:27 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/4023900547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3010ed-edc0-471d-b44b-e700733fe43b"}]: dispatch 2024-04-16T05:30:27.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:27 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3010ed-edc0-471d-b44b-e700733fe43b"}]: dispatch 2024-04-16T05:30:27.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:27 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b3010ed-edc0-471d-b44b-e700733fe43b"}]': finished 2024-04-16T05:30:27.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:27 smithi086 ceph-mon[32028]: osdmap e56: 9 total, 8 up, 9 in 2024-04-16T05:30:27.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:27 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:28 smithi194 ceph-mon[35865]: from='client.? 172.21.15.196:0/1607942706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:28 smithi194 ceph-mon[35865]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:28.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:28 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/1607942706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:28.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:28 smithi086 ceph-mon[32028]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:28 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/1607942706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:28 smithi196 ceph-mon[36373]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:30.728 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:30 smithi196 ceph-mon[36373]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:30.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:30 smithi194 ceph-mon[35865]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:30.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:30 smithi086 ceph-mon[32028]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:32.683 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:32 smithi196 ceph-mon[36373]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:32 smithi194 ceph-mon[35865]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:32.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:32 smithi086 ceph-mon[32028]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:34.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:34 smithi196 ceph-mon[36373]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:34.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:34 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T05:30:34.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:34 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:34.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:34 smithi194 ceph-mon[35865]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:34.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:34 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T05:30:34.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:34 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:34.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:34 smithi086 ceph-mon[32028]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:34.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:34 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T05:30:34.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:34 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:35.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:35 smithi196 ceph-mon[36373]: Deploying daemon osd.8 on smithi196 2024-04-16T05:30:35.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:35 smithi194 ceph-mon[35865]: Deploying daemon osd.8 on smithi196 2024-04-16T05:30:35.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:35 smithi086 ceph-mon[32028]: Deploying daemon osd.8 on smithi196 2024-04-16T05:30:36.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:36 smithi196 ceph-mon[36373]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:36.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:36 smithi194 ceph-mon[35865]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:36.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:36 smithi086 ceph-mon[32028]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:38 smithi194 ceph-mon[35865]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:38.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:38 smithi086 ceph-mon[32028]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:38 smithi196 ceph-mon[36373]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:40.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:40 smithi196 ceph-mon[36373]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:40.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:40 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:40.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:40 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:40.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:40 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:40 smithi194 ceph-mon[35865]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:40 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:40 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:40 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:40.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:40 smithi086 ceph-mon[32028]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:40.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:40 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:40.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:40 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:40.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:40 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:42.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:42 smithi196 ceph-mon[36373]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:42.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:42 smithi194 ceph-mon[35865]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:42.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:42 smithi086 ceph-mon[32028]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:43.201 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-04-16T05:30:43.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.199+0000 7fdefeffd700 1 -- 172.21.15.196:0/1712979096 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf08062f30 con 0x7fdef40a3c40 2024-04-16T05:30:43.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdef40a3c40 msgr2=0x7fdef40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:43.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdef40a3c40 0x7fdef40a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b5c0 tx=0x7fdf04005fd0 comp rx=0 tx=0).stop 2024-04-16T05:30:43.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf081088a0 msgr2=0x7fdf08101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:43.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf081088a0 0x7fdf08101b30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdf000094f0 tx=0x7fdf0000d3b0 comp rx=0 tx=0).stop 2024-04-16T05:30:43.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 shutdown_connections 2024-04-16T05:30:43.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdef40a3c40 0x7fdef40a6100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:43.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf081088a0 0x7fdf08101b30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:43.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf08103640 0x7fdf08102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:43.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 --2- 172.21.15.196:0/1712979096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf08105f70 0x7fdf081015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:43.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 >> 172.21.15.196:0/1712979096 conn(0x7fdf080fd010 msgr2=0x7fdf08103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:30:43.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 shutdown_connections 2024-04-16T05:30:43.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:30:43.202+0000 7fdf0feb5700 1 -- 172.21.15.196:0/1712979096 wait complete. 2024-04-16T05:30:43.857 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.8.service 2024-04-16T05:30:43.860 INFO:tasks.cephadm:Deploying osd.9 on smithi086 with /dev/vg_nvme/lv_1... 2024-04-16T05:30:43.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T05:30:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:30:44.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:44.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:44.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:44 smithi086 ceph-mon[32028]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:44.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:44 smithi086 ceph-mon[32028]: from='osd.8 [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:44.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:44 smithi086 ceph-mon[32028]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:44.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:44.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:44.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:44 smithi196 ceph-mon[36373]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:44.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:44 smithi196 ceph-mon[36373]: from='osd.8 [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:44.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:44 smithi196 ceph-mon[36373]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:44.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:44.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:44.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:44 smithi194 ceph-mon[35865]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:44.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:44 smithi194 ceph-mon[35865]: from='osd.8 [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:44.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:44 smithi194 ceph-mon[35865]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:45.431 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:45 smithi086 ceph-mon[32028]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T05:30:45.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:45 smithi086 ceph-mon[32028]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T05:30:45.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:45 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:45.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:45 smithi086 ceph-mon[32028]: from='osd.8 [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:30:45.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:45 smithi086 ceph-mon[32028]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:30:45.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:45 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:45.432 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:45 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:45.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:45 smithi196 ceph-mon[36373]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T05:30:45.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:45 smithi196 ceph-mon[36373]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T05:30:45.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:45 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:45.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:45 smithi196 ceph-mon[36373]: from='osd.8 [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:30:45.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:45 smithi196 ceph-mon[36373]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:30:45.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:45 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:45.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:45 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:45.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:45 smithi194 ceph-mon[35865]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T05:30:45.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:45 smithi194 ceph-mon[35865]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T05:30:45.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:45 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:45.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:45 smithi194 ceph-mon[35865]: from='osd.8 [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:30:45.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:45 smithi194 ceph-mon[35865]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:30:45.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:45 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:45.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:45 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:45.881 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:30:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:30:45.615+0000 7f3ab2b63700 -1 osd.8 0 waiting for initial osdmap 2024-04-16T05:30:45.881 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:30:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:30:45.627+0000 7f3aaa94e700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:30:46.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:46 smithi086 ceph-mon[32028]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:30:46.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:46 smithi086 ceph-mon[32028]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T05:30:46.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:46 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:46.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:46 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:46.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:46 smithi086 ceph-mon[32028]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:46.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:46 smithi086 ceph-mon[32028]: from='osd.8 ' entity='osd.8' 2024-04-16T05:30:46.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:46 smithi196 ceph-mon[36373]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:30:46.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:46 smithi196 ceph-mon[36373]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T05:30:46.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:46 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:46.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:46 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:46.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:46 smithi196 ceph-mon[36373]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:46.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:46 smithi196 ceph-mon[36373]: from='osd.8 ' entity='osd.8' 2024-04-16T05:30:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T05:30:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T05:30:46.486 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-16T05:30:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-16T05:30:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115464 s, 908 MB/s 2024-04-16T05:30:46.486 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 2024-04-16T05:30:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-16T05:30:46.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:46 smithi194 ceph-mon[35865]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:30:46.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:46 smithi194 ceph-mon[35865]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T05:30:46.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:46 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:46.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:46 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:46.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:46 smithi194 ceph-mon[35865]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:46.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:46 smithi194 ceph-mon[35865]: from='osd.8 ' entity='osd.8' 2024-04-16T05:30:47.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-04-16T05:30:47.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:47 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:30:47.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:47 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:30:47.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:47 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:47.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:47 smithi196 ceph-mon[36373]: osd.8 [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] boot 2024-04-16T05:30:47.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:47 smithi196 ceph-mon[36373]: osdmap e59: 9 total, 9 up, 9 in 2024-04-16T05:30:47.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:47 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:47.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:47 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:30:47.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:47 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:30:47.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:47 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:47.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:47 smithi194 ceph-mon[35865]: osd.8 [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] boot 2024-04-16T05:30:47.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:47 smithi194 ceph-mon[35865]: osdmap e59: 9 total, 9 up, 9 in 2024-04-16T05:30:47.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:47 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:30:47.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:47 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:30:47.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:47 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:30:47.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:47 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:47.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:47 smithi086 ceph-mon[32028]: osd.8 [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] boot 2024-04-16T05:30:47.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:47 smithi086 ceph-mon[32028]: osdmap e59: 9 total, 9 up, 9 in 2024-04-16T05:30:47.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:47 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:48 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T05:30:48.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:48.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:48.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:48.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:48.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:48.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:48.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:48 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T05:30:48.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:48.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:48.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:48.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:48.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:48.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:48.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:48.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:48 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.169+0000 7f73aef0d700 1 -- 172.21.15.86:0/2362984477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a8102c40 msgr2=0x7f73a81030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.169+0000 7f73aef0d700 1 --2- 172.21.15.86:0/2362984477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a8102c40 0x7f73a81030a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f73940099c0 tx=0x7f7394009cd0 comp rx=0 tx=0).stop 2024-04-16T05:30:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.169+0000 7f73aef0d700 1 -- 172.21.15.86:0/2362984477 shutdown_connections 2024-04-16T05:30:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.169+0000 7f73aef0d700 1 --2- 172.21.15.86:0/2362984477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a81035e0 0x7f73a8109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.169+0000 7f73aef0d700 1 --2- 172.21.15.86:0/2362984477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73a8101a40 0x7f73a8101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.169+0000 7f73aef0d700 1 --2- 172.21.15.86:0/2362984477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a8102c40 0x7f73a81030a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.169+0000 7f73aef0d700 1 -- 172.21.15.86:0/2362984477 >> 172.21.15.86:0/2362984477 conn(0x7f73a80fcfe0 msgr2=0x7f73a80ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:30:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.169+0000 7f73aef0d700 1 -- 172.21.15.86:0/2362984477 shutdown_connections 2024-04-16T05:30:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.169+0000 7f73aef0d700 1 -- 172.21.15.86:0/2362984477 wait complete. 2024-04-16T05:30:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.170+0000 7f73aef0d700 1 Processor -- start 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.170+0000 7f73aef0d700 1 -- start start 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.170+0000 7f73aef0d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73a8101a40 0x7f73a8197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.170+0000 7f73aef0d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a8102c40 0x7f73a8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.170+0000 7f73aef0d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a81035e0 0x7f73a819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.170+0000 7f73aef0d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810d6f0 con 0x7f73a81035e0 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.170+0000 7f73aef0d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810d570 con 0x7f73a8101a40 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.170+0000 7f73aef0d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810d870 con 0x7f73a8102c40 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.170+0000 7f73ad4aa700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a81035e0 0x7f73a819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73ad4aa700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a81035e0 0x7f73a819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59040/0 (socket says 172.21.15.86:59040) 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73ad4aa700 1 -- 172.21.15.86:0/1757996945 learned_addr learned my addr 172.21.15.86:0/1757996945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73a7fff700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a8102c40 0x7f73a8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73acca9700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73a8101a40 0x7f73a8197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73ad4aa700 1 -- 172.21.15.86:0/1757996945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a8102c40 msgr2=0x7f73a8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73ad4aa700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a8102c40 0x7f73a8198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73ad4aa700 1 -- 172.21.15.86:0/1757996945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73a8101a40 msgr2=0x7f73a8197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73ad4aa700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73a8101a40 0x7f73a8197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73ad4aa700 1 -- 172.21.15.86:0/1757996945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a819f990 con 0x7f73a81035e0 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73acca9700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73a8101a40 0x7f73a8197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73ad4aa700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a81035e0 0x7f73a819f2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73a000add0 tx=0x7f73a000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73a5ffb700 1 -- 172.21.15.86:0/1757996945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0017020 con 0x7f73a81035e0 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a819fc80 con 0x7f73a81035e0 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73a5ffb700 1 -- 172.21.15.86:0/1757996945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a000ce20 con 0x7f73a81035e0 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73a5ffb700 1 -- 172.21.15.86:0/1757996945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0005520 con 0x7f73a81035e0 2024-04-16T05:30:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.171+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a81a0170 con 0x7f73a81035e0 2024-04-16T05:30:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.172+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a80697e0 con 0x7f73a81035e0 2024-04-16T05:30:49.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.173+0000 7f73a5ffb700 1 -- 172.21.15.86:0/1757996945 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f73a0004ad0 con 0x7f73a81035e0 2024-04-16T05:30:49.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.174+0000 7f73a5ffb700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f73980a3c40 0x7f73980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:49.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.174+0000 7f73acca9700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f73980a3c40 0x7f73980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:49.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.174+0000 7f73a5ffb700 1 -- 172.21.15.86:0/1757996945 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7f73a00cf1f0 con 0x7f73a81035e0 2024-04-16T05:30:49.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.175+0000 7f73acca9700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f73980a3c40 0x7f73980a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f73a8102aa0 tx=0x7f739c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.179+0000 7f73a5ffb700 1 -- 172.21.15.86:0/1757996945 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73a0021020 con 0x7f73a81035e0 2024-04-16T05:30:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:30:49.389+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f73a8062f30 con 0x7f73980a3c40 2024-04-16T05:30:49.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:49 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi196 2024-04-16T05:30:49.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:49 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi196 to 5693M 2024-04-16T05:30:49.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:49.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:49.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:49 smithi086 ceph-mon[32028]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T05:30:49.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:49.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:49 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi196 2024-04-16T05:30:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:49 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi196 to 5693M 2024-04-16T05:30:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:49 smithi196 ceph-mon[36373]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T05:30:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:49.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:49.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:49 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi196 2024-04-16T05:30:49.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:49 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi196 to 5693M 2024-04-16T05:30:49.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:49.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:49.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:49 smithi194 ceph-mon[35865]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T05:30:49.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:49.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:30:50.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:50 smithi086 ceph-mon[32028]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T05:30:50.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:50 smithi086 ceph-mon[32028]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:50.270 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:50.270 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:50.270 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:50.270 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:50 smithi086 ceph-mon[32028]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:50 smithi196 ceph-mon[36373]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T05:30:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:50 smithi196 ceph-mon[36373]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:50.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:50 smithi196 ceph-mon[36373]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:50 smithi194 ceph-mon[35865]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T05:30:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:50 smithi194 ceph-mon[35865]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:50 smithi194 ceph-mon[35865]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:51.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:51 smithi086 ceph-mon[32028]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T05:30:51.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:51 smithi196 ceph-mon[36373]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T05:30:51.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:51 smithi194 ceph-mon[35865]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T05:30:52.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:52 smithi086 ceph-mon[32028]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-04-16T05:30:52.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:52 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2190270742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f95451-1619-47ff-b12f-b8651c020a94"}]: dispatch 2024-04-16T05:30:52.585 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:52 smithi196 ceph-mon[36373]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-04-16T05:30:52.585 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:52 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2190270742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f95451-1619-47ff-b12f-b8651c020a94"}]: dispatch 2024-04-16T05:30:52.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:52 smithi194 ceph-mon[35865]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-04-16T05:30:52.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:52 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2190270742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f95451-1619-47ff-b12f-b8651c020a94"}]: dispatch 2024-04-16T05:30:53.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:53 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2190270742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70f95451-1619-47ff-b12f-b8651c020a94"}]': finished 2024-04-16T05:30:53.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:53 smithi194 ceph-mon[35865]: osdmap e64: 10 total, 9 up, 10 in 2024-04-16T05:30:53.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:53 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:53.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:53 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/46098503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:53.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:53 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2190270742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70f95451-1619-47ff-b12f-b8651c020a94"}]': finished 2024-04-16T05:30:53.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:53 smithi086 ceph-mon[32028]: osdmap e64: 10 total, 9 up, 10 in 2024-04-16T05:30:53.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:53 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:53.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:53 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/46098503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:53.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:53 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2190270742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70f95451-1619-47ff-b12f-b8651c020a94"}]': finished 2024-04-16T05:30:53.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:53 smithi196 ceph-mon[36373]: osdmap e64: 10 total, 9 up, 10 in 2024-04-16T05:30:53.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:53 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:53.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:53 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/46098503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:54.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:54 smithi194 ceph-mon[35865]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-16T05:30:54.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:54 smithi086 ceph-mon[32028]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-16T05:30:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:54 smithi196 ceph-mon[36373]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-16T05:30:56.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:56 smithi086 ceph-mon[32028]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-16T05:30:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:56 smithi194 ceph-mon[35865]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-16T05:30:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:56 smithi196 ceph-mon[36373]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-16T05:30:58.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:30:58 smithi086 ceph-mon[32028]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T05:30:58.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:30:58 smithi194 ceph-mon[35865]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T05:30:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:30:58 smithi196 ceph-mon[36373]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T05:31:00.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:00 smithi086 ceph-mon[32028]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-16T05:31:00.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:00 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T05:31:00.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:00 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:00 smithi194 ceph-mon[35865]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-16T05:31:00.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:00 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T05:31:00.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:00 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:00 smithi196 ceph-mon[36373]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-16T05:31:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:00 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T05:31:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:00 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:01.874 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:01 smithi086 ceph-mon[32028]: Deploying daemon osd.9 on smithi086 2024-04-16T05:31:01.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:01 smithi194 ceph-mon[35865]: Deploying daemon osd.9 on smithi086 2024-04-16T05:31:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:01 smithi196 ceph-mon[36373]: Deploying daemon osd.9 on smithi086 2024-04-16T05:31:02.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:02 smithi194 ceph-mon[35865]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:02.946 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:02 smithi086 ceph-mon[32028]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:02 smithi196 ceph-mon[36373]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:04.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:04 smithi086 ceph-mon[32028]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:04.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:04 smithi194 ceph-mon[35865]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:04.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:04 smithi196 ceph-mon[36373]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:06.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:06 smithi194 ceph-mon[35865]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:06.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:06.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:06.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:06.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:06 smithi086 ceph-mon[32028]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:06.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:06.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:06.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:06 smithi196 ceph-mon[36373]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:08.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:08 smithi086 ceph-mon[32028]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:08 smithi194 ceph-mon[35865]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:08 smithi196 ceph-mon[36373]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:09.797 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 9 on host 'smithi086' 2024-04-16T05:31:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.798+0000 7f73a5ffb700 1 -- 172.21.15.86:0/1757996945 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f73a8062f30 con 0x7f73980a3c40 2024-04-16T05:31:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f73980a3c40 msgr2=0x7f73980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f73980a3c40 0x7f73980a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f73a8102aa0 tx=0x7f739c008040 comp rx=0 tx=0).stop 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a81035e0 msgr2=0x7f73a819f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a81035e0 0x7f73a819f2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73a000add0 tx=0x7f73a000c330 comp rx=0 tx=0).stop 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 shutdown_connections 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a81035e0 0x7f73a819f2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73a8101a40 0x7f73a8197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f73980a3c40 0x7f73980a6100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 --2- 172.21.15.86:0/1757996945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a8102c40 0x7f73a8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 >> 172.21.15.86:0/1757996945 conn(0x7f73a80fcfe0 msgr2=0x7f73a8105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 shutdown_connections 2024-04-16T05:31:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:31:09.800+0000 7f73aef0d700 1 -- 172.21.15.86:0/1757996945 wait complete. 2024-04-16T05:31:10.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:10 smithi086 ceph-mon[32028]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:10.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:10 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:10.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:10 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:10.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:10 smithi086 ceph-mon[32028]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T05:31:10.813 DEBUG:teuthology.orchestra.run.smithi086:osd.9> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.9.service 2024-04-16T05:31:10.815 INFO:tasks.cephadm:Deploying osd.10 on smithi194 with /dev/vg_nvme/lv_1... 2024-04-16T05:31:10.815 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T05:31:10.845 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:10 smithi194 ceph-mon[35865]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:10.845 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:10 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:10.845 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:10 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:10.845 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:10 smithi194 ceph-mon[35865]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T05:31:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:10 smithi196 ceph-mon[36373]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:10 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:10 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:10 smithi196 ceph-mon[36373]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T05:31:11.062 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:31:11.869 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:11 smithi086 ceph-mon[32028]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T05:31:11.869 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:11 smithi086 ceph-mon[32028]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T05:31:11.869 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:11 smithi086 ceph-mon[32028]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:31:11.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:11 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:11.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:11 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:11.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:11 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:11.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:11 smithi086 ceph-mon[32028]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:12.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:11 smithi194 ceph-mon[35865]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T05:31:12.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:11 smithi194 ceph-mon[35865]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T05:31:12.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:11 smithi194 ceph-mon[35865]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:31:12.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:11 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:12.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:11 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:12.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:11 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:12.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:11 smithi194 ceph-mon[35865]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:12.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:11 smithi196 ceph-mon[36373]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T05:31:12.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:11 smithi196 ceph-mon[36373]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T05:31:12.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:11 smithi196 ceph-mon[36373]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-16T05:31:12.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:11 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:12.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:11 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:12.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:11 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:12.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:11 smithi196 ceph-mon[36373]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:12.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:31:12 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:31:12.176+0000 7f14087e3700 -1 osd.9 0 waiting for initial osdmap 2024-04-16T05:31:12.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:31:12 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:31:12.190+0000 7f14005ce700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:31:12.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:12 smithi086 ceph-mon[32028]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:31:12.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:12 smithi086 ceph-mon[32028]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T05:31:12.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:12 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:12.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:12 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:12.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:12 smithi086 ceph-mon[32028]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' 2024-04-16T05:31:13.059 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:12 smithi194 ceph-mon[35865]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:31:13.059 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:12 smithi194 ceph-mon[35865]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T05:31:13.059 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:12 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:13.059 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:12 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:13.059 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:12 smithi194 ceph-mon[35865]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' 2024-04-16T05:31:13.059 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T05:31:13.060 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T05:31:13.060 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-16T05:31:13.060 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-16T05:31:13.060 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011306 s, 927 MB/s 2024-04-16T05:31:13.060 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-16T05:31:13.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:12 smithi196 ceph-mon[36373]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-16T05:31:13.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:12 smithi196 ceph-mon[36373]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T05:31:13.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:12 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:13.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:12 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:13.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:12 smithi196 ceph-mon[36373]: from='osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579]' entity='osd.9' 2024-04-16T05:31:13.770 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_1 2024-04-16T05:31:14.057 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:31:14.103 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:13 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:31:14.103 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:13 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:31:14.103 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:13 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:14.103 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:13 smithi086 ceph-mon[32028]: osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] boot 2024-04-16T05:31:14.103 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:13 smithi086 ceph-mon[32028]: osdmap e67: 10 total, 10 up, 10 in 2024-04-16T05:31:14.103 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:13 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:14.103 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:13 smithi086 ceph-mon[32028]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:14.152 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:13 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:31:14.152 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:13 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:31:14.152 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:13 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:14.153 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:13 smithi194 ceph-mon[35865]: osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] boot 2024-04-16T05:31:14.153 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:13 smithi194 ceph-mon[35865]: osdmap e67: 10 total, 10 up, 10 in 2024-04-16T05:31:14.153 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:13 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:14.153 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:13 smithi194 ceph-mon[35865]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:14.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:13 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:31:14.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:13 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:31:14.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:13 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:14.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:13 smithi196 ceph-mon[36373]: osd.9 [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] boot 2024-04-16T05:31:14.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:13 smithi196 ceph-mon[36373]: osdmap e67: 10 total, 10 up, 10 in 2024-04-16T05:31:14.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:13 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:31:14.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:13 smithi196 ceph-mon[36373]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:31:15.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T05:31:15.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:15.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:15.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:15.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:15.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T05:31:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:15.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.707+0000 7f106e90e700 1 -- 172.21.15.194:0/2285629068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068103de0 msgr2=0x7f1068104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.707+0000 7f106e90e700 1 --2- 172.21.15.194:0/2285629068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068103de0 0x7f1068104240 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1050009990 tx=0x7f1050009ca0 comp rx=0 tx=0).stop 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.707+0000 7f106e90e700 1 -- 172.21.15.194:0/2285629068 shutdown_connections 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.707+0000 7f106e90e700 1 --2- 172.21.15.194:0/2285629068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1068104780 0x7f106810b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.707+0000 7f106e90e700 1 --2- 172.21.15.194:0/2285629068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1068100e00 0x7f1068101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.707+0000 7f106e90e700 1 --2- 172.21.15.194:0/2285629068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068103de0 0x7f1068104240 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.707+0000 7f106e90e700 1 -- 172.21.15.194:0/2285629068 >> 172.21.15.194:0/2285629068 conn(0x7f10680fc780 msgr2=0x7f10680feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.707+0000 7f106e90e700 1 -- 172.21.15.194:0/2285629068 shutdown_connections 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.707+0000 7f106e90e700 1 -- 172.21.15.194:0/2285629068 wait complete. 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106e90e700 1 Processor -- start 2024-04-16T05:31:15.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106e90e700 1 -- start start 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106e90e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068100e00 0x7f1068198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106e90e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1068103de0 0x7f10681991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106e90e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1068104780 0x7f10681a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106e90e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f106810e6f0 con 0x7f1068104780 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106e90e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f106810e570 con 0x7f1068100e00 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106e90e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f106810e870 con 0x7f1068103de0 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f1067fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068100e00 0x7f1068198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f1067fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068100e00 0x7f1068198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:53852/0 (socket says 172.21.15.194:53852) 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f1067fff700 1 -- 172.21.15.194:0/2266102614 learned_addr learned my addr 172.21.15.194:0/2266102614 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106ceab700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1068104780 0x7f10681a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f105ffff700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1068103de0 0x7f10681991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:15.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f1067fff700 1 -- 172.21.15.194:0/2266102614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1068103de0 msgr2=0x7f10681991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f1067fff700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1068103de0 0x7f10681991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f1067fff700 1 -- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1068104780 msgr2=0x7f10681a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f1067fff700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1068104780 0x7f10681a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f1067fff700 1 -- 172.21.15.194:0/2266102614 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10681a0920 con 0x7f1068100e00 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.708+0000 7f106ceab700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1068104780 0x7f10681a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.709+0000 7f1067fff700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068100e00 0x7f1068198ca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f105800d980 tx=0x7f105800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.709+0000 7f1065ffb700 1 -- 172.21.15.194:0/2266102614 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058017030 con 0x7f1068100e00 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.709+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10681a0c10 con 0x7f1068100e00 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.709+0000 7f1065ffb700 1 -- 172.21.15.194:0/2266102614 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1058004500 con 0x7f1068100e00 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.709+0000 7f1065ffb700 1 -- 172.21.15.194:0/2266102614 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058003b20 con 0x7f1068100e00 2024-04-16T05:31:15.710 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.709+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10681a11a0 con 0x7f1068100e00 2024-04-16T05:31:15.711 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.710+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10680697e0 con 0x7f1068100e00 2024-04-16T05:31:15.711 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.711+0000 7f1065ffb700 1 -- 172.21.15.194:0/2266102614 <== mon.2 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f105801a070 con 0x7f1068100e00 2024-04-16T05:31:15.711 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.712+0000 7f1065ffb700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f10540a3cc0 0x7f10540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:15.712 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.712+0000 7f1065ffb700 1 -- 172.21.15.194:0/2266102614 <== mon.2 v2:172.21.15.194:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f10580cf900 con 0x7f1068100e00 2024-04-16T05:31:15.712 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.712+0000 7f105ffff700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f10540a3cc0 0x7f10540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:15.712 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.712+0000 7f105ffff700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f10540a3cc0 0x7f10540a6180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10500039c0 tx=0x7f1050005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:15.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.715+0000 7f1065ffb700 1 -- 172.21.15.194:0/2266102614 <== mon.2 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1058009940 con 0x7f1068100e00 2024-04-16T05:31:15.921 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:15.922+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1068062f30 con 0x7f10540a3cc0 2024-04-16T05:31:16.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:16 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi086 2024-04-16T05:31:16.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:16 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-16T05:31:16.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:16 smithi086 ceph-mon[32028]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:16.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:16 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:31:16.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:16 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:31:16.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:16 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:16 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi086 2024-04-16T05:31:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:16 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-16T05:31:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:16 smithi196 ceph-mon[36373]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:16 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:31:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:16 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:31:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:16 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:16.531 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:16 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi086 2024-04-16T05:31:16.531 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:16 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-16T05:31:16.531 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:16 smithi194 ceph-mon[35865]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:16.531 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:16 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:31:16.531 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:16 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:31:16.531 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:16 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:17.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:17 smithi086 ceph-mon[32028]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:17.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:17 smithi196 ceph-mon[36373]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:17.566 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:17 smithi194 ceph-mon[35865]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:18.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:18 smithi194 ceph-mon[35865]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:18.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:18 smithi086 ceph-mon[32028]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:18.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:18 smithi196 ceph-mon[36373]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:19.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:19 smithi086 ceph-mon[32028]: from='client.? 172.21.15.194:0/3944508022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d"}]: dispatch 2024-04-16T05:31:19.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:19 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d"}]: dispatch 2024-04-16T05:31:19.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:19 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d"}]': finished 2024-04-16T05:31:19.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:19 smithi086 ceph-mon[32028]: osdmap e69: 11 total, 10 up, 11 in 2024-04-16T05:31:19.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:19 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:19.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:19 smithi196 ceph-mon[36373]: from='client.? 172.21.15.194:0/3944508022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d"}]: dispatch 2024-04-16T05:31:19.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:19 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d"}]: dispatch 2024-04-16T05:31:19.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:19 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d"}]': finished 2024-04-16T05:31:19.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:19 smithi196 ceph-mon[36373]: osdmap e69: 11 total, 10 up, 11 in 2024-04-16T05:31:19.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:19 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:19.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:19 smithi194 ceph-mon[35865]: from='client.? 172.21.15.194:0/3944508022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d"}]: dispatch 2024-04-16T05:31:19.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:19 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d"}]: dispatch 2024-04-16T05:31:19.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:19 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d"}]': finished 2024-04-16T05:31:19.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:19 smithi194 ceph-mon[35865]: osdmap e69: 11 total, 10 up, 11 in 2024-04-16T05:31:19.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:19 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:20.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:20 smithi086 ceph-mon[32028]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:20.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:20 smithi086 ceph-mon[32028]: from='client.? 172.21.15.194:0/2011839309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:20 smithi196 ceph-mon[36373]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:20 smithi196 ceph-mon[36373]: from='client.? 172.21.15.194:0/2011839309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:20.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:20 smithi194 ceph-mon[35865]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:20 smithi194 ceph-mon[35865]: from='client.? 172.21.15.194:0/2011839309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:22.795 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:22 smithi194 ceph-mon[35865]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:22.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:22 smithi086 ceph-mon[32028]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:22 smithi196 ceph-mon[36373]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:24 smithi194 ceph-mon[35865]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:24 smithi086 ceph-mon[32028]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:24 smithi196 ceph-mon[36373]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:26.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:26 smithi194 ceph-mon[35865]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:26.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T05:31:26.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:26.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:26 smithi086 ceph-mon[32028]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:26.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T05:31:26.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:26 smithi196 ceph-mon[36373]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T05:31:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:27.877 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:27 smithi194 ceph-mon[35865]: Deploying daemon osd.10 on smithi194 2024-04-16T05:31:27.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:27 smithi086 ceph-mon[32028]: Deploying daemon osd.10 on smithi194 2024-04-16T05:31:27.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:27 smithi196 ceph-mon[36373]: Deploying daemon osd.10 on smithi194 2024-04-16T05:31:28.590 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:28 smithi194 ceph-mon[35865]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:28.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:28 smithi086 ceph-mon[32028]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:28.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:28 smithi196 ceph-mon[36373]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:30 smithi194 ceph-mon[35865]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:30.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:30 smithi086 ceph-mon[32028]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:30 smithi196 ceph-mon[36373]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:32.725 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:32 smithi194 ceph-mon[35865]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:32.725 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:32.725 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:32.725 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:32.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:32 smithi086 ceph-mon[32028]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:32.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:32 smithi196 ceph-mon[36373]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:34.606 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:34 smithi194 ceph-mon[35865]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:34.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:34 smithi086 ceph-mon[32028]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:34.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:34 smithi196 ceph-mon[36373]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:35.937 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.938+0000 7f1065ffb700 1 -- 172.21.15.194:0/2266102614 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1068062f30 con 0x7f10540a3cc0 2024-04-16T05:31:35.937 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 10 on host 'smithi194' 2024-04-16T05:31:35.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f10540a3cc0 msgr2=0x7f10540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:35.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f10540a3cc0 0x7f10540a6180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10500039c0 tx=0x7f1050005fb0 comp rx=0 tx=0).stop 2024-04-16T05:31:35.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068100e00 msgr2=0x7f1068198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:35.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068100e00 0x7f1068198ca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f105800d980 tx=0x7f105800dd40 comp rx=0 tx=0).stop 2024-04-16T05:31:35.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 shutdown_connections 2024-04-16T05:31:35.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1068104780 0x7f10681a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:35.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1068100e00 0x7f1068198ca0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:35.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f10540a3cc0 0x7f10540a6180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:35.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 --2- 172.21.15.194:0/2266102614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1068103de0 0x7f10681991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:35.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 >> 172.21.15.194:0/2266102614 conn(0x7f10680fc780 msgr2=0x7f1068107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:35.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 shutdown_connections 2024-04-16T05:31:35.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-16T05:31:35.941+0000 7f106e90e700 1 -- 172.21.15.194:0/2266102614 wait complete. 2024-04-16T05:31:36.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:36.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:36.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:36.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:36.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='osd.10 [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:36.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:36 smithi194 ceph-mon[35865]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='osd.10 [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:36 smithi086 ceph-mon[32028]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:36.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='osd.10 [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:36.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:36 smithi196 ceph-mon[36373]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:36.599 DEBUG:teuthology.orchestra.run.smithi194:osd.10> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.10.service 2024-04-16T05:31:36.601 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-04-16T05:31:36.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T05:31:36.846 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:31:38.100 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:37 smithi194 ceph-mon[35865]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T05:31:38.100 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:37 smithi194 ceph-mon[35865]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T05:31:38.100 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:37 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:38.100 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:37 smithi194 ceph-mon[35865]: from='osd.10 [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:31:38.100 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:37 smithi194 ceph-mon[35865]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:31:38.100 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:37 smithi194 ceph-mon[35865]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:38.101 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:31:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:31:37.943+0000 7f0a280ef700 -1 osd.10 0 waiting for initial osdmap 2024-04-16T05:31:38.101 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:31:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:31:37.960+0000 7f0a23ee2700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:31:38.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:37 smithi086 ceph-mon[32028]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T05:31:38.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:37 smithi086 ceph-mon[32028]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T05:31:38.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:37 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:38.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:37 smithi086 ceph-mon[32028]: from='osd.10 [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:31:38.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:37 smithi086 ceph-mon[32028]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:31:38.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:37 smithi086 ceph-mon[32028]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:38.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:37 smithi196 ceph-mon[36373]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T05:31:38.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:37 smithi196 ceph-mon[36373]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T05:31:38.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:37 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:38.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:37 smithi196 ceph-mon[36373]: from='osd.10 [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:31:38.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:37 smithi196 ceph-mon[36373]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-16T05:31:38.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:37 smithi196 ceph-mon[36373]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:39.037 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T05:31:39.037 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T05:31:39.037 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-16T05:31:39.037 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-16T05:31:39.037 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111301 s, 942 MB/s 2024-04-16T05:31:39.037 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-16T05:31:39.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:38 smithi194 ceph-mon[35865]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:31:39.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:38 smithi194 ceph-mon[35865]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T05:31:39.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:38 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:39.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:38 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:39.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:38 smithi086 ceph-mon[32028]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:31:39.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:38 smithi086 ceph-mon[32028]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T05:31:39.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:38 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:39.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:38 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:39.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:38 smithi196 ceph-mon[36373]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-16T05:31:39.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:38 smithi196 ceph-mon[36373]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T05:31:39.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:38 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:39.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:38 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:39.683 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-04-16T05:31:39.933 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:31:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:31:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:31:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: osd.10 [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] boot 2024-04-16T05:31:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: osdmap e72: 11 total, 11 up, 11 in 2024-04-16T05:31:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:40.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:39 smithi086 ceph-mon[32028]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T05:31:40.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:31:40.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:31:40.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:40.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: osd.10 [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] boot 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: osdmap e72: 11 total, 11 up, 11 in 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:39 smithi196 ceph-mon[36373]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: osd.10 [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] boot 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: osdmap e72: 11 total, 11 up, 11 in 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:40.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:40.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:40.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:40.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:39 smithi194 ceph-mon[35865]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T05:31:41.076 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:40 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi194 2024-04-16T05:31:41.076 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:40 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi194 to 4270M 2024-04-16T05:31:41.077 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:40 smithi196 ceph-mon[36373]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T05:31:41.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:40 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi194 2024-04-16T05:31:41.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:40 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi194 to 4270M 2024-04-16T05:31:41.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:40 smithi086 ceph-mon[32028]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T05:31:41.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:40 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi194 2024-04-16T05:31:41.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:40 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi194 to 4270M 2024-04-16T05:31:41.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:40 smithi194 ceph-mon[35865]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T05:31:41.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.633+0000 7fad5401c700 1 -- 172.21.15.196:0/50907962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 msgr2=0x7fad4c101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:41.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.633+0000 7fad5401c700 1 --2- 172.21.15.196:0/50907962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 0x7fad4c101e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad48009990 tx=0x7fad48009ca0 comp rx=0 tx=0).stop 2024-04-16T05:31:41.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.633+0000 7fad5401c700 1 -- 172.21.15.196:0/50907962 shutdown_connections 2024-04-16T05:31:41.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.633+0000 7fad5401c700 1 --2- 172.21.15.196:0/50907962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c1035f0 0x7fad4c109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:41.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.633+0000 7fad5401c700 1 --2- 172.21.15.196:0/50907962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 0x7fad4c101e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:41.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.633+0000 7fad5401c700 1 --2- 172.21.15.196:0/50907962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad4c102c50 0x7fad4c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:41.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.633+0000 7fad5401c700 1 -- 172.21.15.196:0/50907962 >> 172.21.15.196:0/50907962 conn(0x7fad4c0fcff0 msgr2=0x7fad4c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:41.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.633+0000 7fad5401c700 1 -- 172.21.15.196:0/50907962 shutdown_connections 2024-04-16T05:31:41.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.633+0000 7fad5401c700 1 -- 172.21.15.196:0/50907962 wait complete. 2024-04-16T05:31:41.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad5401c700 1 Processor -- start 2024-04-16T05:31:41.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad5401c700 1 -- start start 2024-04-16T05:31:41.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad5401c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 0x7fad4c105990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:41.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad5401c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c102c50 0x7fad4c104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad5401c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad4c1035f0 0x7fad4c104570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad5401c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10d710 con 0x7fad4c102c50 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad5401c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10d590 con 0x7fad4c1035f0 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad5401c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10d890 con 0x7fad4c101a50 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad51db8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 0x7fad4c105990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad51db8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 0x7fad4c105990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53628/0 (socket says 172.21.15.196:53628) 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad51db8700 1 -- 172.21.15.196:0/1598278566 learned_addr learned my addr 172.21.15.196:0/1598278566 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad525b9700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad4c1035f0 0x7fad4c104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad515b7700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c102c50 0x7fad4c104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad51db8700 1 -- 172.21.15.196:0/1598278566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad4c1035f0 msgr2=0x7fad4c104570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad51db8700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad4c1035f0 0x7fad4c104570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad51db8700 1 -- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c102c50 msgr2=0x7fad4c104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad51db8700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c102c50 0x7fad4c104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:41.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad51db8700 1 -- 172.21.15.196:0/1598278566 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c104e40 con 0x7fad4c101a50 2024-04-16T05:31:41.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.634+0000 7fad515b7700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c102c50 0x7fad4c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:31:41.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.635+0000 7fad51db8700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 0x7fad4c105990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad48000c00 tx=0x7fad480047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:41.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.635+0000 7fad42ffd700 1 -- 172.21.15.196:0/1598278566 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48003d30 con 0x7fad4c101a50 2024-04-16T05:31:41.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.635+0000 7fad42ffd700 1 -- 172.21.15.196:0/1598278566 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad48004830 con 0x7fad4c101a50 2024-04-16T05:31:41.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.635+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c1a2ff0 con 0x7fad4c101a50 2024-04-16T05:31:41.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.635+0000 7fad42ffd700 1 -- 172.21.15.196:0/1598278566 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48005580 con 0x7fad4c101a50 2024-04-16T05:31:41.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.635+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c1a3390 con 0x7fad4c101a50 2024-04-16T05:31:41.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.636+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c0697e0 con 0x7fad4c101a50 2024-04-16T05:31:41.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.637+0000 7fad42ffd700 1 -- 172.21.15.196:0/1598278566 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fad48003ed0 con 0x7fad4c101a50 2024-04-16T05:31:41.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.637+0000 7fad42ffd700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fad380a3c40 0x7fad380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:41.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.638+0000 7fad42ffd700 1 -- 172.21.15.196:0/1598278566 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7fad480d1420 con 0x7fad4c101a50 2024-04-16T05:31:41.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.638+0000 7fad515b7700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fad380a3c40 0x7fad380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:41.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.638+0000 7fad515b7700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fad380a3c40 0x7fad380a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0099d0 tx=0x7fad3c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:41.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.641+0000 7fad42ffd700 1 -- 172.21.15.196:0/1598278566 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad48013070 con 0x7fad4c101a50 2024-04-16T05:31:41.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:31:41.853+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fad4c06b930 con 0x7fad380a3c40 2024-04-16T05:31:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:41 smithi196 ceph-mon[36373]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:31:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:41 smithi196 ceph-mon[36373]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T05:31:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:31:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:31:42.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:42.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:41 smithi194 ceph-mon[35865]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:31:42.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:41 smithi194 ceph-mon[35865]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T05:31:42.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:31:42.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:31:42.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:42.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:41 smithi086 ceph-mon[32028]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:31:42.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:41 smithi086 ceph-mon[32028]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T05:31:42.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:31:42.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:31:42.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:43.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:42 smithi196 ceph-mon[36373]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:43.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:42 smithi196 ceph-mon[36373]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T05:31:43.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:42 smithi194 ceph-mon[35865]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:43.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:42 smithi194 ceph-mon[35865]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T05:31:43.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:42 smithi086 ceph-mon[32028]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:43.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:42 smithi086 ceph-mon[32028]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T05:31:44.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:43 smithi196 ceph-mon[36373]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:31:44.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:43 smithi194 ceph-mon[35865]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:31:44.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:43 smithi086 ceph-mon[32028]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:31:45.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:44 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/853591087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66ee3110-b716-48c3-b625-d3a5f40beb5f"}]: dispatch 2024-04-16T05:31:45.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:44 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66ee3110-b716-48c3-b625-d3a5f40beb5f"}]: dispatch 2024-04-16T05:31:45.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:44 smithi196 ceph-mon[36373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66ee3110-b716-48c3-b625-d3a5f40beb5f"}]': finished 2024-04-16T05:31:45.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:44 smithi196 ceph-mon[36373]: osdmap e77: 12 total, 11 up, 12 in 2024-04-16T05:31:45.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:45.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:44 smithi194 ceph-mon[35865]: from='client.? 172.21.15.196:0/853591087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66ee3110-b716-48c3-b625-d3a5f40beb5f"}]: dispatch 2024-04-16T05:31:45.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:44 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66ee3110-b716-48c3-b625-d3a5f40beb5f"}]: dispatch 2024-04-16T05:31:45.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:44 smithi194 ceph-mon[35865]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66ee3110-b716-48c3-b625-d3a5f40beb5f"}]': finished 2024-04-16T05:31:45.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:44 smithi194 ceph-mon[35865]: osdmap e77: 12 total, 11 up, 12 in 2024-04-16T05:31:45.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:45.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:44 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/853591087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66ee3110-b716-48c3-b625-d3a5f40beb5f"}]: dispatch 2024-04-16T05:31:45.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:44 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66ee3110-b716-48c3-b625-d3a5f40beb5f"}]: dispatch 2024-04-16T05:31:45.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:44 smithi086 ceph-mon[32028]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66ee3110-b716-48c3-b625-d3a5f40beb5f"}]': finished 2024-04-16T05:31:45.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:44 smithi086 ceph-mon[32028]: osdmap e77: 12 total, 11 up, 12 in 2024-04-16T05:31:45.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:46.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:45 smithi194 ceph-mon[35865]: from='client.? 172.21.15.196:0/2381466188' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:46.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:45 smithi194 ceph-mon[35865]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-16T05:31:46.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:45 smithi086 ceph-mon[32028]: from='client.? 172.21.15.196:0/2381466188' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:46.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:45 smithi086 ceph-mon[32028]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-16T05:31:46.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:45 smithi196 ceph-mon[36373]: from='client.? 172.21.15.196:0/2381466188' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:46.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:45 smithi196 ceph-mon[36373]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-16T05:31:48.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:48 smithi196 ceph-mon[36373]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:48.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:48 smithi194 ceph-mon[35865]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:48.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:48 smithi086 ceph-mon[32028]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:50.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:50 smithi196 ceph-mon[36373]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:50.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:50 smithi194 ceph-mon[35865]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:50.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:50 smithi086 ceph-mon[32028]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:52.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:52 smithi196 ceph-mon[36373]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:52.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T05:31:52.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:52 smithi194 ceph-mon[35865]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T05:31:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:52.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:52 smithi086 ceph-mon[32028]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:52.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T05:31:52.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:53.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:53 smithi196 ceph-mon[36373]: Deploying daemon osd.11 on smithi196 2024-04-16T05:31:53.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:53 smithi194 ceph-mon[35865]: Deploying daemon osd.11 on smithi196 2024-04-16T05:31:53.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:53 smithi086 ceph-mon[32028]: Deploying daemon osd.11 on smithi196 2024-04-16T05:31:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:54 smithi194 ceph-mon[35865]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:54.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:54 smithi086 ceph-mon[32028]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:54 smithi196 ceph-mon[36373]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:56 smithi196 ceph-mon[36373]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:56.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:56 smithi194 ceph-mon[35865]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:56.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:56 smithi086 ceph-mon[32028]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:58.879 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:58 smithi196 ceph-mon[36373]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:58.879 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:58.879 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:58.879 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:31:58 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:58 smithi194 ceph-mon[35865]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:31:58 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:58.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:58 smithi086 ceph-mon[32028]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:31:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:31:58 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:00.800 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:00 smithi196 ceph-mon[36373]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:32:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:00 smithi194 ceph-mon[35865]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:32:00.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:00 smithi086 ceph-mon[32028]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:32:02.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.121+0000 7fad42ffd700 1 -- 172.21.15.196:0/1598278566 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fad4c06b930 con 0x7fad380a3c40 2024-04-16T05:32:02.120 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-04-16T05:32:02.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fad380a3c40 msgr2=0x7fad380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:02.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fad380a3c40 0x7fad380a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0099d0 tx=0x7fad3c008040 comp rx=0 tx=0).stop 2024-04-16T05:32:02.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 msgr2=0x7fad4c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:02.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 0x7fad4c105990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad48000c00 tx=0x7fad480047b0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 shutdown_connections 2024-04-16T05:32:02.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad4c1035f0 0x7fad4c104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fad380a3c40 0x7fad380a6100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4c101a50 0x7fad4c105990 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 --2- 172.21.15.196:0/1598278566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c102c50 0x7fad4c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 >> 172.21.15.196:0/1598278566 conn(0x7fad4c0fcff0 msgr2=0x7fad4c06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:02.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 shutdown_connections 2024-04-16T05:32:02.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:02.123+0000 7fad5401c700 1 -- 172.21.15.196:0/1598278566 wait complete. 2024-04-16T05:32:02.431 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.431 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.431 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:02.432 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:02.432 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.432 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:32:02.432 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.432 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.432 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='osd.11 [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:32:02.432 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:32:02.432 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:02 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='osd.11 [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:32:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:02 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:02.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='osd.11 [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:32:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:02 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:02.778 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.11.service 2024-04-16T05:32:02.780 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-16T05:32:02.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd stat -f json 2024-04-16T05:32:03.032 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:32:04.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:04 smithi196 ceph-mon[36373]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T05:32:04.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:04 smithi196 ceph-mon[36373]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T05:32:04.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:04 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:32:04.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:04 smithi196 ceph-mon[36373]: from='osd.11 [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:32:04.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:04 smithi196 ceph-mon[36373]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:32:04.236 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:04 smithi196 ceph-mon[36373]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:32:04.236 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:32:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:32:04.116+0000 7fe85bf69700 -1 osd.11 0 waiting for initial osdmap 2024-04-16T05:32:04.236 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:32:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:32:04.132+0000 7fe857d5c700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:32:04.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:04 smithi194 ceph-mon[35865]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T05:32:04.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:04 smithi194 ceph-mon[35865]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T05:32:04.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:04 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:32:04.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:04 smithi194 ceph-mon[35865]: from='osd.11 [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:32:04.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:04 smithi194 ceph-mon[35865]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:32:04.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:04 smithi194 ceph-mon[35865]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:32:04.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:04 smithi086 ceph-mon[32028]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T05:32:04.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:04 smithi086 ceph-mon[32028]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T05:32:04.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:04 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:32:04.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:04 smithi086 ceph-mon[32028]: from='osd.11 [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:32:04.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:04 smithi086 ceph-mon[32028]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-16T05:32:04.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:04 smithi086 ceph-mon[32028]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:32:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/4163511827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98103d50 msgr2=0x7f6a981041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/4163511827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98103d50 0x7f6a981041b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a880099c0 tx=0x7f6a88009cd0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/4163511827 shutdown_connections 2024-04-16T05:32:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/4163511827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a981046f0 0x7f6a9810afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/4163511827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a98102b50 0x7f6a98102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/4163511827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98103d50 0x7f6a981041b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/4163511827 >> 172.21.15.86:0/4163511827 conn(0x7f6a980fe0f0 msgr2=0x7f6a98100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/4163511827 shutdown_connections 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/4163511827 wait complete. 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 Processor -- start 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.973+0000 7f6a9fa6b700 1 -- start start 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9fa6b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98102b50 0x7f6a98076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9fa6b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a98103d50 0x7f6a98074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9fa6b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a981046f0 0x7f6a98075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9fa6b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9810e740 con 0x7f6a98102b50 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9fa6b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9810e5c0 con 0x7f6a981046f0 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9fa6b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9810e8c0 con 0x7f6a98103d50 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d807700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98102b50 0x7f6a98076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d807700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98102b50 0x7f6a98076890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48188/0 (socket says 172.21.15.86:48188) 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d807700 1 -- 172.21.15.86:0/411260093 learned_addr learned my addr 172.21.15.86:0/411260093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9e008700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a981046f0 0x7f6a98075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d006700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a98103d50 0x7f6a98074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d807700 1 -- 172.21.15.86:0/411260093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a98103d50 msgr2=0x7f6a98074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d807700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a98103d50 0x7f6a98074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d807700 1 -- 172.21.15.86:0/411260093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a981046f0 msgr2=0x7f6a98075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d807700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a981046f0 0x7f6a98075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d807700 1 -- 172.21.15.86:0/411260093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a98075cd0 con 0x7f6a98102b50 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9d807700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98102b50 0x7f6a98076890 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d980 tx=0x7f6a9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a8effd700 1 -- 172.21.15.86:0/411260093 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94016070 con 0x7f6a98102b50 2024-04-16T05:32:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a8effd700 1 -- 172.21.15.86:0/411260093 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9400fd70 con 0x7f6a98102b50 2024-04-16T05:32:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a981a2ef0 con 0x7f6a98102b50 2024-04-16T05:32:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.974+0000 7f6a8effd700 1 -- 172.21.15.86:0/411260093 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94019360 con 0x7f6a98102b50 2024-04-16T05:32:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.975+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a981a32f0 con 0x7f6a98102b50 2024-04-16T05:32:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.976+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9802dfe0 con 0x7f6a98102b50 2024-04-16T05:32:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.977+0000 7f6a8effd700 1 -- 172.21.15.86:0/411260093 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6a94019070 con 0x7f6a98102b50 2024-04-16T05:32:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.977+0000 7f6a8effd700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a840a3c40 0x7f6a840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.977+0000 7f6a9d006700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a840a3c40 0x7f6a840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.977+0000 7f6a8effd700 1 -- 172.21.15.86:0/411260093 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f6a940cf4e0 con 0x7f6a98102b50 2024-04-16T05:32:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.978+0000 7f6a9d006700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a840a3c40 0x7f6a840a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:04.982+0000 7f6a8effd700 1 -- 172.21.15.86:0/411260093 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a9409c480 con 0x7f6a98102b50 2024-04-16T05:32:05.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.186+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6a98063810 con 0x7f6a98102b50 2024-04-16T05:32:05.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.186+0000 7f6a8effd700 1 -- 172.21.15.86:0/411260093 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f6a9409bd80 con 0x7f6a98102b50 2024-04-16T05:32:05.185 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:05.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a840a3c40 msgr2=0x7f6a840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:05.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a840a3c40 0x7f6a840a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88005fb0 comp rx=0 tx=0).stop 2024-04-16T05:32:05.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98102b50 msgr2=0x7f6a98076890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:05.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98102b50 0x7f6a98076890 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d980 tx=0x7f6a9400dc90 comp rx=0 tx=0).stop 2024-04-16T05:32:05.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 shutdown_connections 2024-04-16T05:32:05.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a981046f0 0x7f6a98075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98102b50 0x7f6a98076890 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a840a3c40 0x7f6a840a6100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 --2- 172.21.15.86:0/411260093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a98103d50 0x7f6a98074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 >> 172.21.15.86:0/411260093 conn(0x7f6a980fe0f0 msgr2=0x7f6a98107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 shutdown_connections 2024-04-16T05:32:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:05.188+0000 7f6a9fa6b700 1 -- 172.21.15.86:0/411260093 wait complete. 2024-04-16T05:32:05.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:05 smithi194 ceph-mon[35865]: purged_snaps scrub starts 2024-04-16T05:32:05.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:05 smithi194 ceph-mon[35865]: purged_snaps scrub ok 2024-04-16T05:32:05.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:05 smithi194 ceph-mon[35865]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:32:05.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:05 smithi194 ceph-mon[35865]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T05:32:05.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:05 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:32:05.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:05 smithi086 ceph-mon[32028]: purged_snaps scrub starts 2024-04-16T05:32:05.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:05 smithi086 ceph-mon[32028]: purged_snaps scrub ok 2024-04-16T05:32:05.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:05 smithi086 ceph-mon[32028]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:32:05.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:05 smithi086 ceph-mon[32028]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T05:32:05.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:05 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:32:05.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:05 smithi196 ceph-mon[36373]: purged_snaps scrub starts 2024-04-16T05:32:05.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:05 smithi196 ceph-mon[36373]: purged_snaps scrub ok 2024-04-16T05:32:05.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:05 smithi196 ceph-mon[36373]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-16T05:32:05.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:05 smithi196 ceph-mon[36373]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T05:32:05.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:05 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:32:05.974 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1713245525,"num_in_osds":12,"osd_in_since":1713245504,"num_remapped_pgs":0} 2024-04-16T05:32:05.975 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd dump --format=json 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: osd.11 [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] boot 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: osdmap e80: 12 total, 12 up, 12 in 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/411260093' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:06.219 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:06 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.223 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:32:06.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: osd.11 [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] boot 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: osdmap e80: 12 total, 12 up, 12 in 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/411260093' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:06.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:06.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:06 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: osd.11 [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] boot 2024-04-16T05:32:06.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: osdmap e80: 12 total, 12 up, 12 in 2024-04-16T05:32:06.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:32:06.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/411260093' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T05:32:06.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:06.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:32:06.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:06.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:06.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:06.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:06 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:07.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:07 smithi086 ceph-mon[32028]: Detected new or changed devices on smithi196 2024-04-16T05:32:07.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:07 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi196 to 4270M 2024-04-16T05:32:07.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:07 smithi086 ceph-mon[32028]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T05:32:07.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:07 smithi194 ceph-mon[35865]: Detected new or changed devices on smithi196 2024-04-16T05:32:07.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:07 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi196 to 4270M 2024-04-16T05:32:07.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:07 smithi194 ceph-mon[35865]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T05:32:07.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:07 smithi196 ceph-mon[36373]: Detected new or changed devices on smithi196 2024-04-16T05:32:07.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:07 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi196 to 4270M 2024-04-16T05:32:07.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:07 smithi196 ceph-mon[36373]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T05:32:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.887+0000 7f4ad4762700 1 -- 172.21.15.86:0/1590090825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 msgr2=0x7f4acc108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.887+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1590090825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 0x7f4acc108ea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009990 tx=0x7f4ac8009ca0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.887+0000 7f4ad4762700 1 -- 172.21.15.86:0/1590090825 shutdown_connections 2024-04-16T05:32:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.887+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1590090825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4acc103570 0x7f4acc10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.887+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1590090825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 0x7f4acc108ea0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.887+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1590090825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc102bb0 0x7f4acc103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.887+0000 7f4ad4762700 1 -- 172.21.15.86:0/1590090825 >> 172.21.15.86:0/1590090825 conn(0x7f4acc079cb0 msgr2=0x7f4acc07a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.887+0000 7f4ad4762700 1 -- 172.21.15.86:0/1590090825 shutdown_connections 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.887+0000 7f4ad4762700 1 -- 172.21.15.86:0/1590090825 wait complete. 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad4762700 1 Processor -- start 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad4762700 1 -- start start 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad4762700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc102bb0 0x7f4acc199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad4762700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4acc103570 0x7f4acc199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad4762700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 0x7f4acc19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad4762700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc112eb0 con 0x7f4acc108ac0 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad4762700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc112d30 con 0x7f4acc103570 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad4762700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc113030 con 0x7f4acc102bb0 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad2cff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 0x7f4acc19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad2cff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 0x7f4acc19dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48212/0 (socket says 172.21.15.86:48212) 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad2cff700 1 -- 172.21.15.86:0/1882645720 learned_addr learned my addr 172.21.15.86:0/1882645720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.888+0000 7f4ad24fe700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc102bb0 0x7f4acc199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4ad1cfd700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4acc103570 0x7f4acc199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4ad2cff700 1 -- 172.21.15.86:0/1882645720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc102bb0 msgr2=0x7f4acc199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4ad2cff700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc102bb0 0x7f4acc199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4ad2cff700 1 -- 172.21.15.86:0/1882645720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4acc103570 msgr2=0x7f4acc199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4ad2cff700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4acc103570 0x7f4acc199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4ad2cff700 1 -- 172.21.15.86:0/1882645720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc19e410 con 0x7f4acc108ac0 2024-04-16T05:32:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4ad2cff700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 0x7f4acc19dcd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400b970 tx=0x7f4ac400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4abf7fe700 1 -- 172.21.15.86:0/1882645720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4019070 con 0x7f4acc108ac0 2024-04-16T05:32:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc19e700 con 0x7f4acc108ac0 2024-04-16T05:32:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4abf7fe700 1 -- 172.21.15.86:0/1882645720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac400ce10 con 0x7f4acc108ac0 2024-04-16T05:32:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4abf7fe700 1 -- 172.21.15.86:0/1882645720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4007370 con 0x7f4acc108ac0 2024-04-16T05:32:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.889+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc1a5f80 con 0x7f4acc108ac0 2024-04-16T05:32:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.890+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4acc10b050 con 0x7f4acc108ac0 2024-04-16T05:32:07.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.891+0000 7f4abf7fe700 1 -- 172.21.15.86:0/1882645720 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4ac401c070 con 0x7f4acc108ac0 2024-04-16T05:32:07.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.892+0000 7f4abf7fe700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ab80a3c40 0x7f4ab80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:07.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.892+0000 7f4ad24fe700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ab80a3c40 0x7f4ab80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:07.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.892+0000 7f4abf7fe700 1 -- 172.21.15.86:0/1882645720 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4ac40cf760 con 0x7f4acc108ac0 2024-04-16T05:32:07.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.892+0000 7f4ad24fe700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ab80a3c40 0x7f4ab80a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800b5c0 tx=0x7f4ac8005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:07.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:07.895+0000 7f4abf7fe700 1 -- 172.21.15.86:0/1882645720 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ac409c5e0 con 0x7f4acc108ac0 2024-04-16T05:32:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.095+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4acc19a760 con 0x7f4acc108ac0 2024-04-16T05:32:08.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.096+0000 7f4abf7fe700 1 -- 172.21.15.86:0/1882645720 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18408 (secure 0 0 0) 0x7f4ac409bee0 con 0x7f4acc108ac0 2024-04-16T05:32:08.095 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:08.095 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:32:06.931554+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ab80a3c40 msgr2=0x7f4ab80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ab80a3c40 0x7f4ab80a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800b5c0 tx=0x7f4ac8005960 comp rx=0 tx=0).stop 2024-04-16T05:32:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 msgr2=0x7f4acc19dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 0x7f4acc19dcd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400b970 tx=0x7f4ac400bc80 comp rx=0 tx=0).stop 2024-04-16T05:32:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 shutdown_connections 2024-04-16T05:32:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc108ac0 0x7f4acc19dcd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ab80a3c40 0x7f4ab80a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc102bb0 0x7f4acc199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 --2- 172.21.15.86:0/1882645720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4acc103570 0x7f4acc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 >> 172.21.15.86:0/1882645720 conn(0x7f4acc079cb0 msgr2=0x7f4acc100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 shutdown_connections 2024-04-16T05:32:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:08.098+0000 7f4ad4762700 1 -- 172.21.15.86:0/1882645720 wait complete. 2024-04-16T05:32:08.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:08 smithi194 ceph-mon[35865]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:08.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:08 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1882645720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:08.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:08 smithi086 ceph-mon[32028]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:08.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:08 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1882645720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:08.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:08 smithi196 ceph-mon[36373]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:08.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:08 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1882645720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:08.829 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T05:32:08.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-16T05:32:09.074 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:32:10.693 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:10 smithi086 ceph-mon[32028]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:10.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:10 smithi194 ceph-mon[35865]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.924+0000 7fae5089e700 1 -- 172.21.15.86:0/2452136428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 msgr2=0x7fae4c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.924+0000 7fae5089e700 1 --2- 172.21.15.86:0/2452136428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 0x7fae4c102fc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fae40009990 tx=0x7fae40009ca0 comp rx=0 tx=0).stop 2024-04-16T05:32:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.924+0000 7fae5089e700 1 -- 172.21.15.86:0/2452136428 shutdown_connections 2024-04-16T05:32:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.924+0000 7fae5089e700 1 --2- 172.21.15.86:0/2452136428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae4c103500 0x7fae4c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.924+0000 7fae5089e700 1 --2- 172.21.15.86:0/2452136428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae4c108b90 0x7fae4c108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.924+0000 7fae5089e700 1 --2- 172.21.15.86:0/2452136428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 0x7fae4c102fc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.924+0000 7fae5089e700 1 -- 172.21.15.86:0/2452136428 >> 172.21.15.86:0/2452136428 conn(0x7fae4c079c50 msgr2=0x7fae4c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.925+0000 7fae5089e700 1 -- 172.21.15.86:0/2452136428 shutdown_connections 2024-04-16T05:32:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.925+0000 7fae5089e700 1 -- 172.21.15.86:0/2452136428 wait complete. 2024-04-16T05:32:10.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.925+0000 7fae5089e700 1 Processor -- start 2024-04-16T05:32:10.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.925+0000 7fae5089e700 1 -- start start 2024-04-16T05:32:10.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.925+0000 7fae5089e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 0x7fae4c074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:10.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.925+0000 7fae5089e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae4c103500 0x7fae4c075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:10.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae5089e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae4c108b90 0x7fae4c075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:10.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae5089e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c112e80 con 0x7fae4c102b40 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae5089e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c112d00 con 0x7fae4c108b90 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae5089e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c113000 con 0x7fae4c103500 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4a59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 0x7fae4c074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4a59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 0x7fae4c074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53770/0 (socket says 172.21.15.86:53770) 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4a59c700 1 -- 172.21.15.86:0/357863953 learned_addr learned my addr 172.21.15.86:0/357863953 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae49d9b700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae4c103500 0x7fae4c075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4ad9d700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae4c108b90 0x7fae4c075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4a59c700 1 -- 172.21.15.86:0/357863953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae4c103500 msgr2=0x7fae4c075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4a59c700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae4c103500 0x7fae4c075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4a59c700 1 -- 172.21.15.86:0/357863953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae4c108b90 msgr2=0x7fae4c075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4a59c700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae4c108b90 0x7fae4c075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4a59c700 1 -- 172.21.15.86:0/357863953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c1abcc0 con 0x7fae4c102b40 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae4a59c700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 0x7fae4c074df0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00c970 tx=0x7fae3c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae3b7fe700 1 -- 172.21.15.86:0/357863953 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c007aa0 con 0x7fae4c102b40 2024-04-16T05:32:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae3b7fe700 1 -- 172.21.15.86:0/357863953 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3c004d10 con 0x7fae4c102b40 2024-04-16T05:32:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae3b7fe700 1 -- 172.21.15.86:0/357863953 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c005690 con 0x7fae4c102b40 2024-04-16T05:32:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c1abf50 con 0x7fae4c102b40 2024-04-16T05:32:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.926+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c1ac350 con 0x7fae4c102b40 2024-04-16T05:32:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.927+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4c1041f0 con 0x7fae4c102b40 2024-04-16T05:32:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.928+0000 7fae3b7fe700 1 -- 172.21.15.86:0/357863953 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fae3c004750 con 0x7fae4c102b40 2024-04-16T05:32:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.929+0000 7fae3b7fe700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae340a3c40 0x7fae340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.929+0000 7fae49d9b700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae340a3c40 0x7fae340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.929+0000 7fae3b7fe700 1 -- 172.21.15.86:0/357863953 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fae3c0cfd30 con 0x7fae4c102b40 2024-04-16T05:32:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.929+0000 7fae49d9b700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae340a3c40 0x7fae340a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0765b0 tx=0x7fae4000b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:10.934+0000 7fae3b7fe700 1 -- 172.21.15.86:0/357863953 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae3c09cc40 con 0x7fae4c102b40 2024-04-16T05:32:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:10 smithi196 ceph-mon[36373]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:11.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.130+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae4c02dfe0 con 0x7fae4c102b40 2024-04-16T05:32:11.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.131+0000 7fae3b7fe700 1 -- 172.21.15.86:0/357863953 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fae3c09c540 con 0x7fae4c102b40 2024-04-16T05:32:11.129 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae340a3c40 msgr2=0x7fae340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae340a3c40 0x7fae340a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0765b0 tx=0x7fae4000b540 comp rx=0 tx=0).stop 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 msgr2=0x7fae4c074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 0x7fae4c074df0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00c970 tx=0x7fae3c00cd30 comp rx=0 tx=0).stop 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 shutdown_connections 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae4c108b90 0x7fae4c075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c102b40 0x7fae4c074df0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fae340a3c40 0x7fae340a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 --2- 172.21.15.86:0/357863953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae4c103500 0x7fae4c075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 >> 172.21.15.86:0/357863953 conn(0x7fae4c079c50 msgr2=0x7fae4c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 shutdown_connections 2024-04-16T05:32:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:11.133+0000 7fae5089e700 1 -- 172.21.15.86:0/357863953 wait complete. 2024-04-16T05:32:11.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:11 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/357863953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:11.822 INFO:tasks.cephadm:Adding mds.a on smithi086 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.d on smithi086 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.g on smithi086 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.j on smithi086 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.b on smithi194 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.e on smithi194 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.h on smithi194 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.k on smithi194 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-04-16T05:32:11.823 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-04-16T05:32:11.823 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch apply mds all '12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi194=b;smithi194=e;smithi194=h;smithi194=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-04-16T05:32:11.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:11 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/357863953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:11.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:11 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/357863953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:12.072 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:32:12.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:12 smithi194 ceph-mon[35865]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:12.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:12 smithi086 ceph-mon[32028]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:12 smithi196 ceph-mon[36373]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:13.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 -- 172.21.15.196:0/3258974086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec103710 msgr2=0x7f8dec109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 --2- 172.21.15.196:0/3258974086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec103710 0x7f8dec109fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8de400b0a0 tx=0x7f8de400b3b0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 -- 172.21.15.196:0/3258974086 shutdown_connections 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 --2- 172.21.15.196:0/3258974086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec103710 0x7f8dec109fa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 --2- 172.21.15.196:0/3258974086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dec102030 0x7f8dec102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 --2- 172.21.15.196:0/3258974086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec102d50 0x7f8dec1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 -- 172.21.15.196:0/3258974086 >> 172.21.15.196:0/3258974086 conn(0x7f8dec0fc830 msgr2=0x7f8dec0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 -- 172.21.15.196:0/3258974086 shutdown_connections 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 -- 172.21.15.196:0/3258974086 wait complete. 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 Processor -- start 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.765+0000 7f8df1e82700 1 -- start start 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8df1e82700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec102030 0x7f8dec198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:13.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8df1e82700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dec102d50 0x7f8dec1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8df1e82700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec103710 0x7f8dec1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8df1e82700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec10d630 con 0x7f8dec103710 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8df1e82700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec10d4b0 con 0x7f8dec102d50 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8df1e82700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec10d7b0 con 0x7f8dec102030 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8deb7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec102030 0x7f8dec198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8deb7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec102030 0x7f8dec198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53784/0 (socket says 172.21.15.196:53784) 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8deb7fe700 1 -- 172.21.15.196:0/2121835552 learned_addr learned my addr 172.21.15.196:0/2121835552 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8de2ffd700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dec102d50 0x7f8dec1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8debfff700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec103710 0x7f8dec1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8deb7fe700 1 -- 172.21.15.196:0/2121835552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dec102d50 msgr2=0x7f8dec1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8deb7fe700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dec102d50 0x7f8dec1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8deb7fe700 1 -- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec103710 msgr2=0x7f8dec1a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8deb7fe700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec103710 0x7f8dec1a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8deb7fe700 1 -- 172.21.15.196:0/2121835552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec1a0920 con 0x7f8dec102030 2024-04-16T05:32:13.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.766+0000 7f8deb7fe700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec102030 0x7f8dec198ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00c970 tx=0x7f8ddc00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:13.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.767+0000 7f8de97fa700 1 -- 172.21.15.196:0/2121835552 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc004220 con 0x7f8dec102030 2024-04-16T05:32:13.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.767+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec1a0c10 con 0x7f8dec102030 2024-04-16T05:32:13.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.767+0000 7f8de97fa700 1 -- 172.21.15.196:0/2121835552 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ddc0043c0 con 0x7f8dec102030 2024-04-16T05:32:13.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.767+0000 7f8de97fa700 1 -- 172.21.15.196:0/2121835552 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc003c40 con 0x7f8dec102030 2024-04-16T05:32:13.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.767+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec1a1150 con 0x7f8dec102030 2024-04-16T05:32:13.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.768+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dec0697e0 con 0x7f8dec102030 2024-04-16T05:32:13.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.769+0000 7f8de97fa700 1 -- 172.21.15.196:0/2121835552 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8ddc0076d0 con 0x7f8dec102030 2024-04-16T05:32:13.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.769+0000 7f8de97fa700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8dd80a3c40 0x7f8dd80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:13.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.770+0000 7f8de97fa700 1 -- 172.21.15.196:0/2121835552 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8ddc0cf6c0 con 0x7f8dec102030 2024-04-16T05:32:13.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.770+0000 7f8de2ffd700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8dd80a3c40 0x7f8dd80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:13.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.770+0000 7f8de2ffd700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8dd80a3c40 0x7f8dd80a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8dec19a1b0 tx=0x7f8dd400a3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.774+0000 7f8de97fa700 1 -- 172.21.15.196:0/2121835552 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ddc014030 con 0x7f8dec102030 2024-04-16T05:32:13.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.980+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi194=b;smithi194=e;smithi194=h;smithi194=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7f8dec0fe860 con 0x7f8dd80a3c40 2024-04-16T05:32:13.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.988+0000 7f8de97fa700 1 -- 172.21.15.196:0/2121835552 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8dec0fe860 con 0x7f8dd80a3c40 2024-04-16T05:32:13.988 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-04-16T05:32:13.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8dd80a3c40 msgr2=0x7f8dd80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:13.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8dd80a3c40 0x7f8dd80a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8dec19a1b0 tx=0x7f8dd400a3b0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec102030 msgr2=0x7f8dec198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:13.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec102030 0x7f8dec198ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00c970 tx=0x7f8ddc00cc80 comp rx=0 tx=0).stop 2024-04-16T05:32:13.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 shutdown_connections 2024-04-16T05:32:13.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec103710 0x7f8dec1a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8dd80a3c40 0x7f8dd80a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dec102030 0x7f8dec198ca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 --2- 172.21.15.196:0/2121835552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dec102d50 0x7f8dec1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:13.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 >> 172.21.15.196:0/2121835552 conn(0x7f8dec0fc830 msgr2=0x7f8dec0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:13.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 shutdown_connections 2024-04-16T05:32:13.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-16T05:32:13.990+0000 7f8df1e82700 1 -- 172.21.15.196:0/2121835552 wait complete. 2024-04-16T05:32:14.672 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:14 smithi196 ceph-mon[36373]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:14.672 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:14 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:14.672 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:14 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:14.701 DEBUG:teuthology.orchestra.run.smithi086:mds.a> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.a.service 2024-04-16T05:32:14.705 DEBUG:teuthology.orchestra.run.smithi086:mds.d> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.d.service 2024-04-16T05:32:14.747 DEBUG:teuthology.orchestra.run.smithi086:mds.g> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.g.service 2024-04-16T05:32:14.790 DEBUG:teuthology.orchestra.run.smithi086:mds.j> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.j.service 2024-04-16T05:32:14.814 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:14 smithi086 ceph-mon[32028]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:14.814 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:14.814 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:14 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:14.815 DEBUG:teuthology.orchestra.run.smithi194:mds.b> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.b.service 2024-04-16T05:32:14.819 DEBUG:teuthology.orchestra.run.smithi194:mds.e> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.e.service 2024-04-16T05:32:14.842 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:14 smithi194 ceph-mon[35865]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:14.842 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:14 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:14.842 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:14 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:14.843 DEBUG:teuthology.orchestra.run.smithi194:mds.h> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.h.service 2024-04-16T05:32:14.885 DEBUG:teuthology.orchestra.run.smithi194:mds.k> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.k.service 2024-04-16T05:32:14.928 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.c.service 2024-04-16T05:32:14.930 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.f.service 2024-04-16T05:32:14.973 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.i.service 2024-04-16T05:32:15.016 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.l.service 2024-04-16T05:32:15.059 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-16T05:32:15.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1111786238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4102a50 msgr2=0x7fdaa4109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1111786238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4102a50 0x7fdaa4109370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009ca0 tx=0x7fda9c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1111786238 shutdown_connections 2024-04-16T05:32:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1111786238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4102a50 0x7fdaa4109370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1111786238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa4102090 0x7fdaa4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1111786238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa4100f40 0x7fdaa4101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1111786238 >> 172.21.15.86:0/1111786238 conn(0x7fdaa40fc4b0 msgr2=0x7fdaa40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1111786238 shutdown_connections 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1111786238 wait complete. 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 Processor -- start 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.191+0000 7fdaab1ba640 1 -- start start 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaab1ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4100f40 0x7fdaa4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaab1ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa4102090 0x7fdaa4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaab1ba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa4102a50 0x7fdaa419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaab1ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa410dd30 con 0x7fdaa4100f40 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaab1ba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa410dbb0 con 0x7fdaa4102a50 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaab1ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa410deb0 con 0x7fdaa4102090 2024-04-16T05:32:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa8f56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4100f40 0x7fdaa4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa8f56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4100f40 0x7fdaa4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53788/0 (socket says 172.21.15.86:53788) 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa8f56640 1 -- 172.21.15.86:0/1713621705 learned_addr learned my addr 172.21.15.86:0/1713621705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa4102090 0x7fdaa4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa9757640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa4102a50 0x7fdaa419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa8f56640 1 -- 172.21.15.86:0/1713621705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa4102090 msgr2=0x7fdaa4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa8f56640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa4102090 0x7fdaa4198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa8f56640 1 -- 172.21.15.86:0/1713621705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa4102a50 msgr2=0x7fdaa419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa8f56640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa4102a50 0x7fdaa419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa8f56640 1 -- 172.21.15.86:0/1713621705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa419ff20 con 0x7fdaa4100f40 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa8f56640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4100f40 0x7fdaa4198300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fda9800e9f0 tx=0x7fda9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa1ffb640 1 -- 172.21.15.86:0/1713621705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9800cd80 con 0x7fdaa4100f40 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa1ffb640 1 -- 172.21.15.86:0/1713621705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda98005500 con 0x7fdaa4100f40 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaa1ffb640 1 -- 172.21.15.86:0/1713621705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98010690 con 0x7fdaa4100f40 2024-04-16T05:32:15.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.192+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa41a0210 con 0x7fdaa4100f40 2024-04-16T05:32:15.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.193+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa41a0710 con 0x7fdaa4100f40 2024-04-16T05:32:15.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.193+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa41081f0 con 0x7fdaa4100f40 2024-04-16T05:32:15.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.194+0000 7fdaa1ffb640 1 -- 172.21.15.86:0/1713621705 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fda980046c0 con 0x7fdaa4100f40 2024-04-16T05:32:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.195+0000 7fdaa1ffb640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fda940a3bb0 0x7fda940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.195+0000 7fdaa3fff640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fda940a3bb0 0x7fda940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.195+0000 7fdaa1ffb640 1 -- 172.21.15.86:0/1713621705 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fda980d0360 con 0x7fdaa4100f40 2024-04-16T05:32:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.196+0000 7fdaa3fff640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fda940a3bb0 0x7fda940a6070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4199810 tx=0x7fda9000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:15.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.200+0000 7fdaa1ffb640 1 -- 172.21.15.86:0/1713621705 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda9809d1e0 con 0x7fdaa4100f40 2024-04-16T05:32:15.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.379+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdaa4062c00 con 0x7fdaa4100f40 2024-04-16T05:32:15.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.380+0000 7fdaa1ffb640 1 -- 172.21.15.86:0/1713621705 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18408 (secure 0 0 0) 0x7fda9809cae0 con 0x7fdaa4100f40 2024-04-16T05:32:15.379 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:15.379 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:32:06.931554+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:15.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fda940a3bb0 msgr2=0x7fda940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fda940a3bb0 0x7fda940a6070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4199810 tx=0x7fda9000a2d0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4100f40 msgr2=0x7fdaa4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4100f40 0x7fdaa4198300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fda9800e9f0 tx=0x7fda9800edf0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 shutdown_connections 2024-04-16T05:32:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fda940a3bb0 0x7fda940a6070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa4102a50 0x7fdaa419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa4102090 0x7fdaa4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 --2- 172.21.15.86:0/1713621705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa4100f40 0x7fdaa4198300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 >> 172.21.15.86:0/1713621705 conn(0x7fdaa40fc4b0 msgr2=0x7fdaa41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 shutdown_connections 2024-04-16T05:32:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.382+0000 7fdaab1ba640 1 -- 172.21.15.86:0/1713621705 wait complete. 2024-04-16T05:32:15.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T05:32:15.393 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:32:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.502+0000 7f9469ed9640 1 -- 172.21.15.86:0/2974422753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 msgr2=0x7f9464109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.502+0000 7f9469ed9640 1 --2- 172.21.15.86:0/2974422753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 0x7f9464109370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f945c009ca0 tx=0x7f945c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.502+0000 7f9469ed9640 1 -- 172.21.15.86:0/2974422753 shutdown_connections 2024-04-16T05:32:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.502+0000 7f9469ed9640 1 --2- 172.21.15.86:0/2974422753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 0x7f9464109370 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.502+0000 7f9469ed9640 1 --2- 172.21.15.86:0/2974422753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9464102090 0x7f9464102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.502+0000 7f9469ed9640 1 --2- 172.21.15.86:0/2974422753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464100f40 0x7f9464101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.502+0000 7f9469ed9640 1 -- 172.21.15.86:0/2974422753 >> 172.21.15.86:0/2974422753 conn(0x7f94640fc4f0 msgr2=0x7f94640fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.502+0000 7f9469ed9640 1 -- 172.21.15.86:0/2974422753 shutdown_connections 2024-04-16T05:32:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.502+0000 7f9469ed9640 1 -- 172.21.15.86:0/2974422753 wait complete. 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9469ed9640 1 Processor -- start 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9469ed9640 1 -- start start 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9469ed9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9464100f40 0x7f9464198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9469ed9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464102090 0x7f9464198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9469ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 0x7f946419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9469ed9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f946410cc10 con 0x7f9464102a50 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9469ed9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f946410ca90 con 0x7f9464102090 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9469ed9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f946410cd90 con 0x7f9464100f40 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9463fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 0x7f946419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9463fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 0x7f946419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53804/0 (socket says 172.21.15.86:53804) 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9463fff640 1 -- 172.21.15.86:0/1468957404 learned_addr learned my addr 172.21.15.86:0/1468957404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f94637fe640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9464100f40 0x7f9464198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f945bfff640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464102090 0x7f9464198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9463fff640 1 -- 172.21.15.86:0/1468957404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9464100f40 msgr2=0x7f9464198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9463fff640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9464100f40 0x7f9464198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9463fff640 1 -- 172.21.15.86:0/1468957404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464102090 msgr2=0x7f9464198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9463fff640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464102090 0x7f9464198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9463fff640 1 -- 172.21.15.86:0/1468957404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946419ff70 con 0x7f9464102a50 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.503+0000 7f9463fff640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 0x7f946419f900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f945c02c700 tx=0x7f945c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.504+0000 7f94617fa640 1 -- 172.21.15.86:0/1468957404 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c039040 con 0x7f9464102a50 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.504+0000 7f94617fa640 1 -- 172.21.15.86:0/1468957404 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f945c044070 con 0x7f9464102a50 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.504+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94641a0200 con 0x7f9464102a50 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.504+0000 7f94617fa640 1 -- 172.21.15.86:0/1468957404 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0042a0 con 0x7f9464102a50 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.504+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94641a06f0 con 0x7f9464102a50 2024-04-16T05:32:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.505+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94641a09b0 con 0x7f9464102a50 2024-04-16T05:32:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.506+0000 7f94617fa640 1 -- 172.21.15.86:0/1468957404 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f945c003970 con 0x7f9464102a50 2024-04-16T05:32:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.507+0000 7f94617fa640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f94500a3bb0 0x7f94500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.507+0000 7f94637fe640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f94500a3bb0 0x7f94500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.507+0000 7f94617fa640 1 -- 172.21.15.86:0/1468957404 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f945c0f2a10 con 0x7f9464102a50 2024-04-16T05:32:15.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.508+0000 7f94637fe640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f94500a3bb0 0x7f94500a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f945400ab40 tx=0x7f945400a250 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:15.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.511+0000 7f94617fa640 1 -- 172.21.15.86:0/1468957404 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f945c0bf920 con 0x7f9464102a50 2024-04-16T05:32:15.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.690+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f94641a0ca0 con 0x7f9464102a50 2024-04-16T05:32:15.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.690+0000 7f94617fa640 1 -- 172.21.15.86:0/1468957404 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f945c0bf220 con 0x7f9464102a50 2024-04-16T05:32:15.689 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:32:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f94500a3bb0 msgr2=0x7f94500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f94500a3bb0 0x7f94500a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f945400ab40 tx=0x7f945400a250 comp rx=0 tx=0).stop 2024-04-16T05:32:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 msgr2=0x7f946419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 0x7f946419f900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f945c02c700 tx=0x7f945c032f20 comp rx=0 tx=0).stop 2024-04-16T05:32:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 shutdown_connections 2024-04-16T05:32:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464102a50 0x7f946419f900 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f94500a3bb0 0x7f94500a6070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464102090 0x7f9464198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 --2- 172.21.15.86:0/1468957404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9464100f40 0x7f9464198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 >> 172.21.15.86:0/1468957404 conn(0x7f94640fc4f0 msgr2=0x7f94640fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 shutdown_connections 2024-04-16T05:32:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.692+0000 7f9469ed9640 1 -- 172.21.15.86:0/1468957404 wait complete. 2024-04-16T05:32:15.701 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:15 smithi086 ceph-mon[32028]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi194=b;smithi194=e;smithi194=h;smithi194=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:15.701 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:15 smithi086 ceph-mon[32028]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi194=b;smithi194=e;smithi194=h;smithi194=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-16T05:32:15.702 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:15.702 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:15.702 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:15.702 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:15.702 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:15.702 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:15 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:15.702 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:15 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1713621705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:15.703 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:15.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:15 smithi196 ceph-mon[36373]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi194=b;smithi194=e;smithi194=h;smithi194=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:15.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:15 smithi196 ceph-mon[36373]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi194=b;smithi194=e;smithi194=h;smithi194=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-16T05:32:15.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:15.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:15.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:15.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:15.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:15.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:15 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:15.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:15 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1713621705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 -- 172.21.15.86:0/2837751936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 msgr2=0x7f2ac410a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/2837751936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 0x7f2ac410a170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009ca0 tx=0x7f2abc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 -- 172.21.15.86:0/2837751936 shutdown_connections 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/2837751936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 0x7f2ac410a170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/2837751936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ac41053f0 0x7f2ac41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/2837751936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac4102ac0 0x7f2ac4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 -- 172.21.15.86:0/2837751936 >> 172.21.15.86:0/2837751936 conn(0x7f2ac40fc4b0 msgr2=0x7f2ac40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 -- 172.21.15.86:0/2837751936 shutdown_connections 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 -- 172.21.15.86:0/2837751936 wait complete. 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 Processor -- start 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 -- start start 2024-04-16T05:32:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ac4102ac0 0x7f2ac4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac41053f0 0x7f2ac41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 0x7f2ac419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac410dd50 con 0x7f2ac4107d20 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac410dbd0 con 0x7f2ac4102ac0 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2aca9bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac410ded0 con 0x7f2ac41053f0 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.810+0000 7f2ac8f5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 0x7f2ac419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac8f5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 0x7f2ac419f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53828/0 (socket says 172.21.15.86:53828) 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac8f5a640 1 -- 172.21.15.86:0/1448228987 learned_addr learned my addr 172.21.15.86:0/1448228987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac3fff640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ac4102ac0 0x7f2ac4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2abbfff640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac41053f0 0x7f2ac41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac8f5a640 1 -- 172.21.15.86:0/1448228987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac41053f0 msgr2=0x7f2ac41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac8f5a640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac41053f0 0x7f2ac41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac8f5a640 1 -- 172.21.15.86:0/1448228987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ac4102ac0 msgr2=0x7f2ac4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac8f5a640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ac4102ac0 0x7f2ac4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac8f5a640 1 -- 172.21.15.86:0/1448228987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac419ff90 con 0x7f2ac4107d20 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2abbfff640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac41053f0 0x7f2ac41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac3fff640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ac4102ac0 0x7f2ac4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac8f5a640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 0x7f2ac419f920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2abc037040 tx=0x7f2abc032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac1ffb640 1 -- 172.21.15.86:0/1448228987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc039040 con 0x7f2ac4107d20 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac41a0220 con 0x7f2ac4107d20 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac1ffb640 1 -- 172.21.15.86:0/1448228987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2abc044070 con 0x7f2ac4107d20 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2ac1ffb640 1 -- 172.21.15.86:0/1448228987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc004330 con 0x7f2ac4107d20 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.811+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac41a0710 con 0x7f2ac4107d20 2024-04-16T05:32:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.812+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac40fdfa0 con 0x7f2ac4107d20 2024-04-16T05:32:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.813+0000 7f2ac1ffb640 1 -- 172.21.15.86:0/1448228987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2abc003a00 con 0x7f2ac4107d20 2024-04-16T05:32:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.814+0000 7f2ac1ffb640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2ab00a3bb0 0x7f2ab00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.814+0000 7f2ac3fff640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2ab00a3bb0 0x7f2ab00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.814+0000 7f2ac1ffb640 1 -- 172.21.15.86:0/1448228987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2abc0f2840 con 0x7f2ac4107d20 2024-04-16T05:32:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.815+0000 7f2ac3fff640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2ab00a3bb0 0x7f2ab00a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4005fd0 tx=0x7f2ab4005ea0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:15.819+0000 7f2ac1ffb640 1 -- 172.21.15.86:0/1448228987 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2abc0bf6c0 con 0x7f2ac4107d20 2024-04-16T05:32:15.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:15 smithi194 ceph-mon[35865]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi194=b;smithi194=e;smithi194=h;smithi194=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:15 smithi194 ceph-mon[35865]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi194=b;smithi194=e;smithi194=h;smithi194=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-16T05:32:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:15 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:15 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1713621705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.033+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ac4062c00 con 0x7f2ac4107d20 2024-04-16T05:32:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.033+0000 7f2ac1ffb640 1 -- 172.21.15.86:0/1448228987 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f2abc0befc0 con 0x7f2ac4107d20 2024-04-16T05:32:16.032 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:16.032 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-16T05:32:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 1 2024-04-16T05:32:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2ab00a3bb0 msgr2=0x7f2ab00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2ab00a3bb0 0x7f2ab00a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4005fd0 tx=0x7f2ab4005ea0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 msgr2=0x7f2ac419f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 0x7f2ac419f920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2abc037040 tx=0x7f2abc032f90 comp rx=0 tx=0).stop 2024-04-16T05:32:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 shutdown_connections 2024-04-16T05:32:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac4107d20 0x7f2ac419f920 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac41053f0 0x7f2ac41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2ab00a3bb0 0x7f2ab00a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 --2- 172.21.15.86:0/1448228987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ac4102ac0 0x7f2ac4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 >> 172.21.15.86:0/1448228987 conn(0x7f2ac40fc4b0 msgr2=0x7f2ac4103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 shutdown_connections 2024-04-16T05:32:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:16.035+0000 7f2aca9bd640 1 -- 172.21.15.86:0/1448228987 wait complete. 2024-04-16T05:32:16.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:16 smithi196 ceph-mon[36373]: Deploying daemon mds.l on smithi196 2024-04-16T05:32:16.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:16 smithi196 ceph-mon[36373]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:16.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:16 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1468957404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:16.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:16 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1448228987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:16 smithi194 ceph-mon[35865]: Deploying daemon mds.l on smithi196 2024-04-16T05:32:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:16 smithi194 ceph-mon[35865]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:16 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1468957404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:16 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1448228987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:16.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:16 smithi086 ceph-mon[32028]: Deploying daemon mds.l on smithi196 2024-04-16T05:32:16.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:16 smithi086 ceph-mon[32028]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:16.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:16 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1468957404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:16.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:16 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1448228987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:17.176 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:32:16 smithi196 systemd[1]: Starting Ceph mds.l for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:17.444 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:32:17 smithi196 podman[66098]: 2024-04-16 05:32:17.080957255 +0000 UTC m=+0.017628248 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:17.444 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:32:17 smithi196 podman[66098]: 2024-04-16 05:32:17.192983296 +0000 UTC m=+0.129654306 container create 0bb4d70276ee4c71a19361b33c24a28f271378b0bbe341577c1727f70b3393db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:32:17.444 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:32:17 smithi196 podman[66098]: 2024-04-16 05:32:17.446195809 +0000 UTC m=+0.382866796 container init 0bb4d70276ee4c71a19361b33c24a28f271378b0bbe341577c1727f70b3393db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:32:17.735 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:32:17 smithi196 podman[66098]: 2024-04-16 05:32:17.448755947 +0000 UTC m=+0.385426941 container start 0bb4d70276ee4c71a19361b33c24a28f271378b0bbe341577c1727f70b3393db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-16T05:32:17.735 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:32:17 smithi196 ceph-mds[66117]: -- 172.21.15.196:0/1500757506 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce5562fa40 con 0x55ce55602c00 2024-04-16T05:32:17.735 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:32:17 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l[66113]: starting mds.l at 2024-04-16T05:32:17.735 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:32:17 smithi196 bash[66098]: 0bb4d70276ee4c71a19361b33c24a28f271378b0bbe341577c1727f70b3393db 2024-04-16T05:32:17.735 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:32:17 smithi196 systemd[1]: Started Ceph mds.l for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:18.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:18.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.164+0000 7ff603d59640 1 -- 172.21.15.86:0/1744474195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 msgr2=0x7ff5fc106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:18.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.164+0000 7ff603d59640 1 --2- 172.21.15.86:0/1744474195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 0x7ff5fc106060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009ca0 tx=0x7ff5f402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 -- 172.21.15.86:0/1744474195 shutdown_connections 2024-04-16T05:32:18.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 --2- 172.21.15.86:0/1744474195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 0x7ff5fc106060 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 --2- 172.21.15.86:0/1744474195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5fc103230 0x7ff5fc1056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 --2- 172.21.15.86:0/1744474195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5fc1008c0 0x7ff5fc102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 -- 172.21.15.86:0/1744474195 >> 172.21.15.86:0/1744474195 conn(0x7ff5fc0fc500 msgr2=0x7ff5fc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 -- 172.21.15.86:0/1744474195 shutdown_connections 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 -- 172.21.15.86:0/1744474195 wait complete. 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 Processor -- start 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 -- start start 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5fc1008c0 0x7ff5fc19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5fc103230 0x7ff5fc19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 0x7ff5fc1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1142d0 con 0x7ff5fc105be0 2024-04-16T05:32:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc114150 con 0x7ff5fc103230 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.165+0000 7ff603d59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc114450 con 0x7ff5fc1008c0 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6022f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 0x7ff5fc1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6022f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 0x7ff5fc1a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53846/0 (socket says 172.21.15.86:53846) 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6022f6640 1 -- 172.21.15.86:0/1321688628 learned_addr learned my addr 172.21.15.86:0/1321688628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6012f4640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5fc103230 0x7ff5fc19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff601af5640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5fc1008c0 0x7ff5fc19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6022f6640 1 -- 172.21.15.86:0/1321688628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5fc1008c0 msgr2=0x7ff5fc19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6022f6640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5fc1008c0 0x7ff5fc19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6022f6640 1 -- 172.21.15.86:0/1321688628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5fc103230 msgr2=0x7ff5fc19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6022f6640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5fc103230 0x7ff5fc19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6022f6640 1 -- 172.21.15.86:0/1321688628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc1a6530 con 0x7ff5fc105be0 2024-04-16T05:32:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff6022f6640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 0x7ff5fc1a5ec0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff5f402c630 tx=0x7ff5f4032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/1321688628 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4041020 con 0x7ff5fc105be0 2024-04-16T05:32:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/1321688628 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f4034cd0 con 0x7ff5fc105be0 2024-04-16T05:32:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc1a67c0 con 0x7ff5fc105be0 2024-04-16T05:32:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/1321688628 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4039040 con 0x7ff5fc105be0 2024-04-16T05:32:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.166+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a6d00 con 0x7ff5fc105be0 2024-04-16T05:32:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.167+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5fc0fdff0 con 0x7ff5fc105be0 2024-04-16T05:32:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.168+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/1321688628 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff5f4044020 con 0x7ff5fc105be0 2024-04-16T05:32:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.169+0000 7ff5f2ffd640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5e80a3c30 0x7ff5e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.169+0000 7ff601af5640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5e80a3c30 0x7ff5e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.169+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/1321688628 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff5f40f1e60 con 0x7ff5fc105be0 2024-04-16T05:32:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.170+0000 7ff601af5640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5e80a3c30 0x7ff5e80a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009b60 tx=0x7ff5f8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:18.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.174+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/1321688628 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5f404b020 con 0x7ff5fc105be0 2024-04-16T05:32:18.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.388+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5fc062c00 con 0x7ff5fc105be0 2024-04-16T05:32:18.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.389+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/1321688628 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7ff5f40bece0 con 0x7ff5fc105be0 2024-04-16T05:32:18.388 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:18.388 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T05:32:18.388 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-04-16T05:32:18.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5e80a3c30 msgr2=0x7ff5e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:18.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5e80a3c30 0x7ff5e80a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009b60 tx=0x7ff5f8005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:18.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 msgr2=0x7ff5fc1a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:18.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 0x7ff5fc1a5ec0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff5f402c630 tx=0x7ff5f4032ea0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 shutdown_connections 2024-04-16T05:32:18.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc105be0 0x7ff5fc1a5ec0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5e80a3c30 0x7ff5e80a60f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5fc103230 0x7ff5fc19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 --2- 172.21.15.86:0/1321688628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5fc1008c0 0x7ff5fc19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 >> 172.21.15.86:0/1321688628 conn(0x7ff5fc0fc500 msgr2=0x7ff5fc101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:18.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 shutdown_connections 2024-04-16T05:32:18.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:18.391+0000 7ff603d59640 1 -- 172.21.15.86:0/1321688628 wait complete. 2024-04-16T05:32:18.507 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:18.507 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6832/1220860047,v1:172.21.15.196:6833/1220860047] up:boot 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: fsmap 1 up:standby 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: Deploying daemon mds.k on smithi194 2024-04-16T05:32:18.508 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:18 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1321688628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6832/1220860047,v1:172.21.15.196:6833/1220860047] up:boot 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: fsmap 1 up:standby 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:18.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:18.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: Deploying daemon mds.k on smithi194 2024-04-16T05:32:18.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:18 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1321688628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6832/1220860047,v1:172.21.15.196:6833/1220860047] up:boot 2024-04-16T05:32:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: fsmap 1 up:standby 2024-04-16T05:32:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:32:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:18.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:18.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:18.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:18.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:18.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: Deploying daemon mds.k on smithi194 2024-04-16T05:32:18.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:18 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1321688628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:20.040 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:32:19 smithi194 systemd[1]: Starting Ceph mds.k for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:20.309 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:32:20 smithi194 podman[66004]: 2024-04-16 05:32:20.041786565 +0000 UTC m=+0.103973593 container create ba3233fc3bde559ddb32356206cd2e6fb8aef82c0bce83a971a7b5a3771200ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-16T05:32:20.309 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:32:20 smithi194 podman[66004]: 2024-04-16 05:32:19.955495008 +0000 UTC m=+0.017682048 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:20.310 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:32:20 smithi194 podman[66004]: 2024-04-16 05:32:20.285915573 +0000 UTC m=+0.348102614 container init ba3233fc3bde559ddb32356206cd2e6fb8aef82c0bce83a971a7b5a3771200ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-16T05:32:20.310 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:32:20 smithi194 podman[66004]: 2024-04-16 05:32:20.288480243 +0000 UTC m=+0.350667279 container start ba3233fc3bde559ddb32356206cd2e6fb8aef82c0bce83a971a7b5a3771200ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409) 2024-04-16T05:32:20.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:20.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 -- 172.21.15.86:0/4164391021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 msgr2=0x7fe3c810a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:20.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/4164391021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 0x7fe3c810a1b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009ca0 tx=0x7fe3c002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 -- 172.21.15.86:0/4164391021 shutdown_connections 2024-04-16T05:32:20.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/4164391021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 0x7fe3c810a1b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/4164391021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c8105430 0x7fe3c8107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/4164391021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c8102b00 0x7fe3c8104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 -- 172.21.15.86:0/4164391021 >> 172.21.15.86:0/4164391021 conn(0x7fe3c80fc4f0 msgr2=0x7fe3c80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:20.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 -- 172.21.15.86:0/4164391021 shutdown_connections 2024-04-16T05:32:20.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 -- 172.21.15.86:0/4164391021 wait complete. 2024-04-16T05:32:20.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 Processor -- start 2024-04-16T05:32:20.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.522+0000 7fe3d05d4640 1 -- start start 2024-04-16T05:32:20.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3d05d4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c8102b00 0x7fe3c8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:20.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3d05d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c8105430 0x7fe3c81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:20.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3d05d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 0x7fe3c819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:20.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3d05d4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c810dd10 con 0x7fe3c8107d60 2024-04-16T05:32:20.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3d05d4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c810db90 con 0x7fe3c8102b00 2024-04-16T05:32:20.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3d05d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c810de90 con 0x7fe3c8105430 2024-04-16T05:32:20.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ceb71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 0x7fe3c819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:20.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ceb71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 0x7fe3c819f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53856/0 (socket says 172.21.15.86:53856) 2024-04-16T05:32:20.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ceb71640 1 -- 172.21.15.86:0/1792565915 learned_addr learned my addr 172.21.15.86:0/1792565915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:20.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ce370640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c8102b00 0x7fe3c8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:20.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3cdb6f640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c8105430 0x7fe3c81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:20.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ceb71640 1 -- 172.21.15.86:0/1792565915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c8105430 msgr2=0x7fe3c81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:20.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ceb71640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c8105430 0x7fe3c81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ceb71640 1 -- 172.21.15.86:0/1792565915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c8102b00 msgr2=0x7fe3c8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:20.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ceb71640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c8102b00 0x7fe3c8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ceb71640 1 -- 172.21.15.86:0/1792565915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c819ffa0 con 0x7fe3c8107d60 2024-04-16T05:32:20.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3ceb71640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 0x7fe3c819f930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005cc0 tx=0x7fe3c0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:20.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3bf7fe640 1 -- 172.21.15.86:0/1792565915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0039040 con 0x7fe3c8107d60 2024-04-16T05:32:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c81a0230 con 0x7fe3c8107d60 2024-04-16T05:32:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3bf7fe640 1 -- 172.21.15.86:0/1792565915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c0044070 con 0x7fe3c8107d60 2024-04-16T05:32:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.523+0000 7fe3bf7fe640 1 -- 172.21.15.86:0/1792565915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00042a0 con 0x7fe3c8107d60 2024-04-16T05:32:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.524+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c81a06f0 con 0x7fe3c8107d60 2024-04-16T05:32:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.525+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c80fdfe0 con 0x7fe3c8107d60 2024-04-16T05:32:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.526+0000 7fe3bf7fe640 1 -- 172.21.15.86:0/1792565915 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe3c0003970 con 0x7fe3c8107d60 2024-04-16T05:32:20.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.526+0000 7fe3bf7fe640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3b40a3bb0 0x7fe3b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:20.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.527+0000 7fe3ce370640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3b40a3bb0 0x7fe3b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:20.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.527+0000 7fe3bf7fe640 1 -- 172.21.15.86:0/1792565915 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe3c00f23f0 con 0x7fe3c8107d60 2024-04-16T05:32:20.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.527+0000 7fe3ce370640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3b40a3bb0 0x7fe3b40a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4005fd0 tx=0x7fe3c400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:20.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.531+0000 7fe3bf7fe640 1 -- 172.21.15.86:0/1792565915 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3c00bf270 con 0x7fe3c8107d60 2024-04-16T05:32:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:20 smithi194 ceph-mon[35865]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:20 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:20 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:20.656 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:32:20 smithi194 ceph-mds[66023]: -- 172.21.15.194:0/4205194780 <== mon.2 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598b538da40 con 0x5598b5360c00 2024-04-16T05:32:20.656 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:32:20 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k[66019]: starting mds.k at 2024-04-16T05:32:20.656 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:32:20 smithi194 bash[66004]: ba3233fc3bde559ddb32356206cd2e6fb8aef82c0bce83a971a7b5a3771200ad 2024-04-16T05:32:20.656 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:32:20 smithi194 systemd[1]: Started Ceph mds.k for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:20 smithi086 ceph-mon[32028]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:20 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:20.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.758+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3c8062c00 con 0x7fe3c8107d60 2024-04-16T05:32:20.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.759+0000 7fe3bf7fe640 1 -- 172.21.15.86:0/1792565915 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fe3c00beb70 con 0x7fe3c8107d60 2024-04-16T05:32:20.757 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:20.757 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T05:32:20.758 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 3 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3b40a3bb0 msgr2=0x7fe3b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3b40a3bb0 0x7fe3b40a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4005fd0 tx=0x7fe3c400b390 comp rx=0 tx=0).stop 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 msgr2=0x7fe3c819f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 0x7fe3c819f930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005cc0 tx=0x7fe3c0032f40 comp rx=0 tx=0).stop 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 shutdown_connections 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c8107d60 0x7fe3c819f930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c8105430 0x7fe3c81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3b40a3bb0 0x7fe3b40a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 --2- 172.21.15.86:0/1792565915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c8102b00 0x7fe3c8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 >> 172.21.15.86:0/1792565915 conn(0x7fe3c80fc4f0 msgr2=0x7fe3c8103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 shutdown_connections 2024-04-16T05:32:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:20.761+0000 7fe3d05d4640 1 -- 172.21.15.86:0/1792565915 wait complete. 2024-04-16T05:32:20.788 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:20 smithi196 ceph-mon[36373]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:20.788 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:20 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:20.788 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:20 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:21 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6832/3318048466,v1:172.21.15.194:6833/3318048466] up:boot 2024-04-16T05:32:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:21 smithi086 ceph-mon[32028]: fsmap 2 up:standby 2024-04-16T05:32:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:32:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:21 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:21 smithi086 ceph-mon[32028]: Deploying daemon mds.j on smithi086 2024-04-16T05:32:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:21 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1792565915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:21.724 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:21 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6832/3318048466,v1:172.21.15.194:6833/3318048466] up:boot 2024-04-16T05:32:21.724 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:21 smithi194 ceph-mon[35865]: fsmap 2 up:standby 2024-04-16T05:32:21.724 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:32:21.724 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:21.724 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:21.724 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:21.724 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:21 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:21.724 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:21 smithi194 ceph-mon[35865]: Deploying daemon mds.j on smithi086 2024-04-16T05:32:21.724 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:21 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1792565915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:21.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:21 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6832/3318048466,v1:172.21.15.194:6833/3318048466] up:boot 2024-04-16T05:32:21.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:21 smithi196 ceph-mon[36373]: fsmap 2 up:standby 2024-04-16T05:32:21.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:32:21.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:21.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:21.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:21.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:21 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:21.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:21 smithi196 ceph-mon[36373]: Deploying daemon mds.j on smithi086 2024-04-16T05:32:21.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:21 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1792565915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:22.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:22.801 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:22 smithi086 ceph-mon[32028]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:22.802 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:32:22 smithi086 systemd[1]: Starting Ceph mds.j for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:22.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.891+0000 7fb54cc7e640 1 -- 172.21.15.86:0/3997950579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548107d80 msgr2=0x7fb54810a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:22.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.891+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/3997950579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548107d80 0x7fb54810a170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb540009ca0 tx=0x7fb54002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:22.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.891+0000 7fb54cc7e640 1 -- 172.21.15.86:0/3997950579 shutdown_connections 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.891+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/3997950579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548107d80 0x7fb54810a170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.891+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/3997950579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb548105450 0x7fb548107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.891+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/3997950579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548102b20 0x7fb548104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.891+0000 7fb54cc7e640 1 -- 172.21.15.86:0/3997950579 >> 172.21.15.86:0/3997950579 conn(0x7fb5480fc4b0 msgr2=0x7fb5480fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.891+0000 7fb54cc7e640 1 -- 172.21.15.86:0/3997950579 shutdown_connections 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.891+0000 7fb54cc7e640 1 -- 172.21.15.86:0/3997950579 wait complete. 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54cc7e640 1 Processor -- start 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54cc7e640 1 -- start start 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54cc7e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548102b20 0x7fb5481982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54cc7e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548105450 0x7fb548198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54cc7e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb548107d80 0x7fb54819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54cc7e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb54810dcd0 con 0x7fb548102b20 2024-04-16T05:32:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54cc7e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb54810db50 con 0x7fb548107d80 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54cc7e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb54810de50 con 0x7fb548105450 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548102b20 0x7fb5481982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548102b20 0x7fb5481982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36246/0 (socket says 172.21.15.86:36246) 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54659c640 1 -- 172.21.15.86:0/591480332 learned_addr learned my addr 172.21.15.86:0/591480332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54659c640 1 -- 172.21.15.86:0/591480332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548105450 msgr2=0x7fb548198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54659c640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548105450 0x7fb548198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54659c640 1 -- 172.21.15.86:0/591480332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb548107d80 msgr2=0x7fb54819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb546d9d640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb548107d80 0x7fb54819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54659c640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb548107d80 0x7fb54819f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.892+0000 7fb54659c640 1 -- 172.21.15.86:0/591480332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54819fee0 con 0x7fb548102b20 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.893+0000 7fb54659c640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548102b20 0x7fb5481982c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb53800c920 tx=0x7fb53800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.893+0000 7fb53f7fe640 1 -- 172.21.15.86:0/591480332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538007c70 con 0x7fb548102b20 2024-04-16T05:32:22.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.893+0000 7fb53f7fe640 1 -- 172.21.15.86:0/591480332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb538007e10 con 0x7fb548102b20 2024-04-16T05:32:22.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.893+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5481a01d0 con 0x7fb548102b20 2024-04-16T05:32:22.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.893+0000 7fb53f7fe640 1 -- 172.21.15.86:0/591480332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538018890 con 0x7fb548102b20 2024-04-16T05:32:22.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.893+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5481a0720 con 0x7fb548102b20 2024-04-16T05:32:22.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.894+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5480fdfa0 con 0x7fb548102b20 2024-04-16T05:32:22.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.895+0000 7fb53f7fe640 1 -- 172.21.15.86:0/591480332 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb538018a70 con 0x7fb548102b20 2024-04-16T05:32:22.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.896+0000 7fb53f7fe640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5340a3bb0 0x7fb5340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:22.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.896+0000 7fb53ffff640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5340a3bb0 0x7fb5340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:22.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.896+0000 7fb53f7fe640 1 -- 172.21.15.86:0/591480332 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb5380d0150 con 0x7fb548102b20 2024-04-16T05:32:22.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.896+0000 7fb53ffff640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5340a3bb0 0x7fb5340a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5300058c0 tx=0x7fb530005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:22.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:22.900+0000 7fb53f7fe640 1 -- 172.21.15.86:0/591480332 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb538014030 con 0x7fb548102b20 2024-04-16T05:32:22.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:22 smithi194 ceph-mon[35865]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:22 smithi196 ceph-mon[36373]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:23.113 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:32:22 smithi086 podman[69826]: 2024-04-16 05:32:22.867556127 +0000 UTC m=+0.018746079 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:23.113 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:32:22 smithi086 podman[69826]: 2024-04-16 05:32:22.991444696 +0000 UTC m=+0.142634639 container create 77388f01fc17018ca44ec6a7120b2711a28ed3c56d6cb1eb52110ed33cd2528b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-04-16T05:32:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.121+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb54802d8b0 con 0x7fb548102b20 2024-04-16T05:32:23.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.121+0000 7fb53f7fe640 1 -- 172.21.15.86:0/591480332 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fb53800de20 con 0x7fb548102b20 2024-04-16T05:32:23.120 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:23.121 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T05:32:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 3 2024-04-16T05:32:23.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.123+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5340a3bb0 msgr2=0x7fb5340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.123+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5340a3bb0 0x7fb5340a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5300058c0 tx=0x7fb530005850 comp rx=0 tx=0).stop 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.123+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548102b20 msgr2=0x7fb5481982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.123+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548102b20 0x7fb5481982c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb53800c920 tx=0x7fb53800cde0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.123+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 shutdown_connections 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.124+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5340a3bb0 0x7fb5340a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.124+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb548107d80 0x7fb54819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.124+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548105450 0x7fb548198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.124+0000 7fb54cc7e640 1 --2- 172.21.15.86:0/591480332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb548102b20 0x7fb5481982c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.124+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 >> 172.21.15.86:0/591480332 conn(0x7fb5480fc4b0 msgr2=0x7fb548103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.124+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 shutdown_connections 2024-04-16T05:32:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:23.124+0000 7fb54cc7e640 1 -- 172.21.15.86:0/591480332 wait complete. 2024-04-16T05:32:23.468 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:32:23 smithi086 podman[69826]: 2024-04-16 05:32:23.235699875 +0000 UTC m=+0.386889818 container init 77388f01fc17018ca44ec6a7120b2711a28ed3c56d6cb1eb52110ed33cd2528b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-j, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-16T05:32:23.468 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:32:23 smithi086 podman[69826]: 2024-04-16 05:32:23.238676413 +0000 UTC m=+0.389866358 container start 77388f01fc17018ca44ec6a7120b2711a28ed3c56d6cb1eb52110ed33cd2528b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-j, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-16T05:32:23.468 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:32:23 smithi086 ceph-mds[69866]: -- 172.21.15.86:0/2849917964 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600f294ba40 con 0x5600f291f000 2024-04-16T05:32:23.468 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:32:23 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-j[69860]: starting mds.j at 2024-04-16T05:32:23.468 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:32:23 smithi086 bash[69826]: 77388f01fc17018ca44ec6a7120b2711a28ed3c56d6cb1eb52110ed33cd2528b 2024-04-16T05:32:23.468 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:32:23 smithi086 systemd[1]: Started Ceph mds.j for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:23 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/591480332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:23 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:23 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:23 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:23 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:23 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:23 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:23.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:23 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/591480332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:23.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:23.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:23.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:23.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:23.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:23.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:23 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:23 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/591480332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:23 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:23 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:23 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:23 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:23 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:23 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:24.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:24 smithi194 ceph-mon[35865]: Deploying daemon mds.d on smithi086 2024-04-16T05:32:24.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:24 smithi194 ceph-mon[35865]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:24.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:24 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.86:6834/2004229798,v1:172.21.15.86:6835/2004229798] up:boot 2024-04-16T05:32:24.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:24 smithi194 ceph-mon[35865]: fsmap 3 up:standby 2024-04-16T05:32:24.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:24 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:32:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:24 smithi086 ceph-mon[32028]: Deploying daemon mds.d on smithi086 2024-04-16T05:32:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:24 smithi086 ceph-mon[32028]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:24 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.86:6834/2004229798,v1:172.21.15.86:6835/2004229798] up:boot 2024-04-16T05:32:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:24 smithi086 ceph-mon[32028]: fsmap 3 up:standby 2024-04-16T05:32:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:24 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:32:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:24 smithi196 ceph-mon[36373]: Deploying daemon mds.d on smithi086 2024-04-16T05:32:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:24 smithi196 ceph-mon[36373]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:24 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.86:6834/2004229798,v1:172.21.15.86:6835/2004229798] up:boot 2024-04-16T05:32:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:24 smithi196 ceph-mon[36373]: fsmap 3 up:standby 2024-04-16T05:32:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:24 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:32:25.135 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 -- 172.21.15.86:0/3600769699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558107de0 msgr2=0x7fa55810a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 --2- 172.21.15.86:0/3600769699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558107de0 0x7fa55810a1d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa548009900 tx=0x7fa54802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 -- 172.21.15.86:0/3600769699 shutdown_connections 2024-04-16T05:32:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 --2- 172.21.15.86:0/3600769699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558107de0 0x7fa55810a1d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 --2- 172.21.15.86:0/3600769699 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5581054b0 0x7fa5581078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 --2- 172.21.15.86:0/3600769699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa558102b80 0x7fa558104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 -- 172.21.15.86:0/3600769699 >> 172.21.15.86:0/3600769699 conn(0x7fa5580fc4b0 msgr2=0x7fa5580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 -- 172.21.15.86:0/3600769699 shutdown_connections 2024-04-16T05:32:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 -- 172.21.15.86:0/3600769699 wait complete. 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 Processor -- start 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.257+0000 7fa55fc14640 1 -- start start 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55fc14640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558102b80 0x7fa558198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55fc14640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5581054b0 0x7fa558198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55fc14640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa558107de0 0x7fa55819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55fc14640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa55810dce0 con 0x7fa558102b80 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55fc14640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa55810db60 con 0x7fa5581054b0 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55fc14640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa55810de60 con 0x7fa558107de0 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d9b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558102b80 0x7fa558198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d9b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558102b80 0x7fa558198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36276/0 (socket says 172.21.15.86:36276) 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d9b0640 1 -- 172.21.15.86:0/4214671078 learned_addr learned my addr 172.21.15.86:0/4214671078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d1af640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5581054b0 0x7fa558198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d9b0640 1 -- 172.21.15.86:0/4214671078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa558107de0 msgr2=0x7fa55819f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55e1b1640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa558107de0 0x7fa55819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d9b0640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa558107de0 0x7fa55819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d9b0640 1 -- 172.21.15.86:0/4214671078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5581054b0 msgr2=0x7fa558198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d9b0640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5581054b0 0x7fa558198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d9b0640 1 -- 172.21.15.86:0/4214671078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55819ff40 con 0x7fa558102b80 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d1af640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5581054b0 0x7fa558198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55d9b0640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558102b80 0x7fa558198320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa55000ea80 tx=0x7fa55000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa556ffd640 1 -- 172.21.15.86:0/4214671078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55000ce40 con 0x7fa558102b80 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa556ffd640 1 -- 172.21.15.86:0/4214671078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa550005020 con 0x7fa558102b80 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5581a0230 con 0x7fa558102b80 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa556ffd640 1 -- 172.21.15.86:0/4214671078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550011760 con 0x7fa558102b80 2024-04-16T05:32:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.258+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5581a0730 con 0x7fa558102b80 2024-04-16T05:32:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.260+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5580fdfa0 con 0x7fa558102b80 2024-04-16T05:32:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.261+0000 7fa556ffd640 1 -- 172.21.15.86:0/4214671078 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa5500051d0 con 0x7fa558102b80 2024-04-16T05:32:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.261+0000 7fa556ffd640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa5400a3bb0 0x7fa5400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.261+0000 7fa55d1af640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa5400a3bb0 0x7fa5400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.261+0000 7fa556ffd640 1 -- 172.21.15.86:0/4214671078 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa5500d0650 con 0x7fa558102b80 2024-04-16T05:32:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.262+0000 7fa55d1af640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa5400a3bb0 0x7fa5400a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa558199830 tx=0x7fa54c005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.266+0000 7fa556ffd640 1 -- 172.21.15.86:0/4214671078 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa550016030 con 0x7fa558102b80 2024-04-16T05:32:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.487+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa55802d620 con 0x7fa558102b80 2024-04-16T05:32:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.487+0000 7fa556ffd640 1 -- 172.21.15.86:0/4214671078 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fa55009d4d0 con 0x7fa558102b80 2024-04-16T05:32:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:25.487 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T05:32:25.488 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-04-16T05:32:25.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa5400a3bb0 msgr2=0x7fa5400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:25.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa5400a3bb0 0x7fa5400a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa558199830 tx=0x7fa54c005ec0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558102b80 msgr2=0x7fa558198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:25.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558102b80 0x7fa558198320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa55000ea80 tx=0x7fa55000ef40 comp rx=0 tx=0).stop 2024-04-16T05:32:25.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 shutdown_connections 2024-04-16T05:32:25.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa558107de0 0x7fa55819f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa5400a3bb0 0x7fa5400a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5581054b0 0x7fa558198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 --2- 172.21.15.86:0/4214671078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa558102b80 0x7fa558198320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:25.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 >> 172.21.15.86:0/4214671078 conn(0x7fa5580fc4b0 msgr2=0x7fa5581034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:25.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 shutdown_connections 2024-04-16T05:32:25.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:25.489+0000 7fa55fc14640 1 -- 172.21.15.86:0/4214671078 wait complete. 2024-04-16T05:32:25.691 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:25 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4214671078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:25.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:25 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4214671078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:25.967 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:32:25 smithi086 systemd[1]: Starting Ceph mds.d for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:25 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4214671078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:26.355 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:32:26 smithi086 podman[70195]: 2024-04-16 05:32:25.995858471 +0000 UTC m=+0.017797986 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:26.355 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:32:26 smithi086 podman[70195]: 2024-04-16 05:32:26.121809543 +0000 UTC m=+0.143749082 container create 695332d77798afec18ae82bfa13e392a2ae975d067e6b0588865b45e8bfb9581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-d, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:32:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:26 smithi086 ceph-mon[32028]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:26 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:26.719 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:32:26 smithi086 podman[70195]: 2024-04-16 05:32:26.357628668 +0000 UTC m=+0.379568164 container init 695332d77798afec18ae82bfa13e392a2ae975d067e6b0588865b45e8bfb9581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, io.buildah.version=1.33.5) 2024-04-16T05:32:26.719 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:32:26 smithi086 podman[70195]: 2024-04-16 05:32:26.360433878 +0000 UTC m=+0.382373375 container start 695332d77798afec18ae82bfa13e392a2ae975d067e6b0588865b45e8bfb9581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-d, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-16T05:32:26.719 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:32:26 smithi086 ceph-mds[70214]: -- 172.21.15.86:0/1336253049 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e35b8ada40 con 0x55e35b880c00 2024-04-16T05:32:26.719 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:32:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-d[70210]: starting mds.d at 2024-04-16T05:32:26.719 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:32:26 smithi086 bash[70195]: 695332d77798afec18ae82bfa13e392a2ae975d067e6b0588865b45e8bfb9581 2024-04-16T05:32:26.719 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:32:26 smithi086 systemd[1]: Started Ceph mds.d for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:26.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:26 smithi194 ceph-mon[35865]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:26.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:26 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:26 smithi196 ceph-mon[36373]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:26.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:26 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:27.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:27.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 -- 172.21.15.86:0/2403881085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103c00 msgr2=0x7f0d3c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:27.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 --2- 172.21.15.86:0/2403881085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103c00 0x7f0d3c10a490 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ca0 tx=0x7f0d3402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 -- 172.21.15.86:0/2403881085 shutdown_connections 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 --2- 172.21.15.86:0/2403881085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103c00 0x7f0d3c10a490 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 --2- 172.21.15.86:0/2403881085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c103240 0x7f0d3c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 --2- 172.21.15.86:0/2403881085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d3c102050 0x7f0d3c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 -- 172.21.15.86:0/2403881085 >> 172.21.15.86:0/2403881085 conn(0x7f0d3c0fd5e0 msgr2=0x7f0d3c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 -- 172.21.15.86:0/2403881085 shutdown_connections 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 -- 172.21.15.86:0/2403881085 wait complete. 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 Processor -- start 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.621+0000 7f0d4371b640 1 -- start start 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d4371b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c102050 0x7f0d3c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d4371b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103240 0x7f0d3c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d4371b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d3c103c00 0x7f0d3c19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d4371b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c10dce0 con 0x7f0d3c103240 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d4371b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c10db60 con 0x7f0d3c103c00 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d4371b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c10de60 con 0x7f0d3c102050 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d40cb6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103240 0x7f0d3c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d40cb6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103240 0x7f0d3c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36298/0 (socket says 172.21.15.86:36298) 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d40cb6640 1 -- 172.21.15.86:0/3040603261 learned_addr learned my addr 172.21.15.86:0/3040603261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d41cb8640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d3c103c00 0x7f0d3c19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d414b7640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c102050 0x7f0d3c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d40cb6640 1 -- 172.21.15.86:0/3040603261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c102050 msgr2=0x7f0d3c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d40cb6640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c102050 0x7f0d3c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d40cb6640 1 -- 172.21.15.86:0/3040603261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d3c103c00 msgr2=0x7f0d3c19f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d40cb6640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d3c103c00 0x7f0d3c19f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d40cb6640 1 -- 172.21.15.86:0/3040603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c19fee0 con 0x7f0d3c103240 2024-04-16T05:32:27.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d40cb6640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103240 0x7f0d3c198800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d950 tx=0x7f0d2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:27.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d327fc640 1 -- 172.21.15.86:0/3040603261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017070 con 0x7f0d3c103240 2024-04-16T05:32:27.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d327fc640 1 -- 172.21.15.86:0/3040603261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2c009c30 con 0x7f0d3c103240 2024-04-16T05:32:27.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c1a01d0 con 0x7f0d3c103240 2024-04-16T05:32:27.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d327fc640 1 -- 172.21.15.86:0/3040603261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00f6b0 con 0x7f0d3c103240 2024-04-16T05:32:27.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.622+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a0720 con 0x7f0d3c103240 2024-04-16T05:32:27.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.623+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c109390 con 0x7f0d3c103240 2024-04-16T05:32:27.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.625+0000 7f0d327fc640 1 -- 172.21.15.86:0/3040603261 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0d2c0109d0 con 0x7f0d3c103240 2024-04-16T05:32:27.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.625+0000 7f0d327fc640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0d280a3bb0 0x7f0d280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:27.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.626+0000 7f0d414b7640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0d280a3bb0 0x7f0d280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:27.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.626+0000 7f0d327fc640 1 -- 172.21.15.86:0/3040603261 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0d2c0cfaf0 con 0x7f0d3c103240 2024-04-16T05:32:27.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.626+0000 7f0d414b7640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0d280a3bb0 0x7f0d280a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c1030a0 tx=0x7f0d38005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.630+0000 7f0d327fc640 1 -- 172.21.15.86:0/3040603261 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d2c009de0 con 0x7f0d3c103240 2024-04-16T05:32:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.843+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d3c02d890 con 0x7f0d3c103240 2024-04-16T05:32:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.844+0000 7f0d327fc640 1 -- 172.21.15.86:0/3040603261 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f0d2c09c320 con 0x7f0d3c103240 2024-04-16T05:32:27.843 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:27.843 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T05:32:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 5 2024-04-16T05:32:27.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0d280a3bb0 msgr2=0x7f0d280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:27.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0d280a3bb0 0x7f0d280a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c1030a0 tx=0x7f0d38005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:27.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103240 msgr2=0x7f0d3c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103240 0x7f0d3c198800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d950 tx=0x7f0d2c00de10 comp rx=0 tx=0).stop 2024-04-16T05:32:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 shutdown_connections 2024-04-16T05:32:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0d280a3bb0 0x7f0d280a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d3c103c00 0x7f0d3c19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c103240 0x7f0d3c198800 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 --2- 172.21.15.86:0/3040603261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c102050 0x7f0d3c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 >> 172.21.15.86:0/3040603261 conn(0x7f0d3c0fd5e0 msgr2=0x7f0d3c106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 shutdown_connections 2024-04-16T05:32:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:27.846+0000 7f0d4371b640 1 -- 172.21.15.86:0/3040603261 wait complete. 2024-04-16T05:32:27.855 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:27 smithi086 ceph-mon[32028]: Deploying daemon mds.e on smithi194 2024-04-16T05:32:27.855 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:27 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.86:6836/2865823959,v1:172.21.15.86:6837/2865823959] up:boot 2024-04-16T05:32:27.855 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:27 smithi086 ceph-mon[32028]: fsmap 4 up:standby 2024-04-16T05:32:27.855 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:27 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:32:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:27 smithi194 ceph-mon[35865]: Deploying daemon mds.e on smithi194 2024-04-16T05:32:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:27 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.86:6836/2865823959,v1:172.21.15.86:6837/2865823959] up:boot 2024-04-16T05:32:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:27 smithi194 ceph-mon[35865]: fsmap 4 up:standby 2024-04-16T05:32:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:27 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:32:27.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:27 smithi196 ceph-mon[36373]: Deploying daemon mds.e on smithi194 2024-04-16T05:32:27.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:27 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.86:6836/2865823959,v1:172.21.15.86:6837/2865823959] up:boot 2024-04-16T05:32:27.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:27 smithi196 ceph-mon[36373]: fsmap 4 up:standby 2024-04-16T05:32:27.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:27 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:32:28.904 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:28 smithi194 ceph-mon[35865]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:28.904 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:28 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3040603261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:28.904 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:32:28 smithi194 systemd[1]: Starting Ceph mds.e for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:28.904 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:32:28 smithi194 podman[66325]: 2024-04-16 05:32:28.808244297 +0000 UTC m=+0.017866629 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:28.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:28 smithi086 ceph-mon[32028]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:28.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:28 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3040603261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:28.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:28 smithi196 ceph-mon[36373]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:28.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:28 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3040603261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:29.156 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:32:28 smithi194 podman[66325]: 2024-04-16 05:32:28.921421822 +0000 UTC m=+0.131044193 container create 8d1011153208e177348e503e966f41c0909e9acce115bd63966690961a66288d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:32:29.656 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:32:29 smithi194 podman[66325]: 2024-04-16 05:32:29.199004133 +0000 UTC m=+0.408626467 container init 8d1011153208e177348e503e966f41c0909e9acce115bd63966690961a66288d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e, maintainer=Guillaume Abrioux , ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-16T05:32:29.656 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:32:29 smithi194 podman[66325]: 2024-04-16 05:32:29.201541032 +0000 UTC m=+0.411163362 container start 8d1011153208e177348e503e966f41c0909e9acce115bd63966690961a66288d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-16T05:32:29.656 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:32:29 smithi194 ceph-mds[66344]: -- 172.21.15.194:0/4235391862 <== mon.2 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c5e2b5a40 con 0x562c5e289000 2024-04-16T05:32:29.656 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:32:29 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e[66340]: starting mds.e at 2024-04-16T05:32:29.656 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:32:29 smithi194 bash[66325]: 8d1011153208e177348e503e966f41c0909e9acce115bd63966690961a66288d 2024-04-16T05:32:29.656 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:32:29 smithi194 systemd[1]: Started Ceph mds.e for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:29.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.976+0000 7ffa73862640 1 -- 172.21.15.86:0/251823444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06fae0 msgr2=0x7ffa6c070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.976+0000 7ffa73862640 1 --2- 172.21.15.86:0/251823444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06fae0 0x7ffa6c070380 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009900 tx=0x7ffa6402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.976+0000 7ffa73862640 1 -- 172.21.15.86:0/251823444 shutdown_connections 2024-04-16T05:32:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.976+0000 7ffa73862640 1 --2- 172.21.15.86:0/251823444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06fae0 0x7ffa6c070380 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.976+0000 7ffa73862640 1 --2- 172.21.15.86:0/251823444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa6c06f140 0x7ffa6c06f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.976+0000 7ffa73862640 1 --2- 172.21.15.86:0/251823444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6c06df00 0x7ffa6c06e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.976+0000 7ffa73862640 1 -- 172.21.15.86:0/251823444 >> 172.21.15.86:0/251823444 conn(0x7ffa6c06caf0 msgr2=0x7ffa6c06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.976+0000 7ffa73862640 1 -- 172.21.15.86:0/251823444 shutdown_connections 2024-04-16T05:32:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.976+0000 7ffa73862640 1 -- 172.21.15.86:0/251823444 wait complete. 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa73862640 1 Processor -- start 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa73862640 1 -- start start 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa73862640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06df00 0x7ffa6c1a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa73862640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa6c06f140 0x7ffa6c1a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa73862640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6c06fae0 0x7ffa6c1ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa73862640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c11cae0 con 0x7ffa6c06df00 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa73862640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c11c960 con 0x7ffa6c06fae0 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa73862640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c11cc60 con 0x7ffa6c06f140 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa715fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06df00 0x7ffa6c1a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa715fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06df00 0x7ffa6c1a70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36310/0 (socket says 172.21.15.86:36310) 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa715fe640 1 -- 172.21.15.86:0/3735408244 learned_addr learned my addr 172.21.15.86:0/3735408244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa70dfd640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa6c06f140 0x7ffa6c1a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa715fe640 1 -- 172.21.15.86:0/3735408244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa6c06f140 msgr2=0x7ffa6c1a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa715fe640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa6c06f140 0x7ffa6c1a7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa715fe640 1 -- 172.21.15.86:0/3735408244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6c06fae0 msgr2=0x7ffa6c1ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa71dff640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6c06fae0 0x7ffa6c1ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa715fe640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6c06fae0 0x7ffa6c1ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.977+0000 7ffa715fe640 1 -- 172.21.15.86:0/3735408244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c1aed00 con 0x7ffa6c06df00 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.978+0000 7ffa70dfd640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa6c06f140 0x7ffa6c1a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.978+0000 7ffa71dff640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6c06fae0 0x7ffa6c1ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.978+0000 7ffa715fe640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06df00 0x7ffa6c1a70e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800ebc0 tx=0x7ffa6800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.978+0000 7ffa627fc640 1 -- 172.21.15.86:0/3735408244 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68010630 con 0x7ffa6c06df00 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.978+0000 7ffa627fc640 1 -- 172.21.15.86:0/3735408244 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68010cb0 con 0x7ffa6c06df00 2024-04-16T05:32:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.978+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c1aeff0 con 0x7ffa6c06df00 2024-04-16T05:32:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.978+0000 7ffa627fc640 1 -- 172.21.15.86:0/3735408244 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68018bd0 con 0x7ffa6c06df00 2024-04-16T05:32:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.978+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c1af450 con 0x7ffa6c06df00 2024-04-16T05:32:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.979+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6c1193c0 con 0x7ffa6c06df00 2024-04-16T05:32:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.980+0000 7ffa627fc640 1 -- 172.21.15.86:0/3735408244 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffa680107d0 con 0x7ffa6c06df00 2024-04-16T05:32:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.981+0000 7ffa627fc640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa580a3bb0 0x7ffa580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:29.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.981+0000 7ffa70dfd640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa580a3bb0 0x7ffa580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:29.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.981+0000 7ffa627fc640 1 -- 172.21.15.86:0/3735408244 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ffa680d06a0 con 0x7ffa6c06df00 2024-04-16T05:32:29.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.981+0000 7ffa70dfd640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa580a3bb0 0x7ffa580a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c1a85f0 tx=0x7ffa5c00b5f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:29.985+0000 7ffa627fc640 1 -- 172.21.15.86:0/3735408244 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa6809d520 con 0x7ffa6c06df00 2024-04-16T05:32:30.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.204+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa6c02d880 con 0x7ffa6c06df00 2024-04-16T05:32:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.204+0000 7ffa627fc640 1 -- 172.21.15.86:0/3735408244 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7ffa6809ce20 con 0x7ffa6c06df00 2024-04-16T05:32:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:30.203 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-16T05:32:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 6 2024-04-16T05:32:30.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa580a3bb0 msgr2=0x7ffa580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:30.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa580a3bb0 0x7ffa580a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c1a85f0 tx=0x7ffa5c00b5f0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06df00 msgr2=0x7ffa6c1a70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06df00 0x7ffa6c1a70e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800ebc0 tx=0x7ffa6800c6a0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 shutdown_connections 2024-04-16T05:32:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ffa580a3bb0 0x7ffa580a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6c06fae0 0x7ffa6c1ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa6c06f140 0x7ffa6c1a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 --2- 172.21.15.86:0/3735408244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c06df00 0x7ffa6c1a70e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 >> 172.21.15.86:0/3735408244 conn(0x7ffa6c06caf0 msgr2=0x7ffa6c10aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 shutdown_connections 2024-04-16T05:32:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:30.206+0000 7ffa73862640 1 -- 172.21.15.86:0/3735408244 wait complete. 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: Deploying daemon mds.f on smithi196 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6834/2618844409,v1:172.21.15.194:6835/2618844409] up:boot 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: fsmap 5 up:standby 2024-04-16T05:32:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:32:30.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:30 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3735408244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: Deploying daemon mds.f on smithi196 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6834/2618844409,v1:172.21.15.194:6835/2618844409] up:boot 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: fsmap 5 up:standby 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:32:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:30 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3735408244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:30.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: Deploying daemon mds.f on smithi196 2024-04-16T05:32:30.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:30.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6834/2618844409,v1:172.21.15.194:6835/2618844409] up:boot 2024-04-16T05:32:30.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: fsmap 5 up:standby 2024-04-16T05:32:30.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:32:30.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:30 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3735408244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:31.713 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:32:31 smithi196 systemd[1]: Starting Ceph mds.f for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:31.964 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:32:31 smithi196 podman[66403]: 2024-04-16 05:32:31.617909415 +0000 UTC m=+0.018186628 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:31.964 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:32:31 smithi196 podman[66403]: 2024-04-16 05:32:31.738519204 +0000 UTC m=+0.138796458 container create e9abd2da7dad6884dd596da632ca505790801e8c78df0003dd8d73e2d7ca2d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-16T05:32:32.217 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:32.235 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:32:31 smithi196 podman[66403]: 2024-04-16 05:32:31.966162339 +0000 UTC m=+0.366439551 container init e9abd2da7dad6884dd596da632ca505790801e8c78df0003dd8d73e2d7ca2d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=) 2024-04-16T05:32:32.235 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:32:31 smithi196 podman[66403]: 2024-04-16 05:32:31.969421109 +0000 UTC m=+0.369698321 container start e9abd2da7dad6884dd596da632ca505790801e8c78df0003dd8d73e2d7ca2d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:32:32.235 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:32:31 smithi196 ceph-mds[66423]: -- 172.21.15.196:0/3875746843 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557316141a40 con 0x557316114c00 2024-04-16T05:32:32.235 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:32:31 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f[66419]: starting mds.f at 2024-04-16T05:32:32.235 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:32:32 smithi196 bash[66403]: e9abd2da7dad6884dd596da632ca505790801e8c78df0003dd8d73e2d7ca2d25 2024-04-16T05:32:32.235 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:32:32 smithi196 systemd[1]: Started Ceph mds.f for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:32.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.332+0000 7f84bcd90640 1 -- 172.21.15.86:0/3669948639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b8102570 msgr2=0x7f84b8102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.332+0000 7f84bcd90640 1 --2- 172.21.15.86:0/3669948639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b8102570 0x7f84b8102990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009900 tx=0x7f84a802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:32.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.332+0000 7f84bcd90640 1 -- 172.21.15.86:0/3669948639 shutdown_connections 2024-04-16T05:32:32.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.332+0000 7f84bcd90640 1 --2- 172.21.15.86:0/3669948639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b8101b80 0x7f84b8109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.332+0000 7f84bcd90640 1 --2- 172.21.15.86:0/3669948639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b81011c0 0x7f84b8101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.332+0000 7f84bcd90640 1 --2- 172.21.15.86:0/3669948639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b8102570 0x7f84b8102990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.332+0000 7f84bcd90640 1 -- 172.21.15.86:0/3669948639 >> 172.21.15.86:0/3669948639 conn(0x7f84b80fbc70 msgr2=0x7f84b80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:32.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.332+0000 7f84bcd90640 1 -- 172.21.15.86:0/3669948639 shutdown_connections 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.332+0000 7f84bcd90640 1 -- 172.21.15.86:0/3669948639 wait complete. 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.333+0000 7f84bcd90640 1 Processor -- start 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.333+0000 7f84bcd90640 1 -- start start 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.333+0000 7f84bcd90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b81011c0 0x7f84b8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.333+0000 7f84bcd90640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b8101b80 0x7f84b81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.333+0000 7f84bcd90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b8102570 0x7f84b819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.333+0000 7f84bcd90640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810cc50 con 0x7f84b81011c0 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.333+0000 7f84bcd90640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810cad0 con 0x7f84b8101b80 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.333+0000 7f84bcd90640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810cdd0 con 0x7f84b8102570 2024-04-16T05:32:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b81011c0 0x7f84b8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b81011c0 0x7f84b8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42698/0 (socket says 172.21.15.86:42698) 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b659c640 1 -- 172.21.15.86:0/1831957648 learned_addr learned my addr 172.21.15.86:0/1831957648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b6d9d640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b8102570 0x7f84b819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b659c640 1 -- 172.21.15.86:0/1831957648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b8102570 msgr2=0x7f84b819f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b659c640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b8102570 0x7f84b819f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b5d9b640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b8101b80 0x7f84b81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b659c640 1 -- 172.21.15.86:0/1831957648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b8101b80 msgr2=0x7f84b81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b659c640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b8101b80 0x7f84b81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b659c640 1 -- 172.21.15.86:0/1831957648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b819ffb0 con 0x7f84b81011c0 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b5d9b640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b8101b80 0x7f84b81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84b659c640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b81011c0 0x7f84b8198390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000c00 tx=0x7f84a8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84a77fe640 1 -- 172.21.15.86:0/1831957648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8041070 con 0x7f84b81011c0 2024-04-16T05:32:32.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84a77fe640 1 -- 172.21.15.86:0/1831957648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a80054e0 con 0x7f84b81011c0 2024-04-16T05:32:32.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b81a0240 con 0x7f84b81011c0 2024-04-16T05:32:32.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84a77fe640 1 -- 172.21.15.86:0/1831957648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a80346a0 con 0x7f84b81011c0 2024-04-16T05:32:32.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.334+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b81a0730 con 0x7f84b81011c0 2024-04-16T05:32:32.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.335+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b81a0990 con 0x7f84b81011c0 2024-04-16T05:32:32.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.336+0000 7f84a77fe640 1 -- 172.21.15.86:0/1831957648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f84a8044070 con 0x7f84b81011c0 2024-04-16T05:32:32.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.337+0000 7f84a77fe640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f84a00a3bb0 0x7f84a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.337+0000 7f84b5d9b640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f84a00a3bb0 0x7f84a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.337+0000 7f84a77fe640 1 -- 172.21.15.86:0/1831957648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f84a80f1c00 con 0x7f84b81011c0 2024-04-16T05:32:32.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.337+0000 7f84b5d9b640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f84a00a3bb0 0x7f84a00a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f84b81998a0 tx=0x7f84ac005e70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:32.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.341+0000 7f84a77fe640 1 -- 172.21.15.86:0/1831957648 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84a80f8080 con 0x7f84b81011c0 2024-04-16T05:32:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.558+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84b81a0b60 con 0x7f84b81011c0 2024-04-16T05:32:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.559+0000 7f84a77fe640 1 -- 172.21.15.86:0/1831957648 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f84a80bea80 con 0x7f84b81011c0 2024-04-16T05:32:32.558 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:32.558 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-16T05:32:32.558 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-16T05:32:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f84a00a3bb0 msgr2=0x7f84a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f84a00a3bb0 0x7f84a00a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f84b81998a0 tx=0x7f84ac005e70 comp rx=0 tx=0).stop 2024-04-16T05:32:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b81011c0 msgr2=0x7f84b8198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b81011c0 0x7f84b8198390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000c00 tx=0x7f84a8005dd0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 shutdown_connections 2024-04-16T05:32:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b8102570 0x7f84b819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f84a00a3bb0 0x7f84a00a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b8101b80 0x7f84b81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 --2- 172.21.15.86:0/1831957648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b81011c0 0x7f84b8198390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 >> 172.21.15.86:0/1831957648 conn(0x7f84b80fbc70 msgr2=0x7f84b80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 shutdown_connections 2024-04-16T05:32:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:32.561+0000 7f84bcd90640 1 -- 172.21.15.86:0/1831957648 wait complete. 2024-04-16T05:32:32.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:32 smithi196 ceph-mon[36373]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:32.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:32.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:32.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:32.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:32.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:32.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:32 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:32 smithi194 ceph-mon[35865]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:32 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:32.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:32 smithi086 ceph-mon[32028]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:32 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:33.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:33 smithi196 ceph-mon[36373]: Deploying daemon mds.c on smithi196 2024-04-16T05:32:33.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:33 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] up:boot 2024-04-16T05:32:33.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:33 smithi196 ceph-mon[36373]: fsmap 6 up:standby 2024-04-16T05:32:33.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:33 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:32:33.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:33 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1831957648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:33.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:33 smithi194 ceph-mon[35865]: Deploying daemon mds.c on smithi196 2024-04-16T05:32:33.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:33 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] up:boot 2024-04-16T05:32:33.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:33 smithi194 ceph-mon[35865]: fsmap 6 up:standby 2024-04-16T05:32:33.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:33 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:32:33.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:33 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1831957648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:33.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:33 smithi086 ceph-mon[32028]: Deploying daemon mds.c on smithi196 2024-04-16T05:32:33.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:33 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] up:boot 2024-04-16T05:32:33.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:33 smithi086 ceph-mon[32028]: fsmap 6 up:standby 2024-04-16T05:32:33.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:33 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:32:33.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:33 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1831957648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:34.485 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:32:34 smithi196 systemd[1]: Starting Ceph mds.c for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:34.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:34.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.688+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/2149453717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 msgr2=0x7f5fc0102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:34.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.688+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/2149453717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 0x7f5fc0102560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009900 tx=0x7f5fa802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.688+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/2149453717 shutdown_connections 2024-04-16T05:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.688+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/2149453717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc0102aa0 0x7f5fc01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.688+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/2149453717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 0x7f5fc0102560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.688+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/2149453717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc0100f10 0x7f5fc0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.688+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/2149453717 >> 172.21.15.86:0/2149453717 conn(0x7f5fc00fc4f0 msgr2=0x7f5fc00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.688+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/2149453717 shutdown_connections 2024-04-16T05:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.688+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/2149453717 wait complete. 2024-04-16T05:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fc4ed9640 1 Processor -- start 2024-04-16T05:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fc4ed9640 1 -- start start 2024-04-16T05:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fc4ed9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc0100f10 0x7f5fc01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fc4ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 0x7f5fc01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fc4ed9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc0102aa0 0x7f5fc019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fc4ed9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc010cbe0 con 0x7f5fc01020e0 2024-04-16T05:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fc4ed9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc010ca60 con 0x7f5fc0100f10 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fc4ed9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc010cd60 con 0x7f5fc0102aa0 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fb7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 0x7f5fc01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fb7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 0x7f5fc01977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42720/0 (socket says 172.21.15.86:42720) 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fb7fff640 1 -- 172.21.15.86:0/3862511894 learned_addr learned my addr 172.21.15.86:0/3862511894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fbed9d640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc0102aa0 0x7f5fc019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.689+0000 7f5fbe59c640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc0100f10 0x7f5fc01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fb7fff640 1 -- 172.21.15.86:0/3862511894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc0102aa0 msgr2=0x7f5fc019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fb7fff640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc0102aa0 0x7f5fc019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fb7fff640 1 -- 172.21.15.86:0/3862511894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc0100f10 msgr2=0x7f5fc01972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fb7fff640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc0100f10 0x7f5fc01972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fb7fff640 1 -- 172.21.15.86:0/3862511894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc019eed0 con 0x7f5fc01020e0 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fb7fff640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 0x7f5fc01977f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009230 tx=0x7f5fa8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fb77fe640 1 -- 172.21.15.86:0/3862511894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8041070 con 0x7f5fc01020e0 2024-04-16T05:32:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fb77fe640 1 -- 172.21.15.86:0/3862511894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa80044d0 con 0x7f5fc01020e0 2024-04-16T05:32:34.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc019f160 con 0x7f5fc01020e0 2024-04-16T05:32:34.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fb77fe640 1 -- 172.21.15.86:0/3862511894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8034440 con 0x7f5fc01020e0 2024-04-16T05:32:34.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.690+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc019f6a0 con 0x7f5fc01020e0 2024-04-16T05:32:34.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.691+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc01083e0 con 0x7f5fc01020e0 2024-04-16T05:32:34.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.692+0000 7f5fb77fe640 1 -- 172.21.15.86:0/3862511894 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5fa8044070 con 0x7f5fc01020e0 2024-04-16T05:32:34.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.693+0000 7f5fb77fe640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fac0a3c30 0x7f5fac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:34.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.693+0000 7f5fbe59c640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fac0a3c30 0x7f5fac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:34.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.693+0000 7f5fb77fe640 1 -- 172.21.15.86:0/3862511894 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5fa80f1aa0 con 0x7f5fc01020e0 2024-04-16T05:32:34.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.693+0000 7f5fbe59c640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fac0a3c30 0x7f5fac0a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0100c30 tx=0x7f5fb0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:34.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.697+0000 7f5fb77fe640 1 -- 172.21.15.86:0/3862511894 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fa80be2d0 con 0x7f5fc01020e0 2024-04-16T05:32:34.807 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:34 smithi194 ceph-mon[35865]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:34.819 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:34 smithi196 ceph-mon[36373]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:34.819 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:32:34 smithi196 podman[66708]: 2024-04-16 05:32:34.419318506 +0000 UTC m=+0.017681735 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:34.819 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:32:34 smithi196 podman[66708]: 2024-04-16 05:32:34.543602538 +0000 UTC m=+0.141965756 container create b11b26d5c298d8e95906863d2432d12ba6c6fad53c3c21ce5f95d3eae2b0be85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:32:34.820 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:32:34 smithi196 podman[66708]: 2024-04-16 05:32:34.821606957 +0000 UTC m=+0.419970176 container init b11b26d5c298d8e95906863d2432d12ba6c6fad53c3c21ce5f95d3eae2b0be85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=) 2024-04-16T05:32:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.914+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fc002d840 con 0x7f5fc01020e0 2024-04-16T05:32:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.915+0000 7f5fb77fe640 1 -- 172.21.15.86:0/3862511894 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f5fa80c25d0 con 0x7f5fc01020e0 2024-04-16T05:32:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:34.914 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-16T05:32:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-16T05:32:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fac0a3c30 msgr2=0x7f5fac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fac0a3c30 0x7f5fac0a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0100c30 tx=0x7f5fb0005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 msgr2=0x7f5fc01977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 0x7f5fc01977f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009230 tx=0x7f5fa8005d50 comp rx=0 tx=0).stop 2024-04-16T05:32:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 shutdown_connections 2024-04-16T05:32:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc0102aa0 0x7f5fc019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc01020e0 0x7f5fc01977f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fac0a3c30 0x7f5fac0a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 --2- 172.21.15.86:0/3862511894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc0100f10 0x7f5fc01972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 >> 172.21.15.86:0/3862511894 conn(0x7f5fc00fc4f0 msgr2=0x7f5fc01055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:34.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 shutdown_connections 2024-04-16T05:32:34.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:34.917+0000 7f5fc4ed9640 1 -- 172.21.15.86:0/3862511894 wait complete. 2024-04-16T05:32:34.926 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:34 smithi086 ceph-mon[32028]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:35.235 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:32:34 smithi196 podman[66708]: 2024-04-16 05:32:34.824465363 +0000 UTC m=+0.422828577 container start b11b26d5c298d8e95906863d2432d12ba6c6fad53c3c21ce5f95d3eae2b0be85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-16T05:32:35.235 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:32:34 smithi196 ceph-mds[66728]: -- 172.21.15.196:0/1000991513 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55943cba5a40 con 0x55943cb78c00 2024-04-16T05:32:35.235 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:32:34 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c[66724]: starting mds.c at 2024-04-16T05:32:35.235 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:32:34 smithi196 bash[66708]: b11b26d5c298d8e95906863d2432d12ba6c6fad53c3c21ce5f95d3eae2b0be85 2024-04-16T05:32:35.235 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:32:34 smithi196 systemd[1]: Started Ceph mds.c for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:35.875 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:35 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3862511894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:35.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:35 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:35.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:35 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:35.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:35 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:35.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:35 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:35.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:35 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:35.876 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:35 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:35.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:35 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3862511894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:35.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:35 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:35.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:35 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:35.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:35 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:35.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:35 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:35.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:35 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:35.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:35 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:35.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:35 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3862511894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:35.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:35 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:35.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:35 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:35.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:35 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:35.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:35 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:35.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:35 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:35.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:35 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:36.848 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:36 smithi194 ceph-mon[35865]: Deploying daemon mds.h on smithi194 2024-04-16T05:32:36.848 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:36 smithi194 ceph-mon[35865]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:36.848 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:36 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] up:boot 2024-04-16T05:32:36.848 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:36 smithi194 ceph-mon[35865]: fsmap 7 up:standby 2024-04-16T05:32:36.848 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:36 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:32:36.928 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:36.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:36 smithi086 ceph-mon[32028]: Deploying daemon mds.h on smithi194 2024-04-16T05:32:36.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:36 smithi086 ceph-mon[32028]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:36.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:36 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] up:boot 2024-04-16T05:32:36.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:36 smithi086 ceph-mon[32028]: fsmap 7 up:standby 2024-04-16T05:32:36.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:36 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:32:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:36 smithi196 ceph-mon[36373]: Deploying daemon mds.h on smithi194 2024-04-16T05:32:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:36 smithi196 ceph-mon[36373]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:36 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] up:boot 2024-04-16T05:32:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:36 smithi196 ceph-mon[36373]: fsmap 7 up:standby 2024-04-16T05:32:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:36 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:32:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.045+0000 7f7dc10e4640 1 -- 172.21.15.86:0/3726337308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc102050 msgr2=0x7f7dbc102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.045+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/3726337308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc102050 0x7f7dbc102470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009900 tx=0x7f7dac02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.045+0000 7f7dc10e4640 1 -- 172.21.15.86:0/3726337308 shutdown_connections 2024-04-16T05:32:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.045+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/3726337308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc103c00 0x7f7dbc10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.045+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/3726337308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dbc103240 0x7f7dbc1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.046+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/3726337308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc102050 0x7f7dbc102470 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.046+0000 7f7dc10e4640 1 -- 172.21.15.86:0/3726337308 >> 172.21.15.86:0/3726337308 conn(0x7f7dbc0fd5e0 msgr2=0x7f7dbc0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.046+0000 7f7dc10e4640 1 -- 172.21.15.86:0/3726337308 shutdown_connections 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.046+0000 7f7dc10e4640 1 -- 172.21.15.86:0/3726337308 wait complete. 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.046+0000 7f7dc10e4640 1 Processor -- start 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.046+0000 7f7dc10e4640 1 -- start start 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dc10e4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc102050 0x7f7dbc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dc10e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dbc103240 0x7f7dbc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dc10e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc103c00 0x7f7dbc19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dc10e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc10dce0 con 0x7f7dbc103c00 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dc10e4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc10db60 con 0x7f7dbc102050 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dc10e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc10de60 con 0x7f7dbc103240 2024-04-16T05:32:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc103c00 0x7f7dbc19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc103c00 0x7f7dbc19f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42740/0 (socket says 172.21.15.86:42740) 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbb59e640 1 -- 172.21.15.86:0/1996033467 learned_addr learned my addr 172.21.15.86:0/1996033467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbad9d640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc102050 0x7f7dbc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dba59c640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dbc103240 0x7f7dbc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbb59e640 1 -- 172.21.15.86:0/1996033467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dbc103240 msgr2=0x7f7dbc1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbb59e640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dbc103240 0x7f7dbc1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbb59e640 1 -- 172.21.15.86:0/1996033467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc102050 msgr2=0x7f7dbc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbb59e640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc102050 0x7f7dbc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbb59e640 1 -- 172.21.15.86:0/1996033467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc19ff90 con 0x7f7dbc103c00 2024-04-16T05:32:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dbb59e640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc103c00 0x7f7dbc19f920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7db400eb10 tx=0x7f7db400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7da3fff640 1 -- 172.21.15.86:0/1996033467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4004b20 con 0x7f7dbc103c00 2024-04-16T05:32:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc1a0280 con 0x7f7dbc103c00 2024-04-16T05:32:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7da3fff640 1 -- 172.21.15.86:0/1996033467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7db4004cc0 con 0x7f7dbc103c00 2024-04-16T05:32:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.047+0000 7f7da3fff640 1 -- 172.21.15.86:0/1996033467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4010760 con 0x7f7dbc103c00 2024-04-16T05:32:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.048+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc1a0780 con 0x7f7dbc103c00 2024-04-16T05:32:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.048+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc109370 con 0x7f7dbc103c00 2024-04-16T05:32:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.049+0000 7f7da3fff640 1 -- 172.21.15.86:0/1996033467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7db4005020 con 0x7f7dbc103c00 2024-04-16T05:32:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.050+0000 7f7da3fff640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7da40a3bb0 0x7f7da40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.050+0000 7f7dbad9d640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7da40a3bb0 0x7f7da40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.050+0000 7f7da3fff640 1 -- 172.21.15.86:0/1996033467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7db40d0580 con 0x7f7dbc103c00 2024-04-16T05:32:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.051+0000 7f7dbad9d640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7da40a3bb0 0x7f7da40a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7dac000c00 tx=0x7f7dac005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:37.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.055+0000 7f7da3fff640 1 -- 172.21.15.86:0/1996033467 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7db409d400 con 0x7f7dbc103c00 2024-04-16T05:32:37.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.270+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dbc062c00 con 0x7f7dbc103c00 2024-04-16T05:32:37.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.270+0000 7f7da3fff640 1 -- 172.21.15.86:0/1996033467 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f7db409cd00 con 0x7f7dbc103c00 2024-04-16T05:32:37.269 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:37.269 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-16T05:32:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 8 2024-04-16T05:32:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7da40a3bb0 msgr2=0x7f7da40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7da40a3bb0 0x7f7da40a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7dac000c00 tx=0x7f7dac005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc103c00 msgr2=0x7f7dbc19f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc103c00 0x7f7dbc19f920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7db400eb10 tx=0x7f7db400c2d0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 shutdown_connections 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dbc103c00 0x7f7dbc19f920 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dbc103240 0x7f7dbc1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7da40a3bb0 0x7f7da40a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 --2- 172.21.15.86:0/1996033467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc102050 0x7f7dbc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 >> 172.21.15.86:0/1996033467 conn(0x7f7dbc0fd5e0 msgr2=0x7f7dbc106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.272+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 shutdown_connections 2024-04-16T05:32:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:37.273+0000 7f7dc10e4640 1 -- 172.21.15.86:0/1996033467 wait complete. 2024-04-16T05:32:37.533 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:32:37 smithi194 podman[66648]: 2024-04-16 05:32:37.535070275 +0000 UTC m=+0.178003710 container create 33598d38e68fd354bea5c4ced1d2bbaaad974db4d538d147fff0539202e2c423 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-16T05:32:37.802 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:37 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1996033467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:37.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:37 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1996033467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:37.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:37 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1996033467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:38.156 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:32:37 smithi194 podman[66648]: 2024-04-16 05:32:37.804539474 +0000 UTC m=+0.447472908 container init 33598d38e68fd354bea5c4ced1d2bbaaad974db4d538d147fff0539202e2c423 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T05:32:38.156 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:32:37 smithi194 podman[66648]: 2024-04-16 05:32:37.807480185 +0000 UTC m=+0.450413622 container start 33598d38e68fd354bea5c4ced1d2bbaaad974db4d538d147fff0539202e2c423 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-16T05:32:38.156 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:32:37 smithi194 ceph-mds[66667]: -- 172.21.15.194:0/2589133486 <== mon.2 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a76b331a40 con 0x55a76b305000 2024-04-16T05:32:38.156 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:32:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h[66663]: starting mds.h at 2024-04-16T05:32:38.156 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:32:37 smithi194 bash[66648]: 33598d38e68fd354bea5c4ced1d2bbaaad974db4d538d147fff0539202e2c423 2024-04-16T05:32:38.156 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:32:37 smithi194 systemd[1]: Started Ceph mds.h for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:38.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:38 smithi196 ceph-mon[36373]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:38.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:38 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:38.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:38 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:38.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:38 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:38.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:38 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:38.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:38 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:38.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:38 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:38 smithi194 ceph-mon[35865]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:38 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:38 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:38 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:38 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:38 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:38 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:38.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:38 smithi086 ceph-mon[32028]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:38.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:38 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:38.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:38 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:38.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:38 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:38.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:38 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:38.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:38 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:38.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:38 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:39.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.400+0000 7ff60ec63640 1 -- 172.21.15.86:0/3741193128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 msgr2=0x7ff6081018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.400+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3741193128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 0x7ff6081018a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009900 tx=0x7ff5fc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.401+0000 7ff60ec63640 1 -- 172.21.15.86:0/3741193128 shutdown_connections 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.401+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3741193128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff608102b50 0x7ff6081093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.401+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3741193128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff608102190 0x7ff608102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.401+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3741193128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 0x7ff6081018a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.401+0000 7ff60ec63640 1 -- 172.21.15.86:0/3741193128 >> 172.21.15.86:0/3741193128 conn(0x7ff6080fbc70 msgr2=0x7ff6080fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.401+0000 7ff60ec63640 1 -- 172.21.15.86:0/3741193128 shutdown_connections 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.401+0000 7ff60ec63640 1 -- 172.21.15.86:0/3741193128 wait complete. 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.401+0000 7ff60ec63640 1 Processor -- start 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.401+0000 7ff60ec63640 1 -- start start 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60ec63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 0x7ff608198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:39.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60ec63640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff608102190 0x7ff608198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60ec63640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff608102b50 0x7ff60819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60ec63640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff60810cc20 con 0x7ff608101480 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60ec63640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff60810caa0 con 0x7ff608102190 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60ec63640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff60810cda0 con 0x7ff608102b50 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60c9ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 0x7ff608198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60c9ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 0x7ff608198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42754/0 (socket says 172.21.15.86:42754) 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60c9ff640 1 -- 172.21.15.86:0/3360007985 learned_addr learned my addr 172.21.15.86:0/3360007985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60d200640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff608102b50 0x7ff60819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff607fff640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff608102190 0x7ff608198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60c9ff640 1 -- 172.21.15.86:0/3360007985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff608102b50 msgr2=0x7ff60819f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60c9ff640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff608102b50 0x7ff60819f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60c9ff640 1 -- 172.21.15.86:0/3360007985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff608102190 msgr2=0x7ff608198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60c9ff640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff608102190 0x7ff608198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60c9ff640 1 -- 172.21.15.86:0/3360007985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6081a0020 con 0x7ff608101480 2024-04-16T05:32:39.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff607fff640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff608102190 0x7ff608198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:39.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60c9ff640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 0x7ff608198400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc005ab0 tx=0x7ff5fc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:39.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff605ffb640 1 -- 172.21.15.86:0/3360007985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc041070 con 0x7ff608101480 2024-04-16T05:32:39.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6081a02b0 con 0x7ff608101480 2024-04-16T05:32:39.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff605ffb640 1 -- 172.21.15.86:0/3360007985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc0054e0 con 0x7ff608101480 2024-04-16T05:32:39.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.402+0000 7ff605ffb640 1 -- 172.21.15.86:0/3360007985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc034440 con 0x7ff608101480 2024-04-16T05:32:39.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.403+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6081a07a0 con 0x7ff608101480 2024-04-16T05:32:39.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.404+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6081a0a00 con 0x7ff608101480 2024-04-16T05:32:39.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.405+0000 7ff605ffb640 1 -- 172.21.15.86:0/3360007985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff5fc044070 con 0x7ff608101480 2024-04-16T05:32:39.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.405+0000 7ff605ffb640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:39.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.405+0000 7ff607fff640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:39.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.406+0000 7ff605ffb640 1 -- 172.21.15.86:0/3360007985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff5fc0f1bb0 con 0x7ff608101480 2024-04-16T05:32:39.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.406+0000 7ff607fff640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff608199910 tx=0x7ff5f4005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:39.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.410+0000 7ff605ffb640 1 -- 172.21.15.86:0/3360007985 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5fc0bea30 con 0x7ff608101480 2024-04-16T05:32:39.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.630+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6081a0bd0 con 0x7ff608101480 2024-04-16T05:32:39.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.631+0000 7ff605ffb640 1 -- 172.21.15.86:0/3360007985 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7ff5fc0be330 con 0x7ff608101480 2024-04-16T05:32:39.631 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:39.631 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-16T05:32:39.631 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-16T05:32:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5f80a3bb0 msgr2=0x7ff5f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff608199910 tx=0x7ff5f4005d30 comp rx=0 tx=0).stop 2024-04-16T05:32:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 msgr2=0x7ff608198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 0x7ff608198400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc005ab0 tx=0x7ff5fc005dc0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 shutdown_connections 2024-04-16T05:32:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff608102b50 0x7ff60819f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff608102190 0x7ff608198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 --2- 172.21.15.86:0/3360007985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608101480 0x7ff608198400 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 >> 172.21.15.86:0/3360007985 conn(0x7ff6080fbc70 msgr2=0x7ff6080fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 shutdown_connections 2024-04-16T05:32:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:39.633+0000 7ff60ec63640 1 -- 172.21.15.86:0/3360007985 wait complete. 2024-04-16T05:32:39.643 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:39 smithi086 ceph-mon[32028]: Deploying daemon mds.i on smithi196 2024-04-16T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:39 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] up:boot 2024-04-16T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:39 smithi086 ceph-mon[32028]: fsmap 8 up:standby 2024-04-16T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:39 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:32:39.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:39 smithi196 ceph-mon[36373]: Deploying daemon mds.i on smithi196 2024-04-16T05:32:39.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:39 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] up:boot 2024-04-16T05:32:39.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:39 smithi196 ceph-mon[36373]: fsmap 8 up:standby 2024-04-16T05:32:39.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:39 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:32:39.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:39 smithi194 ceph-mon[35865]: Deploying daemon mds.i on smithi196 2024-04-16T05:32:39.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:39 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] up:boot 2024-04-16T05:32:39.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:39 smithi194 ceph-mon[35865]: fsmap 8 up:standby 2024-04-16T05:32:39.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:39 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:32:40.235 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:32:40 smithi196 systemd[1]: Starting Ceph mds.i for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:40.550 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:32:40 smithi196 podman[67017]: 2024-04-16 05:32:40.190246524 +0000 UTC m=+0.018138225 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:40.550 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:32:40 smithi196 podman[67017]: 2024-04-16 05:32:40.298101068 +0000 UTC m=+0.125992769 container create cdc608bfa682c78d1675be11462c34807cc7015448d02d09617f627b67f658c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-04-16T05:32:40.550 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:32:40 smithi196 podman[67017]: 2024-04-16 05:32:40.551536315 +0000 UTC m=+0.379428026 container init cdc608bfa682c78d1675be11462c34807cc7015448d02d09617f627b67f658c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:32:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:40 smithi194 ceph-mon[35865]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:40 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3360007985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:40.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:40 smithi086 ceph-mon[32028]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:40.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:40 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3360007985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:40 smithi196 ceph-mon[36373]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:40 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3360007985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:40.985 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:32:40 smithi196 podman[67017]: 2024-04-16 05:32:40.554175281 +0000 UTC m=+0.382066986 container start cdc608bfa682c78d1675be11462c34807cc7015448d02d09617f627b67f658c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-16T05:32:40.985 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:32:40 smithi196 ceph-mds[67036]: -- 172.21.15.196:0/3547485599 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560a87cda40 con 0x5560a87a1000 2024-04-16T05:32:40.985 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:32:40 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i[67032]: starting mds.i at 2024-04-16T05:32:40.985 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:32:40 smithi196 bash[67017]: cdc608bfa682c78d1675be11462c34807cc7015448d02d09617f627b67f658c5 2024-04-16T05:32:40.985 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:32:40 smithi196 systemd[1]: Started Ceph mds.i for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:41.646 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.755+0000 7f9ff9df2640 1 -- 172.21.15.86:0/3074053834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4107d30 msgr2=0x7f9ff410a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.755+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/3074053834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4107d30 0x7f9ff410a180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009ca0 tx=0x7f9fec02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.755+0000 7f9ff9df2640 1 -- 172.21.15.86:0/3074053834 shutdown_connections 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.755+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/3074053834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4107d30 0x7f9ff410a180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.755+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/3074053834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff4105400 0x7f9ff41077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.755+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/3074053834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff4102ad0 0x7f9ff4104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.755+0000 7f9ff9df2640 1 -- 172.21.15.86:0/3074053834 >> 172.21.15.86:0/3074053834 conn(0x7f9ff40fc500 msgr2=0x7f9ff40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.755+0000 7f9ff9df2640 1 -- 172.21.15.86:0/3074053834 shutdown_connections 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff9df2640 1 -- 172.21.15.86:0/3074053834 wait complete. 2024-04-16T05:32:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff9df2640 1 Processor -- start 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff9df2640 1 -- start start 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff9df2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4102ad0 0x7f9ff4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff9df2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff4105400 0x7f9ff4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff9df2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff4107d30 0x7f9ff419f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff9df2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff410ddc0 con 0x7f9ff4102ad0 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff9df2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff410dc40 con 0x7f9ff4107d30 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff9df2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff410df40 con 0x7f9ff4105400 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4102ad0 0x7f9ff4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4102ad0 0x7f9ff4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57744/0 (socket says 172.21.15.86:57744) 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9ff37fe640 1 -- 172.21.15.86:0/1373540455 learned_addr learned my addr 172.21.15.86:0/1373540455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.756+0000 7f9febfff640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff4105400 0x7f9ff4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff37fe640 1 -- 172.21.15.86:0/1373540455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff4105400 msgr2=0x7f9ff4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff37fe640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff4105400 0x7f9ff4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff37fe640 1 -- 172.21.15.86:0/1373540455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff4107d30 msgr2=0x7f9ff419f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff37fe640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff4107d30 0x7f9ff419f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff37fe640 1 -- 172.21.15.86:0/1373540455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff41a0020 con 0x7f9ff4102ad0 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9febfff640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff4105400 0x7f9ff4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff37fe640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4102ad0 0x7f9ff4198400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400b4a0 tx=0x7f9fe400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff17fa640 1 -- 172.21.15.86:0/1373540455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4014070 con 0x7f9ff4102ad0 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff17fa640 1 -- 172.21.15.86:0/1373540455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe4010af0 con 0x7f9ff4102ad0 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff41a0310 con 0x7f9ff4102ad0 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff17fa640 1 -- 172.21.15.86:0/1373540455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4017d10 con 0x7f9ff4102ad0 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.757+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff41a0930 con 0x7f9ff4102ad0 2024-04-16T05:32:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.758+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff40fdff0 con 0x7f9ff4102ad0 2024-04-16T05:32:41.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.759+0000 7f9ff17fa640 1 -- 172.21.15.86:0/1373540455 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9fe4017070 con 0x7f9ff4102ad0 2024-04-16T05:32:41.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.759+0000 7f9ff17fa640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9fe00a3c30 0x7f9fe00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.760+0000 7f9febfff640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9fe00a3c30 0x7f9fe00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.760+0000 7f9ff17fa640 1 -- 172.21.15.86:0/1373540455 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9fe40cf2d0 con 0x7f9ff4102ad0 2024-04-16T05:32:41.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.760+0000 7f9febfff640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9fe00a3c30 0x7f9fe00a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006fd0 tx=0x7f9fdc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.764+0000 7f9ff17fa640 1 -- 172.21.15.86:0/1373540455 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fe409c150 con 0x7f9ff4102ad0 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: Deploying daemon mds.g on smithi086 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] up:boot 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: fsmap 9 up:standby 2024-04-16T05:32:41.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:41 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:32:41.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.984+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ff402d620 con 0x7f9ff4102ad0 2024-04-16T05:32:41.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.984+0000 7f9ff17fa640 1 -- 172.21.15.86:0/1373540455 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f9fe409ba50 con 0x7f9ff4102ad0 2024-04-16T05:32:41.984 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:41.984 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-16T05:32:41.984 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 10 2024-04-16T05:32:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: Deploying daemon mds.g on smithi086 2024-04-16T05:32:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:41.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] up:boot 2024-04-16T05:32:41.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: fsmap 9 up:standby 2024-04-16T05:32:41.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:41 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:32:41.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9fe00a3c30 msgr2=0x7f9fe00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9fe00a3c30 0x7f9fe00a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006fd0 tx=0x7f9fdc008040 comp rx=0 tx=0).stop 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4102ad0 msgr2=0x7f9ff4198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4102ad0 0x7f9ff4198400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400b4a0 tx=0x7f9fe400b960 comp rx=0 tx=0).stop 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 shutdown_connections 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9fe00a3c30 0x7f9fe00a60f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff4107d30 0x7f9ff419f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff4105400 0x7f9ff4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 --2- 172.21.15.86:0/1373540455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff4102ad0 0x7f9ff4198400 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 >> 172.21.15.86:0/1373540455 conn(0x7f9ff40fc500 msgr2=0x7f9ff4103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 shutdown_connections 2024-04-16T05:32:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:41.986+0000 7f9ff9df2640 1 -- 172.21.15.86:0/1373540455 wait complete. 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: Deploying daemon mds.g on smithi086 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] up:boot 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: fsmap 9 up:standby 2024-04-16T05:32:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:41 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:32:42.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:42 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1373540455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:43.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:42 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1373540455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:43.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:42 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1373540455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:43.162 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:32:42 smithi086 systemd[1]: Starting Ceph mds.g for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:43.451 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:32:43 smithi086 podman[70787]: 2024-04-16 05:32:43.066525478 +0000 UTC m=+0.017728663 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:43.451 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:32:43 smithi086 podman[70787]: 2024-04-16 05:32:43.184074469 +0000 UTC m=+0.135277641 container create fde136e54058bfe13cdafaea31a315286802037ccd51f302b52de0fcd0efe054 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-16T05:32:43.452 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:32:43 smithi086 podman[70787]: 2024-04-16 05:32:43.453674805 +0000 UTC m=+0.404877978 container init fde136e54058bfe13cdafaea31a315286802037ccd51f302b52de0fcd0efe054 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-16T05:32:43.718 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:32:43 smithi086 podman[70787]: 2024-04-16 05:32:43.457079645 +0000 UTC m=+0.408282820 container start fde136e54058bfe13cdafaea31a315286802037ccd51f302b52de0fcd0efe054 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:32:43.718 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:32:43 smithi086 ceph-mds[70807]: -- 172.21.15.86:0/1076138090 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5592ab955a40 con 0x5592ab929000 2024-04-16T05:32:43.718 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:32:43 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g[70802]: starting mds.g at 2024-04-16T05:32:43.718 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:32:43 smithi086 bash[70787]: fde136e54058bfe13cdafaea31a315286802037ccd51f302b52de0fcd0efe054 2024-04-16T05:32:43.718 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:32:43 smithi086 systemd[1]: Started Ceph mds.g for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:43.998 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.115+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3135617321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 msgr2=0x7f5fb4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.115+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3135617321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 0x7f5fb4101360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009900 tx=0x7f5fb002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.115+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3135617321 shutdown_connections 2024-04-16T05:32:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.115+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3135617321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fb4102a50 0x7f5fb4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.115+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3135617321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fb4102090 0x7f5fb4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.115+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3135617321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 0x7f5fb4101360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.115+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3135617321 >> 172.21.15.86:0/3135617321 conn(0x7f5fb40fc4b0 msgr2=0x7f5fb40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.116+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3135617321 shutdown_connections 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.116+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3135617321 wait complete. 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.116+0000 7f5fbb5bd640 1 Processor -- start 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.116+0000 7f5fbb5bd640 1 -- start start 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fbb5bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 0x7f5fb4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fbb5bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fb4102090 0x7f5fb4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fbb5bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fb4102a50 0x7f5fb419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fbb5bd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb410cc90 con 0x7f5fb4100f40 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fbb5bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb410cb10 con 0x7f5fb4102090 2024-04-16T05:32:44.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fbb5bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb410ce10 con 0x7f5fb4102a50 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9359640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 0x7f5fb4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9359640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 0x7f5fb4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57792/0 (socket says 172.21.15.86:57792) 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9359640 1 -- 172.21.15.86:0/3349748243 learned_addr learned my addr 172.21.15.86:0/3349748243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9b5a640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fb4102a50 0x7f5fb419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb8b58640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fb4102090 0x7f5fb4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9359640 1 -- 172.21.15.86:0/3349748243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fb4102a50 msgr2=0x7f5fb419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9359640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fb4102a50 0x7f5fb419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9359640 1 -- 172.21.15.86:0/3349748243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fb4102090 msgr2=0x7f5fb4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9359640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fb4102090 0x7f5fb4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9359640 1 -- 172.21.15.86:0/3349748243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb419ef50 con 0x7f5fb4100f40 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5fb9359640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 0x7f5fb4197330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009230 tx=0x7f5fb0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:44.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.117+0000 7f5faa7fc640 1 -- 172.21.15.86:0/3349748243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0041070 con 0x7f5fb4100f40 2024-04-16T05:32:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.118+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb419f1e0 con 0x7f5fb4100f40 2024-04-16T05:32:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.118+0000 7f5faa7fc640 1 -- 172.21.15.86:0/3349748243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb00054e0 con 0x7f5fb4100f40 2024-04-16T05:32:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.118+0000 7f5faa7fc640 1 -- 172.21.15.86:0/3349748243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0034440 con 0x7f5fb4100f40 2024-04-16T05:32:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.118+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb419f720 con 0x7f5fb4100f40 2024-04-16T05:32:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.119+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb4108250 con 0x7f5fb4100f40 2024-04-16T05:32:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.120+0000 7f5faa7fc640 1 -- 172.21.15.86:0/3349748243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5fb0044070 con 0x7f5fb4100f40 2024-04-16T05:32:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.121+0000 7f5faa7fc640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fa00a3bb0 0x7f5fa00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.121+0000 7f5fb8b58640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fa00a3bb0 0x7f5fa00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.121+0000 7f5faa7fc640 1 -- 172.21.15.86:0/3349748243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5fb00f20c0 con 0x7f5fb4100f40 2024-04-16T05:32:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.122+0000 7f5fb8b58640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fa00a3bb0 0x7f5fa00a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4198840 tx=0x7f5fa4009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:44.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.124+0000 7f5faa7fc640 1 -- 172.21.15.86:0/3349748243 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fb00beea0 con 0x7f5fb4100f40 2024-04-16T05:32:44.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.369+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fb402d620 con 0x7f5fb4100f40 2024-04-16T05:32:44.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.370+0000 7f5faa7fc640 1 -- 172.21.15.86:0/3349748243 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7f5fb00be7a0 con 0x7f5fb4100f40 2024-04-16T05:32:44.369 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:44.369 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-16T05:32:44.369 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 11 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fa00a3bb0 msgr2=0x7f5fa00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fa00a3bb0 0x7f5fa00a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4198840 tx=0x7f5fa4009110 comp rx=0 tx=0).stop 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 msgr2=0x7f5fb4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 0x7f5fb4197330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009230 tx=0x7f5fb0005dd0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 shutdown_connections 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fb4102a50 0x7f5fb419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5fa00a3bb0 0x7f5fa00a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fb4102090 0x7f5fb4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 --2- 172.21.15.86:0/3349748243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb4100f40 0x7f5fb4197330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 >> 172.21.15.86:0/3349748243 conn(0x7f5fb40fc4b0 msgr2=0x7f5fb4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:44.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 shutdown_connections 2024-04-16T05:32:44.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:44.372+0000 7f5fbb5bd640 1 -- 172.21.15.86:0/3349748243 wait complete. 2024-04-16T05:32:44.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:44.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:44.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:44.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:44.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:44.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:44.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:44.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.86:6838/1379378484,v1:172.21.15.86:6839/1379378484] up:boot 2024-04-16T05:32:44.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: fsmap 10 up:standby 2024-04-16T05:32:44.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:32:44.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:44 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3349748243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.86:6838/1379378484,v1:172.21.15.86:6839/1379378484] up:boot 2024-04-16T05:32:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: fsmap 10 up:standby 2024-04-16T05:32:44.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:32:44.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:44 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3349748243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:44.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:44.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.86:6838/1379378484,v1:172.21.15.86:6839/1379378484] up:boot 2024-04-16T05:32:44.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: fsmap 10 up:standby 2024-04-16T05:32:44.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:32:44.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:44 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3349748243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:45.807 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:45 smithi086 ceph-mon[32028]: Deploying daemon mds.a on smithi086 2024-04-16T05:32:45.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:45 smithi194 ceph-mon[35865]: Deploying daemon mds.a on smithi086 2024-04-16T05:32:45.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:45 smithi196 ceph-mon[36373]: Deploying daemon mds.a on smithi086 2024-04-16T05:32:46.206 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:32:45 smithi086 systemd[1]: Starting Ceph mds.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:46.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:46.468 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:32:46 smithi086 podman[71143]: 2024-04-16 05:32:46.11103664 +0000 UTC m=+0.017825176 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:46.468 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:32:46 smithi086 podman[71143]: 2024-04-16 05:32:46.238715135 +0000 UTC m=+0.145503655 container create d1005bb154d0e1aee3b4bc9e8334033be98cc81f4531e4f00ccc599e8160f356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:32:46.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.494+0000 7fbc648b1640 1 -- 172.21.15.86:0/381843577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 msgr2=0x7fbc60109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:46.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.494+0000 7fbc648b1640 1 --2- 172.21.15.86:0/381843577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 0x7fbc60109370 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009ca0 tx=0x7fbc5802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.494+0000 7fbc648b1640 1 -- 172.21.15.86:0/381843577 shutdown_connections 2024-04-16T05:32:46.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.494+0000 7fbc648b1640 1 --2- 172.21.15.86:0/381843577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 0x7fbc60109370 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.494+0000 7fbc648b1640 1 --2- 172.21.15.86:0/381843577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc60102090 0x7fbc60102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.494+0000 7fbc648b1640 1 --2- 172.21.15.86:0/381843577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc60100f40 0x7fbc60101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.494+0000 7fbc648b1640 1 -- 172.21.15.86:0/381843577 >> 172.21.15.86:0/381843577 conn(0x7fbc600fc4b0 msgr2=0x7fbc600fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:46.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.494+0000 7fbc648b1640 1 -- 172.21.15.86:0/381843577 shutdown_connections 2024-04-16T05:32:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.494+0000 7fbc648b1640 1 -- 172.21.15.86:0/381843577 wait complete. 2024-04-16T05:32:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.495+0000 7fbc648b1640 1 Processor -- start 2024-04-16T05:32:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.495+0000 7fbc648b1640 1 -- start start 2024-04-16T05:32:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.495+0000 7fbc648b1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc60100f40 0x7fbc60198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.495+0000 7fbc648b1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc60102090 0x7fbc60198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.495+0000 7fbc648b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 0x7fbc6019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.495+0000 7fbc648b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6010cbf0 con 0x7fbc60102a50 2024-04-16T05:32:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.495+0000 7fbc648b1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6010ca70 con 0x7fbc60102090 2024-04-16T05:32:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.495+0000 7fbc648b1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6010cd70 con 0x7fbc60100f40 2024-04-16T05:32:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 0x7fbc6019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 0x7fbc6019f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57814/0 (socket says 172.21.15.86:57814) 2024-04-16T05:32:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5ed9d640 1 -- 172.21.15.86:0/1584739270 learned_addr learned my addr 172.21.15.86:0/1584739270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5e59c640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc60100f40 0x7fbc60198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc55d9b640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc60102090 0x7fbc60198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5ed9d640 1 -- 172.21.15.86:0/1584739270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc60100f40 msgr2=0x7fbc60198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5ed9d640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc60100f40 0x7fbc60198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5ed9d640 1 -- 172.21.15.86:0/1584739270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc60102090 msgr2=0x7fbc60198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5ed9d640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc60102090 0x7fbc60198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5ed9d640 1 -- 172.21.15.86:0/1584739270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6019ff30 con 0x7fbc60102a50 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc5ed9d640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 0x7fbc6019f8c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc5802c700 tx=0x7fbc58032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc57fff640 1 -- 172.21.15.86:0/1584739270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58039040 con 0x7fbc60102a50 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc601a01c0 con 0x7fbc60102a50 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc601a0700 con 0x7fbc60102a50 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc57fff640 1 -- 172.21.15.86:0/1584739270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc58044070 con 0x7fbc60102a50 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.496+0000 7fbc57fff640 1 -- 172.21.15.86:0/1584739270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc580042a0 con 0x7fbc60102a50 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.497+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc601a0a20 con 0x7fbc60102a50 2024-04-16T05:32:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.498+0000 7fbc57fff640 1 -- 172.21.15.86:0/1584739270 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbc58003970 con 0x7fbc60102a50 2024-04-16T05:32:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.499+0000 7fbc57fff640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbc4c0a3bb0 0x7fbc4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.499+0000 7fbc5e59c640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbc4c0a3bb0 0x7fbc4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.500+0000 7fbc57fff640 1 -- 172.21.15.86:0/1584739270 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbc580f23e0 con 0x7fbc60102a50 2024-04-16T05:32:46.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.500+0000 7fbc5e59c640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbc4c0a3bb0 0x7fbc4c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc50005fd0 tx=0x7fbc5000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:46.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.504+0000 7fbc57fff640 1 -- 172.21.15.86:0/1584739270 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc580bf2f0 con 0x7fbc60102a50 2024-04-16T05:32:46.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.748+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc601a0bf0 con 0x7fbc60102a50 2024-04-16T05:32:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.748+0000 7fbc57fff640 1 -- 172.21.15.86:0/1584739270 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7fbc580bebf0 con 0x7fbc60102a50 2024-04-16T05:32:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:46.747 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-16T05:32:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 12 2024-04-16T05:32:46.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbc4c0a3bb0 msgr2=0x7fbc4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:46.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbc4c0a3bb0 0x7fbc4c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc50005fd0 tx=0x7fbc5000b390 comp rx=0 tx=0).stop 2024-04-16T05:32:46.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 msgr2=0x7fbc6019f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:46.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 0x7fbc6019f8c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc5802c700 tx=0x7fbc58032f20 comp rx=0 tx=0).stop 2024-04-16T05:32:46.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 shutdown_connections 2024-04-16T05:32:46.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc60102a50 0x7fbc6019f8c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbc4c0a3bb0 0x7fbc4c0a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc60102090 0x7fbc60198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 --2- 172.21.15.86:0/1584739270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc60100f40 0x7fbc60198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:46.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 >> 172.21.15.86:0/1584739270 conn(0x7fbc600fc4b0 msgr2=0x7fbc60069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:46.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 shutdown_connections 2024-04-16T05:32:46.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:46.751+0000 7fbc648b1640 1 -- 172.21.15.86:0/1584739270 wait complete. 2024-04-16T05:32:46.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:46 smithi086 ceph-mon[32028]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:46.759 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:32:46 smithi086 podman[71143]: 2024-04-16 05:32:46.491357234 +0000 UTC m=+0.398145755 container init d1005bb154d0e1aee3b4bc9e8334033be98cc81f4531e4f00ccc599e8160f356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409) 2024-04-16T05:32:46.759 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:32:46 smithi086 podman[71143]: 2024-04-16 05:32:46.494780532 +0000 UTC m=+0.401569054 container start d1005bb154d0e1aee3b4bc9e8334033be98cc81f4531e4f00ccc599e8160f356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-16T05:32:46.760 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:32:46 smithi086 ceph-mds[71193]: -- 172.21.15.86:0/1232985580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561be8e05a40 con 0x561be8dd8800 2024-04-16T05:32:46.760 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:32:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a[71159]: starting mds.a at 2024-04-16T05:32:46.760 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:32:46 smithi086 bash[71143]: d1005bb154d0e1aee3b4bc9e8334033be98cc81f4531e4f00ccc599e8160f356 2024-04-16T05:32:46.760 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:32:46 smithi086 systemd[1]: Started Ceph mds.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:46.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:46 smithi194 ceph-mon[35865]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:46 smithi196 ceph-mon[36373]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:47.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.86:6840/1989684988,v1:172.21.15.86:6841/1989684988] up:boot 2024-04-16T05:32:47.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: fsmap 11 up:standby 2024-04-16T05:32:47.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:32:47.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:47.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:47.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:47.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:47.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:47.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:47.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:47 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1584739270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.86:6840/1989684988,v1:172.21.15.86:6841/1989684988] up:boot 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: fsmap 11 up:standby 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:47.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:47 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1584739270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.86:6840/1989684988,v1:172.21.15.86:6841/1989684988] up:boot 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: fsmap 11 up:standby 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:47.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:47 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1584739270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:48.762 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:48.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.880+0000 7ff62cb3d640 1 -- 172.21.15.86:0/1195106594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6281020d0 msgr2=0x7ff628102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.880+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/1195106594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6281020d0 0x7ff628102550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff618009900 tx=0x7ff61802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:48.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.881+0000 7ff62cb3d640 1 -- 172.21.15.86:0/1195106594 shutdown_connections 2024-04-16T05:32:48.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.881+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/1195106594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff628102a90 0x7ff6281093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.881+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/1195106594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6281020d0 0x7ff628102550 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.881+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/1195106594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff628100f00 0x7ff628101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.881+0000 7ff62cb3d640 1 -- 172.21.15.86:0/1195106594 >> 172.21.15.86:0/1195106594 conn(0x7ff6280fc4e0 msgr2=0x7ff6280fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:48.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.881+0000 7ff62cb3d640 1 -- 172.21.15.86:0/1195106594 shutdown_connections 2024-04-16T05:32:48.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.881+0000 7ff62cb3d640 1 -- 172.21.15.86:0/1195106594 wait complete. 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.881+0000 7ff62cb3d640 1 Processor -- start 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62cb3d640 1 -- start start 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62cb3d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff628100f00 0x7ff62806b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62cb3d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6281020d0 0x7ff62819d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62cb3d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff628102a90 0x7ff62819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62cb3d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810cbd0 con 0x7ff628100f00 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62cb3d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810ca50 con 0x7ff628102a90 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62cb3d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810cd50 con 0x7ff6281020d0 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff628100f00 0x7ff62806b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff628100f00 0x7ff62806b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57852/0 (socket says 172.21.15.86:57852) 2024-04-16T05:32:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62659c640 1 -- 172.21.15.86:0/3661717804 learned_addr learned my addr 172.21.15.86:0/3661717804 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff626d9d640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff628102a90 0x7ff62819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff61dd9b640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6281020d0 0x7ff62819d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62659c640 1 -- 172.21.15.86:0/3661717804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6281020d0 msgr2=0x7ff62819d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62659c640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6281020d0 0x7ff62819d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62659c640 1 -- 172.21.15.86:0/3661717804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff628102a90 msgr2=0x7ff62819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62659c640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff628102a90 0x7ff62819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62659c640 1 -- 172.21.15.86:0/3661717804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62819fff0 con 0x7ff628100f00 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff61dd9b640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6281020d0 0x7ff62819d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62659c640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff628100f00 0x7ff62806b1d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff61000e9c0 tx=0x7ff61000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff61ffff640 1 -- 172.21.15.86:0/3661717804 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61000cd80 con 0x7ff628100f00 2024-04-16T05:32:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff61ffff640 1 -- 172.21.15.86:0/3661717804 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6100035f0 con 0x7ff628100f00 2024-04-16T05:32:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.882+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6281a0280 con 0x7ff628100f00 2024-04-16T05:32:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.883+0000 7ff61ffff640 1 -- 172.21.15.86:0/3661717804 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610010690 con 0x7ff628100f00 2024-04-16T05:32:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.883+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6281a07d0 con 0x7ff628100f00 2024-04-16T05:32:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.883+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6281082e0 con 0x7ff628100f00 2024-04-16T05:32:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.885+0000 7ff61ffff640 1 -- 172.21.15.86:0/3661717804 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff6100037a0 con 0x7ff628100f00 2024-04-16T05:32:48.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.885+0000 7ff61ffff640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6080a3bb0 0x7ff6080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.885+0000 7ff61dd9b640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6080a3bb0 0x7ff6080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.885+0000 7ff61ffff640 1 -- 172.21.15.86:0/3661717804 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff6100d0410 con 0x7ff628100f00 2024-04-16T05:32:48.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.886+0000 7ff61dd9b640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6080a3bb0 0x7ff6080a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff618036040 tx=0x7ff618000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:48.890+0000 7ff61ffff640 1 -- 172.21.15.86:0/3661717804 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff61009d290 con 0x7ff628100f00 2024-04-16T05:32:48.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:48 smithi194 ceph-mon[35865]: Deploying daemon mds.b on smithi194 2024-04-16T05:32:48.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:48 smithi194 ceph-mon[35865]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:48.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:48 smithi086 ceph-mon[32028]: Deploying daemon mds.b on smithi194 2024-04-16T05:32:48.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:48 smithi086 ceph-mon[32028]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:48.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:48 smithi196 ceph-mon[36373]: Deploying daemon mds.b on smithi194 2024-04-16T05:32:48.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:48 smithi196 ceph-mon[36373]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.106+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff62802d620 con 0x7ff628100f00 2024-04-16T05:32:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.107+0000 7ff61ffff640 1 -- 172.21.15.86:0/3661717804 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7ff61009cb90 con 0x7ff628100f00 2024-04-16T05:32:49.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:49.106 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-16T05:32:49.106 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 12 2024-04-16T05:32:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6080a3bb0 msgr2=0x7ff6080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6080a3bb0 0x7ff6080a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff618036040 tx=0x7ff618000bc0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff628100f00 msgr2=0x7ff62806b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff628100f00 0x7ff62806b1d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff61000e9c0 tx=0x7ff61000ee80 comp rx=0 tx=0).stop 2024-04-16T05:32:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 shutdown_connections 2024-04-16T05:32:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6080a3bb0 0x7ff6080a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff628102a90 0x7ff62819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6281020d0 0x7ff62819d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 --2- 172.21.15.86:0/3661717804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff628100f00 0x7ff62806b1d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 >> 172.21.15.86:0/3661717804 conn(0x7ff6280fc4e0 msgr2=0x7ff6281054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 shutdown_connections 2024-04-16T05:32:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:49.109+0000 7ff62cb3d640 1 -- 172.21.15.86:0/3661717804 wait complete. 2024-04-16T05:32:49.634 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:32:49 smithi194 systemd[1]: Started Ceph mds.b for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:49.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:49 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3661717804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:49.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:49 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:49 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3661717804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:49.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:49 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:49 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3661717804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:49.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:49.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:49 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:50.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:50 smithi196 ceph-mon[36373]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:50.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:50 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] up:boot 2024-04-16T05:32:50.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:50 smithi196 ceph-mon[36373]: fsmap 12 up:standby 2024-04-16T05:32:50.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:32:50.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:50 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:50 smithi194 ceph-mon[35865]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:50 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] up:boot 2024-04-16T05:32:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:50 smithi194 ceph-mon[35865]: fsmap 12 up:standby 2024-04-16T05:32:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:32:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:50 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:50.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:50 smithi086 ceph-mon[32028]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:50.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:50 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] up:boot 2024-04-16T05:32:50.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:50 smithi086 ceph-mon[32028]: fsmap 12 up:standby 2024-04-16T05:32:50.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:32:50.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:50 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:51.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:51.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.228+0000 7fbd10315640 1 -- 172.21.15.86:0/539551916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08105490 msgr2=0x7fbd08107880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.228+0000 7fbd10315640 1 --2- 172.21.15.86:0/539551916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08105490 0x7fbd08107880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009900 tx=0x7fbcf802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:51.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.228+0000 7fbd10315640 1 -- 172.21.15.86:0/539551916 shutdown_connections 2024-04-16T05:32:51.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.228+0000 7fbd10315640 1 --2- 172.21.15.86:0/539551916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd08107dc0 0x7fbd0810a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.228+0000 7fbd10315640 1 --2- 172.21.15.86:0/539551916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08105490 0x7fbd08107880 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.228+0000 7fbd10315640 1 --2- 172.21.15.86:0/539551916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd08102b60 0x7fbd08104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.228+0000 7fbd10315640 1 -- 172.21.15.86:0/539551916 >> 172.21.15.86:0/539551916 conn(0x7fbd080fc4f0 msgr2=0x7fbd080fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.228+0000 7fbd10315640 1 -- 172.21.15.86:0/539551916 shutdown_connections 2024-04-16T05:32:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.228+0000 7fbd10315640 1 -- 172.21.15.86:0/539551916 wait complete. 2024-04-16T05:32:51.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd10315640 1 Processor -- start 2024-04-16T05:32:51.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd10315640 1 -- start start 2024-04-16T05:32:51.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd10315640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08102b60 0x7fbd081983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd10315640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd08105490 0x7fbd08198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd10315640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd08107dc0 0x7fbd0819f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd10315640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810dd10 con 0x7fbd08102b60 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd10315640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810db90 con 0x7fbd08105490 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd10315640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810de90 con 0x7fbd08107dc0 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd0e0b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08102b60 0x7fbd081983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd0e0b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08102b60 0x7fbd081983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45954/0 (socket says 172.21.15.86:45954) 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd0e0b1640 1 -- 172.21.15.86:0/3809795516 learned_addr learned my addr 172.21.15.86:0/3809795516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd0d8b0640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd08105490 0x7fbd08198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.229+0000 7fbd0e8b2640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd08107dc0 0x7fbd0819f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbd0e0b1640 1 -- 172.21.15.86:0/3809795516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd08107dc0 msgr2=0x7fbd0819f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbd0e0b1640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd08107dc0 0x7fbd0819f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbd0e0b1640 1 -- 172.21.15.86:0/3809795516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd08105490 msgr2=0x7fbd08198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbd0e0b1640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd08105490 0x7fbd08198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbd0e0b1640 1 -- 172.21.15.86:0/3809795516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0819ffe0 con 0x7fbd08102b60 2024-04-16T05:32:51.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbd0e0b1640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08102b60 0x7fbd081983c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400e9f0 tx=0x7fbd0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbcff7fe640 1 -- 172.21.15.86:0/3809795516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400cd80 con 0x7fbd08102b60 2024-04-16T05:32:51.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd081a02d0 con 0x7fbd08102b60 2024-04-16T05:32:51.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbcff7fe640 1 -- 172.21.15.86:0/3809795516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd04010910 con 0x7fbd08102b60 2024-04-16T05:32:51.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbcff7fe640 1 -- 172.21.15.86:0/3809795516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd040188f0 con 0x7fbd08102b60 2024-04-16T05:32:51.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.230+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd081a0840 con 0x7fbd08102b60 2024-04-16T05:32:51.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.231+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd080fdfe0 con 0x7fbd08102b60 2024-04-16T05:32:51.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.232+0000 7fbcff7fe640 1 -- 172.21.15.86:0/3809795516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd04010ac0 con 0x7fbd08102b60 2024-04-16T05:32:51.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.233+0000 7fbcff7fe640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcf40a3bb0 0x7fbcf40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.233+0000 7fbd0d8b0640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcf40a3bb0 0x7fbcf40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.233+0000 7fbcff7fe640 1 -- 172.21.15.86:0/3809795516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbd040d0730 con 0x7fbd08102b60 2024-04-16T05:32:51.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.234+0000 7fbd0d8b0640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcf40a3bb0 0x7fbcf40a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8005ab0 tx=0x7fbcf8005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.237+0000 7fbcff7fe640 1 -- 172.21.15.86:0/3809795516 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd0409cec0 con 0x7fbd08102b60 2024-04-16T05:32:51.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.483+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd0802d620 con 0x7fbd08102b60 2024-04-16T05:32:51.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.483+0000 7fbcff7fe640 1 -- 172.21.15.86:0/3809795516 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7fbd040a11c0 con 0x7fbd08102b60 2024-04-16T05:32:51.483 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:51.483 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-16T05:32:51.483 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 13 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcf40a3bb0 msgr2=0x7fbcf40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcf40a3bb0 0x7fbcf40a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8005ab0 tx=0x7fbcf8005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08102b60 msgr2=0x7fbd081983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08102b60 0x7fbd081983c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400e9f0 tx=0x7fbd0400eeb0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 shutdown_connections 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd08107dc0 0x7fbd0819f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcf40a3bb0 0x7fbcf40a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd08105490 0x7fbd08198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 --2- 172.21.15.86:0/3809795516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd08102b60 0x7fbd081983c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 >> 172.21.15.86:0/3809795516 conn(0x7fbd080fc4f0 msgr2=0x7fbd081035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 shutdown_connections 2024-04-16T05:32:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.486+0000 7fbd10315640 1 -- 172.21.15.86:0/3809795516 wait complete. 2024-04-16T05:32:51.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:51.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 -- 172.21.15.86:0/416196078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 msgr2=0x7fd0481020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 --2- 172.21.15.86:0/416196078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 0x7fd0481020d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd040009ca0 tx=0x7fd04002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 -- 172.21.15.86:0/416196078 shutdown_connections 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 --2- 172.21.15.86:0/416196078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 0x7fd0481020d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 --2- 172.21.15.86:0/416196078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0481012b0 0x7fd048101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 --2- 172.21.15.86:0/416196078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd048100950 0x7fd048100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 -- 172.21.15.86:0/416196078 >> 172.21.15.86:0/416196078 conn(0x7fd0480fc4f0 msgr2=0x7fd0480fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 -- 172.21.15.86:0/416196078 shutdown_connections 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 -- 172.21.15.86:0/416196078 wait complete. 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 Processor -- start 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.609+0000 7fd04de37640 1 -- start start 2024-04-16T05:32:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd04de37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd048100950 0x7fd04819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd04de37640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0481012b0 0x7fd04819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd04de37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 0x7fd0481a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd04de37640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd04810ff20 con 0x7fd048101c50 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd04de37640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd04810fda0 con 0x7fd0481012b0 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd04de37640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0481100a0 con 0x7fd048100950 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd047fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 0x7fd0481a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd047fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 0x7fd0481a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45970/0 (socket says 172.21.15.86:45970) 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd0477fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd048100950 0x7fd04819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd047fff640 1 -- 172.21.15.86:0/1746014336 learned_addr learned my addr 172.21.15.86:0/1746014336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd03ffff640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0481012b0 0x7fd04819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd047fff640 1 -- 172.21.15.86:0/1746014336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd048100950 msgr2=0x7fd04819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd047fff640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd048100950 0x7fd04819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd047fff640 1 -- 172.21.15.86:0/1746014336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0481012b0 msgr2=0x7fd04819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd047fff640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0481012b0 0x7fd04819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd047fff640 1 -- 172.21.15.86:0/1746014336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0481a2170 con 0x7fd048101c50 2024-04-16T05:32:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd047fff640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 0x7fd0481a1b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd040005cc0 tx=0x7fd040032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd0457fa640 1 -- 172.21.15.86:0/1746014336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040039040 con 0x7fd048101c50 2024-04-16T05:32:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0481a2400 con 0x7fd048101c50 2024-04-16T05:32:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd0457fa640 1 -- 172.21.15.86:0/1746014336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd040044070 con 0x7fd048101c50 2024-04-16T05:32:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd0457fa640 1 -- 172.21.15.86:0/1746014336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0400042a0 con 0x7fd048101c50 2024-04-16T05:32:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.610+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0481a28c0 con 0x7fd048101c50 2024-04-16T05:32:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.612+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04810b840 con 0x7fd048101c50 2024-04-16T05:32:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.613+0000 7fd0457fa640 1 -- 172.21.15.86:0/1746014336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd040003970 con 0x7fd048101c50 2024-04-16T05:32:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.614+0000 7fd0457fa640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0340a3bb0 0x7fd0340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.614+0000 7fd0477fe640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0340a3bb0 0x7fd0340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.614+0000 7fd0457fa640 1 -- 172.21.15.86:0/1746014336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd0400f2a80 con 0x7fd048101c50 2024-04-16T05:32:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.614+0000 7fd0477fe640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0340a3bb0 0x7fd0340a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd0380058c0 tx=0x7fd03800a250 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.619+0000 7fd0457fa640 1 -- 172.21.15.86:0/1746014336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0400bf900 con 0x7fd048101c50 2024-04-16T05:32:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.806+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd048062c00 con 0x7fd048101c50 2024-04-16T05:32:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.807+0000 7fd0457fa640 1 -- 172.21.15.86:0/1746014336 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18408 (secure 0 0 0) 0x7fd0400bf200 con 0x7fd048101c50 2024-04-16T05:32:51.806 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:51.806 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:32:06.931554+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:51.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0340a3bb0 msgr2=0x7fd0340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0340a3bb0 0x7fd0340a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd0380058c0 tx=0x7fd03800a250 comp rx=0 tx=0).stop 2024-04-16T05:32:51.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 msgr2=0x7fd0481a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 0x7fd0481a1b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd040005cc0 tx=0x7fd040032f40 comp rx=0 tx=0).stop 2024-04-16T05:32:51.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 shutdown_connections 2024-04-16T05:32:51.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd048101c50 0x7fd0481a1b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0340a3bb0 0x7fd0340a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0481012b0 0x7fd04819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 --2- 172.21.15.86:0/1746014336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd048100950 0x7fd04819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 >> 172.21.15.86:0/1746014336 conn(0x7fd0480fc4f0 msgr2=0x7fd0480fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 shutdown_connections 2024-04-16T05:32:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.809+0000 7fd04de37640 1 -- 172.21.15.86:0/1746014336 wait complete. 2024-04-16T05:32:51.820 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T05:32:51.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:32:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 -- 172.21.15.86:0/321224821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 msgr2=0x7f87a8101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 --2- 172.21.15.86:0/321224821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 0x7f87a8101360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8798009900 tx=0x7f879802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 -- 172.21.15.86:0/321224821 shutdown_connections 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 --2- 172.21.15.86:0/321224821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8102a50 0x7f87a8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 --2- 172.21.15.86:0/321224821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8102090 0x7f87a8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 --2- 172.21.15.86:0/321224821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 0x7f87a8101360 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 -- 172.21.15.86:0/321224821 >> 172.21.15.86:0/321224821 conn(0x7f87a80fc4b0 msgr2=0x7f87a80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 -- 172.21.15.86:0/321224821 shutdown_connections 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 -- 172.21.15.86:0/321224821 wait complete. 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 Processor -- start 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.933+0000 7f87ae540640 1 -- start start 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87ae540640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 0x7f87a81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87ae540640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8102090 0x7f87a81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87ae540640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8102a50 0x7f87a819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87ae540640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810cc20 con 0x7f87a8100f40 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87ae540640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810caa0 con 0x7f87a8102a50 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87ae540640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810cda0 con 0x7f87a8102090 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 0x7f87a81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 0x7f87a81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45978/0 (socket says 172.21.15.86:45978) 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a7fff640 1 -- 172.21.15.86:0/3940879336 learned_addr learned my addr 172.21.15.86:0/3940879336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f879ffff640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8102090 0x7f87a81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87acadd640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8102a50 0x7f87a819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a7fff640 1 -- 172.21.15.86:0/3940879336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8102090 msgr2=0x7f87a81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a7fff640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8102090 0x7f87a81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a7fff640 1 -- 172.21.15.86:0/3940879336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8102a50 msgr2=0x7f87a819f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a7fff640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8102a50 0x7f87a819f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a7fff640 1 -- 172.21.15.86:0/3940879336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a819ffd0 con 0x7f87a8100f40 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f879ffff640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8102090 0x7f87a81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:32:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a7fff640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 0x7f87a81983b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8798000c00 tx=0x7f8798005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.934+0000 7f87a5ffb640 1 -- 172.21.15.86:0/3940879336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798043020 con 0x7f87a8100f40 2024-04-16T05:32:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.935+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a81a0260 con 0x7f87a8100f40 2024-04-16T05:32:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.935+0000 7f87a5ffb640 1 -- 172.21.15.86:0/3940879336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87980054e0 con 0x7f87a8100f40 2024-04-16T05:32:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.935+0000 7f87a5ffb640 1 -- 172.21.15.86:0/3940879336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798003e60 con 0x7f87a8100f40 2024-04-16T05:32:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.935+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a81a07b0 con 0x7f87a8100f40 2024-04-16T05:32:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.936+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a81a09b0 con 0x7f87a8100f40 2024-04-16T05:32:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.937+0000 7f87a5ffb640 1 -- 172.21.15.86:0/3940879336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8798046020 con 0x7f87a8100f40 2024-04-16T05:32:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.938+0000 7f87a5ffb640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87940a3bb0 0x7f87940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.938+0000 7f879ffff640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87940a3bb0 0x7f87940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.939+0000 7f87a5ffb640 1 -- 172.21.15.86:0/3940879336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f87980f2120 con 0x7f87a8100f40 2024-04-16T05:32:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.939+0000 7f879ffff640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87940a3bb0 0x7f87940a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87a81998c0 tx=0x7f879000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:51.943+0000 7f87a5ffb640 1 -- 172.21.15.86:0/3940879336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f879804d020 con 0x7f87a8100f40 2024-04-16T05:32:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.130+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87a81a0b80 con 0x7f87a8100f40 2024-04-16T05:32:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.130+0000 7f87a5ffb640 1 -- 172.21.15.86:0/3940879336 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f87980befa0 con 0x7f87a8100f40 2024-04-16T05:32:52.129 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:32:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87940a3bb0 msgr2=0x7f87940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87940a3bb0 0x7f87940a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87a81998c0 tx=0x7f879000b390 comp rx=0 tx=0).stop 2024-04-16T05:32:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 msgr2=0x7f87a81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 0x7f87a81983b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8798000c00 tx=0x7f8798005dd0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 shutdown_connections 2024-04-16T05:32:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87940a3bb0 0x7f87940a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8102a50 0x7f87a819f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8102090 0x7f87a81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 --2- 172.21.15.86:0/3940879336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a8100f40 0x7f87a81983b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 >> 172.21.15.86:0/3940879336 conn(0x7f87a80fc4b0 msgr2=0x7f87a80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 shutdown_connections 2024-04-16T05:32:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.132+0000 7f87ae540640 1 -- 172.21.15.86:0/3940879336 wait complete. 2024-04-16T05:32:52.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 -- 172.21.15.86:0/1814501251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 msgr2=0x7f3550102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 --2- 172.21.15.86:0/1814501251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 0x7f3550102510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3544009900 tx=0x7f354402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 -- 172.21.15.86:0/1814501251 shutdown_connections 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 --2- 172.21.15.86:0/1814501251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3550102a50 0x7f3550109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 --2- 172.21.15.86:0/1814501251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 0x7f3550102510 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 --2- 172.21.15.86:0/1814501251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3550100f40 0x7f3550101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 -- 172.21.15.86:0/1814501251 >> 172.21.15.86:0/1814501251 conn(0x7f35500fc4b0 msgr2=0x7f35500fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 -- 172.21.15.86:0/1814501251 shutdown_connections 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 -- 172.21.15.86:0/1814501251 wait complete. 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.248+0000 7f35566c8640 1 Processor -- start 2024-04-16T05:32:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f35566c8640 1 -- start start 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f35566c8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3550100f40 0x7f3550197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f35566c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 0x7f35501977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f35566c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3550102a50 0x7f355019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f35566c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f355010cbf0 con 0x7f3550102090 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f35566c8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f355010ca70 con 0x7f3550100f40 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f35566c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f355010cd70 con 0x7f3550102a50 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f354f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 0x7f35501977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f354f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 0x7f35501977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46004/0 (socket says 172.21.15.86:46004) 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f354f7fe640 1 -- 172.21.15.86:0/1828612240 learned_addr learned my addr 172.21.15.86:0/1828612240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f3554c65640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3550102a50 0x7f355019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f354ffff640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3550100f40 0x7f3550197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f354f7fe640 1 -- 172.21.15.86:0/1828612240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3550102a50 msgr2=0x7f355019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f354f7fe640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3550102a50 0x7f355019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f354f7fe640 1 -- 172.21.15.86:0/1828612240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3550100f40 msgr2=0x7f3550197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f354f7fe640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3550100f40 0x7f3550197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.249+0000 7f354f7fe640 1 -- 172.21.15.86:0/1828612240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355019eeb0 con 0x7f3550102090 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.250+0000 7f354f7fe640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 0x7f35501977d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3544005ab0 tx=0x7f3544005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.250+0000 7f354d7fa640 1 -- 172.21.15.86:0/1828612240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544041070 con 0x7f3550102090 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.250+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355019f140 con 0x7f3550102090 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.250+0000 7f354d7fa640 1 -- 172.21.15.86:0/1828612240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35440054e0 con 0x7f3550102090 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.250+0000 7f354d7fa640 1 -- 172.21.15.86:0/1828612240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544034440 con 0x7f3550102090 2024-04-16T05:32:52.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.250+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355019f680 con 0x7f3550102090 2024-04-16T05:32:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.251+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3550108250 con 0x7f3550102090 2024-04-16T05:32:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.252+0000 7f354d7fa640 1 -- 172.21.15.86:0/1828612240 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3544044070 con 0x7f3550102090 2024-04-16T05:32:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.253+0000 7f354d7fa640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35380a3bb0 0x7f35380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.253+0000 7f354d7fa640 1 -- 172.21.15.86:0/1828612240 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f35440f1ad0 con 0x7f3550102090 2024-04-16T05:32:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.253+0000 7f354ffff640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35380a3bb0 0x7f35380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.253+0000 7f354ffff640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35380a3bb0 0x7f35380a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3540005d00 tx=0x7f354000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:52.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.261+0000 7f354d7fa640 1 -- 172.21.15.86:0/1828612240 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35440be300 con 0x7f3550102090 2024-04-16T05:32:52.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.469+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f355002d7f0 con 0x7f3550102090 2024-04-16T05:32:52.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.470+0000 7f354d7fa640 1 -- 172.21.15.86:0/1828612240 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f35440c2600 con 0x7f3550102090 2024-04-16T05:32:52.468 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:32:52.468 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-16T05:32:52.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:32:52.469 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-16T05:32:52.469 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-16T05:32:52.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:32:52.469 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35380a3bb0 msgr2=0x7f35380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35380a3bb0 0x7f35380a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3540005d00 tx=0x7f354000b410 comp rx=0 tx=0).stop 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 msgr2=0x7f35501977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 0x7f35501977d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3544005ab0 tx=0x7f3544005dc0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 shutdown_connections 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3550102a50 0x7f355019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3550102090 0x7f35501977d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35380a3bb0 0x7f35380a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 --2- 172.21.15.86:0/1828612240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3550100f40 0x7f3550197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 >> 172.21.15.86:0/1828612240 conn(0x7f35500fc4b0 msgr2=0x7f3550105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 shutdown_connections 2024-04-16T05:32:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.472+0000 7f35566c8640 1 -- 172.21.15.86:0/1828612240 wait complete. 2024-04-16T05:32:52.482 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-16T05:32:52.482 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-16T05:32:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.592+0000 7f047f074640 1 -- 172.21.15.86:0/2000227120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478101290 msgr2=0x7f04781016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.592+0000 7f047f074640 1 --2- 172.21.15.86:0/2000227120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478101290 0x7f04781016f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0460009ca0 tx=0x7f046002c180 comp rx=0 tx=0).stop 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.592+0000 7f047f074640 1 -- 172.21.15.86:0/2000227120 shutdown_connections 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.592+0000 7f047f074640 1 --2- 172.21.15.86:0/2000227120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0478101c30 0x7f04781020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.592+0000 7f047f074640 1 --2- 172.21.15.86:0/2000227120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478101290 0x7f04781016f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.592+0000 7f047f074640 1 --2- 172.21.15.86:0/2000227120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478100930 0x7f0478100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.592+0000 7f047f074640 1 -- 172.21.15.86:0/2000227120 >> 172.21.15.86:0/2000227120 conn(0x7f04780fc4b0 msgr2=0x7f04780fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.592+0000 7f047f074640 1 -- 172.21.15.86:0/2000227120 shutdown_connections 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.592+0000 7f047f074640 1 -- 172.21.15.86:0/2000227120 wait complete. 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047f074640 1 Processor -- start 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047f074640 1 -- start start 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047f074640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478100930 0x7f047819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047f074640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478101290 0x7f047819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047f074640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0478101c30 0x7f04781a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047f074640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f047810fed0 con 0x7f0478100930 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047f074640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f047810fd50 con 0x7f0478101c30 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047f074640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0478110050 con 0x7f0478101290 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047ce10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478100930 0x7f047819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047ce10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478100930 0x7f047819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46030/0 (socket says 172.21.15.86:46030) 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047ce10640 1 -- 172.21.15.86:0/389996219 learned_addr learned my addr 172.21.15.86:0/389996219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f0477fff640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478101290 0x7f047819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047d611640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0478101c30 0x7f04781a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047ce10640 1 -- 172.21.15.86:0/389996219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478101290 msgr2=0x7f047819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.593+0000 7f047ce10640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478101290 0x7f047819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.594+0000 7f047ce10640 1 -- 172.21.15.86:0/389996219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0478101c30 msgr2=0x7f04781a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.594+0000 7f047ce10640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0478101c30 0x7f04781a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.594+0000 7f047ce10640 1 -- 172.21.15.86:0/389996219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04781a21e0 con 0x7f0478100930 2024-04-16T05:32:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.594+0000 7f047ce10640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478100930 0x7f047819a5c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f046800d950 tx=0x7f046800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.594+0000 7f0475ffb640 1 -- 172.21.15.86:0/389996219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468017070 con 0x7f0478100930 2024-04-16T05:32:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.594+0000 7f0475ffb640 1 -- 172.21.15.86:0/389996219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0468009c30 con 0x7f0478100930 2024-04-16T05:32:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.594+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04781a24d0 con 0x7f0478100930 2024-04-16T05:32:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.594+0000 7f0475ffb640 1 -- 172.21.15.86:0/389996219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046800f6b0 con 0x7f0478100930 2024-04-16T05:32:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.594+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04781a29f0 con 0x7f0478100930 2024-04-16T05:32:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.595+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047810b910 con 0x7f0478100930 2024-04-16T05:32:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.596+0000 7f0475ffb640 1 -- 172.21.15.86:0/389996219 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f04680109d0 con 0x7f0478100930 2024-04-16T05:32:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.597+0000 7f0475ffb640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04640a3bb0 0x7f04640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.597+0000 7f0477fff640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04640a3bb0 0x7f04640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.597+0000 7f0475ffb640 1 -- 172.21.15.86:0/389996219 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f04680cfba0 con 0x7f0478100930 2024-04-16T05:32:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.597+0000 7f0477fff640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04640a3bb0 0x7f04640a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0460006010 tx=0x7f0460032670 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.601+0000 7f0475ffb640 1 -- 172.21.15.86:0/389996219 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0468009de0 con 0x7f0478100930 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3809795516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:52.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1746014336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:52 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3940879336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3809795516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1746014336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:52 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3940879336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3809795516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:52.767 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:52.768 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:52.768 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1746014336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.768 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.768 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:52 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3940879336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:52.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:52.789+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f047802d620 con 0x7f0478100930 2024-04-16T05:32:53.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.476+0000 7f0475ffb640 1 -- 172.21.15.86:0/389996219 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f046809ca20 con 0x7f0478100930 2024-04-16T05:32:53.475 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-16T05:32:53.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04640a3bb0 msgr2=0x7f04640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04640a3bb0 0x7f04640a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0460006010 tx=0x7f0460032670 comp rx=0 tx=0).stop 2024-04-16T05:32:53.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478100930 msgr2=0x7f047819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478100930 0x7f047819a5c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f046800d950 tx=0x7f046800de10 comp rx=0 tx=0).stop 2024-04-16T05:32:53.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 shutdown_connections 2024-04-16T05:32:53.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04640a3bb0 0x7f04640a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0478101c30 0x7f04781a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478101290 0x7f047819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 --2- 172.21.15.86:0/389996219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478100930 0x7f047819a5c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 >> 172.21.15.86:0/389996219 conn(0x7f04780fc4b0 msgr2=0x7f04780fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:53.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.478+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 shutdown_connections 2024-04-16T05:32:53.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.479+0000 7f047f074640 1 -- 172.21.15.86:0/389996219 wait complete. 2024-04-16T05:32:53.493 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-16T05:32:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.611+0000 7f224424a640 1 -- 172.21.15.86:0/2453591565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 msgr2=0x7f223c107880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.611+0000 7f224424a640 1 --2- 172.21.15.86:0/2453591565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 0x7f223c107880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f222c009900 tx=0x7f222c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.611+0000 7f224424a640 1 -- 172.21.15.86:0/2453591565 shutdown_connections 2024-04-16T05:32:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.611+0000 7f224424a640 1 --2- 172.21.15.86:0/2453591565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f223c107dc0 0x7f223c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.611+0000 7f224424a640 1 --2- 172.21.15.86:0/2453591565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 0x7f223c107880 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.611+0000 7f224424a640 1 --2- 172.21.15.86:0/2453591565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f223c102b60 0x7f223c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.611+0000 7f224424a640 1 -- 172.21.15.86:0/2453591565 >> 172.21.15.86:0/2453591565 conn(0x7f223c0fc4f0 msgr2=0x7f223c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.611+0000 7f224424a640 1 -- 172.21.15.86:0/2453591565 shutdown_connections 2024-04-16T05:32:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.611+0000 7f224424a640 1 -- 172.21.15.86:0/2453591565 wait complete. 2024-04-16T05:32:53.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f224424a640 1 Processor -- start 2024-04-16T05:32:53.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f224424a640 1 -- start start 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f224424a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f223c102b60 0x7f223c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f224424a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 0x7f223c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f224424a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f223c107dc0 0x7f223c19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f224424a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f223c10dcf0 con 0x7f223c105490 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f224424a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f223c10db70 con 0x7f223c107dc0 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f224424a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f223c10de70 con 0x7f223c102b60 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f22417e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 0x7f223c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f22417e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 0x7f223c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46036/0 (socket says 172.21.15.86:46036) 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f22417e5640 1 -- 172.21.15.86:0/3115542633 learned_addr learned my addr 172.21.15.86:0/3115542633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f2241fe6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f223c102b60 0x7f223c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f22427e7640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f223c107dc0 0x7f223c19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f22417e5640 1 -- 172.21.15.86:0/3115542633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f223c102b60 msgr2=0x7f223c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f22417e5640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f223c102b60 0x7f223c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f22417e5640 1 -- 172.21.15.86:0/3115542633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f223c107dc0 msgr2=0x7f223c19f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f22417e5640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f223c107dc0 0x7f223c19f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f22417e5640 1 -- 172.21.15.86:0/3115542633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c1a0020 con 0x7f223c105490 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.612+0000 7f2241fe6640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f223c102b60 0x7f223c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.613+0000 7f22417e5640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 0x7f223c198940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f222c000c00 tx=0x7f222c004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.613+0000 7f2232ffd640 1 -- 172.21.15.86:0/3115542633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c041070 con 0x7f223c105490 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.613+0000 7f2232ffd640 1 -- 172.21.15.86:0/3115542633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c004450 con 0x7f223c105490 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.613+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c1a02b0 con 0x7f223c105490 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.613+0000 7f2232ffd640 1 -- 172.21.15.86:0/3115542633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c003df0 con 0x7f223c105490 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.613+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c1a07a0 con 0x7f223c105490 2024-04-16T05:32:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.614+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f223c0fdfe0 con 0x7f223c105490 2024-04-16T05:32:53.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.616+0000 7f2232ffd640 1 -- 172.21.15.86:0/3115542633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f222c044070 con 0x7f223c105490 2024-04-16T05:32:53.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.617+0000 7f2232ffd640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f22280a3bb0 0x7f22280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:53.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.617+0000 7f2241fe6640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f22280a3bb0 0x7f22280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:53.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.617+0000 7f2232ffd640 1 -- 172.21.15.86:0/3115542633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f222c0f1d10 con 0x7f223c105490 2024-04-16T05:32:53.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.617+0000 7f2241fe6640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f22280a3bb0 0x7f22280a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2238009fc0 tx=0x7f2238005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:53.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.620+0000 7f2232ffd640 1 -- 172.21.15.86:0/3115542633 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f222c0be3d0 con 0x7f223c105490 2024-04-16T05:32:53.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:53 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1828612240' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T05:32:53.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:53 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/389996219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T05:32:53.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:53 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1828612240' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T05:32:53.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:53 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/389996219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T05:32:53.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:53.801+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f223c02d620 con 0x7f223c105490 2024-04-16T05:32:53.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:53 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1828612240' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T05:32:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:53 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/389996219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T05:32:54.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.484+0000 7f2232ffd640 1 -- 172.21.15.86:0/3115542633 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f222c0c26d0 con 0x7f223c105490 2024-04-16T05:32:54.483 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-16T05:32:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f22280a3bb0 msgr2=0x7f22280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f22280a3bb0 0x7f22280a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2238009fc0 tx=0x7f2238005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 msgr2=0x7f223c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 0x7f223c198940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f222c000c00 tx=0x7f222c004960 comp rx=0 tx=0).stop 2024-04-16T05:32:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 shutdown_connections 2024-04-16T05:32:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f22280a3bb0 0x7f22280a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f223c107dc0 0x7f223c19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c105490 0x7f223c198940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 --2- 172.21.15.86:0/3115542633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f223c102b60 0x7f223c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 >> 172.21.15.86:0/3115542633 conn(0x7f223c0fc4f0 msgr2=0x7f223c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:54.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 shutdown_connections 2024-04-16T05:32:54.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.487+0000 7f224424a640 1 -- 172.21.15.86:0/3115542633 wait complete. 2024-04-16T05:32:54.505 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-16T05:32:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 -- 172.21.15.86:0/302715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac103240 msgr2=0x7f12ac1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 --2- 172.21.15.86:0/302715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac103240 0x7f12ac1036c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f129c009900 tx=0x7f129c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 -- 172.21.15.86:0/302715 shutdown_connections 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 --2- 172.21.15.86:0/302715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ac103c00 0x7f12ac10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 --2- 172.21.15.86:0/302715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac103240 0x7f12ac1036c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 --2- 172.21.15.86:0/302715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ac102050 0x7f12ac102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 -- 172.21.15.86:0/302715 >> 172.21.15.86:0/302715 conn(0x7f12ac0fd5e0 msgr2=0x7f12ac0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 -- 172.21.15.86:0/302715 shutdown_connections 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 -- 172.21.15.86:0/302715 wait complete. 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 Processor -- start 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 -- start start 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac102050 0x7f12ac198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ac103240 0x7f12ac198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ac103c00 0x7f12ac19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac10dd90 con 0x7f12ac102050 2024-04-16T05:32:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac10dc10 con 0x7f12ac103c00 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.617+0000 7f12b45e9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac10df10 con 0x7f12ac103240 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b1b84640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ac103240 0x7f12ac198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b2385640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac102050 0x7f12ac198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b1b84640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ac103240 0x7f12ac198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:40796/0 (socket says 172.21.15.86:40796) 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b2385640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac102050 0x7f12ac198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46056/0 (socket says 172.21.15.86:46056) 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b1b84640 1 -- 172.21.15.86:0/568353817 learned_addr learned my addr 172.21.15.86:0/568353817 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b2b86640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ac103c00 0x7f12ac19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b1b84640 1 -- 172.21.15.86:0/568353817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ac103c00 msgr2=0x7f12ac19f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b1b84640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ac103c00 0x7f12ac19f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b1b84640 1 -- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac102050 msgr2=0x7f12ac198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b1b84640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac102050 0x7f12ac198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b1b84640 1 -- 172.21.15.86:0/568353817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac1a0020 con 0x7f12ac103240 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b2385640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac102050 0x7f12ac198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:32:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b1b84640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ac103240 0x7f12ac198940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f129c000c00 tx=0x7f129c005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:54.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12a37fe640 1 -- 172.21.15.86:0/568353817 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c043020 con 0x7f12ac103240 2024-04-16T05:32:54.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac1a02b0 con 0x7f12ac103240 2024-04-16T05:32:54.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12a37fe640 1 -- 172.21.15.86:0/568353817 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129c0054e0 con 0x7f12ac103240 2024-04-16T05:32:54.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12a37fe640 1 -- 172.21.15.86:0/568353817 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c003df0 con 0x7f12ac103240 2024-04-16T05:32:54.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.618+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac1a0800 con 0x7f12ac103240 2024-04-16T05:32:54.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.620+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac109370 con 0x7f12ac103240 2024-04-16T05:32:54.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.621+0000 7f12a37fe640 1 -- 172.21.15.86:0/568353817 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f129c046020 con 0x7f12ac103240 2024-04-16T05:32:54.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.621+0000 7f12a37fe640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12980a3bb0 0x7f12980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:54.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.621+0000 7f12b2385640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12980a3bb0 0x7f12980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:54.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.621+0000 7f12a37fe640 1 -- 172.21.15.86:0/568353817 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f129c0f2510 con 0x7f12ac103240 2024-04-16T05:32:54.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.622+0000 7f12b2385640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12980a3bb0 0x7f12980a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f12ac1030a0 tx=0x7f12a8009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:54.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.626+0000 7f12a37fe640 1 -- 172.21.15.86:0/568353817 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f129c04d020 con 0x7f12ac103240 2024-04-16T05:32:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/389996219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T05:32:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T05:32:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:54.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:54.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:54 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3115542633' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T05:32:54.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:54.851+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f12ac062c00 con 0x7f12ac103240 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/389996219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:54 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3115542633' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T05:32:54.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/389996219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:32:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:54 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3115542633' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T05:32:55.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.846+0000 7f12a37fe640 1 -- 172.21.15.86:0/568353817 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f129c0bf0a0 con 0x7f12ac103240 2024-04-16T05:32:55.845 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-04-16T05:32:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12980a3bb0 msgr2=0x7f12980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12980a3bb0 0x7f12980a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f12ac1030a0 tx=0x7f12a8009110 comp rx=0 tx=0).stop 2024-04-16T05:32:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ac103240 msgr2=0x7f12ac198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ac103240 0x7f12ac198940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f129c000c00 tx=0x7f129c005dd0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 shutdown_connections 2024-04-16T05:32:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12980a3bb0 0x7f12980a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ac103c00 0x7f12ac19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ac103240 0x7f12ac198940 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 --2- 172.21.15.86:0/568353817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ac102050 0x7f12ac198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 >> 172.21.15.86:0/568353817 conn(0x7f12ac0fd5e0 msgr2=0x7f12ac106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 shutdown_connections 2024-04-16T05:32:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.849+0000 7f12b45e9640 1 -- 172.21.15.86:0/568353817 wait complete. 2024-04-16T05:32:55.863 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:55 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3115542633' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T05:32:55.863 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:55 smithi086 ceph-mon[32028]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T05:32:55.863 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:55 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/568353817' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T05:32:55.863 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:55 smithi086 ceph-mon[32028]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T05:32:55.863 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:55 smithi086 ceph-mon[32028]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T05:32:55.864 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-16T05:32:55.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-16T05:32:55.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:55 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3115542633' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T05:32:55.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:55 smithi194 ceph-mon[35865]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T05:32:55.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:55 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/568353817' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T05:32:55.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:55 smithi194 ceph-mon[35865]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T05:32:55.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:55 smithi194 ceph-mon[35865]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.981+0000 7f989bdba640 1 -- 172.21.15.86:0/393445908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 msgr2=0x7f98941036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.981+0000 7f989bdba640 1 --2- 172.21.15.86:0/393445908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 0x7f98941036c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9884009900 tx=0x7f988402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.981+0000 7f989bdba640 1 -- 172.21.15.86:0/393445908 shutdown_connections 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.981+0000 7f989bdba640 1 --2- 172.21.15.86:0/393445908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9894103c00 0x7f989410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.981+0000 7f989bdba640 1 --2- 172.21.15.86:0/393445908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 0x7f98941036c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.981+0000 7f989bdba640 1 --2- 172.21.15.86:0/393445908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9894102050 0x7f9894102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.981+0000 7f989bdba640 1 -- 172.21.15.86:0/393445908 >> 172.21.15.86:0/393445908 conn(0x7f98940fd5e0 msgr2=0x7f98940ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.981+0000 7f989bdba640 1 -- 172.21.15.86:0/393445908 shutdown_connections 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.981+0000 7f989bdba640 1 -- 172.21.15.86:0/393445908 wait complete. 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f989bdba640 1 Processor -- start 2024-04-16T05:32:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f989bdba640 1 -- start start 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f989bdba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9894102050 0x7f9894198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f989bdba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 0x7f98941988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f989bdba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9894103c00 0x7f989419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f989bdba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f989410dce0 con 0x7f9894103240 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f989bdba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f989410db60 con 0x7f9894103c00 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f989bdba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f989410de60 con 0x7f9894102050 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899355640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 0x7f98941988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899355640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 0x7f98941988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46072/0 (socket says 172.21.15.86:46072) 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899355640 1 -- 172.21.15.86:0/3847045475 learned_addr learned my addr 172.21.15.86:0/3847045475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899b56640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9894102050 0x7f9894198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f989a357640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9894103c00 0x7f989419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899355640 1 -- 172.21.15.86:0/3847045475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9894102050 msgr2=0x7f9894198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899355640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9894102050 0x7f9894198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899355640 1 -- 172.21.15.86:0/3847045475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9894103c00 msgr2=0x7f989419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899355640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9894103c00 0x7f989419f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899355640 1 -- 172.21.15.86:0/3847045475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989419ff90 con 0x7f9894103240 2024-04-16T05:32:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f9899355640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 0x7f98941988b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9884009230 tx=0x7f9884005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.982+0000 7f988affd640 1 -- 172.21.15.86:0/3847045475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884041070 con 0x7f9894103240 2024-04-16T05:32:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.983+0000 7f988affd640 1 -- 172.21.15.86:0/3847045475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98840054e0 con 0x7f9894103240 2024-04-16T05:32:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.983+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98941a0220 con 0x7f9894103240 2024-04-16T05:32:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.983+0000 7f988affd640 1 -- 172.21.15.86:0/3847045475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884034440 con 0x7f9894103240 2024-04-16T05:32:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.983+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98941a0710 con 0x7f9894103240 2024-04-16T05:32:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.983+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9894109370 con 0x7f9894103240 2024-04-16T05:32:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.985+0000 7f988affd640 1 -- 172.21.15.86:0/3847045475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9884044070 con 0x7f9894103240 2024-04-16T05:32:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.985+0000 7f988affd640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f98800a3bb0 0x7f98800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.985+0000 7f9899b56640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f98800a3bb0 0x7f98800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.985+0000 7f988affd640 1 -- 172.21.15.86:0/3847045475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f98840f2380 con 0x7f9894103240 2024-04-16T05:32:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.986+0000 7f9899b56640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f98800a3bb0 0x7f98800a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98941030a0 tx=0x7f989000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:55.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:55 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3115542633' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T05:32:55.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:55 smithi196 ceph-mon[36373]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T05:32:55.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:55 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/568353817' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T05:32:55.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:55 smithi196 ceph-mon[36373]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T05:32:55.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:55 smithi196 ceph-mon[36373]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T05:32:55.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:55.989+0000 7f988affd640 1 -- 172.21.15.86:0/3847045475 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98840beeb0 con 0x7f9894103240 2024-04-16T05:32:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.175+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f989402d870 con 0x7f9894103240 2024-04-16T05:32:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.494+0000 7f988affd640 1 -- 172.21.15.86:0/3847045475 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f98840be7b0 con 0x7f9894103240 2024-04-16T05:32:56.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f98800a3bb0 msgr2=0x7f98800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f98800a3bb0 0x7f98800a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98941030a0 tx=0x7f989000b390 comp rx=0 tx=0).stop 2024-04-16T05:32:56.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 msgr2=0x7f98941988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 0x7f98941988b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9884009230 tx=0x7f9884005dd0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 shutdown_connections 2024-04-16T05:32:56.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f98800a3bb0 0x7f98800a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9894103c00 0x7f989419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894103240 0x7f98941988b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 --2- 172.21.15.86:0/3847045475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9894102050 0x7f9894198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 >> 172.21.15.86:0/3847045475 conn(0x7f98940fd5e0 msgr2=0x7f9894106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 shutdown_connections 2024-04-16T05:32:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.497+0000 7f989bdba640 1 -- 172.21.15.86:0/3847045475 wait complete. 2024-04-16T05:32:56.507 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-16T05:32:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.622+0000 7f68c0972640 1 -- 172.21.15.86:0/2244130362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc102a50 msgr2=0x7f68bc109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.622+0000 7f68c0972640 1 --2- 172.21.15.86:0/2244130362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc102a50 0x7f68bc109370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009ca0 tx=0x7f68b402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.622+0000 7f68c0972640 1 -- 172.21.15.86:0/2244130362 shutdown_connections 2024-04-16T05:32:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.622+0000 7f68c0972640 1 --2- 172.21.15.86:0/2244130362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc102a50 0x7f68bc109370 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.622+0000 7f68c0972640 1 --2- 172.21.15.86:0/2244130362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68bc102090 0x7f68bc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.622+0000 7f68c0972640 1 --2- 172.21.15.86:0/2244130362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc100f40 0x7f68bc101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.622+0000 7f68c0972640 1 -- 172.21.15.86:0/2244130362 >> 172.21.15.86:0/2244130362 conn(0x7f68bc0fc4b0 msgr2=0x7f68bc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.622+0000 7f68c0972640 1 -- 172.21.15.86:0/2244130362 shutdown_connections 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.622+0000 7f68c0972640 1 -- 172.21.15.86:0/2244130362 wait complete. 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68c0972640 1 Processor -- start 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68c0972640 1 -- start start 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68c0972640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc100f40 0x7f68bc069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68c0972640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68bc102090 0x7f68bc19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68c0972640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc102a50 0x7f68bc19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68c0972640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc10cbf0 con 0x7f68bc100f40 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68c0972640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc10ca70 con 0x7f68bc102090 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68c0972640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc10cd70 con 0x7f68bc102a50 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ba59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc100f40 0x7f68bc069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ba59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc100f40 0x7f68bc069d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46086/0 (socket says 172.21.15.86:46086) 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ba59c640 1 -- 172.21.15.86:0/342314667 learned_addr learned my addr 172.21.15.86:0/342314667 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ba59c640 1 -- 172.21.15.86:0/342314667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc102a50 msgr2=0x7f68bc19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ba59c640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc102a50 0x7f68bc19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ba59c640 1 -- 172.21.15.86:0/342314667 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68bc102090 msgr2=0x7f68bc19d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ba59c640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68bc102090 0x7f68bc19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ba59c640 1 -- 172.21.15.86:0/342314667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc19ff60 con 0x7f68bc100f40 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ba59c640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc100f40 0x7f68bc069d00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d770 tx=0x7f68ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ab7fe640 1 -- 172.21.15.86:0/342314667 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac017070 con 0x7f68bc100f40 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ab7fe640 1 -- 172.21.15.86:0/342314667 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac005500 con 0x7f68bc100f40 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68ab7fe640 1 -- 172.21.15.86:0/342314667 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac004e00 con 0x7f68bc100f40 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.623+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc1a01f0 con 0x7f68bc100f40 2024-04-16T05:32:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.624+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc1a0740 con 0x7f68bc100f40 2024-04-16T05:32:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.625+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc1081f0 con 0x7f68bc100f40 2024-04-16T05:32:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.626+0000 7f68ab7fe640 1 -- 172.21.15.86:0/342314667 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f68ac004fa0 con 0x7f68bc100f40 2024-04-16T05:32:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.627+0000 7f68ab7fe640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f68a40a3c30 0x7f68a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.627+0000 7f68b9d9b640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f68a40a3c30 0x7f68a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.627+0000 7f68ab7fe640 1 -- 172.21.15.86:0/342314667 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f68ac0d0390 con 0x7f68bc100f40 2024-04-16T05:32:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.627+0000 7f68b9d9b640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f68a40a3c30 0x7f68a40a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f68b0005fd0 tx=0x7f68b000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.631+0000 7f68ab7fe640 1 -- 172.21.15.86:0/342314667 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68ac09cd60 con 0x7f68bc100f40 2024-04-16T05:32:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:56.807+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f68bc062ca0 con 0x7f68bc100f40 2024-04-16T05:32:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:56 smithi194 ceph-mon[35865]: pgmap v285: 97 pgs: 23 creating+peering, 12 active+clean, 62 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:56 smithi194 ceph-mon[35865]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T05:32:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:56 smithi194 ceph-mon[35865]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:32:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:56 smithi194 ceph-mon[35865]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T05:32:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:56 smithi194 ceph-mon[35865]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T05:32:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:56 smithi194 ceph-mon[35865]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T05:32:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:56 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3847045475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T05:32:56.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:56 smithi086 ceph-mon[32028]: pgmap v285: 97 pgs: 23 creating+peering, 12 active+clean, 62 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:56.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:56 smithi086 ceph-mon[32028]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T05:32:56.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:56 smithi086 ceph-mon[32028]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:32:56.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:56 smithi086 ceph-mon[32028]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T05:32:56.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:56 smithi086 ceph-mon[32028]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T05:32:56.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:56 smithi086 ceph-mon[32028]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T05:32:56.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:56 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3847045475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T05:32:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:56 smithi196 ceph-mon[36373]: pgmap v285: 97 pgs: 23 creating+peering, 12 active+clean, 62 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:56 smithi196 ceph-mon[36373]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T05:32:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:56 smithi196 ceph-mon[36373]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:32:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:56 smithi196 ceph-mon[36373]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T05:32:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:56 smithi196 ceph-mon[36373]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T05:32:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:56 smithi196 ceph-mon[36373]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T05:32:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:56 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3847045475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T05:32:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:57 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3847045475' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T05:32:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:57 smithi194 ceph-mon[35865]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T05:32:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:57 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/342314667' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T05:32:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:57 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] up:active 2024-04-16T05:32:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:57 smithi194 ceph-mon[35865]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:57 smithi194 ceph-mon[35865]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:57 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3847045475' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T05:32:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:57 smithi086 ceph-mon[32028]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T05:32:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:57 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/342314667' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T05:32:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:57 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] up:active 2024-04-16T05:32:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:57 smithi086 ceph-mon[32028]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:57.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:57 smithi086 ceph-mon[32028]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:57 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3847045475' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T05:32:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:57 smithi196 ceph-mon[36373]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T05:32:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:57 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/342314667' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T05:32:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:57 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] up:active 2024-04-16T05:32:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:57 smithi196 ceph-mon[36373]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:57.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:57 smithi196 ceph-mon[36373]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:58.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.498+0000 7f68ab7fe640 1 -- 172.21.15.86:0/342314667 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f68ac09c660 con 0x7f68bc100f40 2024-04-16T05:32:58.497 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' created 2024-04-16T05:32:58.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f68a40a3c30 msgr2=0x7f68a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f68a40a3c30 0x7f68a40a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f68b0005fd0 tx=0x7f68b000b410 comp rx=0 tx=0).stop 2024-04-16T05:32:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc100f40 msgr2=0x7f68bc069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc100f40 0x7f68bc069d00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d770 tx=0x7f68ac00dc30 comp rx=0 tx=0).stop 2024-04-16T05:32:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 shutdown_connections 2024-04-16T05:32:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc102a50 0x7f68bc19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f68a40a3c30 0x7f68a40a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68bc102090 0x7f68bc19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 --2- 172.21.15.86:0/342314667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68bc100f40 0x7f68bc069d00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 >> 172.21.15.86:0/342314667 conn(0x7f68bc0fc4b0 msgr2=0x7f68bc1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.501+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 shutdown_connections 2024-04-16T05:32:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.502+0000 7f68c0972640 1 -- 172.21.15.86:0/342314667 wait complete. 2024-04-16T05:32:58.520 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-16T05:32:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.634+0000 7fd50b59d640 1 -- 172.21.15.86:0/2809981359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 msgr2=0x7fd504109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.634+0000 7fd50b59d640 1 --2- 172.21.15.86:0/2809981359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 0x7fd504109f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009ca0 tx=0x7fd4fc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.634+0000 7fd50b59d640 1 -- 172.21.15.86:0/2809981359 shutdown_connections 2024-04-16T05:32:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.634+0000 7fd50b59d640 1 --2- 172.21.15.86:0/2809981359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 0x7fd504109f70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.634+0000 7fd50b59d640 1 --2- 172.21.15.86:0/2809981359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041051f0 0x7fd5041075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.634+0000 7fd50b59d640 1 --2- 172.21.15.86:0/2809981359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5041028c0 0x7fd504104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.634+0000 7fd50b59d640 1 -- 172.21.15.86:0/2809981359 >> 172.21.15.86:0/2809981359 conn(0x7fd5040fc2f0 msgr2=0x7fd5040fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.634+0000 7fd50b59d640 1 -- 172.21.15.86:0/2809981359 shutdown_connections 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.634+0000 7fd50b59d640 1 -- 172.21.15.86:0/2809981359 wait complete. 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50b59d640 1 Processor -- start 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50b59d640 1 -- start start 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50b59d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5041028c0 0x7fd504198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50b59d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041051f0 0x7fd504198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50b59d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 0x7fd50419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50b59d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410db30 con 0x7fd504107b20 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50b59d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410d9b0 con 0x7fd5041028c0 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50b59d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410dcb0 con 0x7fd5041051f0 2024-04-16T05:32:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50ad9c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 0x7fd50419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50ad9c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 0x7fd50419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46108/0 (socket says 172.21.15.86:46108) 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50ad9c640 1 -- 172.21.15.86:0/1230892765 learned_addr learned my addr 172.21.15.86:0/1230892765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50a59b640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5041028c0 0x7fd504198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd509d9a640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041051f0 0x7fd504198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50ad9c640 1 -- 172.21.15.86:0/1230892765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041051f0 msgr2=0x7fd504198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50ad9c640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041051f0 0x7fd504198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50ad9c640 1 -- 172.21.15.86:0/1230892765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5041028c0 msgr2=0x7fd504198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50ad9c640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5041028c0 0x7fd504198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50ad9c640 1 -- 172.21.15.86:0/1230892765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50419fd60 con 0x7fd504107b20 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50a59b640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5041028c0 0x7fd504198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd50ad9c640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 0x7fd50419f6f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc037040 tx=0x7fd4fc032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.635+0000 7fd4fb7fe640 1 -- 172.21.15.86:0/1230892765 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc039040 con 0x7fd504107b20 2024-04-16T05:32:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.636+0000 7fd4fb7fe640 1 -- 172.21.15.86:0/1230892765 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4fc044070 con 0x7fd504107b20 2024-04-16T05:32:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.636+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50419fff0 con 0x7fd504107b20 2024-04-16T05:32:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.636+0000 7fd4fb7fe640 1 -- 172.21.15.86:0/1230892765 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc004330 con 0x7fd504107b20 2024-04-16T05:32:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.636+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5041a0450 con 0x7fd504107b20 2024-04-16T05:32:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.637+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5040fdde0 con 0x7fd504107b20 2024-04-16T05:32:58.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.638+0000 7fd4fb7fe640 1 -- 172.21.15.86:0/1230892765 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd4fc003a00 con 0x7fd504107b20 2024-04-16T05:32:58.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.638+0000 7fd4fb7fe640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd4f00a3bb0 0x7fd4f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.638+0000 7fd50a59b640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd4f00a3bb0 0x7fd4f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.638+0000 7fd4fb7fe640 1 -- 172.21.15.86:0/1230892765 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7fd4fc0f3510 con 0x7fd504107b20 2024-04-16T05:32:58.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.639+0000 7fd50a59b640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd4f00a3bb0 0x7fd4f00a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40058c0 tx=0x7fd4f4005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:58.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.642+0000 7fd4fb7fe640 1 -- 172.21.15.86:0/1230892765 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4fc0bfcd0 con 0x7fd504107b20 2024-04-16T05:32:58.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:58.820+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fd50402d620 con 0x7fd504107b20 2024-04-16T05:32:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:58 smithi194 ceph-mon[35865]: pgmap v288: 97 pgs: 23 creating+peering, 64 active+clean, 10 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 6 op/s 2024-04-16T05:32:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:58 smithi194 ceph-mon[35865]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T05:32:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:58 smithi194 ceph-mon[35865]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-16T05:32:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:58 smithi194 ceph-mon[35865]: Cluster is now healthy 2024-04-16T05:32:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:58 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/342314667' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T05:32:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:58 smithi194 ceph-mon[35865]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T05:32:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:58 smithi086 ceph-mon[32028]: pgmap v288: 97 pgs: 23 creating+peering, 64 active+clean, 10 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 6 op/s 2024-04-16T05:32:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:58 smithi086 ceph-mon[32028]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T05:32:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:58 smithi086 ceph-mon[32028]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-16T05:32:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:58 smithi086 ceph-mon[32028]: Cluster is now healthy 2024-04-16T05:32:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:58 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/342314667' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T05:32:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:58 smithi086 ceph-mon[32028]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T05:32:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:58 smithi196 ceph-mon[36373]: pgmap v288: 97 pgs: 23 creating+peering, 64 active+clean, 10 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 6 op/s 2024-04-16T05:32:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:58 smithi196 ceph-mon[36373]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T05:32:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:58 smithi196 ceph-mon[36373]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-16T05:32:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:58 smithi196 ceph-mon[36373]: Cluster is now healthy 2024-04-16T05:32:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:58 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/342314667' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T05:32:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:58 smithi196 ceph-mon[36373]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T05:32:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.500+0000 7fd4fb7fe640 1 -- 172.21.15.86:0/1230892765 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7fd4fc0bf5d0 con 0x7fd504107b20 2024-04-16T05:32:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:set pool 4 allow_ec_overwrites to true 2024-04-16T05:32:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd4f00a3bb0 msgr2=0x7fd4f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd4f00a3bb0 0x7fd4f00a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40058c0 tx=0x7fd4f4005ec0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 msgr2=0x7fd50419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 0x7fd50419f6f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc037040 tx=0x7fd4fc032f90 comp rx=0 tx=0).stop 2024-04-16T05:32:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 shutdown_connections 2024-04-16T05:32:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd504107b20 0x7fd50419f6f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041051f0 0x7fd504198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd4f00a3bb0 0x7fd4f00a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 --2- 172.21.15.86:0/1230892765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5041028c0 0x7fd504198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 >> 172.21.15.86:0/1230892765 conn(0x7fd5040fc2f0 msgr2=0x7fd504103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 shutdown_connections 2024-04-16T05:32:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.503+0000 7fd50b59d640 1 -- 172.21.15.86:0/1230892765 wait complete. 2024-04-16T05:32:59.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-16T05:32:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/1457294863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 msgr2=0x7fd0f4101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/1457294863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 0x7fd0f4101350 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009900 tx=0x7fd0e802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/1457294863 shutdown_connections 2024-04-16T05:32:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/1457294863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f4102aa0 0x7fd0f41093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/1457294863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0f41020e0 0x7fd0f4102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/1457294863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 0x7fd0f4101350 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/1457294863 >> 172.21.15.86:0/1457294863 conn(0x7fd0f40fc4f0 msgr2=0x7fd0f40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/1457294863 shutdown_connections 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/1457294863 wait complete. 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.633+0000 7fd0fc0eb640 1 Processor -- start 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fc0eb640 1 -- start start 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fc0eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 0x7fd0f410b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fc0eb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0f41020e0 0x7fd0f4109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fc0eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f4102aa0 0x7fd0f410a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fc0eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f410dd40 con 0x7fd0f4100f10 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fc0eb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f410dbc0 con 0x7fd0f41020e0 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fc0eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f410dec0 con 0x7fd0f4102aa0 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9e87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 0x7fd0f410b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9e87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 0x7fd0f410b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46124/0 (socket says 172.21.15.86:46124) 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9e87640 1 -- 172.21.15.86:0/2579223534 learned_addr learned my addr 172.21.15.86:0/2579223534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:32:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9686640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0f41020e0 0x7fd0f4109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fa688640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f4102aa0 0x7fd0f410a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9e87640 1 -- 172.21.15.86:0/2579223534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f4102aa0 msgr2=0x7fd0f410a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9e87640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f4102aa0 0x7fd0f410a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9e87640 1 -- 172.21.15.86:0/2579223534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0f41020e0 msgr2=0x7fd0f4109ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9e87640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0f41020e0 0x7fd0f4109ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9e87640 1 -- 172.21.15.86:0/2579223534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f410acc0 con 0x7fd0f4100f10 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0f9e87640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 0x7fd0f410b8b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8000c00 tx=0x7fd0e8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0e6ffd640 1 -- 172.21.15.86:0/2579223534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8041070 con 0x7fd0f4100f10 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f41a2440 con 0x7fd0f4100f10 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0e6ffd640 1 -- 172.21.15.86:0/2579223534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0e80054e0 con 0x7fd0f4100f10 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0e6ffd640 1 -- 172.21.15.86:0/2579223534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80346a0 con 0x7fd0f4100f10 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.634+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f41a29e0 con 0x7fd0f4100f10 2024-04-16T05:32:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.635+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f41082f0 con 0x7fd0f4100f10 2024-04-16T05:32:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.636+0000 7fd0e6ffd640 1 -- 172.21.15.86:0/2579223534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd0e8044070 con 0x7fd0f4100f10 2024-04-16T05:32:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.637+0000 7fd0e6ffd640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0e00a3bb0 0x7fd0e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.637+0000 7fd0f9686640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0e00a3bb0 0x7fd0e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.637+0000 7fd0e6ffd640 1 -- 172.21.15.86:0/2579223534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fd0e80f2460 con 0x7fd0f4100f10 2024-04-16T05:32:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.637+0000 7fd0f9686640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0e00a3bb0 0x7fd0e00a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005e10 tx=0x7fd0f0005da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.642+0000 7fd0e6ffd640 1 -- 172.21.15.86:0/2579223534 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0e80bec20 con 0x7fd0f4100f10 2024-04-16T05:32:59.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:32:59.859+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fd0f402d620 con 0x7fd0f4100f10 2024-04-16T05:32:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:59 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1230892765' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T05:32:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:59 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1230892765' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T05:32:59.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:32:59 smithi194 ceph-mon[35865]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T05:32:59.906 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:32:59 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b[66970]: 2024-04-16T05:32:59.641+0000 7fa7479cd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-16T05:32:59.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:59 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1230892765' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T05:32:59.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:59 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1230892765' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T05:32:59.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:32:59 smithi086 ceph-mon[32028]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T05:32:59.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:59 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1230892765' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T05:32:59.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:59 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1230892765' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T05:32:59.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:32:59 smithi196 ceph-mon[36373]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T05:33:00.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.841+0000 7fd0e6ffd640 1 -- 172.21.15.86:0/2579223534 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fd0e80be520 con 0x7fd0f4100f10 2024-04-16T05:33:00.839 INFO:teuthology.orchestra.run.smithi086.stderr:added data pool 4 to fsmap 2024-04-16T05:33:00.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0e00a3bb0 msgr2=0x7fd0e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0e00a3bb0 0x7fd0e00a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005e10 tx=0x7fd0f0005da0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 msgr2=0x7fd0f410b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 0x7fd0f410b8b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8000c00 tx=0x7fd0e8004040 comp rx=0 tx=0).stop 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 shutdown_connections 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f4102aa0 0x7fd0f410a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd0e00a3bb0 0x7fd0e00a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0f41020e0 0x7fd0f4109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 --2- 172.21.15.86:0/2579223534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f4100f10 0x7fd0f410b8b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.843+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 >> 172.21.15.86:0/2579223534 conn(0x7fd0f40fc4f0 msgr2=0x7fd0f41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.844+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 shutdown_connections 2024-04-16T05:33:00.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.844+0000 7fd0fc0eb640 1 -- 172.21.15.86:0/2579223534 wait complete. 2024-04-16T05:33:00.854 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:00 smithi086 ceph-mon[32028]: pgmap v291: 129 pgs: 24 creating+peering, 77 active+clean, 28 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 7 op/s 2024-04-16T05:33:00.854 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:00 smithi086 ceph-mon[32028]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:33:00.854 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:00 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2579223534' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T05:33:00.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:33:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:00 smithi194 ceph-mon[35865]: pgmap v291: 129 pgs: 24 creating+peering, 77 active+clean, 28 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 7 op/s 2024-04-16T05:33:00.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:00 smithi194 ceph-mon[35865]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:33:00.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:00 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2579223534' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T05:33:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.964+0000 7fb59d688640 1 -- 172.21.15.86:0/2629628601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 msgr2=0x7fb598102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.964+0000 7fb59d688640 1 --2- 172.21.15.86:0/2629628601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 0x7fb598102510 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb580009900 tx=0x7fb58002bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.964+0000 7fb59d688640 1 -- 172.21.15.86:0/2629628601 shutdown_connections 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 --2- 172.21.15.86:0/2629628601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb598102a50 0x7fb598109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 --2- 172.21.15.86:0/2629628601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 0x7fb598102510 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 --2- 172.21.15.86:0/2629628601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb598100f40 0x7fb598101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 -- 172.21.15.86:0/2629628601 >> 172.21.15.86:0/2629628601 conn(0x7fb5980fc4b0 msgr2=0x7fb5980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 -- 172.21.15.86:0/2629628601 shutdown_connections 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 -- 172.21.15.86:0/2629628601 wait complete. 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 Processor -- start 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 -- start start 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb598100f40 0x7fb598198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 0x7fb598198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb598102a50 0x7fb59819fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb59810cc60 con 0x7fb598102090 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb59810cae0 con 0x7fb598100f40 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.965+0000 7fb59d688640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb59810cde0 con 0x7fb598102a50 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb58e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 0x7fb598198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb58e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 0x7fb598198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56786/0 (socket says 172.21.15.86:56786) 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb58e7fc640 1 -- 172.21.15.86:0/3818757517 learned_addr learned my addr 172.21.15.86:0/3818757517 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb5977fe640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb598102a50 0x7fb59819fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb596ffd640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb598100f40 0x7fb598198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb58e7fc640 1 -- 172.21.15.86:0/3818757517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb598102a50 msgr2=0x7fb59819fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb58e7fc640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb598102a50 0x7fb59819fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb58e7fc640 1 -- 172.21.15.86:0/3818757517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb598100f40 msgr2=0x7fb598198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:00.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb58e7fc640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb598100f40 0x7fb598198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:00.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb58e7fc640 1 -- 172.21.15.86:0/3818757517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5981a0070 con 0x7fb598102090 2024-04-16T05:33:00.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb5977fe640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb598102a50 0x7fb59819fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:00.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb596ffd640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb598100f40 0x7fb598198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:00.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb58e7fc640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 0x7fb598198990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb580000c00 tx=0x7fb580004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:00.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb594ff9640 1 -- 172.21.15.86:0/3818757517 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580041070 con 0x7fb598102090 2024-04-16T05:33:00.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5981a0300 con 0x7fb598102090 2024-04-16T05:33:00.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb594ff9640 1 -- 172.21.15.86:0/3818757517 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5800054e0 con 0x7fb598102090 2024-04-16T05:33:00.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb594ff9640 1 -- 172.21.15.86:0/3818757517 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800346a0 con 0x7fb598102090 2024-04-16T05:33:00.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.966+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5981a07f0 con 0x7fb598102090 2024-04-16T05:33:00.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.967+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5981a0a50 con 0x7fb598102090 2024-04-16T05:33:00.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.969+0000 7fb594ff9640 1 -- 172.21.15.86:0/3818757517 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb580044070 con 0x7fb598102090 2024-04-16T05:33:00.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.969+0000 7fb594ff9640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5840a3bb0 0x7fb5840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:00.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.969+0000 7fb596ffd640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5840a3bb0 0x7fb5840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:00.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.969+0000 7fb594ff9640 1 -- 172.21.15.86:0/3818757517 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb5800f2480 con 0x7fb598102090 2024-04-16T05:33:00.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.970+0000 7fb596ffd640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5840a3bb0 0x7fb5840a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5880058c0 tx=0x7fb588005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:00.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:00.974+0000 7fb594ff9640 1 -- 172.21.15.86:0/3818757517 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5800bec20 con 0x7fb598102090 2024-04-16T05:33:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:00 smithi196 ceph-mon[36373]: pgmap v291: 129 pgs: 24 creating+peering, 77 active+clean, 28 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 7 op/s 2024-04-16T05:33:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:00 smithi196 ceph-mon[36373]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:33:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:00 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2579223534' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T05:33:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.187+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5981a0de0 con 0x7fb598102090 2024-04-16T05:33:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.187+0000 7fb594ff9640 1 -- 172.21.15.86:0/3818757517 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11812 (secure 0 0 0) 0x7fb5800be520 con 0x7fb598102090 2024-04-16T05:33:01.187 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:32:59.861363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-04-16T05:33:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-04-16T05:33:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5840a3bb0 msgr2=0x7fb5840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5840a3bb0 0x7fb5840a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5880058c0 tx=0x7fb588005d30 comp rx=0 tx=0).stop 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 msgr2=0x7fb598198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 0x7fb598198990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb580000c00 tx=0x7fb580004040 comp rx=0 tx=0).stop 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 shutdown_connections 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb598102a50 0x7fb59819fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb598102090 0x7fb598198990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5840a3bb0 0x7fb5840a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 --2- 172.21.15.86:0/3818757517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb598100f40 0x7fb598198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.189+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 >> 172.21.15.86:0/3818757517 conn(0x7fb5980fc4b0 msgr2=0x7fb598069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.190+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 shutdown_connections 2024-04-16T05:33:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.190+0000 7fb59d688640 1 -- 172.21.15.86:0/3818757517 wait complete. 2024-04-16T05:33:01.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 -- 172.21.15.86:0/3665820527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0102070 msgr2=0x7fc0a0102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/3665820527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0102070 0x7fc0a0102490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc098009ca0 tx=0x7fc09802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 -- 172.21.15.86:0/3665820527 shutdown_connections 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/3665820527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0a0103c20 0x7fc0a010a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/3665820527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a0103260 0x7fc0a01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/3665820527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0102070 0x7fc0a0102490 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 -- 172.21.15.86:0/3665820527 >> 172.21.15.86:0/3665820527 conn(0x7fc0a00fd600 msgr2=0x7fc0a00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 -- 172.21.15.86:0/3665820527 shutdown_connections 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 -- 172.21.15.86:0/3665820527 wait complete. 2024-04-16T05:33:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.306+0000 7fc0a7b59640 1 Processor -- start 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a7b59640 1 -- start start 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a7b59640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0a0102070 0x7fc0a0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a7b59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a0103260 0x7fc0a0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a7b59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0103c20 0x7fc0a019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a7b59640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010dd00 con 0x7fc0a0103c20 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a7b59640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010db80 con 0x7fc0a0102070 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a7b59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010de80 con 0x7fc0a0103260 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a60f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0103c20 0x7fc0a019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a60f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0103c20 0x7fc0a019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56812/0 (socket says 172.21.15.86:56812) 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a60f6640 1 -- 172.21.15.86:0/694316578 learned_addr learned my addr 172.21.15.86:0/694316578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a58f5640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0a0102070 0x7fc0a0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a50f4640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a0103260 0x7fc0a0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a60f6640 1 -- 172.21.15.86:0/694316578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a0103260 msgr2=0x7fc0a0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a60f6640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a0103260 0x7fc0a0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a60f6640 1 -- 172.21.15.86:0/694316578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0a0102070 msgr2=0x7fc0a0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a60f6640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0a0102070 0x7fc0a0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a60f6640 1 -- 172.21.15.86:0/694316578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a019ff40 con 0x7fc0a0103c20 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a58f5640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0a0102070 0x7fc0a0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a60f6640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0103c20 0x7fc0a019f8d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c970 tx=0x7fc09c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc096ffd640 1 -- 172.21.15.86:0/694316578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c007bb0 con 0x7fc0a0103c20 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc096ffd640 1 -- 172.21.15.86:0/694316578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc09c007d50 con 0x7fc0a0103c20 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc096ffd640 1 -- 172.21.15.86:0/694316578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c018800 con 0x7fc0a0103c20 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a01a0230 con 0x7fc0a0103c20 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.307+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a01a0730 con 0x7fc0a0103c20 2024-04-16T05:33:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.308+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a0109390 con 0x7fc0a0103c20 2024-04-16T05:33:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.309+0000 7fc096ffd640 1 -- 172.21.15.86:0/694316578 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc09c00fa50 con 0x7fc0a0103c20 2024-04-16T05:33:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.310+0000 7fc096ffd640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc08c0a3bb0 0x7fc08c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.310+0000 7fc0a58f5640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc08c0a3bb0 0x7fc08c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.310+0000 7fc096ffd640 1 -- 172.21.15.86:0/694316578 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc09c0d0740 con 0x7fc0a0103c20 2024-04-16T05:33:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.310+0000 7fc0a58f5640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc08c0a3bb0 0x7fc08c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc09802c700 tx=0x7fc0980325b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.314+0000 7fc096ffd640 1 -- 172.21.15.86:0/694316578 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc09c014030 con 0x7fc0a0103c20 2024-04-16T05:33:01.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.494+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0a0062c00 con 0x7fc0a0103c20 2024-04-16T05:33:01.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.496+0000 7fc096ffd640 1 -- 172.21.15.86:0/694316578 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24093 (secure 0 0 0) 0x7fc09c09cee0 con 0x7fc0a0103c20 2024-04-16T05:33:01.495 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:01.495 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:32:59.861175+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:01.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc08c0a3bb0 msgr2=0x7fc08c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc08c0a3bb0 0x7fc08c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc09802c700 tx=0x7fc0980325b0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0103c20 msgr2=0x7fc0a019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0103c20 0x7fc0a019f8d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c970 tx=0x7fc09c00ce30 comp rx=0 tx=0).stop 2024-04-16T05:33:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 shutdown_connections 2024-04-16T05:33:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0103c20 0x7fc0a019f8d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a0103260 0x7fc0a0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc08c0a3bb0 0x7fc08c0a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 --2- 172.21.15.86:0/694316578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0a0102070 0x7fc0a0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 >> 172.21.15.86:0/694316578 conn(0x7fc0a00fd600 msgr2=0x7fc0a0106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 shutdown_connections 2024-04-16T05:33:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.498+0000 7fc0a7b59640 1 -- 172.21.15.86:0/694316578 wait complete. 2024-04-16T05:33:01.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:33:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.617+0000 7f35805a8640 1 -- 172.21.15.86:0/3933895231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 msgr2=0x7f357810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.617+0000 7f35805a8640 1 --2- 172.21.15.86:0/3933895231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 0x7f357810a490 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3570009ca0 tx=0x7f357002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.617+0000 7f35805a8640 1 -- 172.21.15.86:0/3933895231 shutdown_connections 2024-04-16T05:33:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.617+0000 7f35805a8640 1 --2- 172.21.15.86:0/3933895231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 0x7f357810a490 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.617+0000 7f35805a8640 1 --2- 172.21.15.86:0/3933895231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3578103240 0x7f35781036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.617+0000 7f35805a8640 1 --2- 172.21.15.86:0/3933895231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3578102050 0x7f3578102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.617+0000 7f35805a8640 1 -- 172.21.15.86:0/3933895231 >> 172.21.15.86:0/3933895231 conn(0x7f35780fd5e0 msgr2=0x7f35780ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.617+0000 7f35805a8640 1 -- 172.21.15.86:0/3933895231 shutdown_connections 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.617+0000 7f35805a8640 1 -- 172.21.15.86:0/3933895231 wait complete. 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f35805a8640 1 Processor -- start 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f35805a8640 1 -- start start 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f35805a8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3578102050 0x7f35781982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f35805a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3578103240 0x7f35781987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f35805a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 0x7f357819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f35805a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f357810dce0 con 0x7f3578103c00 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f35805a8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f357810db60 con 0x7f3578102050 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f35805a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f357810de60 con 0x7f3578103240 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f357eb45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 0x7f357819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f357eb45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 0x7f357819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56834/0 (socket says 172.21.15.86:56834) 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f357eb45640 1 -- 172.21.15.86:0/3079482536 learned_addr learned my addr 172.21.15.86:0/3079482536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f357e344640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3578102050 0x7f35781982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f357eb45640 1 -- 172.21.15.86:0/3079482536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3578103240 msgr2=0x7f35781987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f357eb45640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3578103240 0x7f35781987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f357eb45640 1 -- 172.21.15.86:0/3079482536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3578102050 msgr2=0x7f35781982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f357eb45640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3578102050 0x7f35781982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.618+0000 7f357eb45640 1 -- 172.21.15.86:0/3079482536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357819fed0 con 0x7f3578103c00 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.619+0000 7f357eb45640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 0x7f357819f860 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f357002c700 tx=0x7f3570032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.619+0000 7f356f7fe640 1 -- 172.21.15.86:0/3079482536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570039040 con 0x7f3578103c00 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.619+0000 7f356f7fe640 1 -- 172.21.15.86:0/3079482536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3570044070 con 0x7f3578103c00 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.619+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35781a0160 con 0x7f3578103c00 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.619+0000 7f356f7fe640 1 -- 172.21.15.86:0/3079482536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700042a0 con 0x7f3578103c00 2024-04-16T05:33:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.619+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35781a0650 con 0x7f3578103c00 2024-04-16T05:33:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.620+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3578109370 con 0x7f3578103c00 2024-04-16T05:33:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.621+0000 7f356f7fe640 1 -- 172.21.15.86:0/3079482536 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3570003970 con 0x7f3578103c00 2024-04-16T05:33:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.621+0000 7f356f7fe640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35640a3bb0 0x7f35640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.621+0000 7f357e344640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35640a3bb0 0x7f35640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.621+0000 7f356f7fe640 1 -- 172.21.15.86:0/3079482536 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f35700f2b90 con 0x7f3578103c00 2024-04-16T05:33:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.622+0000 7f357e344640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35640a3bb0 0x7f35640a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35781030a0 tx=0x7f357400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:01.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.626+0000 7f356f7fe640 1 -- 172.21.15.86:0/3079482536 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35700bf3c0 con 0x7f3578103c00 2024-04-16T05:33:01.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.809+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3578062c00 con 0x7f3578103c00 2024-04-16T05:33:01.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.810+0000 7f356f7fe640 1 -- 172.21.15.86:0/3079482536 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24080 (secure 0 0 0) 0x7f35700becc0 con 0x7f3578103c00 2024-04-16T05:33:01.809 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:01.809 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:01.527267+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:01.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35640a3bb0 msgr2=0x7f35640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35640a3bb0 0x7f35640a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35781030a0 tx=0x7f357400b390 comp rx=0 tx=0).stop 2024-04-16T05:33:01.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 msgr2=0x7f357819f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 0x7f357819f860 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f357002c700 tx=0x7f3570032f20 comp rx=0 tx=0).stop 2024-04-16T05:33:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 shutdown_connections 2024-04-16T05:33:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3578103c00 0x7f357819f860 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3578103240 0x7f35781987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35640a3bb0 0x7f35640a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 --2- 172.21.15.86:0/3079482536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3578102050 0x7f35781982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 >> 172.21.15.86:0/3079482536 conn(0x7f35780fd5e0 msgr2=0x7f3578106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 shutdown_connections 2024-04-16T05:33:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.812+0000 7f35805a8640 1 -- 172.21.15.86:0/3079482536 wait complete. 2024-04-16T05:33:01.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:01 smithi086 ceph-mon[32028]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T05:33:01.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:01 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2579223534' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T05:33:01.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:01 smithi086 ceph-mon[32028]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:33:01.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:01 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3818757517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:33:01.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:01 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/694316578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:01.838 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:33:01.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:01 smithi194 ceph-mon[35865]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T05:33:01.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:01 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2579223534' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T05:33:01.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:01 smithi194 ceph-mon[35865]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:33:01.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:01 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3818757517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:33:01.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:01 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/694316578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:01.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.945+0000 7fe2ba860640 1 -- 172.21.15.86:0/2761984100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 msgr2=0x7fe2b410c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.945+0000 7fe2ba860640 1 --2- 172.21.15.86:0/2761984100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 0x7fe2b410c3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009ca0 tx=0x7fe2ac02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.945+0000 7fe2ba860640 1 -- 172.21.15.86:0/2761984100 shutdown_connections 2024-04-16T05:33:01.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.945+0000 7fe2ba860640 1 --2- 172.21.15.86:0/2761984100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 0x7fe2b410c3c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.945+0000 7fe2ba860640 1 --2- 172.21.15.86:0/2761984100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4074a60 0x7fe2b4074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.945+0000 7fe2ba860640 1 --2- 172.21.15.86:0/2761984100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2b4076040 0x7fe2b4074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.945+0000 7fe2ba860640 1 -- 172.21.15.86:0/2761984100 >> 172.21.15.86:0/2761984100 conn(0x7fe2b40fd5e0 msgr2=0x7fe2b40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.945+0000 7fe2ba860640 1 -- 172.21.15.86:0/2761984100 shutdown_connections 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.945+0000 7fe2ba860640 1 -- 172.21.15.86:0/2761984100 wait complete. 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2ba860640 1 Processor -- start 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2ba860640 1 -- start start 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2ba860640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2b4074a60 0x7fe2b419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2ba860640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4076040 0x7fe2b419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2ba860640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 0x7fe2b41a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2ba860640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410ff50 con 0x7fe2b4109f30 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2ba860640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410fdd0 con 0x7fe2b4074a60 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2ba860640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41100d0 con 0x7fe2b4076040 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b8dfd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 0x7fe2b41a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b8dfd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 0x7fe2b41a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56852/0 (socket says 172.21.15.86:56852) 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b8dfd640 1 -- 172.21.15.86:0/3245369442 learned_addr learned my addr 172.21.15.86:0/3245369442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b3fff640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2b4074a60 0x7fe2b419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2abfff640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4076040 0x7fe2b419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b8dfd640 1 -- 172.21.15.86:0/3245369442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4076040 msgr2=0x7fe2b419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b8dfd640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4076040 0x7fe2b419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b8dfd640 1 -- 172.21.15.86:0/3245369442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2b4074a60 msgr2=0x7fe2b419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b8dfd640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2b4074a60 0x7fe2b419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b8dfd640 1 -- 172.21.15.86:0/3245369442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b41a2190 con 0x7fe2b4109f30 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.946+0000 7fe2b8dfd640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 0x7fe2b41a1b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac037040 tx=0x7fe2ac032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.947+0000 7fe2b1ffb640 1 -- 172.21.15.86:0/3245369442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac039040 con 0x7fe2b4109f30 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.947+0000 7fe2b1ffb640 1 -- 172.21.15.86:0/3245369442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac044070 con 0x7fe2b4109f30 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.947+0000 7fe2b1ffb640 1 -- 172.21.15.86:0/3245369442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac004260 con 0x7fe2b4109f30 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.947+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b41a2420 con 0x7fe2b4109f30 2024-04-16T05:33:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.947+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41a2960 con 0x7fe2b4109f30 2024-04-16T05:33:01.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.947+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b4108c60 con 0x7fe2b4109f30 2024-04-16T05:33:01.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.949+0000 7fe2b1ffb640 1 -- 172.21.15.86:0/3245369442 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe2ac003970 con 0x7fe2b4109f30 2024-04-16T05:33:01.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.949+0000 7fe2b1ffb640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.949+0000 7fe2b3fff640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.949+0000 7fe2b1ffb640 1 -- 172.21.15.86:0/3245369442 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe2ac0f2d60 con 0x7fe2b4109f30 2024-04-16T05:33:01.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.950+0000 7fe2b3fff640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009990 tx=0x7fe2a4009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:01.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:01.953+0000 7fe2b1ffb640 1 -- 172.21.15.86:0/3245369442 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2ac0bf500 con 0x7fe2b4109f30 2024-04-16T05:33:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:01 smithi196 ceph-mon[36373]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T05:33:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:01 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2579223534' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T05:33:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:01 smithi196 ceph-mon[36373]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:33:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:01 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3818757517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:33:01.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:01 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/694316578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:02.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.168+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2b4062c00 con 0x7fe2b4109f30 2024-04-16T05:33:02.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.169+0000 7fe2b1ffb640 1 -- 172.21.15.86:0/3245369442 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11812 (secure 0 0 0) 0x7fe2ac0bee00 con 0x7fe2b4109f30 2024-04-16T05:33:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:02.168 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:32:59.861363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-04-16T05:33:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-04-16T05:33:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe2a00a3c30 msgr2=0x7fe2a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009990 tx=0x7fe2a4009110 comp rx=0 tx=0).stop 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 msgr2=0x7fe2b41a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 0x7fe2b41a1b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac037040 tx=0x7fe2ac032f90 comp rx=0 tx=0).stop 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 shutdown_connections 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b4109f30 0x7fe2b41a1b20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4076040 0x7fe2b419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 --2- 172.21.15.86:0/3245369442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2b4074a60 0x7fe2b419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 >> 172.21.15.86:0/3245369442 conn(0x7fe2b40fd5e0 msgr2=0x7fe2b4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 shutdown_connections 2024-04-16T05:33:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.171+0000 7fe2ba860640 1 -- 172.21.15.86:0/3245369442 wait complete. 2024-04-16T05:33:02.181 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:33:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.286+0000 7f2c4755f640 1 -- 172.21.15.86:0/1166484836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40100f40 msgr2=0x7f2c40101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.286+0000 7f2c4755f640 1 --2- 172.21.15.86:0/1166484836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40100f40 0x7f2c40101360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009900 tx=0x7f2c3c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 -- 172.21.15.86:0/1166484836 shutdown_connections 2024-04-16T05:33:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 --2- 172.21.15.86:0/1166484836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c40102a50 0x7f2c40109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 --2- 172.21.15.86:0/1166484836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c40102090 0x7f2c40102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 --2- 172.21.15.86:0/1166484836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40100f40 0x7f2c40101360 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 -- 172.21.15.86:0/1166484836 >> 172.21.15.86:0/1166484836 conn(0x7f2c400fc4b0 msgr2=0x7f2c400fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 -- 172.21.15.86:0/1166484836 shutdown_connections 2024-04-16T05:33:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 -- 172.21.15.86:0/1166484836 wait complete. 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 Processor -- start 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 -- start start 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c40100f40 0x7f2c40198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c40102090 0x7f2c401987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40102a50 0x7f2c4019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4010dcf0 con 0x7f2c40102a50 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4010db70 con 0x7f2c40100f40 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c4755f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4010de70 con 0x7f2c40102090 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c45afc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40102a50 0x7f2c4019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c45afc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40102a50 0x7f2c4019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56866/0 (socket says 172.21.15.86:56866) 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.287+0000 7f2c45afc640 1 -- 172.21.15.86:0/3834004086 learned_addr learned my addr 172.21.15.86:0/3834004086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c44afa640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c40102090 0x7f2c401987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c45afc640 1 -- 172.21.15.86:0/3834004086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c40102090 msgr2=0x7f2c401987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c45afc640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c40102090 0x7f2c401987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c45afc640 1 -- 172.21.15.86:0/3834004086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c40100f40 msgr2=0x7f2c40198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c45afc640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c40100f40 0x7f2c40198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c45afc640 1 -- 172.21.15.86:0/3834004086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4019feb0 con 0x7f2c40102a50 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c45afc640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40102a50 0x7f2c4019f840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800cce0 tx=0x7f2c3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c367fc640 1 -- 172.21.15.86:0/3834004086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3800f450 con 0x7f2c40102a50 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c367fc640 1 -- 172.21.15.86:0/3834004086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3800fad0 con 0x7f2c40102a50 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c401a01a0 con 0x7f2c40102a50 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c367fc640 1 -- 172.21.15.86:0/3834004086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38018c10 con 0x7f2c40102a50 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.288+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401a06a0 con 0x7f2c40102a50 2024-04-16T05:33:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.289+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c401081f0 con 0x7f2c40102a50 2024-04-16T05:33:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.290+0000 7f2c367fc640 1 -- 172.21.15.86:0/3834004086 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2c3800f5f0 con 0x7f2c40102a50 2024-04-16T05:33:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.291+0000 7f2c367fc640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2c2c0a3bb0 0x7f2c2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.291+0000 7f2c452fb640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2c2c0a3bb0 0x7f2c2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.291+0000 7f2c367fc640 1 -- 172.21.15.86:0/3834004086 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2c380d0e10 con 0x7f2c40102a50 2024-04-16T05:33:02.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.291+0000 7f2c452fb640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2c2c0a3bb0 0x7f2c2c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005c20 tx=0x7f2c3c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.294+0000 7f2c367fc640 1 -- 172.21.15.86:0/3834004086 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c38014030 con 0x7f2c40102a50 2024-04-16T05:33:02.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.474+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c40062c00 con 0x7f2c40102a50 2024-04-16T05:33:02.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.475+0000 7f2c367fc640 1 -- 172.21.15.86:0/3834004086 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24080 (secure 0 0 0) 0x7f2c3809d640 con 0x7f2c40102a50 2024-04-16T05:33:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:02.475 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:01.527267+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:02.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2c2c0a3bb0 msgr2=0x7f2c2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2c2c0a3bb0 0x7f2c2c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005c20 tx=0x7f2c3c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40102a50 msgr2=0x7f2c4019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40102a50 0x7f2c4019f840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800cce0 tx=0x7f2c3800a620 comp rx=0 tx=0).stop 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 shutdown_connections 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c40102a50 0x7f2c4019f840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c40102090 0x7f2c401987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f2c2c0a3bb0 0x7f2c2c0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 --2- 172.21.15.86:0/3834004086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c40100f40 0x7f2c40198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.477+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 >> 172.21.15.86:0/3834004086 conn(0x7f2c400fc4b0 msgr2=0x7f2c401053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.478+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 shutdown_connections 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.478+0000 7f2c4755f640 1 -- 172.21.15.86:0/3834004086 wait complete. 2024-04-16T05:33:02.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-16T05:32:54.852973+0000', 'modified': '2024-04-16T05:32:59.861363+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.194:6839/601781370', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 601781370}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 601781370}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24482, 'qdb_cluster': [24482]} 2024-04-16T05:33:02.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-16T05:33:02.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 -- 172.21.15.86:0/3296135829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 msgr2=0x7f0f98109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 --2- 172.21.15.86:0/3296135829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 0x7f0f98109370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0f90009ca0 tx=0x7f0f9002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 -- 172.21.15.86:0/3296135829 shutdown_connections 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 --2- 172.21.15.86:0/3296135829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 0x7f0f98109370 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 --2- 172.21.15.86:0/3296135829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f98102090 0x7f0f98102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 --2- 172.21.15.86:0/3296135829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f98100f40 0x7f0f98101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 -- 172.21.15.86:0/3296135829 >> 172.21.15.86:0/3296135829 conn(0x7f0f980fc4b0 msgr2=0x7f0f980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 -- 172.21.15.86:0/3296135829 shutdown_connections 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 -- 172.21.15.86:0/3296135829 wait complete. 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 Processor -- start 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 -- start start 2024-04-16T05:33:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f98100f40 0x7f0f98197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f98102090 0x7f0f98197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 0x7f0f9819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9810cc60 con 0x7f0f98102a50 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9810cae0 con 0x7f0f98100f40 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9e752640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9810cde0 con 0x7f0f98102090 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.599+0000 7f0f9ccef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 0x7f0f9819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9ccef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 0x7f0f9819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56880/0 (socket says 172.21.15.86:56880) 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9ccef640 1 -- 172.21.15.86:0/1200287247 learned_addr learned my addr 172.21.15.86:0/1200287247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f97fff640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f98100f40 0x7f0f98197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f98102090 0x7f0f98197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9ccef640 1 -- 172.21.15.86:0/1200287247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f98102090 msgr2=0x7f0f98197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f98102090 0x7f0f98197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9ccef640 1 -- 172.21.15.86:0/1200287247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f98100f40 msgr2=0x7f0f98197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f98100f40 0x7f0f98197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9ccef640 1 -- 172.21.15.86:0/1200287247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9819ee40 con 0x7f0f98102a50 2024-04-16T05:33:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 0x7f0f9819e7d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0f90037040 tx=0x7f0f900346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f95ffb640 1 -- 172.21.15.86:0/1200287247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90034cd0 con 0x7f0f98102a50 2024-04-16T05:33:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f95ffb640 1 -- 172.21.15.86:0/1200287247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f90003970 con 0x7f0f98102a50 2024-04-16T05:33:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9e752640 1 -- 172.21.15.86:0/1200287247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f98069190 con 0x7f0f98102a50 2024-04-16T05:33:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f95ffb640 1 -- 172.21.15.86:0/1200287247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90004350 con 0x7f0f98102a50 2024-04-16T05:33:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.600+0000 7f0f9e752640 1 -- 172.21.15.86:0/1200287247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f980696d0 con 0x7f0f98102a50 2024-04-16T05:33:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.602+0000 7f0f95ffb640 1 -- 172.21.15.86:0/1200287247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0f90003b20 con 0x7f0f98102a50 2024-04-16T05:33:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0fa1419740 1 Processor -- start 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0fa1419740 1 -- start start 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f95ffb640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0f840a3c30 0x7f0f840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720170 0x5629c8720590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c8725660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8728160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0fa1419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629c862f240 con 0x5629c8725cd0 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0fa1419740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5629c868c9f0 con 0x5629c8720170 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0fa1419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5629c86479e0 con 0x5629c8720b60 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0f840a3c30 0x7f0f840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f95ffb640 1 -- 172.21.15.86:0/1200287247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0f900f47f0 con 0x7f0f98102a50 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f9ccef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8728160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720170 0x5629c8720590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f9ccef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8728160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56894/0 (socket says 172.21.15.86:56894) 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720170 0x5629c8720590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35870/0 (socket says 172.21.15.86:35870) 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f9ccef640 1 -- 172.21.15.86:0/1561325504 learned_addr learned my addr 172.21.15.86:0/1561325504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/1561325504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c8725660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f9ccef640 1 -- 172.21.15.86:0/1561325504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 msgr2=0x5629c8725660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/1561325504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c8725660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f9ccef640 1 -- 172.21.15.86:0/1561325504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720170 msgr2=0x5629c8720590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/1561325504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720170 0x5629c8720590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f9ccef640 1 -- 172.21.15.86:0/1561325504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c863abc0 con 0x5629c8725cd0 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0f840a3c30 0x7f0f840a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0f80006fd0 tx=0x7f0f8000a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.603+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/1561325504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8728160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f900c8fd0 tx=0x7f0f900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62244f16f32ee72 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0f8e7fc640 1 -- 172.21.15.86:0/1561325504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90039070 con 0x5629c8725cd0 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0f8e7fc640 1 -- 172.21.15.86:0/1561325504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f900098d0 con 0x5629c8725cd0 2024-04-16T05:33:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0f8e7fc640 1 -- 172.21.15.86:0/1561325504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f900beb90 con 0x5629c8725cd0 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0fa1419740 1 -- 172.21.15.86:0/1561325504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 msgr2=0x5629c8728160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0fa1419740 1 --2- 172.21.15.86:0/1561325504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8728160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f900c8fd0 tx=0x7f0f900095c0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0fa1419740 1 -- 172.21.15.86:0/1561325504 shutdown_connections 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0fa1419740 1 --2- 172.21.15.86:0/1561325504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8728160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0fa1419740 1 --2- 172.21.15.86:0/1561325504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c8725660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0fa1419740 1 --2- 172.21.15.86:0/1561325504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720170 0x5629c8720590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0fa1419740 1 -- 172.21.15.86:0/1561325504 >> 172.21.15.86:0/1561325504 conn(0x5629c87172d0 msgr2=0x5629c87176d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0fa1419740 1 -- 172.21.15.86:0/1561325504 shutdown_connections 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.604+0000 7f0fa1419740 1 -- 172.21.15.86:0/1561325504 wait complete. 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0fa1419740 1 -- start start 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0fa1419740 1 Processor -- start 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720170 0x5629c872a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720b60 0x5629c872a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8731660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0fa1419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629c86860f0 con 0x5629c8725cd0 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0fa1419740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5629c8680dd0 con 0x5629c8720b60 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0fa1419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5629c868ecd0 con 0x5629c8720170 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0fa1419740 10 client.0 _ll_register_callbacks cb 0x5629c8677570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f9ccef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8731660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f9ccef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8731660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56896/0 (socket says 172.21.15.86:56896) 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f9ccef640 1 -- 172.21.15.86:0/40353285 learned_addr learned my addr 172.21.15.86:0/40353285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f97fff640 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720170 0x5629c872a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f9ccef640 1 -- 172.21.15.86:0/40353285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720170 msgr2=0x5629c872a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720170 0x5629c872a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720b60 0x5629c872a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f9ccef640 1 -- 172.21.15.86:0/40353285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720b60 msgr2=0x5629c872a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720b60 0x5629c872a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f9ccef640 1 -- 172.21.15.86:0/40353285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c83942d0 con 0x5629c8725cd0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.605+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720b60 0x5629c872a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8731660 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f900bcca0 tx=0x7f0f90048290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc29c040c175a3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0f777fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0f777fe640 1 -- 172.21.15.86:0/40353285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90039070 con 0x5629c8725cd0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0f777fe640 1 -- 172.21.15.86:0/40353285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0f900bcf00 con 0x5629c8725cd0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0fa1419740 1 -- 172.21.15.86:0/40353285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629c86084c0 con 0x5629c8725cd0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0f777fe640 1 -- 172.21.15.86:0/40353285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0f900c0f90 con 0x5629c8725cd0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0fa1419740 10 client.14748 fetch_fsmap learned FSMap version 17 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0fa1419740 1 -- 172.21.15.86:0/40353285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5629c845e6b0 con 0x5629c8725cd0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0f777fe640 1 -- 172.21.15.86:0/40353285 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5629c86084c0 con 0x5629c8725cd0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0f777fe640 1 -- 172.21.15.86:0/40353285 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f0f900c9070 con 0x5629c8725cd0 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0fa1419740 10 client.14748 fetch_fsmap finished waiting for FSMap version 17 2024-04-16T05:33:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0fa1419740 10 client.14748 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0fa1419740 20 client.14748 populate_metadata read hostname 'smithi086' 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0fa1419740 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5629c8736a10 0x5629c8737e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0fa1419740 4 client.14748 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0fa1419740 1 -- 172.21.15.86:0/40353285 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c868f4e0 con 0x5629c8736a10 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.606+0000 7f0f97fff640 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5629c8736a10 0x5629c8737e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.607+0000 7f0f777fe640 10 client.14748 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.607+0000 7f0f97fff640 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5629c8736a10 0x5629c8737e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7680a3ec951329a9 server_cookie=e0f268ff91d0d13b in_seq=0 out_seq=0 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0f777fe640 1 -- 172.21.15.86:0/40353285 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629c868f4e0 con 0x5629c8736a10 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0f777fe640 10 client.14748 handle_command_reply: tid=0 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 client.14748 shutdown 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 -- 172.21.15.86:0/40353285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 msgr2=0x5629c8731660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8731660 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f900bcca0 tx=0x7f0f90048290 comp rx=0 tx=0).stop 2024-04-16T05:33:02.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 -- 172.21.15.86:0/40353285 shutdown_connections 2024-04-16T05:33:02.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5629c8736a10 0x5629c8737e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8725cd0 0x5629c8731660 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8720b60 0x5629c872a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0f777fe640 0 client.14748 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:33:02.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 --2- 172.21.15.86:0/40353285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720170 0x5629c872a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 -- 172.21.15.86:0/40353285 >> 172.21.15.86:0/40353285 conn(0x5629c87172d0 msgr2=0x5629c8716410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:02.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 -- 172.21.15.86:0/40353285 shutdown_connections 2024-04-16T05:33:02.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 1 -- 172.21.15.86:0/40353285 wait complete. 2024-04-16T05:33:02.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.608+0000 7f0fa1419740 20 client.14748 trim_cache size 0 max 16384 2024-04-16T05:33:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0fa1419740 1 Processor -- start 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0fa1419740 1 -- start start 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c8721600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c8731620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 0x5629c87176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0fa1419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629c87361c0 con 0x5629c8720170 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0fa1419740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5629c8736890 con 0x5629c8725cd0 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0fa1419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5629c8736f60 con 0x5629c8720b60 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c8721600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c8721600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56900/0 (socket says 172.21.15.86:56900) 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f97fff640 1 -- 172.21.15.86:0/2391448749 learned_addr learned my addr 172.21.15.86:0/2391448749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f97fff640 1 -- 172.21.15.86:0/2391448749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 msgr2=0x5629c8731620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f97fff640 1 --2- 172.21.15.86:0/2391448749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c8731620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f97fff640 1 -- 172.21.15.86:0/2391448749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 msgr2=0x5629c87176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f97fff640 1 --2- 172.21.15.86:0/2391448749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 0x5629c87176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f97fff640 1 -- 172.21.15.86:0/2391448749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c87213f0 con 0x5629c8720170 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f97fff640 1 --2- 172.21.15.86:0/2391448749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c8721600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5629c8738c90 tx=0x7f0f8800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273cb5bf68920c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f8dffb640 1 -- 172.21.15.86:0/2391448749 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88020030 con 0x5629c8720170 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f8dffb640 1 -- 172.21.15.86:0/2391448749 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8801c420 con 0x5629c8720170 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.626+0000 7f0f8dffb640 1 -- 172.21.15.86:0/2391448749 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f880047a0 con 0x5629c8720170 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 -- 172.21.15.86:0/2391448749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 msgr2=0x5629c8721600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 --2- 172.21.15.86:0/2391448749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c8721600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5629c8738c90 tx=0x7f0f8800cd10 comp rx=0 tx=0).stop 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 -- 172.21.15.86:0/2391448749 shutdown_connections 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 --2- 172.21.15.86:0/2391448749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 0x5629c87176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 --2- 172.21.15.86:0/2391448749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c8731620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 --2- 172.21.15.86:0/2391448749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c8721600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 -- 172.21.15.86:0/2391448749 >> 172.21.15.86:0/2391448749 conn(0x5629c87172d0 msgr2=0x5629c871adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 -- 172.21.15.86:0/2391448749 shutdown_connections 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 -- 172.21.15.86:0/2391448749 wait complete. 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.627+0000 7f0fa1419740 1 -- start start 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0fa1419740 1 Processor -- start 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c872f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c87989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0fa1419740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 0x5629c879aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0fa1419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629c8737d00 con 0x5629c8720170 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0fa1419740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5629c8737630 con 0x5629c8725cd0 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0fa1419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5629c87383d0 con 0x5629c8720b60 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0fa1419740 10 client.0 _ll_register_callbacks cb 0x7f0f8800c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c872f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c872f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56908/0 (socket says 172.21.15.86:56908) 2024-04-16T05:33:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f97fff640 1 -- 172.21.15.86:0/3175339885 learned_addr learned my addr 172.21.15.86:0/3175339885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 0x5629c879aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c87989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f97fff640 1 -- 172.21.15.86:0/3175339885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 msgr2=0x5629c87989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f97fff640 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c87989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f97fff640 1 -- 172.21.15.86:0/3175339885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 msgr2=0x5629c879aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f97fff640 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 0x5629c879aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f97fff640 1 -- 172.21.15.86:0/3175339885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c879b500 con 0x5629c8720170 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f9ccef640 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 0x5629c879aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c87989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f97fff640 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c872f610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f88005760 tx=0x7f0f88005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28581b5dbf796e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f777fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f777fe640 1 -- 172.21.15.86:0/3175339885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88029070 con 0x5629c8720170 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f777fe640 1 -- 172.21.15.86:0/3175339885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0f88003b20 con 0x5629c8720170 2024-04-16T05:33:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0fa1419740 1 -- 172.21.15.86:0/3175339885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8801c420 con 0x5629c8720170 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.628+0000 7f0f777fe640 1 -- 172.21.15.86:0/3175339885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0f88035050 con 0x5629c8720170 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0fa1419740 10 client.14754 fetch_fsmap learned FSMap version 17 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0fa1419740 1 -- 172.21.15.86:0/3175339885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5629c879c830 con 0x5629c8720170 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0f777fe640 1 -- 172.21.15.86:0/3175339885 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8801c420 con 0x5629c8720170 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0f777fe640 1 -- 172.21.15.86:0/3175339885 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f0f88025540 con 0x5629c8720170 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0fa1419740 10 client.14754 fetch_fsmap finished waiting for FSMap version 17 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0fa1419740 10 client.14754 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0fa1419740 20 client.14754 populate_metadata read hostname 'smithi086' 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0fa1419740 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5629c87a10b0 0x5629c879fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0fa1419740 4 client.14754 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0fa1419740 1 -- 172.21.15.86:0/3175339885 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "status"}) -- 0x5629c8689350 con 0x5629c87a10b0 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5629c87a10b0 0x5629c879fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0f777fe640 10 client.14754 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.629+0000 7f0f8ffff640 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5629c87a10b0 0x5629c879fdc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e16919c1f0a41ca server_cookie=e895fa4ffa750bd6 in_seq=0 out_seq=0 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0f777fe640 1 -- 172.21.15.86:0/3175339885 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5629c8689350 con 0x5629c87a10b0 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0f777fe640 10 client.14754 handle_command_reply: tid=0 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 client.14754 shutdown 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 -- 172.21.15.86:0/3175339885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 msgr2=0x5629c872f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c872f610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f88005760 tx=0x7f0f88005160 comp rx=0 tx=0).stop 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 -- 172.21.15.86:0/3175339885 shutdown_connections 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5629c87a10b0 0x5629c879fdc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629c8725cd0 0x5629c879aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629c8720b60 0x5629c87989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 --2- 172.21.15.86:0/3175339885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629c8720170 0x5629c872f610 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0f777fe640 0 client.14754 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 -- 172.21.15.86:0/3175339885 >> 172.21.15.86:0/3175339885 conn(0x5629c87172d0 msgr2=0x5629c871adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 -- 172.21.15.86:0/3175339885 shutdown_connections 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 1 -- 172.21.15.86:0/3175339885 wait complete. 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.630+0000 7f0fa1419740 20 client.14754 trim_cache size 0 max 16384 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24482, 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:33:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.7844391509999999, 2024-04-16T05:33:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 17, 2024-04-16T05:33:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 90, 2024-04-16T05:33:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T05:33:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 13.151992086 2024-04-16T05:33:02.637 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 -- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0f840a3c30 msgr2=0x7f0f840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0f840a3c30 0x7f0f840a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0f80006fd0 tx=0x7f0f8000a300 comp rx=0 tx=0).stop 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 -- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 msgr2=0x7f0f9819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 0x7f0f9819e7d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0f90037040 tx=0x7f0f900346f0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 -- 172.21.15.86:0/1200287247 shutdown_connections 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f98102a50 0x7f0f9819e7d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f98102090 0x7f0f98197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0f840a3c30 0x7f0f840a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 --2- 172.21.15.86:0/1200287247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f98100f40 0x7f0f98197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 -- 172.21.15.86:0/1200287247 >> 172.21.15.86:0/1200287247 conn(0x7f0f980fc4b0 msgr2=0x7f0f98105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 -- 172.21.15.86:0/1200287247 shutdown_connections 2024-04-16T05:33:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:02.631+0000 7f0f9e752640 1 -- 172.21.15.86:0/1200287247 wait complete. 2024-04-16T05:33:02.640 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T05:33:02.640 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:33:02.641 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph fs ls 2024-04-16T05:33:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:02.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:02 smithi194 ceph-mon[35865]: pgmap v294: 129 pgs: 1 creating+activating, 4 creating+peering, 118 active+clean, 6 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 4 op/s 2024-04-16T05:33:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:02 smithi194 ceph-mon[35865]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T05:33:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:02 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3079482536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:02 smithi194 ceph-mon[35865]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:33:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:02 smithi194 ceph-mon[35865]: Cluster is now healthy 2024-04-16T05:33:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:02 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3245369442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:33:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:02 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3834004086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:02 smithi086 ceph-mon[32028]: pgmap v294: 129 pgs: 1 creating+activating, 4 creating+peering, 118 active+clean, 6 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 4 op/s 2024-04-16T05:33:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:02 smithi086 ceph-mon[32028]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T05:33:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:02 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3079482536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:02 smithi086 ceph-mon[32028]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:33:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:02 smithi086 ceph-mon[32028]: Cluster is now healthy 2024-04-16T05:33:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:02 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3245369442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:33:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:02 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3834004086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:02 smithi196 ceph-mon[36373]: pgmap v294: 129 pgs: 1 creating+activating, 4 creating+peering, 118 active+clean, 6 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 4 op/s 2024-04-16T05:33:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:02 smithi196 ceph-mon[36373]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T05:33:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:02 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3079482536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:02 smithi196 ceph-mon[36373]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:33:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:02 smithi196 ceph-mon[36373]: Cluster is now healthy 2024-04-16T05:33:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:02 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3245369442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:33:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:02 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3834004086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:04.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.606+0000 7fc9b2464700 1 -- 172.21.15.86:0/2086158673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac108b90 msgr2=0x7fc9ac108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.606+0000 7fc9b2464700 1 --2- 172.21.15.86:0/2086158673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac108b90 0x7fc9ac108f70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009990 tx=0x7fc99c009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.606+0000 7fc9b2464700 1 -- 172.21.15.86:0/2086158673 shutdown_connections 2024-04-16T05:33:04.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.606+0000 7fc9b2464700 1 --2- 172.21.15.86:0/2086158673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9ac103500 0x7fc9ac10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.606+0000 7fc9b2464700 1 --2- 172.21.15.86:0/2086158673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac108b90 0x7fc9ac108f70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.606+0000 7fc9b2464700 1 --2- 172.21.15.86:0/2086158673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ac102b40 0x7fc9ac102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.606+0000 7fc9b2464700 1 -- 172.21.15.86:0/2086158673 >> 172.21.15.86:0/2086158673 conn(0x7fc9ac079c50 msgr2=0x7fc9ac07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:04.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.606+0000 7fc9b2464700 1 -- 172.21.15.86:0/2086158673 shutdown_connections 2024-04-16T05:33:04.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.606+0000 7fc9b2464700 1 -- 172.21.15.86:0/2086158673 wait complete. 2024-04-16T05:33:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9b2464700 1 Processor -- start 2024-04-16T05:33:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9b2464700 1 -- start start 2024-04-16T05:33:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9b2464700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ac102b40 0x7fc9ac199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9b2464700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac103500 0x7fc9ac1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9b2464700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9ac108b90 0x7fc9ac19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9b2464700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac112df0 con 0x7fc9ac103500 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9b2464700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac112c70 con 0x7fc9ac108b90 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9b2464700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac112f70 con 0x7fc9ac102b40 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9a3fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac103500 0x7fc9ac1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9a3fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac103500 0x7fc9ac1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56924/0 (socket says 172.21.15.86:56924) 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.607+0000 7fc9a3fff700 1 -- 172.21.15.86:0/4284899902 learned_addr learned my addr 172.21.15.86:0/4284899902 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9abfff700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ac102b40 0x7fc9ac199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9b0a01700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9ac108b90 0x7fc9ac19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9a3fff700 1 -- 172.21.15.86:0/4284899902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ac102b40 msgr2=0x7fc9ac199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9a3fff700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ac102b40 0x7fc9ac199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9a3fff700 1 -- 172.21.15.86:0/4284899902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9ac108b90 msgr2=0x7fc9ac19db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9a3fff700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9ac108b90 0x7fc9ac19db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9a3fff700 1 -- 172.21.15.86:0/4284899902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac19e270 con 0x7fc9ac103500 2024-04-16T05:33:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9a3fff700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac103500 0x7fc9ac1997a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc99400c920 tx=0x7fc99400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9a9ffb700 1 -- 172.21.15.86:0/4284899902 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994004a30 con 0x7fc9ac103500 2024-04-16T05:33:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9a9ffb700 1 -- 172.21.15.86:0/4284899902 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc994005500 con 0x7fc9ac103500 2024-04-16T05:33:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac19e560 con 0x7fc9ac103500 2024-04-16T05:33:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9a9ffb700 1 -- 172.21.15.86:0/4284899902 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994007d50 con 0x7fc9ac103500 2024-04-16T05:33:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.608+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac1a5de0 con 0x7fc9ac103500 2024-04-16T05:33:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.609+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ac0697e0 con 0x7fc9ac103500 2024-04-16T05:33:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.610+0000 7fc9a9ffb700 1 -- 172.21.15.86:0/4284899902 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc994004d60 con 0x7fc9ac103500 2024-04-16T05:33:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.611+0000 7fc9a9ffb700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc9980a3cc0 0x7fc9980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.611+0000 7fc9abfff700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc9980a3cc0 0x7fc9980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.611+0000 7fc9a9ffb700 1 -- 172.21.15.86:0/4284899902 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc9940d04d0 con 0x7fc9ac103500 2024-04-16T05:33:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.611+0000 7fc9abfff700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc9980a3cc0 0x7fc9980a6180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00b5c0 tx=0x7fc99c005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.615+0000 7fc9a9ffb700 1 -- 172.21.15.86:0/4284899902 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc994014030 con 0x7fc9ac103500 2024-04-16T05:33:04.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:04 smithi086 ceph-mon[32028]: pgmap v296: 129 pgs: 1 creating+activating, 4 creating+peering, 124 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 3 op/s 2024-04-16T05:33:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.858+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc9ac02dfe0 con 0x7fc9ac103500 2024-04-16T05:33:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.858+0000 7fc9a9ffb700 1 -- 172.21.15.86:0/4284899902 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fc99409ccf0 con 0x7fc9ac103500 2024-04-16T05:33:04.857 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T05:33:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.860+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc9980a3cc0 msgr2=0x7fc9980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.860+0000 7fc9b2464700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc9980a3cc0 0x7fc9980a6180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00b5c0 tx=0x7fc99c005960 comp rx=0 tx=0).stop 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.860+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac103500 msgr2=0x7fc9ac1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.860+0000 7fc9b2464700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac103500 0x7fc9ac1997a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc99400c920 tx=0x7fc99400cc30 comp rx=0 tx=0).stop 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.860+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 shutdown_connections 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.860+0000 7fc9b2464700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9ac108b90 0x7fc9ac19db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.861+0000 7fc9b2464700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc9980a3cc0 0x7fc9980a6180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.861+0000 7fc9b2464700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ac102b40 0x7fc9ac199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.861+0000 7fc9b2464700 1 --2- 172.21.15.86:0/4284899902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac103500 0x7fc9ac1997a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.861+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 >> 172.21.15.86:0/4284899902 conn(0x7fc9ac079c50 msgr2=0x7fc9ac101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.861+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 shutdown_connections 2024-04-16T05:33:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:04.861+0000 7fc9b2464700 1 -- 172.21.15.86:0/4284899902 wait complete. 2024-04-16T05:33:04.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:04 smithi194 ceph-mon[35865]: pgmap v296: 129 pgs: 1 creating+activating, 4 creating+peering, 124 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 3 op/s 2024-04-16T05:33:04.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:04 smithi196 ceph-mon[36373]: pgmap v296: 129 pgs: 1 creating+activating, 4 creating+peering, 124 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 3 op/s 2024-04-16T05:33:05.631 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-16T05:33:05.631 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T05:33:05.631 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-16T05:33:05.632 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-16T05:33:05.632 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:33:05.632 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-16T05:33:05.632 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T05:33:05.632 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T05:33:05.632 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T05:33:05.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:05.632 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-16T05:33:05.648 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:33:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:33:05.650 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T05:33:05.650 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:33:05.650 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-16T05:33:05.650 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42052sec preferred_lft 42052sec 2024-04-16T05:33:05.650 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-16T05:33:05.650 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:33:05.650 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T05:33:05.651 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:33:05.651 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:33:05.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-16T05:33:05.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-16T05:33:05.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-16T05:33:05.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T05:33:05.651 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:33:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:33:05.828 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:05 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4284899902' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:33:05.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:33:05.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:05.882 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T05:33:05.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:05 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4284899902' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:33:05.910 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-16T05:33:05.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:05.912 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-16T05:33:05.981 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-16T05:33:05.982 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T05:33:05.982 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-16T05:33:05.982 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-16T05:33:05.982 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T05:33:05.982 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:33:05.982 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:33:05.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-16T05:33:05.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-16T05:33:05.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-16T05:33:05.983 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:33:05.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:05 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4284899902' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:33:06.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:33:06.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:33:06.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:06.331 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-16T05:33:06.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:06.346 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-16T05:33:06.403 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:33:06.403 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:33:06.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:33:06.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-16T05:33:06.403 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:33:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:33:06.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:33:06.527 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-16T05:33:06.528 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:33:06.528 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:33:06.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-16T05:33:06.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T05:33:06.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-16T05:33:06.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-16T05:33:06.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-16T05:33:06.528 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:33:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:33:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:33:06.673 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:33:06.673 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:33:06.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-16T05:33:06.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T05:33:06.673 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:33:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:33:06.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:33:06.758 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-16T05:33:06.758 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T05:33:06.758 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:33:06.785 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T05:33:06.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T05:33:06.785 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:33:06.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-16T05:33:06.905 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:06 smithi086 ceph-mon[32028]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-04-16T05:33:06.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:06 smithi194 ceph-mon[35865]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-04-16T05:33:06.910 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-16T05:33:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-16T05:33:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/b6c50a6da260f0cb308a6348521cbeed1d8d12e29579cee745793e589f38d8c0/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/edf9593d40cbdf892f93c670fcda7c7ce5141042081e3834ce6a663807a5c2f9/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/f5747f57fb823e996ecf42ce5431d6998086effe07cf3e06d545be1ff2893e57/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/56fe7a21c83e3e13470bc9ebea9a946c75d1a7c230829a06c1adf0963298605a/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/ea385a6136a039499d148c1bb785febb2577b27a5dfd3407800db1940d50bb8b/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/4433118abfd891a00d57c636d1b84838a0e83611941ef55b1fee789a2168e1f7/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/156e17e4b4de05df087f6253b2c2a9c543faf058d2081a10fddaca26c021dd7a/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/3242fbac9f218213651c2deb6c361a8cfd0e55a3c8b085bafab3922dac2ea1a8/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/05fb00820607f20def66ba0ae16c957480fb9632f219d32a62ce6d44681f1781/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/a0eb818cbbe5f4a03ece36aa8b0b586862d209d9356886aa468ab00112b9fba4/merged 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-16T05:33:06.969 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:33:06.970 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:33:06.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:06.970 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-16T05:33:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:06 smithi196 ceph-mon[36373]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-04-16T05:33:07.022 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-16T05:33:07.023 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-16T05:33:07.065 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-16T05:33:07.092 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T05:33:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-16T05:33:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-16T05:33:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-16T05:33:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/b6c50a6da260f0cb308a6348521cbeed1d8d12e29579cee745793e589f38d8c0/merged 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/edf9593d40cbdf892f93c670fcda7c7ce5141042081e3834ce6a663807a5c2f9/merged 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/f5747f57fb823e996ecf42ce5431d6998086effe07cf3e06d545be1ff2893e57/merged 2024-04-16T05:33:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/56fe7a21c83e3e13470bc9ebea9a946c75d1a7c230829a06c1adf0963298605a/merged 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/ea385a6136a039499d148c1bb785febb2577b27a5dfd3407800db1940d50bb8b/merged 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/4433118abfd891a00d57c636d1b84838a0e83611941ef55b1fee789a2168e1f7/merged 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/156e17e4b4de05df087f6253b2c2a9c543faf058d2081a10fddaca26c021dd7a/merged 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/3242fbac9f218213651c2deb6c361a8cfd0e55a3c8b085bafab3922dac2ea1a8/merged 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/05fb00820607f20def66ba0ae16c957480fb9632f219d32a62ce6d44681f1781/merged 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/a0eb818cbbe5f4a03ece36aa8b0b586862d209d9356886aa468ab00112b9fba4/merged 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:33:07.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:07.151 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-16T05:33:07.166 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:2024-04-16T05:33:07.167+0000 7f7cdda73480 -1 init, newargv = 0x5609448e2170 newargc=15 2024-04-16T05:33:07.166 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[73314]: starting ceph client 2024-04-16T05:33:07.172 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[73314]: starting fuse 2024-04-16T05:33:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:119 2024-04-16T05:33:07.181 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-16T05:33:07.181 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-16T05:33:07.181 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-16T05:33:07.181 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-16T05:33:07.182 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-16T05:33:07.183 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-16T05:33:07.183 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-16T05:33:07.183 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-16T05:33:07.183 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-16T05:33:07.183 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-16T05:33:07.183 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-16T05:33:07.183 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.admin")) 2024-04-16T05:33:07.183 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-16T05:33:07.303 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.admin.73314.asok 2024-04-16T05:33:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:33:07.314 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73314.asok 2024-04-16T05:33:07.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:07.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73314.asok status 2024-04-16T05:33:07.454 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:33:07.454 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-16T05:33:07.454 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a87fcff6340baeedd43af126e793f3241b6400ea", 2024-04-16T05:33:07.454 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev)", 2024-04-16T05:33:07.454 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "admin", 2024-04-16T05:33:07.454 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-16T05:33:07.454 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "73314", 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 14766, 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 14766 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-16T05:33:07.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 944593937 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 944593937 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.14766 192.168.0.1:0/944593937", 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/944593937", 2024-04-16T05:33:07.456 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-16T05:33:07.457 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 17, 2024-04-16T05:33:07.457 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 90, 2024-04-16T05:33:07.457 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-16T05:33:07.457 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-16T05:33:07.457 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-16T05:33:07.457 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:33:07.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:07.463 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-16T05:33:07.463 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:33:07.478 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-16T05:33:07.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:33:07.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:07.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:33:07.548 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-16T05:33:07.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T05:33:07.659 INFO:teuthology.orchestra.run.smithi086.stdout:# file: . 2024-04-16T05:33:07.659 INFO:teuthology.orchestra.run.smithi086.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-16T05:33:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:07.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:33:07.666 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:33:07.666 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:33:07.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:33:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 -- 172.21.15.86:0/4090478234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4100f40 msgr2=0x7f0cb4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4090478234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4100f40 0x7f0cb4101360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009900 tx=0x7f0ca402bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 -- 172.21.15.86:0/4090478234 shutdown_connections 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4090478234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cb4102a50 0x7f0cb4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4090478234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb4102090 0x7f0cb4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4090478234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4100f40 0x7f0cb4101360 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 -- 172.21.15.86:0/4090478234 >> 172.21.15.86:0/4090478234 conn(0x7f0cb40fc4b0 msgr2=0x7f0cb40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 -- 172.21.15.86:0/4090478234 shutdown_connections 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 -- 172.21.15.86:0/4090478234 wait complete. 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 Processor -- start 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.828+0000 7f0cba5db640 1 -- start start 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cba5db640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cb4100f40 0x7f0cb4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cba5db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4102090 0x7f0cb41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cba5db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb4102a50 0x7f0cb419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cba5db640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb410cbf0 con 0x7f0cb4102090 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cba5db640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb410ca70 con 0x7f0cb4100f40 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cba5db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb410cd70 con 0x7f0cb4102a50 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4102090 0x7f0cb41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4102090 0x7f0cb41977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56944/0 (socket says 172.21.15.86:56944) 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb37fe640 1 -- 172.21.15.86:0/4055518454 learned_addr learned my addr 172.21.15.86:0/4055518454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb3fff640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cb4100f40 0x7f0cb4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb8b78640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb4102a50 0x7f0cb419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb37fe640 1 -- 172.21.15.86:0/4055518454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb4102a50 msgr2=0x7f0cb419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb37fe640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb4102a50 0x7f0cb419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb37fe640 1 -- 172.21.15.86:0/4055518454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cb4100f40 msgr2=0x7f0cb4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb37fe640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cb4100f40 0x7f0cb4197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb37fe640 1 -- 172.21.15.86:0/4055518454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb419eeb0 con 0x7f0cb4102090 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb37fe640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4102090 0x7f0cb41977d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800d950 tx=0x7f0ca800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.829+0000 7f0cb17fa640 1 -- 172.21.15.86:0/4055518454 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8017070 con 0x7f0cb4102090 2024-04-16T05:33:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.830+0000 7f0cb17fa640 1 -- 172.21.15.86:0/4055518454 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca8009c30 con 0x7f0cb4102090 2024-04-16T05:33:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.830+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb419f1a0 con 0x7f0cb4102090 2024-04-16T05:33:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.830+0000 7f0cb17fa640 1 -- 172.21.15.86:0/4055518454 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800f600 con 0x7f0cb4102090 2024-04-16T05:33:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.830+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb419f6a0 con 0x7f0cb4102090 2024-04-16T05:33:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.831+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb4108250 con 0x7f0cb4102090 2024-04-16T05:33:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.832+0000 7f0cb17fa640 1 -- 172.21.15.86:0/4055518454 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ca801a070 con 0x7f0cb4102090 2024-04-16T05:33:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.832+0000 7f0cb17fa640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c9c0a3bb0 0x7f0c9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.833+0000 7f0cb3fff640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c9c0a3bb0 0x7f0c9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.833+0000 7f0cb17fa640 1 -- 172.21.15.86:0/4055518454 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0ca80cffd0 con 0x7f0cb4102090 2024-04-16T05:33:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.833+0000 7f0cb3fff640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c9c0a3bb0 0x7f0c9c0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ca402c7b0 tx=0x7f0ca4005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:07.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:07.836+0000 7f0cb17fa640 1 -- 172.21.15.86:0/4055518454 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ca8009de0 con 0x7f0cb4102090 2024-04-16T05:33:08.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.015+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cb402d870 con 0x7f0cb4102090 2024-04-16T05:33:08.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.016+0000 7f0cb17fa640 1 -- 172.21.15.86:0/4055518454 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24080 (secure 0 0 0) 0x7f0ca809c120 con 0x7f0cb4102090 2024-04-16T05:33:08.015 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:08.015 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:01.527267+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:08.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.018+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c9c0a3bb0 msgr2=0x7f0c9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.018+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c9c0a3bb0 0x7f0c9c0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ca402c7b0 tx=0x7f0ca4005fd0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.018+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4102090 msgr2=0x7f0cb41977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.018+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4102090 0x7f0cb41977d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800d950 tx=0x7f0ca800de10 comp rx=0 tx=0).stop 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.018+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 shutdown_connections 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.019+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb4102a50 0x7f0cb419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.019+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb4102090 0x7f0cb41977d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.019+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c9c0a3bb0 0x7f0c9c0a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.019+0000 7f0cba5db640 1 --2- 172.21.15.86:0/4055518454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cb4100f40 0x7f0cb4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.019+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 >> 172.21.15.86:0/4055518454 conn(0x7f0cb40fc4b0 msgr2=0x7f0cb4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.019+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 shutdown_connections 2024-04-16T05:33:08.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.019+0000 7f0cba5db640 1 -- 172.21.15.86:0/4055518454 wait complete. 2024-04-16T05:33:08.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:33:08.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:33:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 -- 172.21.15.86:0/58370313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 msgr2=0x7f80b01093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 --2- 172.21.15.86:0/58370313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 0x7f80b01093c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009900 tx=0x7f80a802bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 -- 172.21.15.86:0/58370313 shutdown_connections 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 --2- 172.21.15.86:0/58370313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 0x7f80b01093c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 --2- 172.21.15.86:0/58370313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b01020e0 0x7f80b0102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 --2- 172.21.15.86:0/58370313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80b0100f10 0x7f80b0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 -- 172.21.15.86:0/58370313 >> 172.21.15.86:0/58370313 conn(0x7f80b00fc4d0 msgr2=0x7f80b00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 -- 172.21.15.86:0/58370313 shutdown_connections 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 -- 172.21.15.86:0/58370313 wait complete. 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 Processor -- start 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.138+0000 7f80b57e1640 1 -- start start 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80b57e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b0100f10 0x7f80b006b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80b57e1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80b01020e0 0x7f80b019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80b57e1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 0x7f80b019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80b57e1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010cbb0 con 0x7f80b0102aa0 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80b57e1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010ca30 con 0x7f80b01020e0 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80b57e1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010cd30 con 0x7f80b0100f10 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 0x7f80b019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 0x7f80b019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56974/0 (socket says 172.21.15.86:56974) 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80af7fe640 1 -- 172.21.15.86:0/2921741786 learned_addr learned my addr 172.21.15.86:0/2921741786 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80ae7fc640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80b01020e0 0x7f80b019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80aeffd640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b0100f10 0x7f80b006b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80af7fe640 1 -- 172.21.15.86:0/2921741786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b0100f10 msgr2=0x7f80b006b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80af7fe640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b0100f10 0x7f80b006b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80af7fe640 1 -- 172.21.15.86:0/2921741786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80b01020e0 msgr2=0x7f80b019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80af7fe640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80b01020e0 0x7f80b019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80af7fe640 1 -- 172.21.15.86:0/2921741786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b019ff10 con 0x7f80b0102aa0 2024-04-16T05:33:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80ae7fc640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80b01020e0 0x7f80b019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80af7fe640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 0x7f80b019f650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f80a802ce70 tx=0x7f80a8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80a7fff640 1 -- 172.21.15.86:0/2921741786 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8041070 con 0x7f80b0102aa0 2024-04-16T05:33:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80a7fff640 1 -- 172.21.15.86:0/2921741786 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a80054e0 con 0x7f80b0102aa0 2024-04-16T05:33:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b01a01a0 con 0x7f80b0102aa0 2024-04-16T05:33:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80a7fff640 1 -- 172.21.15.86:0/2921741786 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a80344a0 con 0x7f80b0102aa0 2024-04-16T05:33:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.139+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b01a0690 con 0x7f80b0102aa0 2024-04-16T05:33:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.140+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b0108320 con 0x7f80b0102aa0 2024-04-16T05:33:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.141+0000 7f80a7fff640 1 -- 172.21.15.86:0/2921741786 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f80a8044070 con 0x7f80b0102aa0 2024-04-16T05:33:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.142+0000 7f80a7fff640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f809c0a3bb0 0x7f809c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.142+0000 7f80a7fff640 1 -- 172.21.15.86:0/2921741786 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f80a80f27b0 con 0x7f80b0102aa0 2024-04-16T05:33:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.142+0000 7f80aeffd640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f809c0a3bb0 0x7f809c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.142+0000 7f80aeffd640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f809c0a3bb0 0x7f809c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80a0005c40 tx=0x7f80a0005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.146+0000 7f80a7fff640 1 -- 172.21.15.86:0/2921741786 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80a80bef50 con 0x7f80b0102aa0 2024-04-16T05:33:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.321+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f80b002d620 con 0x7f80b0102aa0 2024-04-16T05:33:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.322+0000 7f80a7fff640 1 -- 172.21.15.86:0/2921741786 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f80a80be850 con 0x7f80b0102aa0 2024-04-16T05:33:08.320 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f809c0a3bb0 msgr2=0x7f809c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f809c0a3bb0 0x7f809c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80a0005c40 tx=0x7f80a0005bd0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 msgr2=0x7f80b019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 0x7f80b019f650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f80a802ce70 tx=0x7f80a8005e40 comp rx=0 tx=0).stop 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 shutdown_connections 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0102aa0 0x7f80b019f650 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f809c0a3bb0 0x7f809c0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80b01020e0 0x7f80b019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 --2- 172.21.15.86:0/2921741786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b0100f10 0x7f80b006b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 >> 172.21.15.86:0/2921741786 conn(0x7f80b00fc4d0 msgr2=0x7f80b01054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 shutdown_connections 2024-04-16T05:33:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.324+0000 7f80b57e1640 1 -- 172.21.15.86:0/2921741786 wait complete. 2024-04-16T05:33:08.334 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:33:08.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.443+0000 7f22a8081640 1 -- 172.21.15.86:0/3576564767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 msgr2=0x7f22a0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.443+0000 7f22a8081640 1 --2- 172.21.15.86:0/3576564767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 0x7f22a0102470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f229c009900 tx=0x7f229c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:08.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.443+0000 7f22a8081640 1 -- 172.21.15.86:0/3576564767 shutdown_connections 2024-04-16T05:33:08.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.443+0000 7f22a8081640 1 --2- 172.21.15.86:0/3576564767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a0103c00 0x7f22a010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.443+0000 7f22a8081640 1 --2- 172.21.15.86:0/3576564767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0103240 0x7f22a01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.443+0000 7f22a8081640 1 --2- 172.21.15.86:0/3576564767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 0x7f22a0102470 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.443+0000 7f22a8081640 1 -- 172.21.15.86:0/3576564767 >> 172.21.15.86:0/3576564767 conn(0x7f22a00fd5e0 msgr2=0x7f22a00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:08.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.443+0000 7f22a8081640 1 -- 172.21.15.86:0/3576564767 shutdown_connections 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.443+0000 7f22a8081640 1 -- 172.21.15.86:0/3576564767 wait complete. 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a8081640 1 Processor -- start 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a8081640 1 -- start start 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a8081640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 0x7f22a0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a8081640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a0103240 0x7f22a0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a8081640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0103c00 0x7f22a019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a8081640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22a010dce0 con 0x7f22a0102050 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a8081640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22a010db60 con 0x7f22a0103240 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a8081640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22a010de60 con 0x7f22a0103c00 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a5e1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 0x7f22a0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a5e1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 0x7f22a0198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56990/0 (socket says 172.21.15.86:56990) 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a5e1d640 1 -- 172.21.15.86:0/114192620 learned_addr learned my addr 172.21.15.86:0/114192620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a561c640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a0103240 0x7f22a0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a661e640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0103c00 0x7f22a019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a5e1d640 1 -- 172.21.15.86:0/114192620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0103c00 msgr2=0x7f22a019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a5e1d640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0103c00 0x7f22a019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a5e1d640 1 -- 172.21.15.86:0/114192620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a0103240 msgr2=0x7f22a0198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a5e1d640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a0103240 0x7f22a0198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a5e1d640 1 -- 172.21.15.86:0/114192620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a019ff30 con 0x7f22a0102050 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a661e640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0103c00 0x7f22a019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a561c640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a0103240 0x7f22a0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.444+0000 7f22a5e1d640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 0x7f22a0198310 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f229c000c00 tx=0x7f229c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.445+0000 7f2296ffd640 1 -- 172.21.15.86:0/114192620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c041070 con 0x7f22a0102050 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.445+0000 7f2296ffd640 1 -- 172.21.15.86:0/114192620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229c0054e0 con 0x7f22a0102050 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.445+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a01a01c0 con 0x7f22a0102050 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.445+0000 7f2296ffd640 1 -- 172.21.15.86:0/114192620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c0346a0 con 0x7f22a0102050 2024-04-16T05:33:08.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.445+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a01a0700 con 0x7f22a0102050 2024-04-16T05:33:08.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.446+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a01093f0 con 0x7f22a0102050 2024-04-16T05:33:08.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.447+0000 7f2296ffd640 1 -- 172.21.15.86:0/114192620 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f229c044070 con 0x7f22a0102050 2024-04-16T05:33:08.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.447+0000 7f2296ffd640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f228c0a3c30 0x7f228c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.448+0000 7f22a561c640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f228c0a3c30 0x7f228c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.448+0000 7f2296ffd640 1 -- 172.21.15.86:0/114192620 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f229c0f25b0 con 0x7f22a0102050 2024-04-16T05:33:08.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.448+0000 7f22a561c640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f228c0a3c30 0x7f228c0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f22a0199820 tx=0x7f229000a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.451+0000 7f2296ffd640 1 -- 172.21.15.86:0/114192620 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f229c0bed50 con 0x7f22a0102050 2024-04-16T05:33:08.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.628+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22a0062c00 con 0x7f22a0102050 2024-04-16T05:33:08.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.629+0000 7f2296ffd640 1 -- 172.21.15.86:0/114192620 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f229c003e20 con 0x7f22a0102050 2024-04-16T05:33:08.628 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f228c0a3c30 msgr2=0x7f228c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f228c0a3c30 0x7f228c0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f22a0199820 tx=0x7f229000a5c0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 msgr2=0x7f22a0198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 0x7f22a0198310 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f229c000c00 tx=0x7f229c004040 comp rx=0 tx=0).stop 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 shutdown_connections 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0103c00 0x7f22a019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f228c0a3c30 0x7f228c0a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a0103240 0x7f22a0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 --2- 172.21.15.86:0/114192620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0102050 0x7f22a0198310 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 >> 172.21.15.86:0/114192620 conn(0x7f22a00fd5e0 msgr2=0x7f22a01065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 shutdown_connections 2024-04-16T05:33:08.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.631+0000 7f22a8081640 1 -- 172.21.15.86:0/114192620 wait complete. 2024-04-16T05:33:08.640 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:08 smithi086 ceph-mon[32028]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-04-16T05:33:08.640 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:08 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4055518454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:08.640 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:08 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2921741786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:33:08.641 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:33:08.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.749+0000 7f1a36ac5640 1 -- 172.21.15.86:0/951927894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30107d20 msgr2=0x7f1a3010a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.749+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/951927894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30107d20 0x7f1a3010a170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009900 tx=0x7f1a1802bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:08.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.749+0000 7f1a36ac5640 1 -- 172.21.15.86:0/951927894 shutdown_connections 2024-04-16T05:33:08.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.749+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/951927894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30107d20 0x7f1a3010a170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.749+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/951927894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a301053f0 0x7f1a301077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.749+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/951927894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a30102ac0 0x7f1a30104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.749+0000 7f1a36ac5640 1 -- 172.21.15.86:0/951927894 >> 172.21.15.86:0/951927894 conn(0x7f1a300fc4b0 msgr2=0x7f1a300fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.749+0000 7f1a36ac5640 1 -- 172.21.15.86:0/951927894 shutdown_connections 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.749+0000 7f1a36ac5640 1 -- 172.21.15.86:0/951927894 wait complete. 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a36ac5640 1 Processor -- start 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a36ac5640 1 -- start start 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a36ac5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30102ac0 0x7f1a30198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a36ac5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301053f0 0x7f1a30198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a36ac5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a30107d20 0x7f1a3019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a36ac5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3010dcd0 con 0x7f1a30102ac0 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a36ac5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3010db50 con 0x7f1a30107d20 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a36ac5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3010de50 con 0x7f1a301053f0 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a34861640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30102ac0 0x7f1a30198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a34861640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30102ac0 0x7f1a30198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57012/0 (socket says 172.21.15.86:57012) 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a34861640 1 -- 172.21.15.86:0/1590535688 learned_addr learned my addr 172.21.15.86:0/1590535688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:08.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a2ffff640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301053f0 0x7f1a30198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a34861640 1 -- 172.21.15.86:0/1590535688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301053f0 msgr2=0x7f1a30198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a35062640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a30107d20 0x7f1a3019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a34861640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301053f0 0x7f1a30198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a34861640 1 -- 172.21.15.86:0/1590535688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a30107d20 msgr2=0x7f1a3019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a34861640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a30107d20 0x7f1a3019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a34861640 1 -- 172.21.15.86:0/1590535688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3019ff50 con 0x7f1a30102ac0 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.750+0000 7f1a35062640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a30107d20 0x7f1a3019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.751+0000 7f1a34861640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30102ac0 0x7f1a30198330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e9f0 tx=0x7f1a2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.751+0000 7f1a2dffb640 1 -- 172.21.15.86:0/1590535688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800cd80 con 0x7f1a30102ac0 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.751+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a301a0240 con 0x7f1a30102ac0 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.751+0000 7f1a2dffb640 1 -- 172.21.15.86:0/1590535688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a28010910 con 0x7f1a30102ac0 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.751+0000 7f1a2dffb640 1 -- 172.21.15.86:0/1590535688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28018990 con 0x7f1a30102ac0 2024-04-16T05:33:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.751+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a301a0790 con 0x7f1a30102ac0 2024-04-16T05:33:08.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.752+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a300fdfa0 con 0x7f1a30102ac0 2024-04-16T05:33:08.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.753+0000 7f1a2dffb640 1 -- 172.21.15.86:0/1590535688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1a28010ac0 con 0x7f1a30102ac0 2024-04-16T05:33:08.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.754+0000 7f1a2dffb640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1a1c0a3bb0 0x7f1a1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:08.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.754+0000 7f1a2ffff640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1a1c0a3bb0 0x7f1a1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:08.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.754+0000 7f1a2dffb640 1 -- 172.21.15.86:0/1590535688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1a280d0ac0 con 0x7f1a30102ac0 2024-04-16T05:33:08.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.754+0000 7f1a2ffff640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1a1c0a3bb0 0x7f1a1c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a30199840 tx=0x7f1a20008230 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:08.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.758+0000 7f1a2dffb640 1 -- 172.21.15.86:0/1590535688 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a2809d260 con 0x7f1a30102ac0 2024-04-16T05:33:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:08 smithi194 ceph-mon[35865]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-04-16T05:33:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:08 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4055518454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:08 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2921741786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:33:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.936+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1a30062c30 con 0x7f1a30102ac0 2024-04-16T05:33:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.937+0000 7f1a2dffb640 1 -- 172.21.15.86:0/1590535688 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f1a2809cb60 con 0x7f1a30102ac0 2024-04-16T05:33:08.935 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:33:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1a1c0a3bb0 msgr2=0x7f1a1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1a1c0a3bb0 0x7f1a1c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a30199840 tx=0x7f1a20008230 comp rx=0 tx=0).stop 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30102ac0 msgr2=0x7f1a30198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30102ac0 0x7f1a30198330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e9f0 tx=0x7f1a2800eeb0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 shutdown_connections 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1a1c0a3bb0 0x7f1a1c0a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a30107d20 0x7f1a3019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301053f0 0x7f1a30198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 --2- 172.21.15.86:0/1590535688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a30102ac0 0x7f1a30198330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 >> 172.21.15.86:0/1590535688 conn(0x7f1a300fc4b0 msgr2=0x7f1a30103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 shutdown_connections 2024-04-16T05:33:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:08.939+0000 7f1a36ac5640 1 -- 172.21.15.86:0/1590535688 wait complete. 2024-04-16T05:33:08.948 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:33:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:08 smithi196 ceph-mon[36373]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-04-16T05:33:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:08 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4055518454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:08 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2921741786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:33:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.057+0000 7fb09090e640 1 -- 172.21.15.86:0/2257285106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c101be0 msgr2=0x7fb08c109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.057+0000 7fb09090e640 1 --2- 172.21.15.86:0/2257285106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c101be0 0x7fb08c109480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb084009ca0 tx=0x7fb08402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.057+0000 7fb09090e640 1 -- 172.21.15.86:0/2257285106 shutdown_connections 2024-04-16T05:33:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.057+0000 7fb09090e640 1 --2- 172.21.15.86:0/2257285106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c101be0 0x7fb08c109480 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.057+0000 7fb09090e640 1 --2- 172.21.15.86:0/2257285106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb08c101220 0x7fb08c1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.057+0000 7fb09090e640 1 --2- 172.21.15.86:0/2257285106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb08c1024f0 0x7fb08c102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.057+0000 7fb09090e640 1 -- 172.21.15.86:0/2257285106 >> 172.21.15.86:0/2257285106 conn(0x7fb08c0fbc30 msgr2=0x7fb08c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.057+0000 7fb09090e640 1 -- 172.21.15.86:0/2257285106 shutdown_connections 2024-04-16T05:33:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.057+0000 7fb09090e640 1 -- 172.21.15.86:0/2257285106 wait complete. 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb09090e640 1 Processor -- start 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb09090e640 1 -- start start 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb09090e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb08c101220 0x7fb08c100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb09090e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb08c101be0 0x7fb08c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb09090e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c1024f0 0x7fb08c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb09090e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c10cbf0 con 0x7fb08c1024f0 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb09090e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c10ca70 con 0x7fb08c101be0 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb09090e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c10cd70 con 0x7fb08c101220 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb08ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c1024f0 0x7fb08c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb08ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c1024f0 0x7fb08c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57036/0 (socket says 172.21.15.86:57036) 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb08ad9d640 1 -- 172.21.15.86:0/3303611041 learned_addr learned my addr 172.21.15.86:0/3303611041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb083fff640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb08c101be0 0x7fb08c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb08a59c640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb08c101220 0x7fb08c100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb08ad9d640 1 -- 172.21.15.86:0/3303611041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb08c101220 msgr2=0x7fb08c100ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb08ad9d640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb08c101220 0x7fb08c100ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb08ad9d640 1 -- 172.21.15.86:0/3303611041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb08c101be0 msgr2=0x7fb08c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb08ad9d640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb08c101be0 0x7fb08c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb08ad9d640 1 -- 172.21.15.86:0/3303611041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c19ff30 con 0x7fb08c1024f0 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.058+0000 7fb083fff640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb08c101be0 0x7fb08c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.059+0000 7fb08ad9d640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c1024f0 0x7fb08c19f670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb08402c700 tx=0x7fb0840346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.059+0000 7fb0837fe640 1 -- 172.21.15.86:0/3303611041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084041020 con 0x7fb08c1024f0 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.059+0000 7fb0837fe640 1 -- 172.21.15.86:0/3303611041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb084032890 con 0x7fb08c1024f0 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.059+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c1a01c0 con 0x7fb08c1024f0 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.059+0000 7fb0837fe640 1 -- 172.21.15.86:0/3303611041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084039040 con 0x7fb08c1024f0 2024-04-16T05:33:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.059+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c1a06b0 con 0x7fb08c1024f0 2024-04-16T05:33:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.060+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08c108300 con 0x7fb08c1024f0 2024-04-16T05:33:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.061+0000 7fb0837fe640 1 -- 172.21.15.86:0/3303611041 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb084044020 con 0x7fb08c1024f0 2024-04-16T05:33:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.061+0000 7fb0837fe640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb0780a3bb0 0x7fb0780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.061+0000 7fb0837fe640 1 -- 172.21.15.86:0/3303611041 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb0840f2730 con 0x7fb08c1024f0 2024-04-16T05:33:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.061+0000 7fb08a59c640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb0780a3bb0 0x7fb0780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.062+0000 7fb08a59c640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb0780a3bb0 0x7fb0780a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005fd0 tx=0x7fb07c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.066+0000 7fb0837fe640 1 -- 172.21.15.86:0/3303611041 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb08404b020 con 0x7fb08c1024f0 2024-04-16T05:33:09.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.242+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb08c062c00 con 0x7fb08c1024f0 2024-04-16T05:33:09.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.242+0000 7fb0837fe640 1 -- 172.21.15.86:0/3303611041 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fb0840beed0 con 0x7fb08c1024f0 2024-04-16T05:33:09.241 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:33:09.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb0780a3bb0 msgr2=0x7fb0780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb0780a3bb0 0x7fb0780a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005fd0 tx=0x7fb07c005850 comp rx=0 tx=0).stop 2024-04-16T05:33:09.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c1024f0 msgr2=0x7fb08c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c1024f0 0x7fb08c19f670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb08402c700 tx=0x7fb0840346f0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 shutdown_connections 2024-04-16T05:33:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c1024f0 0x7fb08c19f670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb0780a3bb0 0x7fb0780a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb08c101be0 0x7fb08c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 --2- 172.21.15.86:0/3303611041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb08c101220 0x7fb08c100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 >> 172.21.15.86:0/3303611041 conn(0x7fb08c0fbc30 msgr2=0x7fb08c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 shutdown_connections 2024-04-16T05:33:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.244+0000 7fb09090e640 1 -- 172.21.15.86:0/3303611041 wait complete. 2024-04-16T05:33:09.254 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T05:33:09.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.360+0000 7fb307088640 1 -- 172.21.15.86:0/4206196274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 msgr2=0x7fb300107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.360+0000 7fb307088640 1 --2- 172.21.15.86:0/4206196274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 0x7fb300107840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009900 tx=0x7fb2ec02bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:09.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.360+0000 7fb307088640 1 -- 172.21.15.86:0/4206196274 shutdown_connections 2024-04-16T05:33:09.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.360+0000 7fb307088640 1 --2- 172.21.15.86:0/4206196274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb300107d80 0x7fb30010a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.360+0000 7fb307088640 1 --2- 172.21.15.86:0/4206196274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 0x7fb300107840 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.360+0000 7fb307088640 1 --2- 172.21.15.86:0/4206196274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb300102b20 0x7fb300104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.360+0000 7fb307088640 1 -- 172.21.15.86:0/4206196274 >> 172.21.15.86:0/4206196274 conn(0x7fb3000fc4b0 msgr2=0x7fb3000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.360+0000 7fb307088640 1 -- 172.21.15.86:0/4206196274 shutdown_connections 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.360+0000 7fb307088640 1 -- 172.21.15.86:0/4206196274 wait complete. 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb307088640 1 Processor -- start 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb307088640 1 -- start start 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb307088640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb300102b20 0x7fb300198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb307088640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 0x7fb3001989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb307088640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb300107d80 0x7fb30019fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb307088640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb30010dd90 con 0x7fb300105450 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb307088640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb30010dc10 con 0x7fb300107d80 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb307088640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb30010df10 con 0x7fb300102b20 2024-04-16T05:33:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb2fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 0x7fb3001989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb2fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 0x7fb3001989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57056/0 (socket says 172.21.15.86:57056) 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb2fffff640 1 -- 172.21.15.86:0/3263444394 learned_addr learned my addr 172.21.15.86:0/3263444394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb304e24640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb300102b20 0x7fb300198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb2fffff640 1 -- 172.21.15.86:0/3263444394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb300102b20 msgr2=0x7fb300198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb2fffff640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb300102b20 0x7fb300198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb2fffff640 1 -- 172.21.15.86:0/3263444394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb300107d80 msgr2=0x7fb30019fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb305625640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb300107d80 0x7fb30019fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb2fffff640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb300107d80 0x7fb30019fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb2fffff640 1 -- 172.21.15.86:0/3263444394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3001a00b0 con 0x7fb300105450 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb304e24640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb300102b20 0x7fb300198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.361+0000 7fb305625640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb300107d80 0x7fb30019fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.362+0000 7fb2fffff640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 0x7fb3001989d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000c00 tx=0x7fb2ec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.362+0000 7fb2fdffb640 1 -- 172.21.15.86:0/3263444394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec041070 con 0x7fb300105450 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.362+0000 7fb2fdffb640 1 -- 172.21.15.86:0/3263444394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2ec0054e0 con 0x7fb300105450 2024-04-16T05:33:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.362+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3001a0340 con 0x7fb300105450 2024-04-16T05:33:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.362+0000 7fb2fdffb640 1 -- 172.21.15.86:0/3263444394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0346a0 con 0x7fb300105450 2024-04-16T05:33:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.362+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3001a0830 con 0x7fb300105450 2024-04-16T05:33:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.363+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3000fdfa0 con 0x7fb300105450 2024-04-16T05:33:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.364+0000 7fb2fdffb640 1 -- 172.21.15.86:0/3263444394 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb2ec044070 con 0x7fb300105450 2024-04-16T05:33:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.365+0000 7fb2fdffb640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2f00a3bb0 0x7fb2f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.365+0000 7fb304e24640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2f00a3bb0 0x7fb2f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.365+0000 7fb2fdffb640 1 -- 172.21.15.86:0/3263444394 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb2ec0f2500 con 0x7fb300105450 2024-04-16T05:33:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.365+0000 7fb304e24640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2f00a3bb0 0x7fb2f00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4006fd0 tx=0x7fb2f4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:09.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.369+0000 7fb2fdffb640 1 -- 172.21.15.86:0/3263444394 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2ec0beca0 con 0x7fb300105450 2024-04-16T05:33:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.562+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb30002d620 con 0x7fb300105450 2024-04-16T05:33:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.563+0000 7fb2fdffb640 1 -- 172.21.15.86:0/3263444394 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+590 (secure 0 0 0) 0x7fb2ec003e20 con 0x7fb300105450 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1016266433 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/908662426 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1420225001 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3914995291 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2966713555 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3037484112 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1420225001 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1904754808 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1904754808 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/661459728 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-04-16T05:33:09.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2f00a3bb0 msgr2=0x7fb2f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2f00a3bb0 0x7fb2f00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4006fd0 tx=0x7fb2f4008040 comp rx=0 tx=0).stop 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 msgr2=0x7fb3001989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 0x7fb3001989d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000c00 tx=0x7fb2ec004040 comp rx=0 tx=0).stop 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 shutdown_connections 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2f00a3bb0 0x7fb2f00a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb300107d80 0x7fb30019fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb300105450 0x7fb3001989d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 --2- 172.21.15.86:0/3263444394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb300102b20 0x7fb300198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 >> 172.21.15.86:0/3263444394 conn(0x7fb3000fc4b0 msgr2=0x7fb3001033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 shutdown_connections 2024-04-16T05:33:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.565+0000 7fb307088640 1 -- 172.21.15.86:0/3263444394 wait complete. 2024-04-16T05:33:09.574 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:33:09.574 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:33:09.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:33:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.737+0000 7eff9b59e640 1 -- 172.21.15.86:0/4190582993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 msgr2=0x7eff9c101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.737+0000 7eff9b59e640 1 --2- 172.21.15.86:0/4190582993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 0x7eff9c101190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009900 tx=0x7eff8c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.737+0000 7eff9b59e640 1 -- 172.21.15.86:0/4190582993 shutdown_connections 2024-04-16T05:33:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.737+0000 7eff9b59e640 1 --2- 172.21.15.86:0/4190582993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff9c102920 0x7eff9c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.737+0000 7eff9b59e640 1 --2- 172.21.15.86:0/4190582993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff9c101f60 0x7eff9c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.737+0000 7eff9b59e640 1 --2- 172.21.15.86:0/4190582993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 0x7eff9c101190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.737+0000 7eff9b59e640 1 -- 172.21.15.86:0/4190582993 >> 172.21.15.86:0/4190582993 conn(0x7eff9c0fc2e0 msgr2=0x7eff9c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.737+0000 7eff9b59e640 1 -- 172.21.15.86:0/4190582993 shutdown_connections 2024-04-16T05:33:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.737+0000 7eff9b59e640 1 -- 172.21.15.86:0/4190582993 wait complete. 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9b59e640 1 Processor -- start 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9b59e640 1 -- start start 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 0x7eff9c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9b59e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff9c101f60 0x7eff9c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9b59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff9c102920 0x7eff9c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9b59e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c10c9c0 con 0x7eff9c100d70 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9b59e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c10c840 con 0x7eff9c101f60 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9b59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c10cb40 con 0x7eff9c102920 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 0x7eff9c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 0x7eff9c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57086/0 (socket says 172.21.15.86:57086) 2024-04-16T05:33:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9a59c640 1 -- 172.21.15.86:0/738682214 learned_addr learned my addr 172.21.15.86:0/738682214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9ad9d640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff9c102920 0x7eff9c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff99d9b640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff9c101f60 0x7eff9c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9a59c640 1 -- 172.21.15.86:0/738682214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff9c102920 msgr2=0x7eff9c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.738+0000 7eff9a59c640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff9c102920 0x7eff9c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff9a59c640 1 -- 172.21.15.86:0/738682214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff9c101f60 msgr2=0x7eff9c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff9a59c640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff9c101f60 0x7eff9c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff9a59c640 1 -- 172.21.15.86:0/738682214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9c19fde0 con 0x7eff9c100d70 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff9ad9d640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff9c102920 0x7eff9c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff9a59c640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 0x7eff9c1981c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7eff8c005ab0 tx=0x7eff8c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff937fe640 1 -- 172.21.15.86:0/738682214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c041070 con 0x7eff9c100d70 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff937fe640 1 -- 172.21.15.86:0/738682214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8c0054e0 con 0x7eff9c100d70 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff9c1a0070 con 0x7eff9c100d70 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff937fe640 1 -- 172.21.15.86:0/738682214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c034440 con 0x7eff9c100d70 2024-04-16T05:33:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.739+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff9c1a0560 con 0x7eff9c100d70 2024-04-16T05:33:09.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.740+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff9c1a07c0 con 0x7eff9c100d70 2024-04-16T05:33:09.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.741+0000 7eff937fe640 1 -- 172.21.15.86:0/738682214 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff8c044070 con 0x7eff9c100d70 2024-04-16T05:33:09.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.742+0000 7eff937fe640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7eff880a3bb0 0x7eff880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:09.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.742+0000 7eff99d9b640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7eff880a3bb0 0x7eff880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:09.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.742+0000 7eff937fe640 1 -- 172.21.15.86:0/738682214 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7eff8c0f2340 con 0x7eff9c100d70 2024-04-16T05:33:09.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.742+0000 7eff99d9b640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7eff880a3bb0 0x7eff880a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7eff84005fd0 tx=0x7eff84005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:09.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.746+0000 7eff937fe640 1 -- 172.21.15.86:0/738682214 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff8c0beae0 con 0x7eff9c100d70 2024-04-16T05:33:09.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:09 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/114192620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:33:09.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:09 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1590535688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:33:09.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:09 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3303611041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:33:09.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.925+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff9c1a0990 con 0x7eff9c100d70 2024-04-16T05:33:09.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.927+0000 7eff937fe640 1 -- 172.21.15.86:0/738682214 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24080 (secure 0 0 0) 0x7eff8c0be3e0 con 0x7eff9c100d70 2024-04-16T05:33:09.926 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:09.926 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:01.527267+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7eff880a3bb0 msgr2=0x7eff880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7eff880a3bb0 0x7eff880a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7eff84005fd0 tx=0x7eff84005850 comp rx=0 tx=0).stop 2024-04-16T05:33:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 msgr2=0x7eff9c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 0x7eff9c1981c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7eff8c005ab0 tx=0x7eff8c005dc0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 shutdown_connections 2024-04-16T05:33:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff9c102920 0x7eff9c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7eff880a3bb0 0x7eff880a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff9c101f60 0x7eff9c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 --2- 172.21.15.86:0/738682214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff9c100d70 0x7eff9c1981c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:09.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 >> 172.21.15.86:0/738682214 conn(0x7eff9c0fc2e0 msgr2=0x7eff9c06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:09.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 shutdown_connections 2024-04-16T05:33:09.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:09.929+0000 7eff9b59e640 1 -- 172.21.15.86:0/738682214 wait complete. 2024-04-16T05:33:09.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:09 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/114192620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:33:09.938 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:09 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1590535688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:33:09.938 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:09 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3303611041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:33:09.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:33:09.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:33:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:09 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/114192620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:33:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:09 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1590535688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:33:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:09 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3303611041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:33:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 -- 172.21.15.86:0/2612073175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 msgr2=0x7f698410a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 --2- 172.21.15.86:0/2612073175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 0x7f698410a170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f697c009ca0 tx=0x7f697c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 -- 172.21.15.86:0/2612073175 shutdown_connections 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 --2- 172.21.15.86:0/2612073175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 0x7f698410a170 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 --2- 172.21.15.86:0/2612073175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69841053f0 0x7f69841077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 --2- 172.21.15.86:0/2612073175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6984102ac0 0x7f6984104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 -- 172.21.15.86:0/2612073175 >> 172.21.15.86:0/2612073175 conn(0x7f69840fc4b0 msgr2=0x7f69840fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 -- 172.21.15.86:0/2612073175 shutdown_connections 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 -- 172.21.15.86:0/2612073175 wait complete. 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 Processor -- start 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 -- start start 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6984102ac0 0x7f69841982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69841053f0 0x7f6984198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 0x7f698419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f698410dcd0 con 0x7f6984107d20 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f698410db50 con 0x7f6984102ac0 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.048+0000 7f698a796640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f698410de50 con 0x7f69841053f0 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6988d33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 0x7f698419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6988d33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 0x7f698419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57112/0 (socket says 172.21.15.86:57112) 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6988d33640 1 -- 172.21.15.86:0/3133707963 learned_addr learned my addr 172.21.15.86:0/3133707963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6983fff640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6984102ac0 0x7f69841982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f697bfff640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69841053f0 0x7f6984198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6988d33640 1 -- 172.21.15.86:0/3133707963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69841053f0 msgr2=0x7f6984198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6988d33640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69841053f0 0x7f6984198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6988d33640 1 -- 172.21.15.86:0/3133707963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6984102ac0 msgr2=0x7f69841982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6988d33640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6984102ac0 0x7f69841982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6988d33640 1 -- 172.21.15.86:0/3133707963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698419ff00 con 0x7f6984107d20 2024-04-16T05:33:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6988d33640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 0x7f698419f890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f697c005e30 tx=0x7f697c032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6981ffb640 1 -- 172.21.15.86:0/3133707963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c041020 con 0x7f6984107d20 2024-04-16T05:33:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6981ffb640 1 -- 172.21.15.86:0/3133707963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697c034cd0 con 0x7f6984107d20 2024-04-16T05:33:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69841a0190 con 0x7f6984107d20 2024-04-16T05:33:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f6981ffb640 1 -- 172.21.15.86:0/3133707963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c039040 con 0x7f6984107d20 2024-04-16T05:33:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.049+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841a0680 con 0x7f6984107d20 2024-04-16T05:33:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.050+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69840fdfa0 con 0x7f6984107d20 2024-04-16T05:33:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.051+0000 7f6981ffb640 1 -- 172.21.15.86:0/3133707963 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f697c044020 con 0x7f6984107d20 2024-04-16T05:33:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.052+0000 7f6981ffb640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69700a3bb0 0x7f69700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.052+0000 7f6983fff640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69700a3bb0 0x7f69700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.052+0000 7f6981ffb640 1 -- 172.21.15.86:0/3133707963 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f697c0f25d0 con 0x7f6984107d20 2024-04-16T05:33:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.053+0000 7f6983fff640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69700a3bb0 0x7f69700a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6974005fd0 tx=0x7f697400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.057+0000 7f6981ffb640 1 -- 172.21.15.86:0/3133707963 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f697c04b020 con 0x7f6984107d20 2024-04-16T05:33:10.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.235+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6984062c00 con 0x7f6984107d20 2024-04-16T05:33:10.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.235+0000 7f6981ffb640 1 -- 172.21.15.86:0/3133707963 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f697c0bed70 con 0x7f6984107d20 2024-04-16T05:33:10.234 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:33:10.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.237+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69700a3bb0 msgr2=0x7f69700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.237+0000 7f698a796640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69700a3bb0 0x7f69700a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6974005fd0 tx=0x7f697400b390 comp rx=0 tx=0).stop 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.237+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 msgr2=0x7f698419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.237+0000 7f698a796640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 0x7f698419f890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f697c005e30 tx=0x7f697c032ec0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.238+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 shutdown_connections 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.238+0000 7f698a796640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6984107d20 0x7f698419f890 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.238+0000 7f698a796640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69841053f0 0x7f6984198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.238+0000 7f698a796640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69700a3bb0 0x7f69700a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.238+0000 7f698a796640 1 --2- 172.21.15.86:0/3133707963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6984102ac0 0x7f69841982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.238+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 >> 172.21.15.86:0/3133707963 conn(0x7f69840fc4b0 msgr2=0x7f6984103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.238+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 shutdown_connections 2024-04-16T05:33:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.238+0000 7f698a796640 1 -- 172.21.15.86:0/3133707963 wait complete. 2024-04-16T05:33:10.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:33:10.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.355+0000 7f0536d67640 1 -- 172.21.15.86:0/3546838716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 msgr2=0x7f0530109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.355+0000 7f0536d67640 1 --2- 172.21.15.86:0/3546838716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 0x7f0530109370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0528009ca0 tx=0x7f052802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.355+0000 7f0536d67640 1 -- 172.21.15.86:0/3546838716 shutdown_connections 2024-04-16T05:33:10.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.355+0000 7f0536d67640 1 --2- 172.21.15.86:0/3546838716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 0x7f0530109370 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.355+0000 7f0536d67640 1 --2- 172.21.15.86:0/3546838716 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0530102090 0x7f0530102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.355+0000 7f0536d67640 1 --2- 172.21.15.86:0/3546838716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530100f40 0x7f0530101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.355+0000 7f0536d67640 1 -- 172.21.15.86:0/3546838716 >> 172.21.15.86:0/3546838716 conn(0x7f05300fc4f0 msgr2=0x7f05300fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.355+0000 7f0536d67640 1 -- 172.21.15.86:0/3546838716 shutdown_connections 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.355+0000 7f0536d67640 1 -- 172.21.15.86:0/3546838716 wait complete. 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0536d67640 1 Processor -- start 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0536d67640 1 -- start start 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0536d67640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530100f40 0x7f0530198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0536d67640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0530102090 0x7f0530198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0536d67640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 0x7f053019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0536d67640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f053010dd30 con 0x7f0530102a50 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0536d67640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f053010dbb0 con 0x7f0530102090 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0536d67640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f053010deb0 con 0x7f0530100f40 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0535304640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 0x7f053019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0535304640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 0x7f053019f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57140/0 (socket says 172.21.15.86:57140) 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0535304640 1 -- 172.21.15.86:0/1572291701 learned_addr learned my addr 172.21.15.86:0/1572291701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0534b03640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530100f40 0x7f0530198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f052ffff640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0530102090 0x7f0530198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0535304640 1 -- 172.21.15.86:0/1572291701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530100f40 msgr2=0x7f0530198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0535304640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530100f40 0x7f0530198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0535304640 1 -- 172.21.15.86:0/1572291701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0530102090 msgr2=0x7f0530198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0535304640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0530102090 0x7f0530198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0535304640 1 -- 172.21.15.86:0/1572291701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053019ff20 con 0x7f0530102a50 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f052ffff640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0530102090 0x7f0530198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.356+0000 7f0535304640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 0x7f053019f8b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f052802c700 tx=0x7f0528032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.357+0000 7f052dffb640 1 -- 172.21.15.86:0/1572291701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528039040 con 0x7f0530102a50 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.357+0000 7f052dffb640 1 -- 172.21.15.86:0/1572291701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0528044070 con 0x7f0530102a50 2024-04-16T05:33:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.357+0000 7f052dffb640 1 -- 172.21.15.86:0/1572291701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05280042a0 con 0x7f0530102a50 2024-04-16T05:33:10.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.357+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05301a01b0 con 0x7f0530102a50 2024-04-16T05:33:10.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.357+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05301a06a0 con 0x7f0530102a50 2024-04-16T05:33:10.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.358+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0530108250 con 0x7f0530102a50 2024-04-16T05:33:10.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.359+0000 7f052dffb640 1 -- 172.21.15.86:0/1572291701 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0528003970 con 0x7f0530102a50 2024-04-16T05:33:10.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.360+0000 7f052dffb640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f05200a3bb0 0x7f05200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.360+0000 7f0534b03640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f05200a3bb0 0x7f05200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.360+0000 7f052dffb640 1 -- 172.21.15.86:0/1572291701 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f05280f31a0 con 0x7f0530102a50 2024-04-16T05:33:10.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.360+0000 7f0534b03640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f05200a3bb0 0x7f05200a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0524005fd0 tx=0x7f052400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.363+0000 7f052dffb640 1 -- 172.21.15.86:0/1572291701 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05280bf9d0 con 0x7f0530102a50 2024-04-16T05:33:10.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.543+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f053002d620 con 0x7f0530102a50 2024-04-16T05:33:10.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.544+0000 7f052dffb640 1 -- 172.21.15.86:0/1572291701 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f05280bf2d0 con 0x7f0530102a50 2024-04-16T05:33:10.542 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.545+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f05200a3bb0 msgr2=0x7f05200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f05200a3bb0 0x7f05200a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0524005fd0 tx=0x7f052400b390 comp rx=0 tx=0).stop 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 msgr2=0x7f053019f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 0x7f053019f8b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f052802c700 tx=0x7f0528032f20 comp rx=0 tx=0).stop 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 shutdown_connections 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0530102a50 0x7f053019f8b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f05200a3bb0 0x7f05200a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0530102090 0x7f0530198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 --2- 172.21.15.86:0/1572291701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530100f40 0x7f0530198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 >> 172.21.15.86:0/1572291701 conn(0x7f05300fc4f0 msgr2=0x7f0530105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 shutdown_connections 2024-04-16T05:33:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.546+0000 7f0536d67640 1 -- 172.21.15.86:0/1572291701 wait complete. 2024-04-16T05:33:10.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:33:10.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.664+0000 7f0d05500640 1 -- 172.21.15.86:0/1841570850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 msgr2=0x7f0d00102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.664+0000 7f0d05500640 1 --2- 172.21.15.86:0/1841570850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 0x7f0d00102510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009900 tx=0x7f0ce802bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:10.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.664+0000 7f0d05500640 1 -- 172.21.15.86:0/1841570850 shutdown_connections 2024-04-16T05:33:10.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.664+0000 7f0d05500640 1 --2- 172.21.15.86:0/1841570850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d00102a50 0x7f0d00109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.664+0000 7f0d05500640 1 --2- 172.21.15.86:0/1841570850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 0x7f0d00102510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.664+0000 7f0d05500640 1 --2- 172.21.15.86:0/1841570850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d00100f40 0x7f0d00101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.664+0000 7f0d05500640 1 -- 172.21.15.86:0/1841570850 >> 172.21.15.86:0/1841570850 conn(0x7f0d000fc4f0 msgr2=0x7f0d000fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.664+0000 7f0d05500640 1 -- 172.21.15.86:0/1841570850 shutdown_connections 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.664+0000 7f0d05500640 1 -- 172.21.15.86:0/1841570850 wait complete. 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0d05500640 1 Processor -- start 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0d05500640 1 -- start start 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0d05500640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d00100f40 0x7f0d0006b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0d05500640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 0x7f0d0019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0d05500640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d00102a50 0x7f0d0019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0d05500640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0010cbd0 con 0x7f0d00102090 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0d05500640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0010ca50 con 0x7f0d00100f40 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0d05500640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0010cd50 con 0x7f0d00102a50 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 0x7f0d0019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 0x7f0d0019d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46134/0 (socket says 172.21.15.86:46134) 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0cf7fff640 1 -- 172.21.15.86:0/268760534 learned_addr learned my addr 172.21.15.86:0/268760534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0cff7fe640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d00102a50 0x7f0d0019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.665+0000 7f0cfeffd640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d00100f40 0x7f0d0006b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0cf7fff640 1 -- 172.21.15.86:0/268760534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d00102a50 msgr2=0x7f0d0019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0cf7fff640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d00102a50 0x7f0d0019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0cf7fff640 1 -- 172.21.15.86:0/268760534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d00100f40 msgr2=0x7f0d0006b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0cf7fff640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d00100f40 0x7f0d0006b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0cf7fff640 1 -- 172.21.15.86:0/268760534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0019ffd0 con 0x7f0d00102090 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0cf7fff640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 0x7f0d0019d0d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005ab0 tx=0x7f0ce8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0cfcff9640 1 -- 172.21.15.86:0/268760534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8041070 con 0x7f0d00102090 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0cfcff9640 1 -- 172.21.15.86:0/268760534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce80054e0 con 0x7f0d00102090 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d001a0260 con 0x7f0d00102090 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0cfcff9640 1 -- 172.21.15.86:0/268760534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8034440 con 0x7f0d00102090 2024-04-16T05:33:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.666+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d001a0750 con 0x7f0d00102090 2024-04-16T05:33:10.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.667+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d001082a0 con 0x7f0d00102090 2024-04-16T05:33:10.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.668+0000 7f0cfcff9640 1 -- 172.21.15.86:0/268760534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ce8044070 con 0x7f0d00102090 2024-04-16T05:33:10.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.669+0000 7f0cfcff9640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0cec0a3bb0 0x7f0cec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.669+0000 7f0cfeffd640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0cec0a3bb0 0x7f0cec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.669+0000 7f0cfcff9640 1 -- 172.21.15.86:0/268760534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0ce80f21e0 con 0x7f0d00102090 2024-04-16T05:33:10.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.669+0000 7f0cfeffd640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0cec0a3bb0 0x7f0cec0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0005d00 tx=0x7f0cf000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.673+0000 7f0cfcff9640 1 -- 172.21.15.86:0/268760534 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ce80be330 con 0x7f0d00102090 2024-04-16T05:33:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.850+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d00062c00 con 0x7f0d00102090 2024-04-16T05:33:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.850+0000 7f0cfcff9640 1 -- 172.21.15.86:0/268760534 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f0ce80c2630 con 0x7f0d00102090 2024-04-16T05:33:10.849 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.852+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0cec0a3bb0 msgr2=0x7f0cec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.852+0000 7f0d05500640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0cec0a3bb0 0x7f0cec0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0005d00 tx=0x7f0cf000b410 comp rx=0 tx=0).stop 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.852+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 msgr2=0x7f0d0019d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.852+0000 7f0d05500640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 0x7f0d0019d0d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005ab0 tx=0x7f0ce8005dc0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.852+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 shutdown_connections 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.852+0000 7f0d05500640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d00102a50 0x7f0d0019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.853+0000 7f0d05500640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d00102090 0x7f0d0019d0d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.853+0000 7f0d05500640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0cec0a3bb0 0x7f0cec0a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.853+0000 7f0d05500640 1 --2- 172.21.15.86:0/268760534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d00100f40 0x7f0d0006b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.853+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 >> 172.21.15.86:0/268760534 conn(0x7f0d000fc4f0 msgr2=0x7f0d00105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.853+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 shutdown_connections 2024-04-16T05:33:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.853+0000 7f0d05500640 1 -- 172.21.15.86:0/268760534 wait complete. 2024-04-16T05:33:10.861 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:10 smithi086 ceph-mon[32028]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:33:10.861 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:10 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3263444394' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:33:10.862 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:10 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/738682214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:10.862 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:10 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3133707963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:33:10.862 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:10 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1572291701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:33:10.862 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:33:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:10 smithi194 ceph-mon[35865]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:33:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:10 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3263444394' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:33:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:10 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/738682214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:10 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3133707963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:33:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:10 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1572291701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:33:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.971+0000 7f794dd9d640 1 -- 172.21.15.86:0/343124829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948100e30 msgr2=0x7f7948101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.971+0000 7f794dd9d640 1 --2- 172.21.15.86:0/343124829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948100e30 0x7f7948101250 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f793c009900 tx=0x7f793c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.971+0000 7f794dd9d640 1 -- 172.21.15.86:0/343124829 shutdown_connections 2024-04-16T05:33:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.971+0000 7f794dd9d640 1 --2- 172.21.15.86:0/343124829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7948102940 0x7f7948109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.971+0000 7f794dd9d640 1 --2- 172.21.15.86:0/343124829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7948101f80 0x7f7948102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.971+0000 7f794dd9d640 1 --2- 172.21.15.86:0/343124829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948100e30 0x7f7948101250 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.971+0000 7f794dd9d640 1 -- 172.21.15.86:0/343124829 >> 172.21.15.86:0/343124829 conn(0x7f79480fc3e0 msgr2=0x7f79480fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.971+0000 7f794dd9d640 1 -- 172.21.15.86:0/343124829 shutdown_connections 2024-04-16T05:33:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.971+0000 7f794dd9d640 1 -- 172.21.15.86:0/343124829 wait complete. 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794dd9d640 1 Processor -- start 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794dd9d640 1 -- start start 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794dd9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7948100e30 0x7f7948197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794dd9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7948101f80 0x7f7948197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794dd9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948102940 0x7f794819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794dd9d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f794810cad0 con 0x7f7948102940 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794dd9d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f794810c950 con 0x7f7948100e30 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794dd9d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f794810cc50 con 0x7f7948101f80 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794d59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948102940 0x7f794819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794d59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948102940 0x7f794819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46142/0 (socket says 172.21.15.86:46142) 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794d59c640 1 -- 172.21.15.86:0/2510865333 learned_addr learned my addr 172.21.15.86:0/2510865333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f7947fff640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7948101f80 0x7f7948197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794cd9b640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7948100e30 0x7f7948197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794d59c640 1 -- 172.21.15.86:0/2510865333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7948101f80 msgr2=0x7f7948197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794d59c640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7948101f80 0x7f7948197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794d59c640 1 -- 172.21.15.86:0/2510865333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7948100e30 msgr2=0x7f7948197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794d59c640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7948100e30 0x7f7948197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794d59c640 1 -- 172.21.15.86:0/2510865333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794819ed30 con 0x7f7948102940 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794cd9b640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7948100e30 0x7f7948197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f7947fff640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7948101f80 0x7f7948197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.972+0000 7f794d59c640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948102940 0x7f794819e6c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f794000eaa0 tx=0x7f794000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.973+0000 7f7945ffb640 1 -- 172.21.15.86:0/2510865333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794000cdb0 con 0x7f7948102940 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.973+0000 7f7945ffb640 1 -- 172.21.15.86:0/2510865333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7940005020 con 0x7f7948102940 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.973+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794819efc0 con 0x7f7948102940 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.973+0000 7f7945ffb640 1 -- 172.21.15.86:0/2510865333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79400116b0 con 0x7f7948102940 2024-04-16T05:33:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.973+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794819f530 con 0x7f7948102940 2024-04-16T05:33:10.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.974+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7948108140 con 0x7f7948102940 2024-04-16T05:33:10.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.975+0000 7f7945ffb640 1 -- 172.21.15.86:0/2510865333 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7940005560 con 0x7f7948102940 2024-04-16T05:33:10.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.975+0000 7f7945ffb640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f79380a3c30 0x7f79380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.975+0000 7f794cd9b640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f79380a3c30 0x7f79380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.975+0000 7f7945ffb640 1 -- 172.21.15.86:0/2510865333 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f79400d0890 con 0x7f7948102940 2024-04-16T05:33:10.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.976+0000 7f794cd9b640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f79380a3c30 0x7f79380a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f793c0096c0 tx=0x7f793c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:10.980+0000 7f7945ffb640 1 -- 172.21.15.86:0/2510865333 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7940016030 con 0x7f7948102940 2024-04-16T05:33:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:10 smithi196 ceph-mon[36373]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:33:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:10 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3263444394' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:33:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:10 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/738682214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:10 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3133707963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:33:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:10 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1572291701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:33:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.157+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7948062c00 con 0x7f7948102940 2024-04-16T05:33:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.157+0000 7f7945ffb640 1 -- 172.21.15.86:0/2510865333 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f794009d030 con 0x7f7948102940 2024-04-16T05:33:11.156 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:33:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f79380a3c30 msgr2=0x7f79380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f79380a3c30 0x7f79380a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f793c0096c0 tx=0x7f793c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948102940 msgr2=0x7f794819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948102940 0x7f794819e6c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f794000eaa0 tx=0x7f794000ef60 comp rx=0 tx=0).stop 2024-04-16T05:33:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 shutdown_connections 2024-04-16T05:33:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7948102940 0x7f794819e6c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7948101f80 0x7f7948197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f79380a3c30 0x7f79380a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 --2- 172.21.15.86:0/2510865333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7948100e30 0x7f7948197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 >> 172.21.15.86:0/2510865333 conn(0x7f79480fc3e0 msgr2=0x7f7948105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 shutdown_connections 2024-04-16T05:33:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.159+0000 7f794dd9d640 1 -- 172.21.15.86:0/2510865333 wait complete. 2024-04-16T05:33:11.169 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T05:33:11.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.275+0000 7fe39efe2640 1 -- 172.21.15.86:0/4145562397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 msgr2=0x7fe398101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.275+0000 7fe39efe2640 1 --2- 172.21.15.86:0/4145562397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 0x7fe398101360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe394009900 tx=0x7fe39402bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:11.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.275+0000 7fe39efe2640 1 -- 172.21.15.86:0/4145562397 shutdown_connections 2024-04-16T05:33:11.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.275+0000 7fe39efe2640 1 --2- 172.21.15.86:0/4145562397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe398102a50 0x7fe398109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.275+0000 7fe39efe2640 1 --2- 172.21.15.86:0/4145562397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe398102090 0x7fe398102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.275+0000 7fe39efe2640 1 --2- 172.21.15.86:0/4145562397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 0x7fe398101360 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.275+0000 7fe39efe2640 1 -- 172.21.15.86:0/4145562397 >> 172.21.15.86:0/4145562397 conn(0x7fe3980fc4b0 msgr2=0x7fe3980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.275+0000 7fe39efe2640 1 -- 172.21.15.86:0/4145562397 shutdown_connections 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.275+0000 7fe39efe2640 1 -- 172.21.15.86:0/4145562397 wait complete. 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39efe2640 1 Processor -- start 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39efe2640 1 -- start start 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39efe2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 0x7fe398069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39efe2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe398102090 0x7fe39819d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39efe2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe398102a50 0x7fe39819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39efe2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe39810cc70 con 0x7fe398100f40 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39efe2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe39810caf0 con 0x7fe398102a50 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39efe2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe39810cdf0 con 0x7fe398102090 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39cd7e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 0x7fe398069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39cd7e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 0x7fe398069d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46158/0 (socket says 172.21.15.86:46158) 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe39cd7e640 1 -- 172.21.15.86:0/2674013385 learned_addr learned my addr 172.21.15.86:0/2674013385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.276+0000 7fe38ffff640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe398102090 0x7fe39819d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe39cd7e640 1 -- 172.21.15.86:0/2674013385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe398102090 msgr2=0x7fe39819d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe39cd7e640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe398102090 0x7fe39819d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe39cd7e640 1 -- 172.21.15.86:0/2674013385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe398102a50 msgr2=0x7fe39819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe39cd7e640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe398102a50 0x7fe39819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe39cd7e640 1 -- 172.21.15.86:0/2674013385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3981a0090 con 0x7fe398100f40 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe39cd7e640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 0x7fe398069d00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe394009230 tx=0x7fe394005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe38dffb640 1 -- 172.21.15.86:0/2674013385 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394041070 con 0x7fe398100f40 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe38dffb640 1 -- 172.21.15.86:0/2674013385 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3940054e0 con 0x7fe398100f40 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe38dffb640 1 -- 172.21.15.86:0/2674013385 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394034440 con 0x7fe398100f40 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3981a02c0 con 0x7fe398100f40 2024-04-16T05:33:11.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.277+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3981a07b0 con 0x7fe398100f40 2024-04-16T05:33:11.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.278+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3981082d0 con 0x7fe398100f40 2024-04-16T05:33:11.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.279+0000 7fe38dffb640 1 -- 172.21.15.86:0/2674013385 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe394044070 con 0x7fe398100f40 2024-04-16T05:33:11.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.280+0000 7fe38dffb640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3800a3bb0 0x7fe3800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:11.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.280+0000 7fe38ffff640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3800a3bb0 0x7fe3800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:11.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.280+0000 7fe38dffb640 1 -- 172.21.15.86:0/2674013385 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe3940f23d0 con 0x7fe398100f40 2024-04-16T05:33:11.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.280+0000 7fe38ffff640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3800a3bb0 0x7fe3800a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe398069a70 tx=0x7fe38800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:11.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.284+0000 7fe38dffb640 1 -- 172.21.15.86:0/2674013385 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3940beb70 con 0x7fe398100f40 2024-04-16T05:33:11.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.472+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe398062ca0 con 0x7fe398100f40 2024-04-16T05:33:11.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.473+0000 7fe38dffb640 1 -- 172.21.15.86:0/2674013385 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+590 (secure 0 0 0) 0x7fe3940be470 con 0x7fe398100f40 2024-04-16T05:33:11.471 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1016266433 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/908662426 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1420225001 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3914995291 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2966713555 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3037484112 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1420225001 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1904754808 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1904754808 2024-04-17T05:25:24.654093+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/661459728 2024-04-17T05:25:51.394705+0000 2024-04-16T05:33:11.472 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3800a3bb0 msgr2=0x7fe3800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3800a3bb0 0x7fe3800a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe398069a70 tx=0x7fe38800b390 comp rx=0 tx=0).stop 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 msgr2=0x7fe398069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 0x7fe398069d00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe394009230 tx=0x7fe394005dd0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 shutdown_connections 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe3800a3bb0 0x7fe3800a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe398102a50 0x7fe39819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe398102090 0x7fe39819d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 --2- 172.21.15.86:0/2674013385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe398100f40 0x7fe398069d00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 >> 172.21.15.86:0/2674013385 conn(0x7fe3980fc4b0 msgr2=0x7fe3981054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 shutdown_connections 2024-04-16T05:33:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.475+0000 7fe39efe2640 1 -- 172.21.15.86:0/2674013385 wait complete. 2024-04-16T05:33:11.484 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-16T05:33:11.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:11.484 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:33:11.512 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[73314]: fuse finished with error 0 and tester_r 0 2024-04-16T05:33:11.514 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T05:33:11.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:33:11.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:33:11.547 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-16T05:33:11.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.758+0000 7f561eff4640 1 -- 172.21.15.86:0/3849519414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 msgr2=0x7f5618102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.758+0000 7f561eff4640 1 --2- 172.21.15.86:0/3849519414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 0x7f5618102510 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5600009ca0 tx=0x7f560002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.758+0000 7f561eff4640 1 -- 172.21.15.86:0/3849519414 shutdown_connections 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.758+0000 7f561eff4640 1 --2- 172.21.15.86:0/3849519414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5618102a50 0x7f5618109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.758+0000 7f561eff4640 1 --2- 172.21.15.86:0/3849519414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 0x7f5618102510 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.758+0000 7f561eff4640 1 --2- 172.21.15.86:0/3849519414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5618100f40 0x7f5618101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.758+0000 7f561eff4640 1 -- 172.21.15.86:0/3849519414 >> 172.21.15.86:0/3849519414 conn(0x7f56180fc4b0 msgr2=0x7f56180fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.758+0000 7f561eff4640 1 -- 172.21.15.86:0/3849519414 shutdown_connections 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561eff4640 1 -- 172.21.15.86:0/3849519414 wait complete. 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561eff4640 1 Processor -- start 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561eff4640 1 -- start start 2024-04-16T05:33:11.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561eff4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5618100f40 0x7f5618198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561eff4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 0x7f5618198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561eff4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5618102a50 0x7f561819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561eff4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f561810cc40 con 0x7f5618102090 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561eff4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f561810cac0 con 0x7f5618100f40 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561eff4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f561810cdc0 con 0x7f5618102a50 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f5617fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 0x7f5618198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f5617fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 0x7f5618198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46172/0 (socket says 172.21.15.86:46172) 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f5617fff640 1 -- 172.21.15.86:0/3816992570 learned_addr learned my addr 172.21.15.86:0/3816992570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f561cd90640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5618100f40 0x7f5618198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.759+0000 7f5617fff640 1 -- 172.21.15.86:0/3816992570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5618102a50 msgr2=0x7f561819f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f5617fff640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5618102a50 0x7f561819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f5617fff640 1 -- 172.21.15.86:0/3816992570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5618100f40 msgr2=0x7f5618198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f5617fff640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5618100f40 0x7f5618198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f5617fff640 1 -- 172.21.15.86:0/3816992570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561819fff0 con 0x7f5618102090 2024-04-16T05:33:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f5617fff640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 0x7f5618198960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f56000324c0 tx=0x7f5600032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:11.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f5615ffb640 1 -- 172.21.15.86:0/3816992570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600039040 con 0x7f5618102090 2024-04-16T05:33:11.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56181a0280 con 0x7f5618102090 2024-04-16T05:33:11.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f5615ffb640 1 -- 172.21.15.86:0/3816992570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5600044070 con 0x7f5618102090 2024-04-16T05:33:11.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f5615ffb640 1 -- 172.21.15.86:0/3816992570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56000042a0 con 0x7f5618102090 2024-04-16T05:33:11.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.760+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56181a07c0 con 0x7f5618102090 2024-04-16T05:33:11.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.761+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56181a0ae0 con 0x7f5618102090 2024-04-16T05:33:11.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.763+0000 7f5615ffb640 1 -- 172.21.15.86:0/3816992570 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5600003970 con 0x7f5618102090 2024-04-16T05:33:11.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.764+0000 7f5615ffb640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f56040a3bb0 0x7f56040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:11.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.764+0000 7f561cd90640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f56040a3bb0 0x7f56040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:11.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.764+0000 7f5615ffb640 1 -- 172.21.15.86:0/3816992570 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f56000f35f0 con 0x7f5618102090 2024-04-16T05:33:11.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.764+0000 7f561cd90640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f56040a3bb0 0x7f56040a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f560800bbe0 tx=0x7f560800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:11.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.767+0000 7f5615ffb640 1 -- 172.21.15.86:0/3816992570 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56000bff40 con 0x7f5618102090 2024-04-16T05:33:11.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:11 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/268760534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:33:11.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:11 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2510865333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:33:11.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:11 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2674013385' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:33:11.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.949+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56181a0dd0 con 0x7f5618102090 2024-04-16T05:33:11.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.950+0000 7f5615ffb640 1 -- 172.21.15.86:0/3816992570 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24080 (secure 0 0 0) 0x7f56000bf840 con 0x7f5618102090 2024-04-16T05:33:11.949 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:11.949 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:01.527267+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f56040a3bb0 msgr2=0x7f56040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f56040a3bb0 0x7f56040a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f560800bbe0 tx=0x7f560800b390 comp rx=0 tx=0).stop 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 msgr2=0x7f5618198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 0x7f5618198960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f56000324c0 tx=0x7f5600032f20 comp rx=0 tx=0).stop 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 shutdown_connections 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5618102a50 0x7f561819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618102090 0x7f5618198960 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f56040a3bb0 0x7f56040a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 --2- 172.21.15.86:0/3816992570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5618100f40 0x7f5618198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 >> 172.21.15.86:0/3816992570 conn(0x7f56180fc4b0 msgr2=0x7f5618069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 shutdown_connections 2024-04-16T05:33:11.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:11.953+0000 7f561eff4640 1 -- 172.21.15.86:0/3816992570 wait complete. 2024-04-16T05:33:11.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:11 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/268760534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:33:11.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:11 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2510865333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:33:11.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:11 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2674013385' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:33:11.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:33:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:11 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/268760534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:33:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:11 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2510865333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:33:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:11 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2674013385' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:33:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 -- 172.21.15.86:0/768209944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103c00 msgr2=0x7f7e6010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 --2- 172.21.15.86:0/768209944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103c00 0x7f7e6010a490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009ca0 tx=0x7f7e5802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 -- 172.21.15.86:0/768209944 shutdown_connections 2024-04-16T05:33:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 --2- 172.21.15.86:0/768209944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103c00 0x7f7e6010a490 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 --2- 172.21.15.86:0/768209944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e60103240 0x7f7e601036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 --2- 172.21.15.86:0/768209944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e60102050 0x7f7e60102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 -- 172.21.15.86:0/768209944 >> 172.21.15.86:0/768209944 conn(0x7f7e600fd5e0 msgr2=0x7f7e600ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 -- 172.21.15.86:0/768209944 shutdown_connections 2024-04-16T05:33:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 -- 172.21.15.86:0/768209944 wait complete. 2024-04-16T05:33:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 Processor -- start 2024-04-16T05:33:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.069+0000 7f7e65550640 1 -- start start 2024-04-16T05:33:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e65550640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e60102050 0x7f7e601983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e65550640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103240 0x7f7e60198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e65550640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e60103c00 0x7f7e6019f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e65550640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6010ddc0 con 0x7f7e60103240 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e65550640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6010dc40 con 0x7f7e60102050 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e65550640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6010df40 con 0x7f7e60103c00 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e57fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103240 0x7f7e60198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e57fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103240 0x7f7e60198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46182/0 (socket says 172.21.15.86:46182) 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e57fff640 1 -- 172.21.15.86:0/964436055 learned_addr learned my addr 172.21.15.86:0/964436055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e5f7fe640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e60103c00 0x7f7e6019f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e5effd640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e60102050 0x7f7e601983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e57fff640 1 -- 172.21.15.86:0/964436055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e60103c00 msgr2=0x7f7e6019f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e57fff640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e60103c00 0x7f7e6019f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e57fff640 1 -- 172.21.15.86:0/964436055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e60102050 msgr2=0x7f7e601983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e57fff640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e60102050 0x7f7e601983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e57fff640 1 -- 172.21.15.86:0/964436055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6019fff0 con 0x7f7e60103240 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e5effd640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e60102050 0x7f7e601983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e57fff640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103240 0x7f7e60198910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800b4a0 tx=0x7f7e4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e5cff9640 1 -- 172.21.15.86:0/964436055 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48014070 con 0x7f7e60103240 2024-04-16T05:33:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e5cff9640 1 -- 172.21.15.86:0/964436055 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e48010af0 con 0x7f7e60103240 2024-04-16T05:33:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e601a02e0 con 0x7f7e60103240 2024-04-16T05:33:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e5cff9640 1 -- 172.21.15.86:0/964436055 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48017d10 con 0x7f7e60103240 2024-04-16T05:33:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.070+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e601a0900 con 0x7f7e60103240 2024-04-16T05:33:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.071+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e60109370 con 0x7f7e60103240 2024-04-16T05:33:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.072+0000 7f7e5cff9640 1 -- 172.21.15.86:0/964436055 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7e48017070 con 0x7f7e60103240 2024-04-16T05:33:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.073+0000 7f7e5cff9640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7e4c0a3c30 0x7f7e4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.073+0000 7f7e5effd640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7e4c0a3c30 0x7f7e4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.073+0000 7f7e5cff9640 1 -- 172.21.15.86:0/964436055 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7e480cf900 con 0x7f7e60103240 2024-04-16T05:33:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.074+0000 7f7e5effd640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7e4c0a3c30 0x7f7e4c0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7e601030a0 tx=0x7f7e50008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.078+0000 7f7e5cff9640 1 -- 172.21.15.86:0/964436055 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e4809c0a0 con 0x7f7e60103240 2024-04-16T05:33:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.259+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e60062c00 con 0x7f7e60103240 2024-04-16T05:33:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.261+0000 7f7e5cff9640 1 -- 172.21.15.86:0/964436055 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24080 (secure 0 0 0) 0x7f7e4809b9a0 con 0x7f7e60103240 2024-04-16T05:33:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:12.260 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:01.527267+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7e4c0a3c30 msgr2=0x7f7e4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7e4c0a3c30 0x7f7e4c0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7e601030a0 tx=0x7f7e50008040 comp rx=0 tx=0).stop 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103240 msgr2=0x7f7e60198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103240 0x7f7e60198910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800b4a0 tx=0x7f7e4800b960 comp rx=0 tx=0).stop 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 shutdown_connections 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e60103c00 0x7f7e6019f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e60103240 0x7f7e60198910 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7e4c0a3c30 0x7f7e4c0a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 --2- 172.21.15.86:0/964436055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e60102050 0x7f7e601983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 >> 172.21.15.86:0/964436055 conn(0x7f7e600fd5e0 msgr2=0x7f7e60106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 shutdown_connections 2024-04-16T05:33:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.263+0000 7f7e65550640 1 -- 172.21.15.86:0/964436055 wait complete. 2024-04-16T05:33:12.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-16T05:33:12.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.383+0000 7f048f6d9640 1 -- 172.21.15.86:0/3828505480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488100f40 msgr2=0x7f0488101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:12.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.383+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3828505480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488100f40 0x7f0488101360 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0484009900 tx=0x7f048402bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:12.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.383+0000 7f048f6d9640 1 -- 172.21.15.86:0/3828505480 shutdown_connections 2024-04-16T05:33:12.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.383+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3828505480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0488102a50 0x7f0488109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.383+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3828505480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0488102090 0x7f0488102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.383+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3828505480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488100f40 0x7f0488101360 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.383+0000 7f048f6d9640 1 -- 172.21.15.86:0/3828505480 >> 172.21.15.86:0/3828505480 conn(0x7f04880fc4b0 msgr2=0x7f04880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:12.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.383+0000 7f048f6d9640 1 -- 172.21.15.86:0/3828505480 shutdown_connections 2024-04-16T05:33:12.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.383+0000 7f048f6d9640 1 -- 172.21.15.86:0/3828505480 wait complete. 2024-04-16T05:33:12.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048f6d9640 1 Processor -- start 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048f6d9640 1 -- start start 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048f6d9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0488100f40 0x7f04881983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048f6d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488102090 0x7f0488198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048f6d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0488102a50 0x7f048819f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048f6d9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f048810cbf0 con 0x7f0488102090 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048f6d9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f048810ca70 con 0x7f0488100f40 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048f6d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f048810cd70 con 0x7f0488102a50 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048cc74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488102090 0x7f0488198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048cc74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488102090 0x7f0488198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46200/0 (socket says 172.21.15.86:46200) 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.384+0000 7f048cc74640 1 -- 172.21.15.86:0/3117315366 learned_addr learned my addr 172.21.15.86:0/3117315366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048dc76640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0488102a50 0x7f048819f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048d475640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0488100f40 0x7f04881983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048cc74640 1 -- 172.21.15.86:0/3117315366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0488102a50 msgr2=0x7f048819f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:12.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048cc74640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0488102a50 0x7f048819f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048cc74640 1 -- 172.21.15.86:0/3117315366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0488100f40 msgr2=0x7f04881983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048cc74640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0488100f40 0x7f04881983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048cc74640 1 -- 172.21.15.86:0/3117315366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048819ffc0 con 0x7f0488102090 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048dc76640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0488102a50 0x7f048819f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048cc74640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488102090 0x7f0488198930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f047800d770 tx=0x7f047800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f047e7fc640 1 -- 172.21.15.86:0/3117315366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478017070 con 0x7f0488102090 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f047e7fc640 1 -- 172.21.15.86:0/3117315366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0478009c30 con 0x7f0488102090 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04881a02b0 con 0x7f0488102090 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f047e7fc640 1 -- 172.21.15.86:0/3117315366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478003dd0 con 0x7f0488102090 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.385+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04881a0850 con 0x7f0488102090 2024-04-16T05:33:12.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.386+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04881a0ab0 con 0x7f0488102090 2024-04-16T05:33:12.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.387+0000 7f047e7fc640 1 -- 172.21.15.86:0/3117315366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0478003f90 con 0x7f0488102090 2024-04-16T05:33:12.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.388+0000 7f047e7fc640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04740a3c30 0x7f04740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:12.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.388+0000 7f048d475640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04740a3c30 0x7f04740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:12.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.388+0000 7f047e7fc640 1 -- 172.21.15.86:0/3117315366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f04780d0420 con 0x7f0488102090 2024-04-16T05:33:12.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.388+0000 7f048d475640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04740a3c30 0x7f04740a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0484005c20 tx=0x7f0484005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:12.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.391+0000 7f047e7fc640 1 -- 172.21.15.86:0/3117315366 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0478009de0 con 0x7f0488102090 2024-04-16T05:33:12.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:12.615+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f04881a0c80 con 0x7f0488102090 2024-04-16T05:33:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:12 smithi194 ceph-mon[35865]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-04-16T05:33:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:12 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3816992570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:12 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/964436055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:12.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:12 smithi086 ceph-mon[32028]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-04-16T05:33:12.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:12 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3816992570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:12.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:12 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/964436055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:12 smithi196 ceph-mon[36373]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-04-16T05:33:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:12 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3816992570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:12 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/964436055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.568+0000 7f047e7fc640 1 -- 172.21.15.86:0/3117315366 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f047809cc40 con 0x7f0488102090 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04740a3c30 msgr2=0x7f04740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04740a3c30 0x7f04740a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0484005c20 tx=0x7f0484005fd0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488102090 msgr2=0x7f0488198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488102090 0x7f0488198930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f047800d770 tx=0x7f047800dc30 comp rx=0 tx=0).stop 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 shutdown_connections 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0488102a50 0x7f048819f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488102090 0x7f0488198930 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f04740a3c30 0x7f04740a60f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 --2- 172.21.15.86:0/3117315366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0488100f40 0x7f04881983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 >> 172.21.15.86:0/3117315366 conn(0x7f04880fc4b0 msgr2=0x7f04880fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 shutdown_connections 2024-04-16T05:33:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.571+0000 7f048f6d9640 1 -- 172.21.15.86:0/3117315366 wait complete. 2024-04-16T05:33:13.581 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-16T05:33:13.581 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-16T05:33:13.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.692+0000 7fbcee2a5640 1 -- 172.21.15.86:0/431805789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce8103630 msgr2=0x7fbce8103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:13.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.692+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/431805789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce8103630 0x7fbce8103a50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009900 tx=0x7fbcd802bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:13.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.692+0000 7fbcee2a5640 1 -- 172.21.15.86:0/431805789 shutdown_connections 2024-04-16T05:33:13.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.692+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/431805789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbce8100a60 0x7fbce810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.692+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/431805789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce81000a0 0x7fbce8100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.692+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/431805789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce8103630 0x7fbce8103a50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.692+0000 7fbcee2a5640 1 -- 172.21.15.86:0/431805789 >> 172.21.15.86:0/431805789 conn(0x7fbce80fbc40 msgr2=0x7fbce80fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.692+0000 7fbcee2a5640 1 -- 172.21.15.86:0/431805789 shutdown_connections 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.692+0000 7fbcee2a5640 1 -- 172.21.15.86:0/431805789 wait complete. 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcee2a5640 1 Processor -- start 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcee2a5640 1 -- start start 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcee2a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce81000a0 0x7fbce81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcee2a5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce8100a60 0x7fbce8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcee2a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbce8103630 0x7fbce819f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcee2a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbce810dd30 con 0x7fbce81000a0 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcee2a5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbce810dbb0 con 0x7fbce8100a60 2024-04-16T05:33:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcee2a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbce810deb0 con 0x7fbce8103630 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbce7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce81000a0 0x7fbce81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbce7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce81000a0 0x7fbce81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46220/0 (socket says 172.21.15.86:46220) 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbce7fff640 1 -- 172.21.15.86:0/1170376279 learned_addr learned my addr 172.21.15.86:0/1170376279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbce7fff640 1 -- 172.21.15.86:0/1170376279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbce8103630 msgr2=0x7fbce819f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbce77fe640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce8100a60 0x7fbce8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcec842640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbce8103630 0x7fbce819f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbce7fff640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbce8103630 0x7fbce819f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbce7fff640 1 -- 172.21.15.86:0/1170376279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce8100a60 msgr2=0x7fbce8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbce7fff640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce8100a60 0x7fbce8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbce7fff640 1 -- 172.21.15.86:0/1170376279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce81a0000 con 0x7fbce81000a0 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.693+0000 7fbcec842640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbce8103630 0x7fbce819f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.694+0000 7fbce7fff640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce81000a0 0x7fbce81983e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbcd802efd0 tx=0x7fbcd8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.694+0000 7fbce57fa640 1 -- 172.21.15.86:0/1170376279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8041070 con 0x7fbce81000a0 2024-04-16T05:33:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.694+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce81a0290 con 0x7fbce81000a0 2024-04-16T05:33:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.694+0000 7fbce57fa640 1 -- 172.21.15.86:0/1170376279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd80054e0 con 0x7fbce81000a0 2024-04-16T05:33:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.694+0000 7fbce57fa640 1 -- 172.21.15.86:0/1170376279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80346a0 con 0x7fbce81000a0 2024-04-16T05:33:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.694+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce81a07d0 con 0x7fbce81000a0 2024-04-16T05:33:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.695+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce8109490 con 0x7fbce81000a0 2024-04-16T05:33:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.696+0000 7fbce57fa640 1 -- 172.21.15.86:0/1170376279 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbcd8044070 con 0x7fbce81000a0 2024-04-16T05:33:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.697+0000 7fbce57fa640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcd00a3bb0 0x7fbcd00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.697+0000 7fbce77fe640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcd00a3bb0 0x7fbcd00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.697+0000 7fbce57fa640 1 -- 172.21.15.86:0/1170376279 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbcd80f27b0 con 0x7fbce81000a0 2024-04-16T05:33:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.697+0000 7fbce77fe640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcd00a3bb0 0x7fbcd00a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbce81998f0 tx=0x7fbcdc005cc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:13.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.700+0000 7fbce57fa640 1 -- 172.21.15.86:0/1170376279 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcd80bef50 con 0x7fbce81000a0 2024-04-16T05:33:13.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:13 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3117315366' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T05:33:13.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:13 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3117315366' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T05:33:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:13.918+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fbce8062c00 con 0x7fbce81000a0 2024-04-16T05:33:13.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:13 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3117315366' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T05:33:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.570+0000 7fbce57fa640 1 -- 172.21.15.86:0/1170376279 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fbcd80be850 con 0x7fbce81000a0 2024-04-16T05:33:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.572+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcd00a3bb0 msgr2=0x7fbcd00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.572+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcd00a3bb0 0x7fbcd00a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbce81998f0 tx=0x7fbcdc005cc0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.572+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce81000a0 msgr2=0x7fbce81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.572+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce81000a0 0x7fbce81983e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbcd802efd0 tx=0x7fbcd8004040 comp rx=0 tx=0).stop 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.573+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 shutdown_connections 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.573+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbce8103630 0x7fbce819f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.573+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbcd00a3bb0 0x7fbcd00a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.573+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce8100a60 0x7fbce8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.573+0000 7fbcee2a5640 1 --2- 172.21.15.86:0/1170376279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce81000a0 0x7fbce81983e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.573+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 >> 172.21.15.86:0/1170376279 conn(0x7fbce80fbc40 msgr2=0x7fbce8106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.573+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 shutdown_connections 2024-04-16T05:33:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.573+0000 7fbcee2a5640 1 -- 172.21.15.86:0/1170376279 wait complete. 2024-04-16T05:33:14.587 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.699+0000 7f754eb0e640 1 -- 172.21.15.86:0/2675164091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548105450 msgr2=0x7f7548107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.699+0000 7f754eb0e640 1 --2- 172.21.15.86:0/2675164091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548105450 0x7f7548107840 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7534009900 tx=0x7f753402bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.699+0000 7f754eb0e640 1 -- 172.21.15.86:0/2675164091 shutdown_connections 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.699+0000 7f754eb0e640 1 --2- 172.21.15.86:0/2675164091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7548107d80 0x7f754810a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.699+0000 7f754eb0e640 1 --2- 172.21.15.86:0/2675164091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548105450 0x7f7548107840 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.699+0000 7f754eb0e640 1 --2- 172.21.15.86:0/2675164091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7548102b20 0x7f7548104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.699+0000 7f754eb0e640 1 -- 172.21.15.86:0/2675164091 >> 172.21.15.86:0/2675164091 conn(0x7f75480fc4b0 msgr2=0x7f75480fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.699+0000 7f754eb0e640 1 -- 172.21.15.86:0/2675164091 shutdown_connections 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.699+0000 7f754eb0e640 1 -- 172.21.15.86:0/2675164091 wait complete. 2024-04-16T05:33:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754eb0e640 1 Processor -- start 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754eb0e640 1 -- start start 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754eb0e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548102b20 0x7f7548198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754eb0e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7548105450 0x7f7548198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754eb0e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7548107d80 0x7f754819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754eb0e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f754810dd10 con 0x7f7548102b20 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754eb0e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f754810db90 con 0x7f7548105450 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754eb0e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f754810de90 con 0x7f7548107d80 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754c8aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548102b20 0x7f7548198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754c8aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548102b20 0x7f7548198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46242/0 (socket says 172.21.15.86:46242) 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754c8aa640 1 -- 172.21.15.86:0/1225264747 learned_addr learned my addr 172.21.15.86:0/1225264747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f7547fff640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7548105450 0x7f7548198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754c8aa640 1 -- 172.21.15.86:0/1225264747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7548107d80 msgr2=0x7f754819f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754c8aa640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7548107d80 0x7f754819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754c8aa640 1 -- 172.21.15.86:0/1225264747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7548105450 msgr2=0x7f7548198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754c8aa640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7548105450 0x7f7548198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754c8aa640 1 -- 172.21.15.86:0/1225264747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754819fff0 con 0x7f7548102b20 2024-04-16T05:33:14.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f754c8aa640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548102b20 0x7f7548198420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f753c00b4a0 tx=0x7f753c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:14.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.700+0000 7f7545ffb640 1 -- 172.21.15.86:0/1225264747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c014070 con 0x7f7548102b20 2024-04-16T05:33:14.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.701+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75481a02e0 con 0x7f7548102b20 2024-04-16T05:33:14.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.701+0000 7f7545ffb640 1 -- 172.21.15.86:0/1225264747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f753c010af0 con 0x7f7548102b20 2024-04-16T05:33:14.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.701+0000 7f7545ffb640 1 -- 172.21.15.86:0/1225264747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c017d10 con 0x7f7548102b20 2024-04-16T05:33:14.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.701+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75481a0800 con 0x7f7548102b20 2024-04-16T05:33:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.702+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75480fdfa0 con 0x7f7548102b20 2024-04-16T05:33:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.703+0000 7f7545ffb640 1 -- 172.21.15.86:0/1225264747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f753c017070 con 0x7f7548102b20 2024-04-16T05:33:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.703+0000 7f7545ffb640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f75380a3bb0 0x7f75380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.703+0000 7f7547fff640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f75380a3bb0 0x7f75380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.703+0000 7f7545ffb640 1 -- 172.21.15.86:0/1225264747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f753c0cfa70 con 0x7f7548102b20 2024-04-16T05:33:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.704+0000 7f7547fff640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f75380a3bb0 0x7f75380a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f753402c7b0 tx=0x7f7534000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:14.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.708+0000 7f7545ffb640 1 -- 172.21.15.86:0/1225264747 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f753c09c210 con 0x7f7548102b20 2024-04-16T05:33:14.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:14 smithi194 ceph-mon[35865]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 257 B/s wr, 0 op/s 2024-04-16T05:33:14.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:14 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3117315366' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T05:33:14.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:14 smithi194 ceph-mon[35865]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:33:14.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:14 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1170376279' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-16T05:33:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:14.923+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f7548062c00 con 0x7f7548102b20 2024-04-16T05:33:14.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:14 smithi086 ceph-mon[32028]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 257 B/s wr, 0 op/s 2024-04-16T05:33:14.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:14 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3117315366' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T05:33:14.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:14 smithi086 ceph-mon[32028]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:33:14.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:14 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1170376279' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-16T05:33:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:14 smithi196 ceph-mon[36373]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 257 B/s wr, 0 op/s 2024-04-16T05:33:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:14 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3117315366' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T05:33:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:14 smithi196 ceph-mon[36373]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:33:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:14 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1170376279' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-16T05:33:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.581+0000 7f7545ffb640 1 -- 172.21.15.86:0/1225264747 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f753c09bb10 con 0x7f7548102b20 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f75380a3bb0 msgr2=0x7f75380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f75380a3bb0 0x7f75380a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f753402c7b0 tx=0x7f7534000bc0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548102b20 msgr2=0x7f7548198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548102b20 0x7f7548198420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f753c00b4a0 tx=0x7f753c00b960 comp rx=0 tx=0).stop 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 shutdown_connections 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7548107d80 0x7f754819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f75380a3bb0 0x7f75380a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7548105450 0x7f7548198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 --2- 172.21.15.86:0/1225264747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7548102b20 0x7f7548198420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 >> 172.21.15.86:0/1225264747 conn(0x7f75480fc4b0 msgr2=0x7f7548103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 shutdown_connections 2024-04-16T05:33:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.584+0000 7f754eb0e640 1 -- 172.21.15.86:0/1225264747 wait complete. 2024-04-16T05:33:15.599 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-16T05:33:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.708+0000 7f99e9f7c640 1 -- 172.21.15.86:0/206518344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4102090 msgr2=0x7f99e4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.708+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/206518344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4102090 0x7f99e4102510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009900 tx=0x7f99d802bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.708+0000 7f99e9f7c640 1 -- 172.21.15.86:0/206518344 shutdown_connections 2024-04-16T05:33:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.708+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/206518344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99e4102a50 0x7f99e4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.708+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/206518344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4102090 0x7f99e4102510 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.708+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/206518344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e4100f40 0x7f99e4101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.708+0000 7f99e9f7c640 1 -- 172.21.15.86:0/206518344 >> 172.21.15.86:0/206518344 conn(0x7f99e40fc4b0 msgr2=0x7f99e40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.708+0000 7f99e9f7c640 1 -- 172.21.15.86:0/206518344 shutdown_connections 2024-04-16T05:33:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.708+0000 7f99e9f7c640 1 -- 172.21.15.86:0/206518344 wait complete. 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e9f7c640 1 Processor -- start 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e9f7c640 1 -- start start 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e9f7c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4100f40 0x7f99e4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e9f7c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99e4102090 0x7f99e4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e9f7c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e4102a50 0x7f99e419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e9f7c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410dd50 con 0x7f99e4100f40 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e9f7c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410dbd0 con 0x7f99e4102090 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e9f7c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410ded0 con 0x7f99e4102a50 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4100f40 0x7f99e4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4100f40 0x7f99e4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46262/0 (socket says 172.21.15.86:46262) 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e37fe640 1 -- 172.21.15.86:0/122099073 learned_addr learned my addr 172.21.15.86:0/122099073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.709+0000 7f99e2ffd640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99e4102090 0x7f99e4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e37fe640 1 -- 172.21.15.86:0/122099073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e4102a50 msgr2=0x7f99e419f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e37fe640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e4102a50 0x7f99e419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e37fe640 1 -- 172.21.15.86:0/122099073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99e4102090 msgr2=0x7f99e4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e37fe640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99e4102090 0x7f99e4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e37fe640 1 -- 172.21.15.86:0/122099073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e41a0050 con 0x7f99e4100f40 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e37fe640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4100f40 0x7f99e4198430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f99d400ba50 tx=0x7f99d400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e0ff9640 1 -- 172.21.15.86:0/122099073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4019070 con 0x7f99e4100f40 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e41a0340 con 0x7f99e4100f40 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e0ff9640 1 -- 172.21.15.86:0/122099073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d400cbe0 con 0x7f99e4100f40 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e0ff9640 1 -- 172.21.15.86:0/122099073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d40076e0 con 0x7f99e4100f40 2024-04-16T05:33:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.710+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e41a0810 con 0x7f99e4100f40 2024-04-16T05:33:15.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.711+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e4108270 con 0x7f99e4100f40 2024-04-16T05:33:15.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.712+0000 7f99e0ff9640 1 -- 172.21.15.86:0/122099073 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f99d401c070 con 0x7f99e4100f40 2024-04-16T05:33:15.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.713+0000 7f99e0ff9640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f99cc0a3bb0 0x7f99cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:15.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.713+0000 7f99e2ffd640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f99cc0a3bb0 0x7f99cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:15.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.713+0000 7f99e0ff9640 1 -- 172.21.15.86:0/122099073 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f99d40d0110 con 0x7f99e4100f40 2024-04-16T05:33:15.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.714+0000 7f99e2ffd640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f99cc0a3bb0 0x7f99cc0a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99d8000c00 tx=0x7f99d8005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:15.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.717+0000 7f99e0ff9640 1 -- 172.21.15.86:0/122099073 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99d409c260 con 0x7f99e4100f40 2024-04-16T05:33:15.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:15 smithi194 ceph-mon[35865]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T05:33:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:15 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1170376279' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-16T05:33:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:15 smithi194 ceph-mon[35865]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T05:33:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:15 smithi194 ceph-mon[35865]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:33:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:15 smithi194 ceph-mon[35865]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-16T05:33:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:15 smithi194 ceph-mon[35865]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-16T05:33:15.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:15 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1225264747' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T05:33:15.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:15 smithi086 ceph-mon[32028]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T05:33:15.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:15 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1170376279' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-16T05:33:15.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:15 smithi086 ceph-mon[32028]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T05:33:15.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:15 smithi086 ceph-mon[32028]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:33:15.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:15 smithi086 ceph-mon[32028]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-16T05:33:15.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:15 smithi086 ceph-mon[32028]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-16T05:33:15.918 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:15 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1225264747' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T05:33:15.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:15.934+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f99e402d620 con 0x7f99e4100f40 2024-04-16T05:33:15.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:15 smithi196 ceph-mon[36373]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T05:33:15.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:15 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1170376279' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-16T05:33:15.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:15 smithi196 ceph-mon[36373]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T05:33:15.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:15 smithi196 ceph-mon[36373]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:33:15.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:15 smithi196 ceph-mon[36373]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-16T05:33:15.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:15 smithi196 ceph-mon[36373]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-16T05:33:15.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:15 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1225264747' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T05:33:16.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.592+0000 7f99e0ff9640 1 -- 172.21.15.86:0/122099073 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f99d40a0560 con 0x7f99e4100f40 2024-04-16T05:33:16.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f99cc0a3bb0 msgr2=0x7f99cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:16.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f99cc0a3bb0 0x7f99cc0a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99d8000c00 tx=0x7f99d8005fd0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4100f40 msgr2=0x7f99e4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:16.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4100f40 0x7f99e4198430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f99d400ba50 tx=0x7f99d400bf10 comp rx=0 tx=0).stop 2024-04-16T05:33:16.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 shutdown_connections 2024-04-16T05:33:16.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e4102a50 0x7f99e419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f99cc0a3bb0 0x7f99cc0a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99e4102090 0x7f99e4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 --2- 172.21.15.86:0/122099073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e4100f40 0x7f99e4198430 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 >> 172.21.15.86:0/122099073 conn(0x7f99e40fc4b0 msgr2=0x7f99e4105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:16.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 shutdown_connections 2024-04-16T05:33:16.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.595+0000 7f99e9f7c640 1 -- 172.21.15.86:0/122099073 wait complete. 2024-04-16T05:33:16.611 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-16T05:33:16.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.725+0000 7f44cb3d6640 1 -- 172.21.15.86:0/3881514550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4107d20 msgr2=0x7f44c410a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.725+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/3881514550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4107d20 0x7f44c410a170 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009ca0 tx=0x7f44bc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 -- 172.21.15.86:0/3881514550 shutdown_connections 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/3881514550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4107d20 0x7f44c410a170 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/3881514550 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44c41053f0 0x7f44c41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/3881514550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44c4102ac0 0x7f44c4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 -- 172.21.15.86:0/3881514550 >> 172.21.15.86:0/3881514550 conn(0x7f44c40fc4b0 msgr2=0x7f44c40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 -- 172.21.15.86:0/3881514550 shutdown_connections 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 -- 172.21.15.86:0/3881514550 wait complete. 2024-04-16T05:33:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 Processor -- start 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 -- start start 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4102ac0 0x7f44c4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44c41053f0 0x7f44c41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44c4107d20 0x7f44c419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410dcd0 con 0x7f44c4102ac0 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410db50 con 0x7f44c4107d20 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44cb3d6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410de50 con 0x7f44c41053f0 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44c9172640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4102ac0 0x7f44c4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.726+0000 7f44c9172640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4102ac0 0x7f44c4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46284/0 (socket says 172.21.15.86:46284) 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c9172640 1 -- 172.21.15.86:0/2341177111 learned_addr learned my addr 172.21.15.86:0/2341177111 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c9973640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44c4107d20 0x7f44c419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c8971640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44c41053f0 0x7f44c41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:16.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c9172640 1 -- 172.21.15.86:0/2341177111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44c41053f0 msgr2=0x7f44c41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c9172640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44c41053f0 0x7f44c41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c9172640 1 -- 172.21.15.86:0/2341177111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44c4107d20 msgr2=0x7f44c419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c9172640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44c4107d20 0x7f44c419f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c9172640 1 -- 172.21.15.86:0/2341177111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c419ff90 con 0x7f44c4102ac0 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c8971640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44c41053f0 0x7f44c41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44c9172640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4102ac0 0x7f44c4198370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f44c000c970 tx=0x7f44c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44b67fc640 1 -- 172.21.15.86:0/2341177111 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0007bb0 con 0x7f44c4102ac0 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44b67fc640 1 -- 172.21.15.86:0/2341177111 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44c0007d50 con 0x7f44c4102ac0 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c41a0280 con 0x7f44c4102ac0 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44b67fc640 1 -- 172.21.15.86:0/2341177111 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c000f450 con 0x7f44c4102ac0 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.727+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41a0780 con 0x7f44c4102ac0 2024-04-16T05:33:16.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.728+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c40fdfa0 con 0x7f44c4102ac0 2024-04-16T05:33:16.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.730+0000 7f44b67fc640 1 -- 172.21.15.86:0/2341177111 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f44c0004fe0 con 0x7f44c4102ac0 2024-04-16T05:33:16.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.730+0000 7f44b67fc640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f44b00a3bb0 0x7f44b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:16.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.730+0000 7f44c8971640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f44b00a3bb0 0x7f44b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:16.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.730+0000 7f44b67fc640 1 -- 172.21.15.86:0/2341177111 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f44c00d07e0 con 0x7f44c4102ac0 2024-04-16T05:33:16.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.731+0000 7f44c8971640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f44b00a3bb0 0x7f44b00a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44c4199880 tx=0x7f44b8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:16.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.734+0000 7f44b67fc640 1 -- 172.21.15.86:0/2341177111 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44c0014030 con 0x7f44c4102ac0 2024-04-16T05:33:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:16 smithi194 ceph-mon[35865]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-16T05:33:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:16 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1225264747' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T05:33:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:16 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] up:active 2024-04-16T05:33:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:16 smithi194 ceph-mon[35865]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T05:33:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:16 smithi194 ceph-mon[35865]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-16T05:33:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:16 smithi194 ceph-mon[35865]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-16T05:33:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:16 smithi194 ceph-mon[35865]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-16T05:33:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:16 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/122099073' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T05:33:16.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:16.937+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f44c4062440 con 0x7f44b00a3bb0 2024-04-16T05:33:16.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-mon[32028]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-16T05:33:16.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1225264747' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T05:33:16.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] up:active 2024-04-16T05:33:16.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-mon[32028]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T05:33:16.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-mon[32028]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-16T05:33:16.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-mon[32028]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-16T05:33:16.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-mon[32028]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-16T05:33:16.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/122099073' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T05:33:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:16 smithi196 ceph-mon[36373]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-16T05:33:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:16 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1225264747' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T05:33:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:16 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] up:active 2024-04-16T05:33:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:16 smithi196 ceph-mon[36373]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T05:33:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:16 smithi196 ceph-mon[36373]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-16T05:33:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:16 smithi196 ceph-mon[36373]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-16T05:33:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:16 smithi196 ceph-mon[36373]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-16T05:33:16.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:16 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/122099073' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T05:33:17.217 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:33:16.942+0000 7ffaf16c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:33:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:33:16.942+0000 7ffaf16c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:33:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:33:16.942+0000 7ffaf16c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:33:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:33:16.942+0000 7ffaf16c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:33:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:33:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:33:16.942+0000 7ffaf16c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:33:17.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.625+0000 7f44b67fc640 1 -- 172.21.15.86:0/2341177111 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f44c4062440 con 0x7f44b00a3bb0 2024-04-16T05:33:17.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f44b00a3bb0 msgr2=0x7f44b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:17.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f44b00a3bb0 0x7f44b00a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44c4199880 tx=0x7f44b8005f90 comp rx=0 tx=0).stop 2024-04-16T05:33:17.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4102ac0 msgr2=0x7f44c4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:17.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4102ac0 0x7f44c4198370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f44c000c970 tx=0x7f44c000ce30 comp rx=0 tx=0).stop 2024-04-16T05:33:17.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 shutdown_connections 2024-04-16T05:33:17.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f44b00a3bb0 0x7f44b00a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44c4107d20 0x7f44c419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44c41053f0 0x7f44c41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 --2- 172.21.15.86:0/2341177111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c4102ac0 0x7f44c4198370 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 >> 172.21.15.86:0/2341177111 conn(0x7f44c40fc4b0 msgr2=0x7f44c4103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:17.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 shutdown_connections 2024-04-16T05:33:17.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.627+0000 7f44cb3d6640 1 -- 172.21.15.86:0/2341177111 wait complete. 2024-04-16T05:33:17.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-16T05:33:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.750+0000 7f1f0058d640 1 -- 172.21.15.86:0/3264809199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 msgr2=0x7f1ef81093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.750+0000 7f1f0058d640 1 --2- 172.21.15.86:0/3264809199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 0x7f1ef81093c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009ca0 tx=0x7f1ef002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.750+0000 7f1f0058d640 1 -- 172.21.15.86:0/3264809199 shutdown_connections 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.750+0000 7f1f0058d640 1 --2- 172.21.15.86:0/3264809199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 0x7f1ef81093c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.750+0000 7f1f0058d640 1 --2- 172.21.15.86:0/3264809199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef81020e0 0x7f1ef8102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.750+0000 7f1f0058d640 1 --2- 172.21.15.86:0/3264809199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ef8100f10 0x7f1ef8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.750+0000 7f1f0058d640 1 -- 172.21.15.86:0/3264809199 >> 172.21.15.86:0/3264809199 conn(0x7f1ef80fc4f0 msgr2=0x7f1ef80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.750+0000 7f1f0058d640 1 -- 172.21.15.86:0/3264809199 shutdown_connections 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.750+0000 7f1f0058d640 1 -- 172.21.15.86:0/3264809199 wait complete. 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1f0058d640 1 Processor -- start 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1f0058d640 1 -- start start 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1f0058d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ef8100f10 0x7f1ef8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1f0058d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef81020e0 0x7f1ef81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1f0058d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 0x7f1ef819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1f0058d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef810dd00 con 0x7f1ef8102aa0 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1f0058d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef810db80 con 0x7f1ef8100f10 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1f0058d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef810de80 con 0x7f1ef81020e0 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efeb2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 0x7f1ef819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efeb2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 0x7f1ef819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46322/0 (socket says 172.21.15.86:46322) 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efeb2a640 1 -- 172.21.15.86:0/1392513198 learned_addr learned my addr 172.21.15.86:0/1392513198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efe329640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ef8100f10 0x7f1ef8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efdb28640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef81020e0 0x7f1ef81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efeb2a640 1 -- 172.21.15.86:0/1392513198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef81020e0 msgr2=0x7f1ef81987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efeb2a640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef81020e0 0x7f1ef81987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efeb2a640 1 -- 172.21.15.86:0/1392513198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ef8100f10 msgr2=0x7f1ef8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efeb2a640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ef8100f10 0x7f1ef8198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efeb2a640 1 -- 172.21.15.86:0/1392513198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef819fe90 con 0x7f1ef8102aa0 2024-04-16T05:33:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.751+0000 7f1efeb2a640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 0x7f1ef819f820 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0037040 tx=0x7f1ef0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.752+0000 7f1eef7fe640 1 -- 172.21.15.86:0/1392513198 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0041020 con 0x7f1ef8102aa0 2024-04-16T05:33:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.752+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef81a0120 con 0x7f1ef8102aa0 2024-04-16T05:33:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.752+0000 7f1eef7fe640 1 -- 172.21.15.86:0/1392513198 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef0034d90 con 0x7f1ef8102aa0 2024-04-16T05:33:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.752+0000 7f1eef7fe640 1 -- 172.21.15.86:0/1392513198 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0039040 con 0x7f1ef8102aa0 2024-04-16T05:33:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.752+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef81a0610 con 0x7f1ef8102aa0 2024-04-16T05:33:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.753+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef81082a0 con 0x7f1ef8102aa0 2024-04-16T05:33:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.754+0000 7f1eef7fe640 1 -- 172.21.15.86:0/1392513198 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1ef0044020 con 0x7f1ef8102aa0 2024-04-16T05:33:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.754+0000 7f1eef7fe640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ee40a3bb0 0x7f1ee40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.755+0000 7f1efe329640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ee40a3bb0 0x7f1ee40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.755+0000 7f1eef7fe640 1 -- 172.21.15.86:0/1392513198 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1ef00f29f0 con 0x7f1ef8102aa0 2024-04-16T05:33:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.755+0000 7f1efe329640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ee40a3bb0 0x7f1ee40a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8100c30 tx=0x7f1ef4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.758+0000 7f1eef7fe640 1 -- 172.21.15.86:0/1392513198 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ef004b020 con 0x7f1ef8102aa0 2024-04-16T05:33:17.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:17 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/122099073' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T05:33:17.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:17 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] up:active 2024-04-16T05:33:17.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:17 smithi194 ceph-mon[35865]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-16T05:33:17.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:17 smithi194 ceph-mon[35865]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-16T05:33:17.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:17 smithi194 ceph-mon[35865]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-16T05:33:17.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:17 smithi194 ceph-mon[35865]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-16T05:33:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.962+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1ef80624d0 con 0x7f1ee40a3bb0 2024-04-16T05:33:17.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:17 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/122099073' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T05:33:17.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:17 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] up:active 2024-04-16T05:33:17.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:17 smithi086 ceph-mon[32028]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-16T05:33:17.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:17 smithi086 ceph-mon[32028]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-16T05:33:17.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:17 smithi086 ceph-mon[32028]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-16T05:33:17.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:17 smithi086 ceph-mon[32028]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-16T05:33:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.985+0000 7f1eef7fe640 1 -- 172.21.15.86:0/1392513198 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1ef80624d0 con 0x7f1ee40a3bb0 2024-04-16T05:33:17.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:17 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/122099073' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T05:33:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:17 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] up:active 2024-04-16T05:33:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:17 smithi196 ceph-mon[36373]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-16T05:33:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:17 smithi196 ceph-mon[36373]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-16T05:33:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:17 smithi196 ceph-mon[36373]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-16T05:33:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:17 smithi196 ceph-mon[36373]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-16T05:33:17.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.986+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ee40a3bb0 msgr2=0x7f1ee40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ee40a3bb0 0x7f1ee40a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8100c30 tx=0x7f1ef4005f90 comp rx=0 tx=0).stop 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 msgr2=0x7f1ef819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 0x7f1ef819f820 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0037040 tx=0x7f1ef0032f20 comp rx=0 tx=0).stop 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 shutdown_connections 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef8102aa0 0x7f1ef819f820 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef81020e0 0x7f1ef81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ee40a3bb0 0x7f1ee40a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 --2- 172.21.15.86:0/1392513198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ef8100f10 0x7f1ef8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 >> 172.21.15.86:0/1392513198 conn(0x7f1ef80fc4f0 msgr2=0x7f1ef8105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 shutdown_connections 2024-04-16T05:33:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:17.987+0000 7f1f0058d640 1 -- 172.21.15.86:0/1392513198 wait complete. 2024-04-16T05:33:17.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-16T05:33:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 -- 172.21.15.86:0/7808627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690100f10 msgr2=0x7f8690101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 --2- 172.21.15.86:0/7808627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690100f10 0x7f8690101350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8680009900 tx=0x7f868002bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 -- 172.21.15.86:0/7808627 shutdown_connections 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 --2- 172.21.15.86:0/7808627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8690102aa0 0x7f86901093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 --2- 172.21.15.86:0/7808627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86901020e0 0x7f8690102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 --2- 172.21.15.86:0/7808627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690100f10 0x7f8690101350 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 -- 172.21.15.86:0/7808627 >> 172.21.15.86:0/7808627 conn(0x7f86900fc4d0 msgr2=0x7f86900fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 -- 172.21.15.86:0/7808627 shutdown_connections 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 -- 172.21.15.86:0/7808627 wait complete. 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 Processor -- start 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.108+0000 7f86966ba640 1 -- start start 2024-04-16T05:33:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f86966ba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8690100f10 0x7f86901983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f86966ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86901020e0 0x7f86901988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f86966ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690102aa0 0x7f869019f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f86966ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f869010dd00 con 0x7f8690102aa0 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f86966ba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f869010db80 con 0x7f8690100f10 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f86966ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f869010de80 con 0x7f86901020e0 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f8694c57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690102aa0 0x7f869019f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f8694c57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690102aa0 0x7f869019f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46332/0 (socket says 172.21.15.86:46332) 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f8694c57640 1 -- 172.21.15.86:0/638590022 learned_addr learned my addr 172.21.15.86:0/638590022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f868ffff640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8690100f10 0x7f86901983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f868f7fe640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86901020e0 0x7f86901988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f8694c57640 1 -- 172.21.15.86:0/638590022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86901020e0 msgr2=0x7f86901988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f8694c57640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86901020e0 0x7f86901988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f8694c57640 1 -- 172.21.15.86:0/638590022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8690100f10 msgr2=0x7f86901983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f8694c57640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8690100f10 0x7f86901983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f8694c57640 1 -- 172.21.15.86:0/638590022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869019ffd0 con 0x7f8690102aa0 2024-04-16T05:33:18.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f8694c57640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690102aa0 0x7f869019f960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f868800eaa0 tx=0x7f868800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:18.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f868d7fa640 1 -- 172.21.15.86:0/638590022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868800cdb0 con 0x7f8690102aa0 2024-04-16T05:33:18.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86901a02c0 con 0x7f8690102aa0 2024-04-16T05:33:18.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f868d7fa640 1 -- 172.21.15.86:0/638590022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8688005020 con 0x7f8690102aa0 2024-04-16T05:33:18.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f868d7fa640 1 -- 172.21.15.86:0/638590022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86880116b0 con 0x7f8690102aa0 2024-04-16T05:33:18.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.109+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86901a0810 con 0x7f8690102aa0 2024-04-16T05:33:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.110+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8690108370 con 0x7f8690102aa0 2024-04-16T05:33:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.112+0000 7f868d7fa640 1 -- 172.21.15.86:0/638590022 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8688005560 con 0x7f8690102aa0 2024-04-16T05:33:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.112+0000 7f868d7fa640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f86780a3bb0 0x7f86780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.113+0000 7f868ffff640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f86780a3bb0 0x7f86780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.113+0000 7f868d7fa640 1 -- 172.21.15.86:0/638590022 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f86880d0890 con 0x7f8690102aa0 2024-04-16T05:33:18.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.113+0000 7f868ffff640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f86780a3bb0 0x7f86780a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8680000c00 tx=0x7f8680005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:18.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.117+0000 7f868d7fa640 1 -- 172.21.15.86:0/638590022 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8688016030 con 0x7f8690102aa0 2024-04-16T05:33:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.320+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f8690062440 con 0x7f86780a3bb0 2024-04-16T05:33:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.327+0000 7f868d7fa640 1 -- 172.21.15.86:0/638590022 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8690062440 con 0x7f86780a3bb0 2024-04-16T05:33:18.326 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_0/44db40f2-700b-4b2a-ae58-5fe25032d993 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f86780a3bb0 msgr2=0x7f86780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f86780a3bb0 0x7f86780a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8680000c00 tx=0x7f8680005fd0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690102aa0 msgr2=0x7f869019f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690102aa0 0x7f869019f960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f868800eaa0 tx=0x7f868800ef60 comp rx=0 tx=0).stop 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 shutdown_connections 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8690102aa0 0x7f869019f960 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86901020e0 0x7f86901988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f86780a3bb0 0x7f86780a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 --2- 172.21.15.86:0/638590022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8690100f10 0x7f86901983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 >> 172.21.15.86:0/638590022 conn(0x7f86900fc4d0 msgr2=0x7f8690105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 shutdown_connections 2024-04-16T05:33:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.329+0000 7f86966ba640 1 -- 172.21.15.86:0/638590022 wait complete. 2024-04-16T05:33:18.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-16T05:33:18.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 -- 172.21.15.86:0/3605366240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102090 msgr2=0x7f6084102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 --2- 172.21.15.86:0/3605366240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102090 0x7f6084102510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6074009900 tx=0x7f607402bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 -- 172.21.15.86:0/3605366240 shutdown_connections 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 --2- 172.21.15.86:0/3605366240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6084102a50 0x7f6084109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 --2- 172.21.15.86:0/3605366240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102090 0x7f6084102510 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 --2- 172.21.15.86:0/3605366240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6084100f40 0x7f6084101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 -- 172.21.15.86:0/3605366240 >> 172.21.15.86:0/3605366240 conn(0x7f60840fc4b0 msgr2=0x7f60840fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 -- 172.21.15.86:0/3605366240 shutdown_connections 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 -- 172.21.15.86:0/3605366240 wait complete. 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 Processor -- start 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 -- start start 2024-04-16T05:33:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6084100f40 0x7f6084197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6084102090 0x7f6084197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102a50 0x7f608419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f608410cbd0 con 0x7f6084102a50 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f608410ca50 con 0x7f6084100f40 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.450+0000 7f608b575640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f608410cd50 con 0x7f6084102090 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089b12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102a50 0x7f608419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089311640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6084100f40 0x7f6084197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089b12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102a50 0x7f608419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46342/0 (socket says 172.21.15.86:46342) 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089b12640 1 -- 172.21.15.86:0/3034031316 learned_addr learned my addr 172.21.15.86:0/3034031316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089311640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6084100f40 0x7f6084197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34244/0 (socket says 172.21.15.86:34244) 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6088b10640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6084102090 0x7f6084197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089b12640 1 -- 172.21.15.86:0/3034031316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6084102090 msgr2=0x7f6084197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089b12640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6084102090 0x7f6084197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089b12640 1 -- 172.21.15.86:0/3034031316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6084100f40 msgr2=0x7f6084197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089b12640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6084100f40 0x7f6084197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089b12640 1 -- 172.21.15.86:0/3034031316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608419ee40 con 0x7f6084102a50 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f6089b12640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102a50 0x7f608419e7d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f607c00cce0 tx=0x7f607c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f607a7fc640 1 -- 172.21.15.86:0/3034031316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c00f450 con 0x7f6084102a50 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f607a7fc640 1 -- 172.21.15.86:0/3034031316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f607c00fad0 con 0x7f6084102a50 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608419f130 con 0x7f6084102a50 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f607a7fc640 1 -- 172.21.15.86:0/3034031316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c018c10 con 0x7f6084102a50 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.451+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608419f680 con 0x7f6084102a50 2024-04-16T05:33:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.452+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6084108250 con 0x7f6084102a50 2024-04-16T05:33:18.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.453+0000 7f607a7fc640 1 -- 172.21.15.86:0/3034031316 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f607c00f5f0 con 0x7f6084102a50 2024-04-16T05:33:18.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.454+0000 7f607a7fc640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f60700a3c30 0x7f60700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.454+0000 7f6089311640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f60700a3c30 0x7f60700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.454+0000 7f607a7fc640 1 -- 172.21.15.86:0/3034031316 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f607c0d0d10 con 0x7f6084102a50 2024-04-16T05:33:18.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.454+0000 7f6089311640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f60700a3c30 0x7f60700a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6080009bd0 tx=0x7f6080005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:18.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.457+0000 7f607a7fc640 1 -- 172.21.15.86:0/3034031316 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f607c014030 con 0x7f6084102a50 2024-04-16T05:33:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.672+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6084062440 con 0x7f60700a3c30 2024-04-16T05:33:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.697+0000 7f607a7fc640 1 -- 172.21.15.86:0/3034031316 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6084062440 con 0x7f60700a3c30 2024-04-16T05:33:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f60700a3c30 msgr2=0x7f60700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f60700a3c30 0x7f60700a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6080009bd0 tx=0x7f6080005f90 comp rx=0 tx=0).stop 2024-04-16T05:33:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102a50 msgr2=0x7f608419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102a50 0x7f608419e7d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f607c00cce0 tx=0x7f607c00a620 comp rx=0 tx=0).stop 2024-04-16T05:33:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 shutdown_connections 2024-04-16T05:33:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6084102a50 0x7f608419e7d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6084102090 0x7f6084197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f60700a3c30 0x7f60700a60f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 --2- 172.21.15.86:0/3034031316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6084100f40 0x7f6084197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 >> 172.21.15.86:0/3034031316 conn(0x7f60840fc4b0 msgr2=0x7f6084105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 shutdown_connections 2024-04-16T05:33:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.699+0000 7f608b575640 1 -- 172.21.15.86:0/3034031316 wait complete. 2024-04-16T05:33:18.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-16T05:33:18.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.820+0000 7fce86e47640 1 -- 172.21.15.86:0/1553750622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801018c0 msgr2=0x7fce80101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.820+0000 7fce86e47640 1 --2- 172.21.15.86:0/1553750622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801018c0 0x7fce80101d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009900 tx=0x7fce6c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:18.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.820+0000 7fce86e47640 1 -- 172.21.15.86:0/1553750622 shutdown_connections 2024-04-16T05:33:18.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.820+0000 7fce86e47640 1 --2- 172.21.15.86:0/1553750622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce80102280 0x7fce80108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.820+0000 7fce86e47640 1 --2- 172.21.15.86:0/1553750622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801018c0 0x7fce80101d40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.820+0000 7fce86e47640 1 --2- 172.21.15.86:0/1553750622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce801006d0 0x7fce80100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.820+0000 7fce86e47640 1 -- 172.21.15.86:0/1553750622 >> 172.21.15.86:0/1553750622 conn(0x7fce800fbc40 msgr2=0x7fce800fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.820+0000 7fce86e47640 1 -- 172.21.15.86:0/1553750622 shutdown_connections 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.820+0000 7fce86e47640 1 -- 172.21.15.86:0/1553750622 wait complete. 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce86e47640 1 Processor -- start 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce86e47640 1 -- start start 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce86e47640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801006d0 0x7fce80198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce86e47640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce801018c0 0x7fce80198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce86e47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80102280 0x7fce8019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce86e47640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010bae0 con 0x7fce801006d0 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce86e47640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010b960 con 0x7fce801018c0 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce86e47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010bc60 con 0x7fce80102280 2024-04-16T05:33:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce84be3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801006d0 0x7fce80198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce84be3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801006d0 0x7fce80198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46364/0 (socket says 172.21.15.86:46364) 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce84be3640 1 -- 172.21.15.86:0/1698638230 learned_addr learned my addr 172.21.15.86:0/1698638230 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce7ffff640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce801018c0 0x7fce80198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce853e4640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80102280 0x7fce8019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce84be3640 1 -- 172.21.15.86:0/1698638230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80102280 msgr2=0x7fce8019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce84be3640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80102280 0x7fce8019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce84be3640 1 -- 172.21.15.86:0/1698638230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce801018c0 msgr2=0x7fce80198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.821+0000 7fce84be3640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce801018c0 0x7fce80198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.822+0000 7fce84be3640 1 -- 172.21.15.86:0/1698638230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8019ff60 con 0x7fce801006d0 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.822+0000 7fce853e4640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80102280 0x7fce8019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.822+0000 7fce84be3640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801006d0 0x7fce80198340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce7400db70 tx=0x7fce7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.822+0000 7fce7dffb640 1 -- 172.21.15.86:0/1698638230 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7401b070 con 0x7fce801006d0 2024-04-16T05:33:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.822+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce801a0250 con 0x7fce801006d0 2024-04-16T05:33:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.822+0000 7fce7dffb640 1 -- 172.21.15.86:0/1698638230 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7400ae50 con 0x7fce801006d0 2024-04-16T05:33:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.822+0000 7fce7dffb640 1 -- 172.21.15.86:0/1698638230 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74016a30 con 0x7fce801006d0 2024-04-16T05:33:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.822+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce801a0770 con 0x7fce801006d0 2024-04-16T05:33:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.823+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce801079f0 con 0x7fce801006d0 2024-04-16T05:33:18.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.824+0000 7fce7dffb640 1 -- 172.21.15.86:0/1698638230 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fce7401e070 con 0x7fce801006d0 2024-04-16T05:33:18.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.824+0000 7fce7dffb640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fce700a3bb0 0x7fce700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:18.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.825+0000 7fce7ffff640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fce700a3bb0 0x7fce700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:18.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.825+0000 7fce7dffb640 1 -- 172.21.15.86:0/1698638230 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fce740d0530 con 0x7fce801006d0 2024-04-16T05:33:18.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.825+0000 7fce7ffff640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fce700a3bb0 0x7fce700a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce6c02c7b0 tx=0x7fce6c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:18.829+0000 7fce7dffb640 1 -- 172.21.15.86:0/1698638230 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce7409cc70 con 0x7fce801006d0 2024-04-16T05:33:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:18 smithi194 ceph-mon[35865]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:18 smithi194 ceph-mon[35865]: pgmap v303: 129 pgs: 129 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 5 op/s 2024-04-16T05:33:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:18 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] up:active 2024-04-16T05:33:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:18 smithi194 ceph-mon[35865]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-16T05:33:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:18 smithi194 ceph-mon[35865]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T05:33:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:18 smithi194 ceph-mon[35865]: Cluster is now healthy 2024-04-16T05:33:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:18 smithi194 ceph-mon[35865]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-16T05:33:18.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:18 smithi194 ceph-mon[35865]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-16T05:33:18.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:18 smithi194 ceph-mon[35865]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-16T05:33:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:18 smithi086 ceph-mon[32028]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:18 smithi086 ceph-mon[32028]: pgmap v303: 129 pgs: 129 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 5 op/s 2024-04-16T05:33:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:18 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] up:active 2024-04-16T05:33:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:18 smithi086 ceph-mon[32028]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-16T05:33:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:18 smithi086 ceph-mon[32028]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T05:33:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:18 smithi086 ceph-mon[32028]: Cluster is now healthy 2024-04-16T05:33:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:18 smithi086 ceph-mon[32028]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-16T05:33:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:18 smithi086 ceph-mon[32028]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-16T05:33:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:18 smithi086 ceph-mon[32028]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-16T05:33:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:18 smithi196 ceph-mon[36373]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:18 smithi196 ceph-mon[36373]: pgmap v303: 129 pgs: 129 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 5 op/s 2024-04-16T05:33:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:18 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] up:active 2024-04-16T05:33:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:18 smithi196 ceph-mon[36373]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-16T05:33:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:18 smithi196 ceph-mon[36373]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T05:33:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:18 smithi196 ceph-mon[36373]: Cluster is now healthy 2024-04-16T05:33:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:18 smithi196 ceph-mon[36373]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-16T05:33:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:18 smithi196 ceph-mon[36373]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-16T05:33:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:18 smithi196 ceph-mon[36373]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-16T05:33:19.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.036+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fce80062440 con 0x7fce700a3bb0 2024-04-16T05:33:19.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.042+0000 7fce7dffb640 1 -- 172.21.15.86:0/1698638230 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fce80062440 con 0x7fce700a3bb0 2024-04-16T05:33:19.040 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_1/304f66f3-2ffd-4c6a-8666-2669b193ec6a 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fce700a3bb0 msgr2=0x7fce700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fce700a3bb0 0x7fce700a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce6c02c7b0 tx=0x7fce6c005fd0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801006d0 msgr2=0x7fce80198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801006d0 0x7fce80198340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce7400db70 tx=0x7fce7400a620 comp rx=0 tx=0).stop 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 shutdown_connections 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80102280 0x7fce8019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fce700a3bb0 0x7fce700a6070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce801018c0 0x7fce80198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 --2- 172.21.15.86:0/1698638230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce801006d0 0x7fce80198340 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 >> 172.21.15.86:0/1698638230 conn(0x7fce800fbc40 msgr2=0x7fce80104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 shutdown_connections 2024-04-16T05:33:19.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.044+0000 7fce86e47640 1 -- 172.21.15.86:0/1698638230 wait complete. 2024-04-16T05:33:19.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:33:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.159+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/1442267721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103240 msgr2=0x7f1cf81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.159+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/1442267721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103240 0x7f1cf81036c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009900 tx=0x7f1cec02bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.159+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/1442267721 shutdown_connections 2024-04-16T05:33:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.159+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/1442267721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8103c00 0x7f1cf810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.159+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/1442267721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103240 0x7f1cf81036c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.159+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/1442267721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cf8102050 0x7f1cf8102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.159+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/1442267721 >> 172.21.15.86:0/1442267721 conn(0x7f1cf80fd5e0 msgr2=0x7f1cf80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.159+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/1442267721 shutdown_connections 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.159+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/1442267721 wait complete. 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.160+0000 7f1cfdaeb640 1 Processor -- start 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.160+0000 7f1cfdaeb640 1 -- start start 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.160+0000 7f1cfdaeb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cf8102050 0x7f1cf8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.160+0000 7f1cfdaeb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8103240 0x7f1cf8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.160+0000 7f1cfdaeb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103c00 0x7f1cf819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.160+0000 7f1cfdaeb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf810dce0 con 0x7f1cf8103c00 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.160+0000 7f1cfdaeb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf810db60 con 0x7f1cf8102050 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.160+0000 7f1cfdaeb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf810de60 con 0x7f1cf8103240 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103c00 0x7f1cf819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103c00 0x7f1cf819f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46378/0 (socket says 172.21.15.86:46378) 2024-04-16T05:33:19.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf7fff640 1 -- 172.21.15.86:0/2614031757 learned_addr learned my addr 172.21.15.86:0/2614031757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf77fe640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cf8102050 0x7f1cf8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf6ffd640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8103240 0x7f1cf8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf7fff640 1 -- 172.21.15.86:0/2614031757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8103240 msgr2=0x7f1cf8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf7fff640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8103240 0x7f1cf8198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf7fff640 1 -- 172.21.15.86:0/2614031757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cf8102050 msgr2=0x7f1cf8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf7fff640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cf8102050 0x7f1cf8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf7fff640 1 -- 172.21.15.86:0/2614031757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf819ff50 con 0x7f1cf8103c00 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf7fff640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103c00 0x7f1cf819f8e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000eb10 tx=0x7f1cf000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf4ff9640 1 -- 172.21.15.86:0/2614031757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0004b20 con 0x7f1cf8103c00 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf4ff9640 1 -- 172.21.15.86:0/2614031757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf0004cc0 con 0x7f1cf8103c00 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf81a0240 con 0x7f1cf8103c00 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cf4ff9640 1 -- 172.21.15.86:0/2614031757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0010760 con 0x7f1cf8103c00 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.161+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf81a0790 con 0x7f1cf8103c00 2024-04-16T05:33:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.162+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf8109370 con 0x7f1cf8103c00 2024-04-16T05:33:19.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.163+0000 7f1cf4ff9640 1 -- 172.21.15.86:0/2614031757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1cf0005020 con 0x7f1cf8103c00 2024-04-16T05:33:19.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.164+0000 7f1cf4ff9640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.164+0000 7f1cf77fe640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.164+0000 7f1cf4ff9640 1 -- 172.21.15.86:0/2614031757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1cf00d0d80 con 0x7f1cf8103c00 2024-04-16T05:33:19.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.165+0000 7f1cf77fe640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80095f0 tx=0x7f1ce8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:19.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.169+0000 7f1cf4ff9640 1 -- 172.21.15.86:0/2614031757 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1cf009d520 con 0x7f1cf8103c00 2024-04-16T05:33:19.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.385+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cf8062c00 con 0x7f1cf8103c00 2024-04-16T05:33:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.385+0000 7f1cf4ff9640 1 -- 172.21.15.86:0/2614031757 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7f1cf009ce20 con 0x7f1cf8103c00 2024-04-16T05:33:19.384 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:19.385 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:33:18.613317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24463,24455,24445]},"id":1}]} 2024-04-16T05:33:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ce00a3bb0 msgr2=0x7f1ce00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80095f0 tx=0x7f1ce8005f90 comp rx=0 tx=0).stop 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103c00 msgr2=0x7f1cf819f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103c00 0x7f1cf819f8e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000eb10 tx=0x7f1cf000c2d0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 shutdown_connections 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf8103c00 0x7f1cf819f8e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8103240 0x7f1cf8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 --2- 172.21.15.86:0/2614031757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cf8102050 0x7f1cf8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 >> 172.21.15.86:0/2614031757 conn(0x7f1cf80fd5e0 msgr2=0x7f1cf8106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 shutdown_connections 2024-04-16T05:33:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.388+0000 7f1cfdaeb640 1 -- 172.21.15.86:0/2614031757 wait complete. 2024-04-16T05:33:19.398 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:33:19.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.504+0000 7f0e67929640 1 -- 172.21.15.86:0/319527850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e60102b30 msgr2=0x7f0e601093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.504+0000 7f0e67929640 1 --2- 172.21.15.86:0/319527850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e60102b30 0x7f0e601093c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009ca0 tx=0x7f0e5802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.504+0000 7f0e67929640 1 -- 172.21.15.86:0/319527850 shutdown_connections 2024-04-16T05:33:19.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.504+0000 7f0e67929640 1 --2- 172.21.15.86:0/319527850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e60102b30 0x7f0e601093c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.504+0000 7f0e67929640 1 --2- 172.21.15.86:0/319527850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102170 0x7f0e601025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.504+0000 7f0e67929640 1 --2- 172.21.15.86:0/319527850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e60100f80 0x7f0e601013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.504+0000 7f0e67929640 1 -- 172.21.15.86:0/319527850 >> 172.21.15.86:0/319527850 conn(0x7f0e600fc4f0 msgr2=0x7f0e600fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:19.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 -- 172.21.15.86:0/319527850 shutdown_connections 2024-04-16T05:33:19.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 -- 172.21.15.86:0/319527850 wait complete. 2024-04-16T05:33:19.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 Processor -- start 2024-04-16T05:33:19.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 -- start start 2024-04-16T05:33:19.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e60100f80 0x7f0e601983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102170 0x7f0e60198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e60102b30 0x7f0e6019f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6010cc50 con 0x7f0e60100f80 2024-04-16T05:33:19.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6010cad0 con 0x7f0e60102b30 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e67929640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6010cdd0 con 0x7f0e60102170 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e656c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e60100f80 0x7f0e601983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.505+0000 7f0e656c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e60100f80 0x7f0e601983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46388/0 (socket says 172.21.15.86:46388) 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e656c5640 1 -- 172.21.15.86:0/1059746738 learned_addr learned my addr 172.21.15.86:0/1059746738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e65ec6640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e60102b30 0x7f0e6019f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e64ec4640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102170 0x7f0e60198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e65ec6640 1 -- 172.21.15.86:0/1059746738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102170 msgr2=0x7f0e60198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e65ec6640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102170 0x7f0e60198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e65ec6640 1 -- 172.21.15.86:0/1059746738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e60100f80 msgr2=0x7f0e601983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e65ec6640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e60100f80 0x7f0e601983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e65ec6640 1 -- 172.21.15.86:0/1059746738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6019fff0 con 0x7f0e60102b30 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e65ec6640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e60102b30 0x7f0e6019f980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e5802c700 tx=0x7f0e58032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.506+0000 7f0e567fc640 1 -- 172.21.15.86:0/1059746738 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58041020 con 0x7f0e60102b30 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.507+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e601a0280 con 0x7f0e60102b30 2024-04-16T05:33:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.507+0000 7f0e567fc640 1 -- 172.21.15.86:0/1059746738 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e58034cd0 con 0x7f0e60102b30 2024-04-16T05:33:19.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.507+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e601a07c0 con 0x7f0e60102b30 2024-04-16T05:33:19.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.507+0000 7f0e567fc640 1 -- 172.21.15.86:0/1059746738 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58039040 con 0x7f0e60102b30 2024-04-16T05:33:19.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.508+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e601a0ae0 con 0x7f0e60102b30 2024-04-16T05:33:19.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.509+0000 7f0e567fc640 1 -- 172.21.15.86:0/1059746738 <== mon.2 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0e58044020 con 0x7f0e60102b30 2024-04-16T05:33:19.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.510+0000 7f0e567fc640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0e4c0a3bb0 0x7f0e4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.510+0000 7f0e656c5640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0e4c0a3bb0 0x7f0e4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.510+0000 7f0e567fc640 1 -- 172.21.15.86:0/1059746738 <== mon.2 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0e580f27d0 con 0x7f0e60102b30 2024-04-16T05:33:19.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.511+0000 7f0e656c5640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0e4c0a3bb0 0x7f0e4c0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0e500058c0 tx=0x7f0e50005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:19.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.514+0000 7f0e567fc640 1 -- 172.21.15.86:0/1059746738 <== mon.2 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e5804b020 con 0x7f0e60102b30 2024-04-16T05:33:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.628+0000 7f0e567fc640 1 -- 172.21.15.86:0/1059746738 <== mon.2 v2:172.21.15.194:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e580baee0 con 0x7f0e60102b30 2024-04-16T05:33:19.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.700+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e601a0dd0 con 0x7f0e60102b30 2024-04-16T05:33:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.702+0000 7f0e567fc640 1 -- 172.21.15.86:0/1059746738 <== mon.2 v2:172.21.15.194:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24080 (secure 0 0 0) 0x7f0e580bef70 con 0x7f0e60102b30 2024-04-16T05:33:19.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:19.701 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:01.527267+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0e4c0a3bb0 msgr2=0x7f0e4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0e4c0a3bb0 0x7f0e4c0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0e500058c0 tx=0x7f0e50005f90 comp rx=0 tx=0).stop 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e60102b30 msgr2=0x7f0e6019f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e60102b30 0x7f0e6019f980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e5802c700 tx=0x7f0e58032f20 comp rx=0 tx=0).stop 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 shutdown_connections 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0e4c0a3bb0 0x7f0e4c0a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e60102b30 0x7f0e6019f980 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102170 0x7f0e60198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 --2- 172.21.15.86:0/1059746738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e60100f80 0x7f0e601983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 >> 172.21.15.86:0/1059746738 conn(0x7f0e600fc4f0 msgr2=0x7f0e6006a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 shutdown_connections 2024-04-16T05:33:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.704+0000 7f0e67929640 1 -- 172.21.15.86:0/1059746738 wait complete. 2024-04-16T05:33:19.712 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:19 smithi086 ceph-mon[32028]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:19.713 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:19 smithi086 ceph-mon[32028]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:19.713 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:19 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] up:active 2024-04-16T05:33:19.713 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:19 smithi086 ceph-mon[32028]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-16T05:33:19.713 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:19 smithi086 ceph-mon[32028]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-16T05:33:19.713 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:19 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2614031757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:33:19.714 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-16T05:33:19.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:33:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3524730602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 msgr2=0x7fd1c8101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3524730602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 0x7fd1c8101360 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009900 tx=0x7fd1b802bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3524730602 shutdown_connections 2024-04-16T05:33:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3524730602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8102a50 0x7fd1c8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3524730602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8102090 0x7fd1c8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3524730602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 0x7fd1c8101360 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3524730602 >> 172.21.15.86:0/3524730602 conn(0x7fd1c80fc4b0 msgr2=0x7fd1c80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3524730602 shutdown_connections 2024-04-16T05:33:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3524730602 wait complete. 2024-04-16T05:33:19.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 Processor -- start 2024-04-16T05:33:19.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.820+0000 7fd1cdb04640 1 -- start start 2024-04-16T05:33:19.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1cdb04640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 0x7fd1c81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1cdb04640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8102090 0x7fd1c81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1cdb04640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8102a50 0x7fd1c819f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1cdb04640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c810dcf0 con 0x7fd1c8100f40 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1cdb04640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c810db70 con 0x7fd1c8102a50 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1cdb04640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c810de70 con 0x7fd1c8102090 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 0x7fd1c81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 0x7fd1c81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46400/0 (socket says 172.21.15.86:46400) 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c77fe640 1 -- 172.21.15.86:0/3572753057 learned_addr learned my addr 172.21.15.86:0/3572753057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c7fff640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8102a50 0x7fd1c819f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1beffd640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8102090 0x7fd1c81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c77fe640 1 -- 172.21.15.86:0/3572753057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8102090 msgr2=0x7fd1c81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c77fe640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8102090 0x7fd1c81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c77fe640 1 -- 172.21.15.86:0/3572753057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8102a50 msgr2=0x7fd1c819f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:19.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c77fe640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8102a50 0x7fd1c819f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:19.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c77fe640 1 -- 172.21.15.86:0/3572753057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c819ffc0 con 0x7fd1c8100f40 2024-04-16T05:33:19.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1beffd640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8102090 0x7fd1c81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:19.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.821+0000 7fd1c77fe640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 0x7fd1c81983a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009230 tx=0x7fd1b8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:19.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.822+0000 7fd1c57fa640 1 -- 172.21.15.86:0/3572753057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8041070 con 0x7fd1c8100f40 2024-04-16T05:33:19.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.822+0000 7fd1c57fa640 1 -- 172.21.15.86:0/3572753057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b80044d0 con 0x7fd1c8100f40 2024-04-16T05:33:19.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.822+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c81a0250 con 0x7fd1c8100f40 2024-04-16T05:33:19.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.822+0000 7fd1c57fa640 1 -- 172.21.15.86:0/3572753057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8034440 con 0x7fd1c8100f40 2024-04-16T05:33:19.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.822+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c81a0740 con 0x7fd1c8100f40 2024-04-16T05:33:19.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.823+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c81082d0 con 0x7fd1c8100f40 2024-04-16T05:33:19.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.824+0000 7fd1c57fa640 1 -- 172.21.15.86:0/3572753057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd1b8044070 con 0x7fd1c8100f40 2024-04-16T05:33:19.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.824+0000 7fd1c57fa640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:19.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.825+0000 7fd1beffd640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:19.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.825+0000 7fd1c57fa640 1 -- 172.21.15.86:0/3572753057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd1b80f2330 con 0x7fd1c8100f40 2024-04-16T05:33:19.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.825+0000 7fd1beffd640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd1c81998b0 tx=0x7fd1b0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:19.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:19.829+0000 7fd1c57fa640 1 -- 172.21.15.86:0/3572753057 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1b80bea80 con 0x7fd1c8100f40 2024-04-16T05:33:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:19 smithi194 ceph-mon[35865]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:19 smithi194 ceph-mon[35865]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:19 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] up:active 2024-04-16T05:33:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:19 smithi194 ceph-mon[35865]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-16T05:33:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:19 smithi194 ceph-mon[35865]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-16T05:33:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:19 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2614031757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:33:19.906 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:33:19 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b[66970]: 2024-04-16T05:33:19.642+0000 7fa7479cd700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-16T05:33:19.906 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:33:19 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b[66970]: 2024-04-16T05:33:19.642+0000 7fa7479cd700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-16T05:33:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:19 smithi196 ceph-mon[36373]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:19 smithi196 ceph-mon[36373]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:19 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] up:active 2024-04-16T05:33:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:19 smithi196 ceph-mon[36373]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-16T05:33:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:19 smithi196 ceph-mon[36373]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-16T05:33:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:19 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2614031757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:33:20.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.005+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1c8062440 con 0x7fd1b40a3c40 2024-04-16T05:33:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.015+0000 7fd1c57fa640 1 -- 172.21.15.86:0/3572753057 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340606 (secure 0 0 0) 0x7fd1c8062440 con 0x7fd1b40a3c40 2024-04-16T05:33:20.018 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:20.020 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.018+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 msgr2=0x7fd1b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.018+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd1c81998b0 tx=0x7fd1b0005f90 comp rx=0 tx=0).stop 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.018+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 msgr2=0x7fd1c81983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.018+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 0x7fd1c81983a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009230 tx=0x7fd1b8005d50 comp rx=0 tx=0).stop 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.019+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 shutdown_connections 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.019+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1b40a3c40 0x7fd1b40a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.019+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1c8102a50 0x7fd1c819f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.019+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1c8102090 0x7fd1c81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.019+0000 7fd1cdb04640 1 --2- 172.21.15.86:0/3572753057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c8100f40 0x7fd1c81983a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.019+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 >> 172.21.15.86:0/3572753057 conn(0x7fd1c80fc4b0 msgr2=0x7fd1c81054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:20.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.019+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 shutdown_connections 2024-04-16T05:33:20.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.019+0000 7fd1cdb04640 1 -- 172.21.15.86:0/3572753057 wait complete. 2024-04-16T05:33:20.028 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-04-16T05:33:19.482133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597328,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":318,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365104,"kb_used_data":11776,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576720,"statfs":{"total":1151940427776,"available":1151566561280,"internally_reserved":0,"allocated":12058624,"data_stored":7633749,"data_compressed":318378,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4550,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":92,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003147"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898114+0000","last_change":"2024-04-16T05:32:55.507548+0000","last_active":"2024-04-16T05:33:01.898114+0000","last_peered":"2024-04-16T05:33:01.898114+0000","last_clean":"2024-04-16T05:33:01.898114+0000","last_became_active":"2024-04-16T05:32:55.507200+0000","last_became_peered":"2024-04-16T05:32:55.507200+0000","last_unstale":"2024-04-16T05:33:01.898114+0000","last_undegraded":"2024-04-16T05:33:01.898114+0000","last_fullsized":"2024-04-16T05:33:01.898114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:58:07.842226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896558+0000","last_change":"2024-04-16T05:32:55.509792+0000","last_active":"2024-04-16T05:33:01.896558+0000","last_peered":"2024-04-16T05:33:01.896558+0000","last_clean":"2024-04-16T05:33:01.896558+0000","last_became_active":"2024-04-16T05:32:55.509442+0000","last_became_peered":"2024-04-16T05:32:55.509442+0000","last_unstale":"2024-04-16T05:33:01.896558+0000","last_undegraded":"2024-04-16T05:33:01.896558+0000","last_fullsized":"2024-04-16T05:33:01.896558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:08:42.040875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896165+0000","last_change":"2024-04-16T05:32:55.505664+0000","last_active":"2024-04-16T05:33:01.896165+0000","last_peered":"2024-04-16T05:33:01.896165+0000","last_clean":"2024-04-16T05:33:01.896165+0000","last_became_active":"2024-04-16T05:32:55.505281+0000","last_became_peered":"2024-04-16T05:32:55.505281+0000","last_unstale":"2024-04-16T05:33:01.896165+0000","last_undegraded":"2024-04-16T05:33:01.896165+0000","last_fullsized":"2024-04-16T05:33:01.896165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:33.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897618+0000","last_change":"2024-04-16T05:32:55.517072+0000","last_active":"2024-04-16T05:33:01.897618+0000","last_peered":"2024-04-16T05:33:01.897618+0000","last_clean":"2024-04-16T05:33:01.897618+0000","last_became_active":"2024-04-16T05:32:55.516648+0000","last_became_peered":"2024-04-16T05:32:55.516648+0000","last_unstale":"2024-04-16T05:33:01.897618+0000","last_undegraded":"2024-04-16T05:33:01.897618+0000","last_fullsized":"2024-04-16T05:33:01.897618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:41.735720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896639+0000","last_change":"2024-04-16T05:32:55.506773+0000","last_active":"2024-04-16T05:33:01.896639+0000","last_peered":"2024-04-16T05:33:01.896639+0000","last_clean":"2024-04-16T05:33:01.896639+0000","last_became_active":"2024-04-16T05:32:55.506414+0000","last_became_peered":"2024-04-16T05:32:55.506414+0000","last_unstale":"2024-04-16T05:33:01.896639+0000","last_undegraded":"2024-04-16T05:33:01.896639+0000","last_fullsized":"2024-04-16T05:33:01.896639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:59.254334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708559+0000","last_change":"2024-04-16T05:32:55.517875+0000","last_active":"2024-04-16T05:33:01.708559+0000","last_peered":"2024-04-16T05:33:01.708559+0000","last_clean":"2024-04-16T05:33:01.708559+0000","last_became_active":"2024-04-16T05:32:55.517546+0000","last_became_peered":"2024-04-16T05:32:55.517546+0000","last_unstale":"2024-04-16T05:33:01.708559+0000","last_undegraded":"2024-04-16T05:33:01.708559+0000","last_fullsized":"2024-04-16T05:33:01.708559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:12:15.098439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534118+0000","last_change":"2024-04-16T05:32:55.509036+0000","last_active":"2024-04-16T05:33:01.534118+0000","last_peered":"2024-04-16T05:33:01.534118+0000","last_clean":"2024-04-16T05:33:01.534118+0000","last_became_active":"2024-04-16T05:32:55.508690+0000","last_became_peered":"2024-04-16T05:32:55.508690+0000","last_unstale":"2024-04-16T05:33:01.534118+0000","last_undegraded":"2024-04-16T05:33:01.534118+0000","last_fullsized":"2024-04-16T05:33:01.534118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:26.724281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896120+0000","last_change":"2024-04-16T05:32:55.510146+0000","last_active":"2024-04-16T05:33:01.896120+0000","last_peered":"2024-04-16T05:33:01.896120+0000","last_clean":"2024-04-16T05:33:01.896120+0000","last_became_active":"2024-04-16T05:32:55.508665+0000","last_became_peered":"2024-04-16T05:32:55.508665+0000","last_unstale":"2024-04-16T05:33:01.896120+0000","last_undegraded":"2024-04-16T05:33:01.896120+0000","last_fullsized":"2024-04-16T05:33:01.896120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:14.158221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898385+0000","last_change":"2024-04-16T05:32:55.513738+0000","last_active":"2024-04-16T05:33:01.898385+0000","last_peered":"2024-04-16T05:33:01.898385+0000","last_clean":"2024-04-16T05:33:01.898385+0000","last_became_active":"2024-04-16T05:32:55.513374+0000","last_became_peered":"2024-04-16T05:32:55.513374+0000","last_unstale":"2024-04-16T05:33:01.898385+0000","last_undegraded":"2024-04-16T05:33:01.898385+0000","last_fullsized":"2024-04-16T05:33:01.898385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:06:23.390908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708971+0000","last_change":"2024-04-16T05:32:55.519276+0000","last_active":"2024-04-16T05:33:01.708971+0000","last_peered":"2024-04-16T05:33:01.708971+0000","last_clean":"2024-04-16T05:33:01.708971+0000","last_became_active":"2024-04-16T05:32:55.518729+0000","last_became_peered":"2024-04-16T05:32:55.518729+0000","last_unstale":"2024-04-16T05:33:01.708971+0000","last_undegraded":"2024-04-16T05:33:01.708971+0000","last_fullsized":"2024-04-16T05:33:01.708971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:44:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534782+0000","last_change":"2024-04-16T05:32:55.516379+0000","last_active":"2024-04-16T05:33:01.534782+0000","last_peered":"2024-04-16T05:33:01.534782+0000","last_clean":"2024-04-16T05:33:01.534782+0000","last_became_active":"2024-04-16T05:32:55.515574+0000","last_became_peered":"2024-04-16T05:32:55.515574+0000","last_unstale":"2024-04-16T05:33:01.534782+0000","last_undegraded":"2024-04-16T05:33:01.534782+0000","last_fullsized":"2024-04-16T05:33:01.534782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:16:01.744380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897635+0000","last_change":"2024-04-16T05:32:55.518988+0000","last_active":"2024-04-16T05:33:01.897635+0000","last_peered":"2024-04-16T05:33:01.897635+0000","last_clean":"2024-04-16T05:33:01.897635+0000","last_became_active":"2024-04-16T05:32:55.518214+0000","last_became_peered":"2024-04-16T05:32:55.518214+0000","last_unstale":"2024-04-16T05:33:01.897635+0000","last_undegraded":"2024-04-16T05:33:01.897635+0000","last_fullsized":"2024-04-16T05:33:01.897635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:12:57.645418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530497+0000","last_change":"2024-04-16T05:32:55.509682+0000","last_active":"2024-04-16T05:33:01.530497+0000","last_peered":"2024-04-16T05:33:01.530497+0000","last_clean":"2024-04-16T05:33:01.530497+0000","last_became_active":"2024-04-16T05:32:55.509383+0000","last_became_peered":"2024-04-16T05:32:55.509383+0000","last_unstale":"2024-04-16T05:33:01.530497+0000","last_undegraded":"2024-04-16T05:33:01.530497+0000","last_fullsized":"2024-04-16T05:33:01.530497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:25:23.487003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.895881+0000","last_change":"2024-04-16T05:32:55.506338+0000","last_active":"2024-04-16T05:33:01.895881+0000","last_peered":"2024-04-16T05:33:01.895881+0000","last_clean":"2024-04-16T05:33:01.895881+0000","last_became_active":"2024-04-16T05:32:55.506000+0000","last_became_peered":"2024-04-16T05:32:55.506000+0000","last_unstale":"2024-04-16T05:33:01.895881+0000","last_undegraded":"2024-04-16T05:33:01.895881+0000","last_fullsized":"2024-04-16T05:33:01.895881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:37:02.296515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896222+0000","last_change":"2024-04-16T05:32:55.516897+0000","last_active":"2024-04-16T05:33:01.896222+0000","last_peered":"2024-04-16T05:33:01.896222+0000","last_clean":"2024-04-16T05:33:01.896222+0000","last_became_active":"2024-04-16T05:32:55.516489+0000","last_became_peered":"2024-04-16T05:32:55.516489+0000","last_unstale":"2024-04-16T05:33:01.896222+0000","last_undegraded":"2024-04-16T05:33:01.896222+0000","last_fullsized":"2024-04-16T05:33:01.896222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:26:08.289501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896820+0000","last_change":"2024-04-16T05:32:55.507374+0000","last_active":"2024-04-16T05:33:01.896820+0000","last_peered":"2024-04-16T05:33:01.896820+0000","last_clean":"2024-04-16T05:33:01.896820+0000","last_became_active":"2024-04-16T05:32:55.507045+0000","last_became_peered":"2024-04-16T05:32:55.507045+0000","last_unstale":"2024-04-16T05:33:01.896820+0000","last_undegraded":"2024-04-16T05:33:01.896820+0000","last_fullsized":"2024-04-16T05:33:01.896820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:35.968704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897727+0000","last_change":"2024-04-16T05:32:55.508845+0000","last_active":"2024-04-16T05:33:01.897727+0000","last_peered":"2024-04-16T05:33:01.897727+0000","last_clean":"2024-04-16T05:33:01.897727+0000","last_became_active":"2024-04-16T05:32:55.508527+0000","last_became_peered":"2024-04-16T05:32:55.508527+0000","last_unstale":"2024-04-16T05:33:01.897727+0000","last_undegraded":"2024-04-16T05:33:01.897727+0000","last_fullsized":"2024-04-16T05:33:01.897727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:20:15.033043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713430+0000","last_change":"2024-04-16T05:32:55.506059+0000","last_active":"2024-04-16T05:33:01.713430+0000","last_peered":"2024-04-16T05:33:01.713430+0000","last_clean":"2024-04-16T05:33:01.713430+0000","last_became_active":"2024-04-16T05:32:55.505728+0000","last_became_peered":"2024-04-16T05:32:55.505728+0000","last_unstale":"2024-04-16T05:33:01.713430+0000","last_undegraded":"2024-04-16T05:33:01.713430+0000","last_fullsized":"2024-04-16T05:33:01.713430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:19.253543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898293+0000","last_change":"2024-04-16T05:32:55.506791+0000","last_active":"2024-04-16T05:33:01.898293+0000","last_peered":"2024-04-16T05:33:01.898293+0000","last_clean":"2024-04-16T05:33:01.898293+0000","last_became_active":"2024-04-16T05:32:55.506379+0000","last_became_peered":"2024-04-16T05:32:55.506379+0000","last_unstale":"2024-04-16T05:33:01.898293+0000","last_undegraded":"2024-04-16T05:33:01.898293+0000","last_fullsized":"2024-04-16T05:33:01.898293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:09:11.457144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897953+0000","last_change":"2024-04-16T05:32:59.524001+0000","last_active":"2024-04-16T05:33:01.897953+0000","last_peered":"2024-04-16T05:33:01.897953+0000","last_clean":"2024-04-16T05:33:01.897953+0000","last_became_active":"2024-04-16T05:32:59.522443+0000","last_became_peered":"2024-04-16T05:32:59.522443+0000","last_unstale":"2024-04-16T05:33:01.897953+0000","last_undegraded":"2024-04-16T05:33:01.897953+0000","last_fullsized":"2024-04-16T05:33:01.897953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:16:14.777815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:18.621756+0000","last_change":"2024-04-16T05:32:54.504826+0000","last_active":"2024-04-16T05:33:18.621756+0000","last_peered":"2024-04-16T05:33:18.621756+0000","last_clean":"2024-04-16T05:33:18.621756+0000","last_became_active":"2024-04-16T05:32:54.504526+0000","last_became_peered":"2024-04-16T05:32:54.504526+0000","last_unstale":"2024-04-16T05:33:18.621756+0000","last_undegraded":"2024-04-16T05:33:18.621756+0000","last_fullsized":"2024-04-16T05:33:18.621756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:21:28.434692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896636+0000","last_change":"2024-04-16T05:32:55.514760+0000","last_active":"2024-04-16T05:33:01.896636+0000","last_peered":"2024-04-16T05:33:01.896636+0000","last_clean":"2024-04-16T05:33:01.896636+0000","last_became_active":"2024-04-16T05:32:55.514421+0000","last_became_peered":"2024-04-16T05:32:55.514421+0000","last_unstale":"2024-04-16T05:33:01.896636+0000","last_undegraded":"2024-04-16T05:33:01.896636+0000","last_fullsized":"2024-04-16T05:33:01.896636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:24:52.894382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896956+0000","last_change":"2024-04-16T05:32:59.524687+0000","last_active":"2024-04-16T05:33:01.896956+0000","last_peered":"2024-04-16T05:33:01.896956+0000","last_clean":"2024-04-16T05:33:01.896956+0000","last_became_active":"2024-04-16T05:32:59.524228+0000","last_became_peered":"2024-04-16T05:32:59.524228+0000","last_unstale":"2024-04-16T05:33:01.896956+0000","last_undegraded":"2024-04-16T05:33:01.896956+0000","last_fullsized":"2024-04-16T05:33:01.896956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:23:15.244359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896680+0000","last_change":"2024-04-16T05:32:54.504062+0000","last_active":"2024-04-16T05:33:01.896680+0000","last_peered":"2024-04-16T05:33:01.896680+0000","last_clean":"2024-04-16T05:33:01.896680+0000","last_became_active":"2024-04-16T05:32:54.503745+0000","last_became_peered":"2024-04-16T05:32:54.503745+0000","last_unstale":"2024-04-16T05:33:01.896680+0000","last_undegraded":"2024-04-16T05:33:01.896680+0000","last_fullsized":"2024-04-16T05:33:01.896680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:29:34.157937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896624+0000","last_change":"2024-04-16T05:32:55.509820+0000","last_active":"2024-04-16T05:33:01.896624+0000","last_peered":"2024-04-16T05:33:01.896624+0000","last_clean":"2024-04-16T05:33:01.896624+0000","last_became_active":"2024-04-16T05:32:55.509473+0000","last_became_peered":"2024-04-16T05:32:55.509473+0000","last_unstale":"2024-04-16T05:33:01.896624+0000","last_undegraded":"2024-04-16T05:33:01.896624+0000","last_fullsized":"2024-04-16T05:33:01.896624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:12:22.433189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897632+0000","last_change":"2024-04-16T05:32:59.524462+0000","last_active":"2024-04-16T05:33:01.897632+0000","last_peered":"2024-04-16T05:33:01.897632+0000","last_clean":"2024-04-16T05:33:01.897632+0000","last_became_active":"2024-04-16T05:32:59.523989+0000","last_became_peered":"2024-04-16T05:32:59.523989+0000","last_unstale":"2024-04-16T05:33:01.897632+0000","last_undegraded":"2024-04-16T05:33:01.897632+0000","last_fullsized":"2024-04-16T05:33:01.897632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:13:11.687257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:16.608633+0000","last_change":"2024-04-16T05:32:54.511469+0000","last_active":"2024-04-16T05:33:16.608633+0000","last_peered":"2024-04-16T05:33:16.608633+0000","last_clean":"2024-04-16T05:33:16.608633+0000","last_became_active":"2024-04-16T05:32:54.511120+0000","last_became_peered":"2024-04-16T05:32:54.511120+0000","last_unstale":"2024-04-16T05:33:16.608633+0000","last_undegraded":"2024-04-16T05:33:16.608633+0000","last_fullsized":"2024-04-16T05:33:16.608633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897788+0000","last_change":"2024-04-16T05:32:55.512308+0000","last_active":"2024-04-16T05:33:01.897788+0000","last_peered":"2024-04-16T05:33:01.897788+0000","last_clean":"2024-04-16T05:33:01.897788+0000","last_became_active":"2024-04-16T05:32:55.511625+0000","last_became_peered":"2024-04-16T05:32:55.511625+0000","last_unstale":"2024-04-16T05:33:01.897788+0000","last_undegraded":"2024-04-16T05:33:01.897788+0000","last_fullsized":"2024-04-16T05:33:01.897788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:29:49.323092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897419+0000","last_change":"2024-04-16T05:32:59.519306+0000","last_active":"2024-04-16T05:33:01.897419+0000","last_peered":"2024-04-16T05:33:01.897419+0000","last_clean":"2024-04-16T05:33:01.897419+0000","last_became_active":"2024-04-16T05:32:59.518797+0000","last_became_peered":"2024-04-16T05:32:59.518797+0000","last_unstale":"2024-04-16T05:33:01.897419+0000","last_undegraded":"2024-04-16T05:33:01.897419+0000","last_fullsized":"2024-04-16T05:33:01.897419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:04:12.389004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896718+0000","last_change":"2024-04-16T05:32:54.505102+0000","last_active":"2024-04-16T05:33:01.896718+0000","last_peered":"2024-04-16T05:33:01.896718+0000","last_clean":"2024-04-16T05:33:01.896718+0000","last_became_active":"2024-04-16T05:32:54.504806+0000","last_became_peered":"2024-04-16T05:32:54.504806+0000","last_unstale":"2024-04-16T05:33:01.896718+0000","last_undegraded":"2024-04-16T05:33:01.896718+0000","last_fullsized":"2024-04-16T05:33:01.896718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:19.467690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531783+0000","last_change":"2024-04-16T05:32:55.517097+0000","last_active":"2024-04-16T05:33:01.531783+0000","last_peered":"2024-04-16T05:33:01.531783+0000","last_clean":"2024-04-16T05:33:01.531783+0000","last_became_active":"2024-04-16T05:32:55.516717+0000","last_became_peered":"2024-04-16T05:32:55.516717+0000","last_unstale":"2024-04-16T05:33:01.531783+0000","last_undegraded":"2024-04-16T05:33:01.531783+0000","last_fullsized":"2024-04-16T05:33:01.531783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:10:44.544935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898275+0000","last_change":"2024-04-16T05:32:59.521429+0000","last_active":"2024-04-16T05:33:01.898275+0000","last_peered":"2024-04-16T05:33:01.898275+0000","last_clean":"2024-04-16T05:33:01.898275+0000","last_became_active":"2024-04-16T05:32:59.520789+0000","last_became_peered":"2024-04-16T05:32:59.520789+0000","last_unstale":"2024-04-16T05:33:01.898275+0000","last_undegraded":"2024-04-16T05:33:01.898275+0000","last_fullsized":"2024-04-16T05:33:01.898275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:40:14.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.592641+0000","last_change":"2024-04-16T05:32:54.507374+0000","last_active":"2024-04-16T05:33:14.592641+0000","last_peered":"2024-04-16T05:33:14.592641+0000","last_clean":"2024-04-16T05:33:14.592641+0000","last_became_active":"2024-04-16T05:32:54.507055+0000","last_became_peered":"2024-04-16T05:32:54.507055+0000","last_unstale":"2024-04-16T05:33:14.592641+0000","last_undegraded":"2024-04-16T05:33:14.592641+0000","last_fullsized":"2024-04-16T05:33:14.592641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:00:57.761971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896786+0000","last_change":"2024-04-16T05:32:55.513980+0000","last_active":"2024-04-16T05:33:01.896786+0000","last_peered":"2024-04-16T05:33:01.896786+0000","last_clean":"2024-04-16T05:33:01.896786+0000","last_became_active":"2024-04-16T05:32:55.513622+0000","last_became_peered":"2024-04-16T05:32:55.513622+0000","last_unstale":"2024-04-16T05:33:01.896786+0000","last_undegraded":"2024-04-16T05:33:01.896786+0000","last_fullsized":"2024-04-16T05:33:01.896786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:16:10.018381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898152+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:33:01.898152+0000","last_peered":"2024-04-16T05:33:01.898152+0000","last_clean":"2024-04-16T05:33:01.898152+0000","last_became_active":"2024-04-16T05:32:59.523032+0000","last_became_peered":"2024-04-16T05:32:59.523032+0000","last_unstale":"2024-04-16T05:33:01.898152+0000","last_undegraded":"2024-04-16T05:33:01.898152+0000","last_fullsized":"2024-04-16T05:33:01.898152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:33:17.897241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708502+0000","last_change":"2024-04-16T05:32:54.504527+0000","last_active":"2024-04-16T05:33:01.708502+0000","last_peered":"2024-04-16T05:33:01.708502+0000","last_clean":"2024-04-16T05:33:01.708502+0000","last_became_active":"2024-04-16T05:32:54.504187+0000","last_became_peered":"2024-04-16T05:32:54.504187+0000","last_unstale":"2024-04-16T05:33:01.708502+0000","last_undegraded":"2024-04-16T05:33:01.708502+0000","last_fullsized":"2024-04-16T05:33:01.708502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.414362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531776+0000","last_change":"2024-04-16T05:32:55.520270+0000","last_active":"2024-04-16T05:33:01.531776+0000","last_peered":"2024-04-16T05:33:01.531776+0000","last_clean":"2024-04-16T05:33:01.531776+0000","last_became_active":"2024-04-16T05:32:55.519895+0000","last_became_peered":"2024-04-16T05:32:55.519895+0000","last_unstale":"2024-04-16T05:33:01.531776+0000","last_undegraded":"2024-04-16T05:33:01.531776+0000","last_fullsized":"2024-04-16T05:33:01.531776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:09.967836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897725+0000","last_change":"2024-04-16T05:32:59.518997+0000","last_active":"2024-04-16T05:33:01.897725+0000","last_peered":"2024-04-16T05:33:01.897725+0000","last_clean":"2024-04-16T05:33:01.897725+0000","last_became_active":"2024-04-16T05:32:59.518558+0000","last_became_peered":"2024-04-16T05:32:59.518558+0000","last_unstale":"2024-04-16T05:33:01.897725+0000","last_undegraded":"2024-04-16T05:33:01.897725+0000","last_fullsized":"2024-04-16T05:33:01.897725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:04:30.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.602246+0000","last_change":"2024-04-16T05:32:54.502074+0000","last_active":"2024-04-16T05:33:15.602246+0000","last_peered":"2024-04-16T05:33:15.602246+0000","last_clean":"2024-04-16T05:33:15.602246+0000","last_became_active":"2024-04-16T05:32:54.501730+0000","last_became_peered":"2024-04-16T05:32:54.501730+0000","last_unstale":"2024-04-16T05:33:15.602246+0000","last_undegraded":"2024-04-16T05:33:15.602246+0000","last_fullsized":"2024-04-16T05:33:15.602246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:05:02.929799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896032+0000","last_change":"2024-04-16T05:32:55.504557+0000","last_active":"2024-04-16T05:33:01.896032+0000","last_peered":"2024-04-16T05:33:01.896032+0000","last_clean":"2024-04-16T05:33:01.896032+0000","last_became_active":"2024-04-16T05:32:55.504244+0000","last_became_peered":"2024-04-16T05:32:55.504244+0000","last_unstale":"2024-04-16T05:33:01.896032+0000","last_undegraded":"2024-04-16T05:33:01.896032+0000","last_fullsized":"2024-04-16T05:33:01.896032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:33:40.500436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896505+0000","last_change":"2024-04-16T05:32:59.524681+0000","last_active":"2024-04-16T05:33:01.896505+0000","last_peered":"2024-04-16T05:33:01.896505+0000","last_clean":"2024-04-16T05:33:01.896505+0000","last_became_active":"2024-04-16T05:32:59.524217+0000","last_became_peered":"2024-04-16T05:32:59.524217+0000","last_unstale":"2024-04-16T05:33:01.896505+0000","last_undegraded":"2024-04-16T05:33:01.896505+0000","last_fullsized":"2024-04-16T05:33:01.896505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:28.314442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531361+0000","last_change":"2024-04-16T05:32:54.508242+0000","last_active":"2024-04-16T05:33:01.531361+0000","last_peered":"2024-04-16T05:33:01.531361+0000","last_clean":"2024-04-16T05:33:01.531361+0000","last_became_active":"2024-04-16T05:32:54.507782+0000","last_became_peered":"2024-04-16T05:32:54.507782+0000","last_unstale":"2024-04-16T05:33:01.531361+0000","last_undegraded":"2024-04-16T05:33:01.531361+0000","last_fullsized":"2024-04-16T05:33:01.531361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:27:26.805716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713049+0000","last_change":"2024-04-16T05:32:55.508985+0000","last_active":"2024-04-16T05:33:01.713049+0000","last_peered":"2024-04-16T05:33:01.713049+0000","last_clean":"2024-04-16T05:33:01.713049+0000","last_became_active":"2024-04-16T05:32:55.508659+0000","last_became_peered":"2024-04-16T05:32:55.508659+0000","last_unstale":"2024-04-16T05:33:01.713049+0000","last_undegraded":"2024-04-16T05:33:01.713049+0000","last_fullsized":"2024-04-16T05:33:01.713049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:43:41.534155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896485+0000","last_change":"2024-04-16T05:32:59.521257+0000","last_active":"2024-04-16T05:33:01.896485+0000","last_peered":"2024-04-16T05:33:01.896485+0000","last_clean":"2024-04-16T05:33:01.896485+0000","last_became_active":"2024-04-16T05:32:59.520848+0000","last_became_peered":"2024-04-16T05:32:59.520848+0000","last_unstale":"2024-04-16T05:33:01.896485+0000","last_undegraded":"2024-04-16T05:33:01.896485+0000","last_fullsized":"2024-04-16T05:33:01.896485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:14.219378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.590987+0000","last_change":"2024-04-16T05:32:54.500684+0000","last_active":"2024-04-16T05:33:14.590987+0000","last_peered":"2024-04-16T05:33:14.590987+0000","last_clean":"2024-04-16T05:33:14.590987+0000","last_became_active":"2024-04-16T05:32:54.500290+0000","last_became_peered":"2024-04-16T05:32:54.500290+0000","last_unstale":"2024-04-16T05:33:14.590987+0000","last_undegraded":"2024-04-16T05:33:14.590987+0000","last_fullsized":"2024-04-16T05:33:14.590987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:20:55.263313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898402+0000","last_change":"2024-04-16T05:32:55.511224+0000","last_active":"2024-04-16T05:33:01.898402+0000","last_peered":"2024-04-16T05:33:01.898402+0000","last_clean":"2024-04-16T05:33:01.898402+0000","last_became_active":"2024-04-16T05:32:55.510891+0000","last_became_peered":"2024-04-16T05:32:55.510891+0000","last_unstale":"2024-04-16T05:33:01.898402+0000","last_undegraded":"2024-04-16T05:33:01.898402+0000","last_fullsized":"2024-04-16T05:33:01.898402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:27:24.894658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.712961+0000","last_change":"2024-04-16T05:32:59.521695+0000","last_active":"2024-04-16T05:33:01.712961+0000","last_peered":"2024-04-16T05:33:01.712961+0000","last_clean":"2024-04-16T05:33:01.712961+0000","last_became_active":"2024-04-16T05:32:59.520569+0000","last_became_peered":"2024-04-16T05:32:59.520569+0000","last_unstale":"2024-04-16T05:33:01.712961+0000","last_undegraded":"2024-04-16T05:33:01.712961+0000","last_fullsized":"2024-04-16T05:33:01.712961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:01:14.726043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.590135+0000","last_change":"2024-04-16T05:32:54.505058+0000","last_active":"2024-04-16T05:33:14.590135+0000","last_peered":"2024-04-16T05:33:14.590135+0000","last_clean":"2024-04-16T05:33:14.590135+0000","last_became_active":"2024-04-16T05:32:54.504636+0000","last_became_peered":"2024-04-16T05:32:54.504636+0000","last_unstale":"2024-04-16T05:33:14.590135+0000","last_undegraded":"2024-04-16T05:33:14.590135+0000","last_fullsized":"2024-04-16T05:33:14.590135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:49:14.429468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896828+0000","last_change":"2024-04-16T05:32:55.518752+0000","last_active":"2024-04-16T05:33:01.896828+0000","last_peered":"2024-04-16T05:33:01.896828+0000","last_clean":"2024-04-16T05:33:01.896828+0000","last_became_active":"2024-04-16T05:32:55.518413+0000","last_became_peered":"2024-04-16T05:32:55.518413+0000","last_unstale":"2024-04-16T05:33:01.896828+0000","last_undegraded":"2024-04-16T05:33:01.896828+0000","last_fullsized":"2024-04-16T05:33:01.896828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:40:32.381456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898199+0000","last_change":"2024-04-16T05:32:59.520394+0000","last_active":"2024-04-16T05:33:01.898199+0000","last_peered":"2024-04-16T05:33:01.898199+0000","last_clean":"2024-04-16T05:33:01.898199+0000","last_became_active":"2024-04-16T05:32:59.519947+0000","last_became_peered":"2024-04-16T05:32:59.519947+0000","last_unstale":"2024-04-16T05:33:01.898199+0000","last_undegraded":"2024-04-16T05:33:01.898199+0000","last_fullsized":"2024-04-16T05:33:01.898199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:33:27.647919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531903+0000","last_change":"2024-04-16T05:32:54.505350+0000","last_active":"2024-04-16T05:33:01.531903+0000","last_peered":"2024-04-16T05:33:01.531903+0000","last_clean":"2024-04-16T05:33:01.531903+0000","last_became_active":"2024-04-16T05:32:54.505011+0000","last_became_peered":"2024-04-16T05:32:54.505011+0000","last_unstale":"2024-04-16T05:33:01.531903+0000","last_undegraded":"2024-04-16T05:33:01.531903+0000","last_fullsized":"2024-04-16T05:33:01.531903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:03:00.276862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897977+0000","last_change":"2024-04-16T05:32:55.517961+0000","last_active":"2024-04-16T05:33:01.897977+0000","last_peered":"2024-04-16T05:33:01.897977+0000","last_clean":"2024-04-16T05:33:01.897977+0000","last_became_active":"2024-04-16T05:32:55.517713+0000","last_became_peered":"2024-04-16T05:32:55.517713+0000","last_unstale":"2024-04-16T05:33:01.897977+0000","last_undegraded":"2024-04-16T05:33:01.897977+0000","last_fullsized":"2024-04-16T05:33:01.897977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:41:30.801833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713118+0000","last_change":"2024-04-16T05:32:59.519660+0000","last_active":"2024-04-16T05:33:01.713118+0000","last_peered":"2024-04-16T05:33:01.713118+0000","last_clean":"2024-04-16T05:33:01.713118+0000","last_became_active":"2024-04-16T05:32:59.518198+0000","last_became_peered":"2024-04-16T05:32:59.518198+0000","last_unstale":"2024-04-16T05:33:01.713118+0000","last_undegraded":"2024-04-16T05:33:01.713118+0000","last_fullsized":"2024-04-16T05:33:01.713118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:03:10.096252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591382+0000","last_change":"2024-04-16T05:32:54.501530+0000","last_active":"2024-04-16T05:33:14.591382+0000","last_peered":"2024-04-16T05:33:14.591382+0000","last_clean":"2024-04-16T05:33:14.591382+0000","last_became_active":"2024-04-16T05:32:54.501159+0000","last_became_peered":"2024-04-16T05:32:54.501159+0000","last_unstale":"2024-04-16T05:33:14.591382+0000","last_undegraded":"2024-04-16T05:33:14.591382+0000","last_fullsized":"2024-04-16T05:33:14.591382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:26:22.240137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897881+0000","last_change":"2024-04-16T05:32:55.512210+0000","last_active":"2024-04-16T05:33:01.897881+0000","last_peered":"2024-04-16T05:33:01.897881+0000","last_clean":"2024-04-16T05:33:01.897881+0000","last_became_active":"2024-04-16T05:32:55.511672+0000","last_became_peered":"2024-04-16T05:32:55.511672+0000","last_unstale":"2024-04-16T05:33:01.897881+0000","last_undegraded":"2024-04-16T05:33:01.897881+0000","last_fullsized":"2024-04-16T05:33:01.897881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:10:53.755803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.532177+0000","last_change":"2024-04-16T05:32:59.527552+0000","last_active":"2024-04-16T05:33:01.532177+0000","last_peered":"2024-04-16T05:33:01.532177+0000","last_clean":"2024-04-16T05:33:01.532177+0000","last_became_active":"2024-04-16T05:32:59.526919+0000","last_became_peered":"2024-04-16T05:32:59.526919+0000","last_unstale":"2024-04-16T05:33:01.532177+0000","last_undegraded":"2024-04-16T05:33:01.532177+0000","last_fullsized":"2024-04-16T05:33:01.532177+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:00:46.342383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591098+0000","last_change":"2024-04-16T05:32:54.494589+0000","last_active":"2024-04-16T05:33:14.591098+0000","last_peered":"2024-04-16T05:33:14.591098+0000","last_clean":"2024-04-16T05:33:14.591098+0000","last_became_active":"2024-04-16T05:32:54.494221+0000","last_became_peered":"2024-04-16T05:32:54.494221+0000","last_unstale":"2024-04-16T05:33:14.591098+0000","last_undegraded":"2024-04-16T05:33:14.591098+0000","last_fullsized":"2024-04-16T05:33:14.591098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:17:53.245359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530305+0000","last_change":"2024-04-16T05:32:55.502724+0000","last_active":"2024-04-16T05:33:01.530305+0000","last_peered":"2024-04-16T05:33:01.530305+0000","last_clean":"2024-04-16T05:33:01.530305+0000","last_became_active":"2024-04-16T05:32:55.502415+0000","last_became_peered":"2024-04-16T05:32:55.502415+0000","last_unstale":"2024-04-16T05:33:01.530305+0000","last_undegraded":"2024-04-16T05:33:01.530305+0000","last_fullsized":"2024-04-16T05:33:01.530305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:20:44.443926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897486+0000","last_change":"2024-04-16T05:32:59.522243+0000","last_active":"2024-04-16T05:33:01.897486+0000","last_peered":"2024-04-16T05:33:01.897486+0000","last_clean":"2024-04-16T05:33:01.897486+0000","last_became_active":"2024-04-16T05:32:59.521823+0000","last_became_peered":"2024-04-16T05:32:59.521823+0000","last_unstale":"2024-04-16T05:33:01.897486+0000","last_undegraded":"2024-04-16T05:33:01.897486+0000","last_fullsized":"2024-04-16T05:33:01.897486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:09:11.158434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591852+0000","last_change":"2024-04-16T05:32:54.507505+0000","last_active":"2024-04-16T05:33:14.591852+0000","last_peered":"2024-04-16T05:33:14.591852+0000","last_clean":"2024-04-16T05:33:14.591852+0000","last_became_active":"2024-04-16T05:32:54.506558+0000","last_became_peered":"2024-04-16T05:32:54.506558+0000","last_unstale":"2024-04-16T05:33:14.591852+0000","last_undegraded":"2024-04-16T05:33:14.591852+0000","last_fullsized":"2024-04-16T05:33:14.591852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:36:17.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534342+0000","last_change":"2024-04-16T05:32:55.518998+0000","last_active":"2024-04-16T05:33:01.534342+0000","last_peered":"2024-04-16T05:33:01.534342+0000","last_clean":"2024-04-16T05:33:01.534342+0000","last_became_active":"2024-04-16T05:32:55.518640+0000","last_became_peered":"2024-04-16T05:32:55.518640+0000","last_unstale":"2024-04-16T05:33:01.534342+0000","last_undegraded":"2024-04-16T05:33:01.534342+0000","last_fullsized":"2024-04-16T05:33:01.534342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:42:44.667504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713373+0000","last_change":"2024-04-16T05:32:59.519050+0000","last_active":"2024-04-16T05:33:01.713373+0000","last_peered":"2024-04-16T05:33:01.713373+0000","last_clean":"2024-04-16T05:33:01.713373+0000","last_became_active":"2024-04-16T05:32:59.518663+0000","last_became_peered":"2024-04-16T05:32:59.518663+0000","last_unstale":"2024-04-16T05:33:01.713373+0000","last_undegraded":"2024-04-16T05:33:01.713373+0000","last_fullsized":"2024-04-16T05:33:01.713373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:36:19.649605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.600587+0000","last_change":"2024-04-16T05:32:54.500940+0000","last_active":"2024-04-16T05:33:15.600587+0000","last_peered":"2024-04-16T05:33:15.600587+0000","last_clean":"2024-04-16T05:33:15.600587+0000","last_became_active":"2024-04-16T05:32:54.500542+0000","last_became_peered":"2024-04-16T05:32:54.500542+0000","last_unstale":"2024-04-16T05:33:15.600587+0000","last_undegraded":"2024-04-16T05:33:15.600587+0000","last_fullsized":"2024-04-16T05:33:15.600587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:38:00.232244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534185+0000","last_change":"2024-04-16T05:32:55.509889+0000","last_active":"2024-04-16T05:33:01.534185+0000","last_peered":"2024-04-16T05:33:01.534185+0000","last_clean":"2024-04-16T05:33:01.534185+0000","last_became_active":"2024-04-16T05:32:55.508911+0000","last_became_peered":"2024-04-16T05:32:55.508911+0000","last_unstale":"2024-04-16T05:33:01.534185+0000","last_undegraded":"2024-04-16T05:33:01.534185+0000","last_fullsized":"2024-04-16T05:33:01.534185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:14:46.609127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530542+0000","last_change":"2024-04-16T05:32:59.523555+0000","last_active":"2024-04-16T05:33:01.530542+0000","last_peered":"2024-04-16T05:33:01.530542+0000","last_clean":"2024-04-16T05:33:01.530542+0000","last_became_active":"2024-04-16T05:32:59.523187+0000","last_became_peered":"2024-04-16T05:32:59.523187+0000","last_unstale":"2024-04-16T05:33:01.530542+0000","last_undegraded":"2024-04-16T05:33:01.530542+0000","last_fullsized":"2024-04-16T05:33:01.530542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:35.049355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'16","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.602325+0000","last_change":"2024-04-16T05:32:54.503247+0000","last_active":"2024-04-16T05:33:15.602325+0000","last_peered":"2024-04-16T05:33:15.602325+0000","last_clean":"2024-04-16T05:33:15.602325+0000","last_became_active":"2024-04-16T05:32:54.502927+0000","last_became_peered":"2024-04-16T05:32:54.502927+0000","last_unstale":"2024-04-16T05:33:15.602325+0000","last_undegraded":"2024-04-16T05:33:15.602325+0000","last_fullsized":"2024-04-16T05:33:15.602325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:36:17.585827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531264+0000","last_change":"2024-04-16T05:32:55.519265+0000","last_active":"2024-04-16T05:33:01.531264+0000","last_peered":"2024-04-16T05:33:01.531264+0000","last_clean":"2024-04-16T05:33:01.531264+0000","last_became_active":"2024-04-16T05:32:55.518840+0000","last_became_peered":"2024-04-16T05:32:55.518840+0000","last_unstale":"2024-04-16T05:33:01.531264+0000","last_undegraded":"2024-04-16T05:33:01.531264+0000","last_fullsized":"2024-04-16T05:33:01.531264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:56.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531344+0000","last_change":"2024-04-16T05:32:59.525275+0000","last_active":"2024-04-16T05:33:01.531344+0000","last_peered":"2024-04-16T05:33:01.531344+0000","last_clean":"2024-04-16T05:33:01.531344+0000","last_became_active":"2024-04-16T05:32:59.524546+0000","last_became_peered":"2024-04-16T05:32:59.524546+0000","last_unstale":"2024-04-16T05:33:01.531344+0000","last_undegraded":"2024-04-16T05:33:01.531344+0000","last_fullsized":"2024-04-16T05:33:01.531344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:07.561898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898484+0000","last_change":"2024-04-16T05:32:54.506904+0000","last_active":"2024-04-16T05:33:01.898484+0000","last_peered":"2024-04-16T05:33:01.898484+0000","last_clean":"2024-04-16T05:33:01.898484+0000","last_became_active":"2024-04-16T05:32:54.505654+0000","last_became_peered":"2024-04-16T05:32:54.505654+0000","last_unstale":"2024-04-16T05:33:01.898484+0000","last_undegraded":"2024-04-16T05:33:01.898484+0000","last_fullsized":"2024-04-16T05:33:01.898484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:30.444817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896903+0000","last_change":"2024-04-16T05:32:55.515685+0000","last_active":"2024-04-16T05:33:01.896903+0000","last_peered":"2024-04-16T05:33:01.896903+0000","last_clean":"2024-04-16T05:33:01.896903+0000","last_became_active":"2024-04-16T05:32:55.515367+0000","last_became_peered":"2024-04-16T05:32:55.515367+0000","last_unstale":"2024-04-16T05:33:01.896903+0000","last_undegraded":"2024-04-16T05:33:01.896903+0000","last_fullsized":"2024-04-16T05:33:01.896903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:46.670143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896718+0000","last_change":"2024-04-16T05:33:00.419828+0000","last_active":"2024-04-16T05:33:01.896718+0000","last_peered":"2024-04-16T05:33:01.896718+0000","last_clean":"2024-04-16T05:33:01.896718+0000","last_became_active":"2024-04-16T05:33:00.419321+0000","last_became_peered":"2024-04-16T05:33:00.419321+0000","last_unstale":"2024-04-16T05:33:01.896718+0000","last_undegraded":"2024-04-16T05:33:01.896718+0000","last_fullsized":"2024-04-16T05:33:01.896718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:56.694914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713229+0000","last_change":"2024-04-16T05:32:54.505369+0000","last_active":"2024-04-16T05:33:01.713229+0000","last_peered":"2024-04-16T05:33:01.713229+0000","last_clean":"2024-04-16T05:33:01.713229+0000","last_became_active":"2024-04-16T05:32:54.505043+0000","last_became_peered":"2024-04-16T05:32:54.505043+0000","last_unstale":"2024-04-16T05:33:01.713229+0000","last_undegraded":"2024-04-16T05:33:01.713229+0000","last_fullsized":"2024-04-16T05:33:01.713229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:34.242117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708768+0000","last_change":"2024-04-16T05:32:55.509970+0000","last_active":"2024-04-16T05:33:01.708768+0000","last_peered":"2024-04-16T05:33:01.708768+0000","last_clean":"2024-04-16T05:33:01.708768+0000","last_became_active":"2024-04-16T05:32:55.509632+0000","last_became_peered":"2024-04-16T05:32:55.509632+0000","last_unstale":"2024-04-16T05:33:01.708768+0000","last_undegraded":"2024-04-16T05:33:01.708768+0000","last_fullsized":"2024-04-16T05:33:01.708768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:33:55.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530576+0000","last_change":"2024-04-16T05:32:59.527007+0000","last_active":"2024-04-16T05:33:01.530576+0000","last_peered":"2024-04-16T05:33:01.530576+0000","last_clean":"2024-04-16T05:33:01.530576+0000","last_became_active":"2024-04-16T05:32:59.526645+0000","last_became_peered":"2024-04-16T05:32:59.526645+0000","last_unstale":"2024-04-16T05:33:01.530576+0000","last_undegraded":"2024-04-16T05:33:01.530576+0000","last_fullsized":"2024-04-16T05:33:01.530576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:46:06.698912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534301+0000","last_change":"2024-04-16T05:32:54.503409+0000","last_active":"2024-04-16T05:33:01.534301+0000","last_peered":"2024-04-16T05:33:01.534301+0000","last_clean":"2024-04-16T05:33:01.534301+0000","last_became_active":"2024-04-16T05:32:54.503069+0000","last_became_peered":"2024-04-16T05:32:54.503069+0000","last_unstale":"2024-04-16T05:33:01.534301+0000","last_undegraded":"2024-04-16T05:33:01.534301+0000","last_fullsized":"2024-04-16T05:33:01.534301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:45:10.535225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708527+0000","last_change":"2024-04-16T05:32:55.519744+0000","last_active":"2024-04-16T05:33:01.708527+0000","last_peered":"2024-04-16T05:33:01.708527+0000","last_clean":"2024-04-16T05:33:01.708527+0000","last_became_active":"2024-04-16T05:32:55.519091+0000","last_became_peered":"2024-04-16T05:32:55.519091+0000","last_unstale":"2024-04-16T05:33:01.708527+0000","last_undegraded":"2024-04-16T05:33:01.708527+0000","last_fullsized":"2024-04-16T05:33:01.708527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:19.417691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530653+0000","last_change":"2024-04-16T05:33:00.419494+0000","last_active":"2024-04-16T05:33:01.530653+0000","last_peered":"2024-04-16T05:33:01.530653+0000","last_clean":"2024-04-16T05:33:01.530653+0000","last_became_active":"2024-04-16T05:33:00.418954+0000","last_became_peered":"2024-04-16T05:33:00.418954+0000","last_unstale":"2024-04-16T05:33:01.530653+0000","last_undegraded":"2024-04-16T05:33:01.530653+0000","last_fullsized":"2024-04-16T05:33:01.530653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:03:01.835304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531404+0000","last_change":"2024-04-16T05:32:55.518272+0000","last_active":"2024-04-16T05:33:01.531404+0000","last_peered":"2024-04-16T05:33:01.531404+0000","last_clean":"2024-04-16T05:33:01.531404+0000","last_became_active":"2024-04-16T05:32:55.517928+0000","last_became_peered":"2024-04-16T05:32:55.517928+0000","last_unstale":"2024-04-16T05:33:01.531404+0000","last_undegraded":"2024-04-16T05:33:01.531404+0000","last_fullsized":"2024-04-16T05:33:01.531404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:44:47.739994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.601587+0000","last_change":"2024-04-16T05:32:54.503923+0000","last_active":"2024-04-16T05:33:15.601587+0000","last_peered":"2024-04-16T05:33:15.601587+0000","last_clean":"2024-04-16T05:33:15.601587+0000","last_became_active":"2024-04-16T05:32:54.503576+0000","last_became_peered":"2024-04-16T05:32:54.503576+0000","last_unstale":"2024-04-16T05:33:15.601587+0000","last_undegraded":"2024-04-16T05:33:15.601587+0000","last_fullsized":"2024-04-16T05:33:15.601587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:49:36.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534678+0000","last_change":"2024-04-16T05:32:59.517757+0000","last_active":"2024-04-16T05:33:01.534678+0000","last_peered":"2024-04-16T05:33:01.534678+0000","last_clean":"2024-04-16T05:33:01.534678+0000","last_became_active":"2024-04-16T05:32:59.517357+0000","last_became_peered":"2024-04-16T05:32:59.517357+0000","last_unstale":"2024-04-16T05:33:01.534678+0000","last_undegraded":"2024-04-16T05:33:01.534678+0000","last_fullsized":"2024-04-16T05:33:01.534678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:21.767071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896342+0000","last_change":"2024-04-16T05:32:55.514617+0000","last_active":"2024-04-16T05:33:01.896342+0000","last_peered":"2024-04-16T05:33:01.896342+0000","last_clean":"2024-04-16T05:33:01.896342+0000","last_became_active":"2024-04-16T05:32:55.512660+0000","last_became_peered":"2024-04-16T05:32:55.512660+0000","last_unstale":"2024-04-16T05:33:01.896342+0000","last_undegraded":"2024-04-16T05:33:01.896342+0000","last_fullsized":"2024-04-16T05:33:01.896342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:31:45.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591140+0000","last_change":"2024-04-16T05:32:54.498650+0000","last_active":"2024-04-16T05:33:14.591140+0000","last_peered":"2024-04-16T05:33:14.591140+0000","last_clean":"2024-04-16T05:33:14.591140+0000","last_became_active":"2024-04-16T05:32:54.497672+0000","last_became_peered":"2024-04-16T05:32:54.497672+0000","last_unstale":"2024-04-16T05:33:14.591140+0000","last_undegraded":"2024-04-16T05:33:14.591140+0000","last_fullsized":"2024-04-16T05:33:14.591140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:36.908809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.533908+0000","last_change":"2024-04-16T05:31:42.831800+0000","last_active":"2024-04-16T05:33:01.533908+0000","last_peered":"2024-04-16T05:33:01.533908+0000","last_clean":"2024-04-16T05:33:01.533908+0000","last_became_active":"2024-04-16T05:31:42.831391+0000","last_became_peered":"2024-04-16T05:31:42.831391+0000","last_unstale":"2024-04-16T05:33:01.533908+0000","last_undegraded":"2024-04-16T05:33:01.533908+0000","last_fullsized":"2024-04-16T05:33:01.533908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_clean_scrub_stamp":"2024-04-16T05:28:15.969690+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.152803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.532032+0000","last_change":"2024-04-16T05:33:00.420445+0000","last_active":"2024-04-16T05:33:01.532032+0000","last_peered":"2024-04-16T05:33:01.532032+0000","last_clean":"2024-04-16T05:33:01.532032+0000","last_became_active":"2024-04-16T05:33:00.419342+0000","last_became_peered":"2024-04-16T05:33:00.419342+0000","last_unstale":"2024-04-16T05:33:01.532032+0000","last_undegraded":"2024-04-16T05:33:01.532032+0000","last_fullsized":"2024-04-16T05:33:01.532032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:57:15.703251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530088+0000","last_change":"2024-04-16T05:32:55.512324+0000","last_active":"2024-04-16T05:33:01.530088+0000","last_peered":"2024-04-16T05:33:01.530088+0000","last_clean":"2024-04-16T05:33:01.530088+0000","last_became_active":"2024-04-16T05:32:55.511958+0000","last_became_peered":"2024-04-16T05:32:55.511958+0000","last_unstale":"2024-04-16T05:33:01.530088+0000","last_undegraded":"2024-04-16T05:33:01.530088+0000","last_fullsized":"2024-04-16T05:33:01.530088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:38.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.601993+0000","last_change":"2024-04-16T05:32:54.508021+0000","last_active":"2024-04-16T05:33:15.601993+0000","last_peered":"2024-04-16T05:33:15.601993+0000","last_clean":"2024-04-16T05:33:15.601993+0000","last_became_active":"2024-04-16T05:32:54.507645+0000","last_became_peered":"2024-04-16T05:32:54.507645+0000","last_unstale":"2024-04-16T05:33:15.601993+0000","last_undegraded":"2024-04-16T05:33:15.601993+0000","last_fullsized":"2024-04-16T05:33:15.601993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:21:50.721260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896998+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:33:01.896998+0000","last_peered":"2024-04-16T05:33:01.896998+0000","last_clean":"2024-04-16T05:33:01.896998+0000","last_became_active":"2024-04-16T05:32:59.523159+0000","last_became_peered":"2024-04-16T05:32:59.523159+0000","last_unstale":"2024-04-16T05:33:01.896998+0000","last_undegraded":"2024-04-16T05:33:01.896998+0000","last_fullsized":"2024-04-16T05:33:01.896998+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:48:37.696713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:16.608227+0000","last_change":"2024-04-16T05:32:54.503262+0000","last_active":"2024-04-16T05:33:16.608227+0000","last_peered":"2024-04-16T05:33:16.608227+0000","last_clean":"2024-04-16T05:33:16.608227+0000","last_became_active":"2024-04-16T05:32:54.502909+0000","last_became_peered":"2024-04-16T05:32:54.502909+0000","last_unstale":"2024-04-16T05:33:16.608227+0000","last_undegraded":"2024-04-16T05:33:16.608227+0000","last_fullsized":"2024-04-16T05:33:16.608227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:48:32.841921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530699+0000","last_change":"2024-04-16T05:32:55.507126+0000","last_active":"2024-04-16T05:33:01.530699+0000","last_peered":"2024-04-16T05:33:01.530699+0000","last_clean":"2024-04-16T05:33:01.530699+0000","last_became_active":"2024-04-16T05:32:55.506743+0000","last_became_peered":"2024-04-16T05:32:55.506743+0000","last_unstale":"2024-04-16T05:33:01.530699+0000","last_undegraded":"2024-04-16T05:33:01.530699+0000","last_fullsized":"2024-04-16T05:33:01.530699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:10:58.697649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.895885+0000","last_change":"2024-04-16T05:33:00.419435+0000","last_active":"2024-04-16T05:33:01.895885+0000","last_peered":"2024-04-16T05:33:01.895885+0000","last_clean":"2024-04-16T05:33:01.895885+0000","last_became_active":"2024-04-16T05:33:00.418985+0000","last_became_peered":"2024-04-16T05:33:00.418985+0000","last_unstale":"2024-04-16T05:33:01.895885+0000","last_undegraded":"2024-04-16T05:33:01.895885+0000","last_fullsized":"2024-04-16T05:33:01.895885+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:08:31.188911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:16.610279+0000","last_change":"2024-04-16T05:32:54.503349+0000","last_active":"2024-04-16T05:33:16.610279+0000","last_peered":"2024-04-16T05:33:16.610279+0000","last_clean":"2024-04-16T05:33:16.610279+0000","last_became_active":"2024-04-16T05:32:54.503018+0000","last_became_peered":"2024-04-16T05:32:54.503018+0000","last_unstale":"2024-04-16T05:33:16.610279+0000","last_undegraded":"2024-04-16T05:33:16.610279+0000","last_fullsized":"2024-04-16T05:33:16.610279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:34:20.853543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896068+0000","last_change":"2024-04-16T05:32:55.511201+0000","last_active":"2024-04-16T05:33:01.896068+0000","last_peered":"2024-04-16T05:33:01.896068+0000","last_clean":"2024-04-16T05:33:01.896068+0000","last_became_active":"2024-04-16T05:32:55.510906+0000","last_became_peered":"2024-04-16T05:32:55.510906+0000","last_unstale":"2024-04-16T05:33:01.896068+0000","last_undegraded":"2024-04-16T05:33:01.896068+0000","last_fullsized":"2024-04-16T05:33:01.896068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:37:37.147628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897533+0000","last_change":"2024-04-16T05:32:59.518440+0000","last_active":"2024-04-16T05:33:01.897533+0000","last_peered":"2024-04-16T05:33:01.897533+0000","last_clean":"2024-04-16T05:33:01.897533+0000","last_became_active":"2024-04-16T05:32:59.517870+0000","last_became_peered":"2024-04-16T05:32:59.517870+0000","last_unstale":"2024-04-16T05:33:01.897533+0000","last_undegraded":"2024-04-16T05:33:01.897533+0000","last_fullsized":"2024-04-16T05:33:01.897533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:51:31.157418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.589574+0000","last_change":"2024-04-16T05:32:54.503983+0000","last_active":"2024-04-16T05:33:14.589574+0000","last_peered":"2024-04-16T05:33:14.589574+0000","last_clean":"2024-04-16T05:33:14.589574+0000","last_became_active":"2024-04-16T05:32:54.502367+0000","last_became_peered":"2024-04-16T05:32:54.502367+0000","last_unstale":"2024-04-16T05:33:14.589574+0000","last_undegraded":"2024-04-16T05:33:14.589574+0000","last_fullsized":"2024-04-16T05:33:14.589574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:49:36.997692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896650+0000","last_change":"2024-04-16T05:32:55.516812+0000","last_active":"2024-04-16T05:33:01.896650+0000","last_peered":"2024-04-16T05:33:01.896650+0000","last_clean":"2024-04-16T05:33:01.896650+0000","last_became_active":"2024-04-16T05:32:55.516477+0000","last_became_peered":"2024-04-16T05:32:55.516477+0000","last_unstale":"2024-04-16T05:33:01.896650+0000","last_undegraded":"2024-04-16T05:33:01.896650+0000","last_fullsized":"2024-04-16T05:33:01.896650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:24:29.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897710+0000","last_change":"2024-04-16T05:33:00.419622+0000","last_active":"2024-04-16T05:33:01.897710+0000","last_peered":"2024-04-16T05:33:01.897710+0000","last_clean":"2024-04-16T05:33:01.897710+0000","last_became_active":"2024-04-16T05:33:00.419135+0000","last_became_peered":"2024-04-16T05:33:00.419135+0000","last_unstale":"2024-04-16T05:33:01.897710+0000","last_undegraded":"2024-04-16T05:33:01.897710+0000","last_fullsized":"2024-04-16T05:33:01.897710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:38:02.529192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.590517+0000","last_change":"2024-04-16T05:32:54.503652+0000","last_active":"2024-04-16T05:33:14.590517+0000","last_peered":"2024-04-16T05:33:14.590517+0000","last_clean":"2024-04-16T05:33:14.590517+0000","last_became_active":"2024-04-16T05:32:54.503311+0000","last_became_peered":"2024-04-16T05:32:54.503311+0000","last_unstale":"2024-04-16T05:33:14.590517+0000","last_undegraded":"2024-04-16T05:33:14.590517+0000","last_fullsized":"2024-04-16T05:33:14.590517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:49:21.584029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896506+0000","last_change":"2024-04-16T05:32:55.508145+0000","last_active":"2024-04-16T05:33:01.896506+0000","last_peered":"2024-04-16T05:33:01.896506+0000","last_clean":"2024-04-16T05:33:01.896506+0000","last_became_active":"2024-04-16T05:32:55.507794+0000","last_became_peered":"2024-04-16T05:32:55.507794+0000","last_unstale":"2024-04-16T05:33:01.896506+0000","last_undegraded":"2024-04-16T05:33:01.896506+0000","last_fullsized":"2024-04-16T05:33:01.896506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:11:57.925733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713507+0000","last_change":"2024-04-16T05:32:59.527273+0000","last_active":"2024-04-16T05:33:01.713507+0000","last_peered":"2024-04-16T05:33:01.713507+0000","last_clean":"2024-04-16T05:33:01.713507+0000","last_became_active":"2024-04-16T05:32:59.522925+0000","last_became_peered":"2024-04-16T05:32:59.522925+0000","last_unstale":"2024-04-16T05:33:01.713507+0000","last_undegraded":"2024-04-16T05:33:01.713507+0000","last_fullsized":"2024-04-16T05:33:01.713507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:33:16.297503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591026+0000","last_change":"2024-04-16T05:32:54.500909+0000","last_active":"2024-04-16T05:33:14.591026+0000","last_peered":"2024-04-16T05:33:14.591026+0000","last_clean":"2024-04-16T05:33:14.591026+0000","last_became_active":"2024-04-16T05:32:54.499348+0000","last_became_peered":"2024-04-16T05:32:54.499348+0000","last_unstale":"2024-04-16T05:33:14.591026+0000","last_undegraded":"2024-04-16T05:33:14.591026+0000","last_fullsized":"2024-04-16T05:33:14.591026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:51:49.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896672+0000","last_change":"2024-04-16T05:32:55.510922+0000","last_active":"2024-04-16T05:33:01.896672+0000","last_peered":"2024-04-16T05:33:01.896672+0000","last_clean":"2024-04-16T05:33:01.896672+0000","last_became_active":"2024-04-16T05:32:55.510546+0000","last_became_peered":"2024-04-16T05:32:55.510546+0000","last_unstale":"2024-04-16T05:33:01.896672+0000","last_undegraded":"2024-04-16T05:33:01.896672+0000","last_fullsized":"2024-04-16T05:33:01.896672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:27.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897451+0000","last_change":"2024-04-16T05:32:59.524257+0000","last_active":"2024-04-16T05:33:01.897451+0000","last_peered":"2024-04-16T05:33:01.897451+0000","last_clean":"2024-04-16T05:33:01.897451+0000","last_became_active":"2024-04-16T05:32:59.523768+0000","last_became_peered":"2024-04-16T05:32:59.523768+0000","last_unstale":"2024-04-16T05:33:01.897451+0000","last_undegraded":"2024-04-16T05:33:01.897451+0000","last_fullsized":"2024-04-16T05:33:01.897451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:38:27.594048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896965+0000","last_change":"2024-04-16T05:32:54.504430+0000","last_active":"2024-04-16T05:33:01.896965+0000","last_peered":"2024-04-16T05:33:01.896965+0000","last_clean":"2024-04-16T05:33:01.896965+0000","last_became_active":"2024-04-16T05:32:54.504054+0000","last_became_peered":"2024-04-16T05:32:54.504054+0000","last_unstale":"2024-04-16T05:33:01.896965+0000","last_undegraded":"2024-04-16T05:33:01.896965+0000","last_fullsized":"2024-04-16T05:33:01.896965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:53:24.875786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896270+0000","last_change":"2024-04-16T05:32:55.508005+0000","last_active":"2024-04-16T05:33:01.896270+0000","last_peered":"2024-04-16T05:33:01.896270+0000","last_clean":"2024-04-16T05:33:01.896270+0000","last_became_active":"2024-04-16T05:32:55.507698+0000","last_became_peered":"2024-04-16T05:32:55.507698+0000","last_unstale":"2024-04-16T05:33:01.896270+0000","last_undegraded":"2024-04-16T05:33:01.896270+0000","last_fullsized":"2024-04-16T05:33:01.896270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:11:12.423294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897885+0000","last_change":"2024-04-16T05:33:00.419442+0000","last_active":"2024-04-16T05:33:01.897885+0000","last_peered":"2024-04-16T05:33:01.897885+0000","last_clean":"2024-04-16T05:33:01.897885+0000","last_became_active":"2024-04-16T05:33:00.418906+0000","last_became_peered":"2024-04-16T05:33:00.418906+0000","last_unstale":"2024-04-16T05:33:01.897885+0000","last_undegraded":"2024-04-16T05:33:01.897885+0000","last_fullsized":"2024-04-16T05:33:01.897885+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:21:22.124879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591283+0000","last_change":"2024-04-16T05:32:54.509090+0000","last_active":"2024-04-16T05:33:14.591283+0000","last_peered":"2024-04-16T05:33:14.591283+0000","last_clean":"2024-04-16T05:33:14.591283+0000","last_became_active":"2024-04-16T05:32:54.508700+0000","last_became_peered":"2024-04-16T05:32:54.508700+0000","last_unstale":"2024-04-16T05:33:14.591283+0000","last_undegraded":"2024-04-16T05:33:14.591283+0000","last_fullsized":"2024-04-16T05:33:14.591283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:53:34.344073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534659+0000","last_change":"2024-04-16T05:32:55.509658+0000","last_active":"2024-04-16T05:33:01.534659+0000","last_peered":"2024-04-16T05:33:01.534659+0000","last_clean":"2024-04-16T05:33:01.534659+0000","last_became_active":"2024-04-16T05:32:55.509258+0000","last_became_peered":"2024-04-16T05:32:55.509258+0000","last_unstale":"2024-04-16T05:33:01.534659+0000","last_undegraded":"2024-04-16T05:33:01.534659+0000","last_fullsized":"2024-04-16T05:33:01.534659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:17:03.490544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897075+0000","last_change":"2024-04-16T05:32:59.519260+0000","last_active":"2024-04-16T05:33:01.897075+0000","last_peered":"2024-04-16T05:33:01.897075+0000","last_clean":"2024-04-16T05:33:01.897075+0000","last_became_active":"2024-04-16T05:32:59.518839+0000","last_became_peered":"2024-04-16T05:32:59.518839+0000","last_unstale":"2024-04-16T05:33:01.897075+0000","last_undegraded":"2024-04-16T05:33:01.897075+0000","last_fullsized":"2024-04-16T05:33:01.897075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:54:50.555214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898392+0000","last_change":"2024-04-16T05:32:54.503226+0000","last_active":"2024-04-16T05:33:01.898392+0000","last_peered":"2024-04-16T05:33:01.898392+0000","last_clean":"2024-04-16T05:33:01.898392+0000","last_became_active":"2024-04-16T05:32:54.502687+0000","last_became_peered":"2024-04-16T05:32:54.502687+0000","last_unstale":"2024-04-16T05:33:01.898392+0000","last_undegraded":"2024-04-16T05:33:01.898392+0000","last_fullsized":"2024-04-16T05:33:01.898392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:34:24.228464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534906+0000","last_change":"2024-04-16T05:32:55.515203+0000","last_active":"2024-04-16T05:33:01.534906+0000","last_peered":"2024-04-16T05:33:01.534906+0000","last_clean":"2024-04-16T05:33:01.534906+0000","last_became_active":"2024-04-16T05:32:55.514815+0000","last_became_peered":"2024-04-16T05:32:55.514815+0000","last_unstale":"2024-04-16T05:33:01.534906+0000","last_undegraded":"2024-04-16T05:33:01.534906+0000","last_fullsized":"2024-04-16T05:33:01.534906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:11:18.464532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.895954+0000","last_change":"2024-04-16T05:32:59.520442+0000","last_active":"2024-04-16T05:33:01.895954+0000","last_peered":"2024-04-16T05:33:01.895954+0000","last_clean":"2024-04-16T05:33:01.895954+0000","last_became_active":"2024-04-16T05:32:59.520071+0000","last_became_peered":"2024-04-16T05:32:59.520071+0000","last_unstale":"2024-04-16T05:33:01.895954+0000","last_undegraded":"2024-04-16T05:33:01.895954+0000","last_fullsized":"2024-04-16T05:33:01.895954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:19:16.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531948+0000","last_change":"2024-04-16T05:32:54.510098+0000","last_active":"2024-04-16T05:33:01.531948+0000","last_peered":"2024-04-16T05:33:01.531948+0000","last_clean":"2024-04-16T05:33:01.531948+0000","last_became_active":"2024-04-16T05:32:54.509750+0000","last_became_peered":"2024-04-16T05:32:54.509750+0000","last_unstale":"2024-04-16T05:33:01.531948+0000","last_undegraded":"2024-04-16T05:33:01.531948+0000","last_fullsized":"2024-04-16T05:33:01.531948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:00:09.528958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534448+0000","last_change":"2024-04-16T05:32:55.510872+0000","last_active":"2024-04-16T05:33:01.534448+0000","last_peered":"2024-04-16T05:33:01.534448+0000","last_clean":"2024-04-16T05:33:01.534448+0000","last_became_active":"2024-04-16T05:32:55.510471+0000","last_became_peered":"2024-04-16T05:32:55.510471+0000","last_unstale":"2024-04-16T05:33:01.534448+0000","last_undegraded":"2024-04-16T05:33:01.534448+0000","last_fullsized":"2024-04-16T05:33:01.534448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:47:36.294039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531085+0000","last_change":"2024-04-16T05:32:59.521334+0000","last_active":"2024-04-16T05:33:01.531085+0000","last_peered":"2024-04-16T05:33:01.531085+0000","last_clean":"2024-04-16T05:33:01.531085+0000","last_became_active":"2024-04-16T05:32:59.520348+0000","last_became_peered":"2024-04-16T05:32:59.520348+0000","last_unstale":"2024-04-16T05:33:01.531085+0000","last_undegraded":"2024-04-16T05:33:01.531085+0000","last_fullsized":"2024-04-16T05:33:01.531085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:40.628552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530512+0000","last_change":"2024-04-16T05:32:54.503062+0000","last_active":"2024-04-16T05:33:01.530512+0000","last_peered":"2024-04-16T05:33:01.530512+0000","last_clean":"2024-04-16T05:33:01.530512+0000","last_became_active":"2024-04-16T05:32:54.502762+0000","last_became_peered":"2024-04-16T05:32:54.502762+0000","last_unstale":"2024-04-16T05:33:01.530512+0000","last_undegraded":"2024-04-16T05:33:01.530512+0000","last_fullsized":"2024-04-16T05:33:01.530512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:43:57.911296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534903+0000","last_change":"2024-04-16T05:32:55.510461+0000","last_active":"2024-04-16T05:33:01.534903+0000","last_peered":"2024-04-16T05:33:01.534903+0000","last_clean":"2024-04-16T05:33:01.534903+0000","last_became_active":"2024-04-16T05:32:55.510142+0000","last_became_peered":"2024-04-16T05:32:55.510142+0000","last_unstale":"2024-04-16T05:33:01.534903+0000","last_undegraded":"2024-04-16T05:33:01.534903+0000","last_fullsized":"2024-04-16T05:33:01.534903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:41.346078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896784+0000","last_change":"2024-04-16T05:32:55.512600+0000","last_active":"2024-04-16T05:33:01.896784+0000","last_peered":"2024-04-16T05:33:01.896784+0000","last_clean":"2024-04-16T05:33:01.896784+0000","last_became_active":"2024-04-16T05:32:55.512265+0000","last_became_peered":"2024-04-16T05:32:55.512265+0000","last_unstale":"2024-04-16T05:33:01.896784+0000","last_undegraded":"2024-04-16T05:33:01.896784+0000","last_fullsized":"2024-04-16T05:33:01.896784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:04:46.112644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897803+0000","last_change":"2024-04-16T05:32:55.514481+0000","last_active":"2024-04-16T05:33:01.897803+0000","last_peered":"2024-04-16T05:33:01.897803+0000","last_clean":"2024-04-16T05:33:01.897803+0000","last_became_active":"2024-04-16T05:32:55.512485+0000","last_became_peered":"2024-04-16T05:32:55.512485+0000","last_unstale":"2024-04-16T05:33:01.897803+0000","last_undegraded":"2024-04-16T05:33:01.897803+0000","last_fullsized":"2024-04-16T05:33:01.897803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:33:02.179774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896827+0000","last_change":"2024-04-16T05:32:55.509849+0000","last_active":"2024-04-16T05:33:01.896827+0000","last_peered":"2024-04-16T05:33:01.896827+0000","last_clean":"2024-04-16T05:33:01.896827+0000","last_became_active":"2024-04-16T05:32:55.509338+0000","last_became_peered":"2024-04-16T05:32:55.509338+0000","last_unstale":"2024-04-16T05:33:01.896827+0000","last_undegraded":"2024-04-16T05:33:01.896827+0000","last_fullsized":"2024-04-16T05:33:01.896827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:32:38.900455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897030+0000","last_change":"2024-04-16T05:32:55.518556+0000","last_active":"2024-04-16T05:33:01.897030+0000","last_peered":"2024-04-16T05:33:01.897030+0000","last_clean":"2024-04-16T05:33:01.897030+0000","last_became_active":"2024-04-16T05:32:55.518240+0000","last_became_peered":"2024-04-16T05:32:55.518240+0000","last_unstale":"2024-04-16T05:33:01.897030+0000","last_undegraded":"2024-04-16T05:33:01.897030+0000","last_fullsized":"2024-04-16T05:33:01.897030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:25:46.002516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530711+0000","last_change":"2024-04-16T05:32:55.518106+0000","last_active":"2024-04-16T05:33:01.530711+0000","last_peered":"2024-04-16T05:33:01.530711+0000","last_clean":"2024-04-16T05:33:01.530711+0000","last_became_active":"2024-04-16T05:32:55.517403+0000","last_became_peered":"2024-04-16T05:32:55.517403+0000","last_unstale":"2024-04-16T05:33:01.530711+0000","last_undegraded":"2024-04-16T05:33:01.530711+0000","last_fullsized":"2024-04-16T05:33:01.530711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:36:34.731644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897351+0000","last_change":"2024-04-16T05:32:55.512889+0000","last_active":"2024-04-16T05:33:01.897351+0000","last_peered":"2024-04-16T05:33:01.897351+0000","last_clean":"2024-04-16T05:33:01.897351+0000","last_became_active":"2024-04-16T05:32:55.511670+0000","last_became_peered":"2024-04-16T05:32:55.511670+0000","last_unstale":"2024-04-16T05:33:01.897351+0000","last_undegraded":"2024-04-16T05:33:01.897351+0000","last_fullsized":"2024-04-16T05:33:01.897351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:30:39.320608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897707+0000","last_change":"2024-04-16T05:32:55.507670+0000","last_active":"2024-04-16T05:33:01.897707+0000","last_peered":"2024-04-16T05:33:01.897707+0000","last_clean":"2024-04-16T05:33:01.897707+0000","last_became_active":"2024-04-16T05:32:55.506672+0000","last_became_peered":"2024-04-16T05:32:55.506672+0000","last_unstale":"2024-04-16T05:33:01.897707+0000","last_undegraded":"2024-04-16T05:33:01.897707+0000","last_fullsized":"2024-04-16T05:33:01.897707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:54.579672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896257+0000","last_change":"2024-04-16T05:32:55.516987+0000","last_active":"2024-04-16T05:33:01.896257+0000","last_peered":"2024-04-16T05:33:01.896257+0000","last_clean":"2024-04-16T05:33:01.896257+0000","last_became_active":"2024-04-16T05:32:55.516654+0000","last_became_peered":"2024-04-16T05:32:55.516654+0000","last_unstale":"2024-04-16T05:33:01.896257+0000","last_undegraded":"2024-04-16T05:33:01.896257+0000","last_fullsized":"2024-04-16T05:33:01.896257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:32:42.957658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531293+0000","last_change":"2024-04-16T05:32:55.506539+0000","last_active":"2024-04-16T05:33:01.531293+0000","last_peered":"2024-04-16T05:33:01.531293+0000","last_clean":"2024-04-16T05:33:01.531293+0000","last_became_active":"2024-04-16T05:32:55.503658+0000","last_became_peered":"2024-04-16T05:32:55.503658+0000","last_unstale":"2024-04-16T05:33:01.531293+0000","last_undegraded":"2024-04-16T05:33:01.531293+0000","last_fullsized":"2024-04-16T05:33:01.531293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:22.116213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896604+0000","last_change":"2024-04-16T05:32:55.507363+0000","last_active":"2024-04-16T05:33:01.896604+0000","last_peered":"2024-04-16T05:33:01.896604+0000","last_clean":"2024-04-16T05:33:01.896604+0000","last_became_active":"2024-04-16T05:32:55.506923+0000","last_became_peered":"2024-04-16T05:32:55.506923+0000","last_unstale":"2024-04-16T05:33:01.896604+0000","last_undegraded":"2024-04-16T05:33:01.896604+0000","last_fullsized":"2024-04-16T05:33:01.896604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:19:34.356834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897402+0000","last_change":"2024-04-16T05:32:55.508531+0000","last_active":"2024-04-16T05:33:01.897402+0000","last_peered":"2024-04-16T05:33:01.897402+0000","last_clean":"2024-04-16T05:33:01.897402+0000","last_became_active":"2024-04-16T05:32:55.508177+0000","last_became_peered":"2024-04-16T05:32:55.508177+0000","last_unstale":"2024-04-16T05:33:01.897402+0000","last_undegraded":"2024-04-16T05:33:01.897402+0000","last_fullsized":"2024-04-16T05:33:01.897402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:49:59.352572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530373+0000","last_change":"2024-04-16T05:32:55.507793+0000","last_active":"2024-04-16T05:33:01.530373+0000","last_peered":"2024-04-16T05:33:01.530373+0000","last_clean":"2024-04-16T05:33:01.530373+0000","last_became_active":"2024-04-16T05:32:55.507443+0000","last_became_peered":"2024-04-16T05:32:55.507443+0000","last_unstale":"2024-04-16T05:33:01.530373+0000","last_undegraded":"2024-04-16T05:33:01.530373+0000","last_fullsized":"2024-04-16T05:33:01.530373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:35.982879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897524+0000","last_change":"2024-04-16T05:32:55.517057+0000","last_active":"2024-04-16T05:33:01.897524+0000","last_peered":"2024-04-16T05:33:01.897524+0000","last_clean":"2024-04-16T05:33:01.897524+0000","last_became_active":"2024-04-16T05:32:55.516696+0000","last_became_peered":"2024-04-16T05:32:55.516696+0000","last_unstale":"2024-04-16T05:33:01.897524+0000","last_undegraded":"2024-04-16T05:33:01.897524+0000","last_fullsized":"2024-04-16T05:33:01.897524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:43:10.391125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280341,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6960,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":137,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":21763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":486622,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1081026,"data_compressed":53275,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":491840,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":487360,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1078782,"data_compressed":53275,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":486712,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":490464,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":489191,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":491120,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":486816,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":486644,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":1077172,"data_compressed":53275,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:33:20.034 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.174+0000 7f9e98688640 1 -- 172.21.15.86:0/1359781055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 msgr2=0x7f9e901036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.174+0000 7f9e98688640 1 --2- 172.21.15.86:0/1359781055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 0x7f9e901036c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009900 tx=0x7f9e8002bc30 comp rx=0 tx=0).stop 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.174+0000 7f9e98688640 1 -- 172.21.15.86:0/1359781055 shutdown_connections 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.174+0000 7f9e98688640 1 --2- 172.21.15.86:0/1359781055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e90103c00 0x7f9e9010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.174+0000 7f9e98688640 1 --2- 172.21.15.86:0/1359781055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 0x7f9e901036c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.174+0000 7f9e98688640 1 --2- 172.21.15.86:0/1359781055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102050 0x7f9e90102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.174+0000 7f9e98688640 1 -- 172.21.15.86:0/1359781055 >> 172.21.15.86:0/1359781055 conn(0x7f9e900fd5e0 msgr2=0x7f9e900ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.174+0000 7f9e98688640 1 -- 172.21.15.86:0/1359781055 shutdown_connections 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.174+0000 7f9e98688640 1 -- 172.21.15.86:0/1359781055 wait complete. 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e98688640 1 Processor -- start 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e98688640 1 -- start start 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e98688640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102050 0x7f9e90198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e98688640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 0x7f9e90198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e98688640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e90103c00 0x7f9e9019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e98688640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010dce0 con 0x7f9e90103240 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e98688640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010db60 con 0x7f9e90103c00 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e98688640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010de60 con 0x7f9e90102050 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e95c23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 0x7f9e90198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e95c23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 0x7f9e90198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46414/0 (socket says 172.21.15.86:46414) 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e95c23640 1 -- 172.21.15.86:0/3959739445 learned_addr learned my addr 172.21.15.86:0/3959739445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e96c25640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e90103c00 0x7f9e9019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e96424640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102050 0x7f9e90198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e95c23640 1 -- 172.21.15.86:0/3959739445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102050 msgr2=0x7f9e90198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e95c23640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102050 0x7f9e90198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e95c23640 1 -- 172.21.15.86:0/3959739445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e90103c00 msgr2=0x7f9e9019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e95c23640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e90103c00 0x7f9e9019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.175+0000 7f9e95c23640 1 -- 172.21.15.86:0/3959739445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9019ff40 con 0x7f9e90103240 2024-04-16T05:33:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.176+0000 7f9e95c23640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 0x7f9e90198860 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009230 tx=0x7f9e80005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.176+0000 7f9e877fe640 1 -- 172.21.15.86:0/3959739445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80041070 con 0x7f9e90103240 2024-04-16T05:33:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.176+0000 7f9e877fe640 1 -- 172.21.15.86:0/3959739445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e800054e0 con 0x7f9e90103240 2024-04-16T05:33:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.176+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e901a01d0 con 0x7f9e90103240 2024-04-16T05:33:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.176+0000 7f9e877fe640 1 -- 172.21.15.86:0/3959739445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80034440 con 0x7f9e90103240 2024-04-16T05:33:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.176+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e901a06c0 con 0x7f9e90103240 2024-04-16T05:33:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.177+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e90109370 con 0x7f9e90103240 2024-04-16T05:33:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.178+0000 7f9e877fe640 1 -- 172.21.15.86:0/3959739445 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9e80044070 con 0x7f9e90103240 2024-04-16T05:33:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.178+0000 7f9e877fe640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9e7c0a3cc0 0x7f9e7c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.179+0000 7f9e96424640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9e7c0a3cc0 0x7f9e7c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.179+0000 7f9e877fe640 1 -- 172.21.15.86:0/3959739445 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9e800f2390 con 0x7f9e90103240 2024-04-16T05:33:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.179+0000 7f9e96424640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9e7c0a3cc0 0x7f9e7c0a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e901030a0 tx=0x7f9e8c00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:20.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.183+0000 7f9e877fe640 1 -- 172.21.15.86:0/3959739445 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e800be430 con 0x7f9e90103240 2024-04-16T05:33:20.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.357+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e90062440 con 0x7f9e7c0a3cc0 2024-04-16T05:33:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.366+0000 7f9e877fe640 1 -- 172.21.15.86:0/3959739445 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340606 (secure 0 0 0) 0x7f9e90062440 con 0x7f9e7c0a3cc0 2024-04-16T05:33:20.368 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9e7c0a3cc0 msgr2=0x7f9e7c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9e7c0a3cc0 0x7f9e7c0a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e901030a0 tx=0x7f9e8c00a2d0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 msgr2=0x7f9e90198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 0x7f9e90198860 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009230 tx=0x7f9e80005dd0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 shutdown_connections 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9e7c0a3cc0 0x7f9e7c0a6180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e90103c00 0x7f9e9019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e90103240 0x7f9e90198860 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 --2- 172.21.15.86:0/3959739445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102050 0x7f9e90198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.369+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 >> 172.21.15.86:0/3959739445 conn(0x7f9e900fd5e0 msgr2=0x7f9e90106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.370+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 shutdown_connections 2024-04-16T05:33:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:20.370+0000 7f9e98688640 1 -- 172.21.15.86:0/3959739445 wait complete. 2024-04-16T05:33:20.379 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-04-16T05:33:19.482133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597328,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":318,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365104,"kb_used_data":11776,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576720,"statfs":{"total":1151940427776,"available":1151566561280,"internally_reserved":0,"allocated":12058624,"data_stored":7633749,"data_compressed":318378,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4550,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":92,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003147"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898114+0000","last_change":"2024-04-16T05:32:55.507548+0000","last_active":"2024-04-16T05:33:01.898114+0000","last_peered":"2024-04-16T05:33:01.898114+0000","last_clean":"2024-04-16T05:33:01.898114+0000","last_became_active":"2024-04-16T05:32:55.507200+0000","last_became_peered":"2024-04-16T05:32:55.507200+0000","last_unstale":"2024-04-16T05:33:01.898114+0000","last_undegraded":"2024-04-16T05:33:01.898114+0000","last_fullsized":"2024-04-16T05:33:01.898114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:58:07.842226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896558+0000","last_change":"2024-04-16T05:32:55.509792+0000","last_active":"2024-04-16T05:33:01.896558+0000","last_peered":"2024-04-16T05:33:01.896558+0000","last_clean":"2024-04-16T05:33:01.896558+0000","last_became_active":"2024-04-16T05:32:55.509442+0000","last_became_peered":"2024-04-16T05:32:55.509442+0000","last_unstale":"2024-04-16T05:33:01.896558+0000","last_undegraded":"2024-04-16T05:33:01.896558+0000","last_fullsized":"2024-04-16T05:33:01.896558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:08:42.040875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896165+0000","last_change":"2024-04-16T05:32:55.505664+0000","last_active":"2024-04-16T05:33:01.896165+0000","last_peered":"2024-04-16T05:33:01.896165+0000","last_clean":"2024-04-16T05:33:01.896165+0000","last_became_active":"2024-04-16T05:32:55.505281+0000","last_became_peered":"2024-04-16T05:32:55.505281+0000","last_unstale":"2024-04-16T05:33:01.896165+0000","last_undegraded":"2024-04-16T05:33:01.896165+0000","last_fullsized":"2024-04-16T05:33:01.896165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:33.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897618+0000","last_change":"2024-04-16T05:32:55.517072+0000","last_active":"2024-04-16T05:33:01.897618+0000","last_peered":"2024-04-16T05:33:01.897618+0000","last_clean":"2024-04-16T05:33:01.897618+0000","last_became_active":"2024-04-16T05:32:55.516648+0000","last_became_peered":"2024-04-16T05:32:55.516648+0000","last_unstale":"2024-04-16T05:33:01.897618+0000","last_undegraded":"2024-04-16T05:33:01.897618+0000","last_fullsized":"2024-04-16T05:33:01.897618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:41.735720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896639+0000","last_change":"2024-04-16T05:32:55.506773+0000","last_active":"2024-04-16T05:33:01.896639+0000","last_peered":"2024-04-16T05:33:01.896639+0000","last_clean":"2024-04-16T05:33:01.896639+0000","last_became_active":"2024-04-16T05:32:55.506414+0000","last_became_peered":"2024-04-16T05:32:55.506414+0000","last_unstale":"2024-04-16T05:33:01.896639+0000","last_undegraded":"2024-04-16T05:33:01.896639+0000","last_fullsized":"2024-04-16T05:33:01.896639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:59.254334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708559+0000","last_change":"2024-04-16T05:32:55.517875+0000","last_active":"2024-04-16T05:33:01.708559+0000","last_peered":"2024-04-16T05:33:01.708559+0000","last_clean":"2024-04-16T05:33:01.708559+0000","last_became_active":"2024-04-16T05:32:55.517546+0000","last_became_peered":"2024-04-16T05:32:55.517546+0000","last_unstale":"2024-04-16T05:33:01.708559+0000","last_undegraded":"2024-04-16T05:33:01.708559+0000","last_fullsized":"2024-04-16T05:33:01.708559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:12:15.098439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534118+0000","last_change":"2024-04-16T05:32:55.509036+0000","last_active":"2024-04-16T05:33:01.534118+0000","last_peered":"2024-04-16T05:33:01.534118+0000","last_clean":"2024-04-16T05:33:01.534118+0000","last_became_active":"2024-04-16T05:32:55.508690+0000","last_became_peered":"2024-04-16T05:32:55.508690+0000","last_unstale":"2024-04-16T05:33:01.534118+0000","last_undegraded":"2024-04-16T05:33:01.534118+0000","last_fullsized":"2024-04-16T05:33:01.534118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:26.724281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896120+0000","last_change":"2024-04-16T05:32:55.510146+0000","last_active":"2024-04-16T05:33:01.896120+0000","last_peered":"2024-04-16T05:33:01.896120+0000","last_clean":"2024-04-16T05:33:01.896120+0000","last_became_active":"2024-04-16T05:32:55.508665+0000","last_became_peered":"2024-04-16T05:32:55.508665+0000","last_unstale":"2024-04-16T05:33:01.896120+0000","last_undegraded":"2024-04-16T05:33:01.896120+0000","last_fullsized":"2024-04-16T05:33:01.896120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:14.158221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898385+0000","last_change":"2024-04-16T05:32:55.513738+0000","last_active":"2024-04-16T05:33:01.898385+0000","last_peered":"2024-04-16T05:33:01.898385+0000","last_clean":"2024-04-16T05:33:01.898385+0000","last_became_active":"2024-04-16T05:32:55.513374+0000","last_became_peered":"2024-04-16T05:32:55.513374+0000","last_unstale":"2024-04-16T05:33:01.898385+0000","last_undegraded":"2024-04-16T05:33:01.898385+0000","last_fullsized":"2024-04-16T05:33:01.898385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:06:23.390908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708971+0000","last_change":"2024-04-16T05:32:55.519276+0000","last_active":"2024-04-16T05:33:01.708971+0000","last_peered":"2024-04-16T05:33:01.708971+0000","last_clean":"2024-04-16T05:33:01.708971+0000","last_became_active":"2024-04-16T05:32:55.518729+0000","last_became_peered":"2024-04-16T05:32:55.518729+0000","last_unstale":"2024-04-16T05:33:01.708971+0000","last_undegraded":"2024-04-16T05:33:01.708971+0000","last_fullsized":"2024-04-16T05:33:01.708971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:44:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534782+0000","last_change":"2024-04-16T05:32:55.516379+0000","last_active":"2024-04-16T05:33:01.534782+0000","last_peered":"2024-04-16T05:33:01.534782+0000","last_clean":"2024-04-16T05:33:01.534782+0000","last_became_active":"2024-04-16T05:32:55.515574+0000","last_became_peered":"2024-04-16T05:32:55.515574+0000","last_unstale":"2024-04-16T05:33:01.534782+0000","last_undegraded":"2024-04-16T05:33:01.534782+0000","last_fullsized":"2024-04-16T05:33:01.534782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:16:01.744380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897635+0000","last_change":"2024-04-16T05:32:55.518988+0000","last_active":"2024-04-16T05:33:01.897635+0000","last_peered":"2024-04-16T05:33:01.897635+0000","last_clean":"2024-04-16T05:33:01.897635+0000","last_became_active":"2024-04-16T05:32:55.518214+0000","last_became_peered":"2024-04-16T05:32:55.518214+0000","last_unstale":"2024-04-16T05:33:01.897635+0000","last_undegraded":"2024-04-16T05:33:01.897635+0000","last_fullsized":"2024-04-16T05:33:01.897635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:12:57.645418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530497+0000","last_change":"2024-04-16T05:32:55.509682+0000","last_active":"2024-04-16T05:33:01.530497+0000","last_peered":"2024-04-16T05:33:01.530497+0000","last_clean":"2024-04-16T05:33:01.530497+0000","last_became_active":"2024-04-16T05:32:55.509383+0000","last_became_peered":"2024-04-16T05:32:55.509383+0000","last_unstale":"2024-04-16T05:33:01.530497+0000","last_undegraded":"2024-04-16T05:33:01.530497+0000","last_fullsized":"2024-04-16T05:33:01.530497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:25:23.487003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.895881+0000","last_change":"2024-04-16T05:32:55.506338+0000","last_active":"2024-04-16T05:33:01.895881+0000","last_peered":"2024-04-16T05:33:01.895881+0000","last_clean":"2024-04-16T05:33:01.895881+0000","last_became_active":"2024-04-16T05:32:55.506000+0000","last_became_peered":"2024-04-16T05:32:55.506000+0000","last_unstale":"2024-04-16T05:33:01.895881+0000","last_undegraded":"2024-04-16T05:33:01.895881+0000","last_fullsized":"2024-04-16T05:33:01.895881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:37:02.296515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896222+0000","last_change":"2024-04-16T05:32:55.516897+0000","last_active":"2024-04-16T05:33:01.896222+0000","last_peered":"2024-04-16T05:33:01.896222+0000","last_clean":"2024-04-16T05:33:01.896222+0000","last_became_active":"2024-04-16T05:32:55.516489+0000","last_became_peered":"2024-04-16T05:32:55.516489+0000","last_unstale":"2024-04-16T05:33:01.896222+0000","last_undegraded":"2024-04-16T05:33:01.896222+0000","last_fullsized":"2024-04-16T05:33:01.896222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:26:08.289501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896820+0000","last_change":"2024-04-16T05:32:55.507374+0000","last_active":"2024-04-16T05:33:01.896820+0000","last_peered":"2024-04-16T05:33:01.896820+0000","last_clean":"2024-04-16T05:33:01.896820+0000","last_became_active":"2024-04-16T05:32:55.507045+0000","last_became_peered":"2024-04-16T05:32:55.507045+0000","last_unstale":"2024-04-16T05:33:01.896820+0000","last_undegraded":"2024-04-16T05:33:01.896820+0000","last_fullsized":"2024-04-16T05:33:01.896820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:35.968704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897727+0000","last_change":"2024-04-16T05:32:55.508845+0000","last_active":"2024-04-16T05:33:01.897727+0000","last_peered":"2024-04-16T05:33:01.897727+0000","last_clean":"2024-04-16T05:33:01.897727+0000","last_became_active":"2024-04-16T05:32:55.508527+0000","last_became_peered":"2024-04-16T05:32:55.508527+0000","last_unstale":"2024-04-16T05:33:01.897727+0000","last_undegraded":"2024-04-16T05:33:01.897727+0000","last_fullsized":"2024-04-16T05:33:01.897727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:20:15.033043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713430+0000","last_change":"2024-04-16T05:32:55.506059+0000","last_active":"2024-04-16T05:33:01.713430+0000","last_peered":"2024-04-16T05:33:01.713430+0000","last_clean":"2024-04-16T05:33:01.713430+0000","last_became_active":"2024-04-16T05:32:55.505728+0000","last_became_peered":"2024-04-16T05:32:55.505728+0000","last_unstale":"2024-04-16T05:33:01.713430+0000","last_undegraded":"2024-04-16T05:33:01.713430+0000","last_fullsized":"2024-04-16T05:33:01.713430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:19.253543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898293+0000","last_change":"2024-04-16T05:32:55.506791+0000","last_active":"2024-04-16T05:33:01.898293+0000","last_peered":"2024-04-16T05:33:01.898293+0000","last_clean":"2024-04-16T05:33:01.898293+0000","last_became_active":"2024-04-16T05:32:55.506379+0000","last_became_peered":"2024-04-16T05:32:55.506379+0000","last_unstale":"2024-04-16T05:33:01.898293+0000","last_undegraded":"2024-04-16T05:33:01.898293+0000","last_fullsized":"2024-04-16T05:33:01.898293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:09:11.457144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897953+0000","last_change":"2024-04-16T05:32:59.524001+0000","last_active":"2024-04-16T05:33:01.897953+0000","last_peered":"2024-04-16T05:33:01.897953+0000","last_clean":"2024-04-16T05:33:01.897953+0000","last_became_active":"2024-04-16T05:32:59.522443+0000","last_became_peered":"2024-04-16T05:32:59.522443+0000","last_unstale":"2024-04-16T05:33:01.897953+0000","last_undegraded":"2024-04-16T05:33:01.897953+0000","last_fullsized":"2024-04-16T05:33:01.897953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:16:14.777815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:18.621756+0000","last_change":"2024-04-16T05:32:54.504826+0000","last_active":"2024-04-16T05:33:18.621756+0000","last_peered":"2024-04-16T05:33:18.621756+0000","last_clean":"2024-04-16T05:33:18.621756+0000","last_became_active":"2024-04-16T05:32:54.504526+0000","last_became_peered":"2024-04-16T05:32:54.504526+0000","last_unstale":"2024-04-16T05:33:18.621756+0000","last_undegraded":"2024-04-16T05:33:18.621756+0000","last_fullsized":"2024-04-16T05:33:18.621756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:21:28.434692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896636+0000","last_change":"2024-04-16T05:32:55.514760+0000","last_active":"2024-04-16T05:33:01.896636+0000","last_peered":"2024-04-16T05:33:01.896636+0000","last_clean":"2024-04-16T05:33:01.896636+0000","last_became_active":"2024-04-16T05:32:55.514421+0000","last_became_peered":"2024-04-16T05:32:55.514421+0000","last_unstale":"2024-04-16T05:33:01.896636+0000","last_undegraded":"2024-04-16T05:33:01.896636+0000","last_fullsized":"2024-04-16T05:33:01.896636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:24:52.894382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896956+0000","last_change":"2024-04-16T05:32:59.524687+0000","last_active":"2024-04-16T05:33:01.896956+0000","last_peered":"2024-04-16T05:33:01.896956+0000","last_clean":"2024-04-16T05:33:01.896956+0000","last_became_active":"2024-04-16T05:32:59.524228+0000","last_became_peered":"2024-04-16T05:32:59.524228+0000","last_unstale":"2024-04-16T05:33:01.896956+0000","last_undegraded":"2024-04-16T05:33:01.896956+0000","last_fullsized":"2024-04-16T05:33:01.896956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:23:15.244359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896680+0000","last_change":"2024-04-16T05:32:54.504062+0000","last_active":"2024-04-16T05:33:01.896680+0000","last_peered":"2024-04-16T05:33:01.896680+0000","last_clean":"2024-04-16T05:33:01.896680+0000","last_became_active":"2024-04-16T05:32:54.503745+0000","last_became_peered":"2024-04-16T05:32:54.503745+0000","last_unstale":"2024-04-16T05:33:01.896680+0000","last_undegraded":"2024-04-16T05:33:01.896680+0000","last_fullsized":"2024-04-16T05:33:01.896680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:29:34.157937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896624+0000","last_change":"2024-04-16T05:32:55.509820+0000","last_active":"2024-04-16T05:33:01.896624+0000","last_peered":"2024-04-16T05:33:01.896624+0000","last_clean":"2024-04-16T05:33:01.896624+0000","last_became_active":"2024-04-16T05:32:55.509473+0000","last_became_peered":"2024-04-16T05:32:55.509473+0000","last_unstale":"2024-04-16T05:33:01.896624+0000","last_undegraded":"2024-04-16T05:33:01.896624+0000","last_fullsized":"2024-04-16T05:33:01.896624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:12:22.433189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897632+0000","last_change":"2024-04-16T05:32:59.524462+0000","last_active":"2024-04-16T05:33:01.897632+0000","last_peered":"2024-04-16T05:33:01.897632+0000","last_clean":"2024-04-16T05:33:01.897632+0000","last_became_active":"2024-04-16T05:32:59.523989+0000","last_became_peered":"2024-04-16T05:32:59.523989+0000","last_unstale":"2024-04-16T05:33:01.897632+0000","last_undegraded":"2024-04-16T05:33:01.897632+0000","last_fullsized":"2024-04-16T05:33:01.897632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:13:11.687257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:16.608633+0000","last_change":"2024-04-16T05:32:54.511469+0000","last_active":"2024-04-16T05:33:16.608633+0000","last_peered":"2024-04-16T05:33:16.608633+0000","last_clean":"2024-04-16T05:33:16.608633+0000","last_became_active":"2024-04-16T05:32:54.511120+0000","last_became_peered":"2024-04-16T05:32:54.511120+0000","last_unstale":"2024-04-16T05:33:16.608633+0000","last_undegraded":"2024-04-16T05:33:16.608633+0000","last_fullsized":"2024-04-16T05:33:16.608633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897788+0000","last_change":"2024-04-16T05:32:55.512308+0000","last_active":"2024-04-16T05:33:01.897788+0000","last_peered":"2024-04-16T05:33:01.897788+0000","last_clean":"2024-04-16T05:33:01.897788+0000","last_became_active":"2024-04-16T05:32:55.511625+0000","last_became_peered":"2024-04-16T05:32:55.511625+0000","last_unstale":"2024-04-16T05:33:01.897788+0000","last_undegraded":"2024-04-16T05:33:01.897788+0000","last_fullsized":"2024-04-16T05:33:01.897788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:29:49.323092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897419+0000","last_change":"2024-04-16T05:32:59.519306+0000","last_active":"2024-04-16T05:33:01.897419+0000","last_peered":"2024-04-16T05:33:01.897419+0000","last_clean":"2024-04-16T05:33:01.897419+0000","last_became_active":"2024-04-16T05:32:59.518797+0000","last_became_peered":"2024-04-16T05:32:59.518797+0000","last_unstale":"2024-04-16T05:33:01.897419+0000","last_undegraded":"2024-04-16T05:33:01.897419+0000","last_fullsized":"2024-04-16T05:33:01.897419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:04:12.389004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896718+0000","last_change":"2024-04-16T05:32:54.505102+0000","last_active":"2024-04-16T05:33:01.896718+0000","last_peered":"2024-04-16T05:33:01.896718+0000","last_clean":"2024-04-16T05:33:01.896718+0000","last_became_active":"2024-04-16T05:32:54.504806+0000","last_became_peered":"2024-04-16T05:32:54.504806+0000","last_unstale":"2024-04-16T05:33:01.896718+0000","last_undegraded":"2024-04-16T05:33:01.896718+0000","last_fullsized":"2024-04-16T05:33:01.896718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:19.467690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531783+0000","last_change":"2024-04-16T05:32:55.517097+0000","last_active":"2024-04-16T05:33:01.531783+0000","last_peered":"2024-04-16T05:33:01.531783+0000","last_clean":"2024-04-16T05:33:01.531783+0000","last_became_active":"2024-04-16T05:32:55.516717+0000","last_became_peered":"2024-04-16T05:32:55.516717+0000","last_unstale":"2024-04-16T05:33:01.531783+0000","last_undegraded":"2024-04-16T05:33:01.531783+0000","last_fullsized":"2024-04-16T05:33:01.531783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:10:44.544935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898275+0000","last_change":"2024-04-16T05:32:59.521429+0000","last_active":"2024-04-16T05:33:01.898275+0000","last_peered":"2024-04-16T05:33:01.898275+0000","last_clean":"2024-04-16T05:33:01.898275+0000","last_became_active":"2024-04-16T05:32:59.520789+0000","last_became_peered":"2024-04-16T05:32:59.520789+0000","last_unstale":"2024-04-16T05:33:01.898275+0000","last_undegraded":"2024-04-16T05:33:01.898275+0000","last_fullsized":"2024-04-16T05:33:01.898275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:40:14.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.592641+0000","last_change":"2024-04-16T05:32:54.507374+0000","last_active":"2024-04-16T05:33:14.592641+0000","last_peered":"2024-04-16T05:33:14.592641+0000","last_clean":"2024-04-16T05:33:14.592641+0000","last_became_active":"2024-04-16T05:32:54.507055+0000","last_became_peered":"2024-04-16T05:32:54.507055+0000","last_unstale":"2024-04-16T05:33:14.592641+0000","last_undegraded":"2024-04-16T05:33:14.592641+0000","last_fullsized":"2024-04-16T05:33:14.592641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:00:57.761971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896786+0000","last_change":"2024-04-16T05:32:55.513980+0000","last_active":"2024-04-16T05:33:01.896786+0000","last_peered":"2024-04-16T05:33:01.896786+0000","last_clean":"2024-04-16T05:33:01.896786+0000","last_became_active":"2024-04-16T05:32:55.513622+0000","last_became_peered":"2024-04-16T05:32:55.513622+0000","last_unstale":"2024-04-16T05:33:01.896786+0000","last_undegraded":"2024-04-16T05:33:01.896786+0000","last_fullsized":"2024-04-16T05:33:01.896786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:16:10.018381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898152+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:33:01.898152+0000","last_peered":"2024-04-16T05:33:01.898152+0000","last_clean":"2024-04-16T05:33:01.898152+0000","last_became_active":"2024-04-16T05:32:59.523032+0000","last_became_peered":"2024-04-16T05:32:59.523032+0000","last_unstale":"2024-04-16T05:33:01.898152+0000","last_undegraded":"2024-04-16T05:33:01.898152+0000","last_fullsized":"2024-04-16T05:33:01.898152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:33:17.897241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708502+0000","last_change":"2024-04-16T05:32:54.504527+0000","last_active":"2024-04-16T05:33:01.708502+0000","last_peered":"2024-04-16T05:33:01.708502+0000","last_clean":"2024-04-16T05:33:01.708502+0000","last_became_active":"2024-04-16T05:32:54.504187+0000","last_became_peered":"2024-04-16T05:32:54.504187+0000","last_unstale":"2024-04-16T05:33:01.708502+0000","last_undegraded":"2024-04-16T05:33:01.708502+0000","last_fullsized":"2024-04-16T05:33:01.708502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.414362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531776+0000","last_change":"2024-04-16T05:32:55.520270+0000","last_active":"2024-04-16T05:33:01.531776+0000","last_peered":"2024-04-16T05:33:01.531776+0000","last_clean":"2024-04-16T05:33:01.531776+0000","last_became_active":"2024-04-16T05:32:55.519895+0000","last_became_peered":"2024-04-16T05:32:55.519895+0000","last_unstale":"2024-04-16T05:33:01.531776+0000","last_undegraded":"2024-04-16T05:33:01.531776+0000","last_fullsized":"2024-04-16T05:33:01.531776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:09.967836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897725+0000","last_change":"2024-04-16T05:32:59.518997+0000","last_active":"2024-04-16T05:33:01.897725+0000","last_peered":"2024-04-16T05:33:01.897725+0000","last_clean":"2024-04-16T05:33:01.897725+0000","last_became_active":"2024-04-16T05:32:59.518558+0000","last_became_peered":"2024-04-16T05:32:59.518558+0000","last_unstale":"2024-04-16T05:33:01.897725+0000","last_undegraded":"2024-04-16T05:33:01.897725+0000","last_fullsized":"2024-04-16T05:33:01.897725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:04:30.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.602246+0000","last_change":"2024-04-16T05:32:54.502074+0000","last_active":"2024-04-16T05:33:15.602246+0000","last_peered":"2024-04-16T05:33:15.602246+0000","last_clean":"2024-04-16T05:33:15.602246+0000","last_became_active":"2024-04-16T05:32:54.501730+0000","last_became_peered":"2024-04-16T05:32:54.501730+0000","last_unstale":"2024-04-16T05:33:15.602246+0000","last_undegraded":"2024-04-16T05:33:15.602246+0000","last_fullsized":"2024-04-16T05:33:15.602246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:05:02.929799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896032+0000","last_change":"2024-04-16T05:32:55.504557+0000","last_active":"2024-04-16T05:33:01.896032+0000","last_peered":"2024-04-16T05:33:01.896032+0000","last_clean":"2024-04-16T05:33:01.896032+0000","last_became_active":"2024-04-16T05:32:55.504244+0000","last_became_peered":"2024-04-16T05:32:55.504244+0000","last_unstale":"2024-04-16T05:33:01.896032+0000","last_undegraded":"2024-04-16T05:33:01.896032+0000","last_fullsized":"2024-04-16T05:33:01.896032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:33:40.500436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896505+0000","last_change":"2024-04-16T05:32:59.524681+0000","last_active":"2024-04-16T05:33:01.896505+0000","last_peered":"2024-04-16T05:33:01.896505+0000","last_clean":"2024-04-16T05:33:01.896505+0000","last_became_active":"2024-04-16T05:32:59.524217+0000","last_became_peered":"2024-04-16T05:32:59.524217+0000","last_unstale":"2024-04-16T05:33:01.896505+0000","last_undegraded":"2024-04-16T05:33:01.896505+0000","last_fullsized":"2024-04-16T05:33:01.896505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:28.314442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531361+0000","last_change":"2024-04-16T05:32:54.508242+0000","last_active":"2024-04-16T05:33:01.531361+0000","last_peered":"2024-04-16T05:33:01.531361+0000","last_clean":"2024-04-16T05:33:01.531361+0000","last_became_active":"2024-04-16T05:32:54.507782+0000","last_became_peered":"2024-04-16T05:32:54.507782+0000","last_unstale":"2024-04-16T05:33:01.531361+0000","last_undegraded":"2024-04-16T05:33:01.531361+0000","last_fullsized":"2024-04-16T05:33:01.531361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:27:26.805716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713049+0000","last_change":"2024-04-16T05:32:55.508985+0000","last_active":"2024-04-16T05:33:01.713049+0000","last_peered":"2024-04-16T05:33:01.713049+0000","last_clean":"2024-04-16T05:33:01.713049+0000","last_became_active":"2024-04-16T05:32:55.508659+0000","last_became_peered":"2024-04-16T05:32:55.508659+0000","last_unstale":"2024-04-16T05:33:01.713049+0000","last_undegraded":"2024-04-16T05:33:01.713049+0000","last_fullsized":"2024-04-16T05:33:01.713049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:43:41.534155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896485+0000","last_change":"2024-04-16T05:32:59.521257+0000","last_active":"2024-04-16T05:33:01.896485+0000","last_peered":"2024-04-16T05:33:01.896485+0000","last_clean":"2024-04-16T05:33:01.896485+0000","last_became_active":"2024-04-16T05:32:59.520848+0000","last_became_peered":"2024-04-16T05:32:59.520848+0000","last_unstale":"2024-04-16T05:33:01.896485+0000","last_undegraded":"2024-04-16T05:33:01.896485+0000","last_fullsized":"2024-04-16T05:33:01.896485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:14.219378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.590987+0000","last_change":"2024-04-16T05:32:54.500684+0000","last_active":"2024-04-16T05:33:14.590987+0000","last_peered":"2024-04-16T05:33:14.590987+0000","last_clean":"2024-04-16T05:33:14.590987+0000","last_became_active":"2024-04-16T05:32:54.500290+0000","last_became_peered":"2024-04-16T05:32:54.500290+0000","last_unstale":"2024-04-16T05:33:14.590987+0000","last_undegraded":"2024-04-16T05:33:14.590987+0000","last_fullsized":"2024-04-16T05:33:14.590987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:20:55.263313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898402+0000","last_change":"2024-04-16T05:32:55.511224+0000","last_active":"2024-04-16T05:33:01.898402+0000","last_peered":"2024-04-16T05:33:01.898402+0000","last_clean":"2024-04-16T05:33:01.898402+0000","last_became_active":"2024-04-16T05:32:55.510891+0000","last_became_peered":"2024-04-16T05:32:55.510891+0000","last_unstale":"2024-04-16T05:33:01.898402+0000","last_undegraded":"2024-04-16T05:33:01.898402+0000","last_fullsized":"2024-04-16T05:33:01.898402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:27:24.894658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.712961+0000","last_change":"2024-04-16T05:32:59.521695+0000","last_active":"2024-04-16T05:33:01.712961+0000","last_peered":"2024-04-16T05:33:01.712961+0000","last_clean":"2024-04-16T05:33:01.712961+0000","last_became_active":"2024-04-16T05:32:59.520569+0000","last_became_peered":"2024-04-16T05:32:59.520569+0000","last_unstale":"2024-04-16T05:33:01.712961+0000","last_undegraded":"2024-04-16T05:33:01.712961+0000","last_fullsized":"2024-04-16T05:33:01.712961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:01:14.726043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.590135+0000","last_change":"2024-04-16T05:32:54.505058+0000","last_active":"2024-04-16T05:33:14.590135+0000","last_peered":"2024-04-16T05:33:14.590135+0000","last_clean":"2024-04-16T05:33:14.590135+0000","last_became_active":"2024-04-16T05:32:54.504636+0000","last_became_peered":"2024-04-16T05:32:54.504636+0000","last_unstale":"2024-04-16T05:33:14.590135+0000","last_undegraded":"2024-04-16T05:33:14.590135+0000","last_fullsized":"2024-04-16T05:33:14.590135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:49:14.429468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896828+0000","last_change":"2024-04-16T05:32:55.518752+0000","last_active":"2024-04-16T05:33:01.896828+0000","last_peered":"2024-04-16T05:33:01.896828+0000","last_clean":"2024-04-16T05:33:01.896828+0000","last_became_active":"2024-04-16T05:32:55.518413+0000","last_became_peered":"2024-04-16T05:32:55.518413+0000","last_unstale":"2024-04-16T05:33:01.896828+0000","last_undegraded":"2024-04-16T05:33:01.896828+0000","last_fullsized":"2024-04-16T05:33:01.896828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:40:32.381456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898199+0000","last_change":"2024-04-16T05:32:59.520394+0000","last_active":"2024-04-16T05:33:01.898199+0000","last_peered":"2024-04-16T05:33:01.898199+0000","last_clean":"2024-04-16T05:33:01.898199+0000","last_became_active":"2024-04-16T05:32:59.519947+0000","last_became_peered":"2024-04-16T05:32:59.519947+0000","last_unstale":"2024-04-16T05:33:01.898199+0000","last_undegraded":"2024-04-16T05:33:01.898199+0000","last_fullsized":"2024-04-16T05:33:01.898199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:33:27.647919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531903+0000","last_change":"2024-04-16T05:32:54.505350+0000","last_active":"2024-04-16T05:33:01.531903+0000","last_peered":"2024-04-16T05:33:01.531903+0000","last_clean":"2024-04-16T05:33:01.531903+0000","last_became_active":"2024-04-16T05:32:54.505011+0000","last_became_peered":"2024-04-16T05:32:54.505011+0000","last_unstale":"2024-04-16T05:33:01.531903+0000","last_undegraded":"2024-04-16T05:33:01.531903+0000","last_fullsized":"2024-04-16T05:33:01.531903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:03:00.276862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897977+0000","last_change":"2024-04-16T05:32:55.517961+0000","last_active":"2024-04-16T05:33:01.897977+0000","last_peered":"2024-04-16T05:33:01.897977+0000","last_clean":"2024-04-16T05:33:01.897977+0000","last_became_active":"2024-04-16T05:32:55.517713+0000","last_became_peered":"2024-04-16T05:32:55.517713+0000","last_unstale":"2024-04-16T05:33:01.897977+0000","last_undegraded":"2024-04-16T05:33:01.897977+0000","last_fullsized":"2024-04-16T05:33:01.897977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:41:30.801833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713118+0000","last_change":"2024-04-16T05:32:59.519660+0000","last_active":"2024-04-16T05:33:01.713118+0000","last_peered":"2024-04-16T05:33:01.713118+0000","last_clean":"2024-04-16T05:33:01.713118+0000","last_became_active":"2024-04-16T05:32:59.518198+0000","last_became_peered":"2024-04-16T05:32:59.518198+0000","last_unstale":"2024-04-16T05:33:01.713118+0000","last_undegraded":"2024-04-16T05:33:01.713118+0000","last_fullsized":"2024-04-16T05:33:01.713118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:03:10.096252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591382+0000","last_change":"2024-04-16T05:32:54.501530+0000","last_active":"2024-04-16T05:33:14.591382+0000","last_peered":"2024-04-16T05:33:14.591382+0000","last_clean":"2024-04-16T05:33:14.591382+0000","last_became_active":"2024-04-16T05:32:54.501159+0000","last_became_peered":"2024-04-16T05:32:54.501159+0000","last_unstale":"2024-04-16T05:33:14.591382+0000","last_undegraded":"2024-04-16T05:33:14.591382+0000","last_fullsized":"2024-04-16T05:33:14.591382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:26:22.240137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897881+0000","last_change":"2024-04-16T05:32:55.512210+0000","last_active":"2024-04-16T05:33:01.897881+0000","last_peered":"2024-04-16T05:33:01.897881+0000","last_clean":"2024-04-16T05:33:01.897881+0000","last_became_active":"2024-04-16T05:32:55.511672+0000","last_became_peered":"2024-04-16T05:32:55.511672+0000","last_unstale":"2024-04-16T05:33:01.897881+0000","last_undegraded":"2024-04-16T05:33:01.897881+0000","last_fullsized":"2024-04-16T05:33:01.897881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:10:53.755803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.532177+0000","last_change":"2024-04-16T05:32:59.527552+0000","last_active":"2024-04-16T05:33:01.532177+0000","last_peered":"2024-04-16T05:33:01.532177+0000","last_clean":"2024-04-16T05:33:01.532177+0000","last_became_active":"2024-04-16T05:32:59.526919+0000","last_became_peered":"2024-04-16T05:32:59.526919+0000","last_unstale":"2024-04-16T05:33:01.532177+0000","last_undegraded":"2024-04-16T05:33:01.532177+0000","last_fullsized":"2024-04-16T05:33:01.532177+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:00:46.342383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591098+0000","last_change":"2024-04-16T05:32:54.494589+0000","last_active":"2024-04-16T05:33:14.591098+0000","last_peered":"2024-04-16T05:33:14.591098+0000","last_clean":"2024-04-16T05:33:14.591098+0000","last_became_active":"2024-04-16T05:32:54.494221+0000","last_became_peered":"2024-04-16T05:32:54.494221+0000","last_unstale":"2024-04-16T05:33:14.591098+0000","last_undegraded":"2024-04-16T05:33:14.591098+0000","last_fullsized":"2024-04-16T05:33:14.591098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:17:53.245359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530305+0000","last_change":"2024-04-16T05:32:55.502724+0000","last_active":"2024-04-16T05:33:01.530305+0000","last_peered":"2024-04-16T05:33:01.530305+0000","last_clean":"2024-04-16T05:33:01.530305+0000","last_became_active":"2024-04-16T05:32:55.502415+0000","last_became_peered":"2024-04-16T05:32:55.502415+0000","last_unstale":"2024-04-16T05:33:01.530305+0000","last_undegraded":"2024-04-16T05:33:01.530305+0000","last_fullsized":"2024-04-16T05:33:01.530305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:20:44.443926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897486+0000","last_change":"2024-04-16T05:32:59.522243+0000","last_active":"2024-04-16T05:33:01.897486+0000","last_peered":"2024-04-16T05:33:01.897486+0000","last_clean":"2024-04-16T05:33:01.897486+0000","last_became_active":"2024-04-16T05:32:59.521823+0000","last_became_peered":"2024-04-16T05:32:59.521823+0000","last_unstale":"2024-04-16T05:33:01.897486+0000","last_undegraded":"2024-04-16T05:33:01.897486+0000","last_fullsized":"2024-04-16T05:33:01.897486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:09:11.158434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591852+0000","last_change":"2024-04-16T05:32:54.507505+0000","last_active":"2024-04-16T05:33:14.591852+0000","last_peered":"2024-04-16T05:33:14.591852+0000","last_clean":"2024-04-16T05:33:14.591852+0000","last_became_active":"2024-04-16T05:32:54.506558+0000","last_became_peered":"2024-04-16T05:32:54.506558+0000","last_unstale":"2024-04-16T05:33:14.591852+0000","last_undegraded":"2024-04-16T05:33:14.591852+0000","last_fullsized":"2024-04-16T05:33:14.591852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:36:17.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534342+0000","last_change":"2024-04-16T05:32:55.518998+0000","last_active":"2024-04-16T05:33:01.534342+0000","last_peered":"2024-04-16T05:33:01.534342+0000","last_clean":"2024-04-16T05:33:01.534342+0000","last_became_active":"2024-04-16T05:32:55.518640+0000","last_became_peered":"2024-04-16T05:32:55.518640+0000","last_unstale":"2024-04-16T05:33:01.534342+0000","last_undegraded":"2024-04-16T05:33:01.534342+0000","last_fullsized":"2024-04-16T05:33:01.534342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:42:44.667504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713373+0000","last_change":"2024-04-16T05:32:59.519050+0000","last_active":"2024-04-16T05:33:01.713373+0000","last_peered":"2024-04-16T05:33:01.713373+0000","last_clean":"2024-04-16T05:33:01.713373+0000","last_became_active":"2024-04-16T05:32:59.518663+0000","last_became_peered":"2024-04-16T05:32:59.518663+0000","last_unstale":"2024-04-16T05:33:01.713373+0000","last_undegraded":"2024-04-16T05:33:01.713373+0000","last_fullsized":"2024-04-16T05:33:01.713373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:36:19.649605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.600587+0000","last_change":"2024-04-16T05:32:54.500940+0000","last_active":"2024-04-16T05:33:15.600587+0000","last_peered":"2024-04-16T05:33:15.600587+0000","last_clean":"2024-04-16T05:33:15.600587+0000","last_became_active":"2024-04-16T05:32:54.500542+0000","last_became_peered":"2024-04-16T05:32:54.500542+0000","last_unstale":"2024-04-16T05:33:15.600587+0000","last_undegraded":"2024-04-16T05:33:15.600587+0000","last_fullsized":"2024-04-16T05:33:15.600587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:38:00.232244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534185+0000","last_change":"2024-04-16T05:32:55.509889+0000","last_active":"2024-04-16T05:33:01.534185+0000","last_peered":"2024-04-16T05:33:01.534185+0000","last_clean":"2024-04-16T05:33:01.534185+0000","last_became_active":"2024-04-16T05:32:55.508911+0000","last_became_peered":"2024-04-16T05:32:55.508911+0000","last_unstale":"2024-04-16T05:33:01.534185+0000","last_undegraded":"2024-04-16T05:33:01.534185+0000","last_fullsized":"2024-04-16T05:33:01.534185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:14:46.609127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530542+0000","last_change":"2024-04-16T05:32:59.523555+0000","last_active":"2024-04-16T05:33:01.530542+0000","last_peered":"2024-04-16T05:33:01.530542+0000","last_clean":"2024-04-16T05:33:01.530542+0000","last_became_active":"2024-04-16T05:32:59.523187+0000","last_became_peered":"2024-04-16T05:32:59.523187+0000","last_unstale":"2024-04-16T05:33:01.530542+0000","last_undegraded":"2024-04-16T05:33:01.530542+0000","last_fullsized":"2024-04-16T05:33:01.530542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:35.049355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'16","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.602325+0000","last_change":"2024-04-16T05:32:54.503247+0000","last_active":"2024-04-16T05:33:15.602325+0000","last_peered":"2024-04-16T05:33:15.602325+0000","last_clean":"2024-04-16T05:33:15.602325+0000","last_became_active":"2024-04-16T05:32:54.502927+0000","last_became_peered":"2024-04-16T05:32:54.502927+0000","last_unstale":"2024-04-16T05:33:15.602325+0000","last_undegraded":"2024-04-16T05:33:15.602325+0000","last_fullsized":"2024-04-16T05:33:15.602325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:36:17.585827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531264+0000","last_change":"2024-04-16T05:32:55.519265+0000","last_active":"2024-04-16T05:33:01.531264+0000","last_peered":"2024-04-16T05:33:01.531264+0000","last_clean":"2024-04-16T05:33:01.531264+0000","last_became_active":"2024-04-16T05:32:55.518840+0000","last_became_peered":"2024-04-16T05:32:55.518840+0000","last_unstale":"2024-04-16T05:33:01.531264+0000","last_undegraded":"2024-04-16T05:33:01.531264+0000","last_fullsized":"2024-04-16T05:33:01.531264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:56.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531344+0000","last_change":"2024-04-16T05:32:59.525275+0000","last_active":"2024-04-16T05:33:01.531344+0000","last_peered":"2024-04-16T05:33:01.531344+0000","last_clean":"2024-04-16T05:33:01.531344+0000","last_became_active":"2024-04-16T05:32:59.524546+0000","last_became_peered":"2024-04-16T05:32:59.524546+0000","last_unstale":"2024-04-16T05:33:01.531344+0000","last_undegraded":"2024-04-16T05:33:01.531344+0000","last_fullsized":"2024-04-16T05:33:01.531344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:07.561898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898484+0000","last_change":"2024-04-16T05:32:54.506904+0000","last_active":"2024-04-16T05:33:01.898484+0000","last_peered":"2024-04-16T05:33:01.898484+0000","last_clean":"2024-04-16T05:33:01.898484+0000","last_became_active":"2024-04-16T05:32:54.505654+0000","last_became_peered":"2024-04-16T05:32:54.505654+0000","last_unstale":"2024-04-16T05:33:01.898484+0000","last_undegraded":"2024-04-16T05:33:01.898484+0000","last_fullsized":"2024-04-16T05:33:01.898484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:30.444817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896903+0000","last_change":"2024-04-16T05:32:55.515685+0000","last_active":"2024-04-16T05:33:01.896903+0000","last_peered":"2024-04-16T05:33:01.896903+0000","last_clean":"2024-04-16T05:33:01.896903+0000","last_became_active":"2024-04-16T05:32:55.515367+0000","last_became_peered":"2024-04-16T05:32:55.515367+0000","last_unstale":"2024-04-16T05:33:01.896903+0000","last_undegraded":"2024-04-16T05:33:01.896903+0000","last_fullsized":"2024-04-16T05:33:01.896903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:46.670143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896718+0000","last_change":"2024-04-16T05:33:00.419828+0000","last_active":"2024-04-16T05:33:01.896718+0000","last_peered":"2024-04-16T05:33:01.896718+0000","last_clean":"2024-04-16T05:33:01.896718+0000","last_became_active":"2024-04-16T05:33:00.419321+0000","last_became_peered":"2024-04-16T05:33:00.419321+0000","last_unstale":"2024-04-16T05:33:01.896718+0000","last_undegraded":"2024-04-16T05:33:01.896718+0000","last_fullsized":"2024-04-16T05:33:01.896718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:56.694914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713229+0000","last_change":"2024-04-16T05:32:54.505369+0000","last_active":"2024-04-16T05:33:01.713229+0000","last_peered":"2024-04-16T05:33:01.713229+0000","last_clean":"2024-04-16T05:33:01.713229+0000","last_became_active":"2024-04-16T05:32:54.505043+0000","last_became_peered":"2024-04-16T05:32:54.505043+0000","last_unstale":"2024-04-16T05:33:01.713229+0000","last_undegraded":"2024-04-16T05:33:01.713229+0000","last_fullsized":"2024-04-16T05:33:01.713229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:34.242117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708768+0000","last_change":"2024-04-16T05:32:55.509970+0000","last_active":"2024-04-16T05:33:01.708768+0000","last_peered":"2024-04-16T05:33:01.708768+0000","last_clean":"2024-04-16T05:33:01.708768+0000","last_became_active":"2024-04-16T05:32:55.509632+0000","last_became_peered":"2024-04-16T05:32:55.509632+0000","last_unstale":"2024-04-16T05:33:01.708768+0000","last_undegraded":"2024-04-16T05:33:01.708768+0000","last_fullsized":"2024-04-16T05:33:01.708768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:33:55.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530576+0000","last_change":"2024-04-16T05:32:59.527007+0000","last_active":"2024-04-16T05:33:01.530576+0000","last_peered":"2024-04-16T05:33:01.530576+0000","last_clean":"2024-04-16T05:33:01.530576+0000","last_became_active":"2024-04-16T05:32:59.526645+0000","last_became_peered":"2024-04-16T05:32:59.526645+0000","last_unstale":"2024-04-16T05:33:01.530576+0000","last_undegraded":"2024-04-16T05:33:01.530576+0000","last_fullsized":"2024-04-16T05:33:01.530576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:46:06.698912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534301+0000","last_change":"2024-04-16T05:32:54.503409+0000","last_active":"2024-04-16T05:33:01.534301+0000","last_peered":"2024-04-16T05:33:01.534301+0000","last_clean":"2024-04-16T05:33:01.534301+0000","last_became_active":"2024-04-16T05:32:54.503069+0000","last_became_peered":"2024-04-16T05:32:54.503069+0000","last_unstale":"2024-04-16T05:33:01.534301+0000","last_undegraded":"2024-04-16T05:33:01.534301+0000","last_fullsized":"2024-04-16T05:33:01.534301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:45:10.535225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.708527+0000","last_change":"2024-04-16T05:32:55.519744+0000","last_active":"2024-04-16T05:33:01.708527+0000","last_peered":"2024-04-16T05:33:01.708527+0000","last_clean":"2024-04-16T05:33:01.708527+0000","last_became_active":"2024-04-16T05:32:55.519091+0000","last_became_peered":"2024-04-16T05:32:55.519091+0000","last_unstale":"2024-04-16T05:33:01.708527+0000","last_undegraded":"2024-04-16T05:33:01.708527+0000","last_fullsized":"2024-04-16T05:33:01.708527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:19.417691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530653+0000","last_change":"2024-04-16T05:33:00.419494+0000","last_active":"2024-04-16T05:33:01.530653+0000","last_peered":"2024-04-16T05:33:01.530653+0000","last_clean":"2024-04-16T05:33:01.530653+0000","last_became_active":"2024-04-16T05:33:00.418954+0000","last_became_peered":"2024-04-16T05:33:00.418954+0000","last_unstale":"2024-04-16T05:33:01.530653+0000","last_undegraded":"2024-04-16T05:33:01.530653+0000","last_fullsized":"2024-04-16T05:33:01.530653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:03:01.835304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531404+0000","last_change":"2024-04-16T05:32:55.518272+0000","last_active":"2024-04-16T05:33:01.531404+0000","last_peered":"2024-04-16T05:33:01.531404+0000","last_clean":"2024-04-16T05:33:01.531404+0000","last_became_active":"2024-04-16T05:32:55.517928+0000","last_became_peered":"2024-04-16T05:32:55.517928+0000","last_unstale":"2024-04-16T05:33:01.531404+0000","last_undegraded":"2024-04-16T05:33:01.531404+0000","last_fullsized":"2024-04-16T05:33:01.531404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:44:47.739994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.601587+0000","last_change":"2024-04-16T05:32:54.503923+0000","last_active":"2024-04-16T05:33:15.601587+0000","last_peered":"2024-04-16T05:33:15.601587+0000","last_clean":"2024-04-16T05:33:15.601587+0000","last_became_active":"2024-04-16T05:32:54.503576+0000","last_became_peered":"2024-04-16T05:32:54.503576+0000","last_unstale":"2024-04-16T05:33:15.601587+0000","last_undegraded":"2024-04-16T05:33:15.601587+0000","last_fullsized":"2024-04-16T05:33:15.601587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:49:36.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534678+0000","last_change":"2024-04-16T05:32:59.517757+0000","last_active":"2024-04-16T05:33:01.534678+0000","last_peered":"2024-04-16T05:33:01.534678+0000","last_clean":"2024-04-16T05:33:01.534678+0000","last_became_active":"2024-04-16T05:32:59.517357+0000","last_became_peered":"2024-04-16T05:32:59.517357+0000","last_unstale":"2024-04-16T05:33:01.534678+0000","last_undegraded":"2024-04-16T05:33:01.534678+0000","last_fullsized":"2024-04-16T05:33:01.534678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:21.767071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896342+0000","last_change":"2024-04-16T05:32:55.514617+0000","last_active":"2024-04-16T05:33:01.896342+0000","last_peered":"2024-04-16T05:33:01.896342+0000","last_clean":"2024-04-16T05:33:01.896342+0000","last_became_active":"2024-04-16T05:32:55.512660+0000","last_became_peered":"2024-04-16T05:32:55.512660+0000","last_unstale":"2024-04-16T05:33:01.896342+0000","last_undegraded":"2024-04-16T05:33:01.896342+0000","last_fullsized":"2024-04-16T05:33:01.896342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:31:45.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591140+0000","last_change":"2024-04-16T05:32:54.498650+0000","last_active":"2024-04-16T05:33:14.591140+0000","last_peered":"2024-04-16T05:33:14.591140+0000","last_clean":"2024-04-16T05:33:14.591140+0000","last_became_active":"2024-04-16T05:32:54.497672+0000","last_became_peered":"2024-04-16T05:32:54.497672+0000","last_unstale":"2024-04-16T05:33:14.591140+0000","last_undegraded":"2024-04-16T05:33:14.591140+0000","last_fullsized":"2024-04-16T05:33:14.591140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:36.908809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.533908+0000","last_change":"2024-04-16T05:31:42.831800+0000","last_active":"2024-04-16T05:33:01.533908+0000","last_peered":"2024-04-16T05:33:01.533908+0000","last_clean":"2024-04-16T05:33:01.533908+0000","last_became_active":"2024-04-16T05:31:42.831391+0000","last_became_peered":"2024-04-16T05:31:42.831391+0000","last_unstale":"2024-04-16T05:33:01.533908+0000","last_undegraded":"2024-04-16T05:33:01.533908+0000","last_fullsized":"2024-04-16T05:33:01.533908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_clean_scrub_stamp":"2024-04-16T05:28:15.969690+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.152803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.532032+0000","last_change":"2024-04-16T05:33:00.420445+0000","last_active":"2024-04-16T05:33:01.532032+0000","last_peered":"2024-04-16T05:33:01.532032+0000","last_clean":"2024-04-16T05:33:01.532032+0000","last_became_active":"2024-04-16T05:33:00.419342+0000","last_became_peered":"2024-04-16T05:33:00.419342+0000","last_unstale":"2024-04-16T05:33:01.532032+0000","last_undegraded":"2024-04-16T05:33:01.532032+0000","last_fullsized":"2024-04-16T05:33:01.532032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:57:15.703251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530088+0000","last_change":"2024-04-16T05:32:55.512324+0000","last_active":"2024-04-16T05:33:01.530088+0000","last_peered":"2024-04-16T05:33:01.530088+0000","last_clean":"2024-04-16T05:33:01.530088+0000","last_became_active":"2024-04-16T05:32:55.511958+0000","last_became_peered":"2024-04-16T05:32:55.511958+0000","last_unstale":"2024-04-16T05:33:01.530088+0000","last_undegraded":"2024-04-16T05:33:01.530088+0000","last_fullsized":"2024-04-16T05:33:01.530088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:38.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:15.601993+0000","last_change":"2024-04-16T05:32:54.508021+0000","last_active":"2024-04-16T05:33:15.601993+0000","last_peered":"2024-04-16T05:33:15.601993+0000","last_clean":"2024-04-16T05:33:15.601993+0000","last_became_active":"2024-04-16T05:32:54.507645+0000","last_became_peered":"2024-04-16T05:32:54.507645+0000","last_unstale":"2024-04-16T05:33:15.601993+0000","last_undegraded":"2024-04-16T05:33:15.601993+0000","last_fullsized":"2024-04-16T05:33:15.601993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:21:50.721260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896998+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:33:01.896998+0000","last_peered":"2024-04-16T05:33:01.896998+0000","last_clean":"2024-04-16T05:33:01.896998+0000","last_became_active":"2024-04-16T05:32:59.523159+0000","last_became_peered":"2024-04-16T05:32:59.523159+0000","last_unstale":"2024-04-16T05:33:01.896998+0000","last_undegraded":"2024-04-16T05:33:01.896998+0000","last_fullsized":"2024-04-16T05:33:01.896998+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:48:37.696713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:16.608227+0000","last_change":"2024-04-16T05:32:54.503262+0000","last_active":"2024-04-16T05:33:16.608227+0000","last_peered":"2024-04-16T05:33:16.608227+0000","last_clean":"2024-04-16T05:33:16.608227+0000","last_became_active":"2024-04-16T05:32:54.502909+0000","last_became_peered":"2024-04-16T05:32:54.502909+0000","last_unstale":"2024-04-16T05:33:16.608227+0000","last_undegraded":"2024-04-16T05:33:16.608227+0000","last_fullsized":"2024-04-16T05:33:16.608227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:48:32.841921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530699+0000","last_change":"2024-04-16T05:32:55.507126+0000","last_active":"2024-04-16T05:33:01.530699+0000","last_peered":"2024-04-16T05:33:01.530699+0000","last_clean":"2024-04-16T05:33:01.530699+0000","last_became_active":"2024-04-16T05:32:55.506743+0000","last_became_peered":"2024-04-16T05:32:55.506743+0000","last_unstale":"2024-04-16T05:33:01.530699+0000","last_undegraded":"2024-04-16T05:33:01.530699+0000","last_fullsized":"2024-04-16T05:33:01.530699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:10:58.697649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.895885+0000","last_change":"2024-04-16T05:33:00.419435+0000","last_active":"2024-04-16T05:33:01.895885+0000","last_peered":"2024-04-16T05:33:01.895885+0000","last_clean":"2024-04-16T05:33:01.895885+0000","last_became_active":"2024-04-16T05:33:00.418985+0000","last_became_peered":"2024-04-16T05:33:00.418985+0000","last_unstale":"2024-04-16T05:33:01.895885+0000","last_undegraded":"2024-04-16T05:33:01.895885+0000","last_fullsized":"2024-04-16T05:33:01.895885+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:08:31.188911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:16.610279+0000","last_change":"2024-04-16T05:32:54.503349+0000","last_active":"2024-04-16T05:33:16.610279+0000","last_peered":"2024-04-16T05:33:16.610279+0000","last_clean":"2024-04-16T05:33:16.610279+0000","last_became_active":"2024-04-16T05:32:54.503018+0000","last_became_peered":"2024-04-16T05:32:54.503018+0000","last_unstale":"2024-04-16T05:33:16.610279+0000","last_undegraded":"2024-04-16T05:33:16.610279+0000","last_fullsized":"2024-04-16T05:33:16.610279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:34:20.853543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896068+0000","last_change":"2024-04-16T05:32:55.511201+0000","last_active":"2024-04-16T05:33:01.896068+0000","last_peered":"2024-04-16T05:33:01.896068+0000","last_clean":"2024-04-16T05:33:01.896068+0000","last_became_active":"2024-04-16T05:32:55.510906+0000","last_became_peered":"2024-04-16T05:32:55.510906+0000","last_unstale":"2024-04-16T05:33:01.896068+0000","last_undegraded":"2024-04-16T05:33:01.896068+0000","last_fullsized":"2024-04-16T05:33:01.896068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:37:37.147628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897533+0000","last_change":"2024-04-16T05:32:59.518440+0000","last_active":"2024-04-16T05:33:01.897533+0000","last_peered":"2024-04-16T05:33:01.897533+0000","last_clean":"2024-04-16T05:33:01.897533+0000","last_became_active":"2024-04-16T05:32:59.517870+0000","last_became_peered":"2024-04-16T05:32:59.517870+0000","last_unstale":"2024-04-16T05:33:01.897533+0000","last_undegraded":"2024-04-16T05:33:01.897533+0000","last_fullsized":"2024-04-16T05:33:01.897533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:51:31.157418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.589574+0000","last_change":"2024-04-16T05:32:54.503983+0000","last_active":"2024-04-16T05:33:14.589574+0000","last_peered":"2024-04-16T05:33:14.589574+0000","last_clean":"2024-04-16T05:33:14.589574+0000","last_became_active":"2024-04-16T05:32:54.502367+0000","last_became_peered":"2024-04-16T05:32:54.502367+0000","last_unstale":"2024-04-16T05:33:14.589574+0000","last_undegraded":"2024-04-16T05:33:14.589574+0000","last_fullsized":"2024-04-16T05:33:14.589574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:49:36.997692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896650+0000","last_change":"2024-04-16T05:32:55.516812+0000","last_active":"2024-04-16T05:33:01.896650+0000","last_peered":"2024-04-16T05:33:01.896650+0000","last_clean":"2024-04-16T05:33:01.896650+0000","last_became_active":"2024-04-16T05:32:55.516477+0000","last_became_peered":"2024-04-16T05:32:55.516477+0000","last_unstale":"2024-04-16T05:33:01.896650+0000","last_undegraded":"2024-04-16T05:33:01.896650+0000","last_fullsized":"2024-04-16T05:33:01.896650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:24:29.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897710+0000","last_change":"2024-04-16T05:33:00.419622+0000","last_active":"2024-04-16T05:33:01.897710+0000","last_peered":"2024-04-16T05:33:01.897710+0000","last_clean":"2024-04-16T05:33:01.897710+0000","last_became_active":"2024-04-16T05:33:00.419135+0000","last_became_peered":"2024-04-16T05:33:00.419135+0000","last_unstale":"2024-04-16T05:33:01.897710+0000","last_undegraded":"2024-04-16T05:33:01.897710+0000","last_fullsized":"2024-04-16T05:33:01.897710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:38:02.529192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.590517+0000","last_change":"2024-04-16T05:32:54.503652+0000","last_active":"2024-04-16T05:33:14.590517+0000","last_peered":"2024-04-16T05:33:14.590517+0000","last_clean":"2024-04-16T05:33:14.590517+0000","last_became_active":"2024-04-16T05:32:54.503311+0000","last_became_peered":"2024-04-16T05:32:54.503311+0000","last_unstale":"2024-04-16T05:33:14.590517+0000","last_undegraded":"2024-04-16T05:33:14.590517+0000","last_fullsized":"2024-04-16T05:33:14.590517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:49:21.584029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896506+0000","last_change":"2024-04-16T05:32:55.508145+0000","last_active":"2024-04-16T05:33:01.896506+0000","last_peered":"2024-04-16T05:33:01.896506+0000","last_clean":"2024-04-16T05:33:01.896506+0000","last_became_active":"2024-04-16T05:32:55.507794+0000","last_became_peered":"2024-04-16T05:32:55.507794+0000","last_unstale":"2024-04-16T05:33:01.896506+0000","last_undegraded":"2024-04-16T05:33:01.896506+0000","last_fullsized":"2024-04-16T05:33:01.896506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:11:57.925733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.713507+0000","last_change":"2024-04-16T05:32:59.527273+0000","last_active":"2024-04-16T05:33:01.713507+0000","last_peered":"2024-04-16T05:33:01.713507+0000","last_clean":"2024-04-16T05:33:01.713507+0000","last_became_active":"2024-04-16T05:32:59.522925+0000","last_became_peered":"2024-04-16T05:32:59.522925+0000","last_unstale":"2024-04-16T05:33:01.713507+0000","last_undegraded":"2024-04-16T05:33:01.713507+0000","last_fullsized":"2024-04-16T05:33:01.713507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:33:16.297503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591026+0000","last_change":"2024-04-16T05:32:54.500909+0000","last_active":"2024-04-16T05:33:14.591026+0000","last_peered":"2024-04-16T05:33:14.591026+0000","last_clean":"2024-04-16T05:33:14.591026+0000","last_became_active":"2024-04-16T05:32:54.499348+0000","last_became_peered":"2024-04-16T05:32:54.499348+0000","last_unstale":"2024-04-16T05:33:14.591026+0000","last_undegraded":"2024-04-16T05:33:14.591026+0000","last_fullsized":"2024-04-16T05:33:14.591026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:51:49.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896672+0000","last_change":"2024-04-16T05:32:55.510922+0000","last_active":"2024-04-16T05:33:01.896672+0000","last_peered":"2024-04-16T05:33:01.896672+0000","last_clean":"2024-04-16T05:33:01.896672+0000","last_became_active":"2024-04-16T05:32:55.510546+0000","last_became_peered":"2024-04-16T05:32:55.510546+0000","last_unstale":"2024-04-16T05:33:01.896672+0000","last_undegraded":"2024-04-16T05:33:01.896672+0000","last_fullsized":"2024-04-16T05:33:01.896672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:27.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897451+0000","last_change":"2024-04-16T05:32:59.524257+0000","last_active":"2024-04-16T05:33:01.897451+0000","last_peered":"2024-04-16T05:33:01.897451+0000","last_clean":"2024-04-16T05:33:01.897451+0000","last_became_active":"2024-04-16T05:32:59.523768+0000","last_became_peered":"2024-04-16T05:32:59.523768+0000","last_unstale":"2024-04-16T05:33:01.897451+0000","last_undegraded":"2024-04-16T05:33:01.897451+0000","last_fullsized":"2024-04-16T05:33:01.897451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:38:27.594048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896965+0000","last_change":"2024-04-16T05:32:54.504430+0000","last_active":"2024-04-16T05:33:01.896965+0000","last_peered":"2024-04-16T05:33:01.896965+0000","last_clean":"2024-04-16T05:33:01.896965+0000","last_became_active":"2024-04-16T05:32:54.504054+0000","last_became_peered":"2024-04-16T05:32:54.504054+0000","last_unstale":"2024-04-16T05:33:01.896965+0000","last_undegraded":"2024-04-16T05:33:01.896965+0000","last_fullsized":"2024-04-16T05:33:01.896965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:53:24.875786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896270+0000","last_change":"2024-04-16T05:32:55.508005+0000","last_active":"2024-04-16T05:33:01.896270+0000","last_peered":"2024-04-16T05:33:01.896270+0000","last_clean":"2024-04-16T05:33:01.896270+0000","last_became_active":"2024-04-16T05:32:55.507698+0000","last_became_peered":"2024-04-16T05:32:55.507698+0000","last_unstale":"2024-04-16T05:33:01.896270+0000","last_undegraded":"2024-04-16T05:33:01.896270+0000","last_fullsized":"2024-04-16T05:33:01.896270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:11:12.423294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897885+0000","last_change":"2024-04-16T05:33:00.419442+0000","last_active":"2024-04-16T05:33:01.897885+0000","last_peered":"2024-04-16T05:33:01.897885+0000","last_clean":"2024-04-16T05:33:01.897885+0000","last_became_active":"2024-04-16T05:33:00.418906+0000","last_became_peered":"2024-04-16T05:33:00.418906+0000","last_unstale":"2024-04-16T05:33:01.897885+0000","last_undegraded":"2024-04-16T05:33:01.897885+0000","last_fullsized":"2024-04-16T05:33:01.897885+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:21:22.124879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:14.591283+0000","last_change":"2024-04-16T05:32:54.509090+0000","last_active":"2024-04-16T05:33:14.591283+0000","last_peered":"2024-04-16T05:33:14.591283+0000","last_clean":"2024-04-16T05:33:14.591283+0000","last_became_active":"2024-04-16T05:32:54.508700+0000","last_became_peered":"2024-04-16T05:32:54.508700+0000","last_unstale":"2024-04-16T05:33:14.591283+0000","last_undegraded":"2024-04-16T05:33:14.591283+0000","last_fullsized":"2024-04-16T05:33:14.591283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:53:34.344073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534659+0000","last_change":"2024-04-16T05:32:55.509658+0000","last_active":"2024-04-16T05:33:01.534659+0000","last_peered":"2024-04-16T05:33:01.534659+0000","last_clean":"2024-04-16T05:33:01.534659+0000","last_became_active":"2024-04-16T05:32:55.509258+0000","last_became_peered":"2024-04-16T05:32:55.509258+0000","last_unstale":"2024-04-16T05:33:01.534659+0000","last_undegraded":"2024-04-16T05:33:01.534659+0000","last_fullsized":"2024-04-16T05:33:01.534659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:17:03.490544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897075+0000","last_change":"2024-04-16T05:32:59.519260+0000","last_active":"2024-04-16T05:33:01.897075+0000","last_peered":"2024-04-16T05:33:01.897075+0000","last_clean":"2024-04-16T05:33:01.897075+0000","last_became_active":"2024-04-16T05:32:59.518839+0000","last_became_peered":"2024-04-16T05:32:59.518839+0000","last_unstale":"2024-04-16T05:33:01.897075+0000","last_undegraded":"2024-04-16T05:33:01.897075+0000","last_fullsized":"2024-04-16T05:33:01.897075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:54:50.555214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.898392+0000","last_change":"2024-04-16T05:32:54.503226+0000","last_active":"2024-04-16T05:33:01.898392+0000","last_peered":"2024-04-16T05:33:01.898392+0000","last_clean":"2024-04-16T05:33:01.898392+0000","last_became_active":"2024-04-16T05:32:54.502687+0000","last_became_peered":"2024-04-16T05:32:54.502687+0000","last_unstale":"2024-04-16T05:33:01.898392+0000","last_undegraded":"2024-04-16T05:33:01.898392+0000","last_fullsized":"2024-04-16T05:33:01.898392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:34:24.228464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534906+0000","last_change":"2024-04-16T05:32:55.515203+0000","last_active":"2024-04-16T05:33:01.534906+0000","last_peered":"2024-04-16T05:33:01.534906+0000","last_clean":"2024-04-16T05:33:01.534906+0000","last_became_active":"2024-04-16T05:32:55.514815+0000","last_became_peered":"2024-04-16T05:32:55.514815+0000","last_unstale":"2024-04-16T05:33:01.534906+0000","last_undegraded":"2024-04-16T05:33:01.534906+0000","last_fullsized":"2024-04-16T05:33:01.534906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:11:18.464532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.895954+0000","last_change":"2024-04-16T05:32:59.520442+0000","last_active":"2024-04-16T05:33:01.895954+0000","last_peered":"2024-04-16T05:33:01.895954+0000","last_clean":"2024-04-16T05:33:01.895954+0000","last_became_active":"2024-04-16T05:32:59.520071+0000","last_became_peered":"2024-04-16T05:32:59.520071+0000","last_unstale":"2024-04-16T05:33:01.895954+0000","last_undegraded":"2024-04-16T05:33:01.895954+0000","last_fullsized":"2024-04-16T05:33:01.895954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:19:16.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531948+0000","last_change":"2024-04-16T05:32:54.510098+0000","last_active":"2024-04-16T05:33:01.531948+0000","last_peered":"2024-04-16T05:33:01.531948+0000","last_clean":"2024-04-16T05:33:01.531948+0000","last_became_active":"2024-04-16T05:32:54.509750+0000","last_became_peered":"2024-04-16T05:32:54.509750+0000","last_unstale":"2024-04-16T05:33:01.531948+0000","last_undegraded":"2024-04-16T05:33:01.531948+0000","last_fullsized":"2024-04-16T05:33:01.531948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:00:09.528958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534448+0000","last_change":"2024-04-16T05:32:55.510872+0000","last_active":"2024-04-16T05:33:01.534448+0000","last_peered":"2024-04-16T05:33:01.534448+0000","last_clean":"2024-04-16T05:33:01.534448+0000","last_became_active":"2024-04-16T05:32:55.510471+0000","last_became_peered":"2024-04-16T05:32:55.510471+0000","last_unstale":"2024-04-16T05:33:01.534448+0000","last_undegraded":"2024-04-16T05:33:01.534448+0000","last_fullsized":"2024-04-16T05:33:01.534448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:47:36.294039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531085+0000","last_change":"2024-04-16T05:32:59.521334+0000","last_active":"2024-04-16T05:33:01.531085+0000","last_peered":"2024-04-16T05:33:01.531085+0000","last_clean":"2024-04-16T05:33:01.531085+0000","last_became_active":"2024-04-16T05:32:59.520348+0000","last_became_peered":"2024-04-16T05:32:59.520348+0000","last_unstale":"2024-04-16T05:33:01.531085+0000","last_undegraded":"2024-04-16T05:33:01.531085+0000","last_fullsized":"2024-04-16T05:33:01.531085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:40.628552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530512+0000","last_change":"2024-04-16T05:32:54.503062+0000","last_active":"2024-04-16T05:33:01.530512+0000","last_peered":"2024-04-16T05:33:01.530512+0000","last_clean":"2024-04-16T05:33:01.530512+0000","last_became_active":"2024-04-16T05:32:54.502762+0000","last_became_peered":"2024-04-16T05:32:54.502762+0000","last_unstale":"2024-04-16T05:33:01.530512+0000","last_undegraded":"2024-04-16T05:33:01.530512+0000","last_fullsized":"2024-04-16T05:33:01.530512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:43:57.911296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.534903+0000","last_change":"2024-04-16T05:32:55.510461+0000","last_active":"2024-04-16T05:33:01.534903+0000","last_peered":"2024-04-16T05:33:01.534903+0000","last_clean":"2024-04-16T05:33:01.534903+0000","last_became_active":"2024-04-16T05:32:55.510142+0000","last_became_peered":"2024-04-16T05:32:55.510142+0000","last_unstale":"2024-04-16T05:33:01.534903+0000","last_undegraded":"2024-04-16T05:33:01.534903+0000","last_fullsized":"2024-04-16T05:33:01.534903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:41.346078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896784+0000","last_change":"2024-04-16T05:32:55.512600+0000","last_active":"2024-04-16T05:33:01.896784+0000","last_peered":"2024-04-16T05:33:01.896784+0000","last_clean":"2024-04-16T05:33:01.896784+0000","last_became_active":"2024-04-16T05:32:55.512265+0000","last_became_peered":"2024-04-16T05:32:55.512265+0000","last_unstale":"2024-04-16T05:33:01.896784+0000","last_undegraded":"2024-04-16T05:33:01.896784+0000","last_fullsized":"2024-04-16T05:33:01.896784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:04:46.112644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897803+0000","last_change":"2024-04-16T05:32:55.514481+0000","last_active":"2024-04-16T05:33:01.897803+0000","last_peered":"2024-04-16T05:33:01.897803+0000","last_clean":"2024-04-16T05:33:01.897803+0000","last_became_active":"2024-04-16T05:32:55.512485+0000","last_became_peered":"2024-04-16T05:32:55.512485+0000","last_unstale":"2024-04-16T05:33:01.897803+0000","last_undegraded":"2024-04-16T05:33:01.897803+0000","last_fullsized":"2024-04-16T05:33:01.897803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:33:02.179774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896827+0000","last_change":"2024-04-16T05:32:55.509849+0000","last_active":"2024-04-16T05:33:01.896827+0000","last_peered":"2024-04-16T05:33:01.896827+0000","last_clean":"2024-04-16T05:33:01.896827+0000","last_became_active":"2024-04-16T05:32:55.509338+0000","last_became_peered":"2024-04-16T05:32:55.509338+0000","last_unstale":"2024-04-16T05:33:01.896827+0000","last_undegraded":"2024-04-16T05:33:01.896827+0000","last_fullsized":"2024-04-16T05:33:01.896827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:32:38.900455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897030+0000","last_change":"2024-04-16T05:32:55.518556+0000","last_active":"2024-04-16T05:33:01.897030+0000","last_peered":"2024-04-16T05:33:01.897030+0000","last_clean":"2024-04-16T05:33:01.897030+0000","last_became_active":"2024-04-16T05:32:55.518240+0000","last_became_peered":"2024-04-16T05:32:55.518240+0000","last_unstale":"2024-04-16T05:33:01.897030+0000","last_undegraded":"2024-04-16T05:33:01.897030+0000","last_fullsized":"2024-04-16T05:33:01.897030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:25:46.002516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530711+0000","last_change":"2024-04-16T05:32:55.518106+0000","last_active":"2024-04-16T05:33:01.530711+0000","last_peered":"2024-04-16T05:33:01.530711+0000","last_clean":"2024-04-16T05:33:01.530711+0000","last_became_active":"2024-04-16T05:32:55.517403+0000","last_became_peered":"2024-04-16T05:32:55.517403+0000","last_unstale":"2024-04-16T05:33:01.530711+0000","last_undegraded":"2024-04-16T05:33:01.530711+0000","last_fullsized":"2024-04-16T05:33:01.530711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:36:34.731644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897351+0000","last_change":"2024-04-16T05:32:55.512889+0000","last_active":"2024-04-16T05:33:01.897351+0000","last_peered":"2024-04-16T05:33:01.897351+0000","last_clean":"2024-04-16T05:33:01.897351+0000","last_became_active":"2024-04-16T05:32:55.511670+0000","last_became_peered":"2024-04-16T05:32:55.511670+0000","last_unstale":"2024-04-16T05:33:01.897351+0000","last_undegraded":"2024-04-16T05:33:01.897351+0000","last_fullsized":"2024-04-16T05:33:01.897351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:30:39.320608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897707+0000","last_change":"2024-04-16T05:32:55.507670+0000","last_active":"2024-04-16T05:33:01.897707+0000","last_peered":"2024-04-16T05:33:01.897707+0000","last_clean":"2024-04-16T05:33:01.897707+0000","last_became_active":"2024-04-16T05:32:55.506672+0000","last_became_peered":"2024-04-16T05:32:55.506672+0000","last_unstale":"2024-04-16T05:33:01.897707+0000","last_undegraded":"2024-04-16T05:33:01.897707+0000","last_fullsized":"2024-04-16T05:33:01.897707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:54.579672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896257+0000","last_change":"2024-04-16T05:32:55.516987+0000","last_active":"2024-04-16T05:33:01.896257+0000","last_peered":"2024-04-16T05:33:01.896257+0000","last_clean":"2024-04-16T05:33:01.896257+0000","last_became_active":"2024-04-16T05:32:55.516654+0000","last_became_peered":"2024-04-16T05:32:55.516654+0000","last_unstale":"2024-04-16T05:33:01.896257+0000","last_undegraded":"2024-04-16T05:33:01.896257+0000","last_fullsized":"2024-04-16T05:33:01.896257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:32:42.957658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.531293+0000","last_change":"2024-04-16T05:32:55.506539+0000","last_active":"2024-04-16T05:33:01.531293+0000","last_peered":"2024-04-16T05:33:01.531293+0000","last_clean":"2024-04-16T05:33:01.531293+0000","last_became_active":"2024-04-16T05:32:55.503658+0000","last_became_peered":"2024-04-16T05:32:55.503658+0000","last_unstale":"2024-04-16T05:33:01.531293+0000","last_undegraded":"2024-04-16T05:33:01.531293+0000","last_fullsized":"2024-04-16T05:33:01.531293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:22.116213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.896604+0000","last_change":"2024-04-16T05:32:55.507363+0000","last_active":"2024-04-16T05:33:01.896604+0000","last_peered":"2024-04-16T05:33:01.896604+0000","last_clean":"2024-04-16T05:33:01.896604+0000","last_became_active":"2024-04-16T05:32:55.506923+0000","last_became_peered":"2024-04-16T05:32:55.506923+0000","last_unstale":"2024-04-16T05:33:01.896604+0000","last_undegraded":"2024-04-16T05:33:01.896604+0000","last_fullsized":"2024-04-16T05:33:01.896604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:19:34.356834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897402+0000","last_change":"2024-04-16T05:32:55.508531+0000","last_active":"2024-04-16T05:33:01.897402+0000","last_peered":"2024-04-16T05:33:01.897402+0000","last_clean":"2024-04-16T05:33:01.897402+0000","last_became_active":"2024-04-16T05:32:55.508177+0000","last_became_peered":"2024-04-16T05:32:55.508177+0000","last_unstale":"2024-04-16T05:33:01.897402+0000","last_undegraded":"2024-04-16T05:33:01.897402+0000","last_fullsized":"2024-04-16T05:33:01.897402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:49:59.352572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.530373+0000","last_change":"2024-04-16T05:32:55.507793+0000","last_active":"2024-04-16T05:33:01.530373+0000","last_peered":"2024-04-16T05:33:01.530373+0000","last_clean":"2024-04-16T05:33:01.530373+0000","last_became_active":"2024-04-16T05:32:55.507443+0000","last_became_peered":"2024-04-16T05:32:55.507443+0000","last_unstale":"2024-04-16T05:33:01.530373+0000","last_undegraded":"2024-04-16T05:33:01.530373+0000","last_fullsized":"2024-04-16T05:33:01.530373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:35.982879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T05:33:01.897524+0000","last_change":"2024-04-16T05:32:55.517057+0000","last_active":"2024-04-16T05:33:01.897524+0000","last_peered":"2024-04-16T05:33:01.897524+0000","last_clean":"2024-04-16T05:33:01.897524+0000","last_became_active":"2024-04-16T05:32:55.516696+0000","last_became_peered":"2024-04-16T05:32:55.516696+0000","last_unstale":"2024-04-16T05:33:01.897524+0000","last_undegraded":"2024-04-16T05:33:01.897524+0000","last_fullsized":"2024-04-16T05:33:01.897524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:43:10.391125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280341,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6960,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":137,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":21763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":486622,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1081026,"data_compressed":53275,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":491840,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":487360,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1078782,"data_compressed":53275,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":486712,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":490464,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":489191,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":491120,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":486816,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":486644,"data_compressed":17617,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":1077172,"data_compressed":53275,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:33:20.383 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-16T05:33:20.384 INFO:tasks.cephadm:Setting up client nodes... 2024-04-16T05:33:20.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-16T05:33:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:20.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:20 smithi086 ceph-mon[32028]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:20.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:20 smithi086 ceph-mon[32028]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:20 smithi086 ceph-mon[32028]: pgmap v304: 129 pgs: 129 active+clean; 583 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-16T05:33:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:20 smithi086 ceph-mon[32028]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-16T05:33:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:20 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1059746738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:20 smithi196 ceph-mon[36373]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:20 smithi196 ceph-mon[36373]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:20 smithi196 ceph-mon[36373]: pgmap v304: 129 pgs: 129 active+clean; 583 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-16T05:33:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:20 smithi196 ceph-mon[36373]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-16T05:33:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:20 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1059746738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:20 smithi194 ceph-mon[35865]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:20 smithi194 ceph-mon[35865]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:33:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:20 smithi194 ceph-mon[35865]: pgmap v304: 129 pgs: 129 active+clean; 583 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-16T05:33:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:20 smithi194 ceph-mon[35865]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-16T05:33:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:20 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1059746738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:21.900 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:21 smithi196 ceph-mon[36373]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:21.901 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:21 smithi196 ceph-mon[36373]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:21.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:21 smithi194 ceph-mon[35865]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:21.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:21 smithi194 ceph-mon[35865]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:21.959 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:21 smithi086 ceph-mon[32028]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:21.959 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:21 smithi086 ceph-mon[32028]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:22.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.526+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/3087245558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 msgr2=0x7f4eb8102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:22.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.526+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/3087245558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 0x7f4eb8102f90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80099c0 tx=0x7f4ea8009cd0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.526+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/3087245558 shutdown_connections 2024-04-16T05:33:22.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.526+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/3087245558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb8104710 0x7f4eb810afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.526+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/3087245558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 0x7f4eb8102f90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.526+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/3087245558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb8103d70 0x7f4eb81041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.526+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/3087245558 >> 172.21.15.86:0/3087245558 conn(0x7f4eb80fe110 msgr2=0x7f4eb8100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.526+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/3087245558 shutdown_connections 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.526+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/3087245558 wait complete. 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4ebd5f5700 1 Processor -- start 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4ebd5f5700 1 -- start start 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4ebd5f5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 0x7f4eb8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4ebd5f5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb8103d70 0x7f4eb81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4ebd5f5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb8104710 0x7f4eb81a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4ebd5f5700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810e7b0 con 0x7f4eb8102b70 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4ebd5f5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810e630 con 0x7f4eb8104710 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4ebd5f5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810e930 con 0x7f4eb8103d70 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4eb6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 0x7f4eb8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4eb6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 0x7f4eb8198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41960/0 (socket says 172.21.15.86:41960) 2024-04-16T05:33:22.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4eb6ffd700 1 -- 172.21.15.86:0/2654540182 learned_addr learned my addr 172.21.15.86:0/2654540182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.527+0000 7f4eb67fc700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb8103d70 0x7f4eb81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eb77fe700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb8104710 0x7f4eb81a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eb6ffd700 1 -- 172.21.15.86:0/2654540182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb8103d70 msgr2=0x7f4eb81991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eb6ffd700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb8103d70 0x7f4eb81991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eb6ffd700 1 -- 172.21.15.86:0/2654540182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb8104710 msgr2=0x7f4eb81a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eb6ffd700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb8104710 0x7f4eb81a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eb6ffd700 1 -- 172.21.15.86:0/2654540182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81a08c0 con 0x7f4eb8102b70 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eb77fe700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb8104710 0x7f4eb81a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eb6ffd700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 0x7f4eb8198c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c010 tx=0x7f4ea80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eaffff700 1 -- 172.21.15.86:0/2654540182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8027070 con 0x7f4eb8102b70 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eaffff700 1 -- 172.21.15.86:0/2654540182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea8004150 con 0x7f4eb8102b70 2024-04-16T05:33:22.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb81a0b50 con 0x7f4eb8102b70 2024-04-16T05:33:22.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4eaffff700 1 -- 172.21.15.86:0/2654540182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea800f460 con 0x7f4eb8102b70 2024-04-16T05:33:22.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.528+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81a1030 con 0x7f4eb8102b70 2024-04-16T05:33:22.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.529+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb8051030 con 0x7f4eb8102b70 2024-04-16T05:33:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.530+0000 7f4eaffff700 1 -- 172.21.15.86:0/2654540182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4ea802a070 con 0x7f4eb8102b70 2024-04-16T05:33:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.531+0000 7f4eaffff700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ea40a3d80 0x7f4ea40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.531+0000 7f4eb67fc700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ea40a3d80 0x7f4ea40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.531+0000 7f4eaffff700 1 -- 172.21.15.86:0/2654540182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4ea80d18f0 con 0x7f4eb8102b70 2024-04-16T05:33:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.531+0000 7f4eb67fc700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ea40a3d80 0x7f4ea40a6240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000ac50 tx=0x7f4ea000a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:22.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.535+0000 7f4eaffff700 1 -- 172.21.15.86:0/2654540182 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ea805eff0 con 0x7f4eb8102b70 2024-04-16T05:33:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.816+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f4eb802dfe0 con 0x7f4eb8102b70 2024-04-16T05:33:22.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.818+0000 7f4eaffff700 1 -- 172.21.15.86:0/2654540182 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f4ea8013070 con 0x7f4eb8102b70 2024-04-16T05:33:22.817 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-16T05:33:22.817 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCiDR5m4w7DMBAA0i5Ef35x7gmUVTB2mtqN5w== 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ea40a3d80 msgr2=0x7f4ea40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ea40a3d80 0x7f4ea40a6240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000ac50 tx=0x7f4ea000a380 comp rx=0 tx=0).stop 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 msgr2=0x7f4eb8198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 0x7f4eb8198c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c010 tx=0x7f4ea80057c0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 shutdown_connections 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb8104710 0x7f4eb81a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb8102b70 0x7f4eb8198c70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4ea40a3d80 0x7f4ea40a6240 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 --2- 172.21.15.86:0/2654540182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb8103d70 0x7f4eb81991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 >> 172.21.15.86:0/2654540182 conn(0x7f4eb80fe110 msgr2=0x7f4eb8106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 shutdown_connections 2024-04-16T05:33:22.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:22.822+0000 7f4ebd5f5700 1 -- 172.21.15.86:0/2654540182 wait complete. 2024-04-16T05:33:22.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:22 smithi194 ceph-mon[35865]: pgmap v305: 129 pgs: 129 active+clean; 628 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-04-16T05:33:22.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:22 smithi086 ceph-mon[32028]: pgmap v305: 129 pgs: 129 active+clean; 628 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-04-16T05:33:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:22 smithi196 ceph-mon[36373]: pgmap v305: 129 pgs: 129 active+clean; 628 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-04-16T05:33:23.677 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:33:23.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-16T05:33:23.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-16T05:33:23.721 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-16T05:33:23.721 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-16T05:33:23.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph mgr dump --format=json 2024-04-16T05:33:23.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:23 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2654540182' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T05:33:23.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:23 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2654540182' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T05:33:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:23 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2654540182' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T05:33:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:23 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2654540182' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T05:33:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:23 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2654540182' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T05:33:23.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:23 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2654540182' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T05:33:24.005 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:24.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:24 smithi086 ceph-mon[32028]: pgmap v306: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-04-16T05:33:24.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:24 smithi194 ceph-mon[35865]: pgmap v306: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-04-16T05:33:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:24 smithi196 ceph-mon[36373]: pgmap v306: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-04-16T05:33:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.731+0000 7f5d65767700 1 -- 172.21.15.86:0/3382858058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d60107a30 msgr2=0x7f5d60107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.731+0000 7f5d65767700 1 --2- 172.21.15.86:0/3382858058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d60107a30 0x7f5d60107e10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009990 tx=0x7f5d50009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.731+0000 7f5d65767700 1 -- 172.21.15.86:0/3382858058 shutdown_connections 2024-04-16T05:33:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.731+0000 7f5d65767700 1 --2- 172.21.15.86:0/3382858058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d601023a0 0x7f5d6010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.731+0000 7f5d65767700 1 --2- 172.21.15.86:0/3382858058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d60107a30 0x7f5d60107e10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.731+0000 7f5d65767700 1 --2- 172.21.15.86:0/3382858058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d601019e0 0x7f5d60101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.731+0000 7f5d65767700 1 -- 172.21.15.86:0/3382858058 >> 172.21.15.86:0/3382858058 conn(0x7f5d60078960 msgr2=0x7f5d60078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.732+0000 7f5d65767700 1 -- 172.21.15.86:0/3382858058 shutdown_connections 2024-04-16T05:33:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.732+0000 7f5d65767700 1 -- 172.21.15.86:0/3382858058 wait complete. 2024-04-16T05:33:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.732+0000 7f5d65767700 1 Processor -- start 2024-04-16T05:33:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.732+0000 7f5d65767700 1 -- start start 2024-04-16T05:33:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d65767700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d601019e0 0x7f5d60199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d65767700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d601023a0 0x7f5d60199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d65767700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d60107a30 0x7f5d6019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d65767700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d60112de0 con 0x7f5d601023a0 2024-04-16T05:33:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d65767700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5d60112c60 con 0x7f5d601019e0 2024-04-16T05:33:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d65767700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d60112f60 con 0x7f5d60107a30 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5e7fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d601023a0 0x7f5d60199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5e7fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d601023a0 0x7f5d60199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41972/0 (socket says 172.21.15.86:41972) 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5e7fc700 1 -- 172.21.15.86:0/642314903 learned_addr learned my addr 172.21.15.86:0/642314903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5effd700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d601019e0 0x7f5d60199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5f7fe700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d60107a30 0x7f5d6019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5e7fc700 1 -- 172.21.15.86:0/642314903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d60107a30 msgr2=0x7f5d6019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5e7fc700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d60107a30 0x7f5d6019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5e7fc700 1 -- 172.21.15.86:0/642314903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d601019e0 msgr2=0x7f5d60199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5e7fc700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d601019e0 0x7f5d60199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5e7fc700 1 -- 172.21.15.86:0/642314903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6019e240 con 0x7f5d601023a0 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d5e7fc700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d601023a0 0x7f5d60199770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d950 tx=0x7f5d5400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d47fff700 1 -- 172.21.15.86:0/642314903 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54016070 con 0x7f5d601023a0 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6019e530 con 0x7f5d601023a0 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d47fff700 1 -- 172.21.15.86:0/642314903 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5400fd70 con 0x7f5d601023a0 2024-04-16T05:33:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d47fff700 1 -- 172.21.15.86:0/642314903 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400e610 con 0x7f5d601023a0 2024-04-16T05:33:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.733+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d601a5db0 con 0x7f5d601023a0 2024-04-16T05:33:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.734+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d60109f50 con 0x7f5d601023a0 2024-04-16T05:33:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.735+0000 7f5d47fff700 1 -- 172.21.15.86:0/642314903 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d54019070 con 0x7f5d601023a0 2024-04-16T05:33:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.736+0000 7f5d47fff700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d480a3d50 0x7f5d480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.736+0000 7f5d5effd700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d480a3d50 0x7f5d480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.736+0000 7f5d47fff700 1 -- 172.21.15.86:0/642314903 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5d540d01e0 con 0x7f5d601023a0 2024-04-16T05:33:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.737+0000 7f5d5effd700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d480a3d50 0x7f5d480a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006010 tx=0x7f5d5000b560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.741+0000 7f5d47fff700 1 -- 172.21.15.86:0/642314903 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d5409c820 con 0x7f5d601023a0 2024-04-16T05:33:25.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:25.995+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f5d6019a590 con 0x7f5d601023a0 2024-04-16T05:33:25.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.000+0000 7f5d47fff700 1 -- 172.21.15.86:0/642314903 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f5d5409c120 con 0x7f5d601023a0 2024-04-16T05:33:26.003 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:26.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d480a3d50 msgr2=0x7f5d480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d480a3d50 0x7f5d480a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006010 tx=0x7f5d5000b560 comp rx=0 tx=0).stop 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d601023a0 msgr2=0x7f5d60199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d601023a0 0x7f5d60199770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d950 tx=0x7f5d5400dc60 comp rx=0 tx=0).stop 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 shutdown_connections 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d480a3d50 0x7f5d480a6210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d60107a30 0x7f5d6019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d601019e0 0x7f5d60199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 --2- 172.21.15.86:0/642314903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d601023a0 0x7f5d60199770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 >> 172.21.15.86:0/642314903 conn(0x7f5d60078960 msgr2=0x7f5d601006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 shutdown_connections 2024-04-16T05:33:26.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:26.003+0000 7f5d65767700 1 -- 172.21.15.86:0/642314903 wait complete. 2024-04-16T05:33:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:26 smithi086 ceph-mon[32028]: pgmap v307: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 23 op/s 2024-04-16T05:33:26.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:26 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/642314903' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T05:33:26.862 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3194681946},{"type":"v1","addr":"172.21.15.86:6801","nonce":3194681946}]},"active_addr":"172.21.15.86:6801/3194681946","active_change":"2024-04-16T05:25:51.394818+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.86:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2330974702}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3900523320}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":501397249}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2641972995}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1241699997}]}]} 2024-04-16T05:33:26.867 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-16T05:33:26.867 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-16T05:33:26.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd dump --format=json 2024-04-16T05:33:26.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:26 smithi194 ceph-mon[35865]: pgmap v307: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 23 op/s 2024-04-16T05:33:26.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:26 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/642314903' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T05:33:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:26 smithi196 ceph-mon[36373]: pgmap v307: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 23 op/s 2024-04-16T05:33:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:26 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/642314903' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T05:33:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:28.861 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:28 smithi086 ceph-mon[32028]: pgmap v308: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 23 op/s 2024-04-16T05:33:28.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:28 smithi194 ceph-mon[35865]: pgmap v308: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 23 op/s 2024-04-16T05:33:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.961+0000 7fa1293c8700 1 -- 172.21.15.86:0/2581957542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 msgr2=0x7fa124108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.961+0000 7fa1293c8700 1 --2- 172.21.15.86:0/2581957542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 0x7fa124108f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00b0a0 tx=0x7fa11c00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.961+0000 7fa1293c8700 1 -- 172.21.15.86:0/2581957542 shutdown_connections 2024-04-16T05:33:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.961+0000 7fa1293c8700 1 --2- 172.21.15.86:0/2581957542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa12406b1a0 0x7fa12410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.961+0000 7fa1293c8700 1 --2- 172.21.15.86:0/2581957542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 0x7fa124108f50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.961+0000 7fa1293c8700 1 --2- 172.21.15.86:0/2581957542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa12406a800 0x7fa12406ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.961+0000 7fa1293c8700 1 -- 172.21.15.86:0/2581957542 >> 172.21.15.86:0/2581957542 conn(0x7fa1240789a0 msgr2=0x7fa124078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.961+0000 7fa1293c8700 1 -- 172.21.15.86:0/2581957542 shutdown_connections 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.961+0000 7fa1293c8700 1 -- 172.21.15.86:0/2581957542 wait complete. 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1293c8700 1 Processor -- start 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1293c8700 1 -- start start 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1293c8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa12406a800 0x7fa124101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1293c8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa12406b1a0 0x7fa124102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1293c8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 0x7fa124102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1293c8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa124112dd0 con 0x7fa124108b70 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1293c8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa124112c50 con 0x7fa12406a800 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1293c8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa124112f50 con 0x7fa12406b1a0 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1237fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 0x7fa124102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1237fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 0x7fa124102540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41988/0 (socket says 172.21.15.86:41988) 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.962+0000 7fa1237fe700 1 -- 172.21.15.86:0/273951872 learned_addr learned my addr 172.21.15.86:0/273951872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1227fc700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa12406b1a0 0x7fa124102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1237fe700 1 -- 172.21.15.86:0/273951872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa12406b1a0 msgr2=0x7fa124102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa122ffd700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa12406a800 0x7fa124101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1237fe700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa12406b1a0 0x7fa124102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1237fe700 1 -- 172.21.15.86:0/273951872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa12406a800 msgr2=0x7fa124101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1237fe700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa12406a800 0x7fa124101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1237fe700 1 -- 172.21.15.86:0/273951872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1241a7790 con 0x7fa124108b70 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1227fc700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa12406b1a0 0x7fa124102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa122ffd700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa12406a800 0x7fa124101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1237fe700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 0x7fa124102540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa11400c970 tx=0x7fa11400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa11bfff700 1 -- 172.21.15.86:0/273951872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114007b00 con 0x7fa124108b70 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1241a7a80 con 0x7fa124108b70 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa11bfff700 1 -- 172.21.15.86:0/273951872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa114004d10 con 0x7fa124108b70 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa11bfff700 1 -- 172.21.15.86:0/273951872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1140056c0 con 0x7fa124108b70 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.963+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1241a8010 con 0x7fa124108b70 2024-04-16T05:33:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.964+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1241041e0 con 0x7fa124108b70 2024-04-16T05:33:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.965+0000 7fa11bfff700 1 -- 172.21.15.86:0/273951872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa114004750 con 0x7fa124108b70 2024-04-16T05:33:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.966+0000 7fa11bfff700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1100a3cd0 0x7fa1100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.966+0000 7fa122ffd700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1100a3cd0 0x7fa1100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.966+0000 7fa11bfff700 1 -- 172.21.15.86:0/273951872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa1140d0420 con 0x7fa124108b70 2024-04-16T05:33:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.966+0000 7fa122ffd700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1100a3cd0 0x7fa1100a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009e60 tx=0x7fa11c00b7b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:28.970+0000 7fa11bfff700 1 -- 172.21.15.86:0/273951872 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa114014030 con 0x7fa124108b70 2024-04-16T05:33:28.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:28 smithi196 ceph-mon[36373]: pgmap v308: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 23 op/s 2024-04-16T05:33:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.172+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa12402dfe0 con 0x7fa124108b70 2024-04-16T05:33:29.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.173+0000 7fa11bfff700 1 -- 172.21.15.86:0/273951872 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24080 (secure 0 0 0) 0x7fa11409ca00 con 0x7fa124108b70 2024-04-16T05:33:29.172 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:29.172 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:01.527267+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1100a3cd0 msgr2=0x7fa1100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1100a3cd0 0x7fa1100a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009e60 tx=0x7fa11c00b7b0 comp rx=0 tx=0).stop 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 msgr2=0x7fa124102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 0x7fa124102540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa11400c970 tx=0x7fa11400cd30 comp rx=0 tx=0).stop 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 shutdown_connections 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa124108b70 0x7fa124102540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa12406a800 0x7fa124101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1100a3cd0 0x7fa1100a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 --2- 172.21.15.86:0/273951872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa12406b1a0 0x7fa124102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 >> 172.21.15.86:0/273951872 conn(0x7fa1240789a0 msgr2=0x7fa1240ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 shutdown_connections 2024-04-16T05:33:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:29.176+0000 7fa1293c8700 1 -- 172.21.15.86:0/273951872 wait complete. 2024-04-16T05:33:29.898 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:29 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/273951872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:29.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:29 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/273951872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:29.926 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-16T05:33:29.926 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd dump --format=json 2024-04-16T05:33:29.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:29 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/273951872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:30.171 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:30 smithi194 ceph-mon[35865]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 19 op/s 2024-04-16T05:33:30.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:30 smithi086 ceph-mon[32028]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 19 op/s 2024-04-16T05:33:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:30 smithi196 ceph-mon[36373]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 19 op/s 2024-04-16T05:33:31.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:31 smithi194 ceph-mon[35865]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-04-16T05:33:31.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:31 smithi086 ceph-mon[32028]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-04-16T05:33:31.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:31 smithi196 ceph-mon[36373]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-04-16T05:33:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.996+0000 7f1c699a9700 1 -- 172.21.15.86:0/4272207622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 msgr2=0x7f1c64101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.996+0000 7f1c699a9700 1 --2- 172.21.15.86:0/4272207622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 0x7f1c64101e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009990 tx=0x7f1c58009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.996+0000 7f1c699a9700 1 -- 172.21.15.86:0/4272207622 shutdown_connections 2024-04-16T05:33:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.996+0000 7f1c699a9700 1 --2- 172.21.15.86:0/4272207622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c641023b0 0x7f1c6410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.996+0000 7f1c699a9700 1 --2- 172.21.15.86:0/4272207622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c64107a40 0x7f1c64107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.996+0000 7f1c699a9700 1 --2- 172.21.15.86:0/4272207622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 0x7f1c64101e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.996+0000 7f1c699a9700 1 -- 172.21.15.86:0/4272207622 >> 172.21.15.86:0/4272207622 conn(0x7f1c640789a0 msgr2=0x7f1c64078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.996+0000 7f1c699a9700 1 -- 172.21.15.86:0/4272207622 shutdown_connections 2024-04-16T05:33:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.996+0000 7f1c699a9700 1 -- 172.21.15.86:0/4272207622 wait complete. 2024-04-16T05:33:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c699a9700 1 Processor -- start 2024-04-16T05:33:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c699a9700 1 -- start start 2024-04-16T05:33:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c699a9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 0x7f1c6419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c699a9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c641023b0 0x7f1c6419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c699a9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c64107a40 0x7f1c641a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c699a9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c64111ce0 con 0x7f1c641019f0 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c699a9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1c64111b60 con 0x7f1c64107a40 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c699a9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c64111e60 con 0x7f1c641023b0 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c62ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 0x7f1c6419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c62ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 0x7f1c6419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36776/0 (socket says 172.21.15.86:36776) 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c62ffd700 1 -- 172.21.15.86:0/1370358411 learned_addr learned my addr 172.21.15.86:0/1370358411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c627fc700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c641023b0 0x7f1c6419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.997+0000 7f1c637fe700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c64107a40 0x7f1c641a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c62ffd700 1 -- 172.21.15.86:0/1370358411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c641023b0 msgr2=0x7f1c6419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c62ffd700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c641023b0 0x7f1c6419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c62ffd700 1 -- 172.21.15.86:0/1370358411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c64107a40 msgr2=0x7f1c641a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c62ffd700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c64107a40 0x7f1c641a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c62ffd700 1 -- 172.21.15.86:0/1370358411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c641a5740 con 0x7f1c641019f0 2024-04-16T05:33:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c62ffd700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 0x7f1c6419a180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c689a7700 1 -- 172.21.15.86:0/1370358411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54016070 con 0x7f1c641019f0 2024-04-16T05:33:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c641a5910 con 0x7f1c641019f0 2024-04-16T05:33:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c689a7700 1 -- 172.21.15.86:0/1370358411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c5400fd70 con 0x7f1c641019f0 2024-04-16T05:33:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c689a7700 1 -- 172.21.15.86:0/1370358411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400e610 con 0x7f1c641019f0 2024-04-16T05:33:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.998+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c641a5e70 con 0x7f1c641019f0 2024-04-16T05:33:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:31.999+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c640697e0 con 0x7f1c641019f0 2024-04-16T05:33:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.001+0000 7f1c689a7700 1 -- 172.21.15.86:0/1370358411 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1c54019070 con 0x7f1c641019f0 2024-04-16T05:33:32.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.002+0000 7f1c689a7700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1c4c0a3cd0 0x7f1c4c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.002+0000 7f1c627fc700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1c4c0a3cd0 0x7f1c4c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.002+0000 7f1c689a7700 1 -- 172.21.15.86:0/1370358411 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1c540d01d0 con 0x7f1c641019f0 2024-04-16T05:33:32.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.002+0000 7f1c627fc700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1c4c0a3cd0 0x7f1c4c0a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b5c0 tx=0x7f1c58005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:32.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.005+0000 7f1c689a7700 1 -- 172.21.15.86:0/1370358411 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c5409c810 con 0x7f1c641019f0 2024-04-16T05:33:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.202+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c6402dfe0 con 0x7f1c641019f0 2024-04-16T05:33:32.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.203+0000 7f1c689a7700 1 -- 172.21.15.86:0/1370358411 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24080 (secure 0 0 0) 0x7f1c5409c110 con 0x7f1c641019f0 2024-04-16T05:33:32.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:33:32.203 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:01.527267+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1c4c0a3cd0 msgr2=0x7f1c4c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1c4c0a3cd0 0x7f1c4c0a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b5c0 tx=0x7f1c58005960 comp rx=0 tx=0).stop 2024-04-16T05:33:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 msgr2=0x7f1c6419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 0x7f1c6419a180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400dc60 comp rx=0 tx=0).stop 2024-04-16T05:33:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 shutdown_connections 2024-04-16T05:33:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c64107a40 0x7f1c641a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c641019f0 0x7f1c6419a180 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f1c4c0a3cd0 0x7f1c4c0a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 --2- 172.21.15.86:0/1370358411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c641023b0 0x7f1c6419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.205+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 >> 172.21.15.86:0/1370358411 conn(0x7f1c640789a0 msgr2=0x7f1c640ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.206+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 shutdown_connections 2024-04-16T05:33:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:32.206+0000 7f1c699a9700 1 -- 172.21.15.86:0/1370358411 wait complete. 2024-04-16T05:33:32.847 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:32 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1370358411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:32.876 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-16T05:33:32.876 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-16T05:33:32.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-16T05:33:32.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-16T05:33:32.878 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-16T05:33:32.878 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-16T05:33:32.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-16T05:33:32.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-16T05:33:32.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-16T05:33:32.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-16T05:33:32.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-16T05:33:32.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-16T05:33:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:32 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1370358411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:32 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1370358411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:33.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:33 smithi086 ceph-mon[32028]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-04-16T05:33:33.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:33 smithi196 ceph-mon[36373]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-04-16T05:33:34.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:33 smithi194 ceph-mon[35865]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-04-16T05:33:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:34.975 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:34.975 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:34.975 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:34.976 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:34.976 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:35.209 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:35.210 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.518+0000 7f9368931700 1 -- 172.21.15.86:0/1153002174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93641023a0 msgr2=0x7f936410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.518+0000 7f9368931700 1 --2- 172.21.15.86:0/1153002174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93641023a0 0x7f936410e890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f935c00b0a0 tx=0x7f935c00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.518+0000 7f9368931700 1 -- 172.21.15.86:0/1153002174 shutdown_connections 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.518+0000 7f9368931700 1 --2- 172.21.15.86:0/1153002174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93641023a0 0x7f936410e890 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.518+0000 7f9368931700 1 --2- 172.21.15.86:0/1153002174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9364107a30 0x7f9364107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.518+0000 7f9368931700 1 --2- 172.21.15.86:0/1153002174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93641019e0 0x7f9364101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.518+0000 7f9368931700 1 -- 172.21.15.86:0/1153002174 >> 172.21.15.86:0/1153002174 conn(0x7f9364078960 msgr2=0x7f9364078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.518+0000 7f9368931700 1 -- 172.21.15.86:0/1153002174 shutdown_connections 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.518+0000 7f9368931700 1 -- 172.21.15.86:0/1153002174 wait complete. 2024-04-16T05:33:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f9368931700 1 Processor -- start 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f9368931700 1 -- start start 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f9368931700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93641019e0 0x7f9364199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f9368931700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93641023a0 0x7f9364199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f9368931700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9364107a30 0x7f936419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f9368931700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9364112e40 con 0x7f93641019e0 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f9368931700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9364112cc0 con 0x7f93641023a0 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f9368931700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9364112fc0 con 0x7f9364107a30 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f936259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93641019e0 0x7f9364199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f936259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93641019e0 0x7f9364199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36788/0 (socket says 172.21.15.86:36788) 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f936259c700 1 -- 172.21.15.86:0/138789530 learned_addr learned my addr 172.21.15.86:0/138789530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.519+0000 7f9361d9b700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93641023a0 0x7f9364199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f9362d9d700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9364107a30 0x7f936419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f936259c700 1 -- 172.21.15.86:0/138789530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9364107a30 msgr2=0x7f936419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f936259c700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9364107a30 0x7f936419dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f936259c700 1 -- 172.21.15.86:0/138789530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93641023a0 msgr2=0x7f9364199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f936259c700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93641023a0 0x7f9364199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f936259c700 1 -- 172.21.15.86:0/138789530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936419e220 con 0x7f93641019e0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f936259c700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93641019e0 0x7f9364199210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f935400e9a0 tx=0x7f935400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f93537fe700 1 -- 172.21.15.86:0/138789530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935400ca10 con 0x7f93641019e0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936419e510 con 0x7f93641019e0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f93537fe700 1 -- 172.21.15.86:0/138789530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f935400cbb0 con 0x7f93641019e0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f93537fe700 1 -- 172.21.15.86:0/138789530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354018680 con 0x7f93641019e0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.520+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93641a5d90 con 0x7f93641019e0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.521+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9364063200 con 0x7f93641019e0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.522+0000 7f93537fe700 1 -- 172.21.15.86:0/138789530 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9354010430 con 0x7f93641019e0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.523+0000 7f93537fe700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f934c0a3cd0 0x7f934c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.523+0000 7f9361d9b700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f934c0a3cd0 0x7f934c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.523+0000 7f93537fe700 1 -- 172.21.15.86:0/138789530 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f93540d1140 con 0x7f93641019e0 2024-04-16T05:33:35.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.523+0000 7f93537fe700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] conn(0x7f934c0ae990 0x7f934c0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.523+0000 7f93537fe700 1 -- 172.21.15.86:0/138789530 --> [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f934c0b14a0 con 0x7f934c0ae990 2024-04-16T05:33:35.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.523+0000 7f93537fe700 1 -- 172.21.15.86:0/138789530 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f93540d1590 con 0x7f93641019e0 2024-04-16T05:33:35.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.523+0000 7f9361d9b700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f934c0a3cd0 0x7f934c0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9364078fd0 tx=0x7f9358009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:35.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.523+0000 7f9362d9d700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] conn(0x7f934c0ae990 0x7f934c0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.524+0000 7f9362d9d700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] conn(0x7f934c0ae990 0x7f934c0b0dd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:35.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.525+0000 7f93537fe700 1 -- 172.21.15.86:0/138789530 <== osd.5 v2:172.21.15.196:6808/90234472 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f934c0b14a0 con 0x7f934c0ae990 2024-04-16T05:33:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.541+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 --> [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9364063600 con 0x7f934c0ae990 2024-04-16T05:33:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f93537fe700 1 -- 172.21.15.86:0/138789530 <== osd.5 v2:172.21.15.196:6808/90234472 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9364063600 con 0x7f934c0ae990 2024-04-16T05:33:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 >> [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] conn(0x7f934c0ae990 msgr2=0x7f934c0b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] conn(0x7f934c0ae990 0x7f934c0b0dd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f934c0a3cd0 msgr2=0x7f934c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f934c0a3cd0 0x7f934c0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9364078fd0 tx=0x7f9358009260 comp rx=0 tx=0).stop 2024-04-16T05:33:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93641019e0 msgr2=0x7f9364199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 --2- 172.21.15.86:0/138789530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93641019e0 0x7f9364199210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f935400e9a0 tx=0x7f935400ecb0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9362d9d700 1 -- 172.21.15.86:0/138789530 reap_dead start 2024-04-16T05:33:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 shutdown_connections 2024-04-16T05:33:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 >> 172.21.15.86:0/138789530 conn(0x7f9364078960 msgr2=0x7f93640ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 shutdown_connections 2024-04-16T05:33:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:35.542+0000 7f9368931700 1 -- 172.21.15.86:0/138789530 wait complete. 2024-04-16T05:33:36.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:36 smithi086 ceph-mon[32028]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-16T05:33:36.742 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:36.858 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:36 smithi194 ceph-mon[35865]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-16T05:33:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:36 smithi196 ceph-mon[36373]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-16T05:33:38.172 INFO:teuthology.orchestra.run.smithi086.stdout:163208757300 2024-04-16T05:33:38.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-16T05:33:38.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:38 smithi086 ceph-mon[32028]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-16T05:33:38.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:38 smithi194 ceph-mon[35865]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-16T05:33:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:38 smithi196 ceph-mon[36373]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-16T05:33:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:40 smithi194 ceph-mon[35865]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-16T05:33:40.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:40 smithi086 ceph-mon[32028]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-16T05:33:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:40 smithi196 ceph-mon[36373]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-16T05:33:42.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:42 smithi086 ceph-mon[32028]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-16T05:33:42.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:42 smithi194 ceph-mon[35865]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-16T05:33:42.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:42 smithi196 ceph-mon[36373]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-16T05:33:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.459+0000 7fe0e1a65700 1 -- 172.21.15.86:0/1372734524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc108b60 msgr2=0x7fe0dc108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.459+0000 7fe0e1a65700 1 --2- 172.21.15.86:0/1372734524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc108b60 0x7fe0dc108f40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009990 tx=0x7fe0cc009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.459+0000 7fe0e1a65700 1 -- 172.21.15.86:0/1372734524 shutdown_connections 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.459+0000 7fe0e1a65700 1 --2- 172.21.15.86:0/1372734524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0dc06ba30 0x7fe0dc10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.459+0000 7fe0e1a65700 1 --2- 172.21.15.86:0/1372734524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc108b60 0x7fe0dc108f40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.459+0000 7fe0e1a65700 1 --2- 172.21.15.86:0/1372734524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0dc06b090 0x7fe0dc06b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.459+0000 7fe0e1a65700 1 -- 172.21.15.86:0/1372734524 >> 172.21.15.86:0/1372734524 conn(0x7fe0dc078960 msgr2=0x7fe0dc078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.459+0000 7fe0e1a65700 1 -- 172.21.15.86:0/1372734524 shutdown_connections 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.459+0000 7fe0e1a65700 1 -- 172.21.15.86:0/1372734524 wait complete. 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.460+0000 7fe0e1a65700 1 Processor -- start 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.460+0000 7fe0e1a65700 1 -- start start 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.460+0000 7fe0e1a65700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0dc06b090 0x7fe0dc101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0e1a65700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0dc06ba30 0x7fe0dc101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0e1a65700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc108b60 0x7fe0dc102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0e1a65700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc112db0 con 0x7fe0dc108b60 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0e1a65700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc112c30 con 0x7fe0dc06ba30 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0e1a65700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc112f30 con 0x7fe0dc06b090 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0dbfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc108b60 0x7fe0dc102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0dbfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc108b60 0x7fe0dc102530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58602/0 (socket says 172.21.15.86:58602) 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0dbfff700 1 -- 172.21.15.86:0/2160429373 learned_addr learned my addr 172.21.15.86:0/2160429373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0daffd700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0dc06ba30 0x7fe0dc101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0db7fe700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0dc06b090 0x7fe0dc101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0dbfff700 1 -- 172.21.15.86:0/2160429373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0dc06b090 msgr2=0x7fe0dc101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0dbfff700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0dc06b090 0x7fe0dc101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0dbfff700 1 -- 172.21.15.86:0/2160429373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0dc06ba30 msgr2=0x7fe0dc101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0dbfff700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0dc06ba30 0x7fe0dc101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0dbfff700 1 -- 172.21.15.86:0/2160429373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc1a7850 con 0x7fe0dc108b60 2024-04-16T05:33:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0dbfff700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc108b60 0x7fe0dc102530 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400ef30 tx=0x7fe0d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.461+0000 7fe0d8ff9700 1 -- 172.21.15.86:0/2160429373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40092e0 con 0x7fe0dc108b60 2024-04-16T05:33:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.462+0000 7fe0d8ff9700 1 -- 172.21.15.86:0/2160429373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0d4004bb0 con 0x7fe0dc108b60 2024-04-16T05:33:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.462+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc1a7ae0 con 0x7fe0dc108b60 2024-04-16T05:33:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.462+0000 7fe0d8ff9700 1 -- 172.21.15.86:0/2160429373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4004280 con 0x7fe0dc108b60 2024-04-16T05:33:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.462+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc1a7fd0 con 0x7fe0dc108b60 2024-04-16T05:33:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.463+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0dc063180 con 0x7fe0dc108b60 2024-04-16T05:33:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.464+0000 7fe0d8ff9700 1 -- 172.21.15.86:0/2160429373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe0d401b030 con 0x7fe0dc108b60 2024-04-16T05:33:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.465+0000 7fe0d8ff9700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0c40a3cd0 0x7fe0c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.465+0000 7fe0d8ff9700 1 -- 172.21.15.86:0/2160429373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe0d40d0ec0 con 0x7fe0dc108b60 2024-04-16T05:33:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.465+0000 7fe0db7fe700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0c40a3cd0 0x7fe0c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.465+0000 7fe0d8ff9700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] conn(0x7fe0c40ae990 0x7fe0c40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.465+0000 7fe0d8ff9700 1 -- 172.21.15.86:0/2160429373 --> [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0c40b14a0 con 0x7fe0c40ae990 2024-04-16T05:33:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.465+0000 7fe0d8ff9700 1 -- 172.21.15.86:0/2160429373 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe0d40d1310 con 0x7fe0dc108b60 2024-04-16T05:33:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.465+0000 7fe0daffd700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] conn(0x7fe0c40ae990 0x7fe0c40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.465+0000 7fe0db7fe700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0c40a3cd0 0x7fe0c40a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00b2e0 tx=0x7fe0cc005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.466+0000 7fe0daffd700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] conn(0x7fe0c40ae990 0x7fe0c40b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.467+0000 7fe0d8ff9700 1 -- 172.21.15.86:0/2160429373 <== osd.4 v2:172.21.15.194:6808/3924919713 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0c40b14a0 con 0x7fe0c40ae990 2024-04-16T05:33:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.483+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 --> [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe0dc0635b0 con 0x7fe0c40ae990 2024-04-16T05:33:44.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.483+0000 7fe0d8ff9700 1 -- 172.21.15.86:0/2160429373 <== osd.4 v2:172.21.15.194:6808/3924919713 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe0dc0635b0 con 0x7fe0c40ae990 2024-04-16T05:33:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 >> [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] conn(0x7fe0c40ae990 msgr2=0x7fe0c40b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] conn(0x7fe0c40ae990 0x7fe0c40b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0c40a3cd0 msgr2=0x7fe0c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0c40a3cd0 0x7fe0c40a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00b2e0 tx=0x7fe0cc005960 comp rx=0 tx=0).stop 2024-04-16T05:33:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc108b60 msgr2=0x7fe0dc102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 --2- 172.21.15.86:0/2160429373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc108b60 0x7fe0dc102530 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400ef30 tx=0x7fe0d400c3a0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0dbfff700 1 -- 172.21.15.86:0/2160429373 reap_dead start 2024-04-16T05:33:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 shutdown_connections 2024-04-16T05:33:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 >> 172.21.15.86:0/2160429373 conn(0x7fe0dc078960 msgr2=0x7fe0dc0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:44.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 shutdown_connections 2024-04-16T05:33:44.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.484+0000 7fe0e1a65700 1 -- 172.21.15.86:0/2160429373 wait complete. 2024-04-16T05:33:44.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:44 smithi086 ceph-mon[32028]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:33:44.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:44 smithi194 ceph-mon[35865]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:33:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:44 smithi196 ceph-mon[36373]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:33:45.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.355+0000 7f5d9606a700 1 -- 172.21.15.86:0/2348352737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d90107a30 msgr2=0x7f5d90107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.355+0000 7f5d9606a700 1 --2- 172.21.15.86:0/2348352737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d90107a30 0x7f5d90107e10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009990 tx=0x7f5d80009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.355+0000 7f5d9606a700 1 -- 172.21.15.86:0/2348352737 shutdown_connections 2024-04-16T05:33:45.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.355+0000 7f5d9606a700 1 --2- 172.21.15.86:0/2348352737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d901023a0 0x7f5d9010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.355+0000 7f5d9606a700 1 --2- 172.21.15.86:0/2348352737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d90107a30 0x7f5d90107e10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.355+0000 7f5d9606a700 1 --2- 172.21.15.86:0/2348352737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d901019e0 0x7f5d90101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.355+0000 7f5d9606a700 1 -- 172.21.15.86:0/2348352737 >> 172.21.15.86:0/2348352737 conn(0x7f5d90078960 msgr2=0x7f5d90078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.355+0000 7f5d9606a700 1 -- 172.21.15.86:0/2348352737 shutdown_connections 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.355+0000 7f5d9606a700 1 -- 172.21.15.86:0/2348352737 wait complete. 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.356+0000 7f5d9606a700 1 Processor -- start 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d9606a700 1 -- start start 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d9606a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d901019e0 0x7f5d90199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d9606a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d901023a0 0x7f5d901998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d9606a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d90107a30 0x7f5d9019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d9606a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90111d20 con 0x7f5d901019e0 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d9606a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90111ba0 con 0x7f5d90107a30 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d9606a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90111ea0 con 0x7f5d901023a0 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d901019e0 0x7f5d90199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:45.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d901019e0 0x7f5d90199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58590/0 (socket says 172.21.15.86:58590) 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8f7fe700 1 -- 172.21.15.86:0/2960924628 learned_addr learned my addr 172.21.15.86:0/2960924628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8effd700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d901023a0 0x7f5d901998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8ffff700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d90107a30 0x7f5d9019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8f7fe700 1 -- 172.21.15.86:0/2960924628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d901023a0 msgr2=0x7f5d901998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8f7fe700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d901023a0 0x7f5d901998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8f7fe700 1 -- 172.21.15.86:0/2960924628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d90107a30 msgr2=0x7f5d9019dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8f7fe700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d90107a30 0x7f5d9019dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8f7fe700 1 -- 172.21.15.86:0/2960924628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9019e390 con 0x7f5d901019e0 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.357+0000 7f5d8f7fe700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d901019e0 0x7f5d90199380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d80005910 tx=0x7f5d800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.358+0000 7f5d8cff9700 1 -- 172.21.15.86:0/2960924628 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80027070 con 0x7f5d901019e0 2024-04-16T05:33:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.358+0000 7f5d8cff9700 1 -- 172.21.15.86:0/2960924628 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d80004050 con 0x7f5d901019e0 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.358+0000 7f5d8cff9700 1 -- 172.21.15.86:0/2960924628 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000f460 con 0x7f5d901019e0 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.358+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9019e620 con 0x7f5d901019e0 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.358+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d901a5f00 con 0x7f5d901019e0 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.359+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d901006a0 con 0x7f5d901019e0 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.360+0000 7f5d8cff9700 1 -- 172.21.15.86:0/2960924628 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d8002a070 con 0x7f5d901019e0 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.361+0000 7f5d8cff9700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d780a3cd0 0x7f5d780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.361+0000 7f5d8effd700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d780a3cd0 0x7f5d780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.361+0000 7f5d8cff9700 1 -- 172.21.15.86:0/2960924628 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5d800d21a0 con 0x7f5d901019e0 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.362+0000 7f5d8cff9700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] conn(0x7f5d780ae990 0x7f5d780b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:45.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.362+0000 7f5d8cff9700 1 -- 172.21.15.86:0/2960924628 --> [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d780b14a0 con 0x7f5d780ae990 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.362+0000 7f5d8cff9700 1 -- 172.21.15.86:0/2960924628 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5d800d2550 con 0x7f5d901019e0 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.362+0000 7f5d8effd700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d780a3cd0 0x7f5d780a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d9006c0b0 tx=0x7f5d8400a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.362+0000 7f5d8ffff700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] conn(0x7f5d780ae990 0x7f5d780b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.362+0000 7f5d8ffff700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] conn(0x7f5d780ae990 0x7f5d780b0dd0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.363+0000 7f5d8cff9700 1 -- 172.21.15.86:0/2960924628 <== osd.11 v2:172.21.15.196:6824/2166568999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d780b14a0 con 0x7f5d780ae990 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.380+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 --> [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d90103100 con 0x7f5d780ae990 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.380+0000 7f5d8cff9700 1 -- 172.21.15.86:0/2960924628 <== osd.11 v2:172.21.15.196:6824/2166568999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5d90103100 con 0x7f5d780ae990 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.380+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 >> [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] conn(0x7f5d780ae990 msgr2=0x7f5d780b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.380+0000 7f5d9606a700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] conn(0x7f5d780ae990 0x7f5d780b0dd0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.381+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d780a3cd0 msgr2=0x7f5d780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.381+0000 7f5d9606a700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5d780a3cd0 0x7f5d780a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d9006c0b0 tx=0x7f5d8400a300 comp rx=0 tx=0).stop 2024-04-16T05:33:45.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.381+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d901019e0 msgr2=0x7f5d90199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.381+0000 7f5d9606a700 1 --2- 172.21.15.86:0/2960924628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d901019e0 0x7f5d90199380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d80005910 tx=0x7f5d800047b0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.381+0000 7f5d8ffff700 1 -- 172.21.15.86:0/2960924628 reap_dead start 2024-04-16T05:33:45.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.381+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 shutdown_connections 2024-04-16T05:33:45.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.381+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 >> 172.21.15.86:0/2960924628 conn(0x7f5d90078960 msgr2=0x7f5d90105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:45.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.381+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 shutdown_connections 2024-04-16T05:33:45.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:44.381+0000 7f5d9606a700 1 -- 172.21.15.86:0/2960924628 wait complete. 2024-04-16T05:33:46.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:46 smithi086 ceph-mon[32028]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-16T05:33:46.652 INFO:teuthology.orchestra.run.smithi086.stdout:141733920827 2024-04-16T05:33:46.652 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-16T05:33:46.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:46 smithi194 ceph-mon[35865]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-16T05:33:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:46 smithi196 ceph-mon[36373]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-16T05:33:48.747 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:48 smithi086 ceph-mon[32028]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:33:48.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:48 smithi194 ceph-mon[35865]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:33:48.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:48 smithi196 ceph-mon[36373]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:33:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.161+0000 7f6a5765b700 1 -- 172.21.15.86:0/3225786652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a50107ab0 msgr2=0x7f6a50107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.161+0000 7f6a5765b700 1 --2- 172.21.15.86:0/3225786652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a50107ab0 0x7f6a50107e90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009990 tx=0x7f6a4c009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.161+0000 7f6a5765b700 1 -- 172.21.15.86:0/3225786652 shutdown_connections 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.161+0000 7f6a5765b700 1 --2- 172.21.15.86:0/3225786652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50103540 0x7f6a5010fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.161+0000 7f6a5765b700 1 --2- 172.21.15.86:0/3225786652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a50107ab0 0x7f6a50107e90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.161+0000 7f6a5765b700 1 --2- 172.21.15.86:0/3225786652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50102ba0 0x7f6a50103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.161+0000 7f6a5765b700 1 -- 172.21.15.86:0/3225786652 >> 172.21.15.86:0/3225786652 conn(0x7f6a50079c50 msgr2=0x7f6a5007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.161+0000 7f6a5765b700 1 -- 172.21.15.86:0/3225786652 shutdown_connections 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.162+0000 7f6a5765b700 1 -- 172.21.15.86:0/3225786652 wait complete. 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a5765b700 1 Processor -- start 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a5765b700 1 -- start start 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a5765b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50102ba0 0x7f6a50199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a5765b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50103540 0x7f6a50199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a5765b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a50107ab0 0x7f6a5019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a5765b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a50112f20 con 0x7f6a50107ab0 2024-04-16T05:33:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a5765b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6a50112da0 con 0x7f6a50102ba0 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a5765b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501130a0 con 0x7f6a50103540 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a55bf8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a50107ab0 0x7f6a5019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a55bf8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a50107ab0 0x7f6a5019dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58618/0 (socket says 172.21.15.86:58618) 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a55bf8700 1 -- 172.21.15.86:0/2872731479 learned_addr learned my addr 172.21.15.86:0/2872731479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a553f7700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50102ba0 0x7f6a50199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a54bf6700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50103540 0x7f6a50199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a55bf8700 1 -- 172.21.15.86:0/2872731479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50103540 msgr2=0x7f6a50199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a55bf8700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50103540 0x7f6a50199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a55bf8700 1 -- 172.21.15.86:0/2872731479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50102ba0 msgr2=0x7f6a50199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a55bf8700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50102ba0 0x7f6a50199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.163+0000 7f6a55bf8700 1 -- 172.21.15.86:0/2872731479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5019e420 con 0x7f6a50107ab0 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.164+0000 7f6a55bf8700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a50107ab0 0x7f6a5019dce0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b9e0 tx=0x7f6a4800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.164+0000 7f6a467fc700 1 -- 172.21.15.86:0/2872731479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800d8e0 con 0x7f6a50107ab0 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.164+0000 7f6a467fc700 1 -- 172.21.15.86:0/2872731479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a48015420 con 0x7f6a50107ab0 2024-04-16T05:33:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.164+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5019e710 con 0x7f6a50107ab0 2024-04-16T05:33:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.164+0000 7f6a467fc700 1 -- 172.21.15.86:0/2872731479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48014630 con 0x7f6a50107ab0 2024-04-16T05:33:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.164+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a501a5f90 con 0x7f6a50107ab0 2024-04-16T05:33:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.165+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a50063200 con 0x7f6a50107ab0 2024-04-16T05:33:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.167+0000 7f6a467fc700 1 -- 172.21.15.86:0/2872731479 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6a48015940 con 0x7f6a50107ab0 2024-04-16T05:33:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.168+0000 7f6a467fc700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a3c0a3d50 0x7f6a3c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.168+0000 7f6a467fc700 1 -- 172.21.15.86:0/2872731479 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6a480d0bb0 con 0x7f6a50107ab0 2024-04-16T05:33:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.168+0000 7f6a553f7700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a3c0a3d50 0x7f6a3c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.168+0000 7f6a467fc700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] conn(0x7f6a3c0aea10 0x7f6a3c0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.168+0000 7f6a467fc700 1 -- 172.21.15.86:0/2872731479 --> [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a48006c30 con 0x7f6a3c0aea10 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.168+0000 7f6a467fc700 1 -- 172.21.15.86:0/2872731479 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f6a480d1000 con 0x7f6a50107ab0 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.168+0000 7f6a553f7700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a3c0a3d50 0x7f6a3c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c006010 tx=0x7f6a4c005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.169+0000 7f6a54bf6700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] conn(0x7f6a3c0aea10 0x7f6a3c0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.169+0000 7f6a54bf6700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] conn(0x7f6a3c0aea10 0x7f6a3c0b0e50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.170+0000 7f6a467fc700 1 -- 172.21.15.86:0/2872731479 <== osd.8 v2:172.21.15.196:6816/3684108941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a48006c30 con 0x7f6a3c0aea10 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.186+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 --> [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6a50063680 con 0x7f6a3c0aea10 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a467fc700 1 -- 172.21.15.86:0/2872731479 <== osd.8 v2:172.21.15.196:6816/3684108941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6a50063680 con 0x7f6a3c0aea10 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 >> [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] conn(0x7f6a3c0aea10 msgr2=0x7f6a3c0b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] conn(0x7f6a3c0aea10 0x7f6a3c0b0e50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a3c0a3d50 msgr2=0x7f6a3c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6a3c0a3d50 0x7f6a3c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c006010 tx=0x7f6a4c005960 comp rx=0 tx=0).stop 2024-04-16T05:33:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a50107ab0 msgr2=0x7f6a5019dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 --2- 172.21.15.86:0/2872731479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a50107ab0 0x7f6a5019dce0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b9e0 tx=0x7f6a4800bcf0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a55bf8700 1 -- 172.21.15.86:0/2872731479 reap_dead start 2024-04-16T05:33:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 shutdown_connections 2024-04-16T05:33:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 >> 172.21.15.86:0/2872731479 conn(0x7f6a50079c50 msgr2=0x7f6a50100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 shutdown_connections 2024-04-16T05:33:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:50.187+0000 7f6a5765b700 1 -- 172.21.15.86:0/2872731479 wait complete. 2024-04-16T05:33:50.813 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:50 smithi086 ceph-mon[32028]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:33:50.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:50 smithi194 ceph-mon[35865]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:33:50.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:50 smithi196 ceph-mon[36373]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:33:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.268+0000 7efeeb8bf700 1 -- 172.21.15.86:0/3527169529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee406bc20 msgr2=0x7efee410f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.268+0000 7efeeb8bf700 1 --2- 172.21.15.86:0/3527169529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee406bc20 0x7efee410f9c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efedc00b0a0 tx=0x7efedc00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.268+0000 7efeeb8bf700 1 -- 172.21.15.86:0/3527169529 shutdown_connections 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.268+0000 7efeeb8bf700 1 --2- 172.21.15.86:0/3527169529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee406bc20 0x7efee410f9c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.268+0000 7efeeb8bf700 1 --2- 172.21.15.86:0/3527169529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee4108bd0 0x7efee4108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.268+0000 7efeeb8bf700 1 --2- 172.21.15.86:0/3527169529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee406b260 0x7efee406b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.268+0000 7efeeb8bf700 1 -- 172.21.15.86:0/3527169529 >> 172.21.15.86:0/3527169529 conn(0x7efee4077a40 msgr2=0x7efee4077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.268+0000 7efeeb8bf700 1 -- 172.21.15.86:0/3527169529 shutdown_connections 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.268+0000 7efeeb8bf700 1 -- 172.21.15.86:0/3527169529 wait complete. 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efeeb8bf700 1 Processor -- start 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efeeb8bf700 1 -- start start 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efeeb8bf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee406b260 0x7efee4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efeeb8bf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee406bc20 0x7efee4199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efeeb8bf700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4108bd0 0x7efee419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efeeb8bf700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efee4112e90 con 0x7efee4108bd0 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efeeb8bf700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efee4112d10 con 0x7efee406bc20 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efeeb8bf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efee4113010 con 0x7efee406b260 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee9e5c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4108bd0 0x7efee419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee9e5c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4108bd0 0x7efee419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33294/0 (socket says 172.21.15.86:33294) 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee9e5c700 1 -- 172.21.15.86:0/755282921 learned_addr learned my addr 172.21.15.86:0/755282921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee8e5a700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee406bc20 0x7efee4199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee965b700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee406b260 0x7efee4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee9e5c700 1 -- 172.21.15.86:0/755282921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee406b260 msgr2=0x7efee4199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee9e5c700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee406b260 0x7efee4199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee9e5c700 1 -- 172.21.15.86:0/755282921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee406bc20 msgr2=0x7efee4199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee9e5c700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee406bc20 0x7efee4199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.269+0000 7efee9e5c700 1 -- 172.21.15.86:0/755282921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee419e210 con 0x7efee4108bd0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.270+0000 7efee9e5c700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4108bd0 0x7efee419dad0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efedc00b780 tx=0x7efedc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.270+0000 7efeda7fc700 1 -- 172.21.15.86:0/755282921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc021070 con 0x7efee4108bd0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.270+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee419e4a0 con 0x7efee4108bd0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.270+0000 7efeda7fc700 1 -- 172.21.15.86:0/755282921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efedc007b80 con 0x7efee4108bd0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.270+0000 7efeda7fc700 1 -- 172.21.15.86:0/755282921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc005370 con 0x7efee4108bd0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.270+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41a5d80 con 0x7efee4108bd0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.270+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efee4100f10 con 0x7efee4108bd0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.272+0000 7efeda7fc700 1 -- 172.21.15.86:0/755282921 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efedc024070 con 0x7efee4108bd0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.272+0000 7efeda7fc700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7efed00a3cd0 0x7efed00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.273+0000 7efee965b700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7efed00a3cd0 0x7efed00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.273+0000 7efeda7fc700 1 -- 172.21.15.86:0/755282921 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7efedc0d1890 con 0x7efee4108bd0 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.273+0000 7efeda7fc700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] conn(0x7efed00ae990 0x7efed00b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.273+0000 7efeda7fc700 1 -- 172.21.15.86:0/755282921 --> [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efedc00c910 con 0x7efed00ae990 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.273+0000 7efeda7fc700 1 -- 172.21.15.86:0/755282921 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7efedc0d1ce0 con 0x7efee4108bd0 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.273+0000 7efee965b700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7efed00a3cd0 0x7efed00a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efee0006fd0 tx=0x7efee0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.273+0000 7efee8e5a700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] conn(0x7efed00ae990 0x7efed00b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.274+0000 7efee8e5a700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] conn(0x7efed00ae990 0x7efed00b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.275+0000 7efeda7fc700 1 -- 172.21.15.86:0/755282921 <== osd.1 v2:172.21.15.194:6800/2032185507 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efedc00c910 con 0x7efed00ae990 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.291+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 --> [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efee41041f0 con 0x7efed00ae990 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.291+0000 7efeda7fc700 1 -- 172.21.15.86:0/755282921 <== osd.1 v2:172.21.15.194:6800/2032185507 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efee41041f0 con 0x7efed00ae990 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.291+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 >> [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] conn(0x7efed00ae990 msgr2=0x7efed00b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.291+0000 7efeeb8bf700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] conn(0x7efed00ae990 0x7efed00b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.292+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7efed00a3cd0 msgr2=0x7efed00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.292+0000 7efeeb8bf700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7efed00a3cd0 0x7efed00a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efee0006fd0 tx=0x7efee0008040 comp rx=0 tx=0).stop 2024-04-16T05:33:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.292+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4108bd0 msgr2=0x7efee419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.292+0000 7efeeb8bf700 1 --2- 172.21.15.86:0/755282921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4108bd0 0x7efee419dad0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efedc00b780 tx=0x7efedc004d00 comp rx=0 tx=0).stop 2024-04-16T05:33:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.292+0000 7efee9e5c700 1 -- 172.21.15.86:0/755282921 reap_dead start 2024-04-16T05:33:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.292+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 shutdown_connections 2024-04-16T05:33:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.292+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 >> 172.21.15.86:0/755282921 conn(0x7efee4077a40 msgr2=0x7efee41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.292+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 shutdown_connections 2024-04-16T05:33:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.292+0000 7efeeb8bf700 1 -- 172.21.15.86:0/755282921 wait complete. 2024-04-16T05:33:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-16T05:33:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-16T05:33:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-16T05:33:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 7]}]: dispatch 2024-04-16T05:33:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-16T05:33:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-16T05:33:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-16T05:33:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-16T05:33:51.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-16T05:33:51.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2024-04-16T05:33:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-16T05:33:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-16T05:33:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-16T05:33:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 7]}]: dispatch 2024-04-16T05:33:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-16T05:33:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-16T05:33:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-16T05:33:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-16T05:33:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-16T05:33:51.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2024-04-16T05:33:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-16T05:33:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-16T05:33:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-16T05:33:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 7]}]: dispatch 2024-04-16T05:33:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-16T05:33:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-16T05:33:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-16T05:33:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-16T05:33:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-16T05:33:51.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2024-04-16T05:33:52.316 INFO:teuthology.orchestra.run.smithi086.stdout:343597383702 2024-04-16T05:33:52.316 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-16T05:33:52.757 INFO:teuthology.orchestra.run.smithi086.stdout:55834574923 2024-04-16T05:33:52.757 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 -- 172.21.15.86:0/4281991508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e8101180 msgr2=0x7f01e8101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 --2- 172.21.15.86:0/4281991508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e8101180 0x7f01e8101600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009990 tx=0x7f01d8009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 -- 172.21.15.86:0/4281991508 shutdown_connections 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 --2- 172.21.15.86:0/4281991508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01e8101b40 0x7f01e810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 --2- 172.21.15.86:0/4281991508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01e8107230 0x7f01e8107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 --2- 172.21.15.86:0/4281991508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e8101180 0x7f01e8101600 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 -- 172.21.15.86:0/4281991508 >> 172.21.15.86:0/4281991508 conn(0x7f01e8078090 msgr2=0x7f01e80ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 -- 172.21.15.86:0/4281991508 shutdown_connections 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 -- 172.21.15.86:0/4281991508 wait complete. 2024-04-16T05:33:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 Processor -- start 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.402+0000 7f01eff90700 1 -- start start 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01eff90700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01e8101180 0x7f01e8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01eff90700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e8101b40 0x7f01e8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01eff90700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01e8107230 0x7f01e819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01eff90700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8110bd0 con 0x7f01e8101b40 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01eff90700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8110a50 con 0x7f01e8107230 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01eff90700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8110d50 con 0x7f01e8101180 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ed52b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e8101b40 0x7f01e8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ed52b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e8101b40 0x7f01e8199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33322/0 (socket says 172.21.15.86:33322) 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ed52b700 1 -- 172.21.15.86:0/3757020383 learned_addr learned my addr 172.21.15.86:0/3757020383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01edd2c700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01e8101180 0x7f01e8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ee52d700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01e8107230 0x7f01e819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ed52b700 1 -- 172.21.15.86:0/3757020383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01e8101180 msgr2=0x7f01e8199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ed52b700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01e8101180 0x7f01e8199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ed52b700 1 -- 172.21.15.86:0/3757020383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01e8107230 msgr2=0x7f01e819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ed52b700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01e8107230 0x7f01e819db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ed52b700 1 -- 172.21.15.86:0/3757020383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e819e240 con 0x7f01e8101b40 2024-04-16T05:33:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01ed52b700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e8101b40 0x7f01e8199770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f01d8000c00 tx=0x7f01d800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01deffd700 1 -- 172.21.15.86:0/3757020383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8027070 con 0x7f01e8101b40 2024-04-16T05:33:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01deffd700 1 -- 172.21.15.86:0/3757020383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d8003d70 con 0x7f01e8101b40 2024-04-16T05:33:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e819e4d0 con 0x7f01e8101b40 2024-04-16T05:33:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01deffd700 1 -- 172.21.15.86:0/3757020383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d800f6c0 con 0x7f01e8101b40 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.403+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e81a5db0 con 0x7f01e8101b40 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.404+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01e8063280 con 0x7f01e8101b40 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.405+0000 7f01deffd700 1 -- 172.21.15.86:0/3757020383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f01d802a070 con 0x7f01e8101b40 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.406+0000 7f01deffd700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01d40a3cd0 0x7f01d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.406+0000 7f01edd2c700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01d40a3cd0 0x7f01d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.406+0000 7f01deffd700 1 -- 172.21.15.86:0/3757020383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f01d80d1f60 con 0x7f01e8101b40 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.406+0000 7f01deffd700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] conn(0x7f01d40ae990 0x7f01d40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.406+0000 7f01deffd700 1 -- 172.21.15.86:0/3757020383 --> [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01d80043d0 con 0x7f01d40ae990 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.406+0000 7f01deffd700 1 -- 172.21.15.86:0/3757020383 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f01d80d23b0 con 0x7f01e8101b40 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.406+0000 7f01edd2c700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01d40a3cd0 0x7f01d40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01e400acd0 tx=0x7f01e4010040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.406+0000 7f01ee52d700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] conn(0x7f01d40ae990 0x7f01d40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.407+0000 7f01ee52d700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] conn(0x7f01d40ae990 0x7f01d40b0dd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.407+0000 7f01deffd700 1 -- 172.21.15.86:0/3757020383 <== osd.6 v2:172.21.15.86:6818/983213207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01d80043d0 con 0x7f01d40ae990 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.424+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 --> [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01e8063700 con 0x7f01d40ae990 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.424+0000 7f01deffd700 1 -- 172.21.15.86:0/3757020383 <== osd.6 v2:172.21.15.86:6818/983213207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f01e8063700 con 0x7f01d40ae990 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.424+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] conn(0x7f01d40ae990 msgr2=0x7f01d40b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.424+0000 7f01eff90700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] conn(0x7f01d40ae990 0x7f01d40b0dd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.424+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01d40a3cd0 msgr2=0x7f01d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.424+0000 7f01eff90700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01d40a3cd0 0x7f01d40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01e400acd0 tx=0x7f01e4010040 comp rx=0 tx=0).stop 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.424+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e8101b40 msgr2=0x7f01e8199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.424+0000 7f01eff90700 1 --2- 172.21.15.86:0/3757020383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e8101b40 0x7f01e8199770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f01d8000c00 tx=0x7f01d800bab0 comp rx=0 tx=0).stop 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.424+0000 7f01ee52d700 1 -- 172.21.15.86:0/3757020383 reap_dead start 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.425+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 shutdown_connections 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.425+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 >> 172.21.15.86:0/3757020383 conn(0x7f01e8078090 msgr2=0x7f01e80ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.425+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 shutdown_connections 2024-04-16T05:33:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:51.425+0000 7f01eff90700 1 -- 172.21.15.86:0/3757020383 wait complete. 2024-04-16T05:33:52.848 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 7]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2024-04-16T05:33:52.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:52 smithi086 ceph-mon[32028]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T05:33:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:33:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-16T05:33:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-16T05:33:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-16T05:33:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 7]}]': finished 2024-04-16T05:33:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-16T05:33:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-16T05:33:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-16T05:33:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-16T05:33:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-16T05:33:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2024-04-16T05:33:52.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:52 smithi194 ceph-mon[35865]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T05:33:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:33:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-16T05:33:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-16T05:33:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-16T05:33:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 7]}]': finished 2024-04-16T05:33:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-16T05:33:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-16T05:33:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-16T05:33:52.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-16T05:33:52.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-16T05:33:52.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2024-04-16T05:33:52.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:52 smithi196 ceph-mon[36373]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T05:33:53.601 INFO:teuthology.orchestra.run.smithi086.stdout:253403070504 2024-04-16T05:33:53.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-16T05:33:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.242+0000 7fd256385700 1 -- 172.21.15.86:0/68377030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd250107230 msgr2=0x7fd250107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.242+0000 7fd256385700 1 --2- 172.21.15.86:0/68377030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd250107230 0x7fd250107610 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd240009990 tx=0x7fd240009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.242+0000 7fd256385700 1 -- 172.21.15.86:0/68377030 shutdown_connections 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.242+0000 7fd256385700 1 --2- 172.21.15.86:0/68377030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd250101b40 0x7fd25010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.242+0000 7fd256385700 1 --2- 172.21.15.86:0/68377030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd250107230 0x7fd250107610 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.242+0000 7fd256385700 1 --2- 172.21.15.86:0/68377030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd250101180 0x7fd250101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.242+0000 7fd256385700 1 -- 172.21.15.86:0/68377030 >> 172.21.15.86:0/68377030 conn(0x7fd250078090 msgr2=0x7fd2500ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.242+0000 7fd256385700 1 -- 172.21.15.86:0/68377030 shutdown_connections 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.242+0000 7fd256385700 1 -- 172.21.15.86:0/68377030 wait complete. 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd256385700 1 Processor -- start 2024-04-16T05:33:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd256385700 1 -- start start 2024-04-16T05:33:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd256385700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd250101180 0x7fd2501991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd256385700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd250101b40 0x7fd250199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd256385700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd250107230 0x7fd25019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd256385700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd250110bb0 con 0x7fd250101180 2024-04-16T05:33:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd256385700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd250110a30 con 0x7fd250107230 2024-04-16T05:33:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd256385700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd250110d30 con 0x7fd250101b40 2024-04-16T05:33:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd24ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd250101180 0x7fd2501991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd24ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd250101180 0x7fd2501991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33340/0 (socket says 172.21.15.86:33340) 2024-04-16T05:33:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd24ffff700 1 -- 172.21.15.86:0/4259408316 learned_addr learned my addr 172.21.15.86:0/4259408316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.243+0000 7fd254922700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd250107230 0x7fd25019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24f7fe700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd250101b40 0x7fd250199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24ffff700 1 -- 172.21.15.86:0/4259408316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd250101b40 msgr2=0x7fd250199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24ffff700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd250101b40 0x7fd250199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24ffff700 1 -- 172.21.15.86:0/4259408316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd250107230 msgr2=0x7fd25019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24ffff700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd250107230 0x7fd25019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24ffff700 1 -- 172.21.15.86:0/4259408316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25019e1f0 con 0x7fd250101180 2024-04-16T05:33:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24ffff700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd250101180 0x7fd2501991e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd2400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24d7fa700 1 -- 172.21.15.86:0/4259408316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240027070 con 0x7fd250101180 2024-04-16T05:33:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24d7fa700 1 -- 172.21.15.86:0/4259408316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd240003780 con 0x7fd250101180 2024-04-16T05:33:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25019e480 con 0x7fd250101180 2024-04-16T05:33:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd24d7fa700 1 -- 172.21.15.86:0/4259408316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240022dd0 con 0x7fd250101180 2024-04-16T05:33:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.244+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2501a5d60 con 0x7fd250101180 2024-04-16T05:33:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.245+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd250063280 con 0x7fd250101180 2024-04-16T05:33:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.246+0000 7fd24d7fa700 1 -- 172.21.15.86:0/4259408316 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd24002a070 con 0x7fd250101180 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.247+0000 7fd24d7fa700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd2380a3cd0 0x7fd2380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.247+0000 7fd24d7fa700 1 -- 172.21.15.86:0/4259408316 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd2400d1ba0 con 0x7fd250101180 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.247+0000 7fd24f7fe700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd2380a3cd0 0x7fd2380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.247+0000 7fd24d7fa700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] conn(0x7fd2380aee30 0x7fd2380b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.247+0000 7fd24d7fa700 1 -- 172.21.15.86:0/4259408316 --> [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2400043d0 con 0x7fd2380aee30 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.247+0000 7fd24d7fa700 1 -- 172.21.15.86:0/4259408316 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fd2400d1ff0 con 0x7fd250101180 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.248+0000 7fd24f7fe700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd2380a3cd0 0x7fd2380a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd244005ea0 tx=0x7fd244005e30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.248+0000 7fd254922700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] conn(0x7fd2380aee30 0x7fd2380b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.248+0000 7fd254922700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] conn(0x7fd2380aee30 0x7fd2380b12d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.249+0000 7fd24d7fa700 1 -- 172.21.15.86:0/4259408316 <== osd.10 v2:172.21.15.194:6824/656449752 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2400043d0 con 0x7fd2380aee30 2024-04-16T05:33:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.265+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 --> [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd250063700 con 0x7fd2380aee30 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd24d7fa700 1 -- 172.21.15.86:0/4259408316 <== osd.10 v2:172.21.15.194:6824/656449752 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd250063700 con 0x7fd2380aee30 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 >> [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] conn(0x7fd2380aee30 msgr2=0x7fd2380b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] conn(0x7fd2380aee30 0x7fd2380b12d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd2380a3cd0 msgr2=0x7fd2380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd2380a3cd0 0x7fd2380a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd244005ea0 tx=0x7fd244005e30 comp rx=0 tx=0).stop 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd250101180 msgr2=0x7fd2501991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 --2- 172.21.15.86:0/4259408316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd250101180 0x7fd2501991e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd2400047b0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd254922700 1 -- 172.21.15.86:0/4259408316 reap_dead start 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 shutdown_connections 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 >> 172.21.15.86:0/4259408316 conn(0x7fd250078090 msgr2=0x7fd250105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 shutdown_connections 2024-04-16T05:33:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:53.266+0000 7fd256385700 1 -- 172.21.15.86:0/4259408316 wait complete. 2024-04-16T05:33:53.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:53 smithi194 ceph-mon[35865]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T05:33:53.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:53 smithi086 ceph-mon[32028]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T05:33:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:53 smithi196 ceph-mon[36373]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T05:33:54.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.333+0000 7f69ff15c700 1 -- 172.21.15.86:0/3122502940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f81023c0 msgr2=0x7f69f810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:54.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.333+0000 7f69ff15c700 1 --2- 172.21.15.86:0/3122502940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f81023c0 0x7f69f810e8a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f69f000b0a0 tx=0x7f69f000b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:54.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.333+0000 7f69ff15c700 1 -- 172.21.15.86:0/3122502940 shutdown_connections 2024-04-16T05:33:54.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.333+0000 7f69ff15c700 1 --2- 172.21.15.86:0/3122502940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f81023c0 0x7f69f810e8a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:54.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.333+0000 7f69ff15c700 1 --2- 172.21.15.86:0/3122502940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f81079d0 0x7f69f8107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:54.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.333+0000 7f69ff15c700 1 --2- 172.21.15.86:0/3122502940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69f8101a20 0x7f69f8101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:54.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.333+0000 7f69ff15c700 1 -- 172.21.15.86:0/3122502940 >> 172.21.15.86:0/3122502940 conn(0x7f69f8078990 msgr2=0x7f69f8078da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:54.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.333+0000 7f69ff15c700 1 -- 172.21.15.86:0/3122502940 shutdown_connections 2024-04-16T05:33:54.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.333+0000 7f69ff15c700 1 -- 172.21.15.86:0/3122502940 wait complete. 2024-04-16T05:33:54.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69ff15c700 1 Processor -- start 2024-04-16T05:33:54.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69ff15c700 1 -- start start 2024-04-16T05:33:54.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69ff15c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f8101a20 0x7f69f8198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:54.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69ff15c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f81023c0 0x7f69f81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:54.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69ff15c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69f81079d0 0x7f69f819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:54.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69ff15c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8111d60 con 0x7f69f81023c0 2024-04-16T05:33:54.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69ff15c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8111be0 con 0x7f69f8101a20 2024-04-16T05:33:54.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69ff15c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8111ee0 con 0x7f69f81079d0 2024-04-16T05:33:54.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69f7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f81023c0 0x7f69f81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:54.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69f7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f81023c0 0x7f69f81986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33358/0 (socket says 172.21.15.86:33358) 2024-04-16T05:33:54.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69f7fff700 1 -- 172.21.15.86:0/1598589513 learned_addr learned my addr 172.21.15.86:0/1598589513 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:54.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69fd6f9700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69f81079d0 0x7f69f819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:54.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.334+0000 7f69fcef8700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f8101a20 0x7f69f8198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:54.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69f7fff700 1 -- 172.21.15.86:0/1598589513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69f81079d0 msgr2=0x7f69f819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:54.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69f7fff700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69f81079d0 0x7f69f819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:54.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69f7fff700 1 -- 172.21.15.86:0/1598589513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f8101a20 msgr2=0x7f69f8198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:54.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69f7fff700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f8101a20 0x7f69f8198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:54.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69f7fff700 1 -- 172.21.15.86:0/1598589513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f819d180 con 0x7f69f81023c0 2024-04-16T05:33:54.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69f7fff700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f81023c0 0x7f69f81986b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f69e800b530 tx=0x7f69e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:54.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69f5ffb700 1 -- 172.21.15.86:0/1598589513 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8015020 con 0x7f69f81023c0 2024-04-16T05:33:54.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69f5ffb700 1 -- 172.21.15.86:0/1598589513 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e8011d10 con 0x7f69f81023c0 2024-04-16T05:33:54.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f819d470 con 0x7f69f81023c0 2024-04-16T05:33:54.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69f5ffb700 1 -- 172.21.15.86:0/1598589513 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e800f2e0 con 0x7f69f81023c0 2024-04-16T05:33:54.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.335+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81a4cf0 con 0x7f69f81023c0 2024-04-16T05:33:54.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.336+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69f8063280 con 0x7f69f81023c0 2024-04-16T05:33:54.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.337+0000 7f69f5ffb700 1 -- 172.21.15.86:0/1598589513 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f69e8018020 con 0x7f69f81023c0 2024-04-16T05:33:54.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.338+0000 7f69f5ffb700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69e40a3cd0 0x7f69e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:54.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.338+0000 7f69fcef8700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69e40a3cd0 0x7f69e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:54.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.338+0000 7f69f5ffb700 1 -- 172.21.15.86:0/1598589513 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f69e80d0470 con 0x7f69f81023c0 2024-04-16T05:33:54.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.338+0000 7f69f5ffb700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] conn(0x7f69e40aee30 0x7f69e40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:54.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.338+0000 7f69f5ffb700 1 -- 172.21.15.86:0/1598589513 --> [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69e40b19a0 con 0x7f69e40aee30 2024-04-16T05:33:54.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.338+0000 7f69f5ffb700 1 -- 172.21.15.86:0/1598589513 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f69e80d08c0 con 0x7f69f81023c0 2024-04-16T05:33:54.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.339+0000 7f69fd6f9700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] conn(0x7f69e40aee30 0x7f69e40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:54.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.339+0000 7f69fcef8700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69e40a3cd0 0x7f69e40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f69e0007be0 tx=0x7f69e00073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:54.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.339+0000 7f69fd6f9700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] conn(0x7f69e40aee30 0x7f69e40b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:54.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.340+0000 7f69f5ffb700 1 -- 172.21.15.86:0/1598589513 <== osd.7 v2:172.21.15.194:6816/1878006717 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69e40b19a0 con 0x7f69e40aee30 2024-04-16T05:33:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.356+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 --> [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f69f8063700 con 0x7f69e40aee30 2024-04-16T05:33:54.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.357+0000 7f69f5ffb700 1 -- 172.21.15.86:0/1598589513 <== osd.7 v2:172.21.15.194:6816/1878006717 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f69f8063700 con 0x7f69e40aee30 2024-04-16T05:33:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 >> [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] conn(0x7f69e40aee30 msgr2=0x7f69e40b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] conn(0x7f69e40aee30 0x7f69e40b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69e40a3cd0 msgr2=0x7f69e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f69e40a3cd0 0x7f69e40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f69e0007be0 tx=0x7f69e00073d0 comp rx=0 tx=0).stop 2024-04-16T05:33:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f81023c0 msgr2=0x7f69f81986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 --2- 172.21.15.86:0/1598589513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f81023c0 0x7f69f81986b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f69e800b530 tx=0x7f69e800b840 comp rx=0 tx=0).stop 2024-04-16T05:33:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69fd6f9700 1 -- 172.21.15.86:0/1598589513 reap_dead start 2024-04-16T05:33:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 shutdown_connections 2024-04-16T05:33:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 >> 172.21.15.86:0/1598589513 conn(0x7f69f8078990 msgr2=0x7f69f80ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 shutdown_connections 2024-04-16T05:33:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:54.358+0000 7f69ff15c700 1 -- 172.21.15.86:0/1598589513 wait complete. 2024-04-16T05:33:54.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:54 smithi196 ceph-mon[36373]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-16T05:33:54.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:54 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:33:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:54 smithi194 ceph-mon[35865]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-16T05:33:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:54 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:33:54.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:54 smithi086 ceph-mon[32028]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-16T05:33:54.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:54 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:33:55.009 INFO:teuthology.orchestra.run.smithi086.stdout:219043332141 2024-04-16T05:33:55.009 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-16T05:33:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.200+0000 7f26c89f1700 1 -- 172.21.15.86:0/1194657325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c406bb80 msgr2=0x7f26c410f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.200+0000 7f26c89f1700 1 --2- 172.21.15.86:0/1194657325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c406bb80 0x7f26c410f9b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00b0a0 tx=0x7f26bc00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.200+0000 7f26c89f1700 1 -- 172.21.15.86:0/1194657325 shutdown_connections 2024-04-16T05:33:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.200+0000 7f26c89f1700 1 --2- 172.21.15.86:0/1194657325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c406bb80 0x7f26c410f9b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.200+0000 7f26c89f1700 1 --2- 172.21.15.86:0/1194657325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c4108b90 0x7f26c4108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.200+0000 7f26c89f1700 1 --2- 172.21.15.86:0/1194657325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26c406b1c0 0x7f26c406b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:55.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.200+0000 7f26c89f1700 1 -- 172.21.15.86:0/1194657325 >> 172.21.15.86:0/1194657325 conn(0x7f26c4078a30 msgr2=0x7f26c4078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:55.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.200+0000 7f26c89f1700 1 -- 172.21.15.86:0/1194657325 shutdown_connections 2024-04-16T05:33:55.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.200+0000 7f26c89f1700 1 -- 172.21.15.86:0/1194657325 wait complete. 2024-04-16T05:33:55.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.201+0000 7f26c89f1700 1 Processor -- start 2024-04-16T05:33:55.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.201+0000 7f26c89f1700 1 -- start start 2024-04-16T05:33:55.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c89f1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c406b1c0 0x7f26c41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:55.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c89f1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26c406bb80 0x7f26c41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:55.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c89f1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c4108b90 0x7f26c419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:55.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c89f1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4112e70 con 0x7f26c406b1c0 2024-04-16T05:33:55.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c89f1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4112cf0 con 0x7f26c406bb80 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c89f1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4112ff0 con 0x7f26c4108b90 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c406b1c0 0x7f26c41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c406b1c0 0x7f26c41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33376/0 (socket says 172.21.15.86:33376) 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c1d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26c406bb80 0x7f26c41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c259c700 1 -- 172.21.15.86:0/2986735326 learned_addr learned my addr 172.21.15.86:0/2986735326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c2d9d700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c4108b90 0x7f26c419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c259c700 1 -- 172.21.15.86:0/2986735326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c4108b90 msgr2=0x7f26c419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c259c700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c4108b90 0x7f26c419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c259c700 1 -- 172.21.15.86:0/2986735326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26c406bb80 msgr2=0x7f26c41996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c259c700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26c406bb80 0x7f26c41996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c259c700 1 -- 172.21.15.86:0/2986735326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c419e1b0 con 0x7f26c406b1c0 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c1d9b700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26c406bb80 0x7f26c41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:33:55.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.202+0000 7f26c259c700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c406b1c0 0x7f26c41991a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f26b400c9b0 tx=0x7f26b400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:55.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.203+0000 7f26b37fe700 1 -- 172.21.15.86:0/2986735326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b401e070 con 0x7f26c406b1c0 2024-04-16T05:33:55.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.203+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c419e4a0 con 0x7f26c406b1c0 2024-04-16T05:33:55.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.203+0000 7f26b37fe700 1 -- 172.21.15.86:0/2986735326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b400ce50 con 0x7f26c406b1c0 2024-04-16T05:33:55.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.203+0000 7f26b37fe700 1 -- 172.21.15.86:0/2986735326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b40197c0 con 0x7f26c406b1c0 2024-04-16T05:33:55.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.203+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c4101a20 con 0x7f26c406b1c0 2024-04-16T05:33:55.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.204+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f26c4100ed0 con 0x7f26c406b1c0 2024-04-16T05:33:55.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.205+0000 7f26b37fe700 1 -- 172.21.15.86:0/2986735326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f26b40076f0 con 0x7f26c406b1c0 2024-04-16T05:33:55.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.206+0000 7f26b37fe700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f26ac0a3cd0 0x7f26ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:55.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.206+0000 7f26c1d9b700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f26ac0a3cd0 0x7f26ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:55.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.206+0000 7f26b37fe700 1 -- 172.21.15.86:0/2986735326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f26b40d42b0 con 0x7f26c406b1c0 2024-04-16T05:33:55.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.206+0000 7f26b37fe700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] conn(0x7f26ac0aee30 0x7f26ac0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.206+0000 7f26b37fe700 1 -- 172.21.15.86:0/2986735326 --> [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26b4014ec0 con 0x7f26ac0aee30 2024-04-16T05:33:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.206+0000 7f26b37fe700 1 -- 172.21.15.86:0/2986735326 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f26b40d4700 con 0x7f26c406b1c0 2024-04-16T05:33:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.206+0000 7f26c2d9d700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] conn(0x7f26ac0aee30 0x7f26ac0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.206+0000 7f26c1d9b700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f26ac0a3cd0 0x7f26ac0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26c4078f50 tx=0x7f26b80073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.207+0000 7f26c2d9d700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] conn(0x7f26ac0aee30 0x7f26ac0b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.207+0000 7f26b37fe700 1 -- 172.21.15.86:0/2986735326 <== osd.9 v2:172.21.15.86:6826/423267579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f26b4014ec0 con 0x7f26ac0aee30 2024-04-16T05:33:55.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.225+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 --> [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f26c41041e0 con 0x7f26ac0aee30 2024-04-16T05:33:55.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.225+0000 7f26b37fe700 1 -- 172.21.15.86:0/2986735326 <== osd.9 v2:172.21.15.86:6826/423267579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f26c41041e0 con 0x7f26ac0aee30 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] conn(0x7f26ac0aee30 msgr2=0x7f26ac0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] conn(0x7f26ac0aee30 0x7f26ac0b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f26ac0a3cd0 msgr2=0x7f26ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f26ac0a3cd0 0x7f26ac0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26c4078f50 tx=0x7f26b80073d0 comp rx=0 tx=0).stop 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c406b1c0 msgr2=0x7f26c41991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 --2- 172.21.15.86:0/2986735326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c406b1c0 0x7f26c41991a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f26b400c9b0 tx=0x7f26b400ccc0 comp rx=0 tx=0).stop 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c2d9d700 1 -- 172.21.15.86:0/2986735326 reap_dead start 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 shutdown_connections 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 >> 172.21.15.86:0/2986735326 conn(0x7f26c4078a30 msgr2=0x7f26c4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 shutdown_connections 2024-04-16T05:33:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:55.226+0000 7f26c89f1700 1 -- 172.21.15.86:0/2986735326 wait complete. 2024-04-16T05:33:55.971 INFO:teuthology.orchestra.run.smithi086.stdout:197568495665 2024-04-16T05:33:55.972 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-16T05:33:56.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.628 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.628 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:56 smithi086 ceph-mon[32028]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-16T05:33:56.628 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.628 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:56 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:56 smithi196 ceph-mon[36373]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-16T05:33:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:56 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:56 smithi194 ceph-mon[35865]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-16T05:33:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:56 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:33:56.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.984+0000 7fb40eb13700 1 -- 172.21.15.86:0/2019951929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb408108b90 msgr2=0x7fb408108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:56.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.984+0000 7fb40eb13700 1 --2- 172.21.15.86:0/2019951929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb408108b90 0x7fb408108f70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009990 tx=0x7fb3fc009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:56.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.984+0000 7fb40eb13700 1 -- 172.21.15.86:0/2019951929 shutdown_connections 2024-04-16T05:33:56.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.984+0000 7fb40eb13700 1 --2- 172.21.15.86:0/2019951929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb408103500 0x7fb40810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:56.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.984+0000 7fb40eb13700 1 --2- 172.21.15.86:0/2019951929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb408108b90 0x7fb408108f70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.984+0000 7fb40eb13700 1 --2- 172.21.15.86:0/2019951929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb408102b40 0x7fb408102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.984+0000 7fb40eb13700 1 -- 172.21.15.86:0/2019951929 >> 172.21.15.86:0/2019951929 conn(0x7fb408079c50 msgr2=0x7fb40807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.984+0000 7fb40eb13700 1 -- 172.21.15.86:0/2019951929 shutdown_connections 2024-04-16T05:33:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.984+0000 7fb40eb13700 1 -- 172.21.15.86:0/2019951929 wait complete. 2024-04-16T05:33:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40eb13700 1 Processor -- start 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40eb13700 1 -- start start 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40eb13700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb408102b40 0x7fb408199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40eb13700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb408103500 0x7fb4081998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40eb13700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb408108b90 0x7fb40819dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40eb13700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb408112e80 con 0x7fb408108b90 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40eb13700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb408112d00 con 0x7fb408102b40 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40eb13700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb408113000 con 0x7fb408103500 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40d0b0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb408108b90 0x7fb40819dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40d0b0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb408108b90 0x7fb40819dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33402/0 (socket says 172.21.15.86:33402) 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb40d0b0700 1 -- 172.21.15.86:0/3954132921 learned_addr learned my addr 172.21.15.86:0/3954132921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.985+0000 7fb407fff700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb408103500 0x7fb4081998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb40c8af700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb408102b40 0x7fb408199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb40d0b0700 1 -- 172.21.15.86:0/3954132921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb408103500 msgr2=0x7fb4081998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb40d0b0700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb408103500 0x7fb4081998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb40d0b0700 1 -- 172.21.15.86:0/3954132921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb408102b40 msgr2=0x7fb408199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb40d0b0700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb408102b40 0x7fb408199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb40d0b0700 1 -- 172.21.15.86:0/3954132921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40819e380 con 0x7fb408108b90 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb40d0b0700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb408108b90 0x7fb40819dc40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb400007ee0 tx=0x7fb40000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb405ffb700 1 -- 172.21.15.86:0/3954132921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40000dbd0 con 0x7fb408108b90 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb405ffb700 1 -- 172.21.15.86:0/3954132921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40000dd70 con 0x7fb408108b90 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40819e670 con 0x7fb408108b90 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb405ffb700 1 -- 172.21.15.86:0/3954132921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400005a60 con 0x7fb408108b90 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.986+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4081a5ef0 con 0x7fb408108b90 2024-04-16T05:33:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.987+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4080631c0 con 0x7fb408108b90 2024-04-16T05:33:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.988+0000 7fb405ffb700 1 -- 172.21.15.86:0/3954132921 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb40001a020 con 0x7fb408108b90 2024-04-16T05:33:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.989+0000 7fb405ffb700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb3f80a3d50 0x7fb3f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.989+0000 7fb40c8af700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb3f80a3d50 0x7fb3f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.989+0000 7fb405ffb700 1 -- 172.21.15.86:0/3954132921 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb4000d0680 con 0x7fb408108b90 2024-04-16T05:33:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.989+0000 7fb405ffb700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] conn(0x7fb3f80aeeb0 0x7fb3f80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.989+0000 7fb405ffb700 1 -- 172.21.15.86:0/3954132921 --> [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb400012ec0 con 0x7fb3f80aeeb0 2024-04-16T05:33:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.989+0000 7fb405ffb700 1 -- 172.21.15.86:0/3954132921 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb4000d0ad0 con 0x7fb408108b90 2024-04-16T05:33:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.989+0000 7fb407fff700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] conn(0x7fb3f80aeeb0 0x7fb3f80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.989+0000 7fb40c8af700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb3f80a3d50 0x7fb3f80a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc006010 tx=0x7fb3fc00b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.990+0000 7fb407fff700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] conn(0x7fb3f80aeeb0 0x7fb3f80b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:56.990+0000 7fb405ffb700 1 -- 172.21.15.86:0/3954132921 <== osd.3 v2:172.21.15.86:6810/439908484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb400012ec0 con 0x7fb3f80aeeb0 2024-04-16T05:33:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.006+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 --> [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4080635c0 con 0x7fb3f80aeeb0 2024-04-16T05:33:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb405ffb700 1 -- 172.21.15.86:0/3954132921 <== osd.3 v2:172.21.15.86:6810/439908484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4080635c0 con 0x7fb3f80aeeb0 2024-04-16T05:33:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] conn(0x7fb3f80aeeb0 msgr2=0x7fb3f80b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] conn(0x7fb3f80aeeb0 0x7fb3f80b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb3f80a3d50 msgr2=0x7fb3f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb3f80a3d50 0x7fb3f80a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc006010 tx=0x7fb3fc00b540 comp rx=0 tx=0).stop 2024-04-16T05:33:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb408108b90 msgr2=0x7fb40819dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:57.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 --2- 172.21.15.86:0/3954132921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb408108b90 0x7fb40819dc40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb400007ee0 tx=0x7fb40000d3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40d0b0700 1 -- 172.21.15.86:0/3954132921 reap_dead start 2024-04-16T05:33:57.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 shutdown_connections 2024-04-16T05:33:57.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 >> 172.21.15.86:0/3954132921 conn(0x7fb408079c50 msgr2=0x7fb408101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:57.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 shutdown_connections 2024-04-16T05:33:57.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.007+0000 7fb40eb13700 1 -- 172.21.15.86:0/3954132921 wait complete. 2024-04-16T05:33:57.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.223+0000 7f12f38fb700 1 -- 172.21.15.86:0/353508543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ec102bb0 msgr2=0x7f12ec10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:57.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.223+0000 7f12f38fb700 1 --2- 172.21.15.86:0/353508543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ec102bb0 0x7f12ec10e8a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f12e400b0a0 tx=0x7f12e400b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.223+0000 7f12f38fb700 1 -- 172.21.15.86:0/353508543 shutdown_connections 2024-04-16T05:33:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.223+0000 7f12f38fb700 1 --2- 172.21.15.86:0/353508543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ec102bb0 0x7f12ec10e8a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.223+0000 7f12f38fb700 1 --2- 172.21.15.86:0/353508543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ec107a60 0x7f12ec107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.223+0000 7f12f38fb700 1 --2- 172.21.15.86:0/353508543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ec1021f0 0x7f12ec102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.223+0000 7f12f38fb700 1 -- 172.21.15.86:0/353508543 >> 172.21.15.86:0/353508543 conn(0x7f12ec078090 msgr2=0x7f12ec0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.223+0000 7f12f38fb700 1 -- 172.21.15.86:0/353508543 shutdown_connections 2024-04-16T05:33:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.223+0000 7f12f38fb700 1 -- 172.21.15.86:0/353508543 wait complete. 2024-04-16T05:33:57.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.224+0000 7f12f38fb700 1 Processor -- start 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.224+0000 7f12f38fb700 1 -- start start 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.224+0000 7f12f38fb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ec1021f0 0x7f12ec1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.224+0000 7f12f38fb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ec102bb0 0x7f12ec199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.224+0000 7f12f38fb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ec107a60 0x7f12ec19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.224+0000 7f12f38fb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec111d00 con 0x7f12ec1021f0 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.224+0000 7f12f38fb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec111b80 con 0x7f12ec102bb0 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.224+0000 7f12f38fb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec111e80 con 0x7f12ec107a60 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1697700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ec1021f0 0x7f12ec1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1697700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ec1021f0 0x7f12ec1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33412/0 (socket says 172.21.15.86:33412) 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1697700 1 -- 172.21.15.86:0/2144433846 learned_addr learned my addr 172.21.15.86:0/2144433846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1e98700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ec107a60 0x7f12ec19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f0e96700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ec102bb0 0x7f12ec199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1697700 1 -- 172.21.15.86:0/2144433846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ec107a60 msgr2=0x7f12ec19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1697700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12ec107a60 0x7f12ec19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1697700 1 -- 172.21.15.86:0/2144433846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ec102bb0 msgr2=0x7f12ec199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1697700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ec102bb0 0x7f12ec199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1697700 1 -- 172.21.15.86:0/2144433846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ec19e200 con 0x7f12ec1021f0 2024-04-16T05:33:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f0e96700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12ec102bb0 0x7f12ec199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f1697700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ec1021f0 0x7f12ec1991f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c970 tx=0x7f12e800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12e27fc700 1 -- 172.21.15.86:0/2144433846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8007aa0 con 0x7f12ec1021f0 2024-04-16T05:33:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ec19e4f0 con 0x7f12ec1021f0 2024-04-16T05:33:57.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12e27fc700 1 -- 172.21.15.86:0/2144433846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e8004d10 con 0x7f12ec1021f0 2024-04-16T05:33:57.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12e27fc700 1 -- 172.21.15.86:0/2144433846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e800f450 con 0x7f12ec1021f0 2024-04-16T05:33:57.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.225+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ec1a5d70 con 0x7f12ec1021f0 2024-04-16T05:33:57.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.226+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f12ec1013b0 con 0x7f12ec1021f0 2024-04-16T05:33:57.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.228+0000 7f12e27fc700 1 -- 172.21.15.86:0/2144433846 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f12e8004750 con 0x7f12ec1021f0 2024-04-16T05:33:57.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.228+0000 7f12e27fc700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12d80a3d50 0x7f12d80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:57.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.229+0000 7f12f0e96700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12d80a3d50 0x7f12d80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.229+0000 7f12e27fc700 1 -- 172.21.15.86:0/2144433846 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f12e80d0a20 con 0x7f12ec1021f0 2024-04-16T05:33:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.229+0000 7f12e27fc700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] conn(0x7f12d80aeeb0 0x7f12d80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.229+0000 7f12e27fc700 1 -- 172.21.15.86:0/2144433846 --> [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12e801de50 con 0x7f12d80aeeb0 2024-04-16T05:33:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.229+0000 7f12e27fc700 1 -- 172.21.15.86:0/2144433846 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f12e80d0e70 con 0x7f12ec1021f0 2024-04-16T05:33:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.229+0000 7f12f1e98700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] conn(0x7f12d80aeeb0 0x7f12d80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.229+0000 7f12f0e96700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12d80a3d50 0x7f12d80a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f12dc005950 tx=0x7f12dc0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.229+0000 7f12f1e98700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] conn(0x7f12d80aeeb0 0x7f12d80b1350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.230+0000 7f12e27fc700 1 -- 172.21.15.86:0/2144433846 <== osd.2 v2:172.21.15.196:6800/4091288332 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12e801de50 con 0x7f12d80aeeb0 2024-04-16T05:33:57.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.252+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 --> [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f12ec103200 con 0x7f12d80aeeb0 2024-04-16T05:33:57.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12e27fc700 1 -- 172.21.15.86:0/2144433846 <== osd.2 v2:172.21.15.196:6800/4091288332 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f12ec103200 con 0x7f12d80aeeb0 2024-04-16T05:33:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 >> [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] conn(0x7f12d80aeeb0 msgr2=0x7f12d80b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12f38fb700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] conn(0x7f12d80aeeb0 0x7f12d80b1350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12d80a3d50 msgr2=0x7f12d80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12f38fb700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f12d80a3d50 0x7f12d80a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f12dc005950 tx=0x7f12dc0058e0 comp rx=0 tx=0).stop 2024-04-16T05:33:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ec1021f0 msgr2=0x7f12ec1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12f38fb700 1 --2- 172.21.15.86:0/2144433846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12ec1021f0 0x7f12ec1991f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c970 tx=0x7f12e800cc80 comp rx=0 tx=0).stop 2024-04-16T05:33:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12f1e98700 1 -- 172.21.15.86:0/2144433846 reap_dead start 2024-04-16T05:33:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 shutdown_connections 2024-04-16T05:33:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.253+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 >> 172.21.15.86:0/2144433846 conn(0x7f12ec078090 msgr2=0x7f12ec105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.254+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 shutdown_connections 2024-04-16T05:33:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:33:57.254+0000 7f12f38fb700 1 -- 172.21.15.86:0/2144433846 wait complete. 2024-04-16T05:33:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:309237645342 2024-04-16T05:33:58.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-16T05:33:58.591 INFO:teuthology.orchestra.run.smithi086.stdout:77309411400 2024-04-16T05:33:58.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-16T05:33:58.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:33:58 smithi086 ceph-mon[32028]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 12 op/s 2024-04-16T05:33:58.879 INFO:teuthology.orchestra.run.smithi086.stdout:287762808868 2024-04-16T05:33:58.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-16T05:33:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:33:58 smithi194 ceph-mon[35865]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 12 op/s 2024-04-16T05:33:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:33:58 smithi196 ceph-mon[36373]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 12 op/s 2024-04-16T05:33:59.348 INFO:teuthology.orchestra.run.smithi086.stdout:107374182467 2024-04-16T05:33:59.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-16T05:33:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:00.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:00 smithi086 ceph-mon[32028]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:34:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.886+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/2634107987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d806ba30 msgr2=0x7fa1d810f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.886+0000 7fa1dd7a2700 1 --2- 172.21.15.86:0/2634107987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d806ba30 0x7fa1d810f9a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000b0a0 tx=0x7fa1d000b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.886+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/2634107987 shutdown_connections 2024-04-16T05:34:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.886+0000 7fa1dd7a2700 1 --2- 172.21.15.86:0/2634107987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d806ba30 0x7fa1d810f9a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.886+0000 7fa1dd7a2700 1 --2- 172.21.15.86:0/2634107987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1d8108b60 0x7fa1d8108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.886+0000 7fa1dd7a2700 1 --2- 172.21.15.86:0/2634107987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1d806b090 0x7fa1d806b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.886+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/2634107987 >> 172.21.15.86:0/2634107987 conn(0x7fa1d8078960 msgr2=0x7fa1d8078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.886+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/2634107987 shutdown_connections 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.886+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/2634107987 wait complete. 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.887+0000 7fa1dd7a2700 1 Processor -- start 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.887+0000 7fa1dd7a2700 1 -- start start 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.887+0000 7fa1dd7a2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d806b090 0x7fa1d8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.887+0000 7fa1dd7a2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1d806ba30 0x7fa1d81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.887+0000 7fa1dd7a2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1d8108b60 0x7fa1d819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.887+0000 7fa1dd7a2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8112dd0 con 0x7fa1d806b090 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.887+0000 7fa1dd7a2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8112c50 con 0x7fa1d8108b60 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.887+0000 7fa1dd7a2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8112f50 con 0x7fa1d806ba30 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.887+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d806b090 0x7fa1d8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d806b090 0x7fa1d8199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55946/0 (socket says 172.21.15.86:55946) 2024-04-16T05:34:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d6ffd700 1 -- 172.21.15.86:0/1460317795 learned_addr learned my addr 172.21.15.86:0/1460317795 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1ce7fc700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1d806ba30 0x7fa1d81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d77fe700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1d8108b60 0x7fa1d819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d6ffd700 1 -- 172.21.15.86:0/1460317795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1d806ba30 msgr2=0x7fa1d81997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d6ffd700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1d806ba30 0x7fa1d81997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d6ffd700 1 -- 172.21.15.86:0/1460317795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1d8108b60 msgr2=0x7fa1d819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d6ffd700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1d8108b60 0x7fa1d819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d6ffd700 1 -- 172.21.15.86:0/1460317795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d819e270 con 0x7fa1d806b090 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d77fe700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1d8108b60 0x7fa1d819db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d6ffd700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d806b090 0x7fa1d8199260 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800d980 tx=0x7fa1c800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d4ff9700 1 -- 172.21.15.86:0/1460317795 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8017070 con 0x7fa1d806b090 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d4ff9700 1 -- 172.21.15.86:0/1460317795 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c8004d10 con 0x7fa1d806b090 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d819e560 con 0x7fa1d806b090 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1d4ff9700 1 -- 172.21.15.86:0/1460317795 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80055c0 con 0x7fa1d806b090 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.888+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d81a5de0 con 0x7fa1d806b090 2024-04-16T05:34:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.889+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1d8063200 con 0x7fa1d806b090 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.890+0000 7fa1d4ff9700 1 -- 172.21.15.86:0/1460317795 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa1c8004750 con 0x7fa1d806b090 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.891+0000 7fa1d4ff9700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1c40a3cd0 0x7fa1c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.891+0000 7fa1ce7fc700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1c40a3cd0 0x7fa1c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.891+0000 7fa1d4ff9700 1 -- 172.21.15.86:0/1460317795 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa1c80d0e10 con 0x7fa1d806b090 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.891+0000 7fa1d4ff9700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] conn(0x7fa1c40aee30 0x7fa1c40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.891+0000 7fa1d4ff9700 1 -- 172.21.15.86:0/1460317795 --> [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1c8026e50 con 0x7fa1c40aee30 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.891+0000 7fa1d4ff9700 1 -- 172.21.15.86:0/1460317795 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa1c80d1260 con 0x7fa1d806b090 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.891+0000 7fa1d77fe700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] conn(0x7fa1c40aee30 0x7fa1c40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.891+0000 7fa1ce7fc700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1c40a3cd0 0x7fa1c40a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0005950 tx=0x7fa1c00058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.892+0000 7fa1d77fe700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] conn(0x7fa1c40aee30 0x7fa1c40b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.892+0000 7fa1d4ff9700 1 -- 172.21.15.86:0/1460317795 <== osd.0 v2:172.21.15.86:6802/571788251 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa1c8026e50 con 0x7fa1c40aee30 2024-04-16T05:34:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:00 smithi194 ceph-mon[35865]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:34:00.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.912+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 --> [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1d8063600 con 0x7fa1c40aee30 2024-04-16T05:34:00.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.913+0000 7fa1d4ff9700 1 -- 172.21.15.86:0/1460317795 <== osd.0 v2:172.21.15.86:6802/571788251 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa1d8063600 con 0x7fa1c40aee30 2024-04-16T05:34:00.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.913+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] conn(0x7fa1c40aee30 msgr2=0x7fa1c40b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.913+0000 7fa1dd7a2700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] conn(0x7fa1c40aee30 0x7fa1c40b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.914+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1c40a3cd0 msgr2=0x7fa1c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.914+0000 7fa1dd7a2700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1c40a3cd0 0x7fa1c40a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0005950 tx=0x7fa1c00058e0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.914+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d806b090 msgr2=0x7fa1d8199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.914+0000 7fa1dd7a2700 1 --2- 172.21.15.86:0/1460317795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d806b090 0x7fa1d8199260 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800d980 tx=0x7fa1c800dd40 comp rx=0 tx=0).stop 2024-04-16T05:34:00.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.914+0000 7fa1d77fe700 1 -- 172.21.15.86:0/1460317795 reap_dead start 2024-04-16T05:34:00.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.914+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 shutdown_connections 2024-04-16T05:34:00.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.914+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 >> 172.21.15.86:0/1460317795 conn(0x7fa1d8078960 msgr2=0x7fa1d80ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:00.913 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:00.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.914+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 shutdown_connections 2024-04-16T05:34:00.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:00.914+0000 7fa1dd7a2700 1 -- 172.21.15.86:0/1460317795 wait complete. 2024-04-16T05:34:00.915 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:00 smithi196 ceph-mon[36373]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:34:02.714 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:02.714 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:02.753 INFO:teuthology.orchestra.run.smithi086.stdout:34359738450 2024-04-16T05:34:02.754 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-16T05:34:02.828 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:02 smithi086 ceph-mon[32028]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-16T05:34:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:02 smithi194 ceph-mon[35865]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-16T05:34:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:02 smithi196 ceph-mon[36373]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-16T05:34:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:04.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:04 smithi086 ceph-mon[32028]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:34:04.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:04 smithi194 ceph-mon[35865]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:34:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.971+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/1790443986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38101340 msgr2=0x7f7d38101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.971+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/1790443986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38101340 0x7f7d38101720 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800b0a0 tx=0x7f7d2800b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.971+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/1790443986 shutdown_connections 2024-04-16T05:34:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.971+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/1790443986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d38107e30 0x7f7d3810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.971+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/1790443986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38101340 0x7f7d38101720 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.971+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/1790443986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d38101cf0 0x7f7d38107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:04.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.971+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/1790443986 >> 172.21.15.86:0/1790443986 conn(0x7f7d38078090 msgr2=0x7f7d380ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:04.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.971+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/1790443986 shutdown_connections 2024-04-16T05:34:04.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.971+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/1790443986 wait complete. 2024-04-16T05:34:04.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.972+0000 7f7d3fcd1700 1 Processor -- start 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.972+0000 7f7d3fcd1700 1 -- start start 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.972+0000 7f7d3fcd1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d38101340 0x7f7d381022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.972+0000 7f7d3fcd1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d38101cf0 0x7f7d38102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.972+0000 7f7d3fcd1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38107e30 0x7f7d38102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.972+0000 7f7d3fcd1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38112e70 con 0x7f7d38107e30 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.972+0000 7f7d3fcd1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38112cf0 con 0x7f7d38101340 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.972+0000 7f7d3fcd1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38112ff0 con 0x7f7d38101cf0 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3e26e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38107e30 0x7f7d38102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3e26e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38107e30 0x7f7d38102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55964/0 (socket says 172.21.15.86:55964) 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3e26e700 1 -- 172.21.15.86:0/768018191 learned_addr learned my addr 172.21.15.86:0/768018191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3d26c700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d38101cf0 0x7f7d38102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3da6d700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d38101340 0x7f7d381022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3e26e700 1 -- 172.21.15.86:0/768018191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d38101cf0 msgr2=0x7f7d38102800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3e26e700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d38101cf0 0x7f7d38102800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3e26e700 1 -- 172.21.15.86:0/768018191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d38101340 msgr2=0x7f7d381022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3e26e700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d38101340 0x7f7d381022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3e26e700 1 -- 172.21.15.86:0/768018191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d381a7900 con 0x7f7d38107e30 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3e26e700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38107e30 0x7f7d38102d40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c970 tx=0x7f7d3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d36ffd700 1 -- 172.21.15.86:0/768018191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000ce60 con 0x7f7d38107e30 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d381a7bf0 con 0x7f7d38107e30 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d36ffd700 1 -- 172.21.15.86:0/768018191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d30004830 con 0x7f7d38107e30 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d381a8130 con 0x7f7d38107e30 2024-04-16T05:34:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.973+0000 7f7d36ffd700 1 -- 172.21.15.86:0/768018191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d300055a0 con 0x7f7d38107e30 2024-04-16T05:34:04.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.974+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3810b050 con 0x7f7d38107e30 2024-04-16T05:34:04.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.976+0000 7f7d36ffd700 1 -- 172.21.15.86:0/768018191 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d300049e0 con 0x7f7d38107e30 2024-04-16T05:34:04.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.976+0000 7f7d36ffd700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7d200a3cd0 0x7f7d200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:04.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.976+0000 7f7d3da6d700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7d200a3cd0 0x7f7d200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:04.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.976+0000 7f7d36ffd700 1 -- 172.21.15.86:0/768018191 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7d300d0130 con 0x7f7d38107e30 2024-04-16T05:34:04.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.977+0000 7f7d3da6d700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7d200a3cd0 0x7f7d200a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7d28006130 tx=0x7f7d28006080 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:04.981+0000 7f7d36ffd700 1 -- 172.21.15.86:0/768018191 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d30014030 con 0x7f7d38107e30 2024-04-16T05:34:04.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:04 smithi196 ceph-mon[36373]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:34:05.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.186+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7d381043e0 con 0x7f7d38107e30 2024-04-16T05:34:05.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.186+0000 7f7d36ffd700 1 -- 172.21.15.86:0/768018191 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7d3009c650 con 0x7f7d38107e30 2024-04-16T05:34:05.185 INFO:teuthology.orchestra.run.smithi086.stdout:163208757305 2024-04-16T05:34:05.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7d200a3cd0 msgr2=0x7f7d200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:05.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7d200a3cd0 0x7f7d200a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7d28006130 tx=0x7f7d28006080 comp rx=0 tx=0).stop 2024-04-16T05:34:05.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38107e30 msgr2=0x7f7d38102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38107e30 0x7f7d38102d40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c970 tx=0x7f7d3000cc80 comp rx=0 tx=0).stop 2024-04-16T05:34:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 shutdown_connections 2024-04-16T05:34:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d38107e30 0x7f7d38102d40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d38101340 0x7f7d381022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7d200a3cd0 0x7f7d200a6190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 --2- 172.21.15.86:0/768018191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d38101cf0 0x7f7d38102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 >> 172.21.15.86:0/768018191 conn(0x7f7d38078090 msgr2=0x7f7d380ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 shutdown_connections 2024-04-16T05:34:05.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:05.188+0000 7f7d3fcd1700 1 -- 172.21.15.86:0/768018191 wait complete. 2024-04-16T05:34:05.695 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:05 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/768018191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T05:34:05.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:05 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/768018191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T05:34:05.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:05 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/768018191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T05:34:06.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:06 smithi086 ceph-mon[32028]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-16T05:34:06.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:06 smithi194 ceph-mon[35865]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-16T05:34:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:06 smithi196 ceph-mon[36373]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-16T05:34:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.113+0000 7f0125c86700 1 -- 172.21.15.86:0/301618361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 msgr2=0x7f0120108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.113+0000 7f0125c86700 1 --2- 172.21.15.86:0/301618361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 0x7f0120108f70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0110009990 tx=0x7f0110009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.113+0000 7f0125c86700 1 -- 172.21.15.86:0/301618361 shutdown_connections 2024-04-16T05:34:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.113+0000 7f0125c86700 1 --2- 172.21.15.86:0/301618361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012006bc20 0x7f012010f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.113+0000 7f0125c86700 1 --2- 172.21.15.86:0/301618361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 0x7f0120108f70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.113+0000 7f0125c86700 1 --2- 172.21.15.86:0/301618361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f012006b260 0x7f012006b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.113+0000 7f0125c86700 1 -- 172.21.15.86:0/301618361 >> 172.21.15.86:0/301618361 conn(0x7f0120077a60 msgr2=0x7f0120077e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:08.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.113+0000 7f0125c86700 1 -- 172.21.15.86:0/301618361 shutdown_connections 2024-04-16T05:34:08.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.113+0000 7f0125c86700 1 -- 172.21.15.86:0/301618361 wait complete. 2024-04-16T05:34:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f0125c86700 1 Processor -- start 2024-04-16T05:34:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f0125c86700 1 -- start start 2024-04-16T05:34:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f0125c86700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012006b260 0x7f0120074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f0125c86700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f012006bc20 0x7f01200747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f0125c86700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 0x7f012006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f0125c86700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0120112e50 con 0x7f0120108b90 2024-04-16T05:34:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f0125c86700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0120112cd0 con 0x7f012006bc20 2024-04-16T05:34:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f0125c86700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0120112fd0 con 0x7f012006b260 2024-04-16T05:34:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f011ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 0x7f012006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f011ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 0x7f012006d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55982/0 (socket says 172.21.15.86:55982) 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.114+0000 7f011ffff700 1 -- 172.21.15.86:0/1189932705 learned_addr learned my addr 172.21.15.86:0/1189932705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011effd700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f012006bc20 0x7f01200747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011f7fe700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012006b260 0x7f0120074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011ffff700 1 -- 172.21.15.86:0/1189932705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012006b260 msgr2=0x7f0120074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011ffff700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012006b260 0x7f0120074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011ffff700 1 -- 172.21.15.86:0/1189932705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f012006bc20 msgr2=0x7f01200747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011ffff700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f012006bc20 0x7f01200747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011ffff700 1 -- 172.21.15.86:0/1189932705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012006ddd0 con 0x7f0120108b90 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011f7fe700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012006b260 0x7f0120074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011effd700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f012006bc20 0x7f01200747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:08.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011ffff700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 0x7f012006d500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f011800b970 tx=0x7f011800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:08.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011cff9700 1 -- 172.21.15.86:0/1189932705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118019070 con 0x7f0120108b90 2024-04-16T05:34:08.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011cff9700 1 -- 172.21.15.86:0/1189932705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0118014440 con 0x7f0120108b90 2024-04-16T05:34:08.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.115+0000 7f011cff9700 1 -- 172.21.15.86:0/1189932705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01180075d0 con 0x7f0120108b90 2024-04-16T05:34:08.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.116+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012006e0c0 con 0x7f0120108b90 2024-04-16T05:34:08.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.116+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0120072060 con 0x7f0120108b90 2024-04-16T05:34:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.117+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01201041e0 con 0x7f0120108b90 2024-04-16T05:34:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.118+0000 7f011cff9700 1 -- 172.21.15.86:0/1189932705 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f011801c070 con 0x7f0120108b90 2024-04-16T05:34:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.118+0000 7f011cff9700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01080a3d50 0x7f01080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.119+0000 7f011f7fe700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01080a3d50 0x7f01080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.119+0000 7f011cff9700 1 -- 172.21.15.86:0/1189932705 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f01180d0030 con 0x7f0120108b90 2024-04-16T05:34:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.119+0000 7f011f7fe700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01080a3d50 0x7f01080a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0110006010 tx=0x7f011000b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.122+0000 7f011cff9700 1 -- 172.21.15.86:0/1189932705 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f011809c660 con 0x7f0120108b90 2024-04-16T05:34:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.324+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0120072b50 con 0x7f0120108b90 2024-04-16T05:34:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.324+0000 7f011cff9700 1 -- 172.21.15.86:0/1189932705 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f011809bf60 con 0x7f0120108b90 2024-04-16T05:34:08.323 INFO:teuthology.orchestra.run.smithi086.stdout:141733920832 2024-04-16T05:34:08.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.326+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01080a3d50 msgr2=0x7f01080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.326+0000 7f0125c86700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01080a3d50 0x7f01080a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0110006010 tx=0x7f011000b540 comp rx=0 tx=0).stop 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 msgr2=0x7f012006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 0x7f012006d500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f011800b970 tx=0x7f011800bd30 comp rx=0 tx=0).stop 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 shutdown_connections 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0120108b90 0x7f012006d500 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f01080a3d50 0x7f01080a6210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012006b260 0x7f0120074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 --2- 172.21.15.86:0/1189932705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f012006bc20 0x7f01200747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 >> 172.21.15.86:0/1189932705 conn(0x7f0120077a60 msgr2=0x7f0120100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 shutdown_connections 2024-04-16T05:34:08.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:08.327+0000 7f0125c86700 1 -- 172.21.15.86:0/1189932705 wait complete. 2024-04-16T05:34:08.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:08 smithi086 ceph-mon[32028]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-16T05:34:08.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:08 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1189932705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T05:34:08.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:08 smithi194 ceph-mon[35865]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-16T05:34:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:08 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1189932705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T05:34:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:08 smithi196 ceph-mon[36373]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-16T05:34:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:08 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1189932705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T05:34:09.518 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920827 got 141733920832 for osd.4 2024-04-16T05:34:09.519 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:10.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:10 smithi086 ceph-mon[32028]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-16T05:34:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:10 smithi194 ceph-mon[35865]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-16T05:34:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:10 smithi196 ceph-mon[36373]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-16T05:34:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.335+0000 7fe0b49ec700 1 -- 172.21.15.86:0/3471139068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0103530 msgr2=0x7fe0b010f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.335+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/3471139068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0103530 0x7fe0b010f9c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b0a0 tx=0x7fe0a800b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.336+0000 7fe0b49ec700 1 -- 172.21.15.86:0/3471139068 shutdown_connections 2024-04-16T05:34:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.336+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/3471139068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0103530 0x7fe0b010f9c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.336+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/3471139068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0b0108b80 0x7fe0b0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.336+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/3471139068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0102b70 0x7fe0b0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.336+0000 7fe0b49ec700 1 -- 172.21.15.86:0/3471139068 >> 172.21.15.86:0/3471139068 conn(0x7fe0b0079c60 msgr2=0x7fe0b007a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.336+0000 7fe0b49ec700 1 -- 172.21.15.86:0/3471139068 shutdown_connections 2024-04-16T05:34:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.336+0000 7fe0b49ec700 1 -- 172.21.15.86:0/3471139068 wait complete. 2024-04-16T05:34:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.336+0000 7fe0b49ec700 1 Processor -- start 2024-04-16T05:34:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.336+0000 7fe0b49ec700 1 -- start start 2024-04-16T05:34:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0b49ec700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0102b70 0x7fe0b01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0b49ec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0b0103530 0x7fe0b0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0b49ec700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0108b80 0x7fe0b019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0b49ec700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0112e80 con 0x7fe0b0108b80 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0b49ec700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0112d00 con 0x7fe0b0102b70 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0b49ec700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0113000 con 0x7fe0b0103530 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0aed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0108b80 0x7fe0b019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0aed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0108b80 0x7fe0b019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37220/0 (socket says 172.21.15.86:37220) 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0ae59c700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0102b70 0x7fe0b01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0aed9d700 1 -- 172.21.15.86:0/1129205997 learned_addr learned my addr 172.21.15.86:0/1129205997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0add9b700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0b0103530 0x7fe0b0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0aed9d700 1 -- 172.21.15.86:0/1129205997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0b0103530 msgr2=0x7fe0b0199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0aed9d700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0b0103530 0x7fe0b0199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0aed9d700 1 -- 172.21.15.86:0/1129205997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0102b70 msgr2=0x7fe0b01992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0aed9d700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0102b70 0x7fe0b01992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0aed9d700 1 -- 172.21.15.86:0/1129205997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b019e2e0 con 0x7fe0b0108b80 2024-04-16T05:34:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe0aed9d700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0108b80 0x7fe0b019dba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80062a0 tx=0x7fe0a800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.337+0000 7fe09f7fe700 1 -- 172.21.15.86:0/1129205997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8021070 con 0x7fe0b0108b80 2024-04-16T05:34:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.338+0000 7fe09f7fe700 1 -- 172.21.15.86:0/1129205997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a8007c80 con 0x7fe0b0108b80 2024-04-16T05:34:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.338+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b019e570 con 0x7fe0b0108b80 2024-04-16T05:34:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.338+0000 7fe09f7fe700 1 -- 172.21.15.86:0/1129205997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8005420 con 0x7fe0b0108b80 2024-04-16T05:34:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.338+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b01a5e50 con 0x7fe0b0108b80 2024-04-16T05:34:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.339+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b00697e0 con 0x7fe0b0108b80 2024-04-16T05:34:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.340+0000 7fe09f7fe700 1 -- 172.21.15.86:0/1129205997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe0a8024070 con 0x7fe0b0108b80 2024-04-16T05:34:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.340+0000 7fe09f7fe700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0980a3cd0 0x7fe0980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.341+0000 7fe0ae59c700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0980a3cd0 0x7fe0980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.341+0000 7fe09f7fe700 1 -- 172.21.15.86:0/1129205997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe0a800f020 con 0x7fe0b0108b80 2024-04-16T05:34:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.341+0000 7fe0ae59c700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0980a3cd0 0x7fe0980a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0069690 tx=0x7fe0a0009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.345+0000 7fe09f7fe700 1 -- 172.21.15.86:0/1129205997 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0a802b030 con 0x7fe0b0108b80 2024-04-16T05:34:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.561+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe0b002dfe0 con 0x7fe0b0108b80 2024-04-16T05:34:11.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.561+0000 7fe09f7fe700 1 -- 172.21.15.86:0/1129205997 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe0a809e170 con 0x7fe0b0108b80 2024-04-16T05:34:11.561 INFO:teuthology.orchestra.run.smithi086.stdout:253403070508 2024-04-16T05:34:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.563+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0980a3cd0 msgr2=0x7fe0980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.563+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0980a3cd0 0x7fe0980a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0069690 tx=0x7fe0a0009190 comp rx=0 tx=0).stop 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.563+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0108b80 msgr2=0x7fe0b019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.563+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0108b80 0x7fe0b019dba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80062a0 tx=0x7fe0a800be60 comp rx=0 tx=0).stop 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.563+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 shutdown_connections 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.563+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b0108b80 0x7fe0b019dba0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.563+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0102b70 0x7fe0b01992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.563+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe0980a3cd0 0x7fe0980a6190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.563+0000 7fe0b49ec700 1 --2- 172.21.15.86:0/1129205997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0b0103530 0x7fe0b0199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.564+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 >> 172.21.15.86:0/1129205997 conn(0x7fe0b0079c60 msgr2=0x7fe0b0101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.564+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 shutdown_connections 2024-04-16T05:34:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.564+0000 7fe0b49ec700 1 -- 172.21.15.86:0/1129205997 wait complete. 2024-04-16T05:34:11.898 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:11 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1129205997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T05:34:11.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:11 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1129205997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T05:34:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:11 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1129205997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T05:34:12.314 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070508 for osd.8 2024-04-16T05:34:12.315 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:12.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:12 smithi086 ceph-mon[32028]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:12.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:12 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/959996847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T05:34:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:12 smithi194 ceph-mon[35865]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:12.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:12 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/959996847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T05:34:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:12 smithi196 ceph-mon[36373]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:12 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/959996847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T05:34:13.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 -- 172.21.15.86:0/3074809958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1023e0 msgr2=0x7fe62c10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 --2- 172.21.15.86:0/3074809958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1023e0 0x7fe62c10e8a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe62400b0a0 tx=0x7fe62400b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 -- 172.21.15.86:0/3074809958 shutdown_connections 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 --2- 172.21.15.86:0/3074809958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1023e0 0x7fe62c10e8a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 --2- 172.21.15.86:0/3074809958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c1079d0 0x7fe62c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 --2- 172.21.15.86:0/3074809958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe62c101a20 0x7fe62c101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 -- 172.21.15.86:0/3074809958 >> 172.21.15.86:0/3074809958 conn(0x7fe62c0789c0 msgr2=0x7fe62c078dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 -- 172.21.15.86:0/3074809958 shutdown_connections 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 -- 172.21.15.86:0/3074809958 wait complete. 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 Processor -- start 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.687+0000 7fe631970700 1 -- start start 2024-04-16T05:34:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe631970700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe62c101a20 0x7fe62c1035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe631970700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c1023e0 0x7fe62c103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe631970700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1079d0 0x7fe62c104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe631970700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c111d60 con 0x7fe62c1079d0 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe631970700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c111be0 con 0x7fe62c1023e0 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe631970700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c111ee0 con 0x7fe62c101a20 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62b7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1079d0 0x7fe62c104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62b7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1079d0 0x7fe62c104020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37240/0 (socket says 172.21.15.86:37240) 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62b7fe700 1 -- 172.21.15.86:0/959996847 learned_addr learned my addr 172.21.15.86:0/959996847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe6227fc700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c1023e0 0x7fe62c103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62affd700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe62c101a20 0x7fe62c1035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62b7fe700 1 -- 172.21.15.86:0/959996847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe62c101a20 msgr2=0x7fe62c1035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62b7fe700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe62c101a20 0x7fe62c1035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62b7fe700 1 -- 172.21.15.86:0/959996847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c1023e0 msgr2=0x7fe62c103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62b7fe700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c1023e0 0x7fe62c103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62b7fe700 1 -- 172.21.15.86:0/959996847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c1a77d0 con 0x7fe62c1079d0 2024-04-16T05:34:13.012 INFO:teuthology.orchestra.run.smithi086.stdout:309237645346 2024-04-16T05:34:13.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.688+0000 7fe62b7fe700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1079d0 0x7fe62c104020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe6240062a0 tx=0x7fe62400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:13.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.689+0000 7fe63096e700 1 -- 172.21.15.86:0/959996847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624021070 con 0x7fe62c1079d0 2024-04-16T05:34:13.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.689+0000 7fe63096e700 1 -- 172.21.15.86:0/959996847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe624007c80 con 0x7fe62c1079d0 2024-04-16T05:34:13.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.689+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c1a7a60 con 0x7fe62c1079d0 2024-04-16T05:34:13.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.689+0000 7fe63096e700 1 -- 172.21.15.86:0/959996847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624005420 con 0x7fe62c1079d0 2024-04-16T05:34:13.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.689+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c1a7f10 con 0x7fe62c1079d0 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.690+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62c0697e0 con 0x7fe62c1079d0 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.691+0000 7fe63096e700 1 -- 172.21.15.86:0/959996847 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe624024070 con 0x7fe62c1079d0 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.691+0000 7fe63096e700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe6180a3cd0 0x7fe6180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.692+0000 7fe62affd700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe6180a3cd0 0x7fe6180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.692+0000 7fe63096e700 1 -- 172.21.15.86:0/959996847 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe62400f020 con 0x7fe62c1079d0 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.692+0000 7fe62affd700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe6180a3cd0 0x7fe6180a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009a10 tx=0x7fe61c009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.696+0000 7fe63096e700 1 -- 172.21.15.86:0/959996847 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe62402b030 con 0x7fe62c1079d0 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.898+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fe62c02dfe0 con 0x7fe62c1079d0 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.899+0000 7fe63096e700 1 -- 172.21.15.86:0/959996847 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe62409e170 con 0x7fe62c1079d0 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe6180a3cd0 msgr2=0x7fe6180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe6180a3cd0 0x7fe6180a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009a10 tx=0x7fe61c009190 comp rx=0 tx=0).stop 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1079d0 msgr2=0x7fe62c104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1079d0 0x7fe62c104020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe6240062a0 tx=0x7fe62400be60 comp rx=0 tx=0).stop 2024-04-16T05:34:13.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 shutdown_connections 2024-04-16T05:34:13.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c1079d0 0x7fe62c104020 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe6180a3cd0 0x7fe6180a6190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe62c101a20 0x7fe62c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 --2- 172.21.15.86:0/959996847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c1023e0 0x7fe62c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 >> 172.21.15.86:0/959996847 conn(0x7fe62c0789c0 msgr2=0x7fe62c06b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:13.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 shutdown_connections 2024-04-16T05:34:13.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:11.901+0000 7fe631970700 1 -- 172.21.15.86:0/959996847 wait complete. 2024-04-16T05:34:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.108+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/730350695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb0107a40 msgr2=0x7f9eb0107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.108+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/730350695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb0107a40 0x7f9eb0107e20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009990 tx=0x7f9ea4009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.108+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/730350695 shutdown_connections 2024-04-16T05:34:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.108+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/730350695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9eb01023b0 0x7f9eb010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.108+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/730350695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb0107a40 0x7f9eb0107e20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.108+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/730350695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb01019f0 0x7f9eb0101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.108+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/730350695 >> 172.21.15.86:0/730350695 conn(0x7f9eb00789a0 msgr2=0x7f9eb0078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.108+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/730350695 shutdown_connections 2024-04-16T05:34:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.108+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/730350695 wait complete. 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eb6b1e700 1 Processor -- start 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eb6b1e700 1 -- start start 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eb6b1e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb01019f0 0x7f9eb01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eb6b1e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb01023b0 0x7f9eb0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eb6b1e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9eb0107a40 0x7f9eb019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eb6b1e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0111d40 con 0x7f9eb01023b0 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eb6b1e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0111bc0 con 0x7f9eb0107a40 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eb6b1e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0111ec0 con 0x7f9eb01019f0 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eaffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb01023b0 0x7f9eb0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eaffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb01023b0 0x7f9eb0199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37254/0 (socket says 172.21.15.86:37254) 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eaffff700 1 -- 172.21.15.86:0/2980647722 learned_addr learned my addr 172.21.15.86:0/2980647722 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.109+0000 7f9eb48ba700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb01019f0 0x7f9eb01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eb50bb700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9eb0107a40 0x7f9eb019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eaffff700 1 -- 172.21.15.86:0/2980647722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb01019f0 msgr2=0x7f9eb01992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eaffff700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb01019f0 0x7f9eb01992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eaffff700 1 -- 172.21.15.86:0/2980647722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9eb0107a40 msgr2=0x7f9eb019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eaffff700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9eb0107a40 0x7f9eb019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eaffff700 1 -- 172.21.15.86:0/2980647722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb019e2d0 con 0x7f9eb01023b0 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eaffff700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb01023b0 0x7f9eb0199800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c970 tx=0x7f9e9c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eadffb700 1 -- 172.21.15.86:0/2980647722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0078d0 con 0x7f9eb01023b0 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb019e5c0 con 0x7f9eb01023b0 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eadffb700 1 -- 172.21.15.86:0/2980647722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c018480 con 0x7f9eb01023b0 2024-04-16T05:34:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.110+0000 7f9eadffb700 1 -- 172.21.15.86:0/2980647722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00f6b0 con 0x7f9eb01023b0 2024-04-16T05:34:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.111+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01034d0 con 0x7f9eb01023b0 2024-04-16T05:34:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.112+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb0103110 con 0x7f9eb01023b0 2024-04-16T05:34:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.114+0000 7f9eadffb700 1 -- 172.21.15.86:0/2980647722 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9e9c018630 con 0x7f9eb01023b0 2024-04-16T05:34:14.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.114+0000 7f9eadffb700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9ea00a3cd0 0x7f9ea00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:14.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.115+0000 7f9eb48ba700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9ea00a3cd0 0x7f9ea00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:14.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.115+0000 7f9eadffb700 1 -- 172.21.15.86:0/2980647722 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9e9c0d09d0 con 0x7f9eb01023b0 2024-04-16T05:34:14.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.115+0000 7f9eb48ba700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9ea00a3cd0 0x7f9ea00a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006010 tx=0x7f9ea400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.123+0000 7f9eadffb700 1 -- 172.21.15.86:0/2980647722 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e9c09cf80 con 0x7f9eb01023b0 2024-04-16T05:34:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.335+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9eb0063700 con 0x7f9eb01023b0 2024-04-16T05:34:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.336+0000 7f9eadffb700 1 -- 172.21.15.86:0/2980647722 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9e9c09c880 con 0x7f9eb01023b0 2024-04-16T05:34:14.336 INFO:teuthology.orchestra.run.smithi086.stdout:219043332145 2024-04-16T05:34:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9ea00a3cd0 msgr2=0x7f9ea00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9ea00a3cd0 0x7f9ea00a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006010 tx=0x7f9ea400b540 comp rx=0 tx=0).stop 2024-04-16T05:34:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb01023b0 msgr2=0x7f9eb0199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb01023b0 0x7f9eb0199800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c970 tx=0x7f9e9c00cc80 comp rx=0 tx=0).stop 2024-04-16T05:34:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 shutdown_connections 2024-04-16T05:34:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9eb0107a40 0x7f9eb019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f9ea00a3cd0 0x7f9ea00a6190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb01019f0 0x7f9eb01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 --2- 172.21.15.86:0/2980647722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb01023b0 0x7f9eb0199800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 >> 172.21.15.86:0/2980647722 conn(0x7f9eb00789a0 msgr2=0x7f9eb0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 shutdown_connections 2024-04-16T05:34:14.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:14.338+0000 7f9eb6b1e700 1 -- 172.21.15.86:0/2980647722 wait complete. 2024-04-16T05:34:14.525 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757305 for osd.5 2024-04-16T05:34:14.526 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:14.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:14 smithi086 ceph-mon[32028]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:14.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:14 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2980647722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T05:34:14.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:14 smithi194 ceph-mon[35865]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:14.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:14 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2980647722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T05:34:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:14 smithi196 ceph-mon[36373]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:14 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2980647722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T05:34:15.569 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332141 got 219043332145 for osd.7 2024-04-16T05:34:15.569 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:16.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.086+0000 7f67b690c700 1 -- 172.21.15.86:0/3595559783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0101a10 msgr2=0x7f67b0101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.086+0000 7f67b690c700 1 --2- 172.21.15.86:0/3595559783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0101a10 0x7f67b0101e90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6798009990 tx=0x7f6798009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.086+0000 7f67b690c700 1 -- 172.21.15.86:0/3595559783 shutdown_connections 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.086+0000 7f67b690c700 1 --2- 172.21.15.86:0/3595559783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b01023d0 0x7f67b010e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.086+0000 7f67b690c700 1 --2- 172.21.15.86:0/3595559783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b0107a10 0x7f67b0107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.086+0000 7f67b690c700 1 --2- 172.21.15.86:0/3595559783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0101a10 0x7f67b0101e90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.086+0000 7f67b690c700 1 -- 172.21.15.86:0/3595559783 >> 172.21.15.86:0/3595559783 conn(0x7f67b0078a90 msgr2=0x7f67b0078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 -- 172.21.15.86:0/3595559783 shutdown_connections 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 -- 172.21.15.86:0/3595559783 wait complete. 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 Processor -- start 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 -- start start 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b0101a10 0x7f67b0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b01023d0 0x7f67b01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0107a10 0x7f67b019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0111da0 con 0x7f67b0107a10 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0111c20 con 0x7f67b01023d0 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b690c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0111f20 con 0x7f67b0101a10 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.087+0000 7f67b4ea9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0107a10 0x7f67b019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b4ea9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0107a10 0x7f67b019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37276/0 (socket says 172.21.15.86:37276) 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b4ea9700 1 -- 172.21.15.86:0/2133654306 learned_addr learned my addr 172.21.15.86:0/2133654306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67affff700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b0101a10 0x7f67b0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67a7fff700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b01023d0 0x7f67b01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b4ea9700 1 -- 172.21.15.86:0/2133654306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b0101a10 msgr2=0x7f67b0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b4ea9700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b0101a10 0x7f67b0199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b4ea9700 1 -- 172.21.15.86:0/2133654306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b01023d0 msgr2=0x7f67b01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b4ea9700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b01023d0 0x7f67b01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b4ea9700 1 -- 172.21.15.86:0/2133654306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b019e290 con 0x7f67b0107a10 2024-04-16T05:34:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b4ea9700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0107a10 0x7f67b019db50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f67a800ef30 tx=0x7f67a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67adffb700 1 -- 172.21.15.86:0/2133654306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a80092e0 con 0x7f67b0107a10 2024-04-16T05:34:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67adffb700 1 -- 172.21.15.86:0/2133654306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a8004bb0 con 0x7f67b0107a10 2024-04-16T05:34:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b019e520 con 0x7f67b0107a10 2024-04-16T05:34:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67adffb700 1 -- 172.21.15.86:0/2133654306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8004280 con 0x7f67b0107a10 2024-04-16T05:34:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.088+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b01a5e00 con 0x7f67b0107a10 2024-04-16T05:34:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.089+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b00697e0 con 0x7f67b0107a10 2024-04-16T05:34:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.090+0000 7f67adffb700 1 -- 172.21.15.86:0/2133654306 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f67a801b030 con 0x7f67b0107a10 2024-04-16T05:34:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.091+0000 7f67adffb700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f679c0a3cd0 0x7f679c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.091+0000 7f67affff700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f679c0a3cd0 0x7f679c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.091+0000 7f67adffb700 1 -- 172.21.15.86:0/2133654306 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f67a80d0360 con 0x7f67b0107a10 2024-04-16T05:34:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.092+0000 7f67affff700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f679c0a3cd0 0x7f679c0a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005950 tx=0x7f67a000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:16.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.096+0000 7f67adffb700 1 -- 172.21.15.86:0/2133654306 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67a8022020 con 0x7f67b0107a10 2024-04-16T05:34:16.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.304+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f67b002dfe0 con 0x7f67b0107a10 2024-04-16T05:34:16.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.304+0000 7f67adffb700 1 -- 172.21.15.86:0/2133654306 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f67a8010040 con 0x7f67b0107a10 2024-04-16T05:34:16.303 INFO:teuthology.orchestra.run.smithi086.stdout:343597383709 2024-04-16T05:34:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f679c0a3cd0 msgr2=0x7f679c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f679c0a3cd0 0x7f679c0a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005950 tx=0x7f67a000b410 comp rx=0 tx=0).stop 2024-04-16T05:34:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0107a10 msgr2=0x7f67b019db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0107a10 0x7f67b019db50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f67a800ef30 tx=0x7f67a800c3a0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 shutdown_connections 2024-04-16T05:34:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b0107a10 0x7f67b019db50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f679c0a3cd0 0x7f679c0a6190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b0101a10 0x7f67b0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 --2- 172.21.15.86:0/2133654306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b01023d0 0x7f67b01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 >> 172.21.15.86:0/2133654306 conn(0x7f67b0078a90 msgr2=0x7f67b006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 shutdown_connections 2024-04-16T05:34:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:16.306+0000 7f67b690c700 1 -- 172.21.15.86:0/2133654306 wait complete. 2024-04-16T05:34:16.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:16 smithi086 ceph-mon[32028]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:34:16.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:16 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2133654306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T05:34:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:16 smithi194 ceph-mon[35865]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:34:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:16 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2133654306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T05:34:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:16 smithi196 ceph-mon[36373]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:34:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:16 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2133654306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T05:34:17.514 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383702 got 343597383709 for osd.11 2024-04-16T05:34:17.514 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:18.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 -- 172.21.15.86:0/4128035191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4103540 msgr2=0x7fdbd410fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/4128035191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4103540 0x7fdbd410fa20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b0a0 tx=0x7fdbcc00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 -- 172.21.15.86:0/4128035191 shutdown_connections 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/4128035191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4103540 0x7fdbd410fa20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/4128035191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbd4108c20 0x7fdbd4109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/4128035191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102b80 0x7fdbd4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 -- 172.21.15.86:0/4128035191 >> 172.21.15.86:0/4128035191 conn(0x7fdbd4079ce0 msgr2=0x7fdbd407a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 -- 172.21.15.86:0/4128035191 shutdown_connections 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 -- 172.21.15.86:0/4128035191 wait complete. 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.445+0000 7fdbd91f6700 1 Processor -- start 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd91f6700 1 -- start start 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd91f6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102b80 0x7fdbd4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd91f6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbd4103540 0x7fdbd4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd91f6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4108c20 0x7fdbd419dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd91f6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4112f40 con 0x7fdbd4108c20 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd91f6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4112dc0 con 0x7fdbd4103540 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd91f6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41130c0 con 0x7fdbd4102b80 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd359e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4108c20 0x7fdbd419dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd359e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4108c20 0x7fdbd419dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37288/0 (socket says 172.21.15.86:37288) 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd359e700 1 -- 172.21.15.86:0/3088737272 learned_addr learned my addr 172.21.15.86:0/3088737272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd2d9d700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102b80 0x7fdbd4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd259c700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbd4103540 0x7fdbd4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd359e700 1 -- 172.21.15.86:0/3088737272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102b80 msgr2=0x7fdbd4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd359e700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102b80 0x7fdbd4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd359e700 1 -- 172.21.15.86:0/3088737272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbd4103540 msgr2=0x7fdbd4199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd359e700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbd4103540 0x7fdbd4199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.446+0000 7fdbd359e700 1 -- 172.21.15.86:0/3088737272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd419e3d0 con 0x7fdbd4108c20 2024-04-16T05:34:18.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.447+0000 7fdbd359e700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4108c20 0x7fdbd419dc90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b830 tx=0x7fdbcc007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:18.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.447+0000 7fdbbbfff700 1 -- 172.21.15.86:0/3088737272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc021020 con 0x7fdbd4108c20 2024-04-16T05:34:18.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.447+0000 7fdbbbfff700 1 -- 172.21.15.86:0/3088737272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbcc01db50 con 0x7fdbd4108c20 2024-04-16T05:34:18.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.447+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd419e660 con 0x7fdbd4108c20 2024-04-16T05:34:18.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.447+0000 7fdbbbfff700 1 -- 172.21.15.86:0/3088737272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc01bd00 con 0x7fdbd4108c20 2024-04-16T05:34:18.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.447+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd41a5f40 con 0x7fdbd4108c20 2024-04-16T05:34:18.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.448+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd410b0e0 con 0x7fdbd4108c20 2024-04-16T05:34:18.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.450+0000 7fdbbbfff700 1 -- 172.21.15.86:0/3088737272 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdbcc024020 con 0x7fdbd4108c20 2024-04-16T05:34:18.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.450+0000 7fdbbbfff700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdbbc0a3d50 0x7fdbbc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.451+0000 7fdbd2d9d700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdbbc0a3d50 0x7fdbbc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.451+0000 7fdbbbfff700 1 -- 172.21.15.86:0/3088737272 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdbcc00f020 con 0x7fdbd4108c20 2024-04-16T05:34:18.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.451+0000 7fdbd2d9d700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdbbc0a3d50 0x7fdbbc0a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4006fd0 tx=0x7fdbc4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.454+0000 7fdbbbfff700 1 -- 172.21.15.86:0/3088737272 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbcc09ecb0 con 0x7fdbd4108c20 2024-04-16T05:34:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.660+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdbd419a720 con 0x7fdbd4108c20 2024-04-16T05:34:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.660+0000 7fdbbbfff700 1 -- 172.21.15.86:0/3088737272 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdbcc09e5b0 con 0x7fdbd4108c20 2024-04-16T05:34:18.659 INFO:teuthology.orchestra.run.smithi086.stdout:77309411404 2024-04-16T05:34:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdbbc0a3d50 msgr2=0x7fdbbc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdbbc0a3d50 0x7fdbbc0a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4006fd0 tx=0x7fdbc4008040 comp rx=0 tx=0).stop 2024-04-16T05:34:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4108c20 msgr2=0x7fdbd419dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4108c20 0x7fdbd419dc90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b830 tx=0x7fdbcc007b40 comp rx=0 tx=0).stop 2024-04-16T05:34:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 shutdown_connections 2024-04-16T05:34:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd4108c20 0x7fdbd419dc90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdbbc0a3d50 0x7fdbbc0a6210 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102b80 0x7fdbd4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 --2- 172.21.15.86:0/3088737272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbd4103540 0x7fdbd4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 >> 172.21.15.86:0/3088737272 conn(0x7fdbd4079ce0 msgr2=0x7fdbd41017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 shutdown_connections 2024-04-16T05:34:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:18.662+0000 7fdbd91f6700 1 -- 172.21.15.86:0/3088737272 wait complete. 2024-04-16T05:34:18.844 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:18 smithi086 ceph-mon[32028]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:34:18.869 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645342 got 309237645346 for osd.10 2024-04-16T05:34:18.869 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:18 smithi194 ceph-mon[35865]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:34:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:18 smithi196 ceph-mon[36373]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:34:19.775 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:19.886 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:19 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3088737272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T05:34:19.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:19 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3088737272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T05:34:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:19 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3088737272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T05:34:20.730 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411404 for osd.2 2024-04-16T05:34:20.730 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:20.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:20 smithi086 ceph-mon[32028]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:20 smithi196 ceph-mon[36373]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:20.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:20 smithi194 ceph-mon[35865]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.689+0000 7f45b7ca9700 1 -- 172.21.15.86:0/1041115115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 msgr2=0x7f45b0102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.689+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/1041115115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 0x7f45b0102fc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009990 tx=0x7f45a0009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.689+0000 7f45b7ca9700 1 -- 172.21.15.86:0/1041115115 shutdown_connections 2024-04-16T05:34:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.689+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/1041115115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0103500 0x7f45b010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.689+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/1041115115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b0108b90 0x7f45b0108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.689+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/1041115115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 0x7f45b0102fc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.689+0000 7f45b7ca9700 1 -- 172.21.15.86:0/1041115115 >> 172.21.15.86:0/1041115115 conn(0x7f45b0079c50 msgr2=0x7f45b007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.689+0000 7f45b7ca9700 1 -- 172.21.15.86:0/1041115115 shutdown_connections 2024-04-16T05:34:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.689+0000 7f45b7ca9700 1 -- 172.21.15.86:0/1041115115 wait complete. 2024-04-16T05:34:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.690+0000 7f45b7ca9700 1 Processor -- start 2024-04-16T05:34:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.690+0000 7f45b7ca9700 1 -- start start 2024-04-16T05:34:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.690+0000 7f45b7ca9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 0x7f45b0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.690+0000 7f45b7ca9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b0103500 0x7f45b0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.690+0000 7f45b7ca9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0108b90 0x7f45b019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.690+0000 7f45b7ca9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0112e80 con 0x7f45b0102b40 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.690+0000 7f45b7ca9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0112d00 con 0x7f45b0103500 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.690+0000 7f45b7ca9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0113000 con 0x7f45b0108b90 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5a45700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 0x7f45b0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5a45700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 0x7f45b0199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60660/0 (socket says 172.21.15.86:60660) 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5a45700 1 -- 172.21.15.86:0/2911216561 learned_addr learned my addr 172.21.15.86:0/2911216561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5244700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b0103500 0x7f45b0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b6246700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0108b90 0x7f45b019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5a45700 1 -- 172.21.15.86:0/2911216561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0108b90 msgr2=0x7f45b019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5a45700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0108b90 0x7f45b019dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5a45700 1 -- 172.21.15.86:0/2911216561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b0103500 msgr2=0x7f45b0199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5a45700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b0103500 0x7f45b0199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5a45700 1 -- 172.21.15.86:0/2911216561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b019e310 con 0x7f45b0102b40 2024-04-16T05:34:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b5a45700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 0x7f45b0199350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00ba50 tx=0x7f45ac00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45a6ffd700 1 -- 172.21.15.86:0/2911216561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac019070 con 0x7f45b0102b40 2024-04-16T05:34:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b019e600 con 0x7f45b0102b40 2024-04-16T05:34:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.691+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b0074d20 con 0x7f45b0102b40 2024-04-16T05:34:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.692+0000 7f45a6ffd700 1 -- 172.21.15.86:0/2911216561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac00ce10 con 0x7f45b0102b40 2024-04-16T05:34:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.692+0000 7f45a6ffd700 1 -- 172.21.15.86:0/2911216561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac007370 con 0x7f45b0102b40 2024-04-16T05:34:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.692+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4594005330 con 0x7f45b0102b40 2024-04-16T05:34:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.694+0000 7f45a6ffd700 1 -- 172.21.15.86:0/2911216561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f45ac01c070 con 0x7f45b0102b40 2024-04-16T05:34:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.694+0000 7f45a6ffd700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f459c0a3c40 0x7f459c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.694+0000 7f45b5244700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f459c0a3c40 0x7f459c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.694+0000 7f45a6ffd700 1 -- 172.21.15.86:0/2911216561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f45ac0cfe10 con 0x7f45b0102b40 2024-04-16T05:34:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.695+0000 7f45b5244700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f459c0a3c40 0x7f459c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45b007a2c0 tx=0x7f45a000b560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.698+0000 7f45a6ffd700 1 -- 172.21.15.86:0/2911216561 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45ac09c490 con 0x7f45b0102b40 2024-04-16T05:34:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.902+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f45940058f0 con 0x7f45b0102b40 2024-04-16T05:34:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.902+0000 7f45a6ffd700 1 -- 172.21.15.86:0/2911216561 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f45ac09bd90 con 0x7f45b0102b40 2024-04-16T05:34:21.901 INFO:teuthology.orchestra.run.smithi086.stdout:55834574930 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.904+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f459c0a3c40 msgr2=0x7f459c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.904+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f459c0a3c40 0x7f459c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45b007a2c0 tx=0x7f45a000b560 comp rx=0 tx=0).stop 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.904+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 msgr2=0x7f45b0199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.904+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 0x7f45b0199350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00ba50 tx=0x7f45ac00bd60 comp rx=0 tx=0).stop 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.905+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 shutdown_connections 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.905+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f459c0a3c40 0x7f459c0a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.905+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0108b90 0x7f45b019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.905+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0102b40 0x7f45b0199350 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.905+0000 7f45b7ca9700 1 --2- 172.21.15.86:0/2911216561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b0103500 0x7f45b0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.905+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 >> 172.21.15.86:0/2911216561 conn(0x7f45b0079c50 msgr2=0x7f45b0101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.905+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 shutdown_connections 2024-04-16T05:34:21.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:21.905+0000 7f45b7ca9700 1 -- 172.21.15.86:0/2911216561 wait complete. 2024-04-16T05:34:22.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:22 smithi194 ceph-mon[35865]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:22.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:22 smithi194 ceph-mon[35865]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-16T05:34:22.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:22 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2911216561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T05:34:22.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:22 smithi086 ceph-mon[32028]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:22.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:22 smithi086 ceph-mon[32028]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-16T05:34:22.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:22 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2911216561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T05:34:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:22 smithi196 ceph-mon[36373]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:22 smithi196 ceph-mon[36373]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-16T05:34:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:22 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2911216561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T05:34:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.834+0000 7fb11f557700 1 -- 172.21.15.86:0/488401110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb118101a20 msgr2=0x7fb118101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.834+0000 7fb11f557700 1 --2- 172.21.15.86:0/488401110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb118101a20 0x7fb118101ea0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb114009990 tx=0x7fb114009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.834+0000 7fb11f557700 1 -- 172.21.15.86:0/488401110 shutdown_connections 2024-04-16T05:34:23.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.834+0000 7fb11f557700 1 --2- 172.21.15.86:0/488401110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1181023e0 0x7fb11810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:23.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.834+0000 7fb11f557700 1 --2- 172.21.15.86:0/488401110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1181079d0 0x7fb118107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:23.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.834+0000 7fb11f557700 1 --2- 172.21.15.86:0/488401110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb118101a20 0x7fb118101ea0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:23.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.834+0000 7fb11f557700 1 -- 172.21.15.86:0/488401110 >> 172.21.15.86:0/488401110 conn(0x7fb1180789c0 msgr2=0x7fb118078dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:23.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.834+0000 7fb11f557700 1 -- 172.21.15.86:0/488401110 shutdown_connections 2024-04-16T05:34:23.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.834+0000 7fb11f557700 1 -- 172.21.15.86:0/488401110 wait complete. 2024-04-16T05:34:23.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11f557700 1 Processor -- start 2024-04-16T05:34:23.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11f557700 1 -- start start 2024-04-16T05:34:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11f557700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118101a20 0x7fb118198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11f557700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1181023e0 0x7fb118198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11f557700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1181079d0 0x7fb11819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11f557700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb118111d60 con 0x7fb1181023e0 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11f557700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb118111be0 con 0x7fb1181079d0 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11f557700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb118111ee0 con 0x7fb118101a20 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11caf2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1181023e0 0x7fb118198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11caf2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1181023e0 0x7fb118198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60684/0 (socket says 172.21.15.86:60684) 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11caf2700 1 -- 172.21.15.86:0/1225678915 learned_addr learned my addr 172.21.15.86:0/1225678915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11daf4700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1181079d0 0x7fb11819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.835+0000 7fb11d2f3700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118101a20 0x7fb118198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb11caf2700 1 -- 172.21.15.86:0/1225678915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118101a20 msgr2=0x7fb118198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb11caf2700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118101a20 0x7fb118198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:23.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb11caf2700 1 -- 172.21.15.86:0/1225678915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1181079d0 msgr2=0x7fb11819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb11caf2700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1181079d0 0x7fb11819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb11caf2700 1 -- 172.21.15.86:0/1225678915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11819d260 con 0x7fb1181023e0 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb11d2f3700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118101a20 0x7fb118198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb11caf2700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1181023e0 0x7fb118198790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c00 tx=0x7fb114003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb10e7fc700 1 -- 172.21.15.86:0/1225678915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114029070 con 0x7fb1181023e0 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11819d4f0 con 0x7fb1181023e0 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb10e7fc700 1 -- 172.21.15.86:0/1225678915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb114019db0 con 0x7fb1181023e0 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb10e7fc700 1 -- 172.21.15.86:0/1225678915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1140246d0 con 0x7fb1181023e0 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.836+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11806b240 con 0x7fb1181023e0 2024-04-16T05:34:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.837+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb118103110 con 0x7fb1181023e0 2024-04-16T05:34:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.838+0000 7fb10e7fc700 1 -- 172.21.15.86:0/1225678915 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb11402c070 con 0x7fb1181023e0 2024-04-16T05:34:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.839+0000 7fb10e7fc700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb1040a3c40 0x7fb1040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.839+0000 7fb11d2f3700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb1040a3c40 0x7fb1040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.839+0000 7fb10e7fc700 1 -- 172.21.15.86:0/1225678915 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb1140d3aa0 con 0x7fb1181023e0 2024-04-16T05:34:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.839+0000 7fb11d2f3700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb1040a3c40 0x7fb1040a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb108005950 tx=0x7fb108005f50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:23.842+0000 7fb10e7fc700 1 -- 172.21.15.86:0/1225678915 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb114033080 con 0x7fb1181023e0 2024-04-16T05:34:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.047+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb1181995b0 con 0x7fb1181023e0 2024-04-16T05:34:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.047+0000 7fb10e7fc700 1 -- 172.21.15.86:0/1225678915 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb114013070 con 0x7fb1181023e0 2024-04-16T05:34:24.046 INFO:teuthology.orchestra.run.smithi086.stdout:34359738455 2024-04-16T05:34:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.049+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb1040a3c40 msgr2=0x7fb1040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.049+0000 7fb11f557700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb1040a3c40 0x7fb1040a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb108005950 tx=0x7fb108005f50 comp rx=0 tx=0).stop 2024-04-16T05:34:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1181023e0 msgr2=0x7fb118198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1181023e0 0x7fb118198790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c00 tx=0x7fb114003680 comp rx=0 tx=0).stop 2024-04-16T05:34:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 shutdown_connections 2024-04-16T05:34:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1181079d0 0x7fb11819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb1040a3c40 0x7fb1040a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118101a20 0x7fb118198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 --2- 172.21.15.86:0/1225678915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1181023e0 0x7fb118198790 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 >> 172.21.15.86:0/1225678915 conn(0x7fb1180789c0 msgr2=0x7fb1180ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 shutdown_connections 2024-04-16T05:34:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.050+0000 7fb11f557700 1 -- 172.21.15.86:0/1225678915 wait complete. 2024-04-16T05:34:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.557+0000 7f02574bb700 1 -- 172.21.15.86:0/3265980974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250102bb0 msgr2=0x7f0250103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.557+0000 7f02574bb700 1 --2- 172.21.15.86:0/3265980974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250102bb0 0x7f0250103030 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0240009990 tx=0x7f0240009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.557+0000 7f02574bb700 1 -- 172.21.15.86:0/3265980974 shutdown_connections 2024-04-16T05:34:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.557+0000 7f02574bb700 1 --2- 172.21.15.86:0/3265980974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0250103570 0x7f025010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.557+0000 7f02574bb700 1 --2- 172.21.15.86:0/3265980974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0250108ac0 0x7f0250108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.557+0000 7f02574bb700 1 --2- 172.21.15.86:0/3265980974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250102bb0 0x7f0250103030 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.557+0000 7f02574bb700 1 -- 172.21.15.86:0/3265980974 >> 172.21.15.86:0/3265980974 conn(0x7f0250079cb0 msgr2=0x7f025007a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.557+0000 7f02574bb700 1 -- 172.21.15.86:0/3265980974 shutdown_connections 2024-04-16T05:34:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.557+0000 7f02574bb700 1 -- 172.21.15.86:0/3265980974 wait complete. 2024-04-16T05:34:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f02574bb700 1 Processor -- start 2024-04-16T05:34:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f02574bb700 1 -- start start 2024-04-16T05:34:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f02574bb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0250102bb0 0x7f02501992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f02574bb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250103570 0x7f0250199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f02574bb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0250108ac0 0x7f025019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f02574bb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0250112e50 con 0x7f0250103570 2024-04-16T05:34:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f02574bb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0250112cd0 con 0x7f0250102bb0 2024-04-16T05:34:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f02574bb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0250112fd0 con 0x7f0250108ac0 2024-04-16T05:34:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0254a56700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250103570 0x7f0250199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0254a56700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250103570 0x7f0250199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60698/0 (socket says 172.21.15.86:60698) 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0254a56700 1 -- 172.21.15.86:0/492450245 learned_addr learned my addr 172.21.15.86:0/492450245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0255a58700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0250108ac0 0x7f025019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0255257700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0250102bb0 0x7f02501992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0254a56700 1 -- 172.21.15.86:0/492450245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0250108ac0 msgr2=0x7f025019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0254a56700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0250108ac0 0x7f025019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0254a56700 1 -- 172.21.15.86:0/492450245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0250102bb0 msgr2=0x7f02501992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0254a56700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0250102bb0 0x7f02501992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.558+0000 7f0254a56700 1 -- 172.21.15.86:0/492450245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025019e2a0 con 0x7f0250103570 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.559+0000 7f0254a56700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250103570 0x7f0250199820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0240000c00 tx=0x7f02400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.559+0000 7f02467fc700 1 -- 172.21.15.86:0/492450245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240027070 con 0x7f0250103570 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.559+0000 7f02467fc700 1 -- 172.21.15.86:0/492450245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0240003780 con 0x7f0250103570 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.559+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025019e530 con 0x7f0250103570 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.559+0000 7f02467fc700 1 -- 172.21.15.86:0/492450245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240022dd0 con 0x7f0250103570 2024-04-16T05:34:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.559+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02501a5e10 con 0x7f0250103570 2024-04-16T05:34:24.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.560+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025010b050 con 0x7f0250103570 2024-04-16T05:34:24.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.561+0000 7f02467fc700 1 -- 172.21.15.86:0/492450245 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f024002a070 con 0x7f0250103570 2024-04-16T05:34:24.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.562+0000 7f02467fc700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f023c0a3c40 0x7f023c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:24.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.562+0000 7f0255257700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f023c0a3c40 0x7f023c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:24.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.562+0000 7f02467fc700 1 -- 172.21.15.86:0/492450245 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f02400d17f0 con 0x7f0250103570 2024-04-16T05:34:24.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.562+0000 7f0255257700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f023c0a3c40 0x7f023c0a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f024c009c90 tx=0x7f024c009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.565+0000 7f02467fc700 1 -- 172.21.15.86:0/492450245 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0240013070 con 0x7f0250103570 2024-04-16T05:34:24.710 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:24 smithi086 ceph-mon[32028]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:34:24.711 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:24 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1225678915' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T05:34:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.778+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f025019a5f0 con 0x7f0250103570 2024-04-16T05:34:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.779+0000 7f02467fc700 1 -- 172.21.15.86:0/492450245 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0240086e20 con 0x7f0250103570 2024-04-16T05:34:24.778 INFO:teuthology.orchestra.run.smithi086.stdout:287762808873 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f023c0a3c40 msgr2=0x7f023c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f023c0a3c40 0x7f023c0a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f024c009c90 tx=0x7f024c009040 comp rx=0 tx=0).stop 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250103570 msgr2=0x7f0250199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250103570 0x7f0250199820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0240000c00 tx=0x7f02400047b0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 shutdown_connections 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f023c0a3c40 0x7f023c0a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0250108ac0 0x7f025019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0250102bb0 0x7f02501992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 --2- 172.21.15.86:0/492450245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0250103570 0x7f0250199820 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 >> 172.21.15.86:0/492450245 conn(0x7f0250079cb0 msgr2=0x7f0250100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 shutdown_connections 2024-04-16T05:34:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:24.781+0000 7f02574bb700 1 -- 172.21.15.86:0/492450245 wait complete. 2024-04-16T05:34:24.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:24 smithi194 ceph-mon[35865]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:34:24.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:24 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1225678915' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T05:34:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:24 smithi196 ceph-mon[36373]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:34:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:24 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1225678915' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T05:34:25.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 -- 172.21.15.86:0/316067310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101e50 msgr2=0x7fec6410f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:25.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 --2- 172.21.15.86:0/316067310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101e50 0x7fec6410f8c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00b0a0 tx=0x7fec5c00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 -- 172.21.15.86:0/316067310 shutdown_connections 2024-04-16T05:34:25.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 --2- 172.21.15.86:0/316067310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101e50 0x7fec6410f8c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 --2- 172.21.15.86:0/316067310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec64107bb0 0x7fec64107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 --2- 172.21.15.86:0/316067310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec64101490 0x7fec64101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 -- 172.21.15.86:0/316067310 >> 172.21.15.86:0/316067310 conn(0x7fec64076d10 msgr2=0x7fec64077120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:25.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 -- 172.21.15.86:0/316067310 shutdown_connections 2024-04-16T05:34:25.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 -- 172.21.15.86:0/316067310 wait complete. 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 Processor -- start 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.390+0000 7fec68dce700 1 -- start start 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec68dce700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101490 0x7fec641992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec68dce700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec64101e50 0x7fec64199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec68dce700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec64107bb0 0x7fec6419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec68dce700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec64112e90 con 0x7fec64101490 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec68dce700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fec64112d10 con 0x7fec64107bb0 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec68dce700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec64113010 con 0x7fec64101e50 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec6259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101490 0x7fec641992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec6259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101490 0x7fec641992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60706/0 (socket says 172.21.15.86:60706) 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec6259c700 1 -- 172.21.15.86:0/1733920280 learned_addr learned my addr 172.21.15.86:0/1733920280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec61d9b700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec64101e50 0x7fec64199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec62d9d700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec64107bb0 0x7fec6419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:25.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec6259c700 1 -- 172.21.15.86:0/1733920280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec64101e50 msgr2=0x7fec64199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec6259c700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec64101e50 0x7fec64199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec6259c700 1 -- 172.21.15.86:0/1733920280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec64107bb0 msgr2=0x7fec6419dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec6259c700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec64107bb0 0x7fec6419dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec6259c700 1 -- 172.21.15.86:0/1733920280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6419e2f0 con 0x7fec64101490 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.391+0000 7fec6259c700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101490 0x7fec641992e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c970 tx=0x7fec5800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.392+0000 7fec537fe700 1 -- 172.21.15.86:0/1733920280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58004220 con 0x7fec64101490 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.392+0000 7fec537fe700 1 -- 172.21.15.86:0/1733920280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec580043c0 con 0x7fec64101490 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.392+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6419e5e0 con 0x7fec64101490 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.392+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec641025a0 con 0x7fec64101490 2024-04-16T05:34:25.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.392+0000 7fec537fe700 1 -- 172.21.15.86:0/1733920280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58003d40 con 0x7fec64101490 2024-04-16T05:34:25.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.393+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec640697e0 con 0x7fec64101490 2024-04-16T05:34:25.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.394+0000 7fec537fe700 1 -- 172.21.15.86:0/1733920280 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fec580076d0 con 0x7fec64101490 2024-04-16T05:34:25.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.394+0000 7fec537fe700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fec4c0a3c40 0x7fec4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:25.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.394+0000 7fec537fe700 1 -- 172.21.15.86:0/1733920280 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fec580d0760 con 0x7fec64101490 2024-04-16T05:34:25.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.394+0000 7fec61d9b700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fec4c0a3c40 0x7fec4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:25.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.395+0000 7fec61d9b700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fec4c0a3c40 0x7fec4c0a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec64077280 tx=0x7fec54009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.398+0000 7fec537fe700 1 -- 172.21.15.86:0/1733920280 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec58014030 con 0x7fec64101490 2024-04-16T05:34:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.600+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fec6419a670 con 0x7fec64101490 2024-04-16T05:34:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.601+0000 7fec537fe700 1 -- 172.21.15.86:0/1733920280 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fec6419a670 con 0x7fec64101490 2024-04-16T05:34:25.599 INFO:teuthology.orchestra.run.smithi086.stdout:197568495672 2024-04-16T05:34:25.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fec4c0a3c40 msgr2=0x7fec4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fec4c0a3c40 0x7fec4c0a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec64077280 tx=0x7fec54009190 comp rx=0 tx=0).stop 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101490 msgr2=0x7fec641992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101490 0x7fec641992e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c970 tx=0x7fec5800cc80 comp rx=0 tx=0).stop 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 shutdown_connections 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec64107bb0 0x7fec6419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec64101490 0x7fec641992e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fec4c0a3c40 0x7fec4c0a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 --2- 172.21.15.86:0/1733920280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec64101e50 0x7fec64199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 >> 172.21.15.86:0/1733920280 conn(0x7fec64076d10 msgr2=0x7fec640fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 shutdown_connections 2024-04-16T05:34:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:25.603+0000 7fec68dce700 1 -- 172.21.15.86:0/1733920280 wait complete. 2024-04-16T05:34:25.830 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808868 got 287762808873 for osd.9 2024-04-16T05:34:25.830 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:25.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:25 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/492450245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T05:34:25.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:25 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1733920280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T05:34:25.911 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:25 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/492450245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T05:34:25.911 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:25 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1733920280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T05:34:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:25 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/492450245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T05:34:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:25 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1733920280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T05:34:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.015+0000 7f35ba904700 1 -- 172.21.15.86:0/434235891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41019f0 msgr2=0x7f35b4101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.015+0000 7f35ba904700 1 --2- 172.21.15.86:0/434235891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41019f0 0x7f35b4101e70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009990 tx=0x7f35a4009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.015+0000 7f35ba904700 1 -- 172.21.15.86:0/434235891 shutdown_connections 2024-04-16T05:34:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.015+0000 7f35ba904700 1 --2- 172.21.15.86:0/434235891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b41023b0 0x7f35b410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.015+0000 7f35ba904700 1 --2- 172.21.15.86:0/434235891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b4107a40 0x7f35b4107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.015+0000 7f35ba904700 1 --2- 172.21.15.86:0/434235891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41019f0 0x7f35b4101e70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.015+0000 7f35ba904700 1 -- 172.21.15.86:0/434235891 >> 172.21.15.86:0/434235891 conn(0x7f35b40789a0 msgr2=0x7f35b4078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.015+0000 7f35ba904700 1 -- 172.21.15.86:0/434235891 shutdown_connections 2024-04-16T05:34:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.015+0000 7f35ba904700 1 -- 172.21.15.86:0/434235891 wait complete. 2024-04-16T05:34:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35ba904700 1 Processor -- start 2024-04-16T05:34:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35ba904700 1 -- start start 2024-04-16T05:34:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35ba904700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b41019f0 0x7f35b4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35ba904700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41023b0 0x7f35b4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35ba904700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b4107a40 0x7f35b419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35ba904700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4111d50 con 0x7f35b41023b0 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35ba904700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4111bd0 con 0x7f35b4107a40 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35ba904700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4111ed0 con 0x7f35b41019f0 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41023b0 0x7f35b4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41023b0 0x7f35b4199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60738/0 (socket says 172.21.15.86:60738) 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35b37fe700 1 -- 172.21.15.86:0/1228568969 learned_addr learned my addr 172.21.15.86:0/1228568969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35b8ea1700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b4107a40 0x7f35b419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.016+0000 7f35b3fff700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b41019f0 0x7f35b4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35b37fe700 1 -- 172.21.15.86:0/1228568969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b41019f0 msgr2=0x7f35b4199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35b37fe700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b41019f0 0x7f35b4199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35b37fe700 1 -- 172.21.15.86:0/1228568969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b4107a40 msgr2=0x7f35b419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35b37fe700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b4107a40 0x7f35b419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35b37fe700 1 -- 172.21.15.86:0/1228568969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b419e320 con 0x7f35b41023b0 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35b37fe700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41023b0 0x7f35b4199850 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f35a4000c00 tx=0x7f35a4003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35b17fa700 1 -- 172.21.15.86:0/1228568969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4027070 con 0x7f35b41023b0 2024-04-16T05:34:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b419e5b0 con 0x7f35b41023b0 2024-04-16T05:34:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35b17fa700 1 -- 172.21.15.86:0/1228568969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a4004110 con 0x7f35b41023b0 2024-04-16T05:34:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.017+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b41a5e90 con 0x7f35b41023b0 2024-04-16T05:34:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.018+0000 7f35b17fa700 1 -- 172.21.15.86:0/1228568969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a400f460 con 0x7f35b41023b0 2024-04-16T05:34:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.018+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a0005330 con 0x7f35b41023b0 2024-04-16T05:34:26.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.020+0000 7f35b17fa700 1 -- 172.21.15.86:0/1228568969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f35a402a070 con 0x7f35b41023b0 2024-04-16T05:34:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.021+0000 7f35b17fa700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f359c0a3b20 0x7f359c0a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.021+0000 7f35b3fff700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f359c0a3b20 0x7f359c0a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.021+0000 7f35b17fa700 1 -- 172.21.15.86:0/1228568969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f35a40d1540 con 0x7f35b41023b0 2024-04-16T05:34:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.022+0000 7f35b3fff700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f359c0a3b20 0x7f359c0a5fe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35a8005fd0 tx=0x7f35a800a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:26.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.026+0000 7f35b17fa700 1 -- 172.21.15.86:0/1228568969 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35a405eff0 con 0x7f35b41023b0 2024-04-16T05:34:26.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.229+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f35a0005620 con 0x7f35b41023b0 2024-04-16T05:34:26.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.229+0000 7f35b17fa700 1 -- 172.21.15.86:0/1228568969 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f35a4013070 con 0x7f35b41023b0 2024-04-16T05:34:26.228 INFO:teuthology.orchestra.run.smithi086.stdout:107374182472 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f359c0a3b20 msgr2=0x7f359c0a5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f359c0a3b20 0x7f359c0a5fe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35a8005fd0 tx=0x7f35a800a560 comp rx=0 tx=0).stop 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41023b0 msgr2=0x7f35b4199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41023b0 0x7f35b4199850 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f35a4000c00 tx=0x7f35a4003680 comp rx=0 tx=0).stop 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 shutdown_connections 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b4107a40 0x7f35b419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f359c0a3b20 0x7f359c0a5fe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b41019f0 0x7f35b4199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 --2- 172.21.15.86:0/1228568969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b41023b0 0x7f35b4199850 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 >> 172.21.15.86:0/1228568969 conn(0x7f35b40789a0 msgr2=0x7f35b406ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 shutdown_connections 2024-04-16T05:34:26.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:26.231+0000 7f35ba904700 1 -- 172.21.15.86:0/1228568969 wait complete. 2024-04-16T05:34:26.466 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738455 for osd.0 2024-04-16T05:34:26.466 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:26 smithi194 ceph-mon[35865]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-16T05:34:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:26 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1228568969' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T05:34:26.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:26 smithi086 ceph-mon[32028]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-16T05:34:26.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:26 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1228568969' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T05:34:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:26 smithi196 ceph-mon[36373]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-16T05:34:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:26 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1228568969' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T05:34:27.111 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574930 for osd.1 2024-04-16T05:34:27.112 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:27.948 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182467 got 107374182472 for osd.3 2024-04-16T05:34:27.948 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:28.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495672 for osd.6 2024-04-16T05:34:28.421 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:28.421 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-16T05:34:28.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph pg dump --format=json 2024-04-16T05:34:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:34:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:28 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:34:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:28 smithi196 ceph-mon[36373]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-16T05:34:28.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:34:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:28 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:34:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:28 smithi194 ceph-mon[35865]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-16T05:34:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:28.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:34:28.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:28 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:34:28.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:28 smithi086 ceph-mon[32028]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-16T05:34:29.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:34:29.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:34:29.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:29 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:34:29.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:29 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:34:29.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:29 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:34:29.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:29 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:34:29.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:29 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:34:29.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:29 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:34:29.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:29 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' 2024-04-16T05:34:30.521 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:30 smithi086 ceph-mon[32028]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-16T05:34:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.533+0000 7fe007c0d700 1 -- 172.21.15.86:0/2253562065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 msgr2=0x7fe000107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.533+0000 7fe007c0d700 1 --2- 172.21.15.86:0/2253562065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 0x7fe000107fb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009990 tx=0x7fdffc009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.533+0000 7fe007c0d700 1 -- 172.21.15.86:0/2253562065 shutdown_connections 2024-04-16T05:34:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.533+0000 7fe007c0d700 1 --2- 172.21.15.86:0/2253562065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe000101e70 0x7fe00010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.533+0000 7fe007c0d700 1 --2- 172.21.15.86:0/2253562065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 0x7fe000107fb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.533+0000 7fe007c0d700 1 --2- 172.21.15.86:0/2253562065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0001014b0 0x7fe000101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.533+0000 7fe007c0d700 1 -- 172.21.15.86:0/2253562065 >> 172.21.15.86:0/2253562065 conn(0x7fe000076d10 msgr2=0x7fe000077120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.533+0000 7fe007c0d700 1 -- 172.21.15.86:0/2253562065 shutdown_connections 2024-04-16T05:34:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.533+0000 7fe007c0d700 1 -- 172.21.15.86:0/2253562065 wait complete. 2024-04-16T05:34:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe007c0d700 1 Processor -- start 2024-04-16T05:34:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe007c0d700 1 -- start start 2024-04-16T05:34:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe007c0d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0001014b0 0x7fe000102690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe007c0d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe000101e70 0x7fe000102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe007c0d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 0x7fe000103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe007c0d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe000112f10 con 0x7fe000107bd0 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe007c0d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe000112d90 con 0x7fe0001014b0 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe007c0d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe000113090 con 0x7fe000101e70 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0061aa700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 0x7fe000103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0061aa700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 0x7fe000103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60758/0 (socket says 172.21.15.86:60758) 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0061aa700 1 -- 172.21.15.86:0/1454270515 learned_addr learned my addr 172.21.15.86:0/1454270515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0059a9700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0001014b0 0x7fe000102690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0051a8700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe000101e70 0x7fe000102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0061aa700 1 -- 172.21.15.86:0/1454270515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe000101e70 msgr2=0x7fe000102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0061aa700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe000101e70 0x7fe000102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0061aa700 1 -- 172.21.15.86:0/1454270515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0001014b0 msgr2=0x7fe000102690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0061aa700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0001014b0 0x7fe000102690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.534+0000 7fe0061aa700 1 -- 172.21.15.86:0/1454270515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0001a79e0 con 0x7fe000107bd0 2024-04-16T05:34:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.535+0000 7fe0061aa700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 0x7fe000103110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdff800b970 tx=0x7fdff800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.535+0000 7fdff6ffd700 1 -- 172.21.15.86:0/1454270515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff800d860 con 0x7fe000107bd0 2024-04-16T05:34:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.535+0000 7fdff6ffd700 1 -- 172.21.15.86:0/1454270515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff800e430 con 0x7fe000107bd0 2024-04-16T05:34:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.535+0000 7fdff6ffd700 1 -- 172.21.15.86:0/1454270515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8017660 con 0x7fe000107bd0 2024-04-16T05:34:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.535+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0001a7cd0 con 0x7fe000107bd0 2024-04-16T05:34:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.535+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0001a8210 con 0x7fe000107bd0 2024-04-16T05:34:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.536+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0000697e0 con 0x7fe000107bd0 2024-04-16T05:34:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.537+0000 7fdff6ffd700 1 -- 172.21.15.86:0/1454270515 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdff800e5e0 con 0x7fe000107bd0 2024-04-16T05:34:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.537+0000 7fdff6ffd700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdfec0a3c40 0x7fdfec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.538+0000 7fe0059a9700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdfec0a3c40 0x7fdfec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.538+0000 7fdff6ffd700 1 -- 172.21.15.86:0/1454270515 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdff80d02d0 con 0x7fe000107bd0 2024-04-16T05:34:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.538+0000 7fe0059a9700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdfec0a3c40 0x7fdfec0a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdffc006010 tx=0x7fdffc005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:30.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.542+0000 7fdff6ffd700 1 -- 172.21.15.86:0/1454270515 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdff8013030 con 0x7fe000107bd0 2024-04-16T05:34:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:30 smithi194 ceph-mon[35865]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-16T05:34:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:30 smithi196 ceph-mon[36373]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-16T05:34:30.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.738+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe000062f30 con 0x7fdfec0a3c40 2024-04-16T05:34:30.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.747+0000 7fdff6ffd700 1 -- 172.21.15.86:0/1454270515 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340899 (secure 0 0 0) 0x7fe000062f30 con 0x7fdfec0a3c40 2024-04-16T05:34:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:34:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdfec0a3c40 msgr2=0x7fdfec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdfec0a3c40 0x7fdfec0a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdffc006010 tx=0x7fdffc005960 comp rx=0 tx=0).stop 2024-04-16T05:34:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 msgr2=0x7fe000103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 0x7fe000103110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdff800b970 tx=0x7fdff800bc80 comp rx=0 tx=0).stop 2024-04-16T05:34:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 shutdown_connections 2024-04-16T05:34:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe000107bd0 0x7fe000103110 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0001014b0 0x7fe000102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fdfec0a3c40 0x7fdfec0a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 --2- 172.21.15.86:0/1454270515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe000101e70 0x7fe000102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 >> 172.21.15.86:0/1454270515 conn(0x7fe000076d10 msgr2=0x7fe0001000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 shutdown_connections 2024-04-16T05:34:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:30.751+0000 7fe007c0d700 1 -- 172.21.15.86:0/1454270515 wait complete. 2024-04-16T05:34:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-16T05:34:31.522 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-04-16T05:34:29.502371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660059,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":852,"num_read_kb":556,"num_write":436,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369912,"kb_used_data":12440,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571912,"statfs":{"total":1151940427776,"available":1151561637888,"internally_reserved":0,"allocated":12738560,"data_stored":8079059,"data_compressed":389838,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":59,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003348"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511478+0000","last_change":"2024-04-16T05:32:55.507548+0000","last_active":"2024-04-16T05:34:21.511478+0000","last_peered":"2024-04-16T05:34:21.511478+0000","last_clean":"2024-04-16T05:34:21.511478+0000","last_became_active":"2024-04-16T05:32:55.507200+0000","last_became_peered":"2024-04-16T05:32:55.507200+0000","last_unstale":"2024-04-16T05:34:21.511478+0000","last_undegraded":"2024-04-16T05:34:21.511478+0000","last_fullsized":"2024-04-16T05:34:21.511478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:58:07.842226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499106+0000","last_change":"2024-04-16T05:32:55.509792+0000","last_active":"2024-04-16T05:34:21.499106+0000","last_peered":"2024-04-16T05:34:21.499106+0000","last_clean":"2024-04-16T05:34:21.499106+0000","last_became_active":"2024-04-16T05:32:55.509442+0000","last_became_peered":"2024-04-16T05:32:55.509442+0000","last_unstale":"2024-04-16T05:34:21.499106+0000","last_undegraded":"2024-04-16T05:34:21.499106+0000","last_fullsized":"2024-04-16T05:34:21.499106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:08:42.040875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511012+0000","last_change":"2024-04-16T05:32:55.505664+0000","last_active":"2024-04-16T05:34:21.511012+0000","last_peered":"2024-04-16T05:34:21.511012+0000","last_clean":"2024-04-16T05:34:21.511012+0000","last_became_active":"2024-04-16T05:32:55.505281+0000","last_became_peered":"2024-04-16T05:32:55.505281+0000","last_unstale":"2024-04-16T05:34:21.511012+0000","last_undegraded":"2024-04-16T05:34:21.511012+0000","last_fullsized":"2024-04-16T05:34:21.511012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:33.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.500029+0000","last_change":"2024-04-16T05:32:55.517072+0000","last_active":"2024-04-16T05:34:21.500029+0000","last_peered":"2024-04-16T05:34:21.500029+0000","last_clean":"2024-04-16T05:34:21.500029+0000","last_became_active":"2024-04-16T05:32:55.516648+0000","last_became_peered":"2024-04-16T05:32:55.516648+0000","last_unstale":"2024-04-16T05:34:21.500029+0000","last_undegraded":"2024-04-16T05:34:21.500029+0000","last_fullsized":"2024-04-16T05:34:21.500029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:41.735720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.489505+0000","last_change":"2024-04-16T05:32:55.506773+0000","last_active":"2024-04-16T05:34:21.489505+0000","last_peered":"2024-04-16T05:34:21.489505+0000","last_clean":"2024-04-16T05:34:21.489505+0000","last_became_active":"2024-04-16T05:32:55.506414+0000","last_became_peered":"2024-04-16T05:32:55.506414+0000","last_unstale":"2024-04-16T05:34:21.489505+0000","last_undegraded":"2024-04-16T05:34:21.489505+0000","last_fullsized":"2024-04-16T05:34:21.489505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:59.254334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.483709+0000","last_change":"2024-04-16T05:32:55.517875+0000","last_active":"2024-04-16T05:34:21.483709+0000","last_peered":"2024-04-16T05:34:21.483709+0000","last_clean":"2024-04-16T05:34:21.483709+0000","last_became_active":"2024-04-16T05:32:55.517546+0000","last_became_peered":"2024-04-16T05:32:55.517546+0000","last_unstale":"2024-04-16T05:34:21.483709+0000","last_undegraded":"2024-04-16T05:34:21.483709+0000","last_fullsized":"2024-04-16T05:34:21.483709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:12:15.098439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501713+0000","last_change":"2024-04-16T05:32:55.509036+0000","last_active":"2024-04-16T05:34:21.501713+0000","last_peered":"2024-04-16T05:34:21.501713+0000","last_clean":"2024-04-16T05:34:21.501713+0000","last_became_active":"2024-04-16T05:32:55.508690+0000","last_became_peered":"2024-04-16T05:32:55.508690+0000","last_unstale":"2024-04-16T05:34:21.501713+0000","last_undegraded":"2024-04-16T05:34:21.501713+0000","last_fullsized":"2024-04-16T05:34:21.501713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:26.724281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506330+0000","last_change":"2024-04-16T05:32:55.510146+0000","last_active":"2024-04-16T05:34:21.506330+0000","last_peered":"2024-04-16T05:34:21.506330+0000","last_clean":"2024-04-16T05:34:21.506330+0000","last_became_active":"2024-04-16T05:32:55.508665+0000","last_became_peered":"2024-04-16T05:32:55.508665+0000","last_unstale":"2024-04-16T05:34:21.506330+0000","last_undegraded":"2024-04-16T05:34:21.506330+0000","last_fullsized":"2024-04-16T05:34:21.506330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:14.158221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.512368+0000","last_change":"2024-04-16T05:32:55.513738+0000","last_active":"2024-04-16T05:34:21.512368+0000","last_peered":"2024-04-16T05:34:21.512368+0000","last_clean":"2024-04-16T05:34:21.512368+0000","last_became_active":"2024-04-16T05:32:55.513374+0000","last_became_peered":"2024-04-16T05:32:55.513374+0000","last_unstale":"2024-04-16T05:34:21.512368+0000","last_undegraded":"2024-04-16T05:34:21.512368+0000","last_fullsized":"2024-04-16T05:34:21.512368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:06:23.390908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499357+0000","last_change":"2024-04-16T05:32:55.519276+0000","last_active":"2024-04-16T05:34:21.499357+0000","last_peered":"2024-04-16T05:34:21.499357+0000","last_clean":"2024-04-16T05:34:21.499357+0000","last_became_active":"2024-04-16T05:32:55.518729+0000","last_became_peered":"2024-04-16T05:32:55.518729+0000","last_unstale":"2024-04-16T05:34:21.499357+0000","last_undegraded":"2024-04-16T05:34:21.499357+0000","last_fullsized":"2024-04-16T05:34:21.499357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:44:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506133+0000","last_change":"2024-04-16T05:32:55.516379+0000","last_active":"2024-04-16T05:34:21.506133+0000","last_peered":"2024-04-16T05:34:21.506133+0000","last_clean":"2024-04-16T05:34:21.506133+0000","last_became_active":"2024-04-16T05:32:55.515574+0000","last_became_peered":"2024-04-16T05:32:55.515574+0000","last_unstale":"2024-04-16T05:34:21.506133+0000","last_undegraded":"2024-04-16T05:34:21.506133+0000","last_fullsized":"2024-04-16T05:34:21.506133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:16:01.744380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501104+0000","last_change":"2024-04-16T05:33:52.674399+0000","last_active":"2024-04-16T05:34:21.501104+0000","last_peered":"2024-04-16T05:34:21.501104+0000","last_clean":"2024-04-16T05:34:21.501104+0000","last_became_active":"2024-04-16T05:33:52.673964+0000","last_became_peered":"2024-04-16T05:33:52.673964+0000","last_unstale":"2024-04-16T05:34:21.501104+0000","last_undegraded":"2024-04-16T05:34:21.501104+0000","last_fullsized":"2024-04-16T05:34:21.501104+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.166747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501629+0000","last_change":"2024-04-16T05:32:55.509682+0000","last_active":"2024-04-16T05:34:21.501629+0000","last_peered":"2024-04-16T05:34:21.501629+0000","last_clean":"2024-04-16T05:34:21.501629+0000","last_became_active":"2024-04-16T05:32:55.509383+0000","last_became_peered":"2024-04-16T05:32:55.509383+0000","last_unstale":"2024-04-16T05:34:21.501629+0000","last_undegraded":"2024-04-16T05:34:21.501629+0000","last_fullsized":"2024-04-16T05:34:21.501629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:25:23.487003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499529+0000","last_change":"2024-04-16T05:32:55.506338+0000","last_active":"2024-04-16T05:34:21.499529+0000","last_peered":"2024-04-16T05:34:21.499529+0000","last_clean":"2024-04-16T05:34:21.499529+0000","last_became_active":"2024-04-16T05:32:55.506000+0000","last_became_peered":"2024-04-16T05:32:55.506000+0000","last_unstale":"2024-04-16T05:34:21.499529+0000","last_undegraded":"2024-04-16T05:34:21.499529+0000","last_fullsized":"2024-04-16T05:34:21.499529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:37:02.296515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501253+0000","last_change":"2024-04-16T05:32:55.516897+0000","last_active":"2024-04-16T05:34:21.501253+0000","last_peered":"2024-04-16T05:34:21.501253+0000","last_clean":"2024-04-16T05:34:21.501253+0000","last_became_active":"2024-04-16T05:32:55.516489+0000","last_became_peered":"2024-04-16T05:32:55.516489+0000","last_unstale":"2024-04-16T05:34:21.501253+0000","last_undegraded":"2024-04-16T05:34:21.501253+0000","last_fullsized":"2024-04-16T05:34:21.501253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:26:08.289501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511131+0000","last_change":"2024-04-16T05:32:55.507374+0000","last_active":"2024-04-16T05:34:21.511131+0000","last_peered":"2024-04-16T05:34:21.511131+0000","last_clean":"2024-04-16T05:34:21.511131+0000","last_became_active":"2024-04-16T05:32:55.507045+0000","last_became_peered":"2024-04-16T05:32:55.507045+0000","last_unstale":"2024-04-16T05:34:21.511131+0000","last_undegraded":"2024-04-16T05:34:21.511131+0000","last_fullsized":"2024-04-16T05:34:21.511131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:35.968704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511583+0000","last_change":"2024-04-16T05:32:55.508845+0000","last_active":"2024-04-16T05:34:21.511583+0000","last_peered":"2024-04-16T05:34:21.511583+0000","last_clean":"2024-04-16T05:34:21.511583+0000","last_became_active":"2024-04-16T05:32:55.508527+0000","last_became_peered":"2024-04-16T05:32:55.508527+0000","last_unstale":"2024-04-16T05:34:21.511583+0000","last_undegraded":"2024-04-16T05:34:21.511583+0000","last_fullsized":"2024-04-16T05:34:21.511583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:20:15.033043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.510628+0000","last_change":"2024-04-16T05:32:55.506059+0000","last_active":"2024-04-16T05:34:21.510628+0000","last_peered":"2024-04-16T05:34:21.510628+0000","last_clean":"2024-04-16T05:34:21.510628+0000","last_became_active":"2024-04-16T05:32:55.505728+0000","last_became_peered":"2024-04-16T05:32:55.505728+0000","last_unstale":"2024-04-16T05:34:21.510628+0000","last_undegraded":"2024-04-16T05:34:21.510628+0000","last_fullsized":"2024-04-16T05:34:21.510628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:19.253543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.503252+0000","last_change":"2024-04-16T05:32:55.506791+0000","last_active":"2024-04-16T05:34:21.503252+0000","last_peered":"2024-04-16T05:34:21.503252+0000","last_clean":"2024-04-16T05:34:21.503252+0000","last_became_active":"2024-04-16T05:32:55.506379+0000","last_became_peered":"2024-04-16T05:32:55.506379+0000","last_unstale":"2024-04-16T05:34:21.503252+0000","last_undegraded":"2024-04-16T05:34:21.503252+0000","last_fullsized":"2024-04-16T05:34:21.503252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:09:11.457144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.582503+0000","last_change":"2024-04-16T05:33:52.582503+0000","last_active":"2024-04-16T05:33:52.582503+0000","last_peered":"2024-04-16T05:33:52.582503+0000","last_clean":"2024-04-16T05:33:52.582503+0000","last_became_active":"2024-04-16T05:33:52.581996+0000","last_became_peered":"2024-04-16T05:33:52.581996+0000","last_unstale":"2024-04-16T05:33:52.582503+0000","last_undegraded":"2024-04-16T05:33:52.582503+0000","last_fullsized":"2024-04-16T05:33:52.582503+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:34:40.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":114,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:27.715903+0000","last_change":"2024-04-16T05:32:54.504826+0000","last_active":"2024-04-16T05:34:27.715903+0000","last_peered":"2024-04-16T05:34:27.715903+0000","last_clean":"2024-04-16T05:34:27.715903+0000","last_became_active":"2024-04-16T05:32:54.504526+0000","last_became_peered":"2024-04-16T05:32:54.504526+0000","last_unstale":"2024-04-16T05:34:27.715903+0000","last_undegraded":"2024-04-16T05:34:27.715903+0000","last_fullsized":"2024-04-16T05:34:27.715903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:21:28.434692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.496238+0000","last_change":"2024-04-16T05:33:52.580931+0000","last_active":"2024-04-16T05:34:21.496238+0000","last_peered":"2024-04-16T05:34:21.496238+0000","last_clean":"2024-04-16T05:34:21.496238+0000","last_became_active":"2024-04-16T05:33:52.580685+0000","last_became_peered":"2024-04-16T05:33:52.580685+0000","last_unstale":"2024-04-16T05:34:21.496238+0000","last_undegraded":"2024-04-16T05:34:21.496238+0000","last_fullsized":"2024-04-16T05:34:21.496238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:35:34.548838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575474+0000","last_change":"2024-04-16T05:32:59.524687+0000","last_active":"2024-04-16T05:33:52.575474+0000","last_peered":"2024-04-16T05:33:52.575474+0000","last_clean":"2024-04-16T05:33:52.575474+0000","last_became_active":"2024-04-16T05:32:59.524228+0000","last_became_peered":"2024-04-16T05:32:59.524228+0000","last_unstale":"2024-04-16T05:33:52.575474+0000","last_undegraded":"2024-04-16T05:33:52.575474+0000","last_fullsized":"2024-04-16T05:33:52.575474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:23:15.244359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579070+0000","last_change":"2024-04-16T05:32:54.504062+0000","last_active":"2024-04-16T05:33:52.579070+0000","last_peered":"2024-04-16T05:33:52.579070+0000","last_clean":"2024-04-16T05:33:52.579070+0000","last_became_active":"2024-04-16T05:32:54.503745+0000","last_became_peered":"2024-04-16T05:32:54.503745+0000","last_unstale":"2024-04-16T05:33:52.579070+0000","last_undegraded":"2024-04-16T05:33:52.579070+0000","last_fullsized":"2024-04-16T05:33:52.579070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:29:34.157937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511122+0000","last_change":"2024-04-16T05:32:55.509820+0000","last_active":"2024-04-16T05:34:21.511122+0000","last_peered":"2024-04-16T05:34:21.511122+0000","last_clean":"2024-04-16T05:34:21.511122+0000","last_became_active":"2024-04-16T05:32:55.509473+0000","last_became_peered":"2024-04-16T05:32:55.509473+0000","last_unstale":"2024-04-16T05:34:21.511122+0000","last_undegraded":"2024-04-16T05:34:21.511122+0000","last_fullsized":"2024-04-16T05:34:21.511122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:12:22.433189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.678903+0000","last_change":"2024-04-16T05:33:52.678903+0000","last_active":"2024-04-16T05:33:52.678903+0000","last_peered":"2024-04-16T05:33:52.678903+0000","last_clean":"2024-04-16T05:33:52.678903+0000","last_became_active":"2024-04-16T05:33:52.678388+0000","last_became_peered":"2024-04-16T05:33:52.678388+0000","last_unstale":"2024-04-16T05:33:52.678903+0000","last_undegraded":"2024-04-16T05:33:52.678903+0000","last_fullsized":"2024-04-16T05:33:52.678903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:07:01.060645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576294+0000","last_change":"2024-04-16T05:32:54.511469+0000","last_active":"2024-04-16T05:33:52.576294+0000","last_peered":"2024-04-16T05:33:52.576294+0000","last_clean":"2024-04-16T05:33:52.576294+0000","last_became_active":"2024-04-16T05:32:54.511120+0000","last_became_peered":"2024-04-16T05:32:54.511120+0000","last_unstale":"2024-04-16T05:33:52.576294+0000","last_undegraded":"2024-04-16T05:33:52.576294+0000","last_fullsized":"2024-04-16T05:33:52.576294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.508406+0000","last_change":"2024-04-16T05:32:55.512308+0000","last_active":"2024-04-16T05:34:21.508406+0000","last_peered":"2024-04-16T05:34:21.508406+0000","last_clean":"2024-04-16T05:34:21.508406+0000","last_became_active":"2024-04-16T05:32:55.511625+0000","last_became_peered":"2024-04-16T05:32:55.511625+0000","last_unstale":"2024-04-16T05:34:21.508406+0000","last_undegraded":"2024-04-16T05:34:21.508406+0000","last_fullsized":"2024-04-16T05:34:21.508406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:29:49.323092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575546+0000","last_change":"2024-04-16T05:32:59.519306+0000","last_active":"2024-04-16T05:33:52.575546+0000","last_peered":"2024-04-16T05:33:52.575546+0000","last_clean":"2024-04-16T05:33:52.575546+0000","last_became_active":"2024-04-16T05:32:59.518797+0000","last_became_peered":"2024-04-16T05:32:59.518797+0000","last_unstale":"2024-04-16T05:33:52.575546+0000","last_undegraded":"2024-04-16T05:33:52.575546+0000","last_fullsized":"2024-04-16T05:33:52.575546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:04:12.389004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575820+0000","last_change":"2024-04-16T05:32:54.505102+0000","last_active":"2024-04-16T05:33:52.575820+0000","last_peered":"2024-04-16T05:33:52.575820+0000","last_clean":"2024-04-16T05:33:52.575820+0000","last_became_active":"2024-04-16T05:32:54.504806+0000","last_became_peered":"2024-04-16T05:32:54.504806+0000","last_unstale":"2024-04-16T05:33:52.575820+0000","last_undegraded":"2024-04-16T05:33:52.575820+0000","last_fullsized":"2024-04-16T05:33:52.575820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:19.467690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.512005+0000","last_change":"2024-04-16T05:32:55.517097+0000","last_active":"2024-04-16T05:34:21.512005+0000","last_peered":"2024-04-16T05:34:21.512005+0000","last_clean":"2024-04-16T05:34:21.512005+0000","last_became_active":"2024-04-16T05:32:55.516717+0000","last_became_peered":"2024-04-16T05:32:55.516717+0000","last_unstale":"2024-04-16T05:34:21.512005+0000","last_undegraded":"2024-04-16T05:34:21.512005+0000","last_fullsized":"2024-04-16T05:34:21.512005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:10:44.544935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577331+0000","last_change":"2024-04-16T05:32:59.521429+0000","last_active":"2024-04-16T05:33:52.577331+0000","last_peered":"2024-04-16T05:33:52.577331+0000","last_clean":"2024-04-16T05:33:52.577331+0000","last_became_active":"2024-04-16T05:32:59.520789+0000","last_became_peered":"2024-04-16T05:32:59.520789+0000","last_unstale":"2024-04-16T05:33:52.577331+0000","last_undegraded":"2024-04-16T05:33:52.577331+0000","last_fullsized":"2024-04-16T05:33:52.577331+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:40:14.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:22.717148+0000","last_change":"2024-04-16T05:32:54.507374+0000","last_active":"2024-04-16T05:34:22.717148+0000","last_peered":"2024-04-16T05:34:22.717148+0000","last_clean":"2024-04-16T05:34:22.717148+0000","last_became_active":"2024-04-16T05:32:54.507055+0000","last_became_peered":"2024-04-16T05:32:54.507055+0000","last_unstale":"2024-04-16T05:34:22.717148+0000","last_undegraded":"2024-04-16T05:34:22.717148+0000","last_fullsized":"2024-04-16T05:34:22.717148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:00:57.761971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.482151+0000","last_change":"2024-04-16T05:32:55.513980+0000","last_active":"2024-04-16T05:34:21.482151+0000","last_peered":"2024-04-16T05:34:21.482151+0000","last_clean":"2024-04-16T05:34:21.482151+0000","last_became_active":"2024-04-16T05:32:55.513622+0000","last_became_peered":"2024-04-16T05:32:55.513622+0000","last_unstale":"2024-04-16T05:34:21.482151+0000","last_undegraded":"2024-04-16T05:34:21.482151+0000","last_fullsized":"2024-04-16T05:34:21.482151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:16:10.018381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577404+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:33:52.577404+0000","last_peered":"2024-04-16T05:33:52.577404+0000","last_clean":"2024-04-16T05:33:52.577404+0000","last_became_active":"2024-04-16T05:32:59.523032+0000","last_became_peered":"2024-04-16T05:32:59.523032+0000","last_unstale":"2024-04-16T05:33:52.577404+0000","last_undegraded":"2024-04-16T05:33:52.577404+0000","last_fullsized":"2024-04-16T05:33:52.577404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:33:17.897241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.673260+0000","last_change":"2024-04-16T05:32:54.504527+0000","last_active":"2024-04-16T05:33:52.673260+0000","last_peered":"2024-04-16T05:33:52.673260+0000","last_clean":"2024-04-16T05:33:52.673260+0000","last_became_active":"2024-04-16T05:32:54.504187+0000","last_became_peered":"2024-04-16T05:32:54.504187+0000","last_unstale":"2024-04-16T05:33:52.673260+0000","last_undegraded":"2024-04-16T05:33:52.673260+0000","last_fullsized":"2024-04-16T05:33:52.673260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.414362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501955+0000","last_change":"2024-04-16T05:32:55.520270+0000","last_active":"2024-04-16T05:34:21.501955+0000","last_peered":"2024-04-16T05:34:21.501955+0000","last_clean":"2024-04-16T05:34:21.501955+0000","last_became_active":"2024-04-16T05:32:55.519895+0000","last_became_peered":"2024-04-16T05:32:55.519895+0000","last_unstale":"2024-04-16T05:34:21.501955+0000","last_undegraded":"2024-04-16T05:34:21.501955+0000","last_fullsized":"2024-04-16T05:34:21.501955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:09.967836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579270+0000","last_change":"2024-04-16T05:32:59.518997+0000","last_active":"2024-04-16T05:33:52.579270+0000","last_peered":"2024-04-16T05:33:52.579270+0000","last_clean":"2024-04-16T05:33:52.579270+0000","last_became_active":"2024-04-16T05:32:59.518558+0000","last_became_peered":"2024-04-16T05:32:59.518558+0000","last_unstale":"2024-04-16T05:33:52.579270+0000","last_undegraded":"2024-04-16T05:33:52.579270+0000","last_fullsized":"2024-04-16T05:33:52.579270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:04:30.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.671972+0000","last_change":"2024-04-16T05:32:54.502074+0000","last_active":"2024-04-16T05:33:52.671972+0000","last_peered":"2024-04-16T05:33:52.671972+0000","last_clean":"2024-04-16T05:33:52.671972+0000","last_became_active":"2024-04-16T05:32:54.501730+0000","last_became_peered":"2024-04-16T05:32:54.501730+0000","last_unstale":"2024-04-16T05:33:52.671972+0000","last_undegraded":"2024-04-16T05:33:52.671972+0000","last_fullsized":"2024-04-16T05:33:52.671972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:05:02.929799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.507740+0000","last_change":"2024-04-16T05:32:55.504557+0000","last_active":"2024-04-16T05:34:21.507740+0000","last_peered":"2024-04-16T05:34:21.507740+0000","last_clean":"2024-04-16T05:34:21.507740+0000","last_became_active":"2024-04-16T05:32:55.504244+0000","last_became_peered":"2024-04-16T05:32:55.504244+0000","last_unstale":"2024-04-16T05:34:21.507740+0000","last_undegraded":"2024-04-16T05:34:21.507740+0000","last_fullsized":"2024-04-16T05:34:21.507740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:33:40.500436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577138+0000","last_change":"2024-04-16T05:32:59.524681+0000","last_active":"2024-04-16T05:33:52.577138+0000","last_peered":"2024-04-16T05:33:52.577138+0000","last_clean":"2024-04-16T05:33:52.577138+0000","last_became_active":"2024-04-16T05:32:59.524217+0000","last_became_peered":"2024-04-16T05:32:59.524217+0000","last_unstale":"2024-04-16T05:33:52.577138+0000","last_undegraded":"2024-04-16T05:33:52.577138+0000","last_fullsized":"2024-04-16T05:33:52.577138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:28.314442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677063+0000","last_change":"2024-04-16T05:32:54.508242+0000","last_active":"2024-04-16T05:33:52.677063+0000","last_peered":"2024-04-16T05:33:52.677063+0000","last_clean":"2024-04-16T05:33:52.677063+0000","last_became_active":"2024-04-16T05:32:54.507782+0000","last_became_peered":"2024-04-16T05:32:54.507782+0000","last_unstale":"2024-04-16T05:33:52.677063+0000","last_undegraded":"2024-04-16T05:33:52.677063+0000","last_fullsized":"2024-04-16T05:33:52.677063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:27:26.805716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.500686+0000","last_change":"2024-04-16T05:32:55.508985+0000","last_active":"2024-04-16T05:34:21.500686+0000","last_peered":"2024-04-16T05:34:21.500686+0000","last_clean":"2024-04-16T05:34:21.500686+0000","last_became_active":"2024-04-16T05:32:55.508659+0000","last_became_peered":"2024-04-16T05:32:55.508659+0000","last_unstale":"2024-04-16T05:34:21.500686+0000","last_undegraded":"2024-04-16T05:34:21.500686+0000","last_fullsized":"2024-04-16T05:34:21.500686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:43:41.534155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578555+0000","last_change":"2024-04-16T05:32:59.521257+0000","last_active":"2024-04-16T05:33:52.578555+0000","last_peered":"2024-04-16T05:33:52.578555+0000","last_clean":"2024-04-16T05:33:52.578555+0000","last_became_active":"2024-04-16T05:32:59.520848+0000","last_became_peered":"2024-04-16T05:32:59.520848+0000","last_unstale":"2024-04-16T05:33:52.578555+0000","last_undegraded":"2024-04-16T05:33:52.578555+0000","last_fullsized":"2024-04-16T05:33:52.578555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:14.219378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":169,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:24.719372+0000","last_change":"2024-04-16T05:32:54.500684+0000","last_active":"2024-04-16T05:34:24.719372+0000","last_peered":"2024-04-16T05:34:24.719372+0000","last_clean":"2024-04-16T05:34:24.719372+0000","last_became_active":"2024-04-16T05:32:54.500290+0000","last_became_peered":"2024-04-16T05:32:54.500290+0000","last_unstale":"2024-04-16T05:34:24.719372+0000","last_undegraded":"2024-04-16T05:34:24.719372+0000","last_fullsized":"2024-04-16T05:34:24.719372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:20:55.263313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.512532+0000","last_change":"2024-04-16T05:32:55.511224+0000","last_active":"2024-04-16T05:34:21.512532+0000","last_peered":"2024-04-16T05:34:21.512532+0000","last_clean":"2024-04-16T05:34:21.512532+0000","last_became_active":"2024-04-16T05:32:55.510891+0000","last_became_peered":"2024-04-16T05:32:55.510891+0000","last_unstale":"2024-04-16T05:34:21.512532+0000","last_undegraded":"2024-04-16T05:34:21.512532+0000","last_fullsized":"2024-04-16T05:34:21.512532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:27:24.894658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.573962+0000","last_change":"2024-04-16T05:32:59.521695+0000","last_active":"2024-04-16T05:33:52.573962+0000","last_peered":"2024-04-16T05:33:52.573962+0000","last_clean":"2024-04-16T05:33:52.573962+0000","last_became_active":"2024-04-16T05:32:59.520569+0000","last_became_peered":"2024-04-16T05:32:59.520569+0000","last_unstale":"2024-04-16T05:33:52.573962+0000","last_undegraded":"2024-04-16T05:33:52.573962+0000","last_fullsized":"2024-04-16T05:33:52.573962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:01:14.726043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.580829+0000","last_change":"2024-04-16T05:32:54.505058+0000","last_active":"2024-04-16T05:33:52.580829+0000","last_peered":"2024-04-16T05:33:52.580829+0000","last_clean":"2024-04-16T05:33:52.580829+0000","last_became_active":"2024-04-16T05:32:54.504636+0000","last_became_peered":"2024-04-16T05:32:54.504636+0000","last_unstale":"2024-04-16T05:33:52.580829+0000","last_undegraded":"2024-04-16T05:33:52.580829+0000","last_fullsized":"2024-04-16T05:33:52.580829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:49:14.429468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.502136+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:34:21.502136+0000","last_peered":"2024-04-16T05:34:21.502136+0000","last_clean":"2024-04-16T05:34:21.502136+0000","last_became_active":"2024-04-16T05:33:52.678570+0000","last_became_peered":"2024-04-16T05:33:52.678570+0000","last_unstale":"2024-04-16T05:34:21.502136+0000","last_undegraded":"2024-04-16T05:34:21.502136+0000","last_fullsized":"2024-04-16T05:34:21.502136+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:56:32.405491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576513+0000","last_change":"2024-04-16T05:32:59.520394+0000","last_active":"2024-04-16T05:33:52.576513+0000","last_peered":"2024-04-16T05:33:52.576513+0000","last_clean":"2024-04-16T05:33:52.576513+0000","last_became_active":"2024-04-16T05:32:59.519947+0000","last_became_peered":"2024-04-16T05:32:59.519947+0000","last_unstale":"2024-04-16T05:33:52.576513+0000","last_undegraded":"2024-04-16T05:33:52.576513+0000","last_fullsized":"2024-04-16T05:33:52.576513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:33:27.647919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677336+0000","last_change":"2024-04-16T05:32:54.505350+0000","last_active":"2024-04-16T05:33:52.677336+0000","last_peered":"2024-04-16T05:33:52.677336+0000","last_clean":"2024-04-16T05:33:52.677336+0000","last_became_active":"2024-04-16T05:32:54.505011+0000","last_became_peered":"2024-04-16T05:32:54.505011+0000","last_unstale":"2024-04-16T05:33:52.677336+0000","last_undegraded":"2024-04-16T05:33:52.677336+0000","last_fullsized":"2024-04-16T05:33:52.677336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:03:00.276862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.510220+0000","last_change":"2024-04-16T05:33:52.673799+0000","last_active":"2024-04-16T05:34:21.510220+0000","last_peered":"2024-04-16T05:34:21.510220+0000","last_clean":"2024-04-16T05:34:21.510220+0000","last_became_active":"2024-04-16T05:33:52.673344+0000","last_became_peered":"2024-04-16T05:33:52.673344+0000","last_unstale":"2024-04-16T05:34:21.510220+0000","last_undegraded":"2024-04-16T05:34:21.510220+0000","last_fullsized":"2024-04-16T05:34:21.510220+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:07.387912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.574107+0000","last_change":"2024-04-16T05:32:59.519660+0000","last_active":"2024-04-16T05:33:52.574107+0000","last_peered":"2024-04-16T05:33:52.574107+0000","last_clean":"2024-04-16T05:33:52.574107+0000","last_became_active":"2024-04-16T05:32:59.518198+0000","last_became_peered":"2024-04-16T05:32:59.518198+0000","last_unstale":"2024-04-16T05:33:52.574107+0000","last_undegraded":"2024-04-16T05:33:52.574107+0000","last_fullsized":"2024-04-16T05:33:52.574107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:03:10.096252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":207,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:24.718579+0000","last_change":"2024-04-16T05:32:54.501530+0000","last_active":"2024-04-16T05:34:24.718579+0000","last_peered":"2024-04-16T05:34:24.718579+0000","last_clean":"2024-04-16T05:34:24.718579+0000","last_became_active":"2024-04-16T05:32:54.501159+0000","last_became_peered":"2024-04-16T05:32:54.501159+0000","last_unstale":"2024-04-16T05:34:24.718579+0000","last_undegraded":"2024-04-16T05:34:24.718579+0000","last_fullsized":"2024-04-16T05:34:24.718579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:26:22.240137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61439,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":141,"num_read_kb":131,"num_write":39,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.475321+0000","last_change":"2024-04-16T05:32:55.512210+0000","last_active":"2024-04-16T05:34:21.475321+0000","last_peered":"2024-04-16T05:34:21.475321+0000","last_clean":"2024-04-16T05:34:21.475321+0000","last_became_active":"2024-04-16T05:32:55.511672+0000","last_became_peered":"2024-04-16T05:32:55.511672+0000","last_unstale":"2024-04-16T05:34:21.475321+0000","last_undegraded":"2024-04-16T05:34:21.475321+0000","last_fullsized":"2024-04-16T05:34:21.475321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:10:53.755803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677945+0000","last_change":"2024-04-16T05:32:59.527552+0000","last_active":"2024-04-16T05:33:52.677945+0000","last_peered":"2024-04-16T05:33:52.677945+0000","last_clean":"2024-04-16T05:33:52.677945+0000","last_became_active":"2024-04-16T05:32:59.526919+0000","last_became_peered":"2024-04-16T05:32:59.526919+0000","last_unstale":"2024-04-16T05:33:52.677945+0000","last_undegraded":"2024-04-16T05:33:52.677945+0000","last_fullsized":"2024-04-16T05:33:52.677945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:00:46.342383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577328+0000","last_change":"2024-04-16T05:32:54.494589+0000","last_active":"2024-04-16T05:33:52.577328+0000","last_peered":"2024-04-16T05:33:52.577328+0000","last_clean":"2024-04-16T05:33:52.577328+0000","last_became_active":"2024-04-16T05:32:54.494221+0000","last_became_peered":"2024-04-16T05:32:54.494221+0000","last_unstale":"2024-04-16T05:33:52.577328+0000","last_undegraded":"2024-04-16T05:33:52.577328+0000","last_fullsized":"2024-04-16T05:33:52.577328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:17:53.245359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.508841+0000","last_change":"2024-04-16T05:32:55.502724+0000","last_active":"2024-04-16T05:34:21.508841+0000","last_peered":"2024-04-16T05:34:21.508841+0000","last_clean":"2024-04-16T05:34:21.508841+0000","last_became_active":"2024-04-16T05:32:55.502415+0000","last_became_peered":"2024-04-16T05:32:55.502415+0000","last_unstale":"2024-04-16T05:34:21.508841+0000","last_undegraded":"2024-04-16T05:34:21.508841+0000","last_fullsized":"2024-04-16T05:34:21.508841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:20:44.443926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578919+0000","last_change":"2024-04-16T05:32:59.522243+0000","last_active":"2024-04-16T05:33:52.578919+0000","last_peered":"2024-04-16T05:33:52.578919+0000","last_clean":"2024-04-16T05:33:52.578919+0000","last_became_active":"2024-04-16T05:32:59.521823+0000","last_became_peered":"2024-04-16T05:32:59.521823+0000","last_unstale":"2024-04-16T05:33:52.578919+0000","last_undegraded":"2024-04-16T05:33:52.578919+0000","last_fullsized":"2024-04-16T05:33:52.578919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:09:11.158434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677065+0000","last_change":"2024-04-16T05:32:54.507505+0000","last_active":"2024-04-16T05:33:52.677065+0000","last_peered":"2024-04-16T05:33:52.677065+0000","last_clean":"2024-04-16T05:33:52.677065+0000","last_became_active":"2024-04-16T05:32:54.506558+0000","last_became_peered":"2024-04-16T05:32:54.506558+0000","last_unstale":"2024-04-16T05:33:52.677065+0000","last_undegraded":"2024-04-16T05:33:52.677065+0000","last_fullsized":"2024-04-16T05:33:52.677065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:36:17.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.508547+0000","last_change":"2024-04-16T05:32:55.518998+0000","last_active":"2024-04-16T05:34:21.508547+0000","last_peered":"2024-04-16T05:34:21.508547+0000","last_clean":"2024-04-16T05:34:21.508547+0000","last_became_active":"2024-04-16T05:32:55.518640+0000","last_became_peered":"2024-04-16T05:32:55.518640+0000","last_unstale":"2024-04-16T05:34:21.508547+0000","last_undegraded":"2024-04-16T05:34:21.508547+0000","last_fullsized":"2024-04-16T05:34:21.508547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:42:44.667504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.574346+0000","last_change":"2024-04-16T05:32:59.519050+0000","last_active":"2024-04-16T05:33:52.574346+0000","last_peered":"2024-04-16T05:33:52.574346+0000","last_clean":"2024-04-16T05:33:52.574346+0000","last_became_active":"2024-04-16T05:32:59.518663+0000","last_became_peered":"2024-04-16T05:32:59.518663+0000","last_unstale":"2024-04-16T05:33:52.574346+0000","last_undegraded":"2024-04-16T05:33:52.574346+0000","last_fullsized":"2024-04-16T05:33:52.574346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:36:19.649605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:24.721602+0000","last_change":"2024-04-16T05:32:54.500940+0000","last_active":"2024-04-16T05:34:24.721602+0000","last_peered":"2024-04-16T05:34:24.721602+0000","last_clean":"2024-04-16T05:34:24.721602+0000","last_became_active":"2024-04-16T05:32:54.500542+0000","last_became_peered":"2024-04-16T05:32:54.500542+0000","last_unstale":"2024-04-16T05:34:24.721602+0000","last_undegraded":"2024-04-16T05:34:24.721602+0000","last_fullsized":"2024-04-16T05:34:24.721602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:38:00.232244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506632+0000","last_change":"2024-04-16T05:32:55.509889+0000","last_active":"2024-04-16T05:34:21.506632+0000","last_peered":"2024-04-16T05:34:21.506632+0000","last_clean":"2024-04-16T05:34:21.506632+0000","last_became_active":"2024-04-16T05:32:55.508911+0000","last_became_peered":"2024-04-16T05:32:55.508911+0000","last_unstale":"2024-04-16T05:34:21.506632+0000","last_undegraded":"2024-04-16T05:34:21.506632+0000","last_fullsized":"2024-04-16T05:34:21.506632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:14:46.609127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.672428+0000","last_change":"2024-04-16T05:32:59.523555+0000","last_active":"2024-04-16T05:33:52.672428+0000","last_peered":"2024-04-16T05:33:52.672428+0000","last_clean":"2024-04-16T05:33:52.672428+0000","last_became_active":"2024-04-16T05:32:59.523187+0000","last_became_peered":"2024-04-16T05:32:59.523187+0000","last_unstale":"2024-04-16T05:33:52.672428+0000","last_undegraded":"2024-04-16T05:33:52.672428+0000","last_fullsized":"2024-04-16T05:33:52.672428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:35.049355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575135+0000","last_change":"2024-04-16T05:32:54.503247+0000","last_active":"2024-04-16T05:33:52.575135+0000","last_peered":"2024-04-16T05:33:52.575135+0000","last_clean":"2024-04-16T05:33:52.575135+0000","last_became_active":"2024-04-16T05:32:54.502927+0000","last_became_peered":"2024-04-16T05:32:54.502927+0000","last_unstale":"2024-04-16T05:33:52.575135+0000","last_undegraded":"2024-04-16T05:33:52.575135+0000","last_fullsized":"2024-04-16T05:33:52.575135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:36:17.585827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.504335+0000","last_change":"2024-04-16T05:32:55.519265+0000","last_active":"2024-04-16T05:34:21.504335+0000","last_peered":"2024-04-16T05:34:21.504335+0000","last_clean":"2024-04-16T05:34:21.504335+0000","last_became_active":"2024-04-16T05:32:55.518840+0000","last_became_peered":"2024-04-16T05:32:55.518840+0000","last_unstale":"2024-04-16T05:34:21.504335+0000","last_undegraded":"2024-04-16T05:34:21.504335+0000","last_fullsized":"2024-04-16T05:34:21.504335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:56.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677057+0000","last_change":"2024-04-16T05:32:59.525275+0000","last_active":"2024-04-16T05:33:52.677057+0000","last_peered":"2024-04-16T05:33:52.677057+0000","last_clean":"2024-04-16T05:33:52.677057+0000","last_became_active":"2024-04-16T05:32:59.524546+0000","last_became_peered":"2024-04-16T05:32:59.524546+0000","last_unstale":"2024-04-16T05:33:52.677057+0000","last_undegraded":"2024-04-16T05:33:52.677057+0000","last_fullsized":"2024-04-16T05:33:52.677057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:07.561898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576966+0000","last_change":"2024-04-16T05:32:54.506904+0000","last_active":"2024-04-16T05:33:52.576966+0000","last_peered":"2024-04-16T05:33:52.576966+0000","last_clean":"2024-04-16T05:33:52.576966+0000","last_became_active":"2024-04-16T05:32:54.505654+0000","last_became_peered":"2024-04-16T05:32:54.505654+0000","last_unstale":"2024-04-16T05:33:52.576966+0000","last_undegraded":"2024-04-16T05:33:52.576966+0000","last_fullsized":"2024-04-16T05:33:52.576966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:30.444817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499938+0000","last_change":"2024-04-16T05:32:55.515685+0000","last_active":"2024-04-16T05:34:21.499938+0000","last_peered":"2024-04-16T05:34:21.499938+0000","last_clean":"2024-04-16T05:34:21.499938+0000","last_became_active":"2024-04-16T05:32:55.515367+0000","last_became_peered":"2024-04-16T05:32:55.515367+0000","last_unstale":"2024-04-16T05:34:21.499938+0000","last_undegraded":"2024-04-16T05:34:21.499938+0000","last_fullsized":"2024-04-16T05:34:21.499938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:46.670143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575592+0000","last_change":"2024-04-16T05:33:00.419828+0000","last_active":"2024-04-16T05:33:52.575592+0000","last_peered":"2024-04-16T05:33:52.575592+0000","last_clean":"2024-04-16T05:33:52.575592+0000","last_became_active":"2024-04-16T05:33:00.419321+0000","last_became_peered":"2024-04-16T05:33:00.419321+0000","last_unstale":"2024-04-16T05:33:52.575592+0000","last_undegraded":"2024-04-16T05:33:52.575592+0000","last_fullsized":"2024-04-16T05:33:52.575592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:56.694914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.574373+0000","last_change":"2024-04-16T05:32:54.505369+0000","last_active":"2024-04-16T05:33:52.574373+0000","last_peered":"2024-04-16T05:33:52.574373+0000","last_clean":"2024-04-16T05:33:52.574373+0000","last_became_active":"2024-04-16T05:32:54.505043+0000","last_became_peered":"2024-04-16T05:32:54.505043+0000","last_unstale":"2024-04-16T05:33:52.574373+0000","last_undegraded":"2024-04-16T05:33:52.574373+0000","last_fullsized":"2024-04-16T05:33:52.574373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:34.242117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.509337+0000","last_change":"2024-04-16T05:32:55.509970+0000","last_active":"2024-04-16T05:34:21.509337+0000","last_peered":"2024-04-16T05:34:21.509337+0000","last_clean":"2024-04-16T05:34:21.509337+0000","last_became_active":"2024-04-16T05:32:55.509632+0000","last_became_peered":"2024-04-16T05:32:55.509632+0000","last_unstale":"2024-04-16T05:34:21.509337+0000","last_undegraded":"2024-04-16T05:34:21.509337+0000","last_fullsized":"2024-04-16T05:34:21.509337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:33:55.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.672444+0000","last_change":"2024-04-16T05:32:59.527007+0000","last_active":"2024-04-16T05:33:52.672444+0000","last_peered":"2024-04-16T05:33:52.672444+0000","last_clean":"2024-04-16T05:33:52.672444+0000","last_became_active":"2024-04-16T05:32:59.526645+0000","last_became_peered":"2024-04-16T05:32:59.526645+0000","last_unstale":"2024-04-16T05:33:52.672444+0000","last_undegraded":"2024-04-16T05:33:52.672444+0000","last_fullsized":"2024-04-16T05:33:52.672444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:46:06.698912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:24.712473+0000","last_change":"2024-04-16T05:32:54.503409+0000","last_active":"2024-04-16T05:34:24.712473+0000","last_peered":"2024-04-16T05:34:24.712473+0000","last_clean":"2024-04-16T05:34:24.712473+0000","last_became_active":"2024-04-16T05:32:54.503069+0000","last_became_peered":"2024-04-16T05:32:54.503069+0000","last_unstale":"2024-04-16T05:34:24.712473+0000","last_undegraded":"2024-04-16T05:34:24.712473+0000","last_fullsized":"2024-04-16T05:34:24.712473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:45:10.535225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501668+0000","last_change":"2024-04-16T05:32:55.519744+0000","last_active":"2024-04-16T05:34:21.501668+0000","last_peered":"2024-04-16T05:34:21.501668+0000","last_clean":"2024-04-16T05:34:21.501668+0000","last_became_active":"2024-04-16T05:32:55.519091+0000","last_became_peered":"2024-04-16T05:32:55.519091+0000","last_unstale":"2024-04-16T05:34:21.501668+0000","last_undegraded":"2024-04-16T05:34:21.501668+0000","last_fullsized":"2024-04-16T05:34:21.501668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:19.417691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.672436+0000","last_change":"2024-04-16T05:33:00.419494+0000","last_active":"2024-04-16T05:33:52.672436+0000","last_peered":"2024-04-16T05:33:52.672436+0000","last_clean":"2024-04-16T05:33:52.672436+0000","last_became_active":"2024-04-16T05:33:00.418954+0000","last_became_peered":"2024-04-16T05:33:00.418954+0000","last_unstale":"2024-04-16T05:33:52.672436+0000","last_undegraded":"2024-04-16T05:33:52.672436+0000","last_fullsized":"2024-04-16T05:33:52.672436+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:03:01.835304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.504732+0000","last_change":"2024-04-16T05:32:55.518272+0000","last_active":"2024-04-16T05:34:21.504732+0000","last_peered":"2024-04-16T05:34:21.504732+0000","last_clean":"2024-04-16T05:34:21.504732+0000","last_became_active":"2024-04-16T05:32:55.517928+0000","last_became_peered":"2024-04-16T05:32:55.517928+0000","last_unstale":"2024-04-16T05:34:21.504732+0000","last_undegraded":"2024-04-16T05:34:21.504732+0000","last_fullsized":"2024-04-16T05:34:21.504732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:44:47.739994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:24.721140+0000","last_change":"2024-04-16T05:32:54.503923+0000","last_active":"2024-04-16T05:34:24.721140+0000","last_peered":"2024-04-16T05:34:24.721140+0000","last_clean":"2024-04-16T05:34:24.721140+0000","last_became_active":"2024-04-16T05:32:54.503576+0000","last_became_peered":"2024-04-16T05:32:54.503576+0000","last_unstale":"2024-04-16T05:34:24.721140+0000","last_undegraded":"2024-04-16T05:34:24.721140+0000","last_fullsized":"2024-04-16T05:34:24.721140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:49:36.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.580596+0000","last_change":"2024-04-16T05:32:59.517757+0000","last_active":"2024-04-16T05:33:52.580596+0000","last_peered":"2024-04-16T05:33:52.580596+0000","last_clean":"2024-04-16T05:33:52.580596+0000","last_became_active":"2024-04-16T05:32:59.517357+0000","last_became_peered":"2024-04-16T05:32:59.517357+0000","last_unstale":"2024-04-16T05:33:52.580596+0000","last_undegraded":"2024-04-16T05:33:52.580596+0000","last_fullsized":"2024-04-16T05:33:52.580596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:21.767071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.505253+0000","last_change":"2024-04-16T05:32:55.514617+0000","last_active":"2024-04-16T05:34:21.505253+0000","last_peered":"2024-04-16T05:34:21.505253+0000","last_clean":"2024-04-16T05:34:21.505253+0000","last_became_active":"2024-04-16T05:32:55.512660+0000","last_became_peered":"2024-04-16T05:32:55.512660+0000","last_unstale":"2024-04-16T05:34:21.505253+0000","last_undegraded":"2024-04-16T05:34:21.505253+0000","last_fullsized":"2024-04-16T05:34:21.505253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:31:45.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576333+0000","last_change":"2024-04-16T05:32:54.498650+0000","last_active":"2024-04-16T05:33:52.576333+0000","last_peered":"2024-04-16T05:33:52.576333+0000","last_clean":"2024-04-16T05:33:52.576333+0000","last_became_active":"2024-04-16T05:32:54.497672+0000","last_became_peered":"2024-04-16T05:32:54.497672+0000","last_unstale":"2024-04-16T05:33:52.576333+0000","last_undegraded":"2024-04-16T05:33:52.576333+0000","last_fullsized":"2024-04-16T05:33:52.576333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:36.908809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":363,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579966+0000","last_change":"2024-04-16T05:31:42.831800+0000","last_active":"2024-04-16T05:33:52.579966+0000","last_peered":"2024-04-16T05:33:52.579966+0000","last_clean":"2024-04-16T05:33:52.579966+0000","last_became_active":"2024-04-16T05:31:42.831391+0000","last_became_peered":"2024-04-16T05:31:42.831391+0000","last_unstale":"2024-04-16T05:33:52.579966+0000","last_undegraded":"2024-04-16T05:33:52.579966+0000","last_fullsized":"2024-04-16T05:33:52.579966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_clean_scrub_stamp":"2024-04-16T05:28:15.969690+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.152803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.675549+0000","last_change":"2024-04-16T05:33:52.675549+0000","last_active":"2024-04-16T05:33:52.675549+0000","last_peered":"2024-04-16T05:33:52.675549+0000","last_clean":"2024-04-16T05:33:52.675549+0000","last_became_active":"2024-04-16T05:33:52.674969+0000","last_became_peered":"2024-04-16T05:33:52.674969+0000","last_unstale":"2024-04-16T05:33:52.675549+0000","last_undegraded":"2024-04-16T05:33:52.675549+0000","last_fullsized":"2024-04-16T05:33:52.675549+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:58:33.137841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.476259+0000","last_change":"2024-04-16T05:32:55.512324+0000","last_active":"2024-04-16T05:34:21.476259+0000","last_peered":"2024-04-16T05:34:21.476259+0000","last_clean":"2024-04-16T05:34:21.476259+0000","last_became_active":"2024-04-16T05:32:55.511958+0000","last_became_peered":"2024-04-16T05:32:55.511958+0000","last_unstale":"2024-04-16T05:34:21.476259+0000","last_undegraded":"2024-04-16T05:34:21.476259+0000","last_fullsized":"2024-04-16T05:34:21.476259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:38.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579985+0000","last_change":"2024-04-16T05:32:54.508021+0000","last_active":"2024-04-16T05:33:52.579985+0000","last_peered":"2024-04-16T05:33:52.579985+0000","last_clean":"2024-04-16T05:33:52.579985+0000","last_became_active":"2024-04-16T05:32:54.507645+0000","last_became_peered":"2024-04-16T05:32:54.507645+0000","last_unstale":"2024-04-16T05:33:52.579985+0000","last_undegraded":"2024-04-16T05:33:52.579985+0000","last_fullsized":"2024-04-16T05:33:52.579985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:21:50.721260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579034+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:33:52.579034+0000","last_peered":"2024-04-16T05:33:52.579034+0000","last_clean":"2024-04-16T05:33:52.579034+0000","last_became_active":"2024-04-16T05:32:59.523159+0000","last_became_peered":"2024-04-16T05:32:59.523159+0000","last_unstale":"2024-04-16T05:33:52.579034+0000","last_undegraded":"2024-04-16T05:33:52.579034+0000","last_fullsized":"2024-04-16T05:33:52.579034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:48:37.696713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576159+0000","last_change":"2024-04-16T05:32:54.503262+0000","last_active":"2024-04-16T05:33:52.576159+0000","last_peered":"2024-04-16T05:33:52.576159+0000","last_clean":"2024-04-16T05:33:52.576159+0000","last_became_active":"2024-04-16T05:32:54.502909+0000","last_became_peered":"2024-04-16T05:32:54.502909+0000","last_unstale":"2024-04-16T05:33:52.576159+0000","last_undegraded":"2024-04-16T05:33:52.576159+0000","last_fullsized":"2024-04-16T05:33:52.576159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:48:32.841921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.490661+0000","last_change":"2024-04-16T05:32:55.507126+0000","last_active":"2024-04-16T05:34:21.490661+0000","last_peered":"2024-04-16T05:34:21.490661+0000","last_clean":"2024-04-16T05:34:21.490661+0000","last_became_active":"2024-04-16T05:32:55.506743+0000","last_became_peered":"2024-04-16T05:32:55.506743+0000","last_unstale":"2024-04-16T05:34:21.490661+0000","last_undegraded":"2024-04-16T05:34:21.490661+0000","last_fullsized":"2024-04-16T05:34:21.490661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:10:58.697649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577532+0000","last_change":"2024-04-16T05:33:00.419435+0000","last_active":"2024-04-16T05:33:52.577532+0000","last_peered":"2024-04-16T05:33:52.577532+0000","last_clean":"2024-04-16T05:33:52.577532+0000","last_became_active":"2024-04-16T05:33:00.418985+0000","last_became_peered":"2024-04-16T05:33:00.418985+0000","last_unstale":"2024-04-16T05:33:52.577532+0000","last_undegraded":"2024-04-16T05:33:52.577532+0000","last_fullsized":"2024-04-16T05:33:52.577532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:08:31.188911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.574583+0000","last_change":"2024-04-16T05:32:54.503349+0000","last_active":"2024-04-16T05:33:52.574583+0000","last_peered":"2024-04-16T05:33:52.574583+0000","last_clean":"2024-04-16T05:33:52.574583+0000","last_became_active":"2024-04-16T05:32:54.503018+0000","last_became_peered":"2024-04-16T05:32:54.503018+0000","last_unstale":"2024-04-16T05:33:52.574583+0000","last_undegraded":"2024-04-16T05:33:52.574583+0000","last_fullsized":"2024-04-16T05:33:52.574583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:34:20.853543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506525+0000","last_change":"2024-04-16T05:32:55.511201+0000","last_active":"2024-04-16T05:34:21.506525+0000","last_peered":"2024-04-16T05:34:21.506525+0000","last_clean":"2024-04-16T05:34:21.506525+0000","last_became_active":"2024-04-16T05:32:55.510906+0000","last_became_peered":"2024-04-16T05:32:55.510906+0000","last_unstale":"2024-04-16T05:34:21.506525+0000","last_undegraded":"2024-04-16T05:34:21.506525+0000","last_fullsized":"2024-04-16T05:34:21.506525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:37:37.147628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577102+0000","last_change":"2024-04-16T05:32:59.518440+0000","last_active":"2024-04-16T05:33:52.577102+0000","last_peered":"2024-04-16T05:33:52.577102+0000","last_clean":"2024-04-16T05:33:52.577102+0000","last_became_active":"2024-04-16T05:32:59.517870+0000","last_became_peered":"2024-04-16T05:32:59.517870+0000","last_unstale":"2024-04-16T05:33:52.577102+0000","last_undegraded":"2024-04-16T05:33:52.577102+0000","last_fullsized":"2024-04-16T05:33:52.577102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:51:31.157418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576507+0000","last_change":"2024-04-16T05:32:54.503983+0000","last_active":"2024-04-16T05:33:52.576507+0000","last_peered":"2024-04-16T05:33:52.576507+0000","last_clean":"2024-04-16T05:33:52.576507+0000","last_became_active":"2024-04-16T05:32:54.502367+0000","last_became_peered":"2024-04-16T05:32:54.502367+0000","last_unstale":"2024-04-16T05:33:52.576507+0000","last_undegraded":"2024-04-16T05:33:52.576507+0000","last_fullsized":"2024-04-16T05:33:52.576507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:49:36.997692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.505135+0000","last_change":"2024-04-16T05:32:55.516812+0000","last_active":"2024-04-16T05:34:21.505135+0000","last_peered":"2024-04-16T05:34:21.505135+0000","last_clean":"2024-04-16T05:34:21.505135+0000","last_became_active":"2024-04-16T05:32:55.516477+0000","last_became_peered":"2024-04-16T05:32:55.516477+0000","last_unstale":"2024-04-16T05:34:21.505135+0000","last_undegraded":"2024-04-16T05:34:21.505135+0000","last_fullsized":"2024-04-16T05:34:21.505135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:24:29.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576194+0000","last_change":"2024-04-16T05:33:00.419622+0000","last_active":"2024-04-16T05:33:52.576194+0000","last_peered":"2024-04-16T05:33:52.576194+0000","last_clean":"2024-04-16T05:33:52.576194+0000","last_became_active":"2024-04-16T05:33:00.419135+0000","last_became_peered":"2024-04-16T05:33:00.419135+0000","last_unstale":"2024-04-16T05:33:52.576194+0000","last_undegraded":"2024-04-16T05:33:52.576194+0000","last_fullsized":"2024-04-16T05:33:52.576194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:38:02.529192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578401+0000","last_change":"2024-04-16T05:32:54.503652+0000","last_active":"2024-04-16T05:33:52.578401+0000","last_peered":"2024-04-16T05:33:52.578401+0000","last_clean":"2024-04-16T05:33:52.578401+0000","last_became_active":"2024-04-16T05:32:54.503311+0000","last_became_peered":"2024-04-16T05:32:54.503311+0000","last_unstale":"2024-04-16T05:33:52.578401+0000","last_undegraded":"2024-04-16T05:33:52.578401+0000","last_fullsized":"2024-04-16T05:33:52.578401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:49:21.584029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.462985+0000","last_change":"2024-04-16T05:32:55.508145+0000","last_active":"2024-04-16T05:34:21.462985+0000","last_peered":"2024-04-16T05:34:21.462985+0000","last_clean":"2024-04-16T05:34:21.462985+0000","last_became_active":"2024-04-16T05:32:55.507794+0000","last_became_peered":"2024-04-16T05:32:55.507794+0000","last_unstale":"2024-04-16T05:34:21.462985+0000","last_undegraded":"2024-04-16T05:34:21.462985+0000","last_fullsized":"2024-04-16T05:34:21.462985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:11:57.925733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.678920+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:33:52.678920+0000","last_peered":"2024-04-16T05:33:52.678920+0000","last_clean":"2024-04-16T05:33:52.678920+0000","last_became_active":"2024-04-16T05:33:52.678330+0000","last_became_peered":"2024-04-16T05:33:52.678330+0000","last_unstale":"2024-04-16T05:33:52.678920+0000","last_undegraded":"2024-04-16T05:33:52.678920+0000","last_fullsized":"2024-04-16T05:33:52.678920+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:33.323513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578679+0000","last_change":"2024-04-16T05:32:54.500909+0000","last_active":"2024-04-16T05:33:52.578679+0000","last_peered":"2024-04-16T05:33:52.578679+0000","last_clean":"2024-04-16T05:33:52.578679+0000","last_became_active":"2024-04-16T05:32:54.499348+0000","last_became_peered":"2024-04-16T05:32:54.499348+0000","last_unstale":"2024-04-16T05:33:52.578679+0000","last_undegraded":"2024-04-16T05:33:52.578679+0000","last_fullsized":"2024-04-16T05:33:52.578679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:51:49.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.505607+0000","last_change":"2024-04-16T05:32:55.510922+0000","last_active":"2024-04-16T05:34:21.505607+0000","last_peered":"2024-04-16T05:34:21.505607+0000","last_clean":"2024-04-16T05:34:21.505607+0000","last_became_active":"2024-04-16T05:32:55.510546+0000","last_became_peered":"2024-04-16T05:32:55.510546+0000","last_unstale":"2024-04-16T05:34:21.505607+0000","last_undegraded":"2024-04-16T05:34:21.505607+0000","last_fullsized":"2024-04-16T05:34:21.505607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:27.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575904+0000","last_change":"2024-04-16T05:32:59.524257+0000","last_active":"2024-04-16T05:33:52.575904+0000","last_peered":"2024-04-16T05:33:52.575904+0000","last_clean":"2024-04-16T05:33:52.575904+0000","last_became_active":"2024-04-16T05:32:59.523768+0000","last_became_peered":"2024-04-16T05:32:59.523768+0000","last_unstale":"2024-04-16T05:33:52.575904+0000","last_undegraded":"2024-04-16T05:33:52.575904+0000","last_fullsized":"2024-04-16T05:33:52.575904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:38:27.594048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577241+0000","last_change":"2024-04-16T05:32:54.504430+0000","last_active":"2024-04-16T05:33:52.577241+0000","last_peered":"2024-04-16T05:33:52.577241+0000","last_clean":"2024-04-16T05:33:52.577241+0000","last_became_active":"2024-04-16T05:32:54.504054+0000","last_became_peered":"2024-04-16T05:32:54.504054+0000","last_unstale":"2024-04-16T05:33:52.577241+0000","last_undegraded":"2024-04-16T05:33:52.577241+0000","last_fullsized":"2024-04-16T05:33:52.577241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:53:24.875786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.503479+0000","last_change":"2024-04-16T05:32:55.508005+0000","last_active":"2024-04-16T05:34:21.503479+0000","last_peered":"2024-04-16T05:34:21.503479+0000","last_clean":"2024-04-16T05:34:21.503479+0000","last_became_active":"2024-04-16T05:32:55.507698+0000","last_became_peered":"2024-04-16T05:32:55.507698+0000","last_unstale":"2024-04-16T05:34:21.503479+0000","last_undegraded":"2024-04-16T05:34:21.503479+0000","last_fullsized":"2024-04-16T05:34:21.503479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:11:12.423294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575846+0000","last_change":"2024-04-16T05:33:00.419442+0000","last_active":"2024-04-16T05:33:52.575846+0000","last_peered":"2024-04-16T05:33:52.575846+0000","last_clean":"2024-04-16T05:33:52.575846+0000","last_became_active":"2024-04-16T05:33:00.418906+0000","last_became_peered":"2024-04-16T05:33:00.418906+0000","last_unstale":"2024-04-16T05:33:52.575846+0000","last_undegraded":"2024-04-16T05:33:52.575846+0000","last_fullsized":"2024-04-16T05:33:52.575846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:21:22.124879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575661+0000","last_change":"2024-04-16T05:32:54.509090+0000","last_active":"2024-04-16T05:33:52.575661+0000","last_peered":"2024-04-16T05:33:52.575661+0000","last_clean":"2024-04-16T05:33:52.575661+0000","last_became_active":"2024-04-16T05:32:54.508700+0000","last_became_peered":"2024-04-16T05:32:54.508700+0000","last_unstale":"2024-04-16T05:33:52.575661+0000","last_undegraded":"2024-04-16T05:33:52.575661+0000","last_fullsized":"2024-04-16T05:33:52.575661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:53:34.344073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511501+0000","last_change":"2024-04-16T05:32:55.509658+0000","last_active":"2024-04-16T05:34:21.511501+0000","last_peered":"2024-04-16T05:34:21.511501+0000","last_clean":"2024-04-16T05:34:21.511501+0000","last_became_active":"2024-04-16T05:32:55.509258+0000","last_became_peered":"2024-04-16T05:32:55.509258+0000","last_unstale":"2024-04-16T05:34:21.511501+0000","last_undegraded":"2024-04-16T05:34:21.511501+0000","last_fullsized":"2024-04-16T05:34:21.511501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:17:03.490544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578141+0000","last_change":"2024-04-16T05:32:59.519260+0000","last_active":"2024-04-16T05:33:52.578141+0000","last_peered":"2024-04-16T05:33:52.578141+0000","last_clean":"2024-04-16T05:33:52.578141+0000","last_became_active":"2024-04-16T05:32:59.518839+0000","last_became_peered":"2024-04-16T05:32:59.518839+0000","last_unstale":"2024-04-16T05:33:52.578141+0000","last_undegraded":"2024-04-16T05:33:52.578141+0000","last_fullsized":"2024-04-16T05:33:52.578141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:54:50.555214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:23.720218+0000","last_change":"2024-04-16T05:32:54.503226+0000","last_active":"2024-04-16T05:34:23.720218+0000","last_peered":"2024-04-16T05:34:23.720218+0000","last_clean":"2024-04-16T05:34:23.720218+0000","last_became_active":"2024-04-16T05:32:54.502687+0000","last_became_peered":"2024-04-16T05:32:54.502687+0000","last_unstale":"2024-04-16T05:34:23.720218+0000","last_undegraded":"2024-04-16T05:34:23.720218+0000","last_fullsized":"2024-04-16T05:34:23.720218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:34:24.228464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.510895+0000","last_change":"2024-04-16T05:32:55.515203+0000","last_active":"2024-04-16T05:34:21.510895+0000","last_peered":"2024-04-16T05:34:21.510895+0000","last_clean":"2024-04-16T05:34:21.510895+0000","last_became_active":"2024-04-16T05:32:55.514815+0000","last_became_peered":"2024-04-16T05:32:55.514815+0000","last_unstale":"2024-04-16T05:34:21.510895+0000","last_undegraded":"2024-04-16T05:34:21.510895+0000","last_fullsized":"2024-04-16T05:34:21.510895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:11:18.464532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577618+0000","last_change":"2024-04-16T05:32:59.520442+0000","last_active":"2024-04-16T05:33:52.577618+0000","last_peered":"2024-04-16T05:33:52.577618+0000","last_clean":"2024-04-16T05:33:52.577618+0000","last_became_active":"2024-04-16T05:32:59.520071+0000","last_became_peered":"2024-04-16T05:32:59.520071+0000","last_unstale":"2024-04-16T05:33:52.577618+0000","last_undegraded":"2024-04-16T05:33:52.577618+0000","last_fullsized":"2024-04-16T05:33:52.577618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:19:16.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677720+0000","last_change":"2024-04-16T05:32:54.510098+0000","last_active":"2024-04-16T05:33:52.677720+0000","last_peered":"2024-04-16T05:33:52.677720+0000","last_clean":"2024-04-16T05:33:52.677720+0000","last_became_active":"2024-04-16T05:32:54.509750+0000","last_became_peered":"2024-04-16T05:32:54.509750+0000","last_unstale":"2024-04-16T05:33:52.677720+0000","last_undegraded":"2024-04-16T05:33:52.677720+0000","last_fullsized":"2024-04-16T05:33:52.677720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:00:09.528958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501136+0000","last_change":"2024-04-16T05:33:52.678823+0000","last_active":"2024-04-16T05:34:21.501136+0000","last_peered":"2024-04-16T05:34:21.501136+0000","last_clean":"2024-04-16T05:34:21.501136+0000","last_became_active":"2024-04-16T05:33:52.678276+0000","last_became_peered":"2024-04-16T05:33:52.678276+0000","last_unstale":"2024-04-16T05:34:21.501136+0000","last_undegraded":"2024-04-16T05:34:21.501136+0000","last_fullsized":"2024-04-16T05:34:21.501136+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:58:33.186382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.676262+0000","last_change":"2024-04-16T05:32:59.521334+0000","last_active":"2024-04-16T05:33:52.676262+0000","last_peered":"2024-04-16T05:33:52.676262+0000","last_clean":"2024-04-16T05:33:52.676262+0000","last_became_active":"2024-04-16T05:32:59.520348+0000","last_became_peered":"2024-04-16T05:32:59.520348+0000","last_unstale":"2024-04-16T05:33:52.676262+0000","last_undegraded":"2024-04-16T05:33:52.676262+0000","last_fullsized":"2024-04-16T05:33:52.676262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:40.628552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.672455+0000","last_change":"2024-04-16T05:32:54.503062+0000","last_active":"2024-04-16T05:33:52.672455+0000","last_peered":"2024-04-16T05:33:52.672455+0000","last_clean":"2024-04-16T05:33:52.672455+0000","last_became_active":"2024-04-16T05:32:54.502762+0000","last_became_peered":"2024-04-16T05:32:54.502762+0000","last_unstale":"2024-04-16T05:33:52.672455+0000","last_undegraded":"2024-04-16T05:33:52.672455+0000","last_fullsized":"2024-04-16T05:33:52.672455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:43:57.911296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.502415+0000","last_change":"2024-04-16T05:32:55.510461+0000","last_active":"2024-04-16T05:34:21.502415+0000","last_peered":"2024-04-16T05:34:21.502415+0000","last_clean":"2024-04-16T05:34:21.502415+0000","last_became_active":"2024-04-16T05:32:55.510142+0000","last_became_peered":"2024-04-16T05:32:55.510142+0000","last_unstale":"2024-04-16T05:34:21.502415+0000","last_undegraded":"2024-04-16T05:34:21.502415+0000","last_fullsized":"2024-04-16T05:34:21.502415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:41.346078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506149+0000","last_change":"2024-04-16T05:32:55.512600+0000","last_active":"2024-04-16T05:34:21.506149+0000","last_peered":"2024-04-16T05:34:21.506149+0000","last_clean":"2024-04-16T05:34:21.506149+0000","last_became_active":"2024-04-16T05:32:55.512265+0000","last_became_peered":"2024-04-16T05:32:55.512265+0000","last_unstale":"2024-04-16T05:34:21.506149+0000","last_undegraded":"2024-04-16T05:34:21.506149+0000","last_fullsized":"2024-04-16T05:34:21.506149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:04:46.112644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.504122+0000","last_change":"2024-04-16T05:32:55.514481+0000","last_active":"2024-04-16T05:34:21.504122+0000","last_peered":"2024-04-16T05:34:21.504122+0000","last_clean":"2024-04-16T05:34:21.504122+0000","last_became_active":"2024-04-16T05:32:55.512485+0000","last_became_peered":"2024-04-16T05:32:55.512485+0000","last_unstale":"2024-04-16T05:34:21.504122+0000","last_undegraded":"2024-04-16T05:34:21.504122+0000","last_fullsized":"2024-04-16T05:34:21.504122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:33:02.179774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.503157+0000","last_change":"2024-04-16T05:32:55.509849+0000","last_active":"2024-04-16T05:34:21.503157+0000","last_peered":"2024-04-16T05:34:21.503157+0000","last_clean":"2024-04-16T05:34:21.503157+0000","last_became_active":"2024-04-16T05:32:55.509338+0000","last_became_peered":"2024-04-16T05:32:55.509338+0000","last_unstale":"2024-04-16T05:34:21.503157+0000","last_undegraded":"2024-04-16T05:34:21.503157+0000","last_fullsized":"2024-04-16T05:34:21.503157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:32:38.900455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.502558+0000","last_change":"2024-04-16T05:32:55.518556+0000","last_active":"2024-04-16T05:34:21.502558+0000","last_peered":"2024-04-16T05:34:21.502558+0000","last_clean":"2024-04-16T05:34:21.502558+0000","last_became_active":"2024-04-16T05:32:55.518240+0000","last_became_peered":"2024-04-16T05:32:55.518240+0000","last_unstale":"2024-04-16T05:34:21.502558+0000","last_undegraded":"2024-04-16T05:34:21.502558+0000","last_fullsized":"2024-04-16T05:34:21.502558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:25:46.002516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.498586+0000","last_change":"2024-04-16T05:32:55.518106+0000","last_active":"2024-04-16T05:34:21.498586+0000","last_peered":"2024-04-16T05:34:21.498586+0000","last_clean":"2024-04-16T05:34:21.498586+0000","last_became_active":"2024-04-16T05:32:55.517403+0000","last_became_peered":"2024-04-16T05:32:55.517403+0000","last_unstale":"2024-04-16T05:34:21.498586+0000","last_undegraded":"2024-04-16T05:34:21.498586+0000","last_fullsized":"2024-04-16T05:34:21.498586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:36:34.731644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501520+0000","last_change":"2024-04-16T05:33:52.581236+0000","last_active":"2024-04-16T05:34:21.501520+0000","last_peered":"2024-04-16T05:34:21.501520+0000","last_clean":"2024-04-16T05:34:21.501520+0000","last_became_active":"2024-04-16T05:33:52.580748+0000","last_became_peered":"2024-04-16T05:33:52.580748+0000","last_unstale":"2024-04-16T05:34:21.501520+0000","last_undegraded":"2024-04-16T05:34:21.501520+0000","last_fullsized":"2024-04-16T05:34:21.501520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:13:33.382776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511534+0000","last_change":"2024-04-16T05:32:55.507670+0000","last_active":"2024-04-16T05:34:21.511534+0000","last_peered":"2024-04-16T05:34:21.511534+0000","last_clean":"2024-04-16T05:34:21.511534+0000","last_became_active":"2024-04-16T05:32:55.506672+0000","last_became_peered":"2024-04-16T05:32:55.506672+0000","last_unstale":"2024-04-16T05:34:21.511534+0000","last_undegraded":"2024-04-16T05:34:21.511534+0000","last_fullsized":"2024-04-16T05:34:21.511534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:54.579672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.495721+0000","last_change":"2024-04-16T05:32:55.516987+0000","last_active":"2024-04-16T05:34:21.495721+0000","last_peered":"2024-04-16T05:34:21.495721+0000","last_clean":"2024-04-16T05:34:21.495721+0000","last_became_active":"2024-04-16T05:32:55.516654+0000","last_became_peered":"2024-04-16T05:32:55.516654+0000","last_unstale":"2024-04-16T05:34:21.495721+0000","last_undegraded":"2024-04-16T05:34:21.495721+0000","last_fullsized":"2024-04-16T05:34:21.495721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:32:42.957658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.508245+0000","last_change":"2024-04-16T05:32:55.506539+0000","last_active":"2024-04-16T05:34:21.508245+0000","last_peered":"2024-04-16T05:34:21.508245+0000","last_clean":"2024-04-16T05:34:21.508245+0000","last_became_active":"2024-04-16T05:32:55.503658+0000","last_became_peered":"2024-04-16T05:32:55.503658+0000","last_unstale":"2024-04-16T05:34:21.508245+0000","last_undegraded":"2024-04-16T05:34:21.508245+0000","last_fullsized":"2024-04-16T05:34:21.508245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:22.116213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.507351+0000","last_change":"2024-04-16T05:32:55.507363+0000","last_active":"2024-04-16T05:34:21.507351+0000","last_peered":"2024-04-16T05:34:21.507351+0000","last_clean":"2024-04-16T05:34:21.507351+0000","last_became_active":"2024-04-16T05:32:55.506923+0000","last_became_peered":"2024-04-16T05:32:55.506923+0000","last_unstale":"2024-04-16T05:34:21.507351+0000","last_undegraded":"2024-04-16T05:34:21.507351+0000","last_fullsized":"2024-04-16T05:34:21.507351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:19:34.356834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511928+0000","last_change":"2024-04-16T05:32:55.508531+0000","last_active":"2024-04-16T05:34:21.511928+0000","last_peered":"2024-04-16T05:34:21.511928+0000","last_clean":"2024-04-16T05:34:21.511928+0000","last_became_active":"2024-04-16T05:32:55.508177+0000","last_became_peered":"2024-04-16T05:32:55.508177+0000","last_unstale":"2024-04-16T05:34:21.511928+0000","last_undegraded":"2024-04-16T05:34:21.511928+0000","last_fullsized":"2024-04-16T05:34:21.511928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:49:59.352572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.509060+0000","last_change":"2024-04-16T05:32:55.507793+0000","last_active":"2024-04-16T05:34:21.509060+0000","last_peered":"2024-04-16T05:34:21.509060+0000","last_clean":"2024-04-16T05:34:21.509060+0000","last_became_active":"2024-04-16T05:32:55.507443+0000","last_became_peered":"2024-04-16T05:32:55.507443+0000","last_unstale":"2024-04-16T05:34:21.509060+0000","last_undegraded":"2024-04-16T05:34:21.509060+0000","last_fullsized":"2024-04-16T05:34:21.509060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:35.982879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499109+0000","last_change":"2024-04-16T05:32:55.517057+0000","last_active":"2024-04-16T05:34:21.499109+0000","last_peered":"2024-04-16T05:34:21.499109+0000","last_clean":"2024-04-16T05:34:21.499109+0000","last_became_active":"2024-04-16T05:32:55.516696+0000","last_became_peered":"2024-04-16T05:32:55.516696+0000","last_unstale":"2024-04-16T05:34:21.499109+0000","last_undegraded":"2024-04-16T05:34:21.499109+0000","last_fullsized":"2024-04-16T05:34:21.499109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:43:10.391125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280341,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69457,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":716,"num_read_kb":437,"num_write":253,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":208371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383711,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":511388,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":1159643,"data_compressed":59230,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":514882,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":511164,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1108334,"data_compressed":59230,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495673,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":506290,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":568759,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":508278,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":570043,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":505998,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":510896,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":1103384,"data_compressed":59230,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:34:31.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph pg dump --format=json 2024-04-16T05:34:31.774 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:32 smithi194 ceph-mon[35865]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:32 smithi194 ceph-mon[35865]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-16T05:34:32.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:32 smithi086 ceph-mon[32028]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:32.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:32 smithi086 ceph-mon[32028]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-16T05:34:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:32 smithi196 ceph-mon[36373]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:32 smithi196 ceph-mon[36373]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.488+0000 7f4fe544d700 1 -- 172.21.15.86:0/2843722740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 msgr2=0x7f4fe01031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.488+0000 7f4fe544d700 1 --2- 172.21.15.86:0/2843722740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 0x7f4fe01031e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00099c0 tx=0x7f4fd0009cd0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.488+0000 7f4fe544d700 1 -- 172.21.15.86:0/2843722740 shutdown_connections 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.488+0000 7f4fe544d700 1 --2- 172.21.15.86:0/2843722740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe0107ed0 0x7f4fe010fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.488+0000 7f4fe544d700 1 --2- 172.21.15.86:0/2843722740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 0x7f4fe01031e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.488+0000 7f4fe544d700 1 --2- 172.21.15.86:0/2843722740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe01037b0 0x7f4fe01077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.488+0000 7f4fe544d700 1 -- 172.21.15.86:0/2843722740 >> 172.21.15.86:0/2843722740 conn(0x7f4fe0079b80 msgr2=0x7f4fe0079f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.488+0000 7f4fe544d700 1 -- 172.21.15.86:0/2843722740 shutdown_connections 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.488+0000 7f4fe544d700 1 -- 172.21.15.86:0/2843722740 wait complete. 2024-04-16T05:34:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fe544d700 1 Processor -- start 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fe544d700 1 -- start start 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fe544d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 0x7f4fe01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fe544d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe01037b0 0x7f4fe01996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fe544d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe0107ed0 0x7f4fe019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fe544d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0112e00 con 0x7f4fe0102e00 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fe544d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0112c80 con 0x7f4fe0107ed0 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fe544d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0112f80 con 0x7f4fe01037b0 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fdeffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 0x7f4fe01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fdeffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 0x7f4fe01991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38606/0 (socket says 172.21.15.86:38606) 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.489+0000 7f4fdeffd700 1 -- 172.21.15.86:0/3063586656 learned_addr learned my addr 172.21.15.86:0/3063586656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fd67fc700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe01037b0 0x7f4fe01996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdeffd700 1 -- 172.21.15.86:0/3063586656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe01037b0 msgr2=0x7f4fe01996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdeffd700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe01037b0 0x7f4fe01996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdf7fe700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe0107ed0 0x7f4fe019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdeffd700 1 -- 172.21.15.86:0/3063586656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe0107ed0 msgr2=0x7f4fe019da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdeffd700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe0107ed0 0x7f4fe019da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdeffd700 1 -- 172.21.15.86:0/3063586656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe019e1c0 con 0x7f4fe0102e00 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdf7fe700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe0107ed0 0x7f4fe019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdeffd700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 0x7f4fe01991b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009fd0 tx=0x7f4fd0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdcff9700 1 -- 172.21.15.86:0/3063586656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0027070 con 0x7f4fe0102e00 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe019e450 con 0x7f4fe0102e00 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdcff9700 1 -- 172.21.15.86:0/3063586656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd0004150 con 0x7f4fe0102e00 2024-04-16T05:34:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fdcff9700 1 -- 172.21.15.86:0/3063586656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd000f460 con 0x7f4fe0102e00 2024-04-16T05:34:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.490+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe01a5d30 con 0x7f4fe0102e00 2024-04-16T05:34:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.491+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe010b110 con 0x7f4fe0102e00 2024-04-16T05:34:33.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.492+0000 7f4fdcff9700 1 -- 172.21.15.86:0/3063586656 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4fd002a070 con 0x7f4fe0102e00 2024-04-16T05:34:33.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.493+0000 7f4fdcff9700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4fcc0a3c40 0x7f4fcc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:33.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.493+0000 7f4fd67fc700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4fcc0a3c40 0x7f4fcc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:33.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.493+0000 7f4fdcff9700 1 -- 172.21.15.86:0/3063586656 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4fd00d1730 con 0x7f4fe0102e00 2024-04-16T05:34:33.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.494+0000 7f4fd67fc700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4fcc0a3c40 0x7f4fcc0a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8005fd0 tx=0x7f4fc800a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:33.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.498+0000 7f4fdcff9700 1 -- 172.21.15.86:0/3063586656 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fd005eff0 con 0x7f4fe0102e00 2024-04-16T05:34:33.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.694+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4fe019a510 con 0x7f4fcc0a3c40 2024-04-16T05:34:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.703+0000 7f4fdcff9700 1 -- 172.21.15.86:0/3063586656 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340787 (secure 0 0 0) 0x7f4fe019a510 con 0x7f4fcc0a3c40 2024-04-16T05:34:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:34:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.706+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4fcc0a3c40 msgr2=0x7f4fcc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.706+0000 7f4fe544d700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4fcc0a3c40 0x7f4fcc0a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8005fd0 tx=0x7f4fc800a560 comp rx=0 tx=0).stop 2024-04-16T05:34:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.706+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 msgr2=0x7f4fe01991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.706+0000 7f4fe544d700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 0x7f4fe01991b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009fd0 tx=0x7f4fd0003730 comp rx=0 tx=0).stop 2024-04-16T05:34:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.707+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 shutdown_connections 2024-04-16T05:34:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.707+0000 7f4fe544d700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe0107ed0 0x7f4fe019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.707+0000 7f4fe544d700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe0102e00 0x7f4fe01991b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.707+0000 7f4fe544d700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4fcc0a3c40 0x7f4fcc0a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.707+0000 7f4fe544d700 1 --2- 172.21.15.86:0/3063586656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe01037b0 0x7f4fe01996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.707+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 >> 172.21.15.86:0/3063586656 conn(0x7f4fe0079b80 msgr2=0x7f4fe0102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.707+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 shutdown_connections 2024-04-16T05:34:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:33.707+0000 7f4fe544d700 1 -- 172.21.15.86:0/3063586656 wait complete. 2024-04-16T05:34:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-16T05:34:34.460 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-04-16T05:34:33.503471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660059,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":897,"num_read_kb":576,"num_write":436,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369912,"kb_used_data":12440,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571912,"statfs":{"total":1151940427776,"available":1151561637888,"internally_reserved":0,"allocated":12738560,"data_stored":8079059,"data_compressed":389838,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":59,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003431"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511478+0000","last_change":"2024-04-16T05:32:55.507548+0000","last_active":"2024-04-16T05:34:21.511478+0000","last_peered":"2024-04-16T05:34:21.511478+0000","last_clean":"2024-04-16T05:34:21.511478+0000","last_became_active":"2024-04-16T05:32:55.507200+0000","last_became_peered":"2024-04-16T05:32:55.507200+0000","last_unstale":"2024-04-16T05:34:21.511478+0000","last_undegraded":"2024-04-16T05:34:21.511478+0000","last_fullsized":"2024-04-16T05:34:21.511478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:58:07.842226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499106+0000","last_change":"2024-04-16T05:32:55.509792+0000","last_active":"2024-04-16T05:34:21.499106+0000","last_peered":"2024-04-16T05:34:21.499106+0000","last_clean":"2024-04-16T05:34:21.499106+0000","last_became_active":"2024-04-16T05:32:55.509442+0000","last_became_peered":"2024-04-16T05:32:55.509442+0000","last_unstale":"2024-04-16T05:34:21.499106+0000","last_undegraded":"2024-04-16T05:34:21.499106+0000","last_fullsized":"2024-04-16T05:34:21.499106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:08:42.040875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511012+0000","last_change":"2024-04-16T05:32:55.505664+0000","last_active":"2024-04-16T05:34:21.511012+0000","last_peered":"2024-04-16T05:34:21.511012+0000","last_clean":"2024-04-16T05:34:21.511012+0000","last_became_active":"2024-04-16T05:32:55.505281+0000","last_became_peered":"2024-04-16T05:32:55.505281+0000","last_unstale":"2024-04-16T05:34:21.511012+0000","last_undegraded":"2024-04-16T05:34:21.511012+0000","last_fullsized":"2024-04-16T05:34:21.511012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:33.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.500029+0000","last_change":"2024-04-16T05:32:55.517072+0000","last_active":"2024-04-16T05:34:21.500029+0000","last_peered":"2024-04-16T05:34:21.500029+0000","last_clean":"2024-04-16T05:34:21.500029+0000","last_became_active":"2024-04-16T05:32:55.516648+0000","last_became_peered":"2024-04-16T05:32:55.516648+0000","last_unstale":"2024-04-16T05:34:21.500029+0000","last_undegraded":"2024-04-16T05:34:21.500029+0000","last_fullsized":"2024-04-16T05:34:21.500029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:41.735720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.489505+0000","last_change":"2024-04-16T05:32:55.506773+0000","last_active":"2024-04-16T05:34:21.489505+0000","last_peered":"2024-04-16T05:34:21.489505+0000","last_clean":"2024-04-16T05:34:21.489505+0000","last_became_active":"2024-04-16T05:32:55.506414+0000","last_became_peered":"2024-04-16T05:32:55.506414+0000","last_unstale":"2024-04-16T05:34:21.489505+0000","last_undegraded":"2024-04-16T05:34:21.489505+0000","last_fullsized":"2024-04-16T05:34:21.489505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:59.254334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.483709+0000","last_change":"2024-04-16T05:32:55.517875+0000","last_active":"2024-04-16T05:34:21.483709+0000","last_peered":"2024-04-16T05:34:21.483709+0000","last_clean":"2024-04-16T05:34:21.483709+0000","last_became_active":"2024-04-16T05:32:55.517546+0000","last_became_peered":"2024-04-16T05:32:55.517546+0000","last_unstale":"2024-04-16T05:34:21.483709+0000","last_undegraded":"2024-04-16T05:34:21.483709+0000","last_fullsized":"2024-04-16T05:34:21.483709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:12:15.098439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501713+0000","last_change":"2024-04-16T05:32:55.509036+0000","last_active":"2024-04-16T05:34:21.501713+0000","last_peered":"2024-04-16T05:34:21.501713+0000","last_clean":"2024-04-16T05:34:21.501713+0000","last_became_active":"2024-04-16T05:32:55.508690+0000","last_became_peered":"2024-04-16T05:32:55.508690+0000","last_unstale":"2024-04-16T05:34:21.501713+0000","last_undegraded":"2024-04-16T05:34:21.501713+0000","last_fullsized":"2024-04-16T05:34:21.501713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:26.724281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506330+0000","last_change":"2024-04-16T05:32:55.510146+0000","last_active":"2024-04-16T05:34:21.506330+0000","last_peered":"2024-04-16T05:34:21.506330+0000","last_clean":"2024-04-16T05:34:21.506330+0000","last_became_active":"2024-04-16T05:32:55.508665+0000","last_became_peered":"2024-04-16T05:32:55.508665+0000","last_unstale":"2024-04-16T05:34:21.506330+0000","last_undegraded":"2024-04-16T05:34:21.506330+0000","last_fullsized":"2024-04-16T05:34:21.506330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:14.158221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.512368+0000","last_change":"2024-04-16T05:32:55.513738+0000","last_active":"2024-04-16T05:34:21.512368+0000","last_peered":"2024-04-16T05:34:21.512368+0000","last_clean":"2024-04-16T05:34:21.512368+0000","last_became_active":"2024-04-16T05:32:55.513374+0000","last_became_peered":"2024-04-16T05:32:55.513374+0000","last_unstale":"2024-04-16T05:34:21.512368+0000","last_undegraded":"2024-04-16T05:34:21.512368+0000","last_fullsized":"2024-04-16T05:34:21.512368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:06:23.390908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499357+0000","last_change":"2024-04-16T05:32:55.519276+0000","last_active":"2024-04-16T05:34:21.499357+0000","last_peered":"2024-04-16T05:34:21.499357+0000","last_clean":"2024-04-16T05:34:21.499357+0000","last_became_active":"2024-04-16T05:32:55.518729+0000","last_became_peered":"2024-04-16T05:32:55.518729+0000","last_unstale":"2024-04-16T05:34:21.499357+0000","last_undegraded":"2024-04-16T05:34:21.499357+0000","last_fullsized":"2024-04-16T05:34:21.499357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:44:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506133+0000","last_change":"2024-04-16T05:32:55.516379+0000","last_active":"2024-04-16T05:34:21.506133+0000","last_peered":"2024-04-16T05:34:21.506133+0000","last_clean":"2024-04-16T05:34:21.506133+0000","last_became_active":"2024-04-16T05:32:55.515574+0000","last_became_peered":"2024-04-16T05:32:55.515574+0000","last_unstale":"2024-04-16T05:34:21.506133+0000","last_undegraded":"2024-04-16T05:34:21.506133+0000","last_fullsized":"2024-04-16T05:34:21.506133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:16:01.744380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501104+0000","last_change":"2024-04-16T05:33:52.674399+0000","last_active":"2024-04-16T05:34:21.501104+0000","last_peered":"2024-04-16T05:34:21.501104+0000","last_clean":"2024-04-16T05:34:21.501104+0000","last_became_active":"2024-04-16T05:33:52.673964+0000","last_became_peered":"2024-04-16T05:33:52.673964+0000","last_unstale":"2024-04-16T05:34:21.501104+0000","last_undegraded":"2024-04-16T05:34:21.501104+0000","last_fullsized":"2024-04-16T05:34:21.501104+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.166747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501629+0000","last_change":"2024-04-16T05:32:55.509682+0000","last_active":"2024-04-16T05:34:21.501629+0000","last_peered":"2024-04-16T05:34:21.501629+0000","last_clean":"2024-04-16T05:34:21.501629+0000","last_became_active":"2024-04-16T05:32:55.509383+0000","last_became_peered":"2024-04-16T05:32:55.509383+0000","last_unstale":"2024-04-16T05:34:21.501629+0000","last_undegraded":"2024-04-16T05:34:21.501629+0000","last_fullsized":"2024-04-16T05:34:21.501629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:25:23.487003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499529+0000","last_change":"2024-04-16T05:32:55.506338+0000","last_active":"2024-04-16T05:34:21.499529+0000","last_peered":"2024-04-16T05:34:21.499529+0000","last_clean":"2024-04-16T05:34:21.499529+0000","last_became_active":"2024-04-16T05:32:55.506000+0000","last_became_peered":"2024-04-16T05:32:55.506000+0000","last_unstale":"2024-04-16T05:34:21.499529+0000","last_undegraded":"2024-04-16T05:34:21.499529+0000","last_fullsized":"2024-04-16T05:34:21.499529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:37:02.296515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501253+0000","last_change":"2024-04-16T05:32:55.516897+0000","last_active":"2024-04-16T05:34:21.501253+0000","last_peered":"2024-04-16T05:34:21.501253+0000","last_clean":"2024-04-16T05:34:21.501253+0000","last_became_active":"2024-04-16T05:32:55.516489+0000","last_became_peered":"2024-04-16T05:32:55.516489+0000","last_unstale":"2024-04-16T05:34:21.501253+0000","last_undegraded":"2024-04-16T05:34:21.501253+0000","last_fullsized":"2024-04-16T05:34:21.501253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:26:08.289501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511131+0000","last_change":"2024-04-16T05:32:55.507374+0000","last_active":"2024-04-16T05:34:21.511131+0000","last_peered":"2024-04-16T05:34:21.511131+0000","last_clean":"2024-04-16T05:34:21.511131+0000","last_became_active":"2024-04-16T05:32:55.507045+0000","last_became_peered":"2024-04-16T05:32:55.507045+0000","last_unstale":"2024-04-16T05:34:21.511131+0000","last_undegraded":"2024-04-16T05:34:21.511131+0000","last_fullsized":"2024-04-16T05:34:21.511131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:35.968704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511583+0000","last_change":"2024-04-16T05:32:55.508845+0000","last_active":"2024-04-16T05:34:21.511583+0000","last_peered":"2024-04-16T05:34:21.511583+0000","last_clean":"2024-04-16T05:34:21.511583+0000","last_became_active":"2024-04-16T05:32:55.508527+0000","last_became_peered":"2024-04-16T05:32:55.508527+0000","last_unstale":"2024-04-16T05:34:21.511583+0000","last_undegraded":"2024-04-16T05:34:21.511583+0000","last_fullsized":"2024-04-16T05:34:21.511583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:20:15.033043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.510628+0000","last_change":"2024-04-16T05:32:55.506059+0000","last_active":"2024-04-16T05:34:21.510628+0000","last_peered":"2024-04-16T05:34:21.510628+0000","last_clean":"2024-04-16T05:34:21.510628+0000","last_became_active":"2024-04-16T05:32:55.505728+0000","last_became_peered":"2024-04-16T05:32:55.505728+0000","last_unstale":"2024-04-16T05:34:21.510628+0000","last_undegraded":"2024-04-16T05:34:21.510628+0000","last_fullsized":"2024-04-16T05:34:21.510628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:19.253543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.503252+0000","last_change":"2024-04-16T05:32:55.506791+0000","last_active":"2024-04-16T05:34:21.503252+0000","last_peered":"2024-04-16T05:34:21.503252+0000","last_clean":"2024-04-16T05:34:21.503252+0000","last_became_active":"2024-04-16T05:32:55.506379+0000","last_became_peered":"2024-04-16T05:32:55.506379+0000","last_unstale":"2024-04-16T05:34:21.503252+0000","last_undegraded":"2024-04-16T05:34:21.503252+0000","last_fullsized":"2024-04-16T05:34:21.503252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:09:11.457144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.582503+0000","last_change":"2024-04-16T05:33:52.582503+0000","last_active":"2024-04-16T05:33:52.582503+0000","last_peered":"2024-04-16T05:33:52.582503+0000","last_clean":"2024-04-16T05:33:52.582503+0000","last_became_active":"2024-04-16T05:33:52.581996+0000","last_became_peered":"2024-04-16T05:33:52.581996+0000","last_unstale":"2024-04-16T05:33:52.582503+0000","last_undegraded":"2024-04-16T05:33:52.582503+0000","last_fullsized":"2024-04-16T05:33:52.582503+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:34:40.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":114,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:27.715903+0000","last_change":"2024-04-16T05:32:54.504826+0000","last_active":"2024-04-16T05:34:27.715903+0000","last_peered":"2024-04-16T05:34:27.715903+0000","last_clean":"2024-04-16T05:34:27.715903+0000","last_became_active":"2024-04-16T05:32:54.504526+0000","last_became_peered":"2024-04-16T05:32:54.504526+0000","last_unstale":"2024-04-16T05:34:27.715903+0000","last_undegraded":"2024-04-16T05:34:27.715903+0000","last_fullsized":"2024-04-16T05:34:27.715903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:21:28.434692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.496238+0000","last_change":"2024-04-16T05:33:52.580931+0000","last_active":"2024-04-16T05:34:21.496238+0000","last_peered":"2024-04-16T05:34:21.496238+0000","last_clean":"2024-04-16T05:34:21.496238+0000","last_became_active":"2024-04-16T05:33:52.580685+0000","last_became_peered":"2024-04-16T05:33:52.580685+0000","last_unstale":"2024-04-16T05:34:21.496238+0000","last_undegraded":"2024-04-16T05:34:21.496238+0000","last_fullsized":"2024-04-16T05:34:21.496238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:35:34.548838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575474+0000","last_change":"2024-04-16T05:32:59.524687+0000","last_active":"2024-04-16T05:33:52.575474+0000","last_peered":"2024-04-16T05:33:52.575474+0000","last_clean":"2024-04-16T05:33:52.575474+0000","last_became_active":"2024-04-16T05:32:59.524228+0000","last_became_peered":"2024-04-16T05:32:59.524228+0000","last_unstale":"2024-04-16T05:33:52.575474+0000","last_undegraded":"2024-04-16T05:33:52.575474+0000","last_fullsized":"2024-04-16T05:33:52.575474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:23:15.244359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579070+0000","last_change":"2024-04-16T05:32:54.504062+0000","last_active":"2024-04-16T05:33:52.579070+0000","last_peered":"2024-04-16T05:33:52.579070+0000","last_clean":"2024-04-16T05:33:52.579070+0000","last_became_active":"2024-04-16T05:32:54.503745+0000","last_became_peered":"2024-04-16T05:32:54.503745+0000","last_unstale":"2024-04-16T05:33:52.579070+0000","last_undegraded":"2024-04-16T05:33:52.579070+0000","last_fullsized":"2024-04-16T05:33:52.579070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:29:34.157937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511122+0000","last_change":"2024-04-16T05:32:55.509820+0000","last_active":"2024-04-16T05:34:21.511122+0000","last_peered":"2024-04-16T05:34:21.511122+0000","last_clean":"2024-04-16T05:34:21.511122+0000","last_became_active":"2024-04-16T05:32:55.509473+0000","last_became_peered":"2024-04-16T05:32:55.509473+0000","last_unstale":"2024-04-16T05:34:21.511122+0000","last_undegraded":"2024-04-16T05:34:21.511122+0000","last_fullsized":"2024-04-16T05:34:21.511122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:12:22.433189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.678903+0000","last_change":"2024-04-16T05:33:52.678903+0000","last_active":"2024-04-16T05:33:52.678903+0000","last_peered":"2024-04-16T05:33:52.678903+0000","last_clean":"2024-04-16T05:33:52.678903+0000","last_became_active":"2024-04-16T05:33:52.678388+0000","last_became_peered":"2024-04-16T05:33:52.678388+0000","last_unstale":"2024-04-16T05:33:52.678903+0000","last_undegraded":"2024-04-16T05:33:52.678903+0000","last_fullsized":"2024-04-16T05:33:52.678903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:07:01.060645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576294+0000","last_change":"2024-04-16T05:32:54.511469+0000","last_active":"2024-04-16T05:33:52.576294+0000","last_peered":"2024-04-16T05:33:52.576294+0000","last_clean":"2024-04-16T05:33:52.576294+0000","last_became_active":"2024-04-16T05:32:54.511120+0000","last_became_peered":"2024-04-16T05:32:54.511120+0000","last_unstale":"2024-04-16T05:33:52.576294+0000","last_undegraded":"2024-04-16T05:33:52.576294+0000","last_fullsized":"2024-04-16T05:33:52.576294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.508406+0000","last_change":"2024-04-16T05:32:55.512308+0000","last_active":"2024-04-16T05:34:21.508406+0000","last_peered":"2024-04-16T05:34:21.508406+0000","last_clean":"2024-04-16T05:34:21.508406+0000","last_became_active":"2024-04-16T05:32:55.511625+0000","last_became_peered":"2024-04-16T05:32:55.511625+0000","last_unstale":"2024-04-16T05:34:21.508406+0000","last_undegraded":"2024-04-16T05:34:21.508406+0000","last_fullsized":"2024-04-16T05:34:21.508406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:29:49.323092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575546+0000","last_change":"2024-04-16T05:32:59.519306+0000","last_active":"2024-04-16T05:33:52.575546+0000","last_peered":"2024-04-16T05:33:52.575546+0000","last_clean":"2024-04-16T05:33:52.575546+0000","last_became_active":"2024-04-16T05:32:59.518797+0000","last_became_peered":"2024-04-16T05:32:59.518797+0000","last_unstale":"2024-04-16T05:33:52.575546+0000","last_undegraded":"2024-04-16T05:33:52.575546+0000","last_fullsized":"2024-04-16T05:33:52.575546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:04:12.389004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575820+0000","last_change":"2024-04-16T05:32:54.505102+0000","last_active":"2024-04-16T05:33:52.575820+0000","last_peered":"2024-04-16T05:33:52.575820+0000","last_clean":"2024-04-16T05:33:52.575820+0000","last_became_active":"2024-04-16T05:32:54.504806+0000","last_became_peered":"2024-04-16T05:32:54.504806+0000","last_unstale":"2024-04-16T05:33:52.575820+0000","last_undegraded":"2024-04-16T05:33:52.575820+0000","last_fullsized":"2024-04-16T05:33:52.575820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:19.467690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.512005+0000","last_change":"2024-04-16T05:32:55.517097+0000","last_active":"2024-04-16T05:34:21.512005+0000","last_peered":"2024-04-16T05:34:21.512005+0000","last_clean":"2024-04-16T05:34:21.512005+0000","last_became_active":"2024-04-16T05:32:55.516717+0000","last_became_peered":"2024-04-16T05:32:55.516717+0000","last_unstale":"2024-04-16T05:34:21.512005+0000","last_undegraded":"2024-04-16T05:34:21.512005+0000","last_fullsized":"2024-04-16T05:34:21.512005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:10:44.544935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577331+0000","last_change":"2024-04-16T05:32:59.521429+0000","last_active":"2024-04-16T05:33:52.577331+0000","last_peered":"2024-04-16T05:33:52.577331+0000","last_clean":"2024-04-16T05:33:52.577331+0000","last_became_active":"2024-04-16T05:32:59.520789+0000","last_became_peered":"2024-04-16T05:32:59.520789+0000","last_unstale":"2024-04-16T05:33:52.577331+0000","last_undegraded":"2024-04-16T05:33:52.577331+0000","last_fullsized":"2024-04-16T05:33:52.577331+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:40:14.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:27.723549+0000","last_change":"2024-04-16T05:32:54.507374+0000","last_active":"2024-04-16T05:34:27.723549+0000","last_peered":"2024-04-16T05:34:27.723549+0000","last_clean":"2024-04-16T05:34:27.723549+0000","last_became_active":"2024-04-16T05:32:54.507055+0000","last_became_peered":"2024-04-16T05:32:54.507055+0000","last_unstale":"2024-04-16T05:34:27.723549+0000","last_undegraded":"2024-04-16T05:34:27.723549+0000","last_fullsized":"2024-04-16T05:34:27.723549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:00:57.761971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.482151+0000","last_change":"2024-04-16T05:32:55.513980+0000","last_active":"2024-04-16T05:34:21.482151+0000","last_peered":"2024-04-16T05:34:21.482151+0000","last_clean":"2024-04-16T05:34:21.482151+0000","last_became_active":"2024-04-16T05:32:55.513622+0000","last_became_peered":"2024-04-16T05:32:55.513622+0000","last_unstale":"2024-04-16T05:34:21.482151+0000","last_undegraded":"2024-04-16T05:34:21.482151+0000","last_fullsized":"2024-04-16T05:34:21.482151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:16:10.018381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577404+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:33:52.577404+0000","last_peered":"2024-04-16T05:33:52.577404+0000","last_clean":"2024-04-16T05:33:52.577404+0000","last_became_active":"2024-04-16T05:32:59.523032+0000","last_became_peered":"2024-04-16T05:32:59.523032+0000","last_unstale":"2024-04-16T05:33:52.577404+0000","last_undegraded":"2024-04-16T05:33:52.577404+0000","last_fullsized":"2024-04-16T05:33:52.577404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:33:17.897241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.673260+0000","last_change":"2024-04-16T05:32:54.504527+0000","last_active":"2024-04-16T05:33:52.673260+0000","last_peered":"2024-04-16T05:33:52.673260+0000","last_clean":"2024-04-16T05:33:52.673260+0000","last_became_active":"2024-04-16T05:32:54.504187+0000","last_became_peered":"2024-04-16T05:32:54.504187+0000","last_unstale":"2024-04-16T05:33:52.673260+0000","last_undegraded":"2024-04-16T05:33:52.673260+0000","last_fullsized":"2024-04-16T05:33:52.673260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.414362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501955+0000","last_change":"2024-04-16T05:32:55.520270+0000","last_active":"2024-04-16T05:34:21.501955+0000","last_peered":"2024-04-16T05:34:21.501955+0000","last_clean":"2024-04-16T05:34:21.501955+0000","last_became_active":"2024-04-16T05:32:55.519895+0000","last_became_peered":"2024-04-16T05:32:55.519895+0000","last_unstale":"2024-04-16T05:34:21.501955+0000","last_undegraded":"2024-04-16T05:34:21.501955+0000","last_fullsized":"2024-04-16T05:34:21.501955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:09.967836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579270+0000","last_change":"2024-04-16T05:32:59.518997+0000","last_active":"2024-04-16T05:33:52.579270+0000","last_peered":"2024-04-16T05:33:52.579270+0000","last_clean":"2024-04-16T05:33:52.579270+0000","last_became_active":"2024-04-16T05:32:59.518558+0000","last_became_peered":"2024-04-16T05:32:59.518558+0000","last_unstale":"2024-04-16T05:33:52.579270+0000","last_undegraded":"2024-04-16T05:33:52.579270+0000","last_fullsized":"2024-04-16T05:33:52.579270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:04:30.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.671972+0000","last_change":"2024-04-16T05:32:54.502074+0000","last_active":"2024-04-16T05:33:52.671972+0000","last_peered":"2024-04-16T05:33:52.671972+0000","last_clean":"2024-04-16T05:33:52.671972+0000","last_became_active":"2024-04-16T05:32:54.501730+0000","last_became_peered":"2024-04-16T05:32:54.501730+0000","last_unstale":"2024-04-16T05:33:52.671972+0000","last_undegraded":"2024-04-16T05:33:52.671972+0000","last_fullsized":"2024-04-16T05:33:52.671972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:05:02.929799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.507740+0000","last_change":"2024-04-16T05:32:55.504557+0000","last_active":"2024-04-16T05:34:21.507740+0000","last_peered":"2024-04-16T05:34:21.507740+0000","last_clean":"2024-04-16T05:34:21.507740+0000","last_became_active":"2024-04-16T05:32:55.504244+0000","last_became_peered":"2024-04-16T05:32:55.504244+0000","last_unstale":"2024-04-16T05:34:21.507740+0000","last_undegraded":"2024-04-16T05:34:21.507740+0000","last_fullsized":"2024-04-16T05:34:21.507740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:33:40.500436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577138+0000","last_change":"2024-04-16T05:32:59.524681+0000","last_active":"2024-04-16T05:33:52.577138+0000","last_peered":"2024-04-16T05:33:52.577138+0000","last_clean":"2024-04-16T05:33:52.577138+0000","last_became_active":"2024-04-16T05:32:59.524217+0000","last_became_peered":"2024-04-16T05:32:59.524217+0000","last_unstale":"2024-04-16T05:33:52.577138+0000","last_undegraded":"2024-04-16T05:33:52.577138+0000","last_fullsized":"2024-04-16T05:33:52.577138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:28.314442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677063+0000","last_change":"2024-04-16T05:32:54.508242+0000","last_active":"2024-04-16T05:33:52.677063+0000","last_peered":"2024-04-16T05:33:52.677063+0000","last_clean":"2024-04-16T05:33:52.677063+0000","last_became_active":"2024-04-16T05:32:54.507782+0000","last_became_peered":"2024-04-16T05:32:54.507782+0000","last_unstale":"2024-04-16T05:33:52.677063+0000","last_undegraded":"2024-04-16T05:33:52.677063+0000","last_fullsized":"2024-04-16T05:33:52.677063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:27:26.805716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.500686+0000","last_change":"2024-04-16T05:32:55.508985+0000","last_active":"2024-04-16T05:34:21.500686+0000","last_peered":"2024-04-16T05:34:21.500686+0000","last_clean":"2024-04-16T05:34:21.500686+0000","last_became_active":"2024-04-16T05:32:55.508659+0000","last_became_peered":"2024-04-16T05:32:55.508659+0000","last_unstale":"2024-04-16T05:34:21.500686+0000","last_undegraded":"2024-04-16T05:34:21.500686+0000","last_fullsized":"2024-04-16T05:34:21.500686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:43:41.534155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578555+0000","last_change":"2024-04-16T05:32:59.521257+0000","last_active":"2024-04-16T05:33:52.578555+0000","last_peered":"2024-04-16T05:33:52.578555+0000","last_clean":"2024-04-16T05:33:52.578555+0000","last_became_active":"2024-04-16T05:32:59.520848+0000","last_became_peered":"2024-04-16T05:32:59.520848+0000","last_unstale":"2024-04-16T05:33:52.578555+0000","last_undegraded":"2024-04-16T05:33:52.578555+0000","last_fullsized":"2024-04-16T05:33:52.578555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:14.219378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":179,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:29.725587+0000","last_change":"2024-04-16T05:32:54.500684+0000","last_active":"2024-04-16T05:34:29.725587+0000","last_peered":"2024-04-16T05:34:29.725587+0000","last_clean":"2024-04-16T05:34:29.725587+0000","last_became_active":"2024-04-16T05:32:54.500290+0000","last_became_peered":"2024-04-16T05:32:54.500290+0000","last_unstale":"2024-04-16T05:34:29.725587+0000","last_undegraded":"2024-04-16T05:34:29.725587+0000","last_fullsized":"2024-04-16T05:34:29.725587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:20:55.263313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.512532+0000","last_change":"2024-04-16T05:32:55.511224+0000","last_active":"2024-04-16T05:34:21.512532+0000","last_peered":"2024-04-16T05:34:21.512532+0000","last_clean":"2024-04-16T05:34:21.512532+0000","last_became_active":"2024-04-16T05:32:55.510891+0000","last_became_peered":"2024-04-16T05:32:55.510891+0000","last_unstale":"2024-04-16T05:34:21.512532+0000","last_undegraded":"2024-04-16T05:34:21.512532+0000","last_fullsized":"2024-04-16T05:34:21.512532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:27:24.894658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.573962+0000","last_change":"2024-04-16T05:32:59.521695+0000","last_active":"2024-04-16T05:33:52.573962+0000","last_peered":"2024-04-16T05:33:52.573962+0000","last_clean":"2024-04-16T05:33:52.573962+0000","last_became_active":"2024-04-16T05:32:59.520569+0000","last_became_peered":"2024-04-16T05:32:59.520569+0000","last_unstale":"2024-04-16T05:33:52.573962+0000","last_undegraded":"2024-04-16T05:33:52.573962+0000","last_fullsized":"2024-04-16T05:33:52.573962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:01:14.726043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.580829+0000","last_change":"2024-04-16T05:32:54.505058+0000","last_active":"2024-04-16T05:33:52.580829+0000","last_peered":"2024-04-16T05:33:52.580829+0000","last_clean":"2024-04-16T05:33:52.580829+0000","last_became_active":"2024-04-16T05:32:54.504636+0000","last_became_peered":"2024-04-16T05:32:54.504636+0000","last_unstale":"2024-04-16T05:33:52.580829+0000","last_undegraded":"2024-04-16T05:33:52.580829+0000","last_fullsized":"2024-04-16T05:33:52.580829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:49:14.429468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.502136+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:34:21.502136+0000","last_peered":"2024-04-16T05:34:21.502136+0000","last_clean":"2024-04-16T05:34:21.502136+0000","last_became_active":"2024-04-16T05:33:52.678570+0000","last_became_peered":"2024-04-16T05:33:52.678570+0000","last_unstale":"2024-04-16T05:34:21.502136+0000","last_undegraded":"2024-04-16T05:34:21.502136+0000","last_fullsized":"2024-04-16T05:34:21.502136+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:56:32.405491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576513+0000","last_change":"2024-04-16T05:32:59.520394+0000","last_active":"2024-04-16T05:33:52.576513+0000","last_peered":"2024-04-16T05:33:52.576513+0000","last_clean":"2024-04-16T05:33:52.576513+0000","last_became_active":"2024-04-16T05:32:59.519947+0000","last_became_peered":"2024-04-16T05:32:59.519947+0000","last_unstale":"2024-04-16T05:33:52.576513+0000","last_undegraded":"2024-04-16T05:33:52.576513+0000","last_fullsized":"2024-04-16T05:33:52.576513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:33:27.647919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677336+0000","last_change":"2024-04-16T05:32:54.505350+0000","last_active":"2024-04-16T05:33:52.677336+0000","last_peered":"2024-04-16T05:33:52.677336+0000","last_clean":"2024-04-16T05:33:52.677336+0000","last_became_active":"2024-04-16T05:32:54.505011+0000","last_became_peered":"2024-04-16T05:32:54.505011+0000","last_unstale":"2024-04-16T05:33:52.677336+0000","last_undegraded":"2024-04-16T05:33:52.677336+0000","last_fullsized":"2024-04-16T05:33:52.677336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:03:00.276862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.510220+0000","last_change":"2024-04-16T05:33:52.673799+0000","last_active":"2024-04-16T05:34:21.510220+0000","last_peered":"2024-04-16T05:34:21.510220+0000","last_clean":"2024-04-16T05:34:21.510220+0000","last_became_active":"2024-04-16T05:33:52.673344+0000","last_became_peered":"2024-04-16T05:33:52.673344+0000","last_unstale":"2024-04-16T05:34:21.510220+0000","last_undegraded":"2024-04-16T05:34:21.510220+0000","last_fullsized":"2024-04-16T05:34:21.510220+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:07.387912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.574107+0000","last_change":"2024-04-16T05:32:59.519660+0000","last_active":"2024-04-16T05:33:52.574107+0000","last_peered":"2024-04-16T05:33:52.574107+0000","last_clean":"2024-04-16T05:33:52.574107+0000","last_became_active":"2024-04-16T05:32:59.518198+0000","last_became_peered":"2024-04-16T05:32:59.518198+0000","last_unstale":"2024-04-16T05:33:52.574107+0000","last_undegraded":"2024-04-16T05:33:52.574107+0000","last_fullsized":"2024-04-16T05:33:52.574107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:03:10.096252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":217,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:29.725076+0000","last_change":"2024-04-16T05:32:54.501530+0000","last_active":"2024-04-16T05:34:29.725076+0000","last_peered":"2024-04-16T05:34:29.725076+0000","last_clean":"2024-04-16T05:34:29.725076+0000","last_became_active":"2024-04-16T05:32:54.501159+0000","last_became_peered":"2024-04-16T05:32:54.501159+0000","last_unstale":"2024-04-16T05:34:29.725076+0000","last_undegraded":"2024-04-16T05:34:29.725076+0000","last_fullsized":"2024-04-16T05:34:29.725076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:26:22.240137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61439,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":151,"num_read_kb":136,"num_write":39,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.475321+0000","last_change":"2024-04-16T05:32:55.512210+0000","last_active":"2024-04-16T05:34:21.475321+0000","last_peered":"2024-04-16T05:34:21.475321+0000","last_clean":"2024-04-16T05:34:21.475321+0000","last_became_active":"2024-04-16T05:32:55.511672+0000","last_became_peered":"2024-04-16T05:32:55.511672+0000","last_unstale":"2024-04-16T05:34:21.475321+0000","last_undegraded":"2024-04-16T05:34:21.475321+0000","last_fullsized":"2024-04-16T05:34:21.475321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:10:53.755803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677945+0000","last_change":"2024-04-16T05:32:59.527552+0000","last_active":"2024-04-16T05:33:52.677945+0000","last_peered":"2024-04-16T05:33:52.677945+0000","last_clean":"2024-04-16T05:33:52.677945+0000","last_became_active":"2024-04-16T05:32:59.526919+0000","last_became_peered":"2024-04-16T05:32:59.526919+0000","last_unstale":"2024-04-16T05:33:52.677945+0000","last_undegraded":"2024-04-16T05:33:52.677945+0000","last_fullsized":"2024-04-16T05:33:52.677945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:00:46.342383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577328+0000","last_change":"2024-04-16T05:32:54.494589+0000","last_active":"2024-04-16T05:33:52.577328+0000","last_peered":"2024-04-16T05:33:52.577328+0000","last_clean":"2024-04-16T05:33:52.577328+0000","last_became_active":"2024-04-16T05:32:54.494221+0000","last_became_peered":"2024-04-16T05:32:54.494221+0000","last_unstale":"2024-04-16T05:33:52.577328+0000","last_undegraded":"2024-04-16T05:33:52.577328+0000","last_fullsized":"2024-04-16T05:33:52.577328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:17:53.245359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.508841+0000","last_change":"2024-04-16T05:32:55.502724+0000","last_active":"2024-04-16T05:34:21.508841+0000","last_peered":"2024-04-16T05:34:21.508841+0000","last_clean":"2024-04-16T05:34:21.508841+0000","last_became_active":"2024-04-16T05:32:55.502415+0000","last_became_peered":"2024-04-16T05:32:55.502415+0000","last_unstale":"2024-04-16T05:34:21.508841+0000","last_undegraded":"2024-04-16T05:34:21.508841+0000","last_fullsized":"2024-04-16T05:34:21.508841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:20:44.443926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578919+0000","last_change":"2024-04-16T05:32:59.522243+0000","last_active":"2024-04-16T05:33:52.578919+0000","last_peered":"2024-04-16T05:33:52.578919+0000","last_clean":"2024-04-16T05:33:52.578919+0000","last_became_active":"2024-04-16T05:32:59.521823+0000","last_became_peered":"2024-04-16T05:32:59.521823+0000","last_unstale":"2024-04-16T05:33:52.578919+0000","last_undegraded":"2024-04-16T05:33:52.578919+0000","last_fullsized":"2024-04-16T05:33:52.578919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:09:11.158434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677065+0000","last_change":"2024-04-16T05:32:54.507505+0000","last_active":"2024-04-16T05:33:52.677065+0000","last_peered":"2024-04-16T05:33:52.677065+0000","last_clean":"2024-04-16T05:33:52.677065+0000","last_became_active":"2024-04-16T05:32:54.506558+0000","last_became_peered":"2024-04-16T05:32:54.506558+0000","last_unstale":"2024-04-16T05:33:52.677065+0000","last_undegraded":"2024-04-16T05:33:52.677065+0000","last_fullsized":"2024-04-16T05:33:52.677065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:36:17.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.508547+0000","last_change":"2024-04-16T05:32:55.518998+0000","last_active":"2024-04-16T05:34:21.508547+0000","last_peered":"2024-04-16T05:34:21.508547+0000","last_clean":"2024-04-16T05:34:21.508547+0000","last_became_active":"2024-04-16T05:32:55.518640+0000","last_became_peered":"2024-04-16T05:32:55.518640+0000","last_unstale":"2024-04-16T05:34:21.508547+0000","last_undegraded":"2024-04-16T05:34:21.508547+0000","last_fullsized":"2024-04-16T05:34:21.508547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:42:44.667504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.574346+0000","last_change":"2024-04-16T05:32:59.519050+0000","last_active":"2024-04-16T05:33:52.574346+0000","last_peered":"2024-04-16T05:33:52.574346+0000","last_clean":"2024-04-16T05:33:52.574346+0000","last_became_active":"2024-04-16T05:32:59.518663+0000","last_became_peered":"2024-04-16T05:32:59.518663+0000","last_unstale":"2024-04-16T05:33:52.574346+0000","last_undegraded":"2024-04-16T05:33:52.574346+0000","last_fullsized":"2024-04-16T05:33:52.574346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:36:19.649605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":112,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:29.727111+0000","last_change":"2024-04-16T05:32:54.500940+0000","last_active":"2024-04-16T05:34:29.727111+0000","last_peered":"2024-04-16T05:34:29.727111+0000","last_clean":"2024-04-16T05:34:29.727111+0000","last_became_active":"2024-04-16T05:32:54.500542+0000","last_became_peered":"2024-04-16T05:32:54.500542+0000","last_unstale":"2024-04-16T05:34:29.727111+0000","last_undegraded":"2024-04-16T05:34:29.727111+0000","last_fullsized":"2024-04-16T05:34:29.727111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:38:00.232244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506632+0000","last_change":"2024-04-16T05:32:55.509889+0000","last_active":"2024-04-16T05:34:21.506632+0000","last_peered":"2024-04-16T05:34:21.506632+0000","last_clean":"2024-04-16T05:34:21.506632+0000","last_became_active":"2024-04-16T05:32:55.508911+0000","last_became_peered":"2024-04-16T05:32:55.508911+0000","last_unstale":"2024-04-16T05:34:21.506632+0000","last_undegraded":"2024-04-16T05:34:21.506632+0000","last_fullsized":"2024-04-16T05:34:21.506632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:14:46.609127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.672428+0000","last_change":"2024-04-16T05:32:59.523555+0000","last_active":"2024-04-16T05:33:52.672428+0000","last_peered":"2024-04-16T05:33:52.672428+0000","last_clean":"2024-04-16T05:33:52.672428+0000","last_became_active":"2024-04-16T05:32:59.523187+0000","last_became_peered":"2024-04-16T05:32:59.523187+0000","last_unstale":"2024-04-16T05:33:52.672428+0000","last_undegraded":"2024-04-16T05:33:52.672428+0000","last_fullsized":"2024-04-16T05:33:52.672428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:35.049355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575135+0000","last_change":"2024-04-16T05:32:54.503247+0000","last_active":"2024-04-16T05:33:52.575135+0000","last_peered":"2024-04-16T05:33:52.575135+0000","last_clean":"2024-04-16T05:33:52.575135+0000","last_became_active":"2024-04-16T05:32:54.502927+0000","last_became_peered":"2024-04-16T05:32:54.502927+0000","last_unstale":"2024-04-16T05:33:52.575135+0000","last_undegraded":"2024-04-16T05:33:52.575135+0000","last_fullsized":"2024-04-16T05:33:52.575135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:36:17.585827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.504335+0000","last_change":"2024-04-16T05:32:55.519265+0000","last_active":"2024-04-16T05:34:21.504335+0000","last_peered":"2024-04-16T05:34:21.504335+0000","last_clean":"2024-04-16T05:34:21.504335+0000","last_became_active":"2024-04-16T05:32:55.518840+0000","last_became_peered":"2024-04-16T05:32:55.518840+0000","last_unstale":"2024-04-16T05:34:21.504335+0000","last_undegraded":"2024-04-16T05:34:21.504335+0000","last_fullsized":"2024-04-16T05:34:21.504335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:56.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677057+0000","last_change":"2024-04-16T05:32:59.525275+0000","last_active":"2024-04-16T05:33:52.677057+0000","last_peered":"2024-04-16T05:33:52.677057+0000","last_clean":"2024-04-16T05:33:52.677057+0000","last_became_active":"2024-04-16T05:32:59.524546+0000","last_became_peered":"2024-04-16T05:32:59.524546+0000","last_unstale":"2024-04-16T05:33:52.677057+0000","last_undegraded":"2024-04-16T05:33:52.677057+0000","last_fullsized":"2024-04-16T05:33:52.677057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:07.561898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576966+0000","last_change":"2024-04-16T05:32:54.506904+0000","last_active":"2024-04-16T05:33:52.576966+0000","last_peered":"2024-04-16T05:33:52.576966+0000","last_clean":"2024-04-16T05:33:52.576966+0000","last_became_active":"2024-04-16T05:32:54.505654+0000","last_became_peered":"2024-04-16T05:32:54.505654+0000","last_unstale":"2024-04-16T05:33:52.576966+0000","last_undegraded":"2024-04-16T05:33:52.576966+0000","last_fullsized":"2024-04-16T05:33:52.576966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:30.444817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499938+0000","last_change":"2024-04-16T05:32:55.515685+0000","last_active":"2024-04-16T05:34:21.499938+0000","last_peered":"2024-04-16T05:34:21.499938+0000","last_clean":"2024-04-16T05:34:21.499938+0000","last_became_active":"2024-04-16T05:32:55.515367+0000","last_became_peered":"2024-04-16T05:32:55.515367+0000","last_unstale":"2024-04-16T05:34:21.499938+0000","last_undegraded":"2024-04-16T05:34:21.499938+0000","last_fullsized":"2024-04-16T05:34:21.499938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:46.670143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575592+0000","last_change":"2024-04-16T05:33:00.419828+0000","last_active":"2024-04-16T05:33:52.575592+0000","last_peered":"2024-04-16T05:33:52.575592+0000","last_clean":"2024-04-16T05:33:52.575592+0000","last_became_active":"2024-04-16T05:33:00.419321+0000","last_became_peered":"2024-04-16T05:33:00.419321+0000","last_unstale":"2024-04-16T05:33:52.575592+0000","last_undegraded":"2024-04-16T05:33:52.575592+0000","last_fullsized":"2024-04-16T05:33:52.575592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:56.694914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.574373+0000","last_change":"2024-04-16T05:32:54.505369+0000","last_active":"2024-04-16T05:33:52.574373+0000","last_peered":"2024-04-16T05:33:52.574373+0000","last_clean":"2024-04-16T05:33:52.574373+0000","last_became_active":"2024-04-16T05:32:54.505043+0000","last_became_peered":"2024-04-16T05:32:54.505043+0000","last_unstale":"2024-04-16T05:33:52.574373+0000","last_undegraded":"2024-04-16T05:33:52.574373+0000","last_fullsized":"2024-04-16T05:33:52.574373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:34.242117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.509337+0000","last_change":"2024-04-16T05:32:55.509970+0000","last_active":"2024-04-16T05:34:21.509337+0000","last_peered":"2024-04-16T05:34:21.509337+0000","last_clean":"2024-04-16T05:34:21.509337+0000","last_became_active":"2024-04-16T05:32:55.509632+0000","last_became_peered":"2024-04-16T05:32:55.509632+0000","last_unstale":"2024-04-16T05:34:21.509337+0000","last_undegraded":"2024-04-16T05:34:21.509337+0000","last_fullsized":"2024-04-16T05:34:21.509337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:33:55.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.672444+0000","last_change":"2024-04-16T05:32:59.527007+0000","last_active":"2024-04-16T05:33:52.672444+0000","last_peered":"2024-04-16T05:33:52.672444+0000","last_clean":"2024-04-16T05:33:52.672444+0000","last_became_active":"2024-04-16T05:32:59.526645+0000","last_became_peered":"2024-04-16T05:32:59.526645+0000","last_unstale":"2024-04-16T05:33:52.672444+0000","last_undegraded":"2024-04-16T05:33:52.672444+0000","last_fullsized":"2024-04-16T05:33:52.672444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:46:06.698912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:29.719513+0000","last_change":"2024-04-16T05:32:54.503409+0000","last_active":"2024-04-16T05:34:29.719513+0000","last_peered":"2024-04-16T05:34:29.719513+0000","last_clean":"2024-04-16T05:34:29.719513+0000","last_became_active":"2024-04-16T05:32:54.503069+0000","last_became_peered":"2024-04-16T05:32:54.503069+0000","last_unstale":"2024-04-16T05:34:29.719513+0000","last_undegraded":"2024-04-16T05:34:29.719513+0000","last_fullsized":"2024-04-16T05:34:29.719513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:45:10.535225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501668+0000","last_change":"2024-04-16T05:32:55.519744+0000","last_active":"2024-04-16T05:34:21.501668+0000","last_peered":"2024-04-16T05:34:21.501668+0000","last_clean":"2024-04-16T05:34:21.501668+0000","last_became_active":"2024-04-16T05:32:55.519091+0000","last_became_peered":"2024-04-16T05:32:55.519091+0000","last_unstale":"2024-04-16T05:34:21.501668+0000","last_undegraded":"2024-04-16T05:34:21.501668+0000","last_fullsized":"2024-04-16T05:34:21.501668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:19.417691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.672436+0000","last_change":"2024-04-16T05:33:00.419494+0000","last_active":"2024-04-16T05:33:52.672436+0000","last_peered":"2024-04-16T05:33:52.672436+0000","last_clean":"2024-04-16T05:33:52.672436+0000","last_became_active":"2024-04-16T05:33:00.418954+0000","last_became_peered":"2024-04-16T05:33:00.418954+0000","last_unstale":"2024-04-16T05:33:52.672436+0000","last_undegraded":"2024-04-16T05:33:52.672436+0000","last_fullsized":"2024-04-16T05:33:52.672436+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:03:01.835304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.504732+0000","last_change":"2024-04-16T05:32:55.518272+0000","last_active":"2024-04-16T05:34:21.504732+0000","last_peered":"2024-04-16T05:34:21.504732+0000","last_clean":"2024-04-16T05:34:21.504732+0000","last_became_active":"2024-04-16T05:32:55.517928+0000","last_became_peered":"2024-04-16T05:32:55.517928+0000","last_unstale":"2024-04-16T05:34:21.504732+0000","last_undegraded":"2024-04-16T05:34:21.504732+0000","last_fullsized":"2024-04-16T05:34:21.504732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:44:47.739994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:29.726631+0000","last_change":"2024-04-16T05:32:54.503923+0000","last_active":"2024-04-16T05:34:29.726631+0000","last_peered":"2024-04-16T05:34:29.726631+0000","last_clean":"2024-04-16T05:34:29.726631+0000","last_became_active":"2024-04-16T05:32:54.503576+0000","last_became_peered":"2024-04-16T05:32:54.503576+0000","last_unstale":"2024-04-16T05:34:29.726631+0000","last_undegraded":"2024-04-16T05:34:29.726631+0000","last_fullsized":"2024-04-16T05:34:29.726631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:49:36.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.580596+0000","last_change":"2024-04-16T05:32:59.517757+0000","last_active":"2024-04-16T05:33:52.580596+0000","last_peered":"2024-04-16T05:33:52.580596+0000","last_clean":"2024-04-16T05:33:52.580596+0000","last_became_active":"2024-04-16T05:32:59.517357+0000","last_became_peered":"2024-04-16T05:32:59.517357+0000","last_unstale":"2024-04-16T05:33:52.580596+0000","last_undegraded":"2024-04-16T05:33:52.580596+0000","last_fullsized":"2024-04-16T05:33:52.580596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:21.767071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.505253+0000","last_change":"2024-04-16T05:32:55.514617+0000","last_active":"2024-04-16T05:34:21.505253+0000","last_peered":"2024-04-16T05:34:21.505253+0000","last_clean":"2024-04-16T05:34:21.505253+0000","last_became_active":"2024-04-16T05:32:55.512660+0000","last_became_peered":"2024-04-16T05:32:55.512660+0000","last_unstale":"2024-04-16T05:34:21.505253+0000","last_undegraded":"2024-04-16T05:34:21.505253+0000","last_fullsized":"2024-04-16T05:34:21.505253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:31:45.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576333+0000","last_change":"2024-04-16T05:32:54.498650+0000","last_active":"2024-04-16T05:33:52.576333+0000","last_peered":"2024-04-16T05:33:52.576333+0000","last_clean":"2024-04-16T05:33:52.576333+0000","last_became_active":"2024-04-16T05:32:54.497672+0000","last_became_peered":"2024-04-16T05:32:54.497672+0000","last_unstale":"2024-04-16T05:33:52.576333+0000","last_undegraded":"2024-04-16T05:33:52.576333+0000","last_fullsized":"2024-04-16T05:33:52.576333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:36.908809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":363,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579966+0000","last_change":"2024-04-16T05:31:42.831800+0000","last_active":"2024-04-16T05:33:52.579966+0000","last_peered":"2024-04-16T05:33:52.579966+0000","last_clean":"2024-04-16T05:33:52.579966+0000","last_became_active":"2024-04-16T05:31:42.831391+0000","last_became_peered":"2024-04-16T05:31:42.831391+0000","last_unstale":"2024-04-16T05:33:52.579966+0000","last_undegraded":"2024-04-16T05:33:52.579966+0000","last_fullsized":"2024-04-16T05:33:52.579966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_clean_scrub_stamp":"2024-04-16T05:28:15.969690+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.152803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.675549+0000","last_change":"2024-04-16T05:33:52.675549+0000","last_active":"2024-04-16T05:33:52.675549+0000","last_peered":"2024-04-16T05:33:52.675549+0000","last_clean":"2024-04-16T05:33:52.675549+0000","last_became_active":"2024-04-16T05:33:52.674969+0000","last_became_peered":"2024-04-16T05:33:52.674969+0000","last_unstale":"2024-04-16T05:33:52.675549+0000","last_undegraded":"2024-04-16T05:33:52.675549+0000","last_fullsized":"2024-04-16T05:33:52.675549+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:58:33.137841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.476259+0000","last_change":"2024-04-16T05:32:55.512324+0000","last_active":"2024-04-16T05:34:21.476259+0000","last_peered":"2024-04-16T05:34:21.476259+0000","last_clean":"2024-04-16T05:34:21.476259+0000","last_became_active":"2024-04-16T05:32:55.511958+0000","last_became_peered":"2024-04-16T05:32:55.511958+0000","last_unstale":"2024-04-16T05:34:21.476259+0000","last_undegraded":"2024-04-16T05:34:21.476259+0000","last_fullsized":"2024-04-16T05:34:21.476259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:38.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579985+0000","last_change":"2024-04-16T05:32:54.508021+0000","last_active":"2024-04-16T05:33:52.579985+0000","last_peered":"2024-04-16T05:33:52.579985+0000","last_clean":"2024-04-16T05:33:52.579985+0000","last_became_active":"2024-04-16T05:32:54.507645+0000","last_became_peered":"2024-04-16T05:32:54.507645+0000","last_unstale":"2024-04-16T05:33:52.579985+0000","last_undegraded":"2024-04-16T05:33:52.579985+0000","last_fullsized":"2024-04-16T05:33:52.579985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:21:50.721260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.579034+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:33:52.579034+0000","last_peered":"2024-04-16T05:33:52.579034+0000","last_clean":"2024-04-16T05:33:52.579034+0000","last_became_active":"2024-04-16T05:32:59.523159+0000","last_became_peered":"2024-04-16T05:32:59.523159+0000","last_unstale":"2024-04-16T05:33:52.579034+0000","last_undegraded":"2024-04-16T05:33:52.579034+0000","last_fullsized":"2024-04-16T05:33:52.579034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:48:37.696713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576159+0000","last_change":"2024-04-16T05:32:54.503262+0000","last_active":"2024-04-16T05:33:52.576159+0000","last_peered":"2024-04-16T05:33:52.576159+0000","last_clean":"2024-04-16T05:33:52.576159+0000","last_became_active":"2024-04-16T05:32:54.502909+0000","last_became_peered":"2024-04-16T05:32:54.502909+0000","last_unstale":"2024-04-16T05:33:52.576159+0000","last_undegraded":"2024-04-16T05:33:52.576159+0000","last_fullsized":"2024-04-16T05:33:52.576159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:48:32.841921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.490661+0000","last_change":"2024-04-16T05:32:55.507126+0000","last_active":"2024-04-16T05:34:21.490661+0000","last_peered":"2024-04-16T05:34:21.490661+0000","last_clean":"2024-04-16T05:34:21.490661+0000","last_became_active":"2024-04-16T05:32:55.506743+0000","last_became_peered":"2024-04-16T05:32:55.506743+0000","last_unstale":"2024-04-16T05:34:21.490661+0000","last_undegraded":"2024-04-16T05:34:21.490661+0000","last_fullsized":"2024-04-16T05:34:21.490661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:10:58.697649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577532+0000","last_change":"2024-04-16T05:33:00.419435+0000","last_active":"2024-04-16T05:33:52.577532+0000","last_peered":"2024-04-16T05:33:52.577532+0000","last_clean":"2024-04-16T05:33:52.577532+0000","last_became_active":"2024-04-16T05:33:00.418985+0000","last_became_peered":"2024-04-16T05:33:00.418985+0000","last_unstale":"2024-04-16T05:33:52.577532+0000","last_undegraded":"2024-04-16T05:33:52.577532+0000","last_fullsized":"2024-04-16T05:33:52.577532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:08:31.188911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.574583+0000","last_change":"2024-04-16T05:32:54.503349+0000","last_active":"2024-04-16T05:33:52.574583+0000","last_peered":"2024-04-16T05:33:52.574583+0000","last_clean":"2024-04-16T05:33:52.574583+0000","last_became_active":"2024-04-16T05:32:54.503018+0000","last_became_peered":"2024-04-16T05:32:54.503018+0000","last_unstale":"2024-04-16T05:33:52.574583+0000","last_undegraded":"2024-04-16T05:33:52.574583+0000","last_fullsized":"2024-04-16T05:33:52.574583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:34:20.853543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506525+0000","last_change":"2024-04-16T05:32:55.511201+0000","last_active":"2024-04-16T05:34:21.506525+0000","last_peered":"2024-04-16T05:34:21.506525+0000","last_clean":"2024-04-16T05:34:21.506525+0000","last_became_active":"2024-04-16T05:32:55.510906+0000","last_became_peered":"2024-04-16T05:32:55.510906+0000","last_unstale":"2024-04-16T05:34:21.506525+0000","last_undegraded":"2024-04-16T05:34:21.506525+0000","last_fullsized":"2024-04-16T05:34:21.506525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:37:37.147628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577102+0000","last_change":"2024-04-16T05:32:59.518440+0000","last_active":"2024-04-16T05:33:52.577102+0000","last_peered":"2024-04-16T05:33:52.577102+0000","last_clean":"2024-04-16T05:33:52.577102+0000","last_became_active":"2024-04-16T05:32:59.517870+0000","last_became_peered":"2024-04-16T05:32:59.517870+0000","last_unstale":"2024-04-16T05:33:52.577102+0000","last_undegraded":"2024-04-16T05:33:52.577102+0000","last_fullsized":"2024-04-16T05:33:52.577102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:51:31.157418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576507+0000","last_change":"2024-04-16T05:32:54.503983+0000","last_active":"2024-04-16T05:33:52.576507+0000","last_peered":"2024-04-16T05:33:52.576507+0000","last_clean":"2024-04-16T05:33:52.576507+0000","last_became_active":"2024-04-16T05:32:54.502367+0000","last_became_peered":"2024-04-16T05:32:54.502367+0000","last_unstale":"2024-04-16T05:33:52.576507+0000","last_undegraded":"2024-04-16T05:33:52.576507+0000","last_fullsized":"2024-04-16T05:33:52.576507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:49:36.997692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.505135+0000","last_change":"2024-04-16T05:32:55.516812+0000","last_active":"2024-04-16T05:34:21.505135+0000","last_peered":"2024-04-16T05:34:21.505135+0000","last_clean":"2024-04-16T05:34:21.505135+0000","last_became_active":"2024-04-16T05:32:55.516477+0000","last_became_peered":"2024-04-16T05:32:55.516477+0000","last_unstale":"2024-04-16T05:34:21.505135+0000","last_undegraded":"2024-04-16T05:34:21.505135+0000","last_fullsized":"2024-04-16T05:34:21.505135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:24:29.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.576194+0000","last_change":"2024-04-16T05:33:00.419622+0000","last_active":"2024-04-16T05:33:52.576194+0000","last_peered":"2024-04-16T05:33:52.576194+0000","last_clean":"2024-04-16T05:33:52.576194+0000","last_became_active":"2024-04-16T05:33:00.419135+0000","last_became_peered":"2024-04-16T05:33:00.419135+0000","last_unstale":"2024-04-16T05:33:52.576194+0000","last_undegraded":"2024-04-16T05:33:52.576194+0000","last_fullsized":"2024-04-16T05:33:52.576194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:38:02.529192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578401+0000","last_change":"2024-04-16T05:32:54.503652+0000","last_active":"2024-04-16T05:33:52.578401+0000","last_peered":"2024-04-16T05:33:52.578401+0000","last_clean":"2024-04-16T05:33:52.578401+0000","last_became_active":"2024-04-16T05:32:54.503311+0000","last_became_peered":"2024-04-16T05:32:54.503311+0000","last_unstale":"2024-04-16T05:33:52.578401+0000","last_undegraded":"2024-04-16T05:33:52.578401+0000","last_fullsized":"2024-04-16T05:33:52.578401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:49:21.584029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.462985+0000","last_change":"2024-04-16T05:32:55.508145+0000","last_active":"2024-04-16T05:34:21.462985+0000","last_peered":"2024-04-16T05:34:21.462985+0000","last_clean":"2024-04-16T05:34:21.462985+0000","last_became_active":"2024-04-16T05:32:55.507794+0000","last_became_peered":"2024-04-16T05:32:55.507794+0000","last_unstale":"2024-04-16T05:34:21.462985+0000","last_undegraded":"2024-04-16T05:34:21.462985+0000","last_fullsized":"2024-04-16T05:34:21.462985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:11:57.925733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.678920+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:33:52.678920+0000","last_peered":"2024-04-16T05:33:52.678920+0000","last_clean":"2024-04-16T05:33:52.678920+0000","last_became_active":"2024-04-16T05:33:52.678330+0000","last_became_peered":"2024-04-16T05:33:52.678330+0000","last_unstale":"2024-04-16T05:33:52.678920+0000","last_undegraded":"2024-04-16T05:33:52.678920+0000","last_fullsized":"2024-04-16T05:33:52.678920+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:33.323513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578679+0000","last_change":"2024-04-16T05:32:54.500909+0000","last_active":"2024-04-16T05:33:52.578679+0000","last_peered":"2024-04-16T05:33:52.578679+0000","last_clean":"2024-04-16T05:33:52.578679+0000","last_became_active":"2024-04-16T05:32:54.499348+0000","last_became_peered":"2024-04-16T05:32:54.499348+0000","last_unstale":"2024-04-16T05:33:52.578679+0000","last_undegraded":"2024-04-16T05:33:52.578679+0000","last_fullsized":"2024-04-16T05:33:52.578679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:51:49.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.505607+0000","last_change":"2024-04-16T05:32:55.510922+0000","last_active":"2024-04-16T05:34:21.505607+0000","last_peered":"2024-04-16T05:34:21.505607+0000","last_clean":"2024-04-16T05:34:21.505607+0000","last_became_active":"2024-04-16T05:32:55.510546+0000","last_became_peered":"2024-04-16T05:32:55.510546+0000","last_unstale":"2024-04-16T05:34:21.505607+0000","last_undegraded":"2024-04-16T05:34:21.505607+0000","last_fullsized":"2024-04-16T05:34:21.505607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:27.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575904+0000","last_change":"2024-04-16T05:32:59.524257+0000","last_active":"2024-04-16T05:33:52.575904+0000","last_peered":"2024-04-16T05:33:52.575904+0000","last_clean":"2024-04-16T05:33:52.575904+0000","last_became_active":"2024-04-16T05:32:59.523768+0000","last_became_peered":"2024-04-16T05:32:59.523768+0000","last_unstale":"2024-04-16T05:33:52.575904+0000","last_undegraded":"2024-04-16T05:33:52.575904+0000","last_fullsized":"2024-04-16T05:33:52.575904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:38:27.594048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577241+0000","last_change":"2024-04-16T05:32:54.504430+0000","last_active":"2024-04-16T05:33:52.577241+0000","last_peered":"2024-04-16T05:33:52.577241+0000","last_clean":"2024-04-16T05:33:52.577241+0000","last_became_active":"2024-04-16T05:32:54.504054+0000","last_became_peered":"2024-04-16T05:32:54.504054+0000","last_unstale":"2024-04-16T05:33:52.577241+0000","last_undegraded":"2024-04-16T05:33:52.577241+0000","last_fullsized":"2024-04-16T05:33:52.577241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:53:24.875786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.503479+0000","last_change":"2024-04-16T05:32:55.508005+0000","last_active":"2024-04-16T05:34:21.503479+0000","last_peered":"2024-04-16T05:34:21.503479+0000","last_clean":"2024-04-16T05:34:21.503479+0000","last_became_active":"2024-04-16T05:32:55.507698+0000","last_became_peered":"2024-04-16T05:32:55.507698+0000","last_unstale":"2024-04-16T05:34:21.503479+0000","last_undegraded":"2024-04-16T05:34:21.503479+0000","last_fullsized":"2024-04-16T05:34:21.503479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:11:12.423294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575846+0000","last_change":"2024-04-16T05:33:00.419442+0000","last_active":"2024-04-16T05:33:52.575846+0000","last_peered":"2024-04-16T05:33:52.575846+0000","last_clean":"2024-04-16T05:33:52.575846+0000","last_became_active":"2024-04-16T05:33:00.418906+0000","last_became_peered":"2024-04-16T05:33:00.418906+0000","last_unstale":"2024-04-16T05:33:52.575846+0000","last_undegraded":"2024-04-16T05:33:52.575846+0000","last_fullsized":"2024-04-16T05:33:52.575846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:21:22.124879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.575661+0000","last_change":"2024-04-16T05:32:54.509090+0000","last_active":"2024-04-16T05:33:52.575661+0000","last_peered":"2024-04-16T05:33:52.575661+0000","last_clean":"2024-04-16T05:33:52.575661+0000","last_became_active":"2024-04-16T05:32:54.508700+0000","last_became_peered":"2024-04-16T05:32:54.508700+0000","last_unstale":"2024-04-16T05:33:52.575661+0000","last_undegraded":"2024-04-16T05:33:52.575661+0000","last_fullsized":"2024-04-16T05:33:52.575661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:53:34.344073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511501+0000","last_change":"2024-04-16T05:32:55.509658+0000","last_active":"2024-04-16T05:34:21.511501+0000","last_peered":"2024-04-16T05:34:21.511501+0000","last_clean":"2024-04-16T05:34:21.511501+0000","last_became_active":"2024-04-16T05:32:55.509258+0000","last_became_peered":"2024-04-16T05:32:55.509258+0000","last_unstale":"2024-04-16T05:34:21.511501+0000","last_undegraded":"2024-04-16T05:34:21.511501+0000","last_fullsized":"2024-04-16T05:34:21.511501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:17:03.490544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.578141+0000","last_change":"2024-04-16T05:32:59.519260+0000","last_active":"2024-04-16T05:33:52.578141+0000","last_peered":"2024-04-16T05:33:52.578141+0000","last_clean":"2024-04-16T05:33:52.578141+0000","last_became_active":"2024-04-16T05:32:59.518839+0000","last_became_peered":"2024-04-16T05:32:59.518839+0000","last_unstale":"2024-04-16T05:33:52.578141+0000","last_undegraded":"2024-04-16T05:33:52.578141+0000","last_fullsized":"2024-04-16T05:33:52.578141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:54:50.555214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:28.726058+0000","last_change":"2024-04-16T05:32:54.503226+0000","last_active":"2024-04-16T05:34:28.726058+0000","last_peered":"2024-04-16T05:34:28.726058+0000","last_clean":"2024-04-16T05:34:28.726058+0000","last_became_active":"2024-04-16T05:32:54.502687+0000","last_became_peered":"2024-04-16T05:32:54.502687+0000","last_unstale":"2024-04-16T05:34:28.726058+0000","last_undegraded":"2024-04-16T05:34:28.726058+0000","last_fullsized":"2024-04-16T05:34:28.726058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:34:24.228464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.510895+0000","last_change":"2024-04-16T05:32:55.515203+0000","last_active":"2024-04-16T05:34:21.510895+0000","last_peered":"2024-04-16T05:34:21.510895+0000","last_clean":"2024-04-16T05:34:21.510895+0000","last_became_active":"2024-04-16T05:32:55.514815+0000","last_became_peered":"2024-04-16T05:32:55.514815+0000","last_unstale":"2024-04-16T05:34:21.510895+0000","last_undegraded":"2024-04-16T05:34:21.510895+0000","last_fullsized":"2024-04-16T05:34:21.510895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:11:18.464532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.577618+0000","last_change":"2024-04-16T05:32:59.520442+0000","last_active":"2024-04-16T05:33:52.577618+0000","last_peered":"2024-04-16T05:33:52.577618+0000","last_clean":"2024-04-16T05:33:52.577618+0000","last_became_active":"2024-04-16T05:32:59.520071+0000","last_became_peered":"2024-04-16T05:32:59.520071+0000","last_unstale":"2024-04-16T05:33:52.577618+0000","last_undegraded":"2024-04-16T05:33:52.577618+0000","last_fullsized":"2024-04-16T05:33:52.577618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:19:16.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.677720+0000","last_change":"2024-04-16T05:32:54.510098+0000","last_active":"2024-04-16T05:33:52.677720+0000","last_peered":"2024-04-16T05:33:52.677720+0000","last_clean":"2024-04-16T05:33:52.677720+0000","last_became_active":"2024-04-16T05:32:54.509750+0000","last_became_peered":"2024-04-16T05:32:54.509750+0000","last_unstale":"2024-04-16T05:33:52.677720+0000","last_undegraded":"2024-04-16T05:33:52.677720+0000","last_fullsized":"2024-04-16T05:33:52.677720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:00:09.528958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501136+0000","last_change":"2024-04-16T05:33:52.678823+0000","last_active":"2024-04-16T05:34:21.501136+0000","last_peered":"2024-04-16T05:34:21.501136+0000","last_clean":"2024-04-16T05:34:21.501136+0000","last_became_active":"2024-04-16T05:33:52.678276+0000","last_became_peered":"2024-04-16T05:33:52.678276+0000","last_unstale":"2024-04-16T05:34:21.501136+0000","last_undegraded":"2024-04-16T05:34:21.501136+0000","last_fullsized":"2024-04-16T05:34:21.501136+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:58:33.186382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.676262+0000","last_change":"2024-04-16T05:32:59.521334+0000","last_active":"2024-04-16T05:33:52.676262+0000","last_peered":"2024-04-16T05:33:52.676262+0000","last_clean":"2024-04-16T05:33:52.676262+0000","last_became_active":"2024-04-16T05:32:59.520348+0000","last_became_peered":"2024-04-16T05:32:59.520348+0000","last_unstale":"2024-04-16T05:33:52.676262+0000","last_undegraded":"2024-04-16T05:33:52.676262+0000","last_fullsized":"2024-04-16T05:33:52.676262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:40.628552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:33:52.672455+0000","last_change":"2024-04-16T05:32:54.503062+0000","last_active":"2024-04-16T05:33:52.672455+0000","last_peered":"2024-04-16T05:33:52.672455+0000","last_clean":"2024-04-16T05:33:52.672455+0000","last_became_active":"2024-04-16T05:32:54.502762+0000","last_became_peered":"2024-04-16T05:32:54.502762+0000","last_unstale":"2024-04-16T05:33:52.672455+0000","last_undegraded":"2024-04-16T05:33:52.672455+0000","last_fullsized":"2024-04-16T05:33:52.672455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:43:57.911296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.502415+0000","last_change":"2024-04-16T05:32:55.510461+0000","last_active":"2024-04-16T05:34:21.502415+0000","last_peered":"2024-04-16T05:34:21.502415+0000","last_clean":"2024-04-16T05:34:21.502415+0000","last_became_active":"2024-04-16T05:32:55.510142+0000","last_became_peered":"2024-04-16T05:32:55.510142+0000","last_unstale":"2024-04-16T05:34:21.502415+0000","last_undegraded":"2024-04-16T05:34:21.502415+0000","last_fullsized":"2024-04-16T05:34:21.502415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:41.346078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.506149+0000","last_change":"2024-04-16T05:32:55.512600+0000","last_active":"2024-04-16T05:34:21.506149+0000","last_peered":"2024-04-16T05:34:21.506149+0000","last_clean":"2024-04-16T05:34:21.506149+0000","last_became_active":"2024-04-16T05:32:55.512265+0000","last_became_peered":"2024-04-16T05:32:55.512265+0000","last_unstale":"2024-04-16T05:34:21.506149+0000","last_undegraded":"2024-04-16T05:34:21.506149+0000","last_fullsized":"2024-04-16T05:34:21.506149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:04:46.112644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.504122+0000","last_change":"2024-04-16T05:32:55.514481+0000","last_active":"2024-04-16T05:34:21.504122+0000","last_peered":"2024-04-16T05:34:21.504122+0000","last_clean":"2024-04-16T05:34:21.504122+0000","last_became_active":"2024-04-16T05:32:55.512485+0000","last_became_peered":"2024-04-16T05:32:55.512485+0000","last_unstale":"2024-04-16T05:34:21.504122+0000","last_undegraded":"2024-04-16T05:34:21.504122+0000","last_fullsized":"2024-04-16T05:34:21.504122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:33:02.179774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.503157+0000","last_change":"2024-04-16T05:32:55.509849+0000","last_active":"2024-04-16T05:34:21.503157+0000","last_peered":"2024-04-16T05:34:21.503157+0000","last_clean":"2024-04-16T05:34:21.503157+0000","last_became_active":"2024-04-16T05:32:55.509338+0000","last_became_peered":"2024-04-16T05:32:55.509338+0000","last_unstale":"2024-04-16T05:34:21.503157+0000","last_undegraded":"2024-04-16T05:34:21.503157+0000","last_fullsized":"2024-04-16T05:34:21.503157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:32:38.900455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.502558+0000","last_change":"2024-04-16T05:32:55.518556+0000","last_active":"2024-04-16T05:34:21.502558+0000","last_peered":"2024-04-16T05:34:21.502558+0000","last_clean":"2024-04-16T05:34:21.502558+0000","last_became_active":"2024-04-16T05:32:55.518240+0000","last_became_peered":"2024-04-16T05:32:55.518240+0000","last_unstale":"2024-04-16T05:34:21.502558+0000","last_undegraded":"2024-04-16T05:34:21.502558+0000","last_fullsized":"2024-04-16T05:34:21.502558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:25:46.002516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.498586+0000","last_change":"2024-04-16T05:32:55.518106+0000","last_active":"2024-04-16T05:34:21.498586+0000","last_peered":"2024-04-16T05:34:21.498586+0000","last_clean":"2024-04-16T05:34:21.498586+0000","last_became_active":"2024-04-16T05:32:55.517403+0000","last_became_peered":"2024-04-16T05:32:55.517403+0000","last_unstale":"2024-04-16T05:34:21.498586+0000","last_undegraded":"2024-04-16T05:34:21.498586+0000","last_fullsized":"2024-04-16T05:34:21.498586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:36:34.731644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.501520+0000","last_change":"2024-04-16T05:33:52.581236+0000","last_active":"2024-04-16T05:34:21.501520+0000","last_peered":"2024-04-16T05:34:21.501520+0000","last_clean":"2024-04-16T05:34:21.501520+0000","last_became_active":"2024-04-16T05:33:52.580748+0000","last_became_peered":"2024-04-16T05:33:52.580748+0000","last_unstale":"2024-04-16T05:34:21.501520+0000","last_undegraded":"2024-04-16T05:34:21.501520+0000","last_fullsized":"2024-04-16T05:34:21.501520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:13:33.382776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511534+0000","last_change":"2024-04-16T05:32:55.507670+0000","last_active":"2024-04-16T05:34:21.511534+0000","last_peered":"2024-04-16T05:34:21.511534+0000","last_clean":"2024-04-16T05:34:21.511534+0000","last_became_active":"2024-04-16T05:32:55.506672+0000","last_became_peered":"2024-04-16T05:32:55.506672+0000","last_unstale":"2024-04-16T05:34:21.511534+0000","last_undegraded":"2024-04-16T05:34:21.511534+0000","last_fullsized":"2024-04-16T05:34:21.511534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:54.579672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.495721+0000","last_change":"2024-04-16T05:32:55.516987+0000","last_active":"2024-04-16T05:34:21.495721+0000","last_peered":"2024-04-16T05:34:21.495721+0000","last_clean":"2024-04-16T05:34:21.495721+0000","last_became_active":"2024-04-16T05:32:55.516654+0000","last_became_peered":"2024-04-16T05:32:55.516654+0000","last_unstale":"2024-04-16T05:34:21.495721+0000","last_undegraded":"2024-04-16T05:34:21.495721+0000","last_fullsized":"2024-04-16T05:34:21.495721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:32:42.957658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.508245+0000","last_change":"2024-04-16T05:32:55.506539+0000","last_active":"2024-04-16T05:34:21.508245+0000","last_peered":"2024-04-16T05:34:21.508245+0000","last_clean":"2024-04-16T05:34:21.508245+0000","last_became_active":"2024-04-16T05:32:55.503658+0000","last_became_peered":"2024-04-16T05:32:55.503658+0000","last_unstale":"2024-04-16T05:34:21.508245+0000","last_undegraded":"2024-04-16T05:34:21.508245+0000","last_fullsized":"2024-04-16T05:34:21.508245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:22.116213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.507351+0000","last_change":"2024-04-16T05:32:55.507363+0000","last_active":"2024-04-16T05:34:21.507351+0000","last_peered":"2024-04-16T05:34:21.507351+0000","last_clean":"2024-04-16T05:34:21.507351+0000","last_became_active":"2024-04-16T05:32:55.506923+0000","last_became_peered":"2024-04-16T05:32:55.506923+0000","last_unstale":"2024-04-16T05:34:21.507351+0000","last_undegraded":"2024-04-16T05:34:21.507351+0000","last_fullsized":"2024-04-16T05:34:21.507351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:19:34.356834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.511928+0000","last_change":"2024-04-16T05:32:55.508531+0000","last_active":"2024-04-16T05:34:21.511928+0000","last_peered":"2024-04-16T05:34:21.511928+0000","last_clean":"2024-04-16T05:34:21.511928+0000","last_became_active":"2024-04-16T05:32:55.508177+0000","last_became_peered":"2024-04-16T05:32:55.508177+0000","last_unstale":"2024-04-16T05:34:21.511928+0000","last_undegraded":"2024-04-16T05:34:21.511928+0000","last_fullsized":"2024-04-16T05:34:21.511928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:49:59.352572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.509060+0000","last_change":"2024-04-16T05:32:55.507793+0000","last_active":"2024-04-16T05:34:21.509060+0000","last_peered":"2024-04-16T05:34:21.509060+0000","last_clean":"2024-04-16T05:34:21.509060+0000","last_became_active":"2024-04-16T05:32:55.507443+0000","last_became_peered":"2024-04-16T05:32:55.507443+0000","last_unstale":"2024-04-16T05:34:21.509060+0000","last_undegraded":"2024-04-16T05:34:21.509060+0000","last_fullsized":"2024-04-16T05:34:21.509060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:35.982879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:21.499109+0000","last_change":"2024-04-16T05:32:55.517057+0000","last_active":"2024-04-16T05:34:21.499109+0000","last_peered":"2024-04-16T05:34:21.499109+0000","last_clean":"2024-04-16T05:34:21.499109+0000","last_became_active":"2024-04-16T05:32:55.516696+0000","last_became_peered":"2024-04-16T05:32:55.516696+0000","last_unstale":"2024-04-16T05:34:21.499109+0000","last_undegraded":"2024-04-16T05:34:21.499109+0000","last_fullsized":"2024-04-16T05:34:21.499109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:43:10.391125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280341,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69457,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":761,"num_read_kb":457,"num_write":253,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":208371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":511388,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":1159643,"data_compressed":59230,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":514882,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":511164,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1108334,"data_compressed":59230,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":506290,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":568759,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":508278,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":570043,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":505998,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":510896,"data_compressed":23572,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":1103384,"data_compressed":59230,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34017,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:34:34.466 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-16T05:34:34.466 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-16T05:34:34.466 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-16T05:34:34.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph health --format=json 2024-04-16T05:34:34.711 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:34.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:34 smithi086 ceph-mon[32028]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-16T05:34:34.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:34 smithi194 ceph-mon[35865]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-16T05:34:34.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:34 smithi196 ceph-mon[36373]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-16T05:34:35.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:35 smithi086 ceph-mon[32028]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:35.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:35 smithi194 ceph-mon[35865]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:35.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:35 smithi196 ceph-mon[36373]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.513+0000 7fd978935700 1 -- 172.21.15.86:0/3194863038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9741079d0 msgr2=0x7fd974107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.513+0000 7fd978935700 1 --2- 172.21.15.86:0/3194863038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9741079d0 0x7fd974107db0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd968009990 tx=0x7fd968009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.513+0000 7fd978935700 1 -- 172.21.15.86:0/3194863038 shutdown_connections 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.513+0000 7fd978935700 1 --2- 172.21.15.86:0/3194863038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9741023e0 0x7fd97410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.513+0000 7fd978935700 1 --2- 172.21.15.86:0/3194863038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9741079d0 0x7fd974107db0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.513+0000 7fd978935700 1 --2- 172.21.15.86:0/3194863038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd974101a20 0x7fd974101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.513+0000 7fd978935700 1 -- 172.21.15.86:0/3194863038 >> 172.21.15.86:0/3194863038 conn(0x7fd9740789a0 msgr2=0x7fd974078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.513+0000 7fd978935700 1 -- 172.21.15.86:0/3194863038 shutdown_connections 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.513+0000 7fd978935700 1 -- 172.21.15.86:0/3194863038 wait complete. 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.514+0000 7fd978935700 1 Processor -- start 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.514+0000 7fd978935700 1 -- start start 2024-04-16T05:34:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.514+0000 7fd978935700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd974101a20 0x7fd9741992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.514+0000 7fd978935700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9741023e0 0x7fd974199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.514+0000 7fd978935700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9741079d0 0x7fd97419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.514+0000 7fd978935700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd974111d60 con 0x7fd974101a20 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.514+0000 7fd978935700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd974111be0 con 0x7fd9741079d0 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.514+0000 7fd978935700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd974111ee0 con 0x7fd9741023e0 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd97259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd974101a20 0x7fd9741992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd97259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd974101a20 0x7fd9741992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38612/0 (socket says 172.21.15.86:38612) 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd97259c700 1 -- 172.21.15.86:0/4048724243 learned_addr learned my addr 172.21.15.86:0/4048724243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd972d9d700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9741079d0 0x7fd97419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd971d9b700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9741023e0 0x7fd974199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd97259c700 1 -- 172.21.15.86:0/4048724243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9741023e0 msgr2=0x7fd974199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd97259c700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9741023e0 0x7fd974199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd97259c700 1 -- 172.21.15.86:0/4048724243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9741079d0 msgr2=0x7fd97419db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd97259c700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9741079d0 0x7fd97419db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd97259c700 1 -- 172.21.15.86:0/4048724243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97419e2d0 con 0x7fd974101a20 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd972d9d700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9741079d0 0x7fd97419db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd97259c700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd974101a20 0x7fd9741992c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd968000c00 tx=0x7fd9680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd9637fe700 1 -- 172.21.15.86:0/4048724243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968027070 con 0x7fd974101a20 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd9637fe700 1 -- 172.21.15.86:0/4048724243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd968004050 con 0x7fd974101a20 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97419e560 con 0x7fd974101a20 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd9637fe700 1 -- 172.21.15.86:0/4048724243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96800f460 con 0x7fd974101a20 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.515+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9741a5e40 con 0x7fd974101a20 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.516+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9740697e0 con 0x7fd974101a20 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.517+0000 7fd9637fe700 1 -- 172.21.15.86:0/4048724243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd96802a070 con 0x7fd974101a20 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.518+0000 7fd9637fe700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd95c0a3cc0 0x7fd95c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.518+0000 7fd971d9b700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd95c0a3cc0 0x7fd95c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.518+0000 7fd9637fe700 1 -- 172.21.15.86:0/4048724243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd9680d1ab0 con 0x7fd974101a20 2024-04-16T05:34:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.518+0000 7fd971d9b700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd95c0a3cc0 0x7fd95c0a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd964005950 tx=0x7fd9640058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.522+0000 7fd9637fe700 1 -- 172.21.15.86:0/4048724243 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd968031070 con 0x7fd974101a20 2024-04-16T05:34:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.768+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd97402dfe0 con 0x7fd974101a20 2024-04-16T05:34:36.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:36 smithi086 ceph-mon[32028]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-16T05:34:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.768+0000 7fd9637fe700 1 -- 172.21.15.86:0/4048724243 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd968013070 con 0x7fd974101a20 2024-04-16T05:34:36.769 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:34:36.769 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd95c0a3cc0 msgr2=0x7fd95c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd95c0a3cc0 0x7fd95c0a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd964005950 tx=0x7fd9640058e0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd974101a20 msgr2=0x7fd9741992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd974101a20 0x7fd9741992c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd968000c00 tx=0x7fd9680047b0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 shutdown_connections 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9741079d0 0x7fd97419db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd974101a20 0x7fd9741992c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd95c0a3cc0 0x7fd95c0a6180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 --2- 172.21.15.86:0/4048724243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9741023e0 0x7fd974199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.770+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 >> 172.21.15.86:0/4048724243 conn(0x7fd9740789a0 msgr2=0x7fd97406b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.771+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 shutdown_connections 2024-04-16T05:34:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:36.771+0000 7fd978935700 1 -- 172.21.15.86:0/4048724243 wait complete. 2024-04-16T05:34:36.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:36 smithi194 ceph-mon[35865]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-16T05:34:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:36 smithi196 ceph-mon[36373]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-16T05:34:37.384 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-16T05:34:37.385 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:37.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.493+0000 7fd983cc5640 1 -- 172.21.15.86:0/932347561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103c20 msgr2=0x7fd97c10a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.493+0000 7fd983cc5640 1 --2- 172.21.15.86:0/932347561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103c20 0x7fd97c10a4b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd974009ca0 tx=0x7fd97402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.493+0000 7fd983cc5640 1 -- 172.21.15.86:0/932347561 shutdown_connections 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.493+0000 7fd983cc5640 1 --2- 172.21.15.86:0/932347561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103c20 0x7fd97c10a4b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.493+0000 7fd983cc5640 1 --2- 172.21.15.86:0/932347561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd97c103260 0x7fd97c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.493+0000 7fd983cc5640 1 --2- 172.21.15.86:0/932347561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd97c102070 0x7fd97c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.493+0000 7fd983cc5640 1 -- 172.21.15.86:0/932347561 >> 172.21.15.86:0/932347561 conn(0x7fd97c0fd5e0 msgr2=0x7fd97c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.493+0000 7fd983cc5640 1 -- 172.21.15.86:0/932347561 shutdown_connections 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.493+0000 7fd983cc5640 1 -- 172.21.15.86:0/932347561 wait complete. 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd983cc5640 1 Processor -- start 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd983cc5640 1 -- start start 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd983cc5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd97c102070 0x7fd97c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd983cc5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103260 0x7fd97c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd983cc5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd97c103c20 0x7fd97c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd983cc5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c10dd00 con 0x7fd97c103260 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd983cc5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c10db80 con 0x7fd97c102070 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd983cc5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c10de80 con 0x7fd97c103c20 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd981260640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103260 0x7fd97c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd981260640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103260 0x7fd97c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38632/0 (socket says 172.21.15.86:38632) 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd981260640 1 -- 172.21.15.86:0/1441187399 learned_addr learned my addr 172.21.15.86:0/1441187399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd981a61640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd97c102070 0x7fd97c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd982262640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd97c103c20 0x7fd97c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd981260640 1 -- 172.21.15.86:0/1441187399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd97c103c20 msgr2=0x7fd97c19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd981260640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd97c103c20 0x7fd97c19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd981260640 1 -- 172.21.15.86:0/1441187399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd97c102070 msgr2=0x7fd97c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd981260640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd97c102070 0x7fd97c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.494+0000 7fd981260640 1 -- 172.21.15.86:0/1441187399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c19ff10 con 0x7fd97c103260 2024-04-16T05:34:37.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.495+0000 7fd981260640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103260 0x7fd97c198830 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd97800d950 tx=0x7fd97800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:37.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.495+0000 7fd972ffd640 1 -- 172.21.15.86:0/1441187399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978017070 con 0x7fd97c103260 2024-04-16T05:34:37.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.495+0000 7fd972ffd640 1 -- 172.21.15.86:0/1441187399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd978009c30 con 0x7fd97c103260 2024-04-16T05:34:37.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.495+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c1a0200 con 0x7fd97c103260 2024-04-16T05:34:37.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.495+0000 7fd972ffd640 1 -- 172.21.15.86:0/1441187399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97800f6b0 con 0x7fd97c103260 2024-04-16T05:34:37.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.495+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c1a0750 con 0x7fd97c103260 2024-04-16T05:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.496+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c109390 con 0x7fd97c103260 2024-04-16T05:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.497+0000 7fd972ffd640 1 -- 172.21.15.86:0/1441187399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd9780109d0 con 0x7fd97c103260 2024-04-16T05:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.497+0000 7fd972ffd640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd9680a3bb0 0x7fd9680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.498+0000 7fd981a61640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd9680a3bb0 0x7fd9680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.498+0000 7fd972ffd640 1 -- 172.21.15.86:0/1441187399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd9780d0320 con 0x7fd97c103260 2024-04-16T05:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.498+0000 7fd981a61640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd9680a3bb0 0x7fd9680a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd96c005fd0 tx=0x7fd96c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:37.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.501+0000 7fd972ffd640 1 -- 172.21.15.86:0/1441187399 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd978009de0 con 0x7fd97c103260 2024-04-16T05:34:37.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.684+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd97c02d890 con 0x7fd97c103260 2024-04-16T05:34:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.685+0000 7fd972ffd640 1 -- 172.21.15.86:0/1441187399 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24550 (secure 0 0 0) 0x7fd97809ca40 con 0x7fd97c103260 2024-04-16T05:34:37.684 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:34:37.685 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:52.567005+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:37.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.687+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd9680a3bb0 msgr2=0x7fd9680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.687+0000 7fd983cc5640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd9680a3bb0 0x7fd9680a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd96c005fd0 tx=0x7fd96c00b390 comp rx=0 tx=0).stop 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.687+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103260 msgr2=0x7fd97c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.687+0000 7fd983cc5640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103260 0x7fd97c198830 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd97800d950 tx=0x7fd97800de10 comp rx=0 tx=0).stop 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.688+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 shutdown_connections 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.688+0000 7fd983cc5640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd97c103c20 0x7fd97c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.688+0000 7fd983cc5640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c103260 0x7fd97c198830 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.688+0000 7fd983cc5640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd9680a3bb0 0x7fd9680a6070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.688+0000 7fd983cc5640 1 --2- 172.21.15.86:0/1441187399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd97c102070 0x7fd97c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.688+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 >> 172.21.15.86:0/1441187399 conn(0x7fd97c0fd5e0 msgr2=0x7fd97c106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.688+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 shutdown_connections 2024-04-16T05:34:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.688+0000 7fd983cc5640 1 -- 172.21.15.86:0/1441187399 wait complete. 2024-04-16T05:34:37.697 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:37 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4048724243' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T05:34:37.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:34:37.699 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:34:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.808+0000 7fd19ccc1640 1 -- 172.21.15.86:0/142829715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198103640 msgr2=0x7fd198103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.808+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/142829715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198103640 0x7fd198103a60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd188009900 tx=0x7fd18802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.808+0000 7fd19ccc1640 1 -- 172.21.15.86:0/142829715 shutdown_connections 2024-04-16T05:34:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.808+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/142829715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd198100a70 0x7fd19810a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.808+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/142829715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1981000b0 0x7fd198100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.808+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/142829715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198103640 0x7fd198103a60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.808+0000 7fd19ccc1640 1 -- 172.21.15.86:0/142829715 >> 172.21.15.86:0/142829715 conn(0x7fd1980fbc30 msgr2=0x7fd1980fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.808+0000 7fd19ccc1640 1 -- 172.21.15.86:0/142829715 shutdown_connections 2024-04-16T05:34:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.808+0000 7fd19ccc1640 1 -- 172.21.15.86:0/142829715 wait complete. 2024-04-16T05:34:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19ccc1640 1 Processor -- start 2024-04-16T05:34:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19ccc1640 1 -- start start 2024-04-16T05:34:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19ccc1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1981000b0 0x7fd198198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19ccc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198100a70 0x7fd198198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19ccc1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd198103640 0x7fd19819f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19ccc1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd19810dd60 con 0x7fd198100a70 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19ccc1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd19810dbe0 con 0x7fd1981000b0 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19ccc1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd19810dee0 con 0x7fd198103640 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198100a70 0x7fd198198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198100a70 0x7fd198198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38654/0 (socket says 172.21.15.86:38654) 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd18ffff640 1 -- 172.21.15.86:0/666555585 learned_addr learned my addr 172.21.15.86:0/666555585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd196d9d640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd198103640 0x7fd19819f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19659c640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1981000b0 0x7fd198198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd18ffff640 1 -- 172.21.15.86:0/666555585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd198103640 msgr2=0x7fd19819f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd18ffff640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd198103640 0x7fd19819f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd18ffff640 1 -- 172.21.15.86:0/666555585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1981000b0 msgr2=0x7fd198198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd18ffff640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1981000b0 0x7fd198198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd18ffff640 1 -- 172.21.15.86:0/666555585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981a0040 con 0x7fd198100a70 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.809+0000 7fd19659c640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1981000b0 0x7fd198198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.810+0000 7fd18ffff640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198100a70 0x7fd198198960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd18000b4a0 tx=0x7fd18000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.810+0000 7fd18f7fe640 1 -- 172.21.15.86:0/666555585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180014070 con 0x7fd198100a70 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.810+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1981a0330 con 0x7fd198100a70 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.810+0000 7fd18f7fe640 1 -- 172.21.15.86:0/666555585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd180010af0 con 0x7fd198100a70 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.810+0000 7fd18f7fe640 1 -- 172.21.15.86:0/666555585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180017d10 con 0x7fd198100a70 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.810+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981a08d0 con 0x7fd198100a70 2024-04-16T05:34:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.811+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1981095d0 con 0x7fd198100a70 2024-04-16T05:34:37.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.812+0000 7fd18f7fe640 1 -- 172.21.15.86:0/666555585 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd180017070 con 0x7fd198100a70 2024-04-16T05:34:37.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.812+0000 7fd18f7fe640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1840a3bb0 0x7fd1840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:37.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.812+0000 7fd19659c640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1840a3bb0 0x7fd1840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:37.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.813+0000 7fd18f7fe640 1 -- 172.21.15.86:0/666555585 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd1800cfb90 con 0x7fd198100a70 2024-04-16T05:34:37.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.813+0000 7fd19659c640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1840a3bb0 0x7fd1840a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd188000c00 tx=0x7fd18802cf90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:37.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.817+0000 7fd18f7fe640 1 -- 172.21.15.86:0/666555585 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd18009c210 con 0x7fd198100a70 2024-04-16T05:34:37.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:37 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4048724243' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T05:34:37.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:37 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4048724243' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T05:34:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.994+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd198062c00 con 0x7fd198100a70 2024-04-16T05:34:37.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.994+0000 7fd18f7fe640 1 -- 172.21.15.86:0/666555585 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fd18009bb10 con 0x7fd198100a70 2024-04-16T05:34:37.993 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:34:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1840a3bb0 msgr2=0x7fd1840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1840a3bb0 0x7fd1840a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd188000c00 tx=0x7fd18802cf90 comp rx=0 tx=0).stop 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198100a70 msgr2=0x7fd198198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198100a70 0x7fd198198960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd18000b4a0 tx=0x7fd18000b960 comp rx=0 tx=0).stop 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 shutdown_connections 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd198103640 0x7fd19819f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd198100a70 0x7fd198198960 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd1840a3bb0 0x7fd1840a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 --2- 172.21.15.86:0/666555585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1981000b0 0x7fd198198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.996+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 >> 172.21.15.86:0/666555585 conn(0x7fd1980fbc30 msgr2=0x7fd1981067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.997+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 shutdown_connections 2024-04-16T05:34:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:37.997+0000 7fd19ccc1640 1 -- 172.21.15.86:0/666555585 wait complete. 2024-04-16T05:34:38.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:34:38.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 -- 172.21.15.86:0/3676151356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103240 msgr2=0x7fe1381036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 --2- 172.21.15.86:0/3676151356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103240 0x7fe1381036c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009900 tx=0x7fe12c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:38.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 -- 172.21.15.86:0/3676151356 shutdown_connections 2024-04-16T05:34:38.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 --2- 172.21.15.86:0/3676151356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe138103c00 0x7fe13810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 --2- 172.21.15.86:0/3676151356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103240 0x7fe1381036c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 --2- 172.21.15.86:0/3676151356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe138102050 0x7fe138102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 -- 172.21.15.86:0/3676151356 >> 172.21.15.86:0/3676151356 conn(0x7fe1380fd5e0 msgr2=0x7fe1380ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:38.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 -- 172.21.15.86:0/3676151356 shutdown_connections 2024-04-16T05:34:38.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 -- 172.21.15.86:0/3676151356 wait complete. 2024-04-16T05:34:38.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.116+0000 7fe13e759640 1 Processor -- start 2024-04-16T05:34:38.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13e759640 1 -- start start 2024-04-16T05:34:38.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13e759640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe138102050 0x7fe138198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13e759640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe138103240 0x7fe1381988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13e759640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103c00 0x7fe13819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13e759640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe13810dce0 con 0x7fe138103c00 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13e759640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe13810db60 con 0x7fe138103240 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13e759640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe13810de60 con 0x7fe138102050 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13ccf6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103c00 0x7fe13819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13ccf6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103c00 0x7fe13819f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38676/0 (socket says 172.21.15.86:38676) 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13ccf6640 1 -- 172.21.15.86:0/3223811676 learned_addr learned my addr 172.21.15.86:0/3223811676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe1377fe640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe138103240 0x7fe1381988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe137fff640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe138102050 0x7fe138198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13ccf6640 1 -- 172.21.15.86:0/3223811676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe138102050 msgr2=0x7fe138198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13ccf6640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe138102050 0x7fe138198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13ccf6640 1 -- 172.21.15.86:0/3223811676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe138103240 msgr2=0x7fe1381988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13ccf6640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe138103240 0x7fe1381988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13ccf6640 1 -- 172.21.15.86:0/3223811676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13819ff80 con 0x7fe138103c00 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe1377fe640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe138103240 0x7fe1381988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.117+0000 7fe13ccf6640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103c00 0x7fe13819f910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe13000eaa0 tx=0x7fe13000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.118+0000 7fe1357fa640 1 -- 172.21.15.86:0/3223811676 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13000cdb0 con 0x7fe138103c00 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.118+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1381a0270 con 0x7fe138103c00 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.118+0000 7fe1357fa640 1 -- 172.21.15.86:0/3223811676 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe130005020 con 0x7fe138103c00 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.118+0000 7fe1357fa640 1 -- 172.21.15.86:0/3223811676 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1300116b0 con 0x7fe138103c00 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.118+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1381a0740 con 0x7fe138103c00 2024-04-16T05:34:38.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.119+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1381093f0 con 0x7fe138103c00 2024-04-16T05:34:38.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.120+0000 7fe1357fa640 1 -- 172.21.15.86:0/3223811676 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe130005560 con 0x7fe138103c00 2024-04-16T05:34:38.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.120+0000 7fe1357fa640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe1200a3bb0 0x7fe1200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.121+0000 7fe137fff640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe1200a3bb0 0x7fe1200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.121+0000 7fe1357fa640 1 -- 172.21.15.86:0/3223811676 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe1300d0e00 con 0x7fe138103c00 2024-04-16T05:34:38.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.121+0000 7fe137fff640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe1200a3bb0 0x7fe1200a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe128005fd0 tx=0x7fe128005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:38.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.124+0000 7fe1357fa640 1 -- 172.21.15.86:0/3223811676 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe130016030 con 0x7fe138103c00 2024-04-16T05:34:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.301+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe13802d620 con 0x7fe138103c00 2024-04-16T05:34:38.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.301+0000 7fe1357fa640 1 -- 172.21.15.86:0/3223811676 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fe13009d480 con 0x7fe138103c00 2024-04-16T05:34:38.300 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:34:38.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe1200a3bb0 msgr2=0x7fe1200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe1200a3bb0 0x7fe1200a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe128005fd0 tx=0x7fe128005850 comp rx=0 tx=0).stop 2024-04-16T05:34:38.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103c00 msgr2=0x7fe13819f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103c00 0x7fe13819f910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe13000eaa0 tx=0x7fe13000ef60 comp rx=0 tx=0).stop 2024-04-16T05:34:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 shutdown_connections 2024-04-16T05:34:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138103c00 0x7fe13819f910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe1200a3bb0 0x7fe1200a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe138103240 0x7fe1381988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 --2- 172.21.15.86:0/3223811676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe138102050 0x7fe138198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.303+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 >> 172.21.15.86:0/3223811676 conn(0x7fe1380fd5e0 msgr2=0x7fe1381065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.304+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 shutdown_connections 2024-04-16T05:34:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.304+0000 7fe13e759640 1 -- 172.21.15.86:0/3223811676 wait complete. 2024-04-16T05:34:38.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:34:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2324392686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 msgr2=0x7f4ad010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2324392686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 0x7f4ad010a5d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009ca0 tx=0x7f4ac802c140 comp rx=0 tx=0).stop 2024-04-16T05:34:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2324392686 shutdown_connections 2024-04-16T05:34:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2324392686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 0x7f4ad010a5d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2324392686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad0103380 0x7f4ad0103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2324392686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad0100310 0x7f4ad0100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2324392686 >> 172.21.15.86:0/2324392686 conn(0x7f4ad00fbc40 msgr2=0x7f4ad00fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2324392686 shutdown_connections 2024-04-16T05:34:38.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2324392686 wait complete. 2024-04-16T05:34:38.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 Processor -- start 2024-04-16T05:34:38.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.422+0000 7f4ad5ce4640 1 -- start start 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4ad5ce4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad0100310 0x7f4ad01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4ad5ce4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad0103380 0x7f4ad01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4ad5ce4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 0x7f4ad019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4ad5ce4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010dde0 con 0x7f4ad0103d40 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4ad5ce4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010dc60 con 0x7f4ad0103380 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4ad5ce4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010df60 con 0x7f4ad0100310 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 0x7f4ad019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 0x7f4ad019fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38692/0 (socket says 172.21.15.86:38692) 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acffff640 1 -- 172.21.15.86:0/2841386184 learned_addr learned my addr 172.21.15.86:0/2841386184 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4ac6ffd640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad0103380 0x7f4ad01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acffff640 1 -- 172.21.15.86:0/2841386184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad0100310 msgr2=0x7f4ad01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acf7fe640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad0100310 0x7f4ad01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acffff640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad0100310 0x7f4ad01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acffff640 1 -- 172.21.15.86:0/2841386184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad0103380 msgr2=0x7f4ad01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acffff640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad0103380 0x7f4ad01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acffff640 1 -- 172.21.15.86:0/2841386184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad01a00d0 con 0x7f4ad0103d40 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acf7fe640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad0100310 0x7f4ad01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acffff640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 0x7f4ad019fa60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8005cc0 tx=0x7f4ac8035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acd7fa640 1 -- 172.21.15.86:0/2841386184 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8039040 con 0x7f4ad0103d40 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.423+0000 7f4acd7fa640 1 -- 172.21.15.86:0/2841386184 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac8044070 con 0x7f4ad0103d40 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.424+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad01a0360 con 0x7f4ad0103d40 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.424+0000 7f4acd7fa640 1 -- 172.21.15.86:0/2841386184 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8004600 con 0x7f4ad0103d40 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.424+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad01a0920 con 0x7f4ad0103d40 2024-04-16T05:34:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.425+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad0109580 con 0x7f4ad0103d40 2024-04-16T05:34:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.426+0000 7f4acd7fa640 1 -- 172.21.15.86:0/2841386184 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ac8003970 con 0x7f4ad0103d40 2024-04-16T05:34:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.427+0000 7f4acd7fa640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4abc0a3c30 0x7f4abc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.427+0000 7f4acf7fe640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4abc0a3c30 0x7f4abc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.427+0000 7f4acd7fa640 1 -- 172.21.15.86:0/2841386184 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4ac80f30d0 con 0x7f4ad0103d40 2024-04-16T05:34:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.427+0000 7f4acf7fe640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4abc0a3c30 0x7f4abc0a60f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006fd0 tx=0x7f4ac0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:38.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.431+0000 7f4acd7fa640 1 -- 172.21.15.86:0/2841386184 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ac80bf750 con 0x7f4ad0103d40 2024-04-16T05:34:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.612+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ad0062c00 con 0x7f4ad0103d40 2024-04-16T05:34:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.612+0000 7f4acd7fa640 1 -- 172.21.15.86:0/2841386184 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f4ac80098b0 con 0x7f4ad0103d40 2024-04-16T05:34:38.611 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:34:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4abc0a3c30 msgr2=0x7f4abc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4abc0a3c30 0x7f4abc0a60f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006fd0 tx=0x7f4ac0008040 comp rx=0 tx=0).stop 2024-04-16T05:34:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 msgr2=0x7f4ad019fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 0x7f4ad019fa60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8005cc0 tx=0x7f4ac8035d90 comp rx=0 tx=0).stop 2024-04-16T05:34:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 shutdown_connections 2024-04-16T05:34:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0103d40 0x7f4ad019fa60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4abc0a3c30 0x7f4abc0a60f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad0103380 0x7f4ad01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 --2- 172.21.15.86:0/2841386184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad0100310 0x7f4ad01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 >> 172.21.15.86:0/2841386184 conn(0x7f4ad00fbc40 msgr2=0x7f4ad0106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 shutdown_connections 2024-04-16T05:34:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.614+0000 7f4ad5ce4640 1 -- 172.21.15.86:0/2841386184 wait complete. 2024-04-16T05:34:38.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 -- 172.21.15.86:0/3853401952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 msgr2=0x7fc8b410a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 --2- 172.21.15.86:0/3853401952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 0x7fc8b410a1d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009ca0 tx=0x7fc8ac02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 -- 172.21.15.86:0/3853401952 shutdown_connections 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 --2- 172.21.15.86:0/3853401952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 0x7fc8b410a1d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 --2- 172.21.15.86:0/3853401952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8b41054b0 0x7fc8b41078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 --2- 172.21.15.86:0/3853401952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b4102b80 0x7fc8b4104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 -- 172.21.15.86:0/3853401952 >> 172.21.15.86:0/3853401952 conn(0x7fc8b40fc4d0 msgr2=0x7fc8b40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 -- 172.21.15.86:0/3853401952 shutdown_connections 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 -- 172.21.15.86:0/3853401952 wait complete. 2024-04-16T05:34:38.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.733+0000 7fc8b9174640 1 Processor -- start 2024-04-16T05:34:38.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b9174640 1 -- start start 2024-04-16T05:34:38.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b9174640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8b4102b80 0x7fc8b41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b9174640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b41054b0 0x7fc8b4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b9174640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 0x7fc8b419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b9174640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b410dd10 con 0x7fc8b4107de0 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b9174640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b410db90 con 0x7fc8b4102b80 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b9174640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b410de90 con 0x7fc8b41054b0 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 0x7fc8b419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 0x7fc8b419f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38718/0 (socket says 172.21.15.86:38718) 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b359e640 1 -- 172.21.15.86:0/330390639 learned_addr learned my addr 172.21.15.86:0/330390639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b2d9d640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8b4102b80 0x7fc8b41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8aa59c640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b41054b0 0x7fc8b4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b359e640 1 -- 172.21.15.86:0/330390639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b41054b0 msgr2=0x7fc8b4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b359e640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b41054b0 0x7fc8b4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b359e640 1 -- 172.21.15.86:0/330390639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8b4102b80 msgr2=0x7fc8b41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b359e640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8b4102b80 0x7fc8b41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b359e640 1 -- 172.21.15.86:0/330390639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b419ff60 con 0x7fc8b4107de0 2024-04-16T05:34:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.734+0000 7fc8b359e640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 0x7fc8b419f8f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac02c700 tx=0x7fc8ac032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.735+0000 7fc8b0d99640 1 -- 172.21.15.86:0/330390639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac039040 con 0x7fc8b4107de0 2024-04-16T05:34:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.735+0000 7fc8b0d99640 1 -- 172.21.15.86:0/330390639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ac044070 con 0x7fc8b4107de0 2024-04-16T05:34:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.735+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b41a01f0 con 0x7fc8b4107de0 2024-04-16T05:34:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.735+0000 7fc8b0d99640 1 -- 172.21.15.86:0/330390639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0042a0 con 0x7fc8b4107de0 2024-04-16T05:34:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.735+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b41a06e0 con 0x7fc8b4107de0 2024-04-16T05:34:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.736+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b40fdfc0 con 0x7fc8b4107de0 2024-04-16T05:34:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.737+0000 7fc8b0d99640 1 -- 172.21.15.86:0/330390639 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc8ac003970 con 0x7fc8b4107de0 2024-04-16T05:34:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.738+0000 7fc8b0d99640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc8a00a3bb0 0x7fc8a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.738+0000 7fc8b2d9d640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc8a00a3bb0 0x7fc8a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.738+0000 7fc8b0d99640 1 -- 172.21.15.86:0/330390639 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc8ac0f32c0 con 0x7fc8b4107de0 2024-04-16T05:34:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.738+0000 7fc8b2d9d640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc8a00a3bb0 0x7fc8a00a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40058c0 tx=0x7fc8a400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:38.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.742+0000 7fc8b0d99640 1 -- 172.21.15.86:0/330390639 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8ac0bf9d0 con 0x7fc8b4107de0 2024-04-16T05:34:38.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:38 smithi194 ceph-mon[35865]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-16T05:34:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:38 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1441187399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:38 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/666555585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:38 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3223811676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:38.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.917+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc8b402d620 con 0x7fc8b4107de0 2024-04-16T05:34:38.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.918+0000 7fc8b0d99640 1 -- 172.21.15.86:0/330390639 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fc8ac0bf2d0 con 0x7fc8b4107de0 2024-04-16T05:34:38.916 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:34:38.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc8a00a3bb0 msgr2=0x7fc8a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc8a00a3bb0 0x7fc8a00a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40058c0 tx=0x7fc8a400a2d0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 msgr2=0x7fc8b419f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:38.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 0x7fc8b419f8f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac02c700 tx=0x7fc8ac032f20 comp rx=0 tx=0).stop 2024-04-16T05:34:38.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 shutdown_connections 2024-04-16T05:34:38.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4107de0 0x7fc8b419f8f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b41054b0 0x7fc8b4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fc8a00a3bb0 0x7fc8a00a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 --2- 172.21.15.86:0/330390639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8b4102b80 0x7fc8b41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 >> 172.21.15.86:0/330390639 conn(0x7fc8b40fc4d0 msgr2=0x7fc8b41034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 shutdown_connections 2024-04-16T05:34:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:38.920+0000 7fc8b9174640 1 -- 172.21.15.86:0/330390639 wait complete. 2024-04-16T05:34:38.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:38 smithi086 ceph-mon[32028]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-16T05:34:38.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:38 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1441187399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:38.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:38 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/666555585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:38.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:38 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3223811676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:38.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:34:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:38 smithi196 ceph-mon[36373]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-16T05:34:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:38 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1441187399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:38 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/666555585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:38 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3223811676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:39.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 -- 172.21.15.86:0/2497575970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac102050 msgr2=0x7f71ac102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 --2- 172.21.15.86:0/2497575970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac102050 0x7f71ac102470 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009900 tx=0x7f71a002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:39.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 -- 172.21.15.86:0/2497575970 shutdown_connections 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 --2- 172.21.15.86:0/2497575970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ac103c00 0x7f71ac10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 --2- 172.21.15.86:0/2497575970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac103240 0x7f71ac1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 --2- 172.21.15.86:0/2497575970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac102050 0x7f71ac102470 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 -- 172.21.15.86:0/2497575970 >> 172.21.15.86:0/2497575970 conn(0x7f71ac0fd5e0 msgr2=0x7f71ac0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 -- 172.21.15.86:0/2497575970 shutdown_connections 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 -- 172.21.15.86:0/2497575970 wait complete. 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 Processor -- start 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 -- start start 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ac102050 0x7f71ac1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac103240 0x7f71ac198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac103c00 0x7f71ac19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10dce0 con 0x7f71ac103c00 2024-04-16T05:34:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10db60 con 0x7f71ac103240 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.036+0000 7f71b4693640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10de60 con 0x7f71ac102050 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b2c30640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac103c00 0x7f71ac19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b2c30640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac103c00 0x7f71ac19f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38734/0 (socket says 172.21.15.86:38734) 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b2c30640 1 -- 172.21.15.86:0/2162971308 learned_addr learned my addr 172.21.15.86:0/2162971308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b242f640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ac102050 0x7f71ac1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b1c2e640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac103240 0x7f71ac198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b2c30640 1 -- 172.21.15.86:0/2162971308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ac102050 msgr2=0x7f71ac1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b2c30640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ac102050 0x7f71ac1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b2c30640 1 -- 172.21.15.86:0/2162971308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac103240 msgr2=0x7f71ac198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b2c30640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac103240 0x7f71ac198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b2c30640 1 -- 172.21.15.86:0/2162971308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac19fef0 con 0x7f71ac103c00 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b1c2e640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac103240 0x7f71ac198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:39.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b2c30640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac103c00 0x7f71ac19f880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f71a400eaa0 tx=0x7f71a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f719f7fe640 1 -- 172.21.15.86:0/2162971308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400cdb0 con 0x7f71ac103c00 2024-04-16T05:34:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f719f7fe640 1 -- 172.21.15.86:0/2162971308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4005020 con 0x7f71ac103c00 2024-04-16T05:34:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac1a01e0 con 0x7f71ac103c00 2024-04-16T05:34:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f719f7fe640 1 -- 172.21.15.86:0/2162971308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a40116b0 con 0x7f71ac103c00 2024-04-16T05:34:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.037+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac1a0730 con 0x7f71ac103c00 2024-04-16T05:34:39.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.038+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ac109310 con 0x7f71ac103c00 2024-04-16T05:34:39.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.039+0000 7f719f7fe640 1 -- 172.21.15.86:0/2162971308 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f71a4005560 con 0x7f71ac103c00 2024-04-16T05:34:39.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.040+0000 7f719f7fe640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f71980a3bb0 0x7f71980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.040+0000 7f71b242f640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f71980a3bb0 0x7f71980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.040+0000 7f719f7fe640 1 -- 172.21.15.86:0/2162971308 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f71a40d09b0 con 0x7f71ac103c00 2024-04-16T05:34:39.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.040+0000 7f71b242f640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f71980a3bb0 0x7f71980a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000c00 tx=0x7f71a0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:39.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.044+0000 7f719f7fe640 1 -- 172.21.15.86:0/2162971308 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71a4016030 con 0x7f71ac103c00 2024-04-16T05:34:39.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.260+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71ac062c00 con 0x7f71ac103c00 2024-04-16T05:34:39.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.260+0000 7f719f7fe640 1 -- 172.21.15.86:0/2162971308 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7f71a409d030 con 0x7f71ac103c00 2024-04-16T05:34:39.259 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:34:39.260 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:33:18.613317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24463,24455,24445]},"id":1}]} 2024-04-16T05:34:39.260 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-16T05:34:39.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.262+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f71980a3bb0 msgr2=0x7f71980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.262+0000 7f71b4693640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f71980a3bb0 0x7f71980a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000c00 tx=0x7f71a0005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.262+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac103c00 msgr2=0x7f71ac19f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.262+0000 7f71b4693640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac103c00 0x7f71ac19f880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f71a400eaa0 tx=0x7f71a400ef60 comp rx=0 tx=0).stop 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.263+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 shutdown_connections 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.263+0000 7f71b4693640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71ac103c00 0x7f71ac19f880 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.263+0000 7f71b4693640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f71980a3bb0 0x7f71980a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.263+0000 7f71b4693640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ac103240 0x7f71ac198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.263+0000 7f71b4693640 1 --2- 172.21.15.86:0/2162971308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ac102050 0x7f71ac1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.263+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 >> 172.21.15.86:0/2162971308 conn(0x7f71ac0fd5e0 msgr2=0x7f71ac1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.263+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 shutdown_connections 2024-04-16T05:34:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.263+0000 7f71b4693640 1 -- 172.21.15.86:0/2162971308 wait complete. 2024-04-16T05:34:39.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:39.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 -- 172.21.15.86:0/3850271697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c107dc0 msgr2=0x7fbf6c10a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/3850271697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c107dc0 0x7fbf6c10a1b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009900 tx=0x7fbf6402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 -- 172.21.15.86:0/3850271697 shutdown_connections 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/3850271697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c107dc0 0x7fbf6c10a1b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/3850271697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c105490 0x7fbf6c107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/3850271697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c102b60 0x7fbf6c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 -- 172.21.15.86:0/3850271697 >> 172.21.15.86:0/3850271697 conn(0x7fbf6c0fc4b0 msgr2=0x7fbf6c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 -- 172.21.15.86:0/3850271697 shutdown_connections 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 -- 172.21.15.86:0/3850271697 wait complete. 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 Processor -- start 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 -- start start 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c102b60 0x7fbf6c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.379+0000 7fbf73dd4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c105490 0x7fbf6c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf73dd4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c107dc0 0x7fbf6c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf73dd4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c10dcf0 con 0x7fbf6c102b60 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf73dd4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c10db70 con 0x7fbf6c105490 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf73dd4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c10de70 con 0x7fbf6c107dc0 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf71b70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c102b60 0x7fbf6c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf71b70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c102b60 0x7fbf6c198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38744/0 (socket says 172.21.15.86:38744) 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf71b70640 1 -- 172.21.15.86:0/1800544362 learned_addr learned my addr 172.21.15.86:0/1800544362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf7136f640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c105490 0x7fbf6c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf72371640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c107dc0 0x7fbf6c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf71b70640 1 -- 172.21.15.86:0/1800544362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c107dc0 msgr2=0x7fbf6c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf71b70640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c107dc0 0x7fbf6c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf71b70640 1 -- 172.21.15.86:0/1800544362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c105490 msgr2=0x7fbf6c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf71b70640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c105490 0x7fbf6c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf71b70640 1 -- 172.21.15.86:0/1800544362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c19fe90 con 0x7fbf6c102b60 2024-04-16T05:34:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf72371640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c107dc0 0x7fbf6c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf71b70640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c102b60 0x7fbf6c198270 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c920 tx=0x7fbf5c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf62ffd640 1 -- 172.21.15.86:0/1800544362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c007c70 con 0x7fbf6c102b60 2024-04-16T05:34:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c1a0180 con 0x7fbf6c102b60 2024-04-16T05:34:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf62ffd640 1 -- 172.21.15.86:0/1800544362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf5c007e10 con 0x7fbf6c102b60 2024-04-16T05:34:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf62ffd640 1 -- 172.21.15.86:0/1800544362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c018890 con 0x7fbf6c102b60 2024-04-16T05:34:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.380+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c1a06d0 con 0x7fbf6c102b60 2024-04-16T05:34:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.381+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c0fdfa0 con 0x7fbf6c102b60 2024-04-16T05:34:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.382+0000 7fbf62ffd640 1 -- 172.21.15.86:0/1800544362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbf5c018a70 con 0x7fbf6c102b60 2024-04-16T05:34:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.383+0000 7fbf62ffd640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf580a3bb0 0x7fbf580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.383+0000 7fbf62ffd640 1 -- 172.21.15.86:0/1800544362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbf5c0d08a0 con 0x7fbf6c102b60 2024-04-16T05:34:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.383+0000 7fbf7136f640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf580a3bb0 0x7fbf580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.383+0000 7fbf7136f640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf580a3bb0 0x7fbf580a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c199780 tx=0x7fbf68009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:39.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.388+0000 7fbf62ffd640 1 -- 172.21.15.86:0/1800544362 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf5c014030 con 0x7fbf6c102b60 2024-04-16T05:34:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.571+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf6c062c00 con 0x7fbf6c102b60 2024-04-16T05:34:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.572+0000 7fbf62ffd640 1 -- 172.21.15.86:0/1800544362 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24550 (secure 0 0 0) 0x7fbf5c09cf20 con 0x7fbf6c102b60 2024-04-16T05:34:39.571 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:34:39.572 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:52.567005+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf580a3bb0 msgr2=0x7fbf580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf580a3bb0 0x7fbf580a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c199780 tx=0x7fbf68009680 comp rx=0 tx=0).stop 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c102b60 msgr2=0x7fbf6c198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c102b60 0x7fbf6c198270 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c920 tx=0x7fbf5c00cde0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 shutdown_connections 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c107dc0 0x7fbf6c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf580a3bb0 0x7fbf580a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c105490 0x7fbf6c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 --2- 172.21.15.86:0/1800544362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6c102b60 0x7fbf6c198270 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 >> 172.21.15.86:0/1800544362 conn(0x7fbf6c0fc4b0 msgr2=0x7fbf6c1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 shutdown_connections 2024-04-16T05:34:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.575+0000 7fbf73dd4640 1 -- 172.21.15.86:0/1800544362 wait complete. 2024-04-16T05:34:39.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:34:39.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:34:39.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.695+0000 7f35721ca640 1 -- 172.21.15.86:0/3195698903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 msgr2=0x7f356c107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.695+0000 7f35721ca640 1 --2- 172.21.15.86:0/3195698903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 0x7f356c107860 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3560009900 tx=0x7f356002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:39.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.695+0000 7f35721ca640 1 -- 172.21.15.86:0/3195698903 shutdown_connections 2024-04-16T05:34:39.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 --2- 172.21.15.86:0/3195698903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f356c107da0 0x7f356c10a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 --2- 172.21.15.86:0/3195698903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 0x7f356c107860 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 --2- 172.21.15.86:0/3195698903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f356c102b40 0x7f356c104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 -- 172.21.15.86:0/3195698903 >> 172.21.15.86:0/3195698903 conn(0x7f356c0fc4b0 msgr2=0x7f356c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:39.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 -- 172.21.15.86:0/3195698903 shutdown_connections 2024-04-16T05:34:39.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 -- 172.21.15.86:0/3195698903 wait complete. 2024-04-16T05:34:39.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 Processor -- start 2024-04-16T05:34:39.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 -- start start 2024-04-16T05:34:39.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f356c102b40 0x7f356c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 0x7f356c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f356c107da0 0x7f356c19f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f356c10dcd0 con 0x7f356c105470 2024-04-16T05:34:39.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f356c10db50 con 0x7f356c107da0 2024-04-16T05:34:39.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.696+0000 7f35721ca640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f356c10de50 con 0x7f356c102b40 2024-04-16T05:34:39.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 0x7f356c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 0x7f356c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38750/0 (socket says 172.21.15.86:38750) 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356affd640 1 -- 172.21.15.86:0/4188434895 learned_addr learned my addr 172.21.15.86:0/4188434895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356bfff640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f356c107da0 0x7f356c19f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356b7fe640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f356c102b40 0x7f356c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356affd640 1 -- 172.21.15.86:0/4188434895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f356c102b40 msgr2=0x7f356c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356affd640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f356c102b40 0x7f356c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356affd640 1 -- 172.21.15.86:0/4188434895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f356c107da0 msgr2=0x7f356c19f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356affd640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f356c107da0 0x7f356c19f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356affd640 1 -- 172.21.15.86:0/4188434895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c1a0000 con 0x7f356c105470 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f356affd640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 0x7f356c198920 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3560000c00 tx=0x7f3560004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f3568ff9640 1 -- 172.21.15.86:0/4188434895 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560041070 con 0x7f356c105470 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f3568ff9640 1 -- 172.21.15.86:0/4188434895 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35600054e0 con 0x7f356c105470 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c1a0290 con 0x7f356c105470 2024-04-16T05:34:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f3568ff9640 1 -- 172.21.15.86:0/4188434895 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600346a0 con 0x7f356c105470 2024-04-16T05:34:39.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.697+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c1a0780 con 0x7f356c105470 2024-04-16T05:34:39.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.698+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f356c0fdfa0 con 0x7f356c105470 2024-04-16T05:34:39.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.699+0000 7f3568ff9640 1 -- 172.21.15.86:0/4188434895 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3560044070 con 0x7f356c105470 2024-04-16T05:34:39.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.700+0000 7f3568ff9640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35540a3bb0 0x7f35540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.700+0000 7f356b7fe640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35540a3bb0 0x7f35540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.700+0000 7f3568ff9640 1 -- 172.21.15.86:0/4188434895 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f35600f2450 con 0x7f356c105470 2024-04-16T05:34:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.700+0000 7f356b7fe640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35540a3bb0 0x7f35540a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f355c0058c0 tx=0x7f355c00a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:39.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.704+0000 7f3568ff9640 1 -- 172.21.15.86:0/4188434895 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35600bead0 con 0x7f356c105470 2024-04-16T05:34:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.882+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f356c062c00 con 0x7f356c105470 2024-04-16T05:34:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.882+0000 7f3568ff9640 1 -- 172.21.15.86:0/4188434895 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f35600be3d0 con 0x7f356c105470 2024-04-16T05:34:39.881 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35540a3bb0 msgr2=0x7f35540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35540a3bb0 0x7f35540a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f355c0058c0 tx=0x7f355c00a4c0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 msgr2=0x7f356c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 0x7f356c198920 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3560000c00 tx=0x7f3560004040 comp rx=0 tx=0).stop 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 shutdown_connections 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f35540a3bb0 0x7f35540a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f356c107da0 0x7f356c19f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c105470 0x7f356c198920 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 --2- 172.21.15.86:0/4188434895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f356c102b40 0x7f356c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 >> 172.21.15.86:0/4188434895 conn(0x7f356c0fc4b0 msgr2=0x7f356c104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:39.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 shutdown_connections 2024-04-16T05:34:39.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:39.884+0000 7f35721ca640 1 -- 172.21.15.86:0/4188434895 wait complete. 2024-04-16T05:34:39.893 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:39 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2841386184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:39.893 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:39 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/330390639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:39.893 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:39 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2162971308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:39.893 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:39 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1800544362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:39.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:34:39.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:39 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2841386184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:39.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:39 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/330390639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:39.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:39 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2162971308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:39.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:39 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1800544362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:39 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2841386184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:39 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/330390639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:39 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2162971308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:39 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1800544362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.003+0000 7f404901b640 1 -- 172.21.15.86:0/3177035545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044102050 msgr2=0x7f4044102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.003+0000 7f404901b640 1 --2- 172.21.15.86:0/3177035545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044102050 0x7f4044102470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4034009900 tx=0x7f403402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.003+0000 7f404901b640 1 -- 172.21.15.86:0/3177035545 shutdown_connections 2024-04-16T05:34:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.003+0000 7f404901b640 1 --2- 172.21.15.86:0/3177035545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4044103c00 0x7f404410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.003+0000 7f404901b640 1 --2- 172.21.15.86:0/3177035545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4044103240 0x7f40441036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.003+0000 7f404901b640 1 --2- 172.21.15.86:0/3177035545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044102050 0x7f4044102470 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.003+0000 7f404901b640 1 -- 172.21.15.86:0/3177035545 >> 172.21.15.86:0/3177035545 conn(0x7f40440fd5e0 msgr2=0x7f40440ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.003+0000 7f404901b640 1 -- 172.21.15.86:0/3177035545 shutdown_connections 2024-04-16T05:34:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.003+0000 7f404901b640 1 -- 172.21.15.86:0/3177035545 wait complete. 2024-04-16T05:34:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404901b640 1 Processor -- start 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404901b640 1 -- start start 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404901b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4044102050 0x7f4044198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404901b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4044103240 0x7f40441988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404901b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044103c00 0x7f404419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404901b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f404410dce0 con 0x7f4044103c00 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404901b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f404410db60 con 0x7f4044102050 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404901b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f404410de60 con 0x7f4044103240 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044103c00 0x7f404419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044103c00 0x7f404419f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38770/0 (socket says 172.21.15.86:38770) 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404359e640 1 -- 172.21.15.86:0/2728846467 learned_addr learned my addr 172.21.15.86:0/2728846467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f4042d9d640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4044102050 0x7f4044198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404259c640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4044103240 0x7f40441988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404359e640 1 -- 172.21.15.86:0/2728846467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4044103240 msgr2=0x7f40441988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404359e640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4044103240 0x7f40441988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404359e640 1 -- 172.21.15.86:0/2728846467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4044102050 msgr2=0x7f4044198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404359e640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4044102050 0x7f4044198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.004+0000 7f404359e640 1 -- 172.21.15.86:0/2728846467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404419ffa0 con 0x7f4044103c00 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.005+0000 7f404359e640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044103c00 0x7f404419f930 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f403c00eaa0 tx=0x7f403c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.005+0000 7f402bfff640 1 -- 172.21.15.86:0/2728846467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c00cdb0 con 0x7f4044103c00 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.005+0000 7f402bfff640 1 -- 172.21.15.86:0/2728846467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f403c005020 con 0x7f4044103c00 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.005+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40441a0290 con 0x7f4044103c00 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.005+0000 7f402bfff640 1 -- 172.21.15.86:0/2728846467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0116b0 con 0x7f4044103c00 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.005+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40441a0790 con 0x7f4044103c00 2024-04-16T05:34:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.006+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4044109390 con 0x7f4044103c00 2024-04-16T05:34:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.008+0000 7f402bfff640 1 -- 172.21.15.86:0/2728846467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f403c005560 con 0x7f4044103c00 2024-04-16T05:34:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.008+0000 7f402bfff640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f402c0a3bb0 0x7f402c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.009+0000 7f4042d9d640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f402c0a3bb0 0x7f402c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.009+0000 7f402bfff640 1 -- 172.21.15.86:0/2728846467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f403c0d0d00 con 0x7f4044103c00 2024-04-16T05:34:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.009+0000 7f4042d9d640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f402c0a3bb0 0x7f402c0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4034000c00 tx=0x7f4034005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.012+0000 7f402bfff640 1 -- 172.21.15.86:0/2728846467 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f403c016030 con 0x7f4044103c00 2024-04-16T05:34:40.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.190+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f404402d620 con 0x7f4044103c00 2024-04-16T05:34:40.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.191+0000 7f402bfff640 1 -- 172.21.15.86:0/2728846467 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f403c09ccd0 con 0x7f4044103c00 2024-04-16T05:34:40.190 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f402c0a3bb0 msgr2=0x7f402c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f402c0a3bb0 0x7f402c0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4034000c00 tx=0x7f4034005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044103c00 msgr2=0x7f404419f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044103c00 0x7f404419f930 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f403c00eaa0 tx=0x7f403c00ef60 comp rx=0 tx=0).stop 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 shutdown_connections 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4044103c00 0x7f404419f930 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4044103240 0x7f40441988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f402c0a3bb0 0x7f402c0a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 --2- 172.21.15.86:0/2728846467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4044102050 0x7f4044198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 >> 172.21.15.86:0/2728846467 conn(0x7f40440fd5e0 msgr2=0x7f4044106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 shutdown_connections 2024-04-16T05:34:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.193+0000 7f404901b640 1 -- 172.21.15.86:0/2728846467 wait complete. 2024-04-16T05:34:40.203 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.311+0000 7f14513e9640 1 -- 172.21.15.86:0/3269444349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c103bf0 msgr2=0x7f144c10a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.311+0000 7f14513e9640 1 --2- 172.21.15.86:0/3269444349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c103bf0 0x7f144c10a480 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1444009ca0 tx=0x7f144402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.311+0000 7f14513e9640 1 -- 172.21.15.86:0/3269444349 shutdown_connections 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.311+0000 7f14513e9640 1 --2- 172.21.15.86:0/3269444349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c103bf0 0x7f144c10a480 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.311+0000 7f14513e9640 1 --2- 172.21.15.86:0/3269444349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f144c103230 0x7f144c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.311+0000 7f14513e9640 1 --2- 172.21.15.86:0/3269444349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f144c102040 0x7f144c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.311+0000 7f14513e9640 1 -- 172.21.15.86:0/3269444349 >> 172.21.15.86:0/3269444349 conn(0x7f144c0fd5b0 msgr2=0x7f144c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.311+0000 7f14513e9640 1 -- 172.21.15.86:0/3269444349 shutdown_connections 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.311+0000 7f14513e9640 1 -- 172.21.15.86:0/3269444349 wait complete. 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f14513e9640 1 Processor -- start 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f14513e9640 1 -- start start 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f14513e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c102040 0x7f144c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f14513e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f144c103230 0x7f144c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f14513e9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f144c103bf0 0x7f144c19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f14513e9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10dcd0 con 0x7f144c102040 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f14513e9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10db50 con 0x7f144c103bf0 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f14513e9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10de50 con 0x7f144c103230 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f144affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c102040 0x7f144c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f144affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c102040 0x7f144c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38802/0 (socket says 172.21.15.86:38802) 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.312+0000 7f144affd640 1 -- 172.21.15.86:0/2461877725 learned_addr learned my addr 172.21.15.86:0/2461877725 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f144affd640 1 -- 172.21.15.86:0/2461877725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f144c103230 msgr2=0x7f144c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f144affd640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f144c103230 0x7f144c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f144affd640 1 -- 172.21.15.86:0/2461877725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f144c103bf0 msgr2=0x7f144c19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f144affd640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f144c103bf0 0x7f144c19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f144affd640 1 -- 172.21.15.86:0/2461877725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c19ff60 con 0x7f144c102040 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f144affd640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c102040 0x7f144c198340 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f143c00d950 tx=0x7f143c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f1433fff640 1 -- 172.21.15.86:0/2461877725 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c017020 con 0x7f144c102040 2024-04-16T05:34:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c1a0250 con 0x7f144c102040 2024-04-16T05:34:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f1433fff640 1 -- 172.21.15.86:0/2461877725 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c009c30 con 0x7f144c102040 2024-04-16T05:34:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f1433fff640 1 -- 172.21.15.86:0/2461877725 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00f740 con 0x7f144c102040 2024-04-16T05:34:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.313+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c1a07a0 con 0x7f144c102040 2024-04-16T05:34:40.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.314+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144c109300 con 0x7f144c102040 2024-04-16T05:34:40.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.315+0000 7f1433fff640 1 -- 172.21.15.86:0/2461877725 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f143c01a020 con 0x7f144c102040 2024-04-16T05:34:40.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.315+0000 7f1433fff640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f14340a3bb0 0x7f14340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.316+0000 7f144a7fc640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f14340a3bb0 0x7f14340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.316+0000 7f1433fff640 1 -- 172.21.15.86:0/2461877725 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f143c0d01f0 con 0x7f144c102040 2024-04-16T05:34:40.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.316+0000 7f144a7fc640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f14340a3bb0 0x7f14340a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f144c199850 tx=0x7f1440005b60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:40.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.320+0000 7f1433fff640 1 -- 172.21.15.86:0/2461877725 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f143c021070 con 0x7f144c102040 2024-04-16T05:34:40.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.499+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f144c02d620 con 0x7f144c102040 2024-04-16T05:34:40.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.500+0000 7f1433fff640 1 -- 172.21.15.86:0/2461877725 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f144c02d620 con 0x7f144c102040 2024-04-16T05:34:40.498 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:34:40.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f14340a3bb0 msgr2=0x7f14340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f14340a3bb0 0x7f14340a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f144c199850 tx=0x7f1440005b60 comp rx=0 tx=0).stop 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c102040 msgr2=0x7f144c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c102040 0x7f144c198340 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f143c00d950 tx=0x7f143c00de10 comp rx=0 tx=0).stop 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 shutdown_connections 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f14340a3bb0 0x7f14340a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f144c103bf0 0x7f144c19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f144c103230 0x7f144c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 --2- 172.21.15.86:0/2461877725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144c102040 0x7f144c198340 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 >> 172.21.15.86:0/2461877725 conn(0x7f144c0fd5b0 msgr2=0x7f144c1064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 shutdown_connections 2024-04-16T05:34:40.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.502+0000 7f14513e9640 1 -- 172.21.15.86:0/2461877725 wait complete. 2024-04-16T05:34:40.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:34:40.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.621+0000 7ff313cd9640 1 -- 172.21.15.86:0/2181891160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c100f40 msgr2=0x7ff30c101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.621+0000 7ff313cd9640 1 --2- 172.21.15.86:0/2181891160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c100f40 0x7ff30c101360 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff308009ca0 tx=0x7ff30802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 -- 172.21.15.86:0/2181891160 shutdown_connections 2024-04-16T05:34:40.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 --2- 172.21.15.86:0/2181891160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff30c102a50 0x7ff30c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 --2- 172.21.15.86:0/2181891160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff30c102090 0x7ff30c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 --2- 172.21.15.86:0/2181891160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c100f40 0x7ff30c101360 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 -- 172.21.15.86:0/2181891160 >> 172.21.15.86:0/2181891160 conn(0x7ff30c0fc4b0 msgr2=0x7ff30c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 -- 172.21.15.86:0/2181891160 shutdown_connections 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 -- 172.21.15.86:0/2181891160 wait complete. 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 Processor -- start 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 -- start start 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff30c100f40 0x7ff30c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff30c102090 0x7ff30c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102a50 0x7ff30c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10cbf0 con 0x7ff30c102a50 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10ca70 con 0x7ff30c100f40 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff313cd9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10cd70 con 0x7ff30c102090 2024-04-16T05:34:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.622+0000 7ff312276640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102a50 0x7ff30c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff312276640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102a50 0x7ff30c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60222/0 (socket says 172.21.15.86:60222) 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff312276640 1 -- 172.21.15.86:0/3707607250 learned_addr learned my addr 172.21.15.86:0/3707607250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff311a75640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff30c100f40 0x7ff30c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff311274640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff30c102090 0x7ff30c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff312276640 1 -- 172.21.15.86:0/3707607250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff30c102090 msgr2=0x7ff30c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff312276640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff30c102090 0x7ff30c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff312276640 1 -- 172.21.15.86:0/3707607250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff30c100f40 msgr2=0x7ff30c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff312276640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff30c100f40 0x7ff30c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff312276640 1 -- 172.21.15.86:0/3707607250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c19eed0 con 0x7ff30c102a50 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff312276640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102a50 0x7ff30c19e860 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff30400e970 tx=0x7ff30400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff302ffd640 1 -- 172.21.15.86:0/3707607250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30400ccc0 con 0x7ff30c102a50 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff302ffd640 1 -- 172.21.15.86:0/3707607250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30400ce60 con 0x7ff30c102a50 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c19f1c0 con 0x7ff30c102a50 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff302ffd640 1 -- 172.21.15.86:0/3707607250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304010490 con 0x7ff30c102a50 2024-04-16T05:34:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.623+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c19f710 con 0x7ff30c102a50 2024-04-16T05:34:40.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.624+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30c108320 con 0x7ff30c102a50 2024-04-16T05:34:40.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.625+0000 7ff302ffd640 1 -- 172.21.15.86:0/3707607250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff304005020 con 0x7ff30c102a50 2024-04-16T05:34:40.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.626+0000 7ff302ffd640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff2f80a3bb0 0x7ff2f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.626+0000 7ff311a75640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff2f80a3bb0 0x7ff2f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.626+0000 7ff302ffd640 1 -- 172.21.15.86:0/3707607250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff3040d0b80 con 0x7ff30c102a50 2024-04-16T05:34:40.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.626+0000 7ff311a75640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff2f80a3bb0 0x7ff2f80a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff308006010 tx=0x7ff308032590 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:40.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.630+0000 7ff302ffd640 1 -- 172.21.15.86:0/3707607250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff30409d200 con 0x7ff30c102a50 2024-04-16T05:34:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.809+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff30c062c00 con 0x7ff30c102a50 2024-04-16T05:34:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.810+0000 7ff302ffd640 1 -- 172.21.15.86:0/3707607250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7ff30409cb00 con 0x7ff30c102a50 2024-04-16T05:34:40.808 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:34:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff2f80a3bb0 msgr2=0x7ff2f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff2f80a3bb0 0x7ff2f80a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff308006010 tx=0x7ff308032590 comp rx=0 tx=0).stop 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102a50 msgr2=0x7ff30c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102a50 0x7ff30c19e860 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff30400e970 tx=0x7ff30400ed70 comp rx=0 tx=0).stop 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 shutdown_connections 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102a50 0x7ff30c19e860 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff30c102090 0x7ff30c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff2f80a3bb0 0x7ff2f80a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 --2- 172.21.15.86:0/3707607250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff30c100f40 0x7ff30c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 >> 172.21.15.86:0/3707607250 conn(0x7ff30c0fc4b0 msgr2=0x7ff30c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 shutdown_connections 2024-04-16T05:34:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.812+0000 7ff313cd9640 1 -- 172.21.15.86:0/3707607250 wait complete. 2024-04-16T05:34:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:40 smithi086 ceph-mon[32028]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:40 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4188434895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:40 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2728846467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:40 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2461877725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:40.822 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:34:40.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:40 smithi194 ceph-mon[35865]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:40 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4188434895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:40 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2728846467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:40 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2461877725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.927+0000 7f08e473d640 1 -- 172.21.15.86:0/3472350620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102a90 msgr2=0x7f08dc1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.927+0000 7f08e473d640 1 --2- 172.21.15.86:0/3472350620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102a90 0x7f08dc1093b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009ca0 tx=0x7f08d402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.927+0000 7f08e473d640 1 -- 172.21.15.86:0/3472350620 shutdown_connections 2024-04-16T05:34:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.927+0000 7f08e473d640 1 --2- 172.21.15.86:0/3472350620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102a90 0x7f08dc1093b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.927+0000 7f08e473d640 1 --2- 172.21.15.86:0/3472350620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08dc1020d0 0x7f08dc102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.927+0000 7f08e473d640 1 --2- 172.21.15.86:0/3472350620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08dc100f00 0x7f08dc101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.927+0000 7f08e473d640 1 -- 172.21.15.86:0/3472350620 >> 172.21.15.86:0/3472350620 conn(0x7f08dc0fc4e0 msgr2=0x7f08dc0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.927+0000 7f08e473d640 1 -- 172.21.15.86:0/3472350620 shutdown_connections 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.927+0000 7f08e473d640 1 -- 172.21.15.86:0/3472350620 wait complete. 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e473d640 1 Processor -- start 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e473d640 1 -- start start 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e473d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100f00 0x7f08dc06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e473d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08dc1020d0 0x7f08dc19d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e473d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08dc102a90 0x7f08dc19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e473d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc10ccc0 con 0x7f08dc100f00 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e473d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc10cb40 con 0x7f08dc102a90 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e473d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc10ce40 con 0x7f08dc1020d0 2024-04-16T05:34:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e24d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100f00 0x7f08dc06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e24d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100f00 0x7f08dc06b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60236/0 (socket says 172.21.15.86:60236) 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e24d9640 1 -- 172.21.15.86:0/1572832973 learned_addr learned my addr 172.21.15.86:0/1572832973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e2cda640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08dc102a90 0x7f08dc19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e1cd8640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08dc1020d0 0x7f08dc19d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e24d9640 1 -- 172.21.15.86:0/1572832973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08dc1020d0 msgr2=0x7f08dc19d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e24d9640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08dc1020d0 0x7f08dc19d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e24d9640 1 -- 172.21.15.86:0/1572832973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08dc102a90 msgr2=0x7f08dc19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e24d9640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08dc102a90 0x7f08dc19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e24d9640 1 -- 172.21.15.86:0/1572832973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc1a0090 con 0x7f08dc100f00 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.928+0000 7f08e1cd8640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08dc1020d0 0x7f08dc19d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.929+0000 7f08e24d9640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100f00 0x7f08dc06b1d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b450 tx=0x7f08d800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.929+0000 7f08d37fe640 1 -- 172.21.15.86:0/1572832973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8015020 con 0x7f08dc100f00 2024-04-16T05:34:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.929+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc1a0320 con 0x7f08dc100f00 2024-04-16T05:34:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.929+0000 7f08d37fe640 1 -- 172.21.15.86:0/1572832973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d8011d70 con 0x7f08dc100f00 2024-04-16T05:34:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.929+0000 7f08d37fe640 1 -- 172.21.15.86:0/1572832973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d800f340 con 0x7f08dc100f00 2024-04-16T05:34:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.929+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc1a08f0 con 0x7f08dc100f00 2024-04-16T05:34:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.930+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08dc108360 con 0x7f08dc100f00 2024-04-16T05:34:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.931+0000 7f08d37fe640 1 -- 172.21.15.86:0/1572832973 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f08d8018020 con 0x7f08dc100f00 2024-04-16T05:34:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.931+0000 7f08d37fe640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08c80a3bb0 0x7f08c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.932+0000 7f08e1cd8640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08c80a3bb0 0x7f08c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.932+0000 7f08d37fe640 1 -- 172.21.15.86:0/1572832973 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f08d80d07d0 con 0x7f08dc100f00 2024-04-16T05:34:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.932+0000 7f08e1cd8640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08c80a3bb0 0x7f08c80a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08cc006fd0 tx=0x7f08cc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:40.935+0000 7f08d37fe640 1 -- 172.21.15.86:0/1572832973 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08d801f020 con 0x7f08dc100f00 2024-04-16T05:34:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:40 smithi196 ceph-mon[36373]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:40 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4188434895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:40 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2728846467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:40 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2461877725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.150+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08dc02d870 con 0x7f08dc100f00 2024-04-16T05:34:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.151+0000 7f08d37fe640 1 -- 172.21.15.86:0/1572832973 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7f08d800d040 con 0x7f08dc100f00 2024-04-16T05:34:41.149 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:34:41.149 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:33:18.613317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24463,24455,24445]},"id":1}]} 2024-04-16T05:34:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.152+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08c80a3bb0 msgr2=0x7f08c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.152+0000 7f08e473d640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08c80a3bb0 0x7f08c80a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08cc006fd0 tx=0x7f08cc008040 comp rx=0 tx=0).stop 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.152+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100f00 msgr2=0x7f08dc06b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.153+0000 7f08e473d640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100f00 0x7f08dc06b1d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b450 tx=0x7f08d800b910 comp rx=0 tx=0).stop 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.153+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 shutdown_connections 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.153+0000 7f08e473d640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08c80a3bb0 0x7f08c80a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.153+0000 7f08e473d640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08dc102a90 0x7f08dc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.153+0000 7f08e473d640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08dc1020d0 0x7f08dc19d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.153+0000 7f08e473d640 1 --2- 172.21.15.86:0/1572832973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100f00 0x7f08dc06b1d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.153+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 >> 172.21.15.86:0/1572832973 conn(0x7f08dc0fc4e0 msgr2=0x7f08dc105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.153+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 shutdown_connections 2024-04-16T05:34:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.153+0000 7f08e473d640 1 -- 172.21.15.86:0/1572832973 wait complete. 2024-04-16T05:34:41.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.268+0000 7f4dd4167640 1 -- 172.21.15.86:0/2862602604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 msgr2=0x7f4dcc102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.268+0000 7f4dd4167640 1 --2- 172.21.15.86:0/2862602604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 0x7f4dcc102560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009900 tx=0x7f4dc002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.268+0000 7f4dd4167640 1 -- 172.21.15.86:0/2862602604 shutdown_connections 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.268+0000 7f4dd4167640 1 --2- 172.21.15.86:0/2862602604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4dcc102aa0 0x7f4dcc1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.268+0000 7f4dd4167640 1 --2- 172.21.15.86:0/2862602604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 0x7f4dcc102560 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.268+0000 7f4dd4167640 1 --2- 172.21.15.86:0/2862602604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dcc100f10 0x7f4dcc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.268+0000 7f4dd4167640 1 -- 172.21.15.86:0/2862602604 >> 172.21.15.86:0/2862602604 conn(0x7f4dcc0fc4f0 msgr2=0x7f4dcc0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.268+0000 7f4dd4167640 1 -- 172.21.15.86:0/2862602604 shutdown_connections 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.268+0000 7f4dd4167640 1 -- 172.21.15.86:0/2862602604 wait complete. 2024-04-16T05:34:41.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd4167640 1 Processor -- start 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd4167640 1 -- start start 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd4167640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dcc100f10 0x7f4dcc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd4167640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 0x7f4dcc198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd4167640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4dcc102aa0 0x7f4dcc19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd4167640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc10cbe0 con 0x7f4dcc1020e0 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd4167640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc10ca60 con 0x7f4dcc102aa0 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd4167640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc10cd60 con 0x7f4dcc100f10 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1702640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 0x7f4dcc198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1702640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 0x7f4dcc198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60248/0 (socket says 172.21.15.86:60248) 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1702640 1 -- 172.21.15.86:0/1288710466 learned_addr learned my addr 172.21.15.86:0/1288710466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1f03640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dcc100f10 0x7f4dcc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd2704640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4dcc102aa0 0x7f4dcc19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1702640 1 -- 172.21.15.86:0/1288710466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dcc100f10 msgr2=0x7f4dcc1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1702640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dcc100f10 0x7f4dcc1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1702640 1 -- 172.21.15.86:0/1288710466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4dcc102aa0 msgr2=0x7f4dcc19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1702640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4dcc102aa0 0x7f4dcc19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1702640 1 -- 172.21.15.86:0/1288710466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc19ffb0 con 0x7f4dcc1020e0 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd2704640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4dcc102aa0 0x7f4dcc19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.269+0000 7f4dd1702640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 0x7f4dcc198920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009230 tx=0x7f4dc0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.270+0000 7f4dcaffd640 1 -- 172.21.15.86:0/1288710466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0041070 con 0x7f4dcc1020e0 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.270+0000 7f4dcaffd640 1 -- 172.21.15.86:0/1288710466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc00054e0 con 0x7f4dcc1020e0 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.270+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc1a0240 con 0x7f4dcc1020e0 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.270+0000 7f4dcaffd640 1 -- 172.21.15.86:0/1288710466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0034440 con 0x7f4dcc1020e0 2024-04-16T05:34:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.270+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc1a0730 con 0x7f4dcc1020e0 2024-04-16T05:34:41.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.271+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dcc1a0990 con 0x7f4dcc1020e0 2024-04-16T05:34:41.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.272+0000 7f4dcaffd640 1 -- 172.21.15.86:0/1288710466 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4dc0044070 con 0x7f4dcc1020e0 2024-04-16T05:34:41.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.272+0000 7f4dcaffd640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4db40a3bb0 0x7f4db40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.272+0000 7f4dd1f03640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4db40a3bb0 0x7f4db40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.272+0000 7f4dcaffd640 1 -- 172.21.15.86:0/1288710466 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4dc00f2850 con 0x7f4dcc1020e0 2024-04-16T05:34:41.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.273+0000 7f4dd1f03640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4db40a3bb0 0x7f4db40a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4005fd0 tx=0x7f4dc400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.276+0000 7f4dcaffd640 1 -- 172.21.15.86:0/1288710466 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4dc00beed0 con 0x7f4dcc1020e0 2024-04-16T05:34:41.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.454+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4dcc1a0b60 con 0x7f4dcc1020e0 2024-04-16T05:34:41.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.455+0000 7f4dcaffd640 1 -- 172.21.15.86:0/1288710466 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24550 (secure 0 0 0) 0x7f4dc00be7d0 con 0x7f4dcc1020e0 2024-04-16T05:34:41.454 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:34:41.455 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:33:52.567005+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:41.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.457+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4db40a3bb0 msgr2=0x7f4db40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.457+0000 7f4dd4167640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4db40a3bb0 0x7f4db40a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4005fd0 tx=0x7f4dc400b390 comp rx=0 tx=0).stop 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.457+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 msgr2=0x7f4dcc198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.457+0000 7f4dd4167640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 0x7f4dcc198920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009230 tx=0x7f4dc0005dd0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.458+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 shutdown_connections 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.458+0000 7f4dd4167640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f4db40a3bb0 0x7f4db40a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.458+0000 7f4dd4167640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4dcc102aa0 0x7f4dcc19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.458+0000 7f4dd4167640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1020e0 0x7f4dcc198920 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.458+0000 7f4dd4167640 1 --2- 172.21.15.86:0/1288710466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dcc100f10 0x7f4dcc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.458+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 >> 172.21.15.86:0/1288710466 conn(0x7f4dcc0fc4f0 msgr2=0x7f4dcc0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.458+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 shutdown_connections 2024-04-16T05:34:41.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.458+0000 7f4dd4167640 1 -- 172.21.15.86:0/1288710466 wait complete. 2024-04-16T05:34:41.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:34:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.574+0000 7ff9cba47640 1 -- 172.21.15.86:0/2852112772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4100310 msgr2=0x7ff9c4100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.574+0000 7ff9cba47640 1 --2- 172.21.15.86:0/2852112772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4100310 0x7ff9c4100730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009900 tx=0x7ff9c002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.574+0000 7ff9cba47640 1 -- 172.21.15.86:0/2852112772 shutdown_connections 2024-04-16T05:34:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.574+0000 7ff9cba47640 1 --2- 172.21.15.86:0/2852112772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4101a30 0x7ff9c41062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.574+0000 7ff9cba47640 1 --2- 172.21.15.86:0/2852112772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c4101070 0x7ff9c41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.574+0000 7ff9cba47640 1 --2- 172.21.15.86:0/2852112772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4100310 0x7ff9c4100730 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.574+0000 7ff9cba47640 1 -- 172.21.15.86:0/2852112772 >> 172.21.15.86:0/2852112772 conn(0x7ff9c40fbc40 msgr2=0x7ff9c40fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.574+0000 7ff9cba47640 1 -- 172.21.15.86:0/2852112772 shutdown_connections 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.574+0000 7ff9cba47640 1 -- 172.21.15.86:0/2852112772 wait complete. 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9cba47640 1 Processor -- start 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9cba47640 1 -- start start 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9cba47640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100310 0x7ff9c419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9cba47640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4101070 0x7ff9c419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9cba47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c4101a30 0x7ff9c41a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9cba47640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41142e0 con 0x7ff9c4101070 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9cba47640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4114160 con 0x7ff9c4100310 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9cba47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4114460 con 0x7ff9c4101a30 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c8fe2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4101070 0x7ff9c419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c8fe2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4101070 0x7ff9c419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60256/0 (socket says 172.21.15.86:60256) 2024-04-16T05:34:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c8fe2640 1 -- 172.21.15.86:0/3676429300 learned_addr learned my addr 172.21.15.86:0/3676429300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c9fe4640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c4101a30 0x7ff9c41a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c97e3640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100310 0x7ff9c419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c8fe2640 1 -- 172.21.15.86:0/3676429300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c4101a30 msgr2=0x7ff9c41a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c8fe2640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c4101a30 0x7ff9c41a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c8fe2640 1 -- 172.21.15.86:0/3676429300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100310 msgr2=0x7ff9c419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c8fe2640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100310 0x7ff9c419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c8fe2640 1 -- 172.21.15.86:0/3676429300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c41a65a0 con 0x7ff9c4101070 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.575+0000 7ff9c9fe4640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c4101a30 0x7ff9c41a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.576+0000 7ff9c8fe2640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4101070 0x7ff9c419eec0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400c970 tx=0x7ff9b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.576+0000 7ff9ba7fc640 1 -- 172.21.15.86:0/3676429300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4004220 con 0x7ff9c4101070 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.576+0000 7ff9ba7fc640 1 -- 172.21.15.86:0/3676429300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b40043c0 con 0x7ff9c4101070 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.576+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c41a6890 con 0x7ff9c4101070 2024-04-16T05:34:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.576+0000 7ff9ba7fc640 1 -- 172.21.15.86:0/3676429300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4004a00 con 0x7ff9c4101070 2024-04-16T05:34:41.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.576+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a6d90 con 0x7ff9c4101070 2024-04-16T05:34:41.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.577+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c41a7050 con 0x7ff9c4101070 2024-04-16T05:34:41.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.578+0000 7ff9ba7fc640 1 -- 172.21.15.86:0/3676429300 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff9b40079b0 con 0x7ff9c4101070 2024-04-16T05:34:41.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.579+0000 7ff9ba7fc640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff9b00a3bb0 0x7ff9b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.579+0000 7ff9c97e3640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff9b00a3bb0 0x7ff9b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.579+0000 7ff9ba7fc640 1 -- 172.21.15.86:0/3676429300 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff9b40d0570 con 0x7ff9c4101070 2024-04-16T05:34:41.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.579+0000 7ff9c97e3640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff9b00a3bb0 0x7ff9b00a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005c20 tx=0x7ff9c0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.584+0000 7ff9ba7fc640 1 -- 172.21.15.86:0/3676429300 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9b4014030 con 0x7ff9c4101070 2024-04-16T05:34:41.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:41 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3707607250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:41.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:41 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1572832973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:41.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:41 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1288710466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:41.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.801+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9c41a7220 con 0x7ff9c4101070 2024-04-16T05:34:41.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.802+0000 7ff9ba7fc640 1 -- 172.21.15.86:0/3676429300 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7ff9b409cbf0 con 0x7ff9c4101070 2024-04-16T05:34:41.801 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:34:41.801 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:33:18.613317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24463,24455,24445]},"id":1}]} 2024-04-16T05:34:41.801 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff9b00a3bb0 msgr2=0x7ff9b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff9b00a3bb0 0x7ff9b00a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005c20 tx=0x7ff9c0005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4101070 msgr2=0x7ff9c419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4101070 0x7ff9c419eec0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400c970 tx=0x7ff9b400ce30 comp rx=0 tx=0).stop 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 shutdown_connections 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c4101a30 0x7ff9c41a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c4101070 0x7ff9c419eec0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff9b00a3bb0 0x7ff9b00a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 --2- 172.21.15.86:0/3676429300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100310 0x7ff9c419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 >> 172.21.15.86:0/3676429300 conn(0x7ff9c40fbc40 msgr2=0x7ff9c40fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 shutdown_connections 2024-04-16T05:34:41.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.804+0000 7ff9cba47640 1 -- 172.21.15.86:0/3676429300 wait complete. 2024-04-16T05:34:41.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-16T05:32:54.852973+0000', 'modified': '2024-04-16T05:33:18.613317+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24481, 'mds_2': 24463, 'mds_3': 24455, 'mds_4': 24445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/1379378484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 1379378484}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 1379378484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14631': {'gid': 14631, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/1989684988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 1989684988}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 1989684988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6833/3318048466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6832', 'nonce': 3318048466}, {'type': 'v1', 'addr': '172.21.15.194:6833', 'nonce': 3318048466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/1220860047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 1220860047}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 1220860047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/2618844409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 2618844409}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 2618844409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.196:6835/3736061748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 3736061748}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 3736061748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.194:6837/3400155977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 3400155977}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 3400155977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.196:6837/2445566344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 2445566344}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 2445566344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.196:6839/3455974815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 3455974815}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 3455974815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.194:6839/601781370', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 601781370}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 601781370}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24481, 24482, 24463, 24455, 24445]} 2024-04-16T05:34:41.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-16T05:34:41.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-16T05:34:41.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:41 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3707607250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:41.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:41 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1572832973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:41.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:41 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1288710466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.921+0000 7f278464a640 1 -- 172.21.15.86:0/716968125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c1011b0 msgr2=0x7f277c101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.921+0000 7f278464a640 1 --2- 172.21.15.86:0/716968125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c1011b0 0x7f277c101630 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f276c009900 tx=0x7f276c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.921+0000 7f278464a640 1 -- 172.21.15.86:0/716968125 shutdown_connections 2024-04-16T05:34:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.921+0000 7f278464a640 1 --2- 172.21.15.86:0/716968125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f277c101b70 0x7f277c109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.921+0000 7f278464a640 1 --2- 172.21.15.86:0/716968125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c1011b0 0x7f277c101630 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.921+0000 7f278464a640 1 --2- 172.21.15.86:0/716968125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f277c102560 0x7f277c102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.921+0000 7f278464a640 1 -- 172.21.15.86:0/716968125 >> 172.21.15.86:0/716968125 conn(0x7f277c0fbc40 msgr2=0x7f277c0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.921+0000 7f278464a640 1 -- 172.21.15.86:0/716968125 shutdown_connections 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.921+0000 7f278464a640 1 -- 172.21.15.86:0/716968125 wait complete. 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f278464a640 1 Processor -- start 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f278464a640 1 -- start start 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f278464a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f277c1011b0 0x7f277c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f278464a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c101b70 0x7f277c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f278464a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f277c102560 0x7f277c19f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f278464a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f277c10dd70 con 0x7f277c101b70 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f278464a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f277c10dbf0 con 0x7f277c1011b0 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f278464a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f277c10def0 con 0x7f277c102560 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f2781be5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c101b70 0x7f277c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f2781be5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c101b70 0x7f277c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60282/0 (socket says 172.21.15.86:60282) 2024-04-16T05:34:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f2781be5640 1 -- 172.21.15.86:0/3303131864 learned_addr learned my addr 172.21.15.86:0/3303131864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f2782be7640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f277c102560 0x7f277c19f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f2781be5640 1 -- 172.21.15.86:0/3303131864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f277c102560 msgr2=0x7f277c19f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f2781be5640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f277c102560 0x7f277c19f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f2781be5640 1 -- 172.21.15.86:0/3303131864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f277c1011b0 msgr2=0x7f277c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f2781be5640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f277c1011b0 0x7f277c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.922+0000 7f2781be5640 1 -- 172.21.15.86:0/3303131864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277c1a0000 con 0x7f277c101b70 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.923+0000 7f2781be5640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c101b70 0x7f277c198920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f276c02efd0 tx=0x7f276c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.923+0000 7f27737fe640 1 -- 172.21.15.86:0/3303131864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c041070 con 0x7f277c101b70 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.923+0000 7f27737fe640 1 -- 172.21.15.86:0/3303131864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276c0044c0 con 0x7f277c101b70 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.923+0000 7f278464a640 1 -- 172.21.15.86:0/3303131864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277c1a0290 con 0x7f277c101b70 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.923+0000 7f27737fe640 1 -- 172.21.15.86:0/3303131864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c0346a0 con 0x7f277c101b70 2024-04-16T05:34:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.923+0000 7f278464a640 1 -- 172.21.15.86:0/3303131864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277c1a07d0 con 0x7f277c101b70 2024-04-16T05:34:41.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f27737fe640 1 -- 172.21.15.86:0/3303131864 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f276c044070 con 0x7f277c101b70 2024-04-16T05:34:41.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f2787235740 1 Processor -- start 2024-04-16T05:34:41.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f2787235740 1 -- start start 2024-04-16T05:34:41.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f2787235740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 0x5590e5ec6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f2787235740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec6750 0x5590e5ecb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f2787235740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ecb8c0 0x5590e5ecdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f2787235740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590e5dd9240 con 0x5590e5ec6750 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f2787235740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5590e5e369f0 con 0x5590e5ec5d60 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f2787235740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590e5df19e0 con 0x5590e5ecb8c0 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f27823e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec6750 0x5590e5ecb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.926+0000 7f27823e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec6750 0x5590e5ecb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60290/0 (socket says 172.21.15.86:60290) 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f27823e6640 1 -- 172.21.15.86:0/2467258663 learned_addr learned my addr 172.21.15.86:0/2467258663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f2781be5640 1 --2- 172.21.15.86:0/2467258663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ecb8c0 0x5590e5ecdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f2782be7640 1 --2- 172.21.15.86:0/2467258663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 0x5590e5ec6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f27737fe640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27680a3c30 0x7f27680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f2782be7640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27680a3c30 0x7f27680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f27823e6640 1 -- 172.21.15.86:0/2467258663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ecb8c0 msgr2=0x5590e5ecdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f27823e6640 1 --2- 172.21.15.86:0/2467258663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ecb8c0 0x5590e5ecdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f27823e6640 1 -- 172.21.15.86:0/2467258663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 msgr2=0x5590e5ec6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f27823e6640 1 --2- 172.21.15.86:0/2467258663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 0x5590e5ec6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f27823e6640 1 -- 172.21.15.86:0/2467258663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e5de4bc0 con 0x5590e5ec6750 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f27737fe640 1 -- 172.21.15.86:0/3303131864 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f276c0f2640 con 0x7f277c101b70 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f27823e6640 1 --2- 172.21.15.86:0/2467258663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec6750 0x5590e5ecb250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f27780097f0 tx=0x7f2778009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9405fa6bd918a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f2782be7640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27680a3c30 0x7f27680a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5590e5ec6fb0 tx=0x7f2774004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f2770bfe640 1 -- 172.21.15.86:0/2467258663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778015020 con 0x5590e5ec6750 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f2770bfe640 1 -- 172.21.15.86:0/2467258663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2778011420 con 0x5590e5ec6750 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.927+0000 7f2770bfe640 1 -- 172.21.15.86:0/2467258663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27780105e0 con 0x5590e5ec6750 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 -- 172.21.15.86:0/2467258663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec6750 msgr2=0x5590e5ecb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 --2- 172.21.15.86:0/2467258663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec6750 0x5590e5ecb250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f27780097f0 tx=0x7f2778009cb0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 -- 172.21.15.86:0/2467258663 shutdown_connections 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 --2- 172.21.15.86:0/2467258663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ecb8c0 0x5590e5ecdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 --2- 172.21.15.86:0/2467258663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec6750 0x5590e5ecb250 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 --2- 172.21.15.86:0/2467258663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 0x5590e5ec6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 -- 172.21.15.86:0/2467258663 >> 172.21.15.86:0/2467258663 conn(0x5590e5ec12d0 msgr2=0x5590e5ec16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 -- 172.21.15.86:0/2467258663 shutdown_connections 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 -- 172.21.15.86:0/2467258663 wait complete. 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.928+0000 7f2787235740 1 -- start start 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2787235740 1 Processor -- start 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2787235740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec5d60 0x5590e5ecff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2787235740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec6750 0x5590e5ed0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2787235740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5ed74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2787235740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590e5e300f0 con 0x5590e5ecb8c0 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2787235740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5590e5e2add0 con 0x5590e5ec6750 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2787235740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590e5e38cd0 con 0x5590e5ec5d60 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2787235740 10 client.0 _ll_register_callbacks cb 0x5590e5e21570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2782be7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5ed74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2782be7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5ed74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60306/0 (socket says 172.21.15.86:60306) 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2782be7640 1 -- 172.21.15.86:0/1514759053 learned_addr learned my addr 172.21.15.86:0/1514759053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2781be5640 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec6750 0x5590e5ed0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f27823e6640 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec5d60 0x5590e5ecff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2782be7640 1 -- 172.21.15.86:0/1514759053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec5d60 msgr2=0x5590e5ecff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2782be7640 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec5d60 0x5590e5ecff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2782be7640 1 -- 172.21.15.86:0/1514759053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec6750 msgr2=0x5590e5ed0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2782be7640 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec6750 0x5590e5ed0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2782be7640 1 -- 172.21.15.86:0/1514759053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e5b3e2d0 con 0x5590e5ecb8c0 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2781be5640 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec6750 0x5590e5ed0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2782be7640 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5ed74d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2774010f20 tx=0x7f2774004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55d8b95cb7be92e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2765ffb640 1 -- 172.21.15.86:0/1514759053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774024020 con 0x5590e5ecb8c0 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2765ffb640 1 -- 172.21.15.86:0/1514759053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f2774020410 con 0x5590e5ecb8c0 2024-04-16T05:34:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.929+0000 7f2787235740 1 -- 172.21.15.86:0/1514759053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2778011420 con 0x5590e5ecb8c0 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2765ffb640 1 -- 172.21.15.86:0/1514759053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f277402f050 con 0x5590e5ecb8c0 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2787235740 10 client.15216 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2787235740 1 -- 172.21.15.86:0/1514759053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5590e5c086b0 con 0x5590e5ecb8c0 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2765ffb640 1 -- 172.21.15.86:0/1514759053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2778011420 con 0x5590e5ecb8c0 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2765ffb640 1 -- 172.21.15.86:0/1514759053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f277400e6c0 con 0x5590e5ecb8c0 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2787235740 10 client.15216 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2787235740 10 client.15216 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2787235740 20 client.15216 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2787235740 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5590e5edcd30 0x5590e5ede1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2787235740 4 client.15216 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f2787235740 1 -- 172.21.15.86:0/1514759053 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590e5db24c0 con 0x5590e5edcd30 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.930+0000 7f27823e6640 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5590e5edcd30 0x5590e5ede1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.931+0000 7f2765ffb640 10 client.15216 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.931+0000 7f27823e6640 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5590e5edcd30 0x5590e5ede1a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eff4afee92c5aeb6 server_cookie=1e42ddf076e699b6 in_seq=0 out_seq=0 2024-04-16T05:34:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2765ffb640 1 -- 172.21.15.86:0/1514759053 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590e5db24c0 con 0x5590e5edcd30 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2765ffb640 10 client.15216 handle_command_reply: tid=0 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 client.15216 shutdown 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 -- 172.21.15.86:0/1514759053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 msgr2=0x5590e5ed74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5ed74d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2774010f20 tx=0x7f2774004a40 comp rx=0 tx=0).stop 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 -- 172.21.15.86:0/1514759053 shutdown_connections 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5590e5edcd30 0x5590e5ede1a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5ed74d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec6750 0x5590e5ed0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2765ffb640 0 client.15216 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 --2- 172.21.15.86:0/1514759053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec5d60 0x5590e5ecff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 -- 172.21.15.86:0/1514759053 >> 172.21.15.86:0/1514759053 conn(0x5590e5ec12d0 msgr2=0x5590e5ec0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 -- 172.21.15.86:0/1514759053 shutdown_connections 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 1 -- 172.21.15.86:0/1514759053 wait complete. 2024-04-16T05:34:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.932+0000 7f2787235740 20 client.15216 trim_cache size 0 max 16384 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f2787235740 1 Processor -- start 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f2787235740 1 -- start start 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f2787235740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec5d60 0x5590e5eceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f2787235740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec6750 0x5590e5ed7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f2787235740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ecb8c0 0x5590e5ec7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f2787235740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590e5edc4e0 con 0x5590e5ec5d60 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f2787235740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5590e5edcbb0 con 0x5590e5ecb8c0 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f2787235740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590e5edd280 con 0x5590e5ec6750 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f27823e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec5d60 0x5590e5eceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f27823e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec5d60 0x5590e5eceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60312/0 (socket says 172.21.15.86:60312) 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f27823e6640 1 -- 172.21.15.86:0/3819724382 learned_addr learned my addr 172.21.15.86:0/3819724382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f27823e6640 1 -- 172.21.15.86:0/3819724382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec6750 msgr2=0x5590e5ed7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f27823e6640 1 --2- 172.21.15.86:0/3819724382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec6750 0x5590e5ed7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f27823e6640 1 -- 172.21.15.86:0/3819724382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ecb8c0 msgr2=0x5590e5ec7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f27823e6640 1 --2- 172.21.15.86:0/3819724382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ecb8c0 0x5590e5ec7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f27823e6640 1 -- 172.21.15.86:0/3819724382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e5ec6fe0 con 0x5590e5ec5d60 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.950+0000 7f27823e6640 1 --2- 172.21.15.86:0/3819724382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec5d60 0x5590e5eceaf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x5590e5ec71c0 tx=0x7f2778009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19419eb8e130e809 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2767fff640 1 -- 172.21.15.86:0/3819724382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277802a070 con 0x5590e5ec5d60 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2767fff640 1 -- 172.21.15.86:0/3819724382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2778006ae0 con 0x5590e5ec5d60 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2767fff640 1 -- 172.21.15.86:0/3819724382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778015040 con 0x5590e5ec5d60 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 -- 172.21.15.86:0/3819724382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec5d60 msgr2=0x5590e5eceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 --2- 172.21.15.86:0/3819724382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec5d60 0x5590e5eceaf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x5590e5ec71c0 tx=0x7f2778009190 comp rx=0 tx=0).stop 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 -- 172.21.15.86:0/3819724382 shutdown_connections 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 --2- 172.21.15.86:0/3819724382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ecb8c0 0x5590e5ec7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 --2- 172.21.15.86:0/3819724382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec6750 0x5590e5ed7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 --2- 172.21.15.86:0/3819724382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ec5d60 0x5590e5eceaf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 -- 172.21.15.86:0/3819724382 >> 172.21.15.86:0/3819724382 conn(0x5590e5ec12d0 msgr2=0x5590e5ec16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 -- 172.21.15.86:0/3819724382 shutdown_connections 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 -- 172.21.15.86:0/3819724382 wait complete. 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.951+0000 7f2787235740 1 -- start start 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2787235740 1 Processor -- start 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2787235740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 0x5590e5ed53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2787235740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec6750 0x5590e5f3ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2787235740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5f41210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2787235740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590e5ede020 con 0x5590e5ecb8c0 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2787235740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5590e5edd950 con 0x5590e5ec5d60 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2787235740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590e5ede6f0 con 0x5590e5ec6750 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2787235740 10 client.0 _ll_register_callbacks cb 0x7f2774007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2782be7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5f41210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2782be7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5f41210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60322/0 (socket says 172.21.15.86:60322) 2024-04-16T05:34:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2782be7640 1 -- 172.21.15.86:0/2215708780 learned_addr learned my addr 172.21.15.86:0/2215708780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f27823e6640 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 0x5590e5ed53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2782be7640 1 -- 172.21.15.86:0/2215708780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec6750 msgr2=0x5590e5f3ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2782be7640 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec6750 0x5590e5f3ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2782be7640 1 -- 172.21.15.86:0/2215708780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 msgr2=0x5590e5ed53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2782be7640 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 0x5590e5ed53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2782be7640 1 -- 172.21.15.86:0/2215708780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e5f41750 con 0x5590e5ecb8c0 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2782be7640 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5f41210 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2774007ba0 tx=0x7f2774004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426462178f37abfe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2765ffb640 1 -- 172.21.15.86:0/2215708780 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774024040 con 0x5590e5ecb8c0 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2765ffb640 1 -- 172.21.15.86:0/2215708780 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f277400ed70 con 0x5590e5ecb8c0 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.952+0000 7f2787235740 1 -- 172.21.15.86:0/2215708780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2778006ae0 con 0x5590e5ecb8c0 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2765ffb640 1 -- 172.21.15.86:0/2215708780 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2774041050 con 0x5590e5ecb8c0 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2787235740 10 client.15222 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2787235740 1 -- 172.21.15.86:0/2215708780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5590e5f429a0 con 0x5590e5ecb8c0 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2765ffb640 1 -- 172.21.15.86:0/2215708780 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2778006ae0 con 0x5590e5ecb8c0 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2765ffb640 1 -- 172.21.15.86:0/2215708780 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2774032040 con 0x5590e5ecb8c0 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2787235740 10 client.15222 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2787235740 10 client.15222 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2787235740 20 client.15222 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2787235740 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5590e5f475e0 0x5590e5f462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2787235740 4 client.15222 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f2787235740 1 -- 172.21.15.86:0/2215708780 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "status"}) -- 0x5590e5e394e0 con 0x5590e5f475e0 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.953+0000 7f27823e6640 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5590e5f475e0 0x5590e5f462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2765ffb640 10 client.15222 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f27823e6640 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5590e5f475e0 0x5590e5f462f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8f4222da412da54 server_cookie=f9dc423def94b25a in_seq=0 out_seq=0 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2765ffb640 1 -- 172.21.15.86:0/2215708780 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5590e5e394e0 con 0x5590e5f475e0 2024-04-16T05:34:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2765ffb640 10 client.15222 handle_command_reply: tid=0 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 client.15222 shutdown 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 -- 172.21.15.86:0/2215708780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 msgr2=0x5590e5f41210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5f41210 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2774007ba0 tx=0x7f2774004a20 comp rx=0 tx=0).stop 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 -- 172.21.15.86:0/2215708780 shutdown_connections 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5590e5f475e0 0x5590e5f462f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e5ecb8c0 0x5590e5f41210 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590e5ec6750 0x5590e5f3ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2765ffb640 0 client.15222 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 --2- 172.21.15.86:0/2215708780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590e5ec5d60 0x5590e5ed53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 -- 172.21.15.86:0/2215708780 >> 172.21.15.86:0/2215708780 conn(0x5590e5ec12d0 msgr2=0x5590e5eceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 -- 172.21.15.86:0/2215708780 shutdown_connections 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 1 -- 172.21.15.86:0/2215708780 wait complete. 2024-04-16T05:34:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f2787235740 20 client.15222 trim_cache size 0 max 16384 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24445, 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 84.346589471000001, 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 92, 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 129.96050415299999 2024-04-16T05:34:41.960 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:34:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.954+0000 7f278464a640 1 -- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27680a3c30 msgr2=0x7f27680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27680a3c30 0x7f27680a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5590e5ec6fb0 tx=0x7f2774004ab0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 -- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c101b70 msgr2=0x7f277c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c101b70 0x7f277c198920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f276c02efd0 tx=0x7f276c0040c0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 -- 172.21.15.86:0/3303131864 shutdown_connections 2024-04-16T05:34:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f277c102560 0x7f277c19f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c101b70 0x7f277c198920 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27680a3c30 0x7f27680a60f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 --2- 172.21.15.86:0/3303131864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f277c1011b0 0x7f277c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 -- 172.21.15.86:0/3303131864 >> 172.21.15.86:0/3303131864 conn(0x7f277c0fbc40 msgr2=0x7f277c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 -- 172.21.15.86:0/3303131864 shutdown_connections 2024-04-16T05:34:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:41.955+0000 7f278464a640 1 -- 172.21.15.86:0/3303131864 wait complete. 2024-04-16T05:34:41.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-16T05:34:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:41 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3707607250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:41 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1572832973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:41 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1288710466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:42.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.070+0000 7fcffb4c7640 1 -- 172.21.15.86:0/1821503950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff41020e0 msgr2=0x7fcff4102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.070+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/1821503950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff41020e0 0x7fcff4102560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009900 tx=0x7fcfe802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.070+0000 7fcffb4c7640 1 -- 172.21.15.86:0/1821503950 shutdown_connections 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.070+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/1821503950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcff4102aa0 0x7fcff41093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.070+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/1821503950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff41020e0 0x7fcff4102560 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.070+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/1821503950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff4100f10 0x7fcff4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.070+0000 7fcffb4c7640 1 -- 172.21.15.86:0/1821503950 >> 172.21.15.86:0/1821503950 conn(0x7fcff40fc4d0 msgr2=0x7fcff40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 -- 172.21.15.86:0/1821503950 shutdown_connections 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 -- 172.21.15.86:0/1821503950 wait complete. 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 Processor -- start 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 -- start start 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff4100f10 0x7fcff406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcff41020e0 0x7fcff419d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff4102aa0 0x7fcff419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcff410ccb0 con 0x7fcff4100f10 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcff410cb30 con 0x7fcff41020e0 2024-04-16T05:34:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcffb4c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcff410ce30 con 0x7fcff4102aa0 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcff9263640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff4100f10 0x7fcff406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcff9263640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff4100f10 0x7fcff406b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60346/0 (socket says 172.21.15.86:60346) 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.071+0000 7fcff9263640 1 -- 172.21.15.86:0/3190395693 learned_addr learned my addr 172.21.15.86:0/3190395693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff4102aa0 0x7fcff419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcff41020e0 0x7fcff419d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcff9263640 1 -- 172.21.15.86:0/3190395693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff4102aa0 msgr2=0x7fcff419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcff9263640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff4102aa0 0x7fcff419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcff9263640 1 -- 172.21.15.86:0/3190395693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcff41020e0 msgr2=0x7fcff419d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcff9263640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcff41020e0 0x7fcff419d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcff9263640 1 -- 172.21.15.86:0/3190395693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff41a0040 con 0x7fcff4100f10 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcff41020e0 0x7fcff419d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcff9263640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff4100f10 0x7fcff406b1d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcff000b450 tx=0x7fcff000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcfe67fc640 1 -- 172.21.15.86:0/3190395693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0015020 con 0x7fcff4100f10 2024-04-16T05:34:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcfe67fc640 1 -- 172.21.15.86:0/3190395693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff0011d70 con 0x7fcff4100f10 2024-04-16T05:34:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcffb4c7640 1 -- 172.21.15.86:0/3190395693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff41a0330 con 0x7fcff4100f10 2024-04-16T05:34:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcfe67fc640 1 -- 172.21.15.86:0/3190395693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff000f340 con 0x7fcff4100f10 2024-04-16T05:34:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.072+0000 7fcffb4c7640 1 -- 172.21.15.86:0/3190395693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff41a0880 con 0x7fcff4100f10 2024-04-16T05:34:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.074+0000 7fcfe67fc640 1 -- 172.21.15.86:0/3190395693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcff0018020 con 0x7fcff4100f10 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcffe182740 1 Processor -- start 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcffe182740 1 -- start start 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcffe182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a49d60 0x564595a4a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcffe182740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 0x564595a4f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcffe182740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4f8c0 0x564595a51d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcffe182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56459595d240 con 0x564595a49d60 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcffe182740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5645959ba9f0 con 0x564595a4a750 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcffe182740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5645959759e0 con 0x564595a4f8c0 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcfe67fc640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfe00a3c30 0x7fcfe00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff9a64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a49d60 0x564595a4a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff9a64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a49d60 0x564595a4a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60358/0 (socket says 172.21.15.86:60358) 2024-04-16T05:34:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff9a64640 1 -- 172.21.15.86:0/3387147378 learned_addr learned my addr 172.21.15.86:0/3387147378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfe00a3c30 0x7fcfe00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcfe67fc640 1 -- 172.21.15.86:0/3190395693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcff00d0260 con 0x7fcff4100f10 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3387147378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4f8c0 0x564595a51d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff9a64640 1 -- 172.21.15.86:0/3387147378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4f8c0 msgr2=0x564595a51d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3387147378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4f8c0 0x564595a51d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff9a64640 1 -- 172.21.15.86:0/3387147378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 msgr2=0x564595a4f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3387147378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 0x564595a4f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff9a64640 1 -- 172.21.15.86:0/3387147378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564595968bc0 con 0x564595a49d60 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.075+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3387147378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4f8c0 0x564595a51d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.076+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3387147378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a49d60 0x564595a4a180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0149c0 tx=0x7fcfec014e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4692fa921cb67331 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.076+0000 7fcfdf7fe640 1 -- 172.21.15.86:0/3387147378 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec01ec50 con 0x564595a49d60 2024-04-16T05:34:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.076+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfe00a3c30 0x7fcfe00a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcfec022000 tx=0x7fcfec020000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.076+0000 7fcfdf7fe640 1 -- 172.21.15.86:0/3387147378 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfec023070 con 0x564595a49d60 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.076+0000 7fcfdf7fe640 1 -- 172.21.15.86:0/3387147378 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec013c40 con 0x564595a49d60 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.076+0000 7fcffe182740 1 -- 172.21.15.86:0/3387147378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a49d60 msgr2=0x564595a4a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 --2- 172.21.15.86:0/3387147378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a49d60 0x564595a4a180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0149c0 tx=0x7fcfec014e80 comp rx=0 tx=0).stop 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 -- 172.21.15.86:0/3387147378 shutdown_connections 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 --2- 172.21.15.86:0/3387147378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4f8c0 0x564595a51d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 --2- 172.21.15.86:0/3387147378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 0x564595a4f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 --2- 172.21.15.86:0/3387147378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a49d60 0x564595a4a180 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 -- 172.21.15.86:0/3387147378 >> 172.21.15.86:0/3387147378 conn(0x564595a452d0 msgr2=0x564595a456d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 -- 172.21.15.86:0/3387147378 shutdown_connections 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 -- 172.21.15.86:0/3387147378 wait complete. 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 -- start start 2024-04-16T05:34:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 Processor -- start 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a49d60 0x564595a54030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4a750 0x564595a54570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a5b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5645959b40f0 con 0x564595a4f8c0 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5645959aedd0 con 0x564595a49d60 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5645959bccd0 con 0x564595a4a750 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcffe182740 10 client.0 _ll_register_callbacks cb 0x5645959a5570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.077+0000 7fcff9a64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a5b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff9a64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a5b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60372/0 (socket says 172.21.15.86:60372) 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff9a64640 1 -- 172.21.15.86:0/3883698697 learned_addr learned my addr 172.21.15.86:0/3883698697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4a750 0x564595a54570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff9a64640 1 -- 172.21.15.86:0/3883698697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4a750 msgr2=0x564595a54570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4a750 0x564595a54570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff9a64640 1 -- 172.21.15.86:0/3883698697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a49d60 msgr2=0x564595a54030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a49d60 0x564595a54030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff9a64640 1 -- 172.21.15.86:0/3883698697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645956c22d0 con 0x564595a4f8c0 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4a750 0x564595a54570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a5b5b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcfec012760 tx=0x7fcfec009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de02bfab95042fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcfdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3883698697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec012070 con 0x564595a4f8c0 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3883698697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcfec01ec50 con 0x564595a4f8c0 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcffe182740 1 -- 172.21.15.86:0/3883698697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfec023070 con 0x564595a4f8c0 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3883698697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfec03a050 con 0x564595a4f8c0 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcffe182740 10 client.15234 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcffe182740 1 -- 172.21.15.86:0/3883698697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56459578c6b0 con 0x564595a4f8c0 2024-04-16T05:34:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.078+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3883698697 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfec023070 con 0x564595a4f8c0 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3883698697 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcfec00f040 con 0x564595a4f8c0 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcffe182740 10 client.15234 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcffe182740 10 client.15234 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcffe182740 20 client.15234 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcffe182740 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x564595a60930 0x564595a5fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcffe182740 4 client.15234 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcffe182740 1 -- 172.21.15.86:0/3883698697 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645959364c0 con 0x564595a60930 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcff9263640 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x564595a60930 0x564595a5fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcfdcff9640 10 client.15234 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.079+0000 7fcff9263640 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x564595a60930 0x564595a5fb90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26a56f6ad3d65f39 server_cookie=fcacdd017f137c40 in_seq=0 out_seq=0 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3883698697 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645959364c0 con 0x564595a60930 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcfdcff9640 10 client.15234 handle_command_reply: tid=0 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcffe182740 1 client.15234 shutdown 2024-04-16T05:34:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcffe182740 1 -- 172.21.15.86:0/3883698697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 msgr2=0x564595a5b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcffe182740 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a5b5b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcfec012760 tx=0x7fcfec009910 comp rx=0 tx=0).stop 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcffe182740 1 -- 172.21.15.86:0/3883698697 shutdown_connections 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcffe182740 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a5b5b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcffe182740 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x564595a60930 0x564595a5fb90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcffe182740 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a4a750 0x564595a54570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcffe182740 1 --2- 172.21.15.86:0/3883698697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a49d60 0x564595a54030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcfdcff9640 0 client.15234 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.080+0000 7fcffe182740 1 -- 172.21.15.86:0/3883698697 >> 172.21.15.86:0/3883698697 conn(0x564595a452d0 msgr2=0x564595a44410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.081+0000 7fcffe182740 1 -- 172.21.15.86:0/3883698697 shutdown_connections 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.081+0000 7fcffe182740 1 -- 172.21.15.86:0/3883698697 wait complete. 2024-04-16T05:34:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.081+0000 7fcffe182740 20 client.15234 trim_cache size 0 max 16384 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.098+0000 7fcffe182740 1 Processor -- start 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.098+0000 7fcffe182740 1 -- start start 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.098+0000 7fcffe182740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 0x564595a456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.098+0000 7fcffe182740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 0x564595a5b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcffe182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a43b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcffe182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564595a600e0 con 0x564595a4f8c0 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcffe182740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564595a607b0 con 0x564595a4a750 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcffe182740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564595a60e80 con 0x564595a49d60 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9a64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9a64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a43b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60378/0 (socket says 172.21.15.86:60378) 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9a64640 1 -- 172.21.15.86:0/3459300468 learned_addr learned my addr 172.21.15.86:0/3459300468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9263640 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 0x564595a456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 0x564595a5b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9a64640 1 -- 172.21.15.86:0/3459300468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 msgr2=0x564595a456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 0x564595a456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9a64640 1 -- 172.21.15.86:0/3459300468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 msgr2=0x564595a5b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 0x564595a5b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9a64640 1 -- 172.21.15.86:0/3459300468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564595a46040 con 0x564595a4f8c0 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9263640 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 0x564595a456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 0x564595a5b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a43b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcfec015010 tx=0x7fcfec0143d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47e0fafae7a5f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcfdeffd640 1 -- 172.21.15.86:0/3459300468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec012dd0 con 0x564595a4f8c0 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcfdeffd640 1 -- 172.21.15.86:0/3459300468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfec023030 con 0x564595a4f8c0 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.099+0000 7fcfdeffd640 1 -- 172.21.15.86:0/3459300468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec013c10 con 0x564595a4f8c0 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 -- 172.21.15.86:0/3459300468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 msgr2=0x564595a43b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a43b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcfec015010 tx=0x7fcfec0143d0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 -- 172.21.15.86:0/3459300468 shutdown_connections 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4f8c0 0x564595a43b80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4a750 0x564595a5b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 --2- 172.21.15.86:0/3459300468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 0x564595a456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 -- 172.21.15.86:0/3459300468 >> 172.21.15.86:0/3459300468 conn(0x564595a452d0 msgr2=0x564595a4b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 -- 172.21.15.86:0/3459300468 shutdown_connections 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 -- 172.21.15.86:0/3459300468 wait complete. 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 -- start start 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.100+0000 7fcffe182740 1 Processor -- start 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 0x564595a56a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4a750 0x564595a56f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4f8c0 0x564595a59f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564595a61c20 con 0x564595a4a750 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564595a61550 con 0x564595a4f8c0 2024-04-16T05:34:42.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564595a622f0 con 0x564595a49d60 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 10 client.0 _ll_register_callbacks cb 0x7fcfec0147c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff8a62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4a750 0x564595a56f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff8a62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4a750 0x564595a56f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60382/0 (socket says 172.21.15.86:60382) 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff8a62640 1 -- 172.21.15.86:0/3490316701 learned_addr learned my addr 172.21.15.86:0/3490316701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff9263640 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 0x564595a56a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff9a64640 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4f8c0 0x564595a59f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff8a62640 1 -- 172.21.15.86:0/3490316701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 msgr2=0x564595a56a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 0x564595a56a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff8a62640 1 -- 172.21.15.86:0/3490316701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4f8c0 msgr2=0x564595a59f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4f8c0 0x564595a59f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff8a62640 1 -- 172.21.15.86:0/3490316701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564595a5a570 con 0x564595a4a750 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcff8a62640 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4a750 0x564595a56f60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8000c00 tx=0x7fcfe8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c0547780d3e3bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcfdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3490316701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8044070 con 0x564595a4a750 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 1 -- 172.21.15.86:0/3490316701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfec023030 con 0x564595a4a750 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3490316701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcfe8005000 con 0x564595a4a750 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3490316701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfe8050050 con 0x564595a4a750 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 10 client.15240 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.101+0000 7fcffe182740 1 -- 172.21.15.86:0/3490316701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564595acf0f0 con 0x564595a4a750 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3490316701 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfec023030 con 0x564595a4a750 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3490316701 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcfe80035f0 con 0x564595a4a750 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcffe182740 10 client.15240 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcffe182740 10 client.15240 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcffe182740 20 client.15240 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcffe182740 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x564595ad3b80 0x564595ad2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcffe182740 4 client.15240 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcffe182740 1 -- 172.21.15.86:0/3490316701 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "status"}) -- 0x5645959bd4e0 con 0x564595ad3b80 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcff9263640 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x564595ad3b80 0x564595ad2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcfdcff9640 10 client.15240 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.102+0000 7fcff9263640 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x564595ad3b80 0x564595ad2890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b752200c04369a4 server_cookie=b1f54908f5497b55 in_seq=0 out_seq=0 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcfdcff9640 1 -- 172.21.15.86:0/3490316701 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5645959bd4e0 con 0x564595ad3b80 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcfdcff9640 10 client.15240 handle_command_reply: tid=0 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 client.15240 shutdown 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 -- 172.21.15.86:0/3490316701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4a750 msgr2=0x564595a56f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4a750 0x564595a56f60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8000c00 tx=0x7fcfe8004270 comp rx=0 tx=0).stop 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 -- 172.21.15.86:0/3490316701 shutdown_connections 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564595a4f8c0 0x564595a59f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564595a4a750 0x564595a56f60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x564595ad3b80 0x564595ad2890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 --2- 172.21.15.86:0/3490316701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564595a49d60 0x564595a56a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcfdcff9640 0 client.15240 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 -- 172.21.15.86:0/3490316701 >> 172.21.15.86:0/3490316701 conn(0x564595a452d0 msgr2=0x564595a456b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 -- 172.21.15.86:0/3490316701 shutdown_connections 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 1 -- 172.21.15.86:0/3490316701 wait complete. 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffe182740 20 client.15240 trim_cache size 0 max 16384 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24455, 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:34:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:34:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 85.503003023000005, 2024-04-16T05:34:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:34:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 92, 2024-04-16T05:34:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:34:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 124.271108807 2024-04-16T05:34:42.110 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 -- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfe00a3c30 msgr2=0x7fcfe00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfe00a3c30 0x7fcfe00a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcfec022000 tx=0x7fcfec020000 comp rx=0 tx=0).stop 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 -- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff4100f10 msgr2=0x7fcff406b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff4100f10 0x7fcff406b1d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcff000b450 tx=0x7fcff000b910 comp rx=0 tx=0).stop 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 -- 172.21.15.86:0/3190395693 shutdown_connections 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff4102aa0 0x7fcff419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfe00a3c30 0x7fcfe00a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcff41020e0 0x7fcff419d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 --2- 172.21.15.86:0/3190395693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff4100f10 0x7fcff406b1d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.103+0000 7fcffb4c7640 1 -- 172.21.15.86:0/3190395693 >> 172.21.15.86:0/3190395693 conn(0x7fcff40fc4d0 msgr2=0x7fcff4105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.104+0000 7fcffb4c7640 1 -- 172.21.15.86:0/3190395693 shutdown_connections 2024-04-16T05:34:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.104+0000 7fcffb4c7640 1 -- 172.21.15.86:0/3190395693 wait complete. 2024-04-16T05:34:42.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-16T05:34:42.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.222+0000 7f081f9e8640 1 -- 172.21.15.86:0/1619731383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818102090 msgr2=0x7f0818102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.222+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1619731383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818102090 0x7f0818102510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0808009900 tx=0x7f080802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:42.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.222+0000 7f081f9e8640 1 -- 172.21.15.86:0/1619731383 shutdown_connections 2024-04-16T05:34:42.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.222+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1619731383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0818102a50 0x7f0818109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.222+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1619731383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818102090 0x7f0818102510 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.222+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1619731383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0818100f40 0x7f0818101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.222+0000 7f081f9e8640 1 -- 172.21.15.86:0/1619731383 >> 172.21.15.86:0/1619731383 conn(0x7f08180fc4b0 msgr2=0x7f08180fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.222+0000 7f081f9e8640 1 -- 172.21.15.86:0/1619731383 shutdown_connections 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.222+0000 7f081f9e8640 1 -- 172.21.15.86:0/1619731383 wait complete. 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081f9e8640 1 Processor -- start 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081f9e8640 1 -- start start 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081f9e8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818100f40 0x7f081819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081f9e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0818102090 0x7f081819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081f9e8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0818102a50 0x7f08181a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081f9e8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f081810cbd0 con 0x7f0818100f40 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081f9e8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f081810ca50 con 0x7f0818102a50 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081f9e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f081810cd50 con 0x7f0818102090 2024-04-16T05:34:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081d784640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818100f40 0x7f081819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081d784640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818100f40 0x7f081819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60406/0 (socket says 172.21.15.86:60406) 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081d784640 1 -- 172.21.15.86:0/1443969050 learned_addr learned my addr 172.21.15.86:0/1443969050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.223+0000 7f081cf83640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0818102090 0x7f081819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081df85640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0818102a50 0x7f08181a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081d784640 1 -- 172.21.15.86:0/1443969050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0818102090 msgr2=0x7f081819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081d784640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0818102090 0x7f081819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081d784640 1 -- 172.21.15.86:0/1443969050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0818102a50 msgr2=0x7f08181a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081d784640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0818102a50 0x7f08181a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081d784640 1 -- 172.21.15.86:0/1443969050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181a2180 con 0x7f0818100f40 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081df85640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0818102a50 0x7f08181a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081d784640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818100f40 0x7f081819a560 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f081400c920 tx=0x7f081400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f080e7fc640 1 -- 172.21.15.86:0/1443969050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814007c70 con 0x7f0818100f40 2024-04-16T05:34:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f080e7fc640 1 -- 172.21.15.86:0/1443969050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0814007e10 con 0x7f0818100f40 2024-04-16T05:34:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081f9e8640 1 -- 172.21.15.86:0/1443969050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181a2470 con 0x7f0818100f40 2024-04-16T05:34:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f080e7fc640 1 -- 172.21.15.86:0/1443969050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814018890 con 0x7f0818100f40 2024-04-16T05:34:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.224+0000 7f081f9e8640 1 -- 172.21.15.86:0/1443969050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181a29e0 con 0x7f0818100f40 2024-04-16T05:34:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.226+0000 7f080e7fc640 1 -- 172.21.15.86:0/1443969050 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0814018a70 con 0x7f0818100f40 2024-04-16T05:34:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f08225cc740 1 Processor -- start 2024-04-16T05:34:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f08225cc740 1 -- start start 2024-04-16T05:34:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f08225cc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 0x55c0e07ec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f08225cc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f08225cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f08225cc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0e06ff240 con 0x55c0e07ec750 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f08225cc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c0e075c9f0 con 0x55c0e07ebd60 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f08225cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0e07179e0 con 0x55c0e07f18c0 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f081d784640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f080e7fc640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08040a3c30 0x7f08040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f081cf83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f081d784640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60414/0 (socket says 172.21.15.86:60414) 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f081d784640 1 -- 172.21.15.86:0/3794649607 learned_addr learned my addr 172.21.15.86:0/3794649607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f081df85640 1 --2- 172.21.15.86:0/3794649607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 0x55c0e07ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f081df85640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08040a3c30 0x7f08040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f081cf83640 1 -- 172.21.15.86:0/3794649607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 msgr2=0x55c0e07ec180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f081cf83640 1 --2- 172.21.15.86:0/3794649607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 0x55c0e07ec180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.227+0000 7f081cf83640 1 -- 172.21.15.86:0/3794649607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 msgr2=0x55c0e07f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.228+0000 7f080e7fc640 1 -- 172.21.15.86:0/1443969050 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f08140d0830 con 0x7f0818100f40 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.228+0000 7f081cf83640 1 --2- 172.21.15.86:0/3794649607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.228+0000 7f081cf83640 1 -- 172.21.15.86:0/3794649607 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e070abc0 con 0x55c0e07f18c0 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.228+0000 7f081d784640 1 --2- 172.21.15.86:0/3794649607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.228+0000 7f081df85640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08040a3c30 0x7f08040a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0810009270 tx=0x7f0810005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.228+0000 7f081cf83640 1 --2- 172.21.15.86:0/3794649607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07f3d50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f080802c170 tx=0x7f0808005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d02a3881c7224198 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.228+0000 7f08037fe640 1 -- 172.21.15.86:0/3794649607 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080802cc80 con 0x55c0e07f18c0 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.228+0000 7f08037fe640 1 -- 172.21.15.86:0/3794649607 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0808005040 con 0x55c0e07f18c0 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.228+0000 7f08037fe640 1 -- 172.21.15.86:0/3794649607 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808004ac0 con 0x55c0e07f18c0 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 -- 172.21.15.86:0/3794649607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 msgr2=0x55c0e07f3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 --2- 172.21.15.86:0/3794649607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07f3d50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f080802c170 tx=0x7f0808005dd0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 -- 172.21.15.86:0/3794649607 shutdown_connections 2024-04-16T05:34:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 --2- 172.21.15.86:0/3794649607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07f3d50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 --2- 172.21.15.86:0/3794649607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 --2- 172.21.15.86:0/3794649607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 0x55c0e07ec180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 -- 172.21.15.86:0/3794649607 >> 172.21.15.86:0/3794649607 conn(0x55c0e07e72d0 msgr2=0x55c0e07e76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 -- 172.21.15.86:0/3794649607 shutdown_connections 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 -- 172.21.15.86:0/3794649607 wait complete. 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 -- start start 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.229+0000 7f08225cc740 1 Processor -- start 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f08225cc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f08225cc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ec750 0x55c0e07f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f08225cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f08225cc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0e07560f0 con 0x55c0e07ebd60 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f08225cc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0750dd0 con 0x55c0e07ec750 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f08225cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0e075ecd0 con 0x55c0e07f18c0 2024-04-16T05:34:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f08225cc740 10 client.0 _ll_register_callbacks cb 0x55c0e0747570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081d784640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081d784640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07f5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60424/0 (socket says 172.21.15.86:60424) 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081d784640 1 -- 172.21.15.86:0/1819406882 learned_addr learned my addr 172.21.15.86:0/1819406882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081cf83640 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ec750 0x55c0e07f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081df85640 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081d784640 1 -- 172.21.15.86:0/1819406882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 msgr2=0x55c0e07fd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081d784640 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07fd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081d784640 1 -- 172.21.15.86:0/1819406882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ec750 msgr2=0x55c0e07f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081d784640 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ec750 0x55c0e07f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081d784640 1 -- 172.21.15.86:0/1819406882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e04642d0 con 0x55c0e07ebd60 2024-04-16T05:34:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081cf83640 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ec750 0x55c0e07f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f081d784640 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07f5f40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0814007590 tx=0x7f081409cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f37fe8b0d84422 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f0800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f0800ff9640 1 -- 172.21.15.86:0/1819406882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140a5420 con 0x55c0e07ebd60 2024-04-16T05:34:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f0800ff9640 1 -- 172.21.15.86:0/1819406882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f081400fb30 con 0x55c0e07ebd60 2024-04-16T05:34:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f08225cc740 1 -- 172.21.15.86:0/1819406882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0808005040 con 0x55c0e07ebd60 2024-04-16T05:34:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.230+0000 7f0800ff9640 1 -- 172.21.15.86:0/1819406882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0814087070 con 0x55c0e07ebd60 2024-04-16T05:34:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f08225cc740 10 client.15252 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f08225cc740 1 -- 172.21.15.86:0/1819406882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c0e052e6b0 con 0x55c0e07ebd60 2024-04-16T05:34:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f0800ff9640 1 -- 172.21.15.86:0/1819406882 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0808005040 con 0x55c0e07ebd60 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f0800ff9640 1 -- 172.21.15.86:0/1819406882 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08140d0ce0 con 0x55c0e07ebd60 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f08225cc740 10 client.15252 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f08225cc740 10 client.15252 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f08225cc740 20 client.15252 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f08225cc740 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c0e0802c50 0x55c0e08040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f08225cc740 4 client.15252 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f08225cc740 1 -- 172.21.15.86:0/1819406882 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0e06d84c0 con 0x55c0e0802c50 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.231+0000 7f081cf83640 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c0e0802c50 0x55c0e08040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.232+0000 7f081cf83640 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c0e0802c50 0x55c0e08040c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=778ffe483ec05d58 server_cookie=f0e8113de3eb7f2b in_seq=0 out_seq=0 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.232+0000 7f0800ff9640 10 client.15252 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:34:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.232+0000 7f0800ff9640 1 -- 172.21.15.86:0/1819406882 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0e06d84c0 con 0x55c0e0802c50 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.232+0000 7f0800ff9640 10 client.15252 handle_command_reply: tid=0 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.232+0000 7f08225cc740 1 client.15252 shutdown 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 -- 172.21.15.86:0/1819406882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 msgr2=0x55c0e07f5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07f5f40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0814007590 tx=0x7f081409cf70 comp rx=0 tx=0).stop 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 -- 172.21.15.86:0/1819406882 shutdown_connections 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ec750 0x55c0e07f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c0e0802c50 0x55c0e08040c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 --2- 172.21.15.86:0/1819406882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07f5f40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 -- 172.21.15.86:0/1819406882 >> 172.21.15.86:0/1819406882 conn(0x55c0e07e72d0 msgr2=0x55c0e07e6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f0800ff9640 0 client.15252 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 -- 172.21.15.86:0/1819406882 shutdown_connections 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 1 -- 172.21.15.86:0/1819406882 wait complete. 2024-04-16T05:34:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.233+0000 7f08225cc740 20 client.15252 trim_cache size 0 max 16384 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f08225cc740 1 Processor -- start 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f08225cc740 1 -- start start 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f08225cc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f08225cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07ec750 0x55c0e07ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f08225cc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07f18c0 0x55c0e07fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f08225cc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0802400 con 0x55c0e07ebd60 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f08225cc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0802ad0 con 0x55c0e07f18c0 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f08225cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0e08031a0 con 0x55c0e07ec750 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f081d784640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f081d784640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07edcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60428/0 (socket says 172.21.15.86:60428) 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.251+0000 7f081d784640 1 -- 172.21.15.86:0/3696716110 learned_addr learned my addr 172.21.15.86:0/3696716110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081df85640 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07f18c0 0x55c0e07fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081cf83640 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07ec750 0x55c0e07ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081d784640 1 -- 172.21.15.86:0/3696716110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07ec750 msgr2=0x55c0e07ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081d784640 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07ec750 0x55c0e07ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081d784640 1 -- 172.21.15.86:0/3696716110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07f18c0 msgr2=0x55c0e07fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081d784640 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07f18c0 0x55c0e07fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081d784640 1 -- 172.21.15.86:0/3696716110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e0802e80 con 0x55c0e07ebd60 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081cf83640 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07ec750 0x55c0e07ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081df85640 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07f18c0 0x55c0e07fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f081d784640 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07edcb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f08140ce340 tx=0x7f081400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9662cc6d072d444f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f0802ffd640 1 -- 172.21.15.86:0/3696716110 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140a5420 con 0x55c0e07ebd60 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f0802ffd640 1 -- 172.21.15.86:0/3696716110 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0814018d60 con 0x55c0e07ebd60 2024-04-16T05:34:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.252+0000 7f0802ffd640 1 -- 172.21.15.86:0/3696716110 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081409dd40 con 0x55c0e07ebd60 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 -- 172.21.15.86:0/3696716110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 msgr2=0x55c0e07edcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07edcb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f08140ce340 tx=0x7f081400ff30 comp rx=0 tx=0).stop 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 -- 172.21.15.86:0/3696716110 shutdown_connections 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07f18c0 0x55c0e07fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07ec750 0x55c0e07ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 --2- 172.21.15.86:0/3696716110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ebd60 0x55c0e07edcb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 -- 172.21.15.86:0/3696716110 >> 172.21.15.86:0/3696716110 conn(0x55c0e07e72d0 msgr2=0x55c0e0801ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 -- 172.21.15.86:0/3696716110 shutdown_connections 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 -- 172.21.15.86:0/3696716110 wait complete. 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 -- start start 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.253+0000 7f08225cc740 1 Processor -- start 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f08225cc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 0x55c0e07f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f08225cc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f08225cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f08225cc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0803f40 con 0x55c0e07ec750 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f08225cc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0803870 con 0x55c0e07ebd60 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f08225cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0804610 con 0x55c0e07f18c0 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f08225cc740 10 client.0 _ll_register_callbacks cb 0x7f081400a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081cf83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081cf83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60444/0 (socket says 172.21.15.86:60444) 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081cf83640 1 -- 172.21.15.86:0/628903539 learned_addr learned my addr 172.21.15.86:0/628903539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081df85640 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081d784640 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 0x55c0e07f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081cf83640 1 -- 172.21.15.86:0/628903539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 msgr2=0x55c0e07fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081cf83640 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07fc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081cf83640 1 -- 172.21.15.86:0/628903539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 msgr2=0x55c0e07f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081cf83640 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 0x55c0e07f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081cf83640 1 -- 172.21.15.86:0/628903539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e07fc660 con 0x55c0e07ec750 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081d784640 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 0x55c0e07f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f081cf83640 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f9190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f080802f020 tx=0x7f0808005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab15bb769338906f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f0800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f0800ff9640 1 -- 172.21.15.86:0/628903539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808004130 con 0x55c0e07ec750 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f0800ff9640 1 -- 172.21.15.86:0/628903539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f08080042d0 con 0x55c0e07ec750 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.254+0000 7f08225cc740 1 -- 172.21.15.86:0/628903539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0814018d60 con 0x55c0e07ec750 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f0800ff9640 1 -- 172.21.15.86:0/628903539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f080805c050 con 0x55c0e07ec750 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f08225cc740 10 client.15258 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f08225cc740 1 -- 172.21.15.86:0/628903539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c0e0868c00 con 0x55c0e07ec750 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f0800ff9640 1 -- 172.21.15.86:0/628903539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0814018d60 con 0x55c0e07ec750 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f0800ff9640 1 -- 172.21.15.86:0/628903539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0808005400 con 0x55c0e07ec750 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f08225cc740 10 client.15258 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f08225cc740 10 client.15258 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f08225cc740 20 client.15258 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f08225cc740 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c0e086d330 0x55c0e0869e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f08225cc740 4 client.15258 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f08225cc740 1 -- 172.21.15.86:0/628903539 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "status"}) -- 0x55c0e075f4e0 con 0x55c0e086d330 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.255+0000 7f081d784640 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c0e086d330 0x55c0e0869e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f0800ff9640 10 client.15258 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f081d784640 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c0e086d330 0x55c0e0869e40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91f7a2a4e9d1d3f7 server_cookie=bd983a3eb3d1e2e1 in_seq=0 out_seq=0 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f0800ff9640 1 -- 172.21.15.86:0/628903539 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c0e075f4e0 con 0x55c0e086d330 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f0800ff9640 10 client.15258 handle_command_reply: tid=0 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 client.15258 shutdown 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 -- 172.21.15.86:0/628903539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 msgr2=0x55c0e07f9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f9190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f080802f020 tx=0x7f0808005a20 comp rx=0 tx=0).stop 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 -- 172.21.15.86:0/628903539 shutdown_connections 2024-04-16T05:34:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e07f18c0 0x55c0e07fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c0e086d330 0x55c0e0869e40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0e07ec750 0x55c0e07f9190 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 --2- 172.21.15.86:0/628903539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0e07ebd60 0x55c0e07f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f0800ff9640 0 client.15258 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:34:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 -- 172.21.15.86:0/628903539 >> 172.21.15.86:0/628903539 conn(0x55c0e07e72d0 msgr2=0x55c0e07f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 -- 172.21.15.86:0/628903539 shutdown_connections 2024-04-16T05:34:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 1 -- 172.21.15.86:0/628903539 wait complete. 2024-04-16T05:34:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.256+0000 7f08225cc740 20 client.15258 trim_cache size 0 max 16384 2024-04-16T05:34:42.262 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24463, 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 86.664160090999999, 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 92, 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 127.407788638 2024-04-16T05:34:42.263 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:34:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 -- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08040a3c30 msgr2=0x7f08040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08040a3c30 0x7f08040a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0810009270 tx=0x7f0810005bd0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 -- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818100f40 msgr2=0x7f081819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818100f40 0x7f081819a560 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f081400c920 tx=0x7f081400cde0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 -- 172.21.15.86:0/1443969050 shutdown_connections 2024-04-16T05:34:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f08040a3c30 0x7f08040a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0818102a50 0x7f08181a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0818102090 0x7f081819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 --2- 172.21.15.86:0/1443969050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0818100f40 0x7f081819a560 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 -- 172.21.15.86:0/1443969050 >> 172.21.15.86:0/1443969050 conn(0x7f08180fc4b0 msgr2=0x7f08180fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 -- 172.21.15.86:0/1443969050 shutdown_connections 2024-04-16T05:34:42.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.257+0000 7f081f9e8640 1 -- 172.21.15.86:0/1443969050 wait complete. 2024-04-16T05:34:42.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-16T05:34:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.376+0000 7f61fbfba640 1 -- 172.21.15.86:0/1368725943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f41053f0 msgr2=0x7f61f41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.376+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1368725943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f41053f0 0x7f61f41077e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009900 tx=0x7f61e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.376+0000 7f61fbfba640 1 -- 172.21.15.86:0/1368725943 shutdown_connections 2024-04-16T05:34:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.376+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1368725943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f4107d20 0x7f61f410a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.376+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1368725943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f41053f0 0x7f61f41077e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.376+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1368725943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f4102ac0 0x7f61f4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.376+0000 7f61fbfba640 1 -- 172.21.15.86:0/1368725943 >> 172.21.15.86:0/1368725943 conn(0x7f61f40fc4b0 msgr2=0x7f61f40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.376+0000 7f61fbfba640 1 -- 172.21.15.86:0/1368725943 shutdown_connections 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.376+0000 7f61fbfba640 1 -- 172.21.15.86:0/1368725943 wait complete. 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fbfba640 1 Processor -- start 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fbfba640 1 -- start start 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fbfba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f4102ac0 0x7f61f4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fbfba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f41053f0 0x7f61f4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fbfba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f4107d20 0x7f61f419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fbfba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61f410dcd0 con 0x7f61f4107d20 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fbfba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f61f410db50 con 0x7f61f41053f0 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fbfba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61f410de50 con 0x7f61f4102ac0 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fa557640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f4107d20 0x7f61f419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fa557640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f4107d20 0x7f61f419f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60464/0 (socket says 172.21.15.86:60464) 2024-04-16T05:34:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fa557640 1 -- 172.21.15.86:0/1242837207 learned_addr learned my addr 172.21.15.86:0/1242837207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61f9d56640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f4102ac0 0x7f61f4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61f9555640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f41053f0 0x7f61f4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fa557640 1 -- 172.21.15.86:0/1242837207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f4102ac0 msgr2=0x7f61f4198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fa557640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f4102ac0 0x7f61f4198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fa557640 1 -- 172.21.15.86:0/1242837207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f41053f0 msgr2=0x7f61f4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fa557640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f41053f0 0x7f61f4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61fa557640 1 -- 172.21.15.86:0/1242837207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f41a0040 con 0x7f61f4107d20 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.377+0000 7f61f9555640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f41053f0 0x7f61f4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.378+0000 7f61fa557640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f4107d20 0x7f61f419f9d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009fd0 tx=0x7f61ec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.378+0000 7f61eaffd640 1 -- 172.21.15.86:0/1242837207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec017070 con 0x7f61f4107d20 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.378+0000 7f61eaffd640 1 -- 172.21.15.86:0/1242837207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61ec003e50 con 0x7f61f4107d20 2024-04-16T05:34:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.378+0000 7f61fbfba640 1 -- 172.21.15.86:0/1242837207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f41a0330 con 0x7f61f4107d20 2024-04-16T05:34:42.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.378+0000 7f61eaffd640 1 -- 172.21.15.86:0/1242837207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec0106c0 con 0x7f61f4107d20 2024-04-16T05:34:42.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.378+0000 7f61fbfba640 1 -- 172.21.15.86:0/1242837207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f41a0830 con 0x7f61f4107d20 2024-04-16T05:34:42.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.380+0000 7f61feba0740 1 Processor -- start 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.381+0000 7f61eaffd640 1 -- 172.21.15.86:0/1242837207 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f61ec003970 con 0x7f61f4107d20 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.381+0000 7f61feba0740 1 -- start start 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.381+0000 7f61feba0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x56081391c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.381+0000 7f61feba0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x560813921250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.381+0000 7f61feba0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 0x560813923d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.381+0000 7f61feba0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56081382f240 con 0x56081391bd60 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.381+0000 7f61feba0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56081388c9f0 con 0x5608139218c0 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.381+0000 7f61feba0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5608138479e0 con 0x56081391c750 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9555640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x56081391c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9555640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x56081391c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60468/0 (socket says 172.21.15.86:60468) 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9555640 1 -- 172.21.15.86:0/1811981068 learned_addr learned my addr 172.21.15.86:0/1811981068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9d56640 1 --2- 172.21.15.86:0/1811981068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x560813921250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61eaffd640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f61e00a3bb0 0x7f61e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9555640 1 -- 172.21.15.86:0/1811981068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 msgr2=0x560813921250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9555640 1 --2- 172.21.15.86:0/1811981068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x560813921250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9555640 1 -- 172.21.15.86:0/1811981068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 msgr2=0x560813923d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9555640 1 --2- 172.21.15.86:0/1811981068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 0x560813923d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9555640 1 -- 172.21.15.86:0/1811981068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56081383abc0 con 0x56081391bd60 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61fa557640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f61e00a3bb0 0x7f61e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61eaffd640 1 -- 172.21.15.86:0/1242837207 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f61ec0d04f0 con 0x7f61f4107d20 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61f9555640 1 --2- 172.21.15.86:0/1811981068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x56081391c180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f61e4005ab0 tx=0x7f61e40095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d1126620dc39bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61dffff640 1 -- 172.21.15.86:0/1811981068 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4043020 con 0x56081391bd60 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61dffff640 1 -- 172.21.15.86:0/1811981068 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e402cb80 con 0x56081391bd60 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.382+0000 7f61dffff640 1 -- 172.21.15.86:0/1811981068 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40046c0 con 0x56081391bd60 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61feba0740 1 -- 172.21.15.86:0/1811981068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 msgr2=0x56081391c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61feba0740 1 --2- 172.21.15.86:0/1811981068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x56081391c180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f61e4005ab0 tx=0x7f61e40095f0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61fa557640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f61e00a3bb0 0x7f61e00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f61ec09c400 tx=0x7f61ec0152b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61feba0740 1 -- 172.21.15.86:0/1811981068 shutdown_connections 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61feba0740 1 --2- 172.21.15.86:0/1811981068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 0x560813923d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61feba0740 1 --2- 172.21.15.86:0/1811981068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x560813921250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61feba0740 1 --2- 172.21.15.86:0/1811981068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x56081391c180 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61feba0740 1 -- 172.21.15.86:0/1811981068 >> 172.21.15.86:0/1811981068 conn(0x5608139172d0 msgr2=0x5608139176d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61feba0740 1 -- 172.21.15.86:0/1811981068 shutdown_connections 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.383+0000 7f61feba0740 1 -- 172.21.15.86:0/1811981068 wait complete. 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61feba0740 1 -- start start 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61feba0740 1 Processor -- start 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61feba0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x560813925f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61feba0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x5608139264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61feba0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 0x56081392d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61feba0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5608138860f0 con 0x56081391bd60 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61feba0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560813880dd0 con 0x5608139218c0 2024-04-16T05:34:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61feba0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56081388ecd0 con 0x56081391c750 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61feba0740 10 client.0 _ll_register_callbacks cb 0x560813877570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61f9555640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x560813925f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61f9555640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x560813925f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60482/0 (socket says 172.21.15.86:60482) 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61f9555640 1 -- 172.21.15.86:0/3845801432 learned_addr learned my addr 172.21.15.86:0/3845801432 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61f9d56640 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x5608139264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61f9555640 1 -- 172.21.15.86:0/3845801432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 msgr2=0x5608139264b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61f9555640 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x5608139264b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61f9555640 1 -- 172.21.15.86:0/3845801432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 msgr2=0x56081392d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61f9555640 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 0x56081392d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.384+0000 7f61f9555640 1 -- 172.21.15.86:0/3845801432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608135942d0 con 0x56081391bd60 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61f9555640 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x560813925f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f61e4005910 tx=0x7f61e402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79736527285bb73 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3845801432 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e404f060 con 0x56081391bd60 2024-04-16T05:34:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3845801432 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f61e40037f0 con 0x56081391bd60 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61feba0740 1 -- 172.21.15.86:0/3845801432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e402cb80 con 0x56081391bd60 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3845801432 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61e4056050 con 0x56081391bd60 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61feba0740 10 client.15270 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61feba0740 1 -- 172.21.15.86:0/3845801432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56081365e6b0 con 0x56081391bd60 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3845801432 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e402cb80 con 0x56081391bd60 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3845801432 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f61e4049070 con 0x56081391bd60 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61feba0740 10 client.15270 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61feba0740 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61feba0740 20 client.15270 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.385+0000 7f61feba0740 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5608139319e0 0x560813933b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.386+0000 7f61feba0740 4 client.15270 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.386+0000 7f61feba0740 1 -- 172.21.15.86:0/3845801432 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608138084c0 con 0x5608139319e0 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.386+0000 7f61f9d56640 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5608139319e0 0x560813933b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.386+0000 7f61f9d56640 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5608139319e0 0x560813933b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9539bbde2dd58e6e server_cookie=5ff499bce76190a8 in_seq=0 out_seq=0 2024-04-16T05:34:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.386+0000 7f61dd7fa640 10 client.15270 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3845801432 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608138084c0 con 0x5608139319e0 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61dd7fa640 10 client.15270 handle_command_reply: tid=0 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 client.15270 shutdown 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 -- 172.21.15.86:0/3845801432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 msgr2=0x560813925f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x560813925f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f61e4005910 tx=0x7f61e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 -- 172.21.15.86:0/3845801432 shutdown_connections 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5608139319e0 0x560813933b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 0x56081392d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x5608139264b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61dd7fa640 0 client.15270 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 --2- 172.21.15.86:0/3845801432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x560813925f70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 -- 172.21.15.86:0/3845801432 >> 172.21.15.86:0/3845801432 conn(0x5608139172d0 msgr2=0x560813916410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 -- 172.21.15.86:0/3845801432 shutdown_connections 2024-04-16T05:34:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 1 -- 172.21.15.86:0/3845801432 wait complete. 2024-04-16T05:34:42.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.387+0000 7f61feba0740 20 client.15270 trim_cache size 0 max 16384 2024-04-16T05:34:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61feba0740 1 Processor -- start 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61feba0740 1 -- start start 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61feba0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x5608139176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61feba0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x560813924a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61feba0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 0x560813924f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61feba0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560813931ea0 con 0x56081391bd60 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61feba0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560813932570 con 0x5608139218c0 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61feba0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560813932c40 con 0x56081391c750 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9555640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x5608139176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9555640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x5608139176b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60492/0 (socket says 172.21.15.86:60492) 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9555640 1 -- 172.21.15.86:0/2222205372 learned_addr learned my addr 172.21.15.86:0/2222205372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9d56640 1 --2- 172.21.15.86:0/2222205372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x560813924a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9555640 1 -- 172.21.15.86:0/2222205372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 msgr2=0x560813924a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9555640 1 --2- 172.21.15.86:0/2222205372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x560813924a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9555640 1 -- 172.21.15.86:0/2222205372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 msgr2=0x560813924f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9555640 1 --2- 172.21.15.86:0/2222205372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 0x560813924f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9555640 1 -- 172.21.15.86:0/2222205372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560813926f60 con 0x56081391bd60 2024-04-16T05:34:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9d56640 1 --2- 172.21.15.86:0/2222205372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x560813924a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.406+0000 7f61f9555640 1 --2- 172.21.15.86:0/2222205372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x5608139176b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f61e4000c00 tx=0x7f61e4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eafbae62b721663 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61df7fe640 1 -- 172.21.15.86:0/2222205372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e404f040 con 0x56081391bd60 2024-04-16T05:34:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61df7fe640 1 -- 172.21.15.86:0/2222205372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e4034d40 con 0x56081391bd60 2024-04-16T05:34:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61df7fe640 1 -- 172.21.15.86:0/2222205372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4043030 con 0x56081391bd60 2024-04-16T05:34:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61feba0740 1 -- 172.21.15.86:0/2222205372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 msgr2=0x5608139176b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61feba0740 1 --2- 172.21.15.86:0/2222205372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x5608139176b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f61e4000c00 tx=0x7f61e4005a20 comp rx=0 tx=0).stop 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61feba0740 1 -- 172.21.15.86:0/2222205372 shutdown_connections 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61feba0740 1 --2- 172.21.15.86:0/2222205372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5608139218c0 0x560813924f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61feba0740 1 --2- 172.21.15.86:0/2222205372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56081391c750 0x560813924a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61feba0740 1 --2- 172.21.15.86:0/2222205372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391bd60 0x5608139176b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61feba0740 1 -- 172.21.15.86:0/2222205372 >> 172.21.15.86:0/2222205372 conn(0x5608139172d0 msgr2=0x56081391d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61feba0740 1 -- 172.21.15.86:0/2222205372 shutdown_connections 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.407+0000 7f61feba0740 1 -- 172.21.15.86:0/2222205372 wait complete. 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61feba0740 1 -- start start 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61feba0740 1 Processor -- start 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61feba0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56081391bd60 0x560813928680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61feba0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391c750 0x560813928bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61feba0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5608139218c0 0x56081392b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61feba0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5608139339e0 con 0x56081391c750 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61feba0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560813933310 con 0x56081391bd60 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61feba0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5608139340b0 con 0x5608139218c0 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61feba0740 10 client.0 _ll_register_callbacks cb 0x7f61f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61f9d56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391c750 0x560813928bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61f9d56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391c750 0x560813928bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60504/0 (socket says 172.21.15.86:60504) 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61f9d56640 1 -- 172.21.15.86:0/3922975258 learned_addr learned my addr 172.21.15.86:0/3922975258 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61f9555640 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5608139218c0 0x56081392b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.408+0000 7f61f9555640 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56081391bd60 0x560813928680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61f9d56640 1 -- 172.21.15.86:0/3922975258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5608139218c0 msgr2=0x56081392b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61f9d56640 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5608139218c0 0x56081392b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61f9d56640 1 -- 172.21.15.86:0/3922975258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56081391bd60 msgr2=0x560813928680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61f9d56640 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56081391bd60 0x560813928680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61f9d56640 1 -- 172.21.15.86:0/3922975258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56081392bdd0 con 0x56081391c750 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61f9d56640 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391c750 0x560813928bc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x560813918240 tx=0x7f61f0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf11db685e72b6e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3922975258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f001e3c0 con 0x56081391c750 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3922975258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f61f001e560 con 0x56081391c750 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61feba0740 1 -- 172.21.15.86:0/3922975258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e4034d40 con 0x56081391c750 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3922975258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61f002c050 con 0x56081391c750 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61feba0740 10 client.15276 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61feba0740 1 -- 172.21.15.86:0/3922975258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560813998890 con 0x56081391c750 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3922975258 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e4034d40 con 0x56081391c750 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3922975258 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f61f00048c0 con 0x56081391c750 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.409+0000 7f61feba0740 10 client.15276 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61feba0740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61feba0740 20 client.15276 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61feba0740 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56081399d510 0x56081399c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61feba0740 4 client.15276 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61feba0740 1 -- 172.21.15.86:0/3922975258 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "status"}) -- 0x56081388f4e0 con 0x56081399d510 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61f9555640 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56081399d510 0x56081399c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61dd7fa640 10 client.15276 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61f9555640 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56081399d510 0x56081399c220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e92bf266e3c28816 server_cookie=3daccb7c40144a0d in_seq=0 out_seq=0 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61dd7fa640 1 -- 172.21.15.86:0/3922975258 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56081388f4e0 con 0x56081399d510 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61dd7fa640 10 client.15276 handle_command_reply: tid=0 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.410+0000 7f61feba0740 1 client.15276 shutdown 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 -- 172.21.15.86:0/3922975258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391c750 msgr2=0x560813928bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391c750 0x560813928bc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x560813918240 tx=0x7f61f0004410 comp rx=0 tx=0).stop 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 -- 172.21.15.86:0/3922975258 shutdown_connections 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56081399d510 0x56081399c220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5608139218c0 0x56081392b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56081391c750 0x560813928bc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61dd7fa640 0 client.15276 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 --2- 172.21.15.86:0/3922975258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56081391bd60 0x560813928680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 -- 172.21.15.86:0/3922975258 >> 172.21.15.86:0/3922975258 conn(0x5608139172d0 msgr2=0x5608139176b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 -- 172.21.15.86:0/3922975258 shutdown_connections 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 1 -- 172.21.15.86:0/3922975258 wait complete. 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.411+0000 7f61feba0740 20 client.15276 trim_cache size 0 max 16384 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-16T05:34:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24481, 2024-04-16T05:34:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:34:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:34:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:34:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 87.832235761999996, 2024-04-16T05:34:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:34:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 92, 2024-04-16T05:34:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:34:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 121.831844465 2024-04-16T05:34:42.417 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:34:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 -- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f61e00a3bb0 msgr2=0x7f61e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f61e00a3bb0 0x7f61e00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f61ec09c400 tx=0x7f61ec0152b0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 -- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f4107d20 msgr2=0x7f61f419f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f4107d20 0x7f61f419f9d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009fd0 tx=0x7f61ec00b260 comp rx=0 tx=0).stop 2024-04-16T05:34:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 -- 172.21.15.86:0/1242837207 shutdown_connections 2024-04-16T05:34:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61f4107d20 0x7f61f419f9d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f61e00a3bb0 0x7f61e00a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f41053f0 0x7f61f4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 --2- 172.21.15.86:0/1242837207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f4102ac0 0x7f61f4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 -- 172.21.15.86:0/1242837207 >> 172.21.15.86:0/1242837207 conn(0x7f61f40fc4b0 msgr2=0x7f61f4103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 -- 172.21.15.86:0/1242837207 shutdown_connections 2024-04-16T05:34:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.412+0000 7f61fbfba640 1 -- 172.21.15.86:0/1242837207 wait complete. 2024-04-16T05:34:42.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T05:34:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 -- 172.21.15.86:0/1676496602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 msgr2=0x7f7b8010a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 --2- 172.21.15.86:0/1676496602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 0x7f7b8010a190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009ca0 tx=0x7f7b7802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 -- 172.21.15.86:0/1676496602 shutdown_connections 2024-04-16T05:34:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 --2- 172.21.15.86:0/1676496602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 0x7f7b8010a190 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 --2- 172.21.15.86:0/1676496602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b80105470 0x7f7b80107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 --2- 172.21.15.86:0/1676496602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b80102b40 0x7f7b80104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 -- 172.21.15.86:0/1676496602 >> 172.21.15.86:0/1676496602 conn(0x7f7b800fc4b0 msgr2=0x7f7b800fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 -- 172.21.15.86:0/1676496602 shutdown_connections 2024-04-16T05:34:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 -- 172.21.15.86:0/1676496602 wait complete. 2024-04-16T05:34:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.532+0000 7f7b85167640 1 Processor -- start 2024-04-16T05:34:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b85167640 1 -- start start 2024-04-16T05:34:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b85167640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b80102b40 0x7f7b801983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b85167640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b80105470 0x7f7b80198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b85167640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 0x7f7b8019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b85167640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8010dd50 con 0x7f7b80107da0 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b85167640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8010dbd0 con 0x7f7b80102b40 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b85167640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8010ded0 con 0x7f7b80105470 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 0x7f7b8019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 0x7f7b8019f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60516/0 (socket says 172.21.15.86:60516) 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7f59e640 1 -- 172.21.15.86:0/1196223917 learned_addr learned my addr 172.21.15.86:0/1196223917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b80102b40 0x7f7b801983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b80105470 0x7f7b80198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7f59e640 1 -- 172.21.15.86:0/1196223917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b80105470 msgr2=0x7f7b80198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7f59e640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b80105470 0x7f7b80198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7f59e640 1 -- 172.21.15.86:0/1196223917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b80102b40 msgr2=0x7f7b801983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7f59e640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b80102b40 0x7f7b801983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7f59e640 1 -- 172.21.15.86:0/1196223917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8019ffe0 con 0x7f7b80107da0 2024-04-16T05:34:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.533+0000 7f7b7f59e640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 0x7f7b8019f970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802c700 tx=0x7f7b78032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.534+0000 7f7b67fff640 1 -- 172.21.15.86:0/1196223917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78039040 con 0x7f7b80107da0 2024-04-16T05:34:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.534+0000 7f7b85167640 1 -- 172.21.15.86:0/1196223917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b801a0270 con 0x7f7b80107da0 2024-04-16T05:34:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.534+0000 7f7b67fff640 1 -- 172.21.15.86:0/1196223917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b78044070 con 0x7f7b80107da0 2024-04-16T05:34:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.534+0000 7f7b67fff640 1 -- 172.21.15.86:0/1196223917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780042a0 con 0x7f7b80107da0 2024-04-16T05:34:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.534+0000 7f7b85167640 1 -- 172.21.15.86:0/1196223917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b801a07b0 con 0x7f7b80107da0 2024-04-16T05:34:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.536+0000 7f7b67fff640 1 -- 172.21.15.86:0/1196223917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7b78032690 con 0x7f7b80107da0 2024-04-16T05:34:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.536+0000 7f7b87e2d740 1 Processor -- start 2024-04-16T05:34:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.536+0000 7f7b87e2d740 1 -- start start 2024-04-16T05:34:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 0x56060392a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392a750 0x56060392f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392f8c0 0x560603931d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56060383d240 con 0x56060392f8c0 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56060389a9f0 con 0x56060392a750 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5606038559e0 con 0x560603929d60 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392f8c0 0x560603931d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392f8c0 0x560603931d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60530/0 (socket says 172.21.15.86:60530) 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 -- 172.21.15.86:0/1390193462 learned_addr learned my addr 172.21.15.86:0/1390193462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/1390193462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 0x56060392a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b67fff640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7b680a3bb0 0x7f7b680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/1390193462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392a750 0x56060392f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7f59e640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7b680a3bb0 0x7f7b680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 -- 172.21.15.86:0/1390193462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 msgr2=0x56060392a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/1390193462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 0x56060392a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 -- 172.21.15.86:0/1390193462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392a750 msgr2=0x56060392f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/1390193462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392a750 0x56060392f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 -- 172.21.15.86:0/1390193462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560603848bc0 con 0x56060392f8c0 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b67fff640 1 -- 172.21.15.86:0/1196223917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7b780f2cd0 con 0x7f7b80107da0 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.537+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/1390193462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392f8c0 0x560603931d50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7b74014960 tx=0x7f7b7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7fac827c316461 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.538+0000 7f7b657fa640 1 -- 172.21.15.86:0/1390193462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740142d0 con 0x56060392f8c0 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.538+0000 7f7b7f59e640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7b680a3bb0 0x7f7b680a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7b780bb6b0 tx=0x7f7b780b41c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.538+0000 7f7b657fa640 1 -- 172.21.15.86:0/1390193462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b74005500 con 0x56060392f8c0 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.538+0000 7f7b657fa640 1 -- 172.21.15.86:0/1390193462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400f760 con 0x56060392f8c0 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 -- 172.21.15.86:0/1390193462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392f8c0 msgr2=0x560603931d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/1390193462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392f8c0 0x560603931d50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7b74014960 tx=0x7f7b7400c490 comp rx=0 tx=0).stop 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 -- 172.21.15.86:0/1390193462 shutdown_connections 2024-04-16T05:34:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/1390193462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392f8c0 0x560603931d50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/1390193462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392a750 0x56060392f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/1390193462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 0x56060392a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 -- 172.21.15.86:0/1390193462 >> 172.21.15.86:0/1390193462 conn(0x5606039252d0 msgr2=0x5606039256d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 -- 172.21.15.86:0/1390193462 shutdown_connections 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 -- 172.21.15.86:0/1390193462 wait complete. 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 -- start start 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.539+0000 7f7b87e2d740 1 Processor -- start 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603933fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 0x5606039344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x56060393b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5606038940f0 con 0x560603929d60 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56060388edd0 con 0x56060392f8c0 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56060389ccd0 con 0x56060392a750 2024-04-16T05:34:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b87e2d740 10 client.0 _ll_register_callbacks cb 0x560603885570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603933fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603933fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60544/0 (socket says 172.21.15.86:60544) 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 -- 172.21.15.86:0/551319266 learned_addr learned my addr 172.21.15.86:0/551319266 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 0x5606039344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x56060393b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 -- 172.21.15.86:0/551319266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 msgr2=0x5606039344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 0x5606039344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 -- 172.21.15.86:0/551319266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 msgr2=0x56060393b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x56060393b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 -- 172.21.15.86:0/551319266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606035a22d0 con 0x560603929d60 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603933fa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7b74001eb0 tx=0x7f7b74005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bee02435df59050 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.540+0000 7f7b62ff5640 1 -- 172.21.15.86:0/551319266 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7402a020 con 0x560603929d60 2024-04-16T05:34:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b62ff5640 1 -- 172.21.15.86:0/551319266 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7b740051b0 con 0x560603929d60 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b87e2d740 1 -- 172.21.15.86:0/551319266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b74005500 con 0x560603929d60 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b62ff5640 1 -- 172.21.15.86:0/551319266 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b7400f730 con 0x560603929d60 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b87e2d740 10 client.15288 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b87e2d740 1 -- 172.21.15.86:0/551319266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56060366c6b0 con 0x560603929d60 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b62ff5640 1 -- 172.21.15.86:0/551319266 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b74005500 con 0x560603929d60 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b62ff5640 1 -- 172.21.15.86:0/551319266 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b740206b0 con 0x560603929d60 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b87e2d740 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b87e2d740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b87e2d740 20 client.15288 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56060393fa10 0x560603941b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b87e2d740 4 client.15288 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b87e2d740 1 -- 172.21.15.86:0/551319266 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606038164c0 con 0x56060393fa10 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.541+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56060393fa10 0x560603941b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.542+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56060393fa10 0x560603941b50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1575d3cfff7301ca server_cookie=98f9b1b17e1d9a44 in_seq=0 out_seq=0 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.542+0000 7f7b62ff5640 10 client.15288 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b62ff5640 1 -- 172.21.15.86:0/551319266 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606038164c0 con 0x56060393fa10 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b62ff5640 10 client.15288 handle_command_reply: tid=0 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 client.15288 shutdown 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 -- 172.21.15.86:0/551319266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 msgr2=0x560603933fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603933fa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7b74001eb0 tx=0x7f7b74005f80 comp rx=0 tx=0).stop 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 -- 172.21.15.86:0/551319266 shutdown_connections 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56060393fa10 0x560603941b50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x56060393b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 0x5606039344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b62ff5640 0 client.15288 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/551319266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603933fa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 -- 172.21.15.86:0/551319266 >> 172.21.15.86:0/551319266 conn(0x5606039252d0 msgr2=0x560603924410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 -- 172.21.15.86:0/551319266 shutdown_connections 2024-04-16T05:34:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 1 -- 172.21.15.86:0/551319266 wait complete. 2024-04-16T05:34:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.543+0000 7f7b87e2d740 20 client.15288 trim_cache size 0 max 16384 2024-04-16T05:34:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.566+0000 7f7b87e2d740 1 Processor -- start 2024-04-16T05:34:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.566+0000 7f7b87e2d740 1 -- start start 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 0x560603933bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392a750 0x56060393b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x5606039340f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56060393fe90 con 0x56060392a750 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560603940560 con 0x56060392f8c0 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560603940c30 con 0x560603929d60 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392a750 0x56060393b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392a750 0x56060393b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60550/0 (socket says 172.21.15.86:60550) 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7ed9d640 1 -- 172.21.15.86:0/3951869461 learned_addr learned my addr 172.21.15.86:0/3951869461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x5606039340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 0x560603933bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7ed9d640 1 -- 172.21.15.86:0/3951869461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 msgr2=0x560603933bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 0x560603933bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7ed9d640 1 -- 172.21.15.86:0/3951869461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 msgr2=0x5606039340f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x5606039340f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7ed9d640 1 -- 172.21.15.86:0/3951869461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560603926040 con 0x56060392a750 2024-04-16T05:34:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x5606039340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392a750 0x56060393b4b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009a70 tx=0x7f7b70009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6402f46aebeed51d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.567+0000 7f7b64ff9640 1 -- 172.21.15.86:0/3951869461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70025070 con 0x56060392a750 2024-04-16T05:34:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.568+0000 7f7b64ff9640 1 -- 172.21.15.86:0/3951869461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7001f8c0 con 0x56060392a750 2024-04-16T05:34:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.568+0000 7f7b64ff9640 1 -- 172.21.15.86:0/3951869461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70014670 con 0x56060392a750 2024-04-16T05:34:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.568+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3951869461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392a750 msgr2=0x56060393b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.568+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392a750 0x56060393b4b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009a70 tx=0x7f7b70009110 comp rx=0 tx=0).stop 2024-04-16T05:34:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.568+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3951869461 shutdown_connections 2024-04-16T05:34:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.568+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x5606039340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.568+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56060392a750 0x56060393b4b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.568+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3951869461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560603929d60 0x560603933bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3951869461 >> 172.21.15.86:0/3951869461 conn(0x5606039252d0 msgr2=0x5606039256b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3951869461 shutdown_connections 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3951869461 wait complete. 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 -- start start 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 Processor -- start 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603936bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 0x560603937110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x56060393a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5606039419d0 con 0x560603929d60 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560603941300 con 0x56060392f8c0 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5606039420a0 con 0x56060392a750 2024-04-16T05:34:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b87e2d740 10 client.0 _ll_register_callbacks cb 0x7f7b700093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603936bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603936bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60554/0 (socket says 172.21.15.86:60554) 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b7e59c640 1 -- 172.21.15.86:0/3855944499 learned_addr learned my addr 172.21.15.86:0/3855944499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 0x560603937110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b7e59c640 1 -- 172.21.15.86:0/3855944499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 msgr2=0x560603937110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 0x560603937110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b7e59c640 1 -- 172.21.15.86:0/3855944499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 msgr2=0x56060393a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x56060393a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.569+0000 7f7b7e59c640 1 -- 172.21.15.86:0/3855944499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56060393a720 con 0x560603929d60 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b7e59c640 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603936bd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7b74020c70 tx=0x7f7b7400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83953c7fe71b0dde server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b62ff5640 1 -- 172.21.15.86:0/3855944499 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7402a040 con 0x560603929d60 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b62ff5640 1 -- 172.21.15.86:0/3855944499 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7b74020650 con 0x560603929d60 2024-04-16T05:34:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3855944499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b7001f8c0 con 0x560603929d60 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b62ff5640 1 -- 172.21.15.86:0/3855944499 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b74004ae0 con 0x560603929d60 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b87e2d740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3855944499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606039a6690 con 0x560603929d60 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b62ff5640 1 -- 172.21.15.86:0/3855944499 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7001f8c0 con 0x560603929d60 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b62ff5640 1 -- 172.21.15.86:0/3855944499 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b7400f9d0 con 0x560603929d60 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b87e2d740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b87e2d740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b87e2d740 20 client.15294 populate_metadata read hostname 'smithi086' 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56060393fa10 0x5606039a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b87e2d740 4 client.15294 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.570+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3855944499 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "status"}) -- 0x56060389d4e0 con 0x56060393fa10 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.571+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56060393fa10 0x5606039a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.571+0000 7f7b7ed9d640 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56060393fa10 0x5606039a9c80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e7c7b1b2313b81 server_cookie=b07074448c0329e in_seq=0 out_seq=0 2024-04-16T05:34:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.571+0000 7f7b62ff5640 10 client.15294 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.571+0000 7f7b62ff5640 1 -- 172.21.15.86:0/3855944499 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56060389d4e0 con 0x56060393fa10 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.571+0000 7f7b62ff5640 10 client.15294 handle_command_reply: tid=0 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.571+0000 7f7b87e2d740 1 client.15294 shutdown 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3855944499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 msgr2=0x560603936bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603936bd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7b74020c70 tx=0x7f7b7400ca80 comp rx=0 tx=0).stop 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3855944499 shutdown_connections 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56060393fa10 0x5606039a9c80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56060392f8c0 0x56060393a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56060392a750 0x560603937110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b62ff5640 0 client.15294 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 --2- 172.21.15.86:0/3855944499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560603929d60 0x560603936bd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3855944499 >> 172.21.15.86:0/3855944499 conn(0x5606039252d0 msgr2=0x560603933bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3855944499 shutdown_connections 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 1 -- 172.21.15.86:0/3855944499 wait complete. 2024-04-16T05:34:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b87e2d740 20 client.15294 trim_cache size 0 max 16384 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24482, 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 106.726100523, 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:34:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 92, 2024-04-16T05:34:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:34:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 113.093650782 2024-04-16T05:34:42.578 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 -- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7b680a3bb0 msgr2=0x7f7b680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7b680a3bb0 0x7f7b680a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7b780bb6b0 tx=0x7f7b780b41c0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 -- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 msgr2=0x7f7b8019f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 0x7f7b8019f970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802c700 tx=0x7f7b78032f20 comp rx=0 tx=0).stop 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 -- 172.21.15.86:0/1196223917 shutdown_connections 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b80107da0 0x7f7b8019f970 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b80105470 0x7f7b80198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7b680a3bb0 0x7f7b680a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 --2- 172.21.15.86:0/1196223917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b80102b40 0x7f7b801983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 -- 172.21.15.86:0/1196223917 >> 172.21.15.86:0/1196223917 conn(0x7f7b800fc4b0 msgr2=0x7f7b80103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 -- 172.21.15.86:0/1196223917 shutdown_connections 2024-04-16T05:34:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:42.572+0000 7f7b85167640 1 -- 172.21.15.86:0/1196223917 wait complete. 2024-04-16T05:34:42.582 INFO:tasks.cephadm:Setup complete, yielding 2024-04-16T05:34:42.583 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-16T05:34:42.592 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:34:42.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch status' 2024-04-16T05:34:42.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:42 smithi086 ceph-mon[32028]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:42.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:42 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3676429300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:42.851 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:42.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:42 smithi194 ceph-mon[35865]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:42.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:42 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3676429300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:42.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:42 smithi196 ceph-mon[36373]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:42.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:42 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3676429300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.700+0000 7fa15c697700 1 -- 172.21.15.86:0/3737123559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 msgr2=0x7fa1541030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.700+0000 7fa15c697700 1 --2- 172.21.15.86:0/3737123559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 0x7fa1541030b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa1440099c0 tx=0x7fa144009cd0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 -- 172.21.15.86:0/3737123559 shutdown_connections 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 --2- 172.21.15.86:0/3737123559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1541035f0 0x7fa154109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 --2- 172.21.15.86:0/3737123559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa154101a50 0x7fa154101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 --2- 172.21.15.86:0/3737123559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 0x7fa1541030b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 -- 172.21.15.86:0/3737123559 >> 172.21.15.86:0/3737123559 conn(0x7fa1540fcff0 msgr2=0x7fa1540ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 -- 172.21.15.86:0/3737123559 shutdown_connections 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 -- 172.21.15.86:0/3737123559 wait complete. 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 Processor -- start 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 -- start start 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa154101a50 0x7fa154198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 0x7fa154199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1541035f0 0x7fa1541a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa15410d730 con 0x7fa154102c50 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa15410d5b0 con 0x7fa154101a50 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.701+0000 7fa15c697700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa15410d8b0 con 0x7fa1541035f0 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa159c32700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 0x7fa154199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa159c32700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 0x7fa154199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60576/0 (socket says 172.21.15.86:60576) 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa159c32700 1 -- 172.21.15.86:0/3894634262 learned_addr learned my addr 172.21.15.86:0/3894634262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa15ac34700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1541035f0 0x7fa1541a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa15a433700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa154101a50 0x7fa154198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa159c32700 1 -- 172.21.15.86:0/3894634262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1541035f0 msgr2=0x7fa1541a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa159c32700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1541035f0 0x7fa1541a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa159c32700 1 -- 172.21.15.86:0/3894634262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa154101a50 msgr2=0x7fa154198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa159c32700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa154101a50 0x7fa154198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa159c32700 1 -- 172.21.15.86:0/3894634262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1541a0b10 con 0x7fa154102c50 2024-04-16T05:34:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa15ac34700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1541035f0 0x7fa1541a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa159c32700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 0x7fa154199440 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa144005910 tx=0x7fa144003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa14b7fe700 1 -- 172.21.15.86:0/3894634262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144027070 con 0x7fa154102c50 2024-04-16T05:34:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1541a0da0 con 0x7fa154102c50 2024-04-16T05:34:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa14b7fe700 1 -- 172.21.15.86:0/3894634262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa144022470 con 0x7fa154102c50 2024-04-16T05:34:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa14b7fe700 1 -- 172.21.15.86:0/3894634262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f6c0 con 0x7fa154102c50 2024-04-16T05:34:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.702+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1541a1270 con 0x7fa154102c50 2024-04-16T05:34:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.703+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1540697e0 con 0x7fa154102c50 2024-04-16T05:34:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.705+0000 7fa14b7fe700 1 -- 172.21.15.86:0/3894634262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa14402a070 con 0x7fa154102c50 2024-04-16T05:34:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.705+0000 7fa14b7fe700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1400a3c40 0x7fa1400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.705+0000 7fa15a433700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1400a3c40 0x7fa1400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.705+0000 7fa14b7fe700 1 -- 172.21.15.86:0/3894634262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa1440d1910 con 0x7fa154102c50 2024-04-16T05:34:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.706+0000 7fa15a433700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1400a3c40 0x7fa1400a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa150006fd0 tx=0x7fa150008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:44.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.709+0000 7fa14b7fe700 1 -- 172.21.15.86:0/3894634262 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa144013070 con 0x7fa154102c50 2024-04-16T05:34:44.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:44 smithi194 ceph-mon[35865]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.923+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa1540ff1f0 con 0x7fa1400a3c40 2024-04-16T05:34:44.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:44 smithi086 ceph-mon[32028]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.924+0000 7fa14b7fe700 1 -- 172.21.15.86:0/3894634262 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa1540ff1f0 con 0x7fa1400a3c40 2024-04-16T05:34:44.923 INFO:teuthology.orchestra.run.smithi086.stdout:Backend: cephadm 2024-04-16T05:34:44.923 INFO:teuthology.orchestra.run.smithi086.stdout:Available: Yes 2024-04-16T05:34:44.923 INFO:teuthology.orchestra.run.smithi086.stdout:Paused: No 2024-04-16T05:34:44.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1400a3c40 msgr2=0x7fa1400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1400a3c40 0x7fa1400a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa150006fd0 tx=0x7fa150008040 comp rx=0 tx=0).stop 2024-04-16T05:34:44.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 msgr2=0x7fa154199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 0x7fa154199440 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa144005910 tx=0x7fa144003730 comp rx=0 tx=0).stop 2024-04-16T05:34:44.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 shutdown_connections 2024-04-16T05:34:44.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1400a3c40 0x7fa1400a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1541035f0 0x7fa1541a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa154101a50 0x7fa154198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 --2- 172.21.15.86:0/3894634262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa154102c50 0x7fa154199440 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 >> 172.21.15.86:0/3894634262 conn(0x7fa1540fcff0 msgr2=0x7fa1540feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:44.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 shutdown_connections 2024-04-16T05:34:44.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:44.927+0000 7fa15c697700 1 -- 172.21.15.86:0/3894634262 wait complete. 2024-04-16T05:34:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:44 smithi196 ceph-mon[36373]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:45.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-16T05:34:45.976 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:46.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:46 smithi086 ceph-mon[32028]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:46.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:46 smithi086 ceph-mon[32028]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:34:46.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:46 smithi194 ceph-mon[35865]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:46.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:46 smithi194 ceph-mon[35865]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:34:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:46 smithi196 ceph-mon[36373]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:46 smithi196 ceph-mon[36373]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:34:47.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.725+0000 7f1888b15700 1 -- 172.21.15.86:0/2196179200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 msgr2=0x7f18841030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.725+0000 7f1888b15700 1 --2- 172.21.15.86:0/2196179200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 0x7f18841030d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f18740099c0 tx=0x7f1874009cd0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.725+0000 7f1888b15700 1 -- 172.21.15.86:0/2196179200 shutdown_connections 2024-04-16T05:34:47.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.725+0000 7f1888b15700 1 --2- 172.21.15.86:0/2196179200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1884103610 0x7f1884109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.725+0000 7f1888b15700 1 --2- 172.21.15.86:0/2196179200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1884101a70 0x7f1884101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.725+0000 7f1888b15700 1 --2- 172.21.15.86:0/2196179200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 0x7f18841030d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.725+0000 7f1888b15700 1 -- 172.21.15.86:0/2196179200 >> 172.21.15.86:0/2196179200 conn(0x7f18840fd050 msgr2=0x7f18840ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.725+0000 7f1888b15700 1 -- 172.21.15.86:0/2196179200 shutdown_connections 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.725+0000 7f1888b15700 1 -- 172.21.15.86:0/2196179200 wait complete. 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.726+0000 7f1888b15700 1 Processor -- start 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.726+0000 7f1888b15700 1 -- start start 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.726+0000 7f1888b15700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1884101a70 0x7f1884197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.726+0000 7f1888b15700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 0x7f18841981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.726+0000 7f1888b15700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1884103610 0x7f188419f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.726+0000 7f1888b15700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f188410d650 con 0x7f1884102c70 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.726+0000 7f1888b15700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f188410d4d0 con 0x7f1884103610 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.726+0000 7f1888b15700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f188410d7d0 con 0x7f1884101a70 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1881d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 0x7f18841981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1881d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 0x7f18841981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60592/0 (socket says 172.21.15.86:60592) 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1881d9b700 1 -- 172.21.15.86:0/1508331899 learned_addr learned my addr 172.21.15.86:0/1508331899 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f188259c700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1884101a70 0x7f1884197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1881d9b700 1 -- 172.21.15.86:0/1508331899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1884101a70 msgr2=0x7f1884197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1881d9b700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1884101a70 0x7f1884197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1881d9b700 1 -- 172.21.15.86:0/1508331899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1884103610 msgr2=0x7f188419f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1881d9b700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1884103610 0x7f188419f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1881d9b700 1 -- 172.21.15.86:0/1508331899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188406b2e0 con 0x7f1884102c70 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1881d9b700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 0x7f18841981a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1874003890 tx=0x7f18740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f18737fe700 1 -- 172.21.15.86:0/1508331899 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874027070 con 0x7f1884102c70 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188406b570 con 0x7f1884102c70 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f18737fe700 1 -- 172.21.15.86:0/1508331899 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18740041d0 con 0x7f1884102c70 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f18737fe700 1 -- 172.21.15.86:0/1508331899 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187400f460 con 0x7f1884102c70 2024-04-16T05:34:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.727+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188406ba50 con 0x7f1884102c70 2024-04-16T05:34:47.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.728+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18840697e0 con 0x7f1884102c70 2024-04-16T05:34:47.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.729+0000 7f18737fe700 1 -- 172.21.15.86:0/1508331899 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f187402a070 con 0x7f1884102c70 2024-04-16T05:34:47.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.730+0000 7f18737fe700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f186c0a3c40 0x7f186c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.730+0000 7f188259c700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f186c0a3c40 0x7f186c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.730+0000 7f18737fe700 1 -- 172.21.15.86:0/1508331899 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f18740d16a0 con 0x7f1884102c70 2024-04-16T05:34:47.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.731+0000 7f188259c700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f186c0a3c40 0x7f186c0a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1878005950 tx=0x7f187800a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.735+0000 7f18737fe700 1 -- 172.21.15.86:0/1508331899 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1874013070 con 0x7f1884102c70 2024-04-16T05:34:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.942+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f188406bcb0 con 0x7f186c0a3c40 2024-04-16T05:34:47.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.955+0000 7f18737fe700 1 -- 172.21.15.86:0/1508331899 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f188406bcb0 con 0x7f186c0a3c40 2024-04-16T05:34:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-16T05:34:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:mds.a smithi086 running (2m) 20s ago 2m 18.7M - 19.0.0-3101-ga87fcff6 3e786b67158f d1005bb154d0 2024-04-16T05:34:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:mds.b smithi194 running (118s) 52s ago 118s 26.0M - 19.0.0-3101-ga87fcff6 3e786b67158f 999ed2367937 2024-04-16T05:34:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:mds.c smithi196 running (2m) 52s ago 2m 17.3M - 19.0.0-3101-ga87fcff6 3e786b67158f b11b26d5c298 2024-04-16T05:34:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:mds.d smithi086 running (2m) 20s ago 2m 15.3M - 19.0.0-3101-ga87fcff6 3e786b67158f 695332d77798 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mds.e smithi194 running (2m) 52s ago 2m 18.5M - 19.0.0-3101-ga87fcff6 3e786b67158f 8d1011153208 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mds.f smithi196 running (2m) 52s ago 2m 17.1M - 19.0.0-3101-ga87fcff6 3e786b67158f e9abd2da7dad 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mds.g smithi086 running (2m) 20s ago 2m 16.4M - 19.0.0-3101-ga87fcff6 3e786b67158f fde136e54058 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mds.h smithi194 running (2m) 52s ago 2m 17.3M - 19.0.0-3101-ga87fcff6 3e786b67158f 33598d38e68f 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mds.i smithi196 running (2m) 52s ago 2m 19.7M - 19.0.0-3101-ga87fcff6 3e786b67158f cdc608bfa682 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mds.j smithi086 running (2m) 20s ago 2m 14.1M - 19.0.0-3101-ga87fcff6 3e786b67158f 77388f01fc17 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mds.k smithi194 running (2m) 52s ago 2m 16.2M - 19.0.0-3101-ga87fcff6 3e786b67158f ba3233fc3bde 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mds.l smithi196 running (2m) 52s ago 2m 18.8M - 19.0.0-3101-ga87fcff6 3e786b67158f 0bb4d70276ee 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.x smithi086 *:9283,8765 running (9m) 20s ago 9m 515M - 19.0.0-3101-ga87fcff6 3e786b67158f 3de8902fe015 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.y smithi194 *:8443,8765 running (7m) 52s ago 7m 454M - 19.0.0-3101-ga87fcff6 3e786b67158f 28612d49764f 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.z smithi196 *:8443,8765 running (7m) 52s ago 7m 454M - 19.0.0-3101-ga87fcff6 3e786b67158f d3858bd2fb30 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mon.a smithi086 running (9m) 20s ago 9m 50.8M 2048M 19.0.0-3101-ga87fcff6 3e786b67158f 29c249e4574d 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mon.b smithi194 running (8m) 52s ago 8m 38.8M 2048M 19.0.0-3101-ga87fcff6 3e786b67158f b922bb74c42a 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:mon.c smithi196 running (8m) 52s ago 8m 39.1M 2048M 19.0.0-3101-ga87fcff6 3e786b67158f 2a64957c51d3 2024-04-16T05:34:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 smithi086 running (7m) 20s ago 7m 59.3M 4272M 19.0.0-3101-ga87fcff6 3e786b67158f f2d410462f5d 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 smithi194 running (7m) 52s ago 7m 58.4M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 27921751cb2c 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 smithi196 running (6m) 52s ago 6m 58.4M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f e8eea4c52f6b 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 smithi086 running (6m) 20s ago 6m 43.3M 4272M 19.0.0-3101-ga87fcff6 3e786b67158f 08ac20212f46 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 smithi194 running (5m) 52s ago 5m 53.5M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 8c3fd860cfd0 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 smithi196 running (5m) 52s ago 5m 55.5M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 8e0b4421f552 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 smithi086 running (4m) 20s ago 4m 44.3M 4272M 19.0.0-3101-ga87fcff6 3e786b67158f ddc58f48ce65 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 smithi194 running (4m) 52s ago 4m 44.9M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 640253ab46ab 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 smithi196 running (4m) 52s ago 4m 56.0M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 0fedd8cc998b 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 smithi086 running (3m) 20s ago 3m 52.9M 4272M 19.0.0-3101-ga87fcff6 3e786b67158f 183b558bb296 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 smithi194 running (3m) 52s ago 3m 44.0M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 785b0be09416 2024-04-16T05:34:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 smithi196 running (2m) 52s ago 2m 42.5M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f fc8b6cf50cae 2024-04-16T05:34:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f186c0a3c40 msgr2=0x7f186c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f186c0a3c40 0x7f186c0a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1878005950 tx=0x7f187800a300 comp rx=0 tx=0).stop 2024-04-16T05:34:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 msgr2=0x7f18841981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 0x7f18841981a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1874003890 tx=0x7f18740047b0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 shutdown_connections 2024-04-16T05:34:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1884103610 0x7f188419f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f186c0a3c40 0x7f186c0a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1884101a70 0x7f1884197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 --2- 172.21.15.86:0/1508331899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1884102c70 0x7f18841981a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 >> 172.21.15.86:0/1508331899 conn(0x7f18840fd050 msgr2=0x7f1884105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 shutdown_connections 2024-04-16T05:34:47.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:47.957+0000 7f1888b15700 1 -- 172.21.15.86:0/1508331899 wait complete. 2024-04-16T05:34:48.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:48 smithi086 ceph-mon[32028]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:34:48.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-16T05:34:48.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:48 smithi194 ceph-mon[35865]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:34:48.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:48 smithi196 ceph-mon[36373]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:34:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:49.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:49 smithi086 ceph-mon[32028]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:49.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:49 smithi194 ceph-mon[35865]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:49.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:49 smithi196 ceph-mon[36373]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:50.863 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:50 smithi086 ceph-mon[32028]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.864+0000 7fb606b23700 1 -- 172.21.15.86:0/3385959837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6001035f0 msgr2=0x7fb600109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.864+0000 7fb606b23700 1 --2- 172.21.15.86:0/3385959837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6001035f0 0x7fb600109ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800b0d0 tx=0x7fb5f800b3e0 comp rx=0 tx=0).stop 2024-04-16T05:34:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.865+0000 7fb606b23700 1 -- 172.21.15.86:0/3385959837 shutdown_connections 2024-04-16T05:34:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.865+0000 7fb606b23700 1 --2- 172.21.15.86:0/3385959837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6001035f0 0x7fb600109ea0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.865+0000 7fb606b23700 1 --2- 172.21.15.86:0/3385959837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb600101a50 0x7fb600101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.865+0000 7fb606b23700 1 --2- 172.21.15.86:0/3385959837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb600102c50 0x7fb6001030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.865+0000 7fb606b23700 1 -- 172.21.15.86:0/3385959837 >> 172.21.15.86:0/3385959837 conn(0x7fb6000fcff0 msgr2=0x7fb6000ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.865+0000 7fb606b23700 1 -- 172.21.15.86:0/3385959837 shutdown_connections 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.865+0000 7fb606b23700 1 -- 172.21.15.86:0/3385959837 wait complete. 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.865+0000 7fb606b23700 1 Processor -- start 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.865+0000 7fb606b23700 1 -- start start 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb606b23700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb600101a50 0x7fb600198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb606b23700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102c50 0x7fb600199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb606b23700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6001035f0 0x7fb6001a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb606b23700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb60010d690 con 0x7fb600102c50 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb606b23700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb60010d510 con 0x7fb6001035f0 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb606b23700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb60010d810 con 0x7fb600101a50 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102c50 0x7fb600199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102c50 0x7fb600199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41726/0 (socket says 172.21.15.86:41726) 2024-04-16T05:34:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fffff700 1 -- 172.21.15.86:0/2666632141 learned_addr learned my addr 172.21.15.86:0/2666632141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb6050c0700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6001035f0 0x7fb6001a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb6048bf700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb600101a50 0x7fb600198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fffff700 1 -- 172.21.15.86:0/2666632141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb600101a50 msgr2=0x7fb600198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fffff700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb600101a50 0x7fb600198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fffff700 1 -- 172.21.15.86:0/2666632141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6001035f0 msgr2=0x7fb6001a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fffff700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6001035f0 0x7fb6001a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fffff700 1 -- 172.21.15.86:0/2666632141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6001a0a30 con 0x7fb600102c50 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb6050c0700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6001035f0 0x7fb6001a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fffff700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102c50 0x7fb600199310 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b560 tx=0x7fb5ec00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fdffb700 1 -- 172.21.15.86:0/2666632141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec015020 con 0x7fb600102c50 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fdffb700 1 -- 172.21.15.86:0/2666632141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ec011d10 con 0x7fb600102c50 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb5fdffb700 1 -- 172.21.15.86:0/2666632141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec010e50 con 0x7fb600102c50 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.866+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6001a0d20 con 0x7fb600102c50 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.867+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6001a12d0 con 0x7fb600102c50 2024-04-16T05:34:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.868+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb600069810 con 0x7fb600102c50 2024-04-16T05:34:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.869+0000 7fb5fdffb700 1 -- 172.21.15.86:0/2666632141 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb5ec018020 con 0x7fb600102c50 2024-04-16T05:34:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.870+0000 7fb5fdffb700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5f00a3e60 0x7fb5f00a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.870+0000 7fb6048bf700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5f00a3e60 0x7fb5f00a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.870+0000 7fb5fdffb700 1 -- 172.21.15.86:0/2666632141 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb5ec0cfe40 con 0x7fb600102c50 2024-04-16T05:34:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.870+0000 7fb6048bf700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5f00a3e60 0x7fb5f00a6320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb600102ab0 tx=0x7fb5f400a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:50.874+0000 7fb5fdffb700 1 -- 172.21.15.86:0/2666632141 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb600069810 con 0x7fb600102c50 2024-04-16T05:34:50.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:50 smithi196 ceph-mon[36373]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:50.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:50 smithi194 ceph-mon[35865]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:34:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.081+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb6000ff0f0 con 0x7fb5f00a3e60 2024-04-16T05:34:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.087+0000 7fb5fdffb700 1 -- 172.21.15.86:0/2666632141 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb6000ff0f0 con 0x7fb5f00a3e60 2024-04-16T05:34:51.086 INFO:teuthology.orchestra.run.smithi086.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-16T05:34:51.086 INFO:teuthology.orchestra.run.smithi086.stdout:mds.all 12/12 55s ago 2m smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi194=b;smithi194=e;smithi194=h;smithi194=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-16T05:34:51.086 INFO:teuthology.orchestra.run.smithi086.stdout:mgr 3/3 55s ago 7m smithi086=x;smithi194=y;smithi196=z;count:3 2024-04-16T05:34:51.087 INFO:teuthology.orchestra.run.smithi086.stdout:mon 3/3 55s ago 8m smithi086:172.21.15.86=a;smithi194:172.21.15.194=b;smithi196:172.21.15.196=c;count:3 2024-04-16T05:34:51.087 INFO:teuthology.orchestra.run.smithi086.stdout:osd 12 55s ago - 2024-04-16T05:34:51.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5f00a3e60 msgr2=0x7fb5f00a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:51.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5f00a3e60 0x7fb5f00a6320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb600102ab0 tx=0x7fb5f400a380 comp rx=0 tx=0).stop 2024-04-16T05:34:51.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102c50 msgr2=0x7fb600199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:51.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102c50 0x7fb600199310 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b560 tx=0x7fb5ec00b870 comp rx=0 tx=0).stop 2024-04-16T05:34:51.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 shutdown_connections 2024-04-16T05:34:51.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6001035f0 0x7fb6001a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb5f00a3e60 0x7fb5f00a6320 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb600101a50 0x7fb600198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 --2- 172.21.15.86:0/2666632141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102c50 0x7fb600199310 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 >> 172.21.15.86:0/2666632141 conn(0x7fb6000fcff0 msgr2=0x7fb6000fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:51.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 shutdown_connections 2024-04-16T05:34:51.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:51.089+0000 7fb606b23700 1 -- 172.21.15.86:0/2666632141 wait complete. 2024-04-16T05:34:51.899 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-16T05:34:51.899 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:51 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-16T05:34:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-16T05:34:51.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:51 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-16T05:34:51.926 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-16T05:34:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-16T05:34:51.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:51 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-16T05:34:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:52.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:52 smithi086 ceph-mon[32028]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:52.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:52 smithi086 ceph-mon[32028]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:52.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-16T05:34:52.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:52 smithi086 ceph-mon[32028]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-16T05:34:52.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:52 smithi086 ceph-mon[32028]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T05:34:52.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:52 smithi194 ceph-mon[35865]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:52.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:52 smithi194 ceph-mon[35865]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-16T05:34:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:52 smithi194 ceph-mon[35865]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-16T05:34:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:52 smithi194 ceph-mon[35865]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T05:34:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:52 smithi196 ceph-mon[36373]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:52 smithi196 ceph-mon[36373]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-16T05:34:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-16T05:34:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:52 smithi196 ceph-mon[36373]: from='mgr.14148 172.21.15.86:0/4009649614' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-16T05:34:52.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:52 smithi196 ceph-mon[36373]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T05:34:53.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:53 smithi086 ceph-mon[32028]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T05:34:53.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:53 smithi194 ceph-mon[35865]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T05:34:53.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:53 smithi196 ceph-mon[36373]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T05:34:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 -- 172.21.15.86:0/3375625245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c104730 msgr2=0x7f8f1c10afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/3375625245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c104730 0x7f8f1c10afe0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400b0d0 tx=0x7f8f1400b3e0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 -- 172.21.15.86:0/3375625245 shutdown_connections 2024-04-16T05:34:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/3375625245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c104730 0x7f8f1c10afe0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/3375625245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f1c102b90 0x7f8f1c102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/3375625245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c103d90 0x7f8f1c1041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 -- 172.21.15.86:0/3375625245 >> 172.21.15.86:0/3375625245 conn(0x7f8f1c0fe130 msgr2=0x7f8f1c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 -- 172.21.15.86:0/3375625245 shutdown_connections 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 -- 172.21.15.86:0/3375625245 wait complete. 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 Processor -- start 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.024+0000 7f8f21c5b700 1 -- start start 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f21c5b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c102b90 0x7f8f1c19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f21c5b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c103d90 0x7f8f1c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f21c5b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f1c104730 0x7f8f1c1a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f21c5b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c10e780 con 0x7f8f1c102b90 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f21c5b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c10e600 con 0x7f8f1c104730 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f21c5b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c10e900 con 0x7f8f1c103d90 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f1b7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c102b90 0x7f8f1c19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f1b7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c102b90 0x7f8f1c19f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41736/0 (socket says 172.21.15.86:41736) 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f1b7fe700 1 -- 172.21.15.86:0/1124923109 learned_addr learned my addr 172.21.15.86:0/1124923109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f13fff700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c103d90 0x7f8f1c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f1bfff700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f1c104730 0x7f8f1c1a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f13fff700 1 -- 172.21.15.86:0/1124923109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f1c104730 msgr2=0x7f8f1c1a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f13fff700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f1c104730 0x7f8f1c1a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f13fff700 1 -- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c102b90 msgr2=0x7f8f1c19f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f13fff700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c102b90 0x7f8f1c19f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f13fff700 1 -- 172.21.15.86:0/1124923109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c1a6f40 con 0x7f8f1c103d90 2024-04-16T05:34:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f1b7fe700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c102b90 0x7f8f1c19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:34:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.025+0000 7f8f13fff700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c103d90 0x7f8f1c19f820 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400d780 tx=0x7f8f0400da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.026+0000 7f8f197fa700 1 -- 172.21.15.86:0/1124923109 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04016070 con 0x7f8f1c103d90 2024-04-16T05:34:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.026+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c1a7230 con 0x7f8f1c103d90 2024-04-16T05:34:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.026+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c1a7770 con 0x7f8f1c103d90 2024-04-16T05:34:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.026+0000 7f8f197fa700 1 -- 172.21.15.86:0/1124923109 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f04005020 con 0x7f8f1c103d90 2024-04-16T05:34:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.026+0000 7f8f197fa700 1 -- 172.21.15.86:0/1124923109 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0400fb30 con 0x7f8f1c103d90 2024-04-16T05:34:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.027+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8efc005360 con 0x7f8f1c103d90 2024-04-16T05:34:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.028+0000 7f8f197fa700 1 -- 172.21.15.86:0/1124923109 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8f0400fcd0 con 0x7f8f1c103d90 2024-04-16T05:34:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.029+0000 7f8f197fa700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8f080a3cc0 0x7f8f080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.029+0000 7f8f1b7fe700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8f080a3cc0 0x7f8f080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.029+0000 7f8f197fa700 1 -- 172.21.15.86:0/1124923109 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8f040d0410 con 0x7f8f1c103d90 2024-04-16T05:34:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.029+0000 7f8f1b7fe700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8f080a3cc0 0x7f8f080a6180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0105c0 tx=0x7f8f0c00b4a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.033+0000 7f8f197fa700 1 -- 172.21.15.86:0/1124923109 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f0409ca60 con 0x7f8f1c103d90 2024-04-16T05:34:54.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.240+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f8efc000bf0 con 0x7f8f080a3cc0 2024-04-16T05:34:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.241+0000 7f8f197fa700 1 -- 172.21.15.86:0/1124923109 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f8efc000bf0 con 0x7f8f080a3cc0 2024-04-16T05:34:54.240 INFO:teuthology.orchestra.run.smithi086.stdout:HOST ADDR LABELS STATUS 2024-04-16T05:34:54.240 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 172.21.15.86 2024-04-16T05:34:54.240 INFO:teuthology.orchestra.run.smithi086.stdout:smithi194 172.21.15.194 2024-04-16T05:34:54.240 INFO:teuthology.orchestra.run.smithi086.stdout:smithi196 172.21.15.196 2024-04-16T05:34:54.240 INFO:teuthology.orchestra.run.smithi086.stdout:3 hosts in cluster 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8f080a3cc0 msgr2=0x7f8f080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8f080a3cc0 0x7f8f080a6180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0105c0 tx=0x7f8f0c00b4a0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c103d90 msgr2=0x7f8f1c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c103d90 0x7f8f1c19f820 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400d780 tx=0x7f8f0400da90 comp rx=0 tx=0).stop 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 shutdown_connections 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f1c104730 0x7f8f1c1a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c102b90 0x7f8f1c19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8f080a3cc0 0x7f8f080a6180 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 --2- 172.21.15.86:0/1124923109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c103d90 0x7f8f1c19f820 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 >> 172.21.15.86:0/1124923109 conn(0x7f8f1c0fe130 msgr2=0x7f8f1c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 shutdown_connections 2024-04-16T05:34:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:54.244+0000 7f8f21c5b700 1 -- 172.21.15.86:0/1124923109 wait complete. 2024-04-16T05:34:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:54 smithi194 ceph-mon[35865]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-16T05:34:54.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:54 smithi086 ceph-mon[32028]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-16T05:34:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:54 smithi196 ceph-mon[36373]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-16T05:34:55.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-16T05:34:55.319 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:55.711 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:55 smithi086 ceph-mon[32028]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:55.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:55 smithi194 ceph-mon[35865]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:55.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:55 smithi196 ceph-mon[36373]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:56.853 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:56 smithi086 ceph-mon[32028]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:34:56.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:56 smithi194 ceph-mon[35865]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:56 smithi196 ceph-mon[36373]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.260+0000 7f87c90db700 1 -- 172.21.15.86:0/713260271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4103d50 msgr2=0x7f87c41041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.260+0000 7f87c90db700 1 --2- 172.21.15.86:0/713260271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4103d50 0x7f87c41041b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f87b80099c0 tx=0x7f87b8009cd0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.260+0000 7f87c90db700 1 -- 172.21.15.86:0/713260271 shutdown_connections 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.260+0000 7f87c90db700 1 --2- 172.21.15.86:0/713260271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87c41046f0 0x7f87c410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.260+0000 7f87c90db700 1 --2- 172.21.15.86:0/713260271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87c4102b50 0x7f87c4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.260+0000 7f87c90db700 1 --2- 172.21.15.86:0/713260271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4103d50 0x7f87c41041b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.260+0000 7f87c90db700 1 -- 172.21.15.86:0/713260271 >> 172.21.15.86:0/713260271 conn(0x7f87c40fe0f0 msgr2=0x7f87c4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.260+0000 7f87c90db700 1 -- 172.21.15.86:0/713260271 shutdown_connections 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.260+0000 7f87c90db700 1 -- 172.21.15.86:0/713260271 wait complete. 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.261+0000 7f87c90db700 1 Processor -- start 2024-04-16T05:34:57.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.261+0000 7f87c90db700 1 -- start start 2024-04-16T05:34:57.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.261+0000 7f87c90db700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87c4102b50 0x7f87c4076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.261+0000 7f87c90db700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4103d50 0x7f87c4074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.261+0000 7f87c90db700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87c41046f0 0x7f87c4075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.261+0000 7f87c90db700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410e7b0 con 0x7f87c4103d50 2024-04-16T05:34:57.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.261+0000 7f87c90db700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410e630 con 0x7f87c41046f0 2024-04-16T05:34:57.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.261+0000 7f87c90db700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410e930 con 0x7f87c4102b50 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c2d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87c4102b50 0x7f87c4076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c2d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87c4102b50 0x7f87c4076890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:55670/0 (socket says 172.21.15.86:55670) 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4103d50 0x7f87c4074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c2d9d700 1 -- 172.21.15.86:0/787398666 learned_addr learned my addr 172.21.15.86:0/787398666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c359e700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87c41046f0 0x7f87c4075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c259c700 1 -- 172.21.15.86:0/787398666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87c4102b50 msgr2=0x7f87c4076890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c259c700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87c4102b50 0x7f87c4076890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c259c700 1 -- 172.21.15.86:0/787398666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87c41046f0 msgr2=0x7f87c4075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c259c700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87c41046f0 0x7f87c4075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c259c700 1 -- 172.21.15.86:0/787398666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c4075cd0 con 0x7f87c4103d50 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c259c700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4103d50 0x7f87c4074ee0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f87b80098e0 tx=0x7f87b80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87abfff700 1 -- 172.21.15.86:0/787398666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8027070 con 0x7f87c4103d50 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87abfff700 1 -- 172.21.15.86:0/787398666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b8004150 con 0x7f87c4103d50 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c41a2ec0 con 0x7f87c4103d50 2024-04-16T05:34:57.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87abfff700 1 -- 172.21.15.86:0/787398666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8022df0 con 0x7f87c4103d50 2024-04-16T05:34:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.262+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c41a3340 con 0x7f87c4103d50 2024-04-16T05:34:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.263+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c402dfe0 con 0x7f87c4103d50 2024-04-16T05:34:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.265+0000 7f87abfff700 1 -- 172.21.15.86:0/787398666 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f87b802a070 con 0x7f87c4103d50 2024-04-16T05:34:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.265+0000 7f87abfff700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87ac0a3c40 0x7f87ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.265+0000 7f87c2d9d700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87ac0a3c40 0x7f87ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.265+0000 7f87abfff700 1 -- 172.21.15.86:0/787398666 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f87b80d1df0 con 0x7f87c4103d50 2024-04-16T05:34:57.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.266+0000 7f87c2d9d700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87ac0a3c40 0x7f87ac0a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009aa0 tx=0x7f87b4009240 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:57.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.269+0000 7f87abfff700 1 -- 172.21.15.86:0/787398666 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87b8031020 con 0x7f87c4103d50 2024-04-16T05:34:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.477+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f87c4062f30 con 0x7f87ac0a3c40 2024-04-16T05:34:57.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.479+0000 7f87abfff700 1 -- 172.21.15.86:0/787398666 <== mgr.14148 v2:172.21.15.86:6800/3194681946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f87c4062f30 con 0x7f87ac0a3c40 2024-04-16T05:34:57.479 INFO:teuthology.orchestra.run.smithi086.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-16T05:34:57.479 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-16T05:34:57.479 INFO:teuthology.orchestra.run.smithi086.stdout:smithi194 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6240002A400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-16T05:34:57.479 INFO:teuthology.orchestra.run.smithi086.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-16T05:34:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87ac0a3c40 msgr2=0x7f87ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87ac0a3c40 0x7f87ac0a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009aa0 tx=0x7f87b4009240 comp rx=0 tx=0).stop 2024-04-16T05:34:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4103d50 msgr2=0x7f87c4074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4103d50 0x7f87c4074ee0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f87b80098e0 tx=0x7f87b80057c0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 shutdown_connections 2024-04-16T05:34:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87c41046f0 0x7f87c4075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f87ac0a3c40 0x7f87ac0a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87c4102b50 0x7f87c4076890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 --2- 172.21.15.86:0/787398666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4103d50 0x7f87c4074ee0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 >> 172.21.15.86:0/787398666 conn(0x7f87c40fe0f0 msgr2=0x7f87c4107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:57.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 shutdown_connections 2024-04-16T05:34:57.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:34:57.481+0000 7f87c90db700 1 -- 172.21.15.86:0/787398666 wait complete. 2024-04-16T05:34:58.210 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-16T05:34:58.219 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:34:58.220 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-16T05:34:58.466 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:58.885 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:58 smithi086 ceph-mon[32028]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:58.885 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:34:58 smithi086 ceph-mon[32028]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:34:58.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:58 smithi194 ceph-mon[35865]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:58.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:34:58 smithi194 ceph-mon[35865]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:34:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:58 smithi196 ceph-mon[36373]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:34:58 smithi196 ceph-mon[36373]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.247+0000 7fcff113b700 1 -- 172.21.15.86:0/4169789792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 msgr2=0x7fcfec10afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.247+0000 7fcff113b700 1 --2- 172.21.15.86:0/4169789792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 0x7fcfec10afa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400b0d0 tx=0x7fcfe400b3e0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.247+0000 7fcff113b700 1 -- 172.21.15.86:0/4169789792 shutdown_connections 2024-04-16T05:35:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.247+0000 7fcff113b700 1 --2- 172.21.15.86:0/4169789792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 0x7fcfec10afa0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.247+0000 7fcff113b700 1 --2- 172.21.15.86:0/4169789792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec102b50 0x7fcfec102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.247+0000 7fcff113b700 1 --2- 172.21.15.86:0/4169789792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec103d50 0x7fcfec1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.247+0000 7fcff113b700 1 -- 172.21.15.86:0/4169789792 >> 172.21.15.86:0/4169789792 conn(0x7fcfec0fe0f0 msgr2=0x7fcfec100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.247+0000 7fcff113b700 1 -- 172.21.15.86:0/4169789792 shutdown_connections 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.247+0000 7fcff113b700 1 -- 172.21.15.86:0/4169789792 wait complete. 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcff113b700 1 Processor -- start 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcff113b700 1 -- start start 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcff113b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec102b50 0x7fcfec198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcff113b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec103d50 0x7fcfec199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcff113b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 0x7fcfec1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcff113b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec10e740 con 0x7fcfec1046f0 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcff113b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec10e5c0 con 0x7fcfec102b50 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcff113b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec10e8c0 con 0x7fcfec103d50 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcfeb59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 0x7fcfec1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcfeb59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 0x7fcfec1a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41754/0 (socket says 172.21.15.86:41754) 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.248+0000 7fcfeb59e700 1 -- 172.21.15.86:0/2009273219 learned_addr learned my addr 172.21.15.86:0/2009273219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfea59c700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec103d50 0x7fcfec199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfead9d700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec102b50 0x7fcfec198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfeb59e700 1 -- 172.21.15.86:0/2009273219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec103d50 msgr2=0x7fcfec199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfeb59e700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec103d50 0x7fcfec199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfeb59e700 1 -- 172.21.15.86:0/2009273219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec102b50 msgr2=0x7fcfec198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfeb59e700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec102b50 0x7fcfec198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfeb59e700 1 -- 172.21.15.86:0/2009273219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec1a08a0 con 0x7fcfec1046f0 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfead9d700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec102b50 0x7fcfec198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfeb59e700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 0x7fcfec1a01d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4006070 tx=0x7fcfe4007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfe3fff700 1 -- 172.21.15.86:0/2009273219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4021020 con 0x7fcfec1046f0 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec1a0b30 con 0x7fcfec1046f0 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfe3fff700 1 -- 172.21.15.86:0/2009273219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe401d950 con 0x7fcfec1046f0 2024-04-16T05:35:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcfe3fff700 1 -- 172.21.15.86:0/2009273219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4005390 con 0x7fcfec1046f0 2024-04-16T05:35:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.249+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec1a10c0 con 0x7fcfec1046f0 2024-04-16T05:35:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.250+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfec0697e0 con 0x7fcfec1046f0 2024-04-16T05:35:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.251+0000 7fcfe3fff700 1 -- 172.21.15.86:0/2009273219 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcfe4024020 con 0x7fcfec1046f0 2024-04-16T05:35:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.252+0000 7fcfe3fff700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfd80a3c40 0x7fcfd80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.252+0000 7fcfead9d700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfd80a3c40 0x7fcfd80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.252+0000 7fcfe3fff700 1 -- 172.21.15.86:0/2009273219 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcfe400f020 con 0x7fcfec1046f0 2024-04-16T05:35:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.252+0000 7fcfead9d700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfd80a3c40 0x7fcfd80a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4005950 tx=0x7fcfd40058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.257+0000 7fcfe3fff700 1 -- 172.21.15.86:0/2009273219 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcfe409e0c0 con 0x7fcfec1046f0 2024-04-16T05:35:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.500+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fcfec063700 con 0x7fcfec1046f0 2024-04-16T05:35:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.500+0000 7fcfe3fff700 1 -- 172.21.15.86:0/2009273219 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3102 (secure 0 0 0) 0x7fcfe409d9c0 con 0x7fcfec1046f0 2024-04-16T05:35:00.500 INFO:teuthology.orchestra.run.smithi086.stdout:e28 2024-04-16T05:35:00.500 INFO:teuthology.orchestra.run.smithi086.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T05:35:00.500 INFO:teuthology.orchestra.run.smithi086.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:legacy client fscid: 1 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:Filesystem 'cephfs' (1) 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:fs_name cephfs 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 28 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-16T05:32:54.852973+0000 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-04-16T05:33:18.613317+0000 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:tableserver 0 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:root 0 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:session_timeout 300 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:session_autoclose 300 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:max_file_size 1099511627776 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:max_xattr_size 65536 2024-04-16T05:35:00.501 INFO:teuthology.orchestra.run.smithi086.stdout:required_client_features {} 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure 0 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure_osd_epoch 0 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:max_mds 5 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:in 0,1,2,3,4 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:up {0=24482,1=24481,2=24463,3=24455,4=24445} 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:failed 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:damaged 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:stopped 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:data_pools [3,4] 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:metadata_pool 2 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:inline_data disabled 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:balancer 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:bal_rank_mask -1 2024-04-16T05:35:00.502 INFO:teuthology.orchestra.run.smithi086.stdout:standby_count_wanted 0 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:qdb_cluster leader: 24482 members: 24481,24482,24463,24455,24445 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b{0:24482} state up:active seq 3 addr [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.e{0:24419} state up:standby-replay seq 1 addr [v2:172.21.15.194:6834/2618844409,v1:172.21.15.194:6835/2618844409] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.i{1:24481} state up:active seq 10 addr [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.l{1:24403} state up:standby-replay seq 1 addr [v2:172.21.15.196:6832/1220860047,v1:172.21.15.196:6833/1220860047] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c{2:24463} state up:active seq 12 addr [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.k{2:24392} state up:standby-replay seq 1 addr [v2:172.21.15.194:6832/3318048466,v1:172.21.15.194:6833/3318048466] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.h{3:24455} state up:active seq 11 addr [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a{3:14631} state up:standby-replay seq 1 addr [v2:172.21.15.86:6840/1989684988,v1:172.21.15.86:6841/1989684988] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.f{4:24445} state up:active seq 13 addr [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.86:6838/1379378484,v1:172.21.15.86:6839/1379378484] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:Standby daemons: 2024-04-16T05:35:00.503 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:00.504 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.86:6834/2004229798,v1:172.21.15.86:6835/2004229798] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.504 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.86:6836/2865823959,v1:172.21.15.86:6837/2865823959] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:35:00.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.502+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfd80a3c40 msgr2=0x7fcfd80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.502+0000 7fcff113b700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfd80a3c40 0x7fcfd80a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4005950 tx=0x7fcfd40058e0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.502+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 msgr2=0x7fcfec1a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.502+0000 7fcff113b700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 0x7fcfec1a01d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4006070 tx=0x7fcfe4007d10 comp rx=0 tx=0).stop 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.503+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 shutdown_connections 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.503+0000 7fcff113b700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfec1046f0 0x7fcfec1a01d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.503+0000 7fcff113b700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec102b50 0x7fcfec198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.503+0000 7fcff113b700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fcfd80a3c40 0x7fcfd80a6100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.503+0000 7fcff113b700 1 --2- 172.21.15.86:0/2009273219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec103d50 0x7fcfec199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.503+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 >> 172.21.15.86:0/2009273219 conn(0x7fcfec0fe0f0 msgr2=0x7fcfec107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.503+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 shutdown_connections 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:00.503+0000 7fcff113b700 1 -- 172.21.15.86:0/2009273219 wait complete. 2024-04-16T05:35:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-16T05:35:00.872 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:00 smithi086 ceph-mon[32028]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:00.872 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:00 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2009273219' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T05:35:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:00 smithi194 ceph-mon[35865]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:00 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2009273219' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T05:35:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:00 smithi196 ceph-mon[36373]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:00 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2009273219' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T05:35:01.251 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-16T05:35:01.497 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:35:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:02 smithi194 ceph-mon[35865]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:02.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:02 smithi086 ceph-mon[32028]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:02 smithi196 ceph-mon[36373]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.247+0000 7f8b211fc700 1 -- 172.21.15.86:0/1920187207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c101a70 msgr2=0x7f8b1c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.247+0000 7f8b211fc700 1 --2- 172.21.15.86:0/1920187207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c101a70 0x7f8b1c101e90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8b040099c0 tx=0x7f8b04009cd0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.247+0000 7f8b211fc700 1 -- 172.21.15.86:0/1920187207 shutdown_connections 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.247+0000 7f8b211fc700 1 --2- 172.21.15.86:0/1920187207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b1c103610 0x7f8b1c109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.247+0000 7f8b211fc700 1 --2- 172.21.15.86:0/1920187207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c101a70 0x7f8b1c101e90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.247+0000 7f8b211fc700 1 --2- 172.21.15.86:0/1920187207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b1c102c70 0x7f8b1c1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.247+0000 7f8b211fc700 1 -- 172.21.15.86:0/1920187207 >> 172.21.15.86:0/1920187207 conn(0x7f8b1c0fd050 msgr2=0x7f8b1c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.247+0000 7f8b211fc700 1 -- 172.21.15.86:0/1920187207 shutdown_connections 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.247+0000 7f8b211fc700 1 -- 172.21.15.86:0/1920187207 wait complete. 2024-04-16T05:35:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.248+0000 7f8b211fc700 1 Processor -- start 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.248+0000 7f8b211fc700 1 -- start start 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b211fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b1c101a70 0x7f8b1c105a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b211fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b1c102c70 0x7f8b1c104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b211fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c103610 0x7f8b1c104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b211fc700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c10d730 con 0x7f8b1c103610 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b211fc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c10d5b0 con 0x7f8b1c101a70 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b211fc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c10d8b0 con 0x7f8b1c102c70 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1b59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c103610 0x7f8b1c104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1b59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c103610 0x7f8b1c104590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52878/0 (socket says 172.21.15.86:52878) 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1b59e700 1 -- 172.21.15.86:0/2705629151 learned_addr learned my addr 172.21.15.86:0/2705629151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1ad9d700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b1c101a70 0x7f8b1c105a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1a59c700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b1c102c70 0x7f8b1c104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1b59e700 1 -- 172.21.15.86:0/2705629151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b1c102c70 msgr2=0x7f8b1c104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1b59e700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b1c102c70 0x7f8b1c104050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1b59e700 1 -- 172.21.15.86:0/2705629151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b1c101a70 msgr2=0x7f8b1c105a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1b59e700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b1c101a70 0x7f8b1c105a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1b59e700 1 -- 172.21.15.86:0/2705629151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c104e40 con 0x7f8b1c103610 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b1b59e700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c103610 0x7f8b1c104590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b970 tx=0x7f8b1400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.249+0000 7f8b13fff700 1 -- 172.21.15.86:0/2705629151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14019070 con 0x7f8b1c103610 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.250+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c1a3060 con 0x7f8b1c103610 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.250+0000 7f8b13fff700 1 -- 172.21.15.86:0/2705629151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1400ce10 con 0x7f8b1c103610 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.250+0000 7f8b13fff700 1 -- 172.21.15.86:0/2705629151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14007370 con 0x7f8b1c103610 2024-04-16T05:35:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.250+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c1a35a0 con 0x7f8b1c103610 2024-04-16T05:35:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.250+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1c0697e0 con 0x7f8b1c103610 2024-04-16T05:35:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.252+0000 7f8b13fff700 1 -- 172.21.15.86:0/2705629151 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8b1401c070 con 0x7f8b1c103610 2024-04-16T05:35:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.252+0000 7f8b13fff700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8b080a3c40 0x7f8b080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.253+0000 7f8b1ad9d700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8b080a3c40 0x7f8b080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.253+0000 7f8b13fff700 1 -- 172.21.15.86:0/2705629151 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8b140cfff0 con 0x7f8b1c103610 2024-04-16T05:35:03.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.253+0000 7f8b1ad9d700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8b080a3c40 0x7f8b080a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b580 tx=0x7f8b04005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.256+0000 7f8b13fff700 1 -- 172.21.15.86:0/2705629151 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b1409c640 con 0x7f8b1c103610 2024-04-16T05:35:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.453+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f8b1c1a3a20 con 0x7f8b1c103610 2024-04-16T05:35:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.454+0000 7f8b13fff700 1 -- 172.21.15.86:0/2705629151 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5637 (secure 0 0 0) 0x7f8b1409bf40 con 0x7f8b1c103610 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 94 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-16T05:24:54.956200+0000 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-04-16T05:34:52.612662+0000 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:crush_version 27 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:full_ratio 0.95 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:backfillfull_ratio 0.9 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:nearfull_ratio 0.85 2024-04-16T05:35:03.453 INFO:teuthology.orchestra.run.smithi086.stdout:require_min_compat_client luminous 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:min_compat_client luminous 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:require_osd_release squid 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:stretch_mode_enabled false 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:max_osd 12 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6802/571788251,v1:172.21.15.86:6803/571788251] [v2:172.21.15.86:6804/571788251,v1:172.21.15.86:6805/571788251] exists,up 249ee95d-f444-4c1b-aeae-e88da2c920fc 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6800/2032185507,v1:172.21.15.194:6801/2032185507] [v2:172.21.15.194:6802/2032185507,v1:172.21.15.194:6803/2032185507] exists,up dfd22f89-9efb-4095-82f4-20ceb80af165 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/4091288332,v1:172.21.15.196:6801/4091288332] [v2:172.21.15.196:6802/4091288332,v1:172.21.15.196:6803/4091288332] exists,up 913b98ff-30fe-4214-877a-7a8cc4765ad6 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 up in weight 1 up_from 25 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6810/439908484,v1:172.21.15.86:6811/439908484] [v2:172.21.15.86:6812/439908484,v1:172.21.15.86:6813/439908484] exists,up 63b377ce-0b9e-48a5-85c1-6903db7c60bc 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6808/3924919713,v1:172.21.15.194:6809/3924919713] [v2:172.21.15.194:6810/3924919713,v1:172.21.15.194:6811/3924919713] exists,up 48c2fbf7-a6c1-4405-92e9-494e58e34298 2024-04-16T05:35:03.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/90234472,v1:172.21.15.196:6809/90234472] [v2:172.21.15.196:6810/90234472,v1:172.21.15.196:6811/90234472] exists,up 4e22bf80-acb0-4744-b7f5-94ae9032b5d8 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6818/983213207,v1:172.21.15.86:6819/983213207] [v2:172.21.15.86:6820/983213207,v1:172.21.15.86:6821/983213207] exists,up 5af57df8-4de1-4092-ae4e-9fb9a9e8b60b 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6816/1878006717,v1:172.21.15.194:6817/1878006717] [v2:172.21.15.194:6818/1878006717,v1:172.21.15.194:6819/1878006717] exists,up b5a76013-993e-4ff1-976d-002d09209edf 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 up in weight 1 up_from 59 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/3684108941,v1:172.21.15.196:6817/3684108941] [v2:172.21.15.196:6818/3684108941,v1:172.21.15.196:6819/3684108941] exists,up 4b3010ed-edc0-471d-b44b-e700733fe43b 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 up in weight 1 up_from 67 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6826/423267579,v1:172.21.15.86:6827/423267579] [v2:172.21.15.86:6828/423267579,v1:172.21.15.86:6829/423267579] exists,up 70f95451-1619-47ff-b12f-b8651c020a94 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6824/656449752,v1:172.21.15.194:6825/656449752] [v2:172.21.15.194:6826/656449752,v1:172.21.15.194:6827/656449752] exists,up d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 up in weight 1 up_from 80 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/2166568999,v1:172.21.15.196:6825/2166568999] [v2:172.21.15.196:6826/2166568999,v1:172.21.15.196:6827/2166568999] exists,up 66ee3110-b716-48c3-b625-d3a5f40beb5f 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.a [2,11] 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.f [8,11] 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.c [3,9] 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.14 [4,6] 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.17 [4,8] 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.1e [5,7] 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.25 [5,9] 2024-04-16T05:35:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.34 [4,9] 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.6 [5,8] 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.f [9,8] 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.19 [9,8] 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.1b [9,10] 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/1016266433 expires 2024-04-17T05:25:51.394705+0000 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/908662426 expires 2024-04-17T05:25:51.394705+0000 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/1420225001 expires 2024-04-17T05:25:51.394705+0000 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3914995291 expires 2024-04-17T05:25:24.654093+0000 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2966713555 expires 2024-04-17T05:25:24.654093+0000 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3037484112 expires 2024-04-17T05:25:24.654093+0000 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/1420225001 expires 2024-04-17T05:25:51.394705+0000 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/1904754808 expires 2024-04-17T05:25:24.654093+0000 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/1904754808 expires 2024-04-17T05:25:24.654093+0000 2024-04-16T05:35:03.456 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/661459728 expires 2024-04-17T05:25:51.394705+0000 2024-04-16T05:35:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8b080a3c40 msgr2=0x7f8b080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8b080a3c40 0x7f8b080a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b580 tx=0x7f8b04005fb0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c103610 msgr2=0x7f8b1c104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c103610 0x7f8b1c104590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b970 tx=0x7f8b1400bc80 comp rx=0 tx=0).stop 2024-04-16T05:35:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 shutdown_connections 2024-04-16T05:35:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1c103610 0x7f8b1c104590 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b1c101a70 0x7f8b1c105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f8b080a3c40 0x7f8b080a6100 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 --2- 172.21.15.86:0/2705629151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b1c102c70 0x7f8b1c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 >> 172.21.15.86:0/2705629151 conn(0x7f8b1c0fd050 msgr2=0x7f8b1c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:03.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 shutdown_connections 2024-04-16T05:35:03.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:03.456+0000 7f8b211fc700 1 -- 172.21.15.86:0/2705629151 wait complete. 2024-04-16T05:35:03.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:03 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2705629151' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T05:35:03.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:03 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2705629151' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T05:35:03.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:03 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2705629151' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T05:35:04.198 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-16T05:35:04.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:35:04.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.318+0000 7fea09cd9640 1 -- 172.21.15.86:0/720920645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04102090 msgr2=0x7fea04102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.318+0000 7fea09cd9640 1 --2- 172.21.15.86:0/720920645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04102090 0x7fea04102510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009900 tx=0x7fe9f402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:04.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.318+0000 7fea09cd9640 1 -- 172.21.15.86:0/720920645 shutdown_connections 2024-04-16T05:35:04.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.318+0000 7fea09cd9640 1 --2- 172.21.15.86:0/720920645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea04102a50 0x7fea04109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.318+0000 7fea09cd9640 1 --2- 172.21.15.86:0/720920645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04102090 0x7fea04102510 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.318+0000 7fea09cd9640 1 --2- 172.21.15.86:0/720920645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea04100f40 0x7fea04101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.318+0000 7fea09cd9640 1 -- 172.21.15.86:0/720920645 >> 172.21.15.86:0/720920645 conn(0x7fea040fc4b0 msgr2=0x7fea040fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.318+0000 7fea09cd9640 1 -- 172.21.15.86:0/720920645 shutdown_connections 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.318+0000 7fea09cd9640 1 -- 172.21.15.86:0/720920645 wait complete. 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea09cd9640 1 Processor -- start 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea09cd9640 1 -- start start 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea09cd9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04100f40 0x7fea04198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea09cd9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea04102090 0x7fea04198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea09cd9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea04102a50 0x7fea0419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea09cd9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea0410dd30 con 0x7fea04100f40 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea09cd9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea0410dbb0 con 0x7fea04102a50 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea09cd9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea0410deb0 con 0x7fea04102090 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04100f40 0x7fea04198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04100f40 0x7fea04198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52906/0 (socket says 172.21.15.86:52906) 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.319+0000 7fea037fe640 1 -- 172.21.15.86:0/4194119142 learned_addr learned my addr 172.21.15.86:0/4194119142 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea03fff640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea04102a50 0x7fea0419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea02ffd640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea04102090 0x7fea04198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea037fe640 1 -- 172.21.15.86:0/4194119142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea04102090 msgr2=0x7fea04198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea037fe640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea04102090 0x7fea04198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea037fe640 1 -- 172.21.15.86:0/4194119142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea04102a50 msgr2=0x7fea0419f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea037fe640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea04102a50 0x7fea0419f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea037fe640 1 -- 172.21.15.86:0/4194119142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0419ff60 con 0x7fea04100f40 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea037fe640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04100f40 0x7fea04198340 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00c890 tx=0x7fe9ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea00ff9640 1 -- 172.21.15.86:0/4194119142 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec004220 con 0x7fea04100f40 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea00ff9640 1 -- 172.21.15.86:0/4194119142 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9ec0043c0 con 0x7fea04100f40 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea00ff9640 1 -- 172.21.15.86:0/4194119142 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0049e0 con 0x7fea04100f40 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea041a0250 con 0x7fea04100f40 2024-04-16T05:35:04.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.320+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea041a0770 con 0x7fea04100f40 2024-04-16T05:35:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.321+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea04108250 con 0x7fea04100f40 2024-04-16T05:35:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.322+0000 7fea00ff9640 1 -- 172.21.15.86:0/4194119142 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe9ec0079b0 con 0x7fea04100f40 2024-04-16T05:35:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.323+0000 7fea00ff9640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.323+0000 7fea02ffd640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.323+0000 7fea00ff9640 1 -- 172.21.15.86:0/4194119142 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe9ec0d0820 con 0x7fea04100f40 2024-04-16T05:35:04.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.324+0000 7fea02ffd640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe9f402c7b0 tx=0x7fe9f4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:04.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.328+0000 7fea00ff9640 1 -- 172.21.15.86:0/4194119142 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9ec014030 con 0x7fea04100f40 2024-04-16T05:35:04.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.507+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea04062c00 con 0x7fea04100f40 2024-04-16T05:35:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.509+0000 7fea00ff9640 1 -- 172.21.15.86:0/4194119142 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24700 (secure 0 0 0) 0x7fe9ec09ce70 con 0x7fea04100f40 2024-04-16T05:35:04.508 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:04.508 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:34:52.612662+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:35:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe9f00a3bb0 msgr2=0x7fe9f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe9f402c7b0 tx=0x7fe9f4000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04100f40 msgr2=0x7fea04198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04100f40 0x7fea04198340 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00c890 tx=0x7fe9ec00cd50 comp rx=0 tx=0).stop 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 shutdown_connections 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea04102a50 0x7fea0419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea04102090 0x7fea04198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 --2- 172.21.15.86:0/4194119142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04100f40 0x7fea04198340 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 >> 172.21.15.86:0/4194119142 conn(0x7fea040fc4b0 msgr2=0x7fea04105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 shutdown_connections 2024-04-16T05:35:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.511+0000 7fea09cd9640 1 -- 172.21.15.86:0/4194119142 wait complete. 2024-04-16T05:35:04.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:35:04.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 -- 172.21.15.86:0/4254716952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 msgr2=0x7f67c410a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 --2- 172.21.15.86:0/4254716952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 0x7f67c410a530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009930 tx=0x7f67b402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 -- 172.21.15.86:0/4254716952 shutdown_connections 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 --2- 172.21.15.86:0/4254716952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 0x7f67c410a530 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 --2- 172.21.15.86:0/4254716952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67c41032e0 0x7f67c4103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 --2- 172.21.15.86:0/4254716952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c4100310 0x7f67c4100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 -- 172.21.15.86:0/4254716952 >> 172.21.15.86:0/4254716952 conn(0x7f67c40fbc40 msgr2=0x7f67c40fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 -- 172.21.15.86:0/4254716952 shutdown_connections 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 -- 172.21.15.86:0/4254716952 wait complete. 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.632+0000 7f67c9a42640 1 Processor -- start 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c9a42640 1 -- start start 2024-04-16T05:35:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c9a42640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67c4100310 0x7f67c4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c9a42640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c41032e0 0x7f67c4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c9a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 0x7f67c419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c9a42640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410dd00 con 0x7f67c4103ca0 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c9a42640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410db80 con 0x7f67c41032e0 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c9a42640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410de80 con 0x7f67c4100310 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 0x7f67c419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 0x7f67c419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52934/0 (socket says 172.21.15.86:52934) 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c37fe640 1 -- 172.21.15.86:0/113754740 learned_addr learned my addr 172.21.15.86:0/113754740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c27fc640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c41032e0 0x7f67c4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c2ffd640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67c4100310 0x7f67c4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c37fe640 1 -- 172.21.15.86:0/113754740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67c4100310 msgr2=0x7f67c4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c37fe640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67c4100310 0x7f67c4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c37fe640 1 -- 172.21.15.86:0/113754740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c41032e0 msgr2=0x7f67c4198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c37fe640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c41032e0 0x7f67c4198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c37fe640 1 -- 172.21.15.86:0/113754740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c419ff70 con 0x7f67c4103ca0 2024-04-16T05:35:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c27fc640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c41032e0 0x7f67c4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.633+0000 7f67c37fe640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 0x7f67c419f900 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f67b402c8e0 tx=0x7f67b4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.634+0000 7f67c8a40640 1 -- 172.21.15.86:0/113754740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4041070 con 0x7f67c4103ca0 2024-04-16T05:35:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.634+0000 7f67c8a40640 1 -- 172.21.15.86:0/113754740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b40054e0 con 0x7f67c4103ca0 2024-04-16T05:35:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.634+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41a0200 con 0x7f67c4103ca0 2024-04-16T05:35:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.634+0000 7f67c8a40640 1 -- 172.21.15.86:0/113754740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4034440 con 0x7f67c4103ca0 2024-04-16T05:35:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.634+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c41a06c0 con 0x7f67c4103ca0 2024-04-16T05:35:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.635+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c4109410 con 0x7f67c4103ca0 2024-04-16T05:35:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.636+0000 7f67c8a40640 1 -- 172.21.15.86:0/113754740 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f67b4044070 con 0x7f67c4103ca0 2024-04-16T05:35:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.636+0000 7f67c8a40640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67b00a3bb0 0x7f67b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.637+0000 7f67c8a40640 1 -- 172.21.15.86:0/113754740 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f67b40f28a0 con 0x7f67c4103ca0 2024-04-16T05:35:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.637+0000 7f67c2ffd640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67b00a3bb0 0x7f67b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.637+0000 7f67c2ffd640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67b00a3bb0 0x7f67b00a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f67c406a440 tx=0x7f67bc0098f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.640+0000 7f67c8a40640 1 -- 172.21.15.86:0/113754740 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67b40beef0 con 0x7f67c4103ca0 2024-04-16T05:35:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.819+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f67c402d620 con 0x7f67c4103ca0 2024-04-16T05:35:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.819+0000 7f67c8a40640 1 -- 172.21.15.86:0/113754740 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f67b40be7f0 con 0x7f67c4103ca0 2024-04-16T05:35:04.818 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67b00a3bb0 msgr2=0x7f67b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67b00a3bb0 0x7f67b00a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f67c406a440 tx=0x7f67bc0098f0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 msgr2=0x7f67c419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 0x7f67c419f900 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f67b402c8e0 tx=0x7f67b4005dc0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 shutdown_connections 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67c4103ca0 0x7f67c419f900 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f67b00a3bb0 0x7f67b00a6070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67c41032e0 0x7f67c4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 --2- 172.21.15.86:0/113754740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67c4100310 0x7f67c4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 >> 172.21.15.86:0/113754740 conn(0x7f67c40fbc40 msgr2=0x7f67c41065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 shutdown_connections 2024-04-16T05:35:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.821+0000 7f67c9a42640 1 -- 172.21.15.86:0/113754740 wait complete. 2024-04-16T05:35:04.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:04 smithi086 ceph-mon[32028]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:04.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:04 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4194119142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:04.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:35:04.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:04 smithi194 ceph-mon[35865]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:04 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4194119142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:04.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.940+0000 7f0c934fb640 1 -- 172.21.15.86:0/978146839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 msgr2=0x7f0c8c100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.940+0000 7f0c934fb640 1 --2- 172.21.15.86:0/978146839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 0x7f0c8c100730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009930 tx=0x7f0c8002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.940+0000 7f0c934fb640 1 -- 172.21.15.86:0/978146839 shutdown_connections 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.940+0000 7f0c934fb640 1 --2- 172.21.15.86:0/978146839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c103ca0 0x7f0c8c10a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.940+0000 7f0c934fb640 1 --2- 172.21.15.86:0/978146839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c8c1032e0 0x7f0c8c103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.940+0000 7f0c934fb640 1 --2- 172.21.15.86:0/978146839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 0x7f0c8c100730 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.940+0000 7f0c934fb640 1 -- 172.21.15.86:0/978146839 >> 172.21.15.86:0/978146839 conn(0x7f0c8c0fbc40 msgr2=0x7f0c8c0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.940+0000 7f0c934fb640 1 -- 172.21.15.86:0/978146839 shutdown_connections 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.940+0000 7f0c934fb640 1 -- 172.21.15.86:0/978146839 wait complete. 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c934fb640 1 Processor -- start 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c934fb640 1 -- start start 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c934fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 0x7f0c8c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c934fb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c8c1032e0 0x7f0c8c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c934fb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c103ca0 0x7f0c8c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c934fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c10dd40 con 0x7f0c8c100310 2024-04-16T05:35:04.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c934fb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c10dbc0 con 0x7f0c8c1032e0 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c934fb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c10dec0 con 0x7f0c8c103ca0 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c91297640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 0x7f0c8c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c91297640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 0x7f0c8c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52946/0 (socket says 172.21.15.86:52946) 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c91297640 1 -- 172.21.15.86:0/220264100 learned_addr learned my addr 172.21.15.86:0/220264100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c90a96640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c8c1032e0 0x7f0c8c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c91297640 1 -- 172.21.15.86:0/220264100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c103ca0 msgr2=0x7f0c8c19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c91297640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c103ca0 0x7f0c8c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c91297640 1 -- 172.21.15.86:0/220264100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c8c1032e0 msgr2=0x7f0c8c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c91297640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c8c1032e0 0x7f0c8c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c91297640 1 -- 172.21.15.86:0/220264100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c19ff80 con 0x7f0c8c100310 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c90a96640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c8c1032e0 0x7f0c8c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c91297640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 0x7f0c8c198360 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0c8002c7b0 tx=0x7f0c80004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c7e7fc640 1 -- 172.21.15.86:0/220264100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80041070 con 0x7f0c8c100310 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c7e7fc640 1 -- 172.21.15.86:0/220264100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c800054e0 con 0x7f0c8c100310 2024-04-16T05:35:04.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.941+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c1a0210 con 0x7f0c8c100310 2024-04-16T05:35:04.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.942+0000 7f0c7e7fc640 1 -- 172.21.15.86:0/220264100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800346a0 con 0x7f0c8c100310 2024-04-16T05:35:04.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.942+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c1a0750 con 0x7f0c8c100310 2024-04-16T05:35:04.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.943+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8c1093b0 con 0x7f0c8c100310 2024-04-16T05:35:04.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.944+0000 7f0c7e7fc640 1 -- 172.21.15.86:0/220264100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0c80044070 con 0x7f0c8c100310 2024-04-16T05:35:04.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.944+0000 7f0c7e7fc640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c780a3c30 0x7f0c780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:04.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.944+0000 7f0c90a96640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c780a3c30 0x7f0c780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:04.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.944+0000 7f0c7e7fc640 1 -- 172.21.15.86:0/220264100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0c800f2920 con 0x7f0c8c100310 2024-04-16T05:35:04.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.945+0000 7f0c90a96640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c780a3c30 0x7f0c780a60f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c199870 tx=0x7f0c8800a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:04.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:04.948+0000 7f0c7e7fc640 1 -- 172.21.15.86:0/220264100 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c800bef70 con 0x7f0c8c100310 2024-04-16T05:35:04.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:04 smithi196 ceph-mon[36373]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-04-16T05:35:04.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:04 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4194119142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.131+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0c8c062ca0 con 0x7f0c8c100310 2024-04-16T05:35:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.131+0000 7f0c7e7fc640 1 -- 172.21.15.86:0/220264100 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f0c800be870 con 0x7f0c8c100310 2024-04-16T05:35:05.130 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c780a3c30 msgr2=0x7f0c780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c780a3c30 0x7f0c780a60f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c199870 tx=0x7f0c8800a5c0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 msgr2=0x7f0c8c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 0x7f0c8c198360 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0c8002c7b0 tx=0x7f0c80004040 comp rx=0 tx=0).stop 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 shutdown_connections 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c103ca0 0x7f0c8c19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f0c780a3c30 0x7f0c780a60f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c8c1032e0 0x7f0c8c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 --2- 172.21.15.86:0/220264100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c100310 0x7f0c8c198360 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 >> 172.21.15.86:0/220264100 conn(0x7f0c8c0fbc40 msgr2=0x7f0c8c106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 shutdown_connections 2024-04-16T05:35:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.134+0000 7f0c934fb640 1 -- 172.21.15.86:0/220264100 wait complete. 2024-04-16T05:35:05.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.252+0000 7ff70798d640 1 -- 172.21.15.86:0/3132037669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700103250 msgr2=0x7ff7001036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.252+0000 7ff70798d640 1 --2- 172.21.15.86:0/3132037669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700103250 0x7ff7001036d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009900 tx=0x7ff6f002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.252+0000 7ff70798d640 1 -- 172.21.15.86:0/3132037669 shutdown_connections 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.252+0000 7ff70798d640 1 --2- 172.21.15.86:0/3132037669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff700103c10 0x7ff70010a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.252+0000 7ff70798d640 1 --2- 172.21.15.86:0/3132037669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700103250 0x7ff7001036d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.252+0000 7ff70798d640 1 --2- 172.21.15.86:0/3132037669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff700102060 0x7ff700102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.252+0000 7ff70798d640 1 -- 172.21.15.86:0/3132037669 >> 172.21.15.86:0/3132037669 conn(0x7ff7000fd5b0 msgr2=0x7ff7000ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.252+0000 7ff70798d640 1 -- 172.21.15.86:0/3132037669 shutdown_connections 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.252+0000 7ff70798d640 1 -- 172.21.15.86:0/3132037669 wait complete. 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff70798d640 1 Processor -- start 2024-04-16T05:35:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff70798d640 1 -- start start 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff70798d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700102060 0x7ff7001983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff70798d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff700103250 0x7ff7001988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff70798d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff700103c10 0x7ff70019f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff70798d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff70010dd50 con 0x7ff700102060 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff70798d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff70010dbd0 con 0x7ff700103250 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff70798d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff70010ded0 con 0x7ff700103c10 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff705729640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700102060 0x7ff7001983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff705729640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700102060 0x7ff7001983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52952/0 (socket says 172.21.15.86:52952) 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff705729640 1 -- 172.21.15.86:0/2176108535 learned_addr learned my addr 172.21.15.86:0/2176108535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff704f28640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff700103250 0x7ff7001988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff705729640 1 -- 172.21.15.86:0/2176108535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff700103c10 msgr2=0x7ff70019f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff705729640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff700103c10 0x7ff70019f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff705729640 1 -- 172.21.15.86:0/2176108535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff700103250 msgr2=0x7ff7001988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff705729640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff700103250 0x7ff7001988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.253+0000 7ff705729640 1 -- 172.21.15.86:0/2176108535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70019ffc0 con 0x7ff700102060 2024-04-16T05:35:05.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.254+0000 7ff705729640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700102060 0x7ff7001983a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00b450 tx=0x7ff6fc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:05.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.254+0000 7ff6f67fc640 1 -- 172.21.15.86:0/2176108535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc015020 con 0x7ff700102060 2024-04-16T05:35:05.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.254+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7001a02b0 con 0x7ff700102060 2024-04-16T05:35:05.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.254+0000 7ff6f67fc640 1 -- 172.21.15.86:0/2176108535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc011d70 con 0x7ff700102060 2024-04-16T05:35:05.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.254+0000 7ff6f67fc640 1 -- 172.21.15.86:0/2176108535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00f340 con 0x7ff700102060 2024-04-16T05:35:05.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.254+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7001a07d0 con 0x7ff700102060 2024-04-16T05:35:05.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.255+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff700109380 con 0x7ff700102060 2024-04-16T05:35:05.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.256+0000 7ff6f67fc640 1 -- 172.21.15.86:0/2176108535 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff6fc018020 con 0x7ff700102060 2024-04-16T05:35:05.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.256+0000 7ff6f67fc640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6ec0a3bb0 0x7ff6ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.257+0000 7ff704f28640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6ec0a3bb0 0x7ff6ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.257+0000 7ff6f67fc640 1 -- 172.21.15.86:0/2176108535 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff6fc0d0810 con 0x7ff700102060 2024-04-16T05:35:05.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.257+0000 7ff704f28640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6ec0a3bb0 0x7ff6ec0a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0006010 tx=0x7ff6f0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:05.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.260+0000 7ff6f67fc640 1 -- 172.21.15.86:0/2176108535 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6fc01f020 con 0x7ff700102060 2024-04-16T05:35:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.436+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff70002d620 con 0x7ff700102060 2024-04-16T05:35:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.437+0000 7ff6f67fc640 1 -- 172.21.15.86:0/2176108535 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7ff6fc00d040 con 0x7ff700102060 2024-04-16T05:35:05.435 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:35:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6ec0a3bb0 msgr2=0x7ff6ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6ec0a3bb0 0x7ff6ec0a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0006010 tx=0x7ff6f0000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700102060 msgr2=0x7ff7001983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700102060 0x7ff7001983a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00b450 tx=0x7ff6fc00b910 comp rx=0 tx=0).stop 2024-04-16T05:35:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 shutdown_connections 2024-04-16T05:35:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff700103c10 0x7ff70019f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff6ec0a3bb0 0x7ff6ec0a6070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff700103250 0x7ff7001988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 --2- 172.21.15.86:0/2176108535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff700102060 0x7ff7001983a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 >> 172.21.15.86:0/2176108535 conn(0x7ff7000fd5b0 msgr2=0x7ff700106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:05.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 shutdown_connections 2024-04-16T05:35:05.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.439+0000 7ff70798d640 1 -- 172.21.15.86:0/2176108535 wait complete. 2024-04-16T05:35:05.449 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:35:05.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.569+0000 7ff043fa8640 1 -- 172.21.15.86:0/331154489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c100300 msgr2=0x7ff03c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.569+0000 7ff043fa8640 1 --2- 172.21.15.86:0/331154489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c100300 0x7ff03c100720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff034009930 tx=0x7ff03402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:05.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.569+0000 7ff043fa8640 1 -- 172.21.15.86:0/331154489 shutdown_connections 2024-04-16T05:35:05.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.569+0000 7ff043fa8640 1 --2- 172.21.15.86:0/331154489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff03c103c90 0x7ff03c10a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.569+0000 7ff043fa8640 1 --2- 172.21.15.86:0/331154489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff03c1032d0 0x7ff03c103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.569+0000 7ff043fa8640 1 --2- 172.21.15.86:0/331154489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c100300 0x7ff03c100720 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.569+0000 7ff043fa8640 1 -- 172.21.15.86:0/331154489 >> 172.21.15.86:0/331154489 conn(0x7ff03c0fbc30 msgr2=0x7ff03c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:05.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.569+0000 7ff043fa8640 1 -- 172.21.15.86:0/331154489 shutdown_connections 2024-04-16T05:35:05.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.569+0000 7ff043fa8640 1 -- 172.21.15.86:0/331154489 wait complete. 2024-04-16T05:35:05.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff043fa8640 1 Processor -- start 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff043fa8640 1 -- start start 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff043fa8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff03c100300 0x7ff03c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff043fa8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff03c1032d0 0x7ff03c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff043fa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c103c90 0x7ff03c19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff043fa8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c10dd40 con 0x7ff03c103c90 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff043fa8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c10dbc0 con 0x7ff03c1032d0 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff043fa8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c10dec0 con 0x7ff03c100300 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff042545640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c103c90 0x7ff03c19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff042545640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c103c90 0x7ff03c19f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52978/0 (socket says 172.21.15.86:52978) 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff042545640 1 -- 172.21.15.86:0/3867882976 learned_addr learned my addr 172.21.15.86:0/3867882976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff041d44640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff03c100300 0x7ff03c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.570+0000 7ff041543640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff03c1032d0 0x7ff03c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff042545640 1 -- 172.21.15.86:0/3867882976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff03c100300 msgr2=0x7ff03c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff042545640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff03c100300 0x7ff03c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff042545640 1 -- 172.21.15.86:0/3867882976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff03c1032d0 msgr2=0x7ff03c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff042545640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff03c1032d0 0x7ff03c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff042545640 1 -- 172.21.15.86:0/3867882976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c1a0040 con 0x7ff03c103c90 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff042545640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c103c90 0x7ff03c19f9d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009fd0 tx=0x7ff02c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff03affd640 1 -- 172.21.15.86:0/3867882976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c017070 con 0x7ff03c103c90 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff03affd640 1 -- 172.21.15.86:0/3867882976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff02c003e50 con 0x7ff03c103c90 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c1a0330 con 0x7ff03c103c90 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff03affd640 1 -- 172.21.15.86:0/3867882976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0106c0 con 0x7ff03c103c90 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.571+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c1a0830 con 0x7ff03c103c90 2024-04-16T05:35:05.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.572+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03c1094d0 con 0x7ff03c103c90 2024-04-16T05:35:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.573+0000 7ff03affd640 1 -- 172.21.15.86:0/3867882976 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff02c003970 con 0x7ff03c103c90 2024-04-16T05:35:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.574+0000 7ff03affd640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff0240a3bb0 0x7ff0240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.574+0000 7ff03affd640 1 -- 172.21.15.86:0/3867882976 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff02c0d0470 con 0x7ff03c103c90 2024-04-16T05:35:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.574+0000 7ff041d44640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff0240a3bb0 0x7ff0240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.574+0000 7ff041d44640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff0240a3bb0 0x7ff0240a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff034009230 tx=0x7ff03402df90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:05.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.578+0000 7ff03affd640 1 -- 172.21.15.86:0/3867882976 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff02c09c470 con 0x7ff03c103c90 2024-04-16T05:35:05.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:05 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/113754740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:05.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:05 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/220264100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:05.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:05 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2176108535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.762+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff03c062c00 con 0x7ff03c103c90 2024-04-16T05:35:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.763+0000 7ff03affd640 1 -- 172.21.15.86:0/3867882976 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7ff02c0a0770 con 0x7ff03c103c90 2024-04-16T05:35:05.761 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff0240a3bb0 msgr2=0x7ff0240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff0240a3bb0 0x7ff0240a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff034009230 tx=0x7ff03402df90 comp rx=0 tx=0).stop 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c103c90 msgr2=0x7ff03c19f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c103c90 0x7ff03c19f9d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009fd0 tx=0x7ff02c00b260 comp rx=0 tx=0).stop 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 shutdown_connections 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c103c90 0x7ff03c19f9d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff0240a3bb0 0x7ff0240a6070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff03c1032d0 0x7ff03c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 --2- 172.21.15.86:0/3867882976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff03c100300 0x7ff03c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 >> 172.21.15.86:0/3867882976 conn(0x7ff03c0fbc30 msgr2=0x7ff03c1066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 shutdown_connections 2024-04-16T05:35:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.765+0000 7ff043fa8640 1 -- 172.21.15.86:0/3867882976 wait complete. 2024-04-16T05:35:05.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:35:05.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 -- 172.21.15.86:0/3161072706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c1018d0 msgr2=0x7f456c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 --2- 172.21.15.86:0/3161072706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c1018d0 0x7f456c101d30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4568009900 tx=0x7f456802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:05.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 -- 172.21.15.86:0/3161072706 shutdown_connections 2024-04-16T05:35:05.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 --2- 172.21.15.86:0/3161072706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f456c102270 0x7f456c108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 --2- 172.21.15.86:0/3161072706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c1018d0 0x7f456c101d30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 --2- 172.21.15.86:0/3161072706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f456c100820 0x7f456c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 -- 172.21.15.86:0/3161072706 >> 172.21.15.86:0/3161072706 conn(0x7f456c0fb7f0 msgr2=0x7f456c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 -- 172.21.15.86:0/3161072706 shutdown_connections 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 -- 172.21.15.86:0/3161072706 wait complete. 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 Processor -- start 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 -- start start 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c100820 0x7f456c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.881+0000 7f4573874640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f456c1018d0 0x7f456c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4573874640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f456c102270 0x7f456c19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4573874640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f456c10baf0 con 0x7f456c100820 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4573874640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f456c10b970 con 0x7f456c102270 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4573874640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f456c10bc70 con 0x7f456c1018d0 2024-04-16T05:35:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571610640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c100820 0x7f456c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571610640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c100820 0x7f456c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52988/0 (socket says 172.21.15.86:52988) 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571610640 1 -- 172.21.15.86:0/2598065004 learned_addr learned my addr 172.21.15.86:0/2598065004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571e11640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f456c102270 0x7f456c19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4570e0f640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f456c1018d0 0x7f456c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571610640 1 -- 172.21.15.86:0/2598065004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f456c1018d0 msgr2=0x7f456c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571610640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f456c1018d0 0x7f456c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571610640 1 -- 172.21.15.86:0/2598065004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f456c102270 msgr2=0x7f456c19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571610640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f456c102270 0x7f456c19f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571610640 1 -- 172.21.15.86:0/2598065004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c19ff90 con 0x7f456c100820 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4571610640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c100820 0x7f456c198370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f455c00b4a0 tx=0x7f455c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f45627fc640 1 -- 172.21.15.86:0/2598065004 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c014070 con 0x7f456c100820 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f45627fc640 1 -- 172.21.15.86:0/2598065004 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455c010af0 con 0x7f456c100820 2024-04-16T05:35:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c1a0280 con 0x7f456c100820 2024-04-16T05:35:05.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f45627fc640 1 -- 172.21.15.86:0/2598065004 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c017d10 con 0x7f456c100820 2024-04-16T05:35:05.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.882+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c1a07a0 con 0x7f456c100820 2024-04-16T05:35:05.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.883+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456c107a00 con 0x7f456c100820 2024-04-16T05:35:05.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.884+0000 7f45627fc640 1 -- 172.21.15.86:0/2598065004 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f455c017070 con 0x7f456c100820 2024-04-16T05:35:05.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.885+0000 7f45627fc640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f45580a3bb0 0x7f45580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:05.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.885+0000 7f4570e0f640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f45580a3bb0 0x7f45580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:05.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.885+0000 7f45627fc640 1 -- 172.21.15.86:0/2598065004 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f455c0cfe70 con 0x7f456c100820 2024-04-16T05:35:05.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.885+0000 7f4570e0f640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f45580a3bb0 0x7f45580a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f456802c7b0 tx=0x7f4568005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:05.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:05.889+0000 7f45627fc640 1 -- 172.21.15.86:0/2598065004 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f455c09be70 con 0x7f456c100820 2024-04-16T05:35:05.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:05 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/113754740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:05.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:05 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/220264100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:05.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:05 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2176108535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:05.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:05 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/113754740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:05.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:05 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/220264100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:05.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:05 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2176108535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:06.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.105+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f456c02d620 con 0x7f456c100820 2024-04-16T05:35:06.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.106+0000 7f45627fc640 1 -- 172.21.15.86:0/2598065004 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7f455c0a0170 con 0x7f456c100820 2024-04-16T05:35:06.105 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:06.105 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:33:18.613317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24463,24455,24445]},"id":1}]} 2024-04-16T05:35:06.105 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-16T05:35:06.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f45580a3bb0 msgr2=0x7f45580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f45580a3bb0 0x7f45580a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f456802c7b0 tx=0x7f4568005fd0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c100820 msgr2=0x7f456c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c100820 0x7f456c198370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f455c00b4a0 tx=0x7f455c00b960 comp rx=0 tx=0).stop 2024-04-16T05:35:06.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 shutdown_connections 2024-04-16T05:35:06.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f45580a3bb0 0x7f45580a6070 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f456c102270 0x7f456c19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f456c1018d0 0x7f456c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 --2- 172.21.15.86:0/2598065004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c100820 0x7f456c198370 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 >> 172.21.15.86:0/2598065004 conn(0x7f456c0fb7f0 msgr2=0x7f456c104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:06.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 shutdown_connections 2024-04-16T05:35:06.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.108+0000 7f4573874640 1 -- 172.21.15.86:0/2598065004 wait complete. 2024-04-16T05:35:06.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:35:06.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 -- 172.21.15.86:0/386195853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8103240 msgr2=0x7ff7b81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 --2- 172.21.15.86:0/386195853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8103240 0x7ff7b81036c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009900 tx=0x7ff7a002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:06.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 -- 172.21.15.86:0/386195853 shutdown_connections 2024-04-16T05:35:06.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 --2- 172.21.15.86:0/386195853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b8103c00 0x7ff7b810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 --2- 172.21.15.86:0/386195853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8103240 0x7ff7b81036c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 --2- 172.21.15.86:0/386195853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7b8102050 0x7ff7b8102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 -- 172.21.15.86:0/386195853 >> 172.21.15.86:0/386195853 conn(0x7ff7b80fd5e0 msgr2=0x7ff7b80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 -- 172.21.15.86:0/386195853 shutdown_connections 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 -- 172.21.15.86:0/386195853 wait complete. 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 Processor -- start 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.224+0000 7ff7be7db640 1 -- start start 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7be7db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8102050 0x7ff7b81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7be7db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7b8103240 0x7ff7b81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7be7db640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b8103c00 0x7ff7b819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7be7db640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b810dd40 con 0x7ff7b8102050 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7be7db640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b810dbc0 con 0x7ff7b8103c00 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7be7db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b810dec0 con 0x7ff7b8103240 2024-04-16T05:35:06.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8102050 0x7ff7b81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8102050 0x7ff7b81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53012/0 (socket says 172.21.15.86:53012) 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b7fff640 1 -- 172.21.15.86:0/913196054 learned_addr learned my addr 172.21.15.86:0/913196054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7bcd78640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b8103c00 0x7ff7b819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b77fe640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7b8103240 0x7ff7b81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b7fff640 1 -- 172.21.15.86:0/913196054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7b8103240 msgr2=0x7ff7b81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b7fff640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7b8103240 0x7ff7b81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b7fff640 1 -- 172.21.15.86:0/913196054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b8103c00 msgr2=0x7ff7b819f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b7fff640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b8103c00 0x7ff7b819f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b7fff640 1 -- 172.21.15.86:0/913196054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b819ffd0 con 0x7ff7b8102050 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7bcd78640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b8103c00 0x7ff7b819f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b7fff640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8102050 0x7ff7b81983b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800b450 tx=0x7ff7a800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b57fa640 1 -- 172.21.15.86:0/913196054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8015020 con 0x7ff7b8102050 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b81a02c0 con 0x7ff7b8102050 2024-04-16T05:35:06.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b57fa640 1 -- 172.21.15.86:0/913196054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a8011d70 con 0x7ff7b8102050 2024-04-16T05:35:06.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7b57fa640 1 -- 172.21.15.86:0/913196054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a800f340 con 0x7ff7b8102050 2024-04-16T05:35:06.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.225+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b81a07e0 con 0x7ff7b8102050 2024-04-16T05:35:06.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.226+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b8109370 con 0x7ff7b8102050 2024-04-16T05:35:06.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.227+0000 7ff7b57fa640 1 -- 172.21.15.86:0/913196054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff7a8018020 con 0x7ff7b8102050 2024-04-16T05:35:06.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.228+0000 7ff7b57fa640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff7a40a3bb0 0x7ff7a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.228+0000 7ff7b57fa640 1 -- 172.21.15.86:0/913196054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff7a80d02c0 con 0x7ff7b8102050 2024-04-16T05:35:06.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.228+0000 7ff7b77fe640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff7a40a3bb0 0x7ff7a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:06.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.229+0000 7ff7b77fe640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff7a40a3bb0 0x7ff7a40a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0006010 tx=0x7ff7a0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.233+0000 7ff7b57fa640 1 -- 172.21.15.86:0/913196054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7a801f020 con 0x7ff7b8102050 2024-04-16T05:35:06.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.411+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7b8062c00 con 0x7ff7b8102050 2024-04-16T05:35:06.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.413+0000 7ff7b57fa640 1 -- 172.21.15.86:0/913196054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24700 (secure 0 0 0) 0x7ff7a800d040 con 0x7ff7b8102050 2024-04-16T05:35:06.412 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:06.412 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:34:52.612662+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:35:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff7a40a3bb0 msgr2=0x7ff7a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff7a40a3bb0 0x7ff7a40a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0006010 tx=0x7ff7a0000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8102050 msgr2=0x7ff7b81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8102050 0x7ff7b81983b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800b450 tx=0x7ff7a800b910 comp rx=0 tx=0).stop 2024-04-16T05:35:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 shutdown_connections 2024-04-16T05:35:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7ff7a40a3bb0 0x7ff7a40a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b8103c00 0x7ff7b819f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7b8103240 0x7ff7b81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 --2- 172.21.15.86:0/913196054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b8102050 0x7ff7b81983b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 >> 172.21.15.86:0/913196054 conn(0x7ff7b80fd5e0 msgr2=0x7ff7b8106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:06.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 shutdown_connections 2024-04-16T05:35:06.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.415+0000 7ff7be7db640 1 -- 172.21.15.86:0/913196054 wait complete. 2024-04-16T05:35:06.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:35:06.426 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 -- 172.21.15.86:0/168297585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c102090 msgr2=0x7f095c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 --2- 172.21.15.86:0/168297585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c102090 0x7f095c102510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0950009900 tx=0x7f095002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 -- 172.21.15.86:0/168297585 shutdown_connections 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 --2- 172.21.15.86:0/168297585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f095c102a50 0x7f095c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 --2- 172.21.15.86:0/168297585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c102090 0x7f095c102510 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 --2- 172.21.15.86:0/168297585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c100f40 0x7f095c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 -- 172.21.15.86:0/168297585 >> 172.21.15.86:0/168297585 conn(0x7f095c0fc4b0 msgr2=0x7f095c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 -- 172.21.15.86:0/168297585 shutdown_connections 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 -- 172.21.15.86:0/168297585 wait complete. 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.534+0000 7f09621a8640 1 Processor -- start 2024-04-16T05:35:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f09621a8640 1 -- start start 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f09621a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c100f40 0x7f095c069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f09621a8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f095c102090 0x7f095c19d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f09621a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c102a50 0x7f095c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f09621a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10cc40 con 0x7f095c100f40 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f09621a8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10cac0 con 0x7f095c102090 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f09621a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10cdc0 con 0x7f095c102a50 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c100f40 0x7f095c069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c100f40 0x7f095c069d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53040/0 (socket says 172.21.15.86:53040) 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095b7fe640 1 -- 172.21.15.86:0/296617085 learned_addr learned my addr 172.21.15.86:0/296617085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095affd640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f095c102090 0x7f095c19d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095b7fe640 1 -- 172.21.15.86:0/296617085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c102a50 msgr2=0x7f095c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095b7fe640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c102a50 0x7f095c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095b7fe640 1 -- 172.21.15.86:0/296617085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f095c102090 msgr2=0x7f095c19d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095b7fe640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f095c102090 0x7f095c19d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095b7fe640 1 -- 172.21.15.86:0/296617085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1a0060 con 0x7f095c100f40 2024-04-16T05:35:06.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095affd640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f095c102090 0x7f095c19d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:06.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f095b7fe640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c100f40 0x7f095c069d00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f094c00b4a0 tx=0x7f094c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:06.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.535+0000 7f0958ff9640 1 -- 172.21.15.86:0/296617085 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c014070 con 0x7f095c100f40 2024-04-16T05:35:06.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.536+0000 7f0958ff9640 1 -- 172.21.15.86:0/296617085 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f094c010af0 con 0x7f095c100f40 2024-04-16T05:35:06.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.536+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1a02f0 con 0x7f095c100f40 2024-04-16T05:35:06.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.536+0000 7f0958ff9640 1 -- 172.21.15.86:0/296617085 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c017d10 con 0x7f095c100f40 2024-04-16T05:35:06.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.536+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1a07f0 con 0x7f095c100f40 2024-04-16T05:35:06.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.537+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f095c108270 con 0x7f095c100f40 2024-04-16T05:35:06.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.538+0000 7f0958ff9640 1 -- 172.21.15.86:0/296617085 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f094c017070 con 0x7f095c100f40 2024-04-16T05:35:06.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.538+0000 7f0958ff9640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f09440a3bb0 0x7f09440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.538+0000 7f095affd640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f09440a3bb0 0x7f09440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:06.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.538+0000 7f0958ff9640 1 -- 172.21.15.86:0/296617085 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f094c0cfc20 con 0x7f095c100f40 2024-04-16T05:35:06.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.539+0000 7f095affd640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f09440a3bb0 0x7f09440a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f095002c170 tx=0x7f0950000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:06.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.543+0000 7f0958ff9640 1 -- 172.21.15.86:0/296617085 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f094c09c270 con 0x7f095c100f40 2024-04-16T05:35:06.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:06 smithi086 ceph-mon[32028]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s; 1 B/s, 0 objects/s recovering 2024-04-16T05:35:06.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:06 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3867882976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:06.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:06 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2598065004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:06 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/913196054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:06.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.721+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f095c062c00 con 0x7f095c100f40 2024-04-16T05:35:06.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.722+0000 7f0958ff9640 1 -- 172.21.15.86:0/296617085 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f094c09bb70 con 0x7f095c100f40 2024-04-16T05:35:06.720 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f09440a3bb0 msgr2=0x7f09440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f09440a3bb0 0x7f09440a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f095002c170 tx=0x7f0950000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c100f40 msgr2=0x7f095c069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c100f40 0x7f095c069d00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f094c00b4a0 tx=0x7f094c00b960 comp rx=0 tx=0).stop 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 shutdown_connections 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c102a50 0x7f095c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f09440a3bb0 0x7f09440a6070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f095c102090 0x7f095c19d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 --2- 172.21.15.86:0/296617085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095c100f40 0x7f095c069d00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 >> 172.21.15.86:0/296617085 conn(0x7f095c0fc4b0 msgr2=0x7f095c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 shutdown_connections 2024-04-16T05:35:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.724+0000 7f09621a8640 1 -- 172.21.15.86:0/296617085 wait complete. 2024-04-16T05:35:06.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:35:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.842+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/3327410991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94103c00 msgr2=0x7fbf9410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.842+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/3327410991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94103c00 0x7fbf9410a490 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009ca0 tx=0x7fbf8c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.842+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/3327410991 shutdown_connections 2024-04-16T05:35:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.842+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/3327410991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94103c00 0x7fbf9410a490 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.842+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/3327410991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94103240 0x7fbf941036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.842+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/3327410991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf94102050 0x7fbf94102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.842+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/3327410991 >> 172.21.15.86:0/3327410991 conn(0x7fbf940fd5e0 msgr2=0x7fbf940ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.842+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/3327410991 shutdown_connections 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.842+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/3327410991 wait complete. 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf9a4f2640 1 Processor -- start 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf9a4f2640 1 -- start start 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf9a4f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94102050 0x7fbf94198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf9a4f2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94103240 0x7fbf94198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf9a4f2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf94103c00 0x7fbf9419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf9a4f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410dd20 con 0x7fbf94102050 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf9a4f2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410dba0 con 0x7fbf94103c00 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf9a4f2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410dea0 con 0x7fbf94103240 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf93fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94102050 0x7fbf94198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf93fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94102050 0x7fbf94198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53060/0 (socket says 172.21.15.86:53060) 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf93fff640 1 -- 172.21.15.86:0/739644464 learned_addr learned my addr 172.21.15.86:0/739644464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.843+0000 7fbf93fff640 1 -- 172.21.15.86:0/739644464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94103240 msgr2=0x7fbf94198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf93fff640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94103240 0x7fbf94198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf93fff640 1 -- 172.21.15.86:0/739644464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf94103c00 msgr2=0x7fbf9419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf93fff640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf94103c00 0x7fbf9419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf93fff640 1 -- 172.21.15.86:0/739644464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9419ff20 con 0x7fbf94102050 2024-04-16T05:35:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf93fff640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94102050 0x7fbf94198300 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c970 tx=0x7fbf8400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf917fa640 1 -- 172.21.15.86:0/739644464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84007c10 con 0x7fbf94102050 2024-04-16T05:35:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf941a0210 con 0x7fbf94102050 2024-04-16T05:35:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf917fa640 1 -- 172.21.15.86:0/739644464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf84007db0 con 0x7fbf94102050 2024-04-16T05:35:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf917fa640 1 -- 172.21.15.86:0/739644464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400f940 con 0x7fbf94102050 2024-04-16T05:35:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.844+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf941a0760 con 0x7fbf94102050 2024-04-16T05:35:06.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.845+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf941092a0 con 0x7fbf94102050 2024-04-16T05:35:06.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.846+0000 7fbf917fa640 1 -- 172.21.15.86:0/739644464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbf8400fae0 con 0x7fbf94102050 2024-04-16T05:35:06.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.846+0000 7fbf917fa640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf7c0a3bb0 0x7fbf7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:06.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.847+0000 7fbf937fe640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf7c0a3bb0 0x7fbf7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:06.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.847+0000 7fbf917fa640 1 -- 172.21.15.86:0/739644464 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbf840d0be0 con 0x7fbf94102050 2024-04-16T05:35:06.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.847+0000 7fbf937fe640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf7c0a3bb0 0x7fbf7c0a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbf94199810 tx=0x7fbf88006c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:06.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:06.851+0000 7fbf917fa640 1 -- 172.21.15.86:0/739644464 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf8409d2c0 con 0x7fbf94102050 2024-04-16T05:35:06.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:06 smithi194 ceph-mon[35865]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s; 1 B/s, 0 objects/s recovering 2024-04-16T05:35:06.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:06 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3867882976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:06.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:06 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2598065004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:06.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:06 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/913196054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:06 smithi196 ceph-mon[36373]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s; 1 B/s, 0 objects/s recovering 2024-04-16T05:35:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:06 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3867882976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:06 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2598065004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:06 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/913196054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.029+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbf94062d00 con 0x7fbf94102050 2024-04-16T05:35:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.029+0000 7fbf917fa640 1 -- 172.21.15.86:0/739644464 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fbf8409cbc0 con 0x7fbf94102050 2024-04-16T05:35:07.028 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:07.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.031+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf7c0a3bb0 msgr2=0x7fbf7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.031+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf7c0a3bb0 0x7fbf7c0a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbf94199810 tx=0x7fbf88006c90 comp rx=0 tx=0).stop 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.031+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94102050 msgr2=0x7fbf94198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.031+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94102050 0x7fbf94198300 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c970 tx=0x7fbf8400ce30 comp rx=0 tx=0).stop 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.031+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 shutdown_connections 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.032+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fbf7c0a3bb0 0x7fbf7c0a6070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.032+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf94103c00 0x7fbf9419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.032+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94103240 0x7fbf94198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.032+0000 7fbf9a4f2640 1 --2- 172.21.15.86:0/739644464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf94102050 0x7fbf94198300 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.032+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 >> 172.21.15.86:0/739644464 conn(0x7fbf940fd5e0 msgr2=0x7fbf94106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.032+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 shutdown_connections 2024-04-16T05:35:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.032+0000 7fbf9a4f2640 1 -- 172.21.15.86:0/739644464 wait complete. 2024-04-16T05:35:07.042 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:35:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 -- 172.21.15.86:0/1741798575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea0100310 msgr2=0x7f6ea0100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1741798575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea0100310 0x7f6ea0100730 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009930 tx=0x7f6e8802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 -- 172.21.15.86:0/1741798575 shutdown_connections 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1741798575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ea0103ca0 0x7f6ea010a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1741798575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea01032e0 0x7f6ea0103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1741798575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea0100310 0x7f6ea0100730 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 -- 172.21.15.86:0/1741798575 >> 172.21.15.86:0/1741798575 conn(0x7f6ea00fbc40 msgr2=0x7f6ea00fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 -- 172.21.15.86:0/1741798575 shutdown_connections 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 -- 172.21.15.86:0/1741798575 wait complete. 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 Processor -- start 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.151+0000 7f6ea6635640 1 -- start start 2024-04-16T05:35:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6ea6635640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea0100310 0x7f6ea0102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6ea6635640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea01032e0 0x7f6ea0101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6ea6635640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ea0103ca0 0x7f6ea01017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6ea6635640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea010dd00 con 0x7f6ea01032e0 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6ea6635640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea010db80 con 0x7f6ea0103ca0 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6ea6635640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea010de80 con 0x7f6ea0100310 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea01032e0 0x7f6ea0101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea01032e0 0x7f6ea0101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53068/0 (socket says 172.21.15.86:53068) 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9f7fe640 1 -- 172.21.15.86:0/1944914793 learned_addr learned my addr 172.21.15.86:0/1944914793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6ea4bd2640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ea0103ca0 0x7f6ea01017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9f7fe640 1 -- 172.21.15.86:0/1944914793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea0100310 msgr2=0x7f6ea0102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9ffff640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea0100310 0x7f6ea0102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9f7fe640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea0100310 0x7f6ea0102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9f7fe640 1 -- 172.21.15.86:0/1944914793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ea0103ca0 msgr2=0x7f6ea01017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9f7fe640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ea0103ca0 0x7f6ea01017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9f7fe640 1 -- 172.21.15.86:0/1944914793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea0102060 con 0x7f6ea01032e0 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6ea4bd2640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ea0103ca0 0x7f6ea01017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9f7fe640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea01032e0 0x7f6ea0101290 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000d770 tx=0x7f6e9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9d7fa640 1 -- 172.21.15.86:0/1944914793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90017070 con 0x7f6ea01032e0 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9d7fa640 1 -- 172.21.15.86:0/1944914793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e90005500 con 0x7f6ea01032e0 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6e9d7fa640 1 -- 172.21.15.86:0/1944914793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90004e00 con 0x7f6ea01032e0 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.152+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea01a2510 con 0x7f6ea01032e0 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.153+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea01a2a60 con 0x7f6ea01032e0 2024-04-16T05:35:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.154+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea0109500 con 0x7f6ea01032e0 2024-04-16T05:35:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.154+0000 7f6e9d7fa640 1 -- 172.21.15.86:0/1944914793 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6e90005000 con 0x7f6ea01032e0 2024-04-16T05:35:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.155+0000 7f6e9d7fa640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.155+0000 7f6e9d7fa640 1 -- 172.21.15.86:0/1944914793 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6e900d0210 con 0x7f6ea01032e0 2024-04-16T05:35:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.155+0000 7f6e9ffff640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.156+0000 7f6e9ffff640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6e8802c170 tx=0x7f6e88005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:07.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.160+0000 7f6e9d7fa640 1 -- 172.21.15.86:0/1944914793 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e9009c860 con 0x7f6ea01032e0 2024-04-16T05:35:07.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.338+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6ea0062c00 con 0x7f6ea01032e0 2024-04-16T05:35:07.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.338+0000 7f6e9d7fa640 1 -- 172.21.15.86:0/1944914793 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f6e9009c160 con 0x7f6ea01032e0 2024-04-16T05:35:07.337 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:35:07.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e8c0a3c30 msgr2=0x7f6e8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6e8802c170 tx=0x7f6e88005fd0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea01032e0 msgr2=0x7f6ea0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea01032e0 0x7f6ea0101290 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000d770 tx=0x7f6e9000dc30 comp rx=0 tx=0).stop 2024-04-16T05:35:07.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 shutdown_connections 2024-04-16T05:35:07.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ea0103ca0 0x7f6ea01017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ea01032e0 0x7f6ea0101290 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 --2- 172.21.15.86:0/1944914793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea0100310 0x7f6ea0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 >> 172.21.15.86:0/1944914793 conn(0x7f6ea00fbc40 msgr2=0x7f6ea01066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:07.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.340+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 shutdown_connections 2024-04-16T05:35:07.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.341+0000 7f6ea6635640 1 -- 172.21.15.86:0/1944914793 wait complete. 2024-04-16T05:35:07.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:35:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.460+0000 7fa1cefba640 1 -- 172.21.15.86:0/2191914377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c81032e0 msgr2=0x7fa1c8103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.460+0000 7fa1cefba640 1 --2- 172.21.15.86:0/2191914377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c81032e0 0x7fa1c8103760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009930 tx=0x7fa1b402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.460+0000 7fa1cefba640 1 -- 172.21.15.86:0/2191914377 shutdown_connections 2024-04-16T05:35:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.460+0000 7fa1cefba640 1 --2- 172.21.15.86:0/2191914377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c8103ca0 0x7fa1c810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.460+0000 7fa1cefba640 1 --2- 172.21.15.86:0/2191914377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c81032e0 0x7fa1c8103760 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.460+0000 7fa1cefba640 1 --2- 172.21.15.86:0/2191914377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c8100310 0x7fa1c8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.460+0000 7fa1cefba640 1 -- 172.21.15.86:0/2191914377 >> 172.21.15.86:0/2191914377 conn(0x7fa1c80fbc40 msgr2=0x7fa1c80fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.460+0000 7fa1cefba640 1 -- 172.21.15.86:0/2191914377 shutdown_connections 2024-04-16T05:35:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.460+0000 7fa1cefba640 1 -- 172.21.15.86:0/2191914377 wait complete. 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cefba640 1 Processor -- start 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cefba640 1 -- start start 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cefba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c8100310 0x7fa1c819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cefba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c81032e0 0x7fa1c819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cefba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c8103ca0 0x7fa1c81a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cefba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c810dd00 con 0x7fa1c8103ca0 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cefba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c810db80 con 0x7fa1c8100310 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cefba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c810de80 con 0x7fa1c81032e0 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cd557640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c8103ca0 0x7fa1c81a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cd557640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c8103ca0 0x7fa1c81a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53082/0 (socket says 172.21.15.86:53082) 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cd557640 1 -- 172.21.15.86:0/1075797629 learned_addr learned my addr 172.21.15.86:0/1075797629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1ccd56640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c8100310 0x7fa1c819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1c7fff640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c81032e0 0x7fa1c819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cd557640 1 -- 172.21.15.86:0/1075797629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c81032e0 msgr2=0x7fa1c819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cd557640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c81032e0 0x7fa1c819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cd557640 1 -- 172.21.15.86:0/1075797629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c8100310 msgr2=0x7fa1c819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cd557640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c8100310 0x7fa1c819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cd557640 1 -- 172.21.15.86:0/1075797629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c81a2190 con 0x7fa1c8103ca0 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1cd557640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c8103ca0 0x7fa1c81a1b20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000cad0 tx=0x7fa1c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.461+0000 7fa1c5ffb640 1 -- 172.21.15.86:0/1075797629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000ae20 con 0x7fa1c8103ca0 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.462+0000 7fa1c5ffb640 1 -- 172.21.15.86:0/1075797629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c0005020 con 0x7fa1c8103ca0 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.462+0000 7fa1c5ffb640 1 -- 172.21.15.86:0/1075797629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000f6f0 con 0x7fa1c8103ca0 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.462+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c81a2480 con 0x7fa1c8103ca0 2024-04-16T05:35:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.462+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c81a29a0 con 0x7fa1c8103ca0 2024-04-16T05:35:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.463+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c81a2c60 con 0x7fa1c8103ca0 2024-04-16T05:35:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.464+0000 7fa1c5ffb640 1 -- 172.21.15.86:0/1075797629 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa1c00051d0 con 0x7fa1c8103ca0 2024-04-16T05:35:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.464+0000 7fa1c5ffb640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.464+0000 7fa1ccd56640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.464+0000 7fa1c5ffb640 1 -- 172.21.15.86:0/1075797629 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1c00d0790 con 0x7fa1c8103ca0 2024-04-16T05:35:07.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.465+0000 7fa1ccd56640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa1c806a420 tx=0x7fa1bc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:07.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.469+0000 7fa1c5ffb640 1 -- 172.21.15.86:0/1075797629 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1c0014030 con 0x7fa1c8103ca0 2024-04-16T05:35:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.651+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa1c81a2e30 con 0x7fa1c8103ca0 2024-04-16T05:35:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.651+0000 7fa1c5ffb640 1 -- 172.21.15.86:0/1075797629 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fa1c009cde0 con 0x7fa1c8103ca0 2024-04-16T05:35:07.650 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1b80a3c30 msgr2=0x7fa1b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa1c806a420 tx=0x7fa1bc005f90 comp rx=0 tx=0).stop 2024-04-16T05:35:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c8103ca0 msgr2=0x7fa1c81a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c8103ca0 0x7fa1c81a1b20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000cad0 tx=0x7fa1c000cf90 comp rx=0 tx=0).stop 2024-04-16T05:35:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 shutdown_connections 2024-04-16T05:35:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c8103ca0 0x7fa1c81a1b20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c81032e0 0x7fa1c819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 --2- 172.21.15.86:0/1075797629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c8100310 0x7fa1c819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 >> 172.21.15.86:0/1075797629 conn(0x7fa1c80fbc40 msgr2=0x7fa1c81010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 shutdown_connections 2024-04-16T05:35:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.653+0000 7fa1cefba640 1 -- 172.21.15.86:0/1075797629 wait complete. 2024-04-16T05:35:07.665 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:35:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.770+0000 7f692837c640 1 -- 172.21.15.86:0/1023600009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920107dc0 msgr2=0x7f692010a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.770+0000 7f692837c640 1 --2- 172.21.15.86:0/1023600009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920107dc0 0x7f692010a1b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6918009ca0 tx=0x7f691802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 -- 172.21.15.86:0/1023600009 shutdown_connections 2024-04-16T05:35:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 --2- 172.21.15.86:0/1023600009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920107dc0 0x7f692010a1b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 --2- 172.21.15.86:0/1023600009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6920105490 0x7f6920107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 --2- 172.21.15.86:0/1023600009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6920102b60 0x7f6920104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 -- 172.21.15.86:0/1023600009 >> 172.21.15.86:0/1023600009 conn(0x7f69200fc4b0 msgr2=0x7f69200fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 -- 172.21.15.86:0/1023600009 shutdown_connections 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 -- 172.21.15.86:0/1023600009 wait complete. 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 Processor -- start 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 -- start start 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6920102b60 0x7f6920198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920105490 0x7f6920198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6920107dc0 0x7f692019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f692010dcf0 con 0x7f6920105490 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f692010db70 con 0x7f6920102b60 2024-04-16T05:35:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.771+0000 7f692837c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f692010de70 con 0x7f6920107dc0 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6925917640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920105490 0x7f6920198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6925917640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920105490 0x7f6920198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53106/0 (socket says 172.21.15.86:53106) 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6925917640 1 -- 172.21.15.86:0/2240320856 learned_addr learned my addr 172.21.15.86:0/2240320856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6926118640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6920102b60 0x7f6920198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6926919640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6920107dc0 0x7f692019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6925917640 1 -- 172.21.15.86:0/2240320856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6920107dc0 msgr2=0x7f692019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6925917640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6920107dc0 0x7f692019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6925917640 1 -- 172.21.15.86:0/2240320856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6920102b60 msgr2=0x7f6920198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6925917640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6920102b60 0x7f6920198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6925917640 1 -- 172.21.15.86:0/2240320856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692019ff40 con 0x7f6920105490 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f6925917640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920105490 0x7f6920198860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f691000d770 tx=0x7f691000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f69177fe640 1 -- 172.21.15.86:0/2240320856 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910017070 con 0x7f6920105490 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f69177fe640 1 -- 172.21.15.86:0/2240320856 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6910009c30 con 0x7f6920105490 2024-04-16T05:35:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69201a0230 con 0x7f6920105490 2024-04-16T05:35:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f69177fe640 1 -- 172.21.15.86:0/2240320856 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691000f620 con 0x7f6920105490 2024-04-16T05:35:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.772+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69201a0780 con 0x7f6920105490 2024-04-16T05:35:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.773+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69200fdfa0 con 0x7f6920105490 2024-04-16T05:35:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.774+0000 7f69177fe640 1 -- 172.21.15.86:0/2240320856 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f691001a070 con 0x7f6920105490 2024-04-16T05:35:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.775+0000 7f69177fe640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f690c0a3bb0 0x7f690c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.775+0000 7f6926118640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f690c0a3bb0 0x7f690c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.775+0000 7f69177fe640 1 -- 172.21.15.86:0/2240320856 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f69100d02a0 con 0x7f6920105490 2024-04-16T05:35:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.776+0000 7f6926118640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f690c0a3bb0 0x7f690c0a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f691c005fd0 tx=0x7f691c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.779+0000 7f69177fe640 1 -- 172.21.15.86:0/2240320856 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6910009de0 con 0x7f6920105490 2024-04-16T05:35:07.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:07 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/296617085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:07.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:07 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/739644464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:07.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:07 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1944914793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:07.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:07 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1075797629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:07.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:07 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/296617085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:07.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:07 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/739644464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:07.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:07 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1944914793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:07.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:07 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1075797629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.997+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6920062c00 con 0x7f6920105490 2024-04-16T05:35:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:07.998+0000 7f69177fe640 1 -- 172.21.15.86:0/2240320856 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7f691009c8f0 con 0x7f6920105490 2024-04-16T05:35:07.997 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:07.997 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:33:18.613317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24463,24455,24445]},"id":1}]} 2024-04-16T05:35:07.997 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-16T05:35:07.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f690c0a3bb0 msgr2=0x7f690c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f690c0a3bb0 0x7f690c0a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f691c005fd0 tx=0x7f691c00b390 comp rx=0 tx=0).stop 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920105490 msgr2=0x7f6920198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920105490 0x7f6920198860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f691000d770 tx=0x7f691000dc30 comp rx=0 tx=0).stop 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 shutdown_connections 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6920107dc0 0x7f692019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6920105490 0x7f6920198860 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f690c0a3bb0 0x7f690c0a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 --2- 172.21.15.86:0/2240320856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6920102b60 0x7f6920198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 >> 172.21.15.86:0/2240320856 conn(0x7f69200fc4b0 msgr2=0x7f69201034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 shutdown_connections 2024-04-16T05:35:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.000+0000 7f692837c640 1 -- 172.21.15.86:0/2240320856 wait complete. 2024-04-16T05:35:08.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:35:08.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 -- 172.21.15.86:0/577892507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103c20 msgr2=0x7f7fe010a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/577892507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103c20 0x7f7fe010a4b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009ca0 tx=0x7f7fd802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 -- 172.21.15.86:0/577892507 shutdown_connections 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/577892507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103c20 0x7f7fe010a4b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/577892507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fe0103260 0x7f7fe01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/577892507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fe0102070 0x7f7fe0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 -- 172.21.15.86:0/577892507 >> 172.21.15.86:0/577892507 conn(0x7f7fe00fd5e0 msgr2=0x7f7fe00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 -- 172.21.15.86:0/577892507 shutdown_connections 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 -- 172.21.15.86:0/577892507 wait complete. 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 Processor -- start 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.117+0000 7f7fe62e0640 1 -- start start 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fe62e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fe0102070 0x7f7fe0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fe62e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103260 0x7f7fe0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fe62e0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fe0103c20 0x7f7fe019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fd7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103260 0x7f7fe0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fd7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103260 0x7f7fe0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53130/0 (socket says 172.21.15.86:53130) 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fd7fff640 1 -- 172.21.15.86:0/1458172598 learned_addr learned my addr 172.21.15.86:0/1458172598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fe62e0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe010dd00 con 0x7f7fe0103260 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe010db80 con 0x7f7fe0103c20 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe010de80 con 0x7f7fe0102070 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fdffff640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fe0102070 0x7f7fe0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fe487d640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fe0103c20 0x7f7fe019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fd7fff640 1 -- 172.21.15.86:0/1458172598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fe0102070 msgr2=0x7f7fe0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fd7fff640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fe0102070 0x7f7fe0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fd7fff640 1 -- 172.21.15.86:0/1458172598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fe0103c20 msgr2=0x7f7fe019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fd7fff640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fe0103c20 0x7f7fe019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fd7fff640 1 -- 172.21.15.86:0/1458172598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe019ff40 con 0x7f7fe0103260 2024-04-16T05:35:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.118+0000 7f7fd7fff640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103260 0x7f7fe0198860 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800d770 tx=0x7f7fc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.119+0000 7f7fddffb640 1 -- 172.21.15.86:0/1458172598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8017070 con 0x7f7fe0103260 2024-04-16T05:35:08.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.119+0000 7f7fddffb640 1 -- 172.21.15.86:0/1458172598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc8005500 con 0x7f7fe0103260 2024-04-16T05:35:08.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.119+0000 7f7fddffb640 1 -- 172.21.15.86:0/1458172598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8004e00 con 0x7f7fe0103260 2024-04-16T05:35:08.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.119+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe01a0230 con 0x7f7fe0103260 2024-04-16T05:35:08.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.119+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe01a0730 con 0x7f7fe0103260 2024-04-16T05:35:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.120+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe0109330 con 0x7f7fe0103260 2024-04-16T05:35:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.121+0000 7f7fddffb640 1 -- 172.21.15.86:0/1458172598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7fc8004fa0 con 0x7f7fe0103260 2024-04-16T05:35:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.121+0000 7f7fddffb640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7fcc0a3bb0 0x7f7fcc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.121+0000 7f7fdffff640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7fcc0a3bb0 0x7f7fcc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.122+0000 7f7fddffb640 1 -- 172.21.15.86:0/1458172598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7fc80d0150 con 0x7f7fe0103260 2024-04-16T05:35:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.122+0000 7f7fdffff640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7fcc0a3bb0 0x7f7fcc0a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7fe01030c0 tx=0x7f7fd000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.126+0000 7f7fddffb640 1 -- 172.21.15.86:0/1458172598 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fc809c150 con 0x7f7fe0103260 2024-04-16T05:35:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:07 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/296617085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:07 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/739644464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:07 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1944914793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:08.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:07 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1075797629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.304+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fe0062c00 con 0x7f7fe0103260 2024-04-16T05:35:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.305+0000 7f7fddffb640 1 -- 172.21.15.86:0/1458172598 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24700 (secure 0 0 0) 0x7f7fc80a0450 con 0x7f7fe0103260 2024-04-16T05:35:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:08.305 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:34:52.612662+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7fcc0a3bb0 msgr2=0x7f7fcc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7fcc0a3bb0 0x7f7fcc0a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7fe01030c0 tx=0x7f7fd000b390 comp rx=0 tx=0).stop 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103260 msgr2=0x7f7fe0198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103260 0x7f7fe0198860 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800d770 tx=0x7f7fc800dc30 comp rx=0 tx=0).stop 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 shutdown_connections 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f7fcc0a3bb0 0x7f7fcc0a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fe0103c20 0x7f7fe019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fe0103260 0x7f7fe0198860 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 --2- 172.21.15.86:0/1458172598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fe0102070 0x7f7fe0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 >> 172.21.15.86:0/1458172598 conn(0x7f7fe00fd5e0 msgr2=0x7f7fe0106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 shutdown_connections 2024-04-16T05:35:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.308+0000 7f7fe62e0640 1 -- 172.21.15.86:0/1458172598 wait complete. 2024-04-16T05:35:08.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-16T05:32:54.852973+0000', 'modified': '2024-04-16T05:33:18.613317+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24481, 'mds_2': 24463, 'mds_3': 24455, 'mds_4': 24445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/1379378484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 1379378484}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 1379378484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14631': {'gid': 14631, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/1989684988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 1989684988}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 1989684988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6833/3318048466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6832', 'nonce': 3318048466}, {'type': 'v1', 'addr': '172.21.15.194:6833', 'nonce': 3318048466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/1220860047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 1220860047}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 1220860047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/2618844409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 2618844409}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 2618844409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.196:6835/3736061748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 3736061748}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 3736061748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.194:6837/3400155977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 3400155977}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 3400155977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.196:6837/2445566344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 2445566344}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 2445566344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.196:6839/3455974815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 3455974815}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 3455974815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.194:6839/601781370', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 601781370}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 601781370}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24481, 24482, 24463, 24455, 24445]} 2024-04-16T05:35:08.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-16T05:35:08.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-16T05:35:08.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.426+0000 7f5a391a8640 1 -- 172.21.15.86:0/3857017309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34102a50 msgr2=0x7f5a34109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.426+0000 7f5a391a8640 1 --2- 172.21.15.86:0/3857017309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34102a50 0x7f5a34109370 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009ca0 tx=0x7f5a2c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.426+0000 7f5a391a8640 1 -- 172.21.15.86:0/3857017309 shutdown_connections 2024-04-16T05:35:08.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.426+0000 7f5a391a8640 1 --2- 172.21.15.86:0/3857017309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34102a50 0x7f5a34109370 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.426+0000 7f5a391a8640 1 --2- 172.21.15.86:0/3857017309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a34102090 0x7f5a34102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.426+0000 7f5a391a8640 1 --2- 172.21.15.86:0/3857017309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a34100f40 0x7f5a34101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.426+0000 7f5a391a8640 1 -- 172.21.15.86:0/3857017309 >> 172.21.15.86:0/3857017309 conn(0x7f5a340fc4b0 msgr2=0x7f5a340fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.426+0000 7f5a391a8640 1 -- 172.21.15.86:0/3857017309 shutdown_connections 2024-04-16T05:35:08.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.426+0000 7f5a391a8640 1 -- 172.21.15.86:0/3857017309 wait complete. 2024-04-16T05:35:08.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a391a8640 1 Processor -- start 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a391a8640 1 -- start start 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a391a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34100f40 0x7f5a341982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a391a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a34102090 0x7f5a341987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a391a8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a34102a50 0x7f5a3419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a391a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3410dd30 con 0x7f5a34100f40 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a391a8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3410dbb0 con 0x7f5a34102a50 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a391a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3410deb0 con 0x7f5a34102090 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a32d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34100f40 0x7f5a341982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a32d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34100f40 0x7f5a341982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53144/0 (socket says 172.21.15.86:53144) 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a32d9d640 1 -- 172.21.15.86:0/1368974418 learned_addr learned my addr 172.21.15.86:0/1368974418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a3259c640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a34102090 0x7f5a341987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a3359e640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a34102a50 0x7f5a3419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a32d9d640 1 -- 172.21.15.86:0/1368974418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a34102090 msgr2=0x7f5a341987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a32d9d640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a34102090 0x7f5a341987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a32d9d640 1 -- 172.21.15.86:0/1368974418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a34102a50 msgr2=0x7f5a3419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a32d9d640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a34102a50 0x7f5a3419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a32d9d640 1 -- 172.21.15.86:0/1368974418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3419fed0 con 0x7f5a34100f40 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a3259c640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a34102090 0x7f5a341987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.427+0000 7f5a3359e640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a34102a50 0x7f5a3419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.428+0000 7f5a32d9d640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34100f40 0x7f5a341982b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400e9f0 tx=0x7f5a2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.428+0000 7f5a1bfff640 1 -- 172.21.15.86:0/1368974418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2400cd80 con 0x7f5a34100f40 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.428+0000 7f5a391a8640 1 -- 172.21.15.86:0/1368974418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a341a01c0 con 0x7f5a34100f40 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.428+0000 7f5a1bfff640 1 -- 172.21.15.86:0/1368974418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a24010910 con 0x7f5a34100f40 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.428+0000 7f5a1bfff640 1 -- 172.21.15.86:0/1368974418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a240188f0 con 0x7f5a34100f40 2024-04-16T05:35:08.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.428+0000 7f5a391a8640 1 -- 172.21.15.86:0/1368974418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a341a0710 con 0x7f5a34100f40 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a1bfff640 1 -- 172.21.15.86:0/1368974418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5a24010ac0 con 0x7f5a34100f40 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3bd8f740 1 Processor -- start 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3bd8f740 1 -- start start 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fb5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb5750 0x563638fba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fba8c0 0x563638fbcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563638ec8240 con 0x563638fb4d60 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563638f259f0 con 0x563638fba8c0 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563638ee09e0 con 0x563638fb5750 2024-04-16T05:35:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fb5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fb5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53146/0 (socket says 172.21.15.86:53146) 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3359e640 1 -- 172.21.15.86:0/1737998994 learned_addr learned my addr 172.21.15.86:0/1737998994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3359e640 1 -- 172.21.15.86:0/1737998994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb5750 msgr2=0x563638fba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3359e640 1 --2- 172.21.15.86:0/1737998994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb5750 0x563638fba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3259c640 1 --2- 172.21.15.86:0/1737998994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fba8c0 0x563638fbcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3359e640 1 -- 172.21.15.86:0/1737998994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fba8c0 msgr2=0x563638fbcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a1bfff640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5a1c0a3bb0 0x7f5a1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3359e640 1 --2- 172.21.15.86:0/1737998994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fba8c0 0x563638fbcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a3359e640 1 -- 172.21.15.86:0/1737998994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563638ed3bc0 con 0x563638fb4d60 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.431+0000 7f5a1bfff640 1 -- 172.21.15.86:0/1368974418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5a240d0c80 con 0x7f5a34100f40 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.432+0000 7f5a3359e640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5a1c0a3bb0 0x7f5a1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.432+0000 7f5a3359e640 1 --2- 172.21.15.86:0/1737998994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fb5180 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0324c0 tx=0x7f5a2c009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7604ea83216e942b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.432+0000 7f5a197fa640 1 -- 172.21.15.86:0/1737998994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c045030 con 0x563638fb4d60 2024-04-16T05:35:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.432+0000 7f5a197fa640 1 -- 172.21.15.86:0/1737998994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a2c003e50 con 0x563638fb4d60 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.432+0000 7f5a197fa640 1 -- 172.21.15.86:0/1737998994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c039040 con 0x563638fb4d60 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1737998994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 msgr2=0x563638fb5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3359e640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5a1c0a3bb0 0x7f5a1c0a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c045000 tx=0x7f5a2c04a000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1737998994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fb5180 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0324c0 tx=0x7f5a2c009910 comp rx=0 tx=0).stop 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1737998994 shutdown_connections 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1737998994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fba8c0 0x563638fbcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1737998994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb5750 0x563638fba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1737998994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fb5180 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1737998994 >> 172.21.15.86:0/1737998994 conn(0x563638fb02d0 msgr2=0x563638fb06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1737998994 shutdown_connections 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1737998994 wait complete. 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 -- start start 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 Processor -- start 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fbf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x563638fbf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fba8c0 0x563638fc65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563638f1f0f0 con 0x563638fb4d60 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563638f19dd0 con 0x563638fb5750 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563638f27cd0 con 0x563638fba8c0 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3bd8f740 10 client.0 _ll_register_callbacks cb 0x563638f10570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a32d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fbf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a32d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fbf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53160/0 (socket says 172.21.15.86:53160) 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a32d9d640 1 -- 172.21.15.86:0/1222342663 learned_addr learned my addr 172.21.15.86:0/1222342663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.433+0000 7f5a3259c640 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x563638fbf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a32d9d640 1 -- 172.21.15.86:0/1222342663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fba8c0 msgr2=0x563638fc65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a32d9d640 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fba8c0 0x563638fc65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a32d9d640 1 -- 172.21.15.86:0/1222342663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 msgr2=0x563638fbf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a32d9d640 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x563638fbf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a32d9d640 1 -- 172.21.15.86:0/1222342663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563638c2d2d0 con 0x563638fb4d60 2024-04-16T05:35:08.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3259c640 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x563638fbf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a32d9d640 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fbf030 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a24099030 tx=0x7f5a2409d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf60e3672696302 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a16ff5640 1 -- 172.21.15.86:0/1222342663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a240a0f80 con 0x563638fb4d60 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a16ff5640 1 -- 172.21.15.86:0/1222342663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5a2409c020 con 0x563638fb4d60 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1222342663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a2c003e50 con 0x563638fb4d60 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a16ff5640 1 -- 172.21.15.86:0/1222342663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5a240a5840 con 0x563638fb4d60 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3bd8f740 10 client.15426 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1222342663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563638cf76b0 con 0x563638fb4d60 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a16ff5640 1 -- 172.21.15.86:0/1222342663 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a2c003e50 con 0x563638fb4d60 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a16ff5640 1 -- 172.21.15.86:0/1222342663 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5a24098a40 con 0x563638fb4d60 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3bd8f740 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3bd8f740 10 client.15426 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3bd8f740 20 client.15426 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x563638fcbcb0 0x563638fcd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3bd8f740 4 client.15426 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.434+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1222342663 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563638ea14c0 con 0x563638fcbcb0 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.435+0000 7f5a3259c640 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x563638fcbcb0 0x563638fcd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.435+0000 7f5a3259c640 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x563638fcbcb0 0x563638fcd120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4618579637dc61db server_cookie=b345c134840a7bc1 in_seq=0 out_seq=0 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.435+0000 7f5a16ff5640 10 client.15426 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a16ff5640 1 -- 172.21.15.86:0/1222342663 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563638ea14c0 con 0x563638fcbcb0 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a16ff5640 10 client.15426 handle_command_reply: tid=0 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 client.15426 shutdown 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1222342663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 msgr2=0x563638fbf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fbf030 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a24099030 tx=0x7f5a2409d390 comp rx=0 tx=0).stop 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1222342663 shutdown_connections 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x563638fcbcb0 0x563638fcd120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fba8c0 0x563638fc65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x563638fbf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/1222342663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fb4d60 0x563638fbf030 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a16ff5640 0 client.15426 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:35:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1222342663 >> 172.21.15.86:0/1222342663 conn(0x563638fb02d0 msgr2=0x563638faf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1222342663 shutdown_connections 2024-04-16T05:35:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/1222342663 wait complete. 2024-04-16T05:35:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.436+0000 7f5a3bd8f740 20 client.15426 trim_cache size 0 max 16384 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3bd8f740 1 Processor -- start 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3bd8f740 1 -- start start 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb4d60 0x563638fcac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb5750 0x563638fb6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563638fbda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563638fcb460 con 0x563638fba8c0 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563638fcbb30 con 0x563638fb4d60 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563638fcc200 con 0x563638fb5750 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563638fbda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563638fbda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53164/0 (socket says 172.21.15.86:53164) 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a3359e640 1 -- 172.21.15.86:0/4098262134 learned_addr learned my addr 172.21.15.86:0/4098262134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.454+0000 7f5a32d9d640 1 --2- 172.21.15.86:0/4098262134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb4d60 0x563638fcac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3259c640 1 --2- 172.21.15.86:0/4098262134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb5750 0x563638fb6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3359e640 1 -- 172.21.15.86:0/4098262134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb5750 msgr2=0x563638fb6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3359e640 1 --2- 172.21.15.86:0/4098262134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb5750 0x563638fb6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3359e640 1 -- 172.21.15.86:0/4098262134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb4d60 msgr2=0x563638fcac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3359e640 1 --2- 172.21.15.86:0/4098262134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb4d60 0x563638fcac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3359e640 1 -- 172.21.15.86:0/4098262134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563638fb5b30 con 0x563638fba8c0 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3359e640 1 --2- 172.21.15.86:0/4098262134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563638fbda90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0325f0 tx=0x7f5a2c04d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16854c8c38b45ede server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a18ff9640 1 -- 172.21.15.86:0/4098262134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c039070 con 0x563638fba8c0 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a18ff9640 1 -- 172.21.15.86:0/4098262134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a2c03f950 con 0x563638fba8c0 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a18ff9640 1 -- 172.21.15.86:0/4098262134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c04f070 con 0x563638fba8c0 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/4098262134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 msgr2=0x563638fbda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/4098262134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563638fbda90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0325f0 tx=0x7f5a2c04d000 comp rx=0 tx=0).stop 2024-04-16T05:35:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/4098262134 shutdown_connections 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.455+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/4098262134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563638fbda90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/4098262134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb5750 0x563638fb6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/4098262134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb4d60 0x563638fcac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/4098262134 >> 172.21.15.86:0/4098262134 conn(0x563638fb02d0 msgr2=0x563638fb06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/4098262134 shutdown_connections 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/4098262134 wait complete. 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 -- start start 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 Processor -- start 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb4d60 0x563638fc37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x56363902dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563639030190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563638fccfa0 con 0x563638fba8c0 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563638fcc8d0 con 0x563638fb5750 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563638fcd670 con 0x563638fb4d60 2024-04-16T05:35:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3bd8f740 10 client.0 _ll_register_callbacks cb 0x7f5a2400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563639030190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563639030190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53168/0 (socket says 172.21.15.86:53168) 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3359e640 1 -- 172.21.15.86:0/2005660545 learned_addr learned my addr 172.21.15.86:0/2005660545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3259c640 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x56363902dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3359e640 1 -- 172.21.15.86:0/2005660545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb4d60 msgr2=0x563638fc37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3359e640 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb4d60 0x563638fc37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3359e640 1 -- 172.21.15.86:0/2005660545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 msgr2=0x56363902dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3359e640 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x56363902dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3359e640 1 -- 172.21.15.86:0/2005660545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636390306d0 con 0x563638fba8c0 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.456+0000 7f5a3259c640 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x56363902dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3359e640 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563639030190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c02cd20 tx=0x7f5a2c02cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2320e041fa6dd21 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a16ff5640 1 -- 172.21.15.86:0/2005660545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c060060 con 0x563638fba8c0 2024-04-16T05:35:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a16ff5640 1 -- 172.21.15.86:0/2005660545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5a2c03f670 con 0x563638fba8c0 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/2005660545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a2c03f950 con 0x563638fba8c0 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a16ff5640 1 -- 172.21.15.86:0/2005660545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5a2c067050 con 0x563638fba8c0 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3bd8f740 10 client.15432 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/2005660545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636390318d0 con 0x563638fba8c0 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a16ff5640 1 -- 172.21.15.86:0/2005660545 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a2c03f950 con 0x563638fba8c0 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a16ff5640 1 -- 172.21.15.86:0/2005660545 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5a2c04b720 con 0x563638fba8c0 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3bd8f740 10 client.15432 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3bd8f740 10 client.15432 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3bd8f740 20 client.15432 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x563639035e90 0x563639034ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3bd8f740 4 client.15432 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.457+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/2005660545 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "status"}) -- 0x563638f284e0 con 0x563639035e90 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a32d9d640 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x563639035e90 0x563639034ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a16ff5640 10 client.15432 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:35:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a32d9d640 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x563639035e90 0x563639034ba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf4aa0fffbf85b68 server_cookie=c3deb28913bf1001 in_seq=0 out_seq=0 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a16ff5640 1 -- 172.21.15.86:0/2005660545 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563638f284e0 con 0x563639035e90 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a16ff5640 10 client.15432 handle_command_reply: tid=0 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 client.15432 shutdown 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/2005660545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 msgr2=0x563639030190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563639030190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c02cd20 tx=0x7f5a2c02cd50 comp rx=0 tx=0).stop 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/2005660545 shutdown_connections 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x563639035e90 0x563639034ba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563638fba8c0 0x563639030190 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563638fb5750 0x56363902dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a16ff5640 0 client.15432 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 --2- 172.21.15.86:0/2005660545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563638fb4d60 0x563638fc37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/2005660545 >> 172.21.15.86:0/2005660545 conn(0x563638fb02d0 msgr2=0x563638fb6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/2005660545 shutdown_connections 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.458+0000 7f5a3bd8f740 1 -- 172.21.15.86:0/2005660545 wait complete. 2024-04-16T05:35:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a3bd8f740 20 client.15432 trim_cache size 0 max 16384 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24445, 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 110.850965128, 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 156.46487850400001 2024-04-16T05:35:08.465 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 -- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5a1c0a3bb0 msgr2=0x7f5a1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5a1c0a3bb0 0x7f5a1c0a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c045000 tx=0x7f5a2c04a000 comp rx=0 tx=0).stop 2024-04-16T05:35:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 -- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34100f40 msgr2=0x7f5a341982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34100f40 0x7f5a341982b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400e9f0 tx=0x7f5a2400eeb0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 -- 172.21.15.86:0/1368974418 shutdown_connections 2024-04-16T05:35:08.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f5a1c0a3bb0 0x7f5a1c0a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a34102a50 0x7f5a3419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a34102090 0x7f5a341987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 --2- 172.21.15.86:0/1368974418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a34100f40 0x7f5a341982b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 -- 172.21.15.86:0/1368974418 >> 172.21.15.86:0/1368974418 conn(0x7f5a340fc4b0 msgr2=0x7f5a34105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 -- 172.21.15.86:0/1368974418 shutdown_connections 2024-04-16T05:35:08.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.459+0000 7f5a391a8640 1 -- 172.21.15.86:0/1368974418 wait complete. 2024-04-16T05:35:08.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 -- 172.21.15.86:0/2952563301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c101210 msgr2=0x7f271c101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 --2- 172.21.15.86:0/2952563301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c101210 0x7f271c101690 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2704009900 tx=0x7f270402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 -- 172.21.15.86:0/2952563301 shutdown_connections 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 --2- 172.21.15.86:0/2952563301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f271c101bd0 0x7f271c109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 --2- 172.21.15.86:0/2952563301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c101210 0x7f271c101690 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 --2- 172.21.15.86:0/2952563301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f271c1024e0 0x7f271c102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 -- 172.21.15.86:0/2952563301 >> 172.21.15.86:0/2952563301 conn(0x7f271c0fbc40 msgr2=0x7f271c0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 -- 172.21.15.86:0/2952563301 shutdown_connections 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 -- 172.21.15.86:0/2952563301 wait complete. 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 Processor -- start 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.576+0000 7f2720ea5640 1 -- start start 2024-04-16T05:35:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2720ea5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f271c101210 0x7f271c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2720ea5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f271c101bd0 0x7f271c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2720ea5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c1024e0 0x7f271c19f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2720ea5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10dd00 con 0x7f271c1024e0 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2720ea5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10db80 con 0x7f271c101bd0 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2720ea5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10de80 con 0x7f271c101210 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c1024e0 0x7f271c19f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c1024e0 0x7f271c19f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53176/0 (socket says 172.21.15.86:53176) 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271ad9d640 1 -- 172.21.15.86:0/3811485757 learned_addr learned my addr 172.21.15.86:0/3811485757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271a59c640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f271c101210 0x7f271c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2713fff640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f271c101bd0 0x7f271c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271ad9d640 1 -- 172.21.15.86:0/3811485757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f271c101210 msgr2=0x7f271c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271ad9d640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f271c101210 0x7f271c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271ad9d640 1 -- 172.21.15.86:0/3811485757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f271c101bd0 msgr2=0x7f271c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271ad9d640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f271c101bd0 0x7f271c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271ad9d640 1 -- 172.21.15.86:0/3811485757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c19ffd0 con 0x7f271c1024e0 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2713fff640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f271c101bd0 0x7f271c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f271ad9d640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c1024e0 0x7f271c19f960 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f271400eaa0 tx=0x7f271400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f27137fe640 1 -- 172.21.15.86:0/3811485757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271400cdb0 con 0x7f271c1024e0 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f27137fe640 1 -- 172.21.15.86:0/3811485757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2714005020 con 0x7f271c1024e0 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2720ea5640 1 -- 172.21.15.86:0/3811485757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c1a02c0 con 0x7f271c1024e0 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f27137fe640 1 -- 172.21.15.86:0/3811485757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27140116b0 con 0x7f271c1024e0 2024-04-16T05:35:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.577+0000 7f2720ea5640 1 -- 172.21.15.86:0/3811485757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c1a0810 con 0x7f271c1024e0 2024-04-16T05:35:08.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.580+0000 7f27137fe640 1 -- 172.21.15.86:0/3811485757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2714005560 con 0x7f271c1024e0 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.580+0000 7f2723b6b740 1 Processor -- start 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.580+0000 7f2723b6b740 1 -- start start 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.580+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 0x561f18b41180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.580+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b41750 0x561f18b46250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.580+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b468c0 0x561f18b48d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.580+0000 7f2723b6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f18a54240 con 0x561f18b468c0 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.580+0000 7f2723b6b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561f18ab19f0 con 0x561f18b40d60 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2723b6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f18a6c9e0 con 0x561f18b41750 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f27137fe640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27080a3bb0 0x7f27080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b468c0 0x561f18b48d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f271ad9d640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27080a3bb0 0x7f27080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b468c0 0x561f18b48d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53188/0 (socket says 172.21.15.86:53188) 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 -- 172.21.15.86:0/1595482508 learned_addr learned my addr 172.21.15.86:0/1595482508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f271a59c640 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b41750 0x561f18b46250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f27137fe640 1 -- 172.21.15.86:0/3811485757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f27140d0a30 con 0x7f271c1024e0 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 0x561f18b41180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 -- 172.21.15.86:0/1595482508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b41750 msgr2=0x561f18b46250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b41750 0x561f18b46250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 -- 172.21.15.86:0/1595482508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 msgr2=0x561f18b41180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 0x561f18b41180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 -- 172.21.15.86:0/1595482508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f18a5fbc0 con 0x561f18b468c0 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f271a59c640 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b41750 0x561f18b46250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f271ad9d640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27080a3bb0 0x7f27080a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27140ce510 tx=0x7f2714066fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2713fff640 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b468c0 0x561f18b48d50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2704005c20 tx=0x7f270402cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05e12cf9ed5672c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2710ff9640 1 -- 172.21.15.86:0/1595482508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704048070 con 0x561f18b468c0 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2710ff9640 1 -- 172.21.15.86:0/1595482508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27040035f0 con 0x561f18b468c0 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.581+0000 7f2710ff9640 1 -- 172.21.15.86:0/1595482508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27040436b0 con 0x561f18b468c0 2024-04-16T05:35:08.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 -- 172.21.15.86:0/1595482508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b468c0 msgr2=0x561f18b48d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b468c0 0x561f18b48d50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2704005c20 tx=0x7f270402cf90 comp rx=0 tx=0).stop 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 -- 172.21.15.86:0/1595482508 shutdown_connections 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b468c0 0x561f18b48d50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b41750 0x561f18b46250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1595482508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 0x561f18b41180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 -- 172.21.15.86:0/1595482508 >> 172.21.15.86:0/1595482508 conn(0x561f18b3c2d0 msgr2=0x561f18b3c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 -- 172.21.15.86:0/1595482508 shutdown_connections 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 -- 172.21.15.86:0/1595482508 wait complete. 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.582+0000 7f2723b6b740 1 -- start start 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2723b6b740 1 Processor -- start 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 0x561f18b4afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b41750 0x561f18b4b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b468c0 0x561f18b52520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2723b6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f18aab0f0 con 0x561f18b41750 2024-04-16T05:35:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2723b6b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561f18aa5dd0 con 0x561f18b40d60 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2723b6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f18ab3cd0 con 0x561f18b468c0 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2723b6b740 10 client.0 _ll_register_callbacks cb 0x561f18a9c570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f271a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b41750 0x561f18b4b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f271a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b41750 0x561f18b4b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53200/0 (socket says 172.21.15.86:53200) 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f271a59c640 1 -- 172.21.15.86:0/482415939 learned_addr learned my addr 172.21.15.86:0/482415939 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2713fff640 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 0x561f18b4afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f271a59c640 1 -- 172.21.15.86:0/482415939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b468c0 msgr2=0x561f18b52520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f271a59c640 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b468c0 0x561f18b52520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f271a59c640 1 -- 172.21.15.86:0/482415939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 msgr2=0x561f18b4afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f271a59c640 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 0x561f18b4afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f271a59c640 1 -- 172.21.15.86:0/482415939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f187b92d0 con 0x561f18b41750 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2713fff640 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 0x561f18b4afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f271a59c640 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b41750 0x561f18b4b4e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f270c00cd60 tx=0x7f270c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324c69e624480560 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2701ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2701ffb640 1 -- 172.21.15.86:0/482415939 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c005220 con 0x561f18b41750 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2701ffb640 1 -- 172.21.15.86:0/482415939 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f270c0053c0 con 0x561f18b41750 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.583+0000 7f2723b6b740 1 -- 172.21.15.86:0/482415939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27040035f0 con 0x561f18b41750 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2701ffb640 1 -- 172.21.15.86:0/482415939 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f270c0055c0 con 0x561f18b41750 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2723b6b740 10 client.15444 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2723b6b740 1 -- 172.21.15.86:0/482415939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f188836b0 con 0x561f18b41750 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2701ffb640 1 -- 172.21.15.86:0/482415939 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27040035f0 con 0x561f18b41750 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2701ffb640 1 -- 172.21.15.86:0/482415939 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f270c018030 con 0x561f18b41750 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2723b6b740 10 client.15444 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2723b6b740 10 client.15444 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2723b6b740 20 client.15444 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2723b6b740 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561f18b57d20 0x561f18b59190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2723b6b740 4 client.15444 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2723b6b740 1 -- 172.21.15.86:0/482415939 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f18a2d4c0 con 0x561f18b57d20 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.584+0000 7f2713fff640 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561f18b57d20 0x561f18b59190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.585+0000 7f2701ffb640 10 client.15444 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.585+0000 7f2713fff640 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561f18b57d20 0x561f18b59190 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=478bc2b1332fa9 server_cookie=44c1fbcb23ecf6ce in_seq=0 out_seq=0 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.585+0000 7f2701ffb640 1 -- 172.21.15.86:0/482415939 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f18a2d4c0 con 0x561f18b57d20 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.585+0000 7f2701ffb640 10 client.15444 handle_command_reply: tid=0 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.585+0000 7f2723b6b740 1 client.15444 shutdown 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 -- 172.21.15.86:0/482415939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b41750 msgr2=0x561f18b4b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b41750 0x561f18b4b4e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f270c00cd60 tx=0x7f270c0039c0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 -- 172.21.15.86:0/482415939 shutdown_connections 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561f18b57d20 0x561f18b59190 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b468c0 0x561f18b52520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b41750 0x561f18b4b4e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2701ffb640 0 client.15444 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 --2- 172.21.15.86:0/482415939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b40d60 0x561f18b4afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 -- 172.21.15.86:0/482415939 >> 172.21.15.86:0/482415939 conn(0x561f18b3c2d0 msgr2=0x561f18b3b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 -- 172.21.15.86:0/482415939 shutdown_connections 2024-04-16T05:35:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 1 -- 172.21.15.86:0/482415939 wait complete. 2024-04-16T05:35:08.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.586+0000 7f2723b6b740 20 client.15444 trim_cache size 0 max 16384 2024-04-16T05:35:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2723b6b740 1 Processor -- start 2024-04-16T05:35:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2723b6b740 1 -- start start 2024-04-16T05:35:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b3c2d0 0x561f18b48cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b40d60 0x561f18b56c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 0x561f18b42c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2723b6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f18b574d0 con 0x561f18b40d60 2024-04-16T05:35:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2723b6b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561f18b57ba0 con 0x561f18b41750 2024-04-16T05:35:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2723b6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f18b58270 con 0x561f18b3c2d0 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f271a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b40d60 0x561f18b56c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f271a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b40d60 0x561f18b56c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53208/0 (socket says 172.21.15.86:53208) 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f271a59c640 1 -- 172.21.15.86:0/2619214084 learned_addr learned my addr 172.21.15.86:0/2619214084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2713fff640 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 0x561f18b42c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2713fff640 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b3c2d0 0x561f18b48cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f271a59c640 1 -- 172.21.15.86:0/2619214084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b3c2d0 msgr2=0x561f18b48cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f271a59c640 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b3c2d0 0x561f18b48cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f271a59c640 1 -- 172.21.15.86:0/2619214084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 msgr2=0x561f18b42c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f271a59c640 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 0x561f18b42c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f271a59c640 1 -- 172.21.15.86:0/2619214084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f18b41b30 con 0x561f18b40d60 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2713fff640 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 0x561f18b42c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f271a59c640 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b40d60 0x561f18b56c50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f270c000c00 tx=0x7f270c004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84edb2900f51547 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2703fff640 1 -- 172.21.15.86:0/2619214084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c02a070 con 0x561f18b40d60 2024-04-16T05:35:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2703fff640 1 -- 172.21.15.86:0/2619214084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270c010900 con 0x561f18b40d60 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.604+0000 7f2703fff640 1 -- 172.21.15.86:0/2619214084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c019650 con 0x561f18b40d60 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 -- 172.21.15.86:0/2619214084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b40d60 msgr2=0x561f18b56c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b40d60 0x561f18b56c50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f270c000c00 tx=0x7f270c004000 comp rx=0 tx=0).stop 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 -- 172.21.15.86:0/2619214084 shutdown_connections 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 0x561f18b42c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b40d60 0x561f18b56c50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 --2- 172.21.15.86:0/2619214084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b3c2d0 0x561f18b48cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 -- 172.21.15.86:0/2619214084 >> 172.21.15.86:0/2619214084 conn(0x561f18b42530 msgr2=0x561f18b3c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 -- 172.21.15.86:0/2619214084 shutdown_connections 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 -- 172.21.15.86:0/2619214084 wait complete. 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 -- start start 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.605+0000 7f2723b6b740 1 Processor -- start 2024-04-16T05:35:08.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b3c2d0 0x561f18b4d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b40d60 0x561f18b4dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 0x561f18b504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f18b59010 con 0x561f18b3c2d0 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561f18b58940 con 0x561f18b41750 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f18b596e0 con 0x561f18b40d60 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 10 client.0 _ll_register_callbacks cb 0x7f270c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b3c2d0 0x561f18b4d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b3c2d0 0x561f18b4d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53216/0 (socket says 172.21.15.86:53216) 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 -- 172.21.15.86:0/1572423564 learned_addr learned my addr 172.21.15.86:0/1572423564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f271a59c640 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b40d60 0x561f18b4dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 0x561f18b504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 -- 172.21.15.86:0/1572423564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b40d60 msgr2=0x561f18b4dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b40d60 0x561f18b4dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 -- 172.21.15.86:0/1572423564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 msgr2=0x561f18b504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 0x561f18b504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 -- 172.21.15.86:0/1572423564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f18b50a50 con 0x561f18b3c2d0 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f271a59c640 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b40d60 0x561f18b4dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2713fff640 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b3c2d0 0x561f18b4d6b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2704000c00 tx=0x7f270402cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e2f53eef1c92cf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2701ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2701ffb640 1 -- 172.21.15.86:0/1572423564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704048040 con 0x561f18b3c2d0 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 1 -- 172.21.15.86:0/1572423564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f270c010900 con 0x561f18b3c2d0 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2701ffb640 1 -- 172.21.15.86:0/1572423564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2704003a20 con 0x561f18b3c2d0 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2701ffb640 1 -- 172.21.15.86:0/1572423564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2704052450 con 0x561f18b3c2d0 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 10 client.15450 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.606+0000 7f2723b6b740 1 -- 172.21.15.86:0/1572423564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f18b51d50 con 0x561f18b3c2d0 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.607+0000 7f2701ffb640 1 -- 172.21.15.86:0/1572423564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270c010900 con 0x561f18b3c2d0 2024-04-16T05:35:08.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.607+0000 7f2701ffb640 1 -- 172.21.15.86:0/1572423564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2704043450 con 0x561f18b3c2d0 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.607+0000 7f2723b6b740 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.607+0000 7f2723b6b740 10 client.15450 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.607+0000 7f2723b6b740 20 client.15450 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.607+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561f18b468c0 0x561f18bc95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.607+0000 7f2723b6b740 4 client.15450 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.607+0000 7f2723b6b740 1 -- 172.21.15.86:0/1572423564 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "status"}) -- 0x561f18ab44e0 con 0x561f18b468c0 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.607+0000 7f271a59c640 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561f18b468c0 0x561f18bc95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2701ffb640 10 client.15450 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f271a59c640 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561f18b468c0 0x561f18bc95c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73e1073ca5c1cc18 server_cookie=311b53cd2dc8d5ed in_seq=0 out_seq=0 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2701ffb640 1 -- 172.21.15.86:0/1572423564 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561f18ab44e0 con 0x561f18b468c0 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2701ffb640 10 client.15450 handle_command_reply: tid=0 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 client.15450 shutdown 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 -- 172.21.15.86:0/1572423564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b3c2d0 msgr2=0x561f18b4d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b3c2d0 0x561f18b4d6b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2704000c00 tx=0x7f270402cf90 comp rx=0 tx=0).stop 2024-04-16T05:35:08.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 -- 172.21.15.86:0/1572423564 shutdown_connections 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f18b41750 0x561f18b504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561f18b468c0 0x561f18bc95c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f18b40d60 0x561f18b4dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 --2- 172.21.15.86:0/1572423564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f18b3c2d0 0x561f18b4d6b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2701ffb640 0 client.15450 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 -- 172.21.15.86:0/1572423564 >> 172.21.15.86:0/1572423564 conn(0x561f18b42530 msgr2=0x561f18b56c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 -- 172.21.15.86:0/1572423564 shutdown_connections 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 1 -- 172.21.15.86:0/1572423564 wait complete. 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2723b6b740 20 client.15450 trim_cache size 0 max 16384 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:08.614 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24455, 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 112.008161798, 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 150.77626731800001 2024-04-16T05:35:08.615 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:08.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2720ea5640 1 -- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27080a3bb0 msgr2=0x7f27080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2720ea5640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27080a3bb0 0x7f27080a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27140ce510 tx=0x7f2714066fd0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2720ea5640 1 -- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c1024e0 msgr2=0x7f271c19f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.608+0000 7f2720ea5640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c1024e0 0x7f271c19f960 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f271400eaa0 tx=0x7f271400ef60 comp rx=0 tx=0).stop 2024-04-16T05:35:08.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.609+0000 7f2720ea5640 1 -- 172.21.15.86:0/3811485757 shutdown_connections 2024-04-16T05:35:08.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.609+0000 7f2720ea5640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271c1024e0 0x7f271c19f960 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.609+0000 7f2720ea5640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f27080a3bb0 0x7f27080a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.609+0000 7f2720ea5640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f271c101bd0 0x7f271c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.609+0000 7f2720ea5640 1 --2- 172.21.15.86:0/3811485757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f271c101210 0x7f271c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.609+0000 7f2720ea5640 1 -- 172.21.15.86:0/3811485757 >> 172.21.15.86:0/3811485757 conn(0x7f271c0fbc40 msgr2=0x7f271c1055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.609+0000 7f2720ea5640 1 -- 172.21.15.86:0/3811485757 shutdown_connections 2024-04-16T05:35:08.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.609+0000 7f2720ea5640 1 -- 172.21.15.86:0/3811485757 wait complete. 2024-04-16T05:35:08.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-16T05:35:08.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.726+0000 7fd541f24640 1 -- 172.21.15.86:0/4113723094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c103c00 msgr2=0x7fd53c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.726+0000 7fd541f24640 1 --2- 172.21.15.86:0/4113723094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c103c00 0x7fd53c10a490 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd534009ca0 tx=0x7fd53402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 -- 172.21.15.86:0/4113723094 shutdown_connections 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 --2- 172.21.15.86:0/4113723094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c103c00 0x7fd53c10a490 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 --2- 172.21.15.86:0/4113723094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd53c103240 0x7fd53c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 --2- 172.21.15.86:0/4113723094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd53c102050 0x7fd53c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 -- 172.21.15.86:0/4113723094 >> 172.21.15.86:0/4113723094 conn(0x7fd53c0fd5e0 msgr2=0x7fd53c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 -- 172.21.15.86:0/4113723094 shutdown_connections 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 -- 172.21.15.86:0/4113723094 wait complete. 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 Processor -- start 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 -- start start 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c102050 0x7fd53c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd53c103240 0x7fd53c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd53c103c00 0x7fd53c19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c10dce0 con 0x7fd53c102050 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c10db60 con 0x7fd53c103c00 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.727+0000 7fd541f24640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c10de60 con 0x7fd53c103240 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c102050 0x7fd53c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c102050 0x7fd53c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53234/0 (socket says 172.21.15.86:53234) 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53b7fe640 1 -- 172.21.15.86:0/2209255137 learned_addr learned my addr 172.21.15.86:0/2209255137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd532ffd640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd53c103240 0x7fd53c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53bfff640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd53c103c00 0x7fd53c19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53b7fe640 1 -- 172.21.15.86:0/2209255137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd53c103240 msgr2=0x7fd53c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd53c103240 0x7fd53c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53b7fe640 1 -- 172.21.15.86:0/2209255137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd53c103c00 msgr2=0x7fd53c19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd53c103c00 0x7fd53c19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53b7fe640 1 -- 172.21.15.86:0/2209255137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c19ff30 con 0x7fd53c102050 2024-04-16T05:35:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd532ffd640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd53c103240 0x7fd53c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53bfff640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd53c103c00 0x7fd53c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c102050 0x7fd53c198310 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00c970 tx=0x7fd52c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd5397fa640 1 -- 172.21.15.86:0/2209255137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c004220 con 0x7fd53c102050 2024-04-16T05:35:08.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd5397fa640 1 -- 172.21.15.86:0/2209255137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c0043c0 con 0x7fd53c102050 2024-04-16T05:35:08.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd5397fa640 1 -- 172.21.15.86:0/2209255137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00f450 con 0x7fd53c102050 2024-04-16T05:35:08.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd541f24640 1 -- 172.21.15.86:0/2209255137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53c1a0220 con 0x7fd53c102050 2024-04-16T05:35:08.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.728+0000 7fd541f24640 1 -- 172.21.15.86:0/2209255137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53c1a0770 con 0x7fd53c102050 2024-04-16T05:35:08.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.730+0000 7fd5397fa640 1 -- 172.21.15.86:0/2209255137 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd52c0079b0 con 0x7fd53c102050 2024-04-16T05:35:08.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd544beb740 1 Processor -- start 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd544beb740 1 -- start start 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd544beb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd544beb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d934250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd544beb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d936d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd544beb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b7d842240 con 0x561b7d92ed60 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd544beb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561b7d89f9f0 con 0x561b7d92f750 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd544beb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b7d85a9e0 con 0x561b7d9348c0 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53238/0 (socket says 172.21.15.86:53238) 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd5397fa640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd5280a3bb0 0x7fd5280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53bfff640 1 -- 172.21.15.86:0/3232155717 learned_addr learned my addr 172.21.15.86:0/3232155717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/3232155717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d934250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd532ffd640 1 --2- 172.21.15.86:0/3232155717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d936d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53bfff640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd5280a3bb0 0x7fd5280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53bfff640 1 -- 172.21.15.86:0/3232155717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 msgr2=0x561b7d936d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53bfff640 1 --2- 172.21.15.86:0/3232155717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d936d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53bfff640 1 -- 172.21.15.86:0/3232155717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 msgr2=0x561b7d934250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53bfff640 1 --2- 172.21.15.86:0/3232155717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d934250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd5397fa640 1 -- 172.21.15.86:0/2209255137 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd52c0d05a0 con 0x7fd53c102050 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.731+0000 7fd53bfff640 1 -- 172.21.15.86:0/3232155717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7d84dbc0 con 0x561b7d92ed60 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd53bfff640 1 --2- 172.21.15.86:0/3232155717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92f180 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd5340325f0 tx=0x7fd534032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80ba615095d7aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd531ffb640 1 -- 172.21.15.86:0/3232155717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534039040 con 0x561b7d92ed60 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd531ffb640 1 -- 172.21.15.86:0/3232155717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd534032870 con 0x561b7d92ed60 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd531ffb640 1 -- 172.21.15.86:0/3232155717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5340346a0 con 0x561b7d92ed60 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd53bfff640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd5280a3bb0 0x7fd5280a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd53404c000 tx=0x7fd534048000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd544beb740 1 -- 172.21.15.86:0/3232155717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 msgr2=0x561b7d92f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd544beb740 1 --2- 172.21.15.86:0/3232155717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92f180 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd5340325f0 tx=0x7fd534032e20 comp rx=0 tx=0).stop 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd544beb740 1 -- 172.21.15.86:0/3232155717 shutdown_connections 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd544beb740 1 --2- 172.21.15.86:0/3232155717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d936d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd544beb740 1 --2- 172.21.15.86:0/3232155717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d934250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd544beb740 1 --2- 172.21.15.86:0/3232155717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92f180 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.732+0000 7fd544beb740 1 -- 172.21.15.86:0/3232155717 >> 172.21.15.86:0/3232155717 conn(0x561b7d92a2d0 msgr2=0x561b7d92a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 -- 172.21.15.86:0/3232155717 shutdown_connections 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 -- 172.21.15.86:0/3232155717 wait complete. 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 -- start start 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 Processor -- start 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d938f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d9394d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d940510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b7d8990f0 con 0x561b7d92ed60 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561b7d893dd0 con 0x561b7d92f750 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b7d8a1cd0 con 0x561b7d9348c0 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd544beb740 10 client.0 _ll_register_callbacks cb 0x561b7d88a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd53b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d938f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd53b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d938f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53252/0 (socket says 172.21.15.86:53252) 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.733+0000 7fd53b7fe640 1 -- 172.21.15.86:0/3948403124 learned_addr learned my addr 172.21.15.86:0/3948403124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd53bfff640 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d940510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd53b7fe640 1 -- 172.21.15.86:0/3948403124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 msgr2=0x561b7d940510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd532ffd640 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d9394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d940510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd53b7fe640 1 -- 172.21.15.86:0/3948403124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 msgr2=0x561b7d9394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d9394d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd53b7fe640 1 -- 172.21.15.86:0/3948403124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b7d5a72d0 con 0x561b7d92ed60 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d938f90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00c940 tx=0x7fd52c098160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bad3a27a5017f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd51affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd51affd640 1 -- 172.21.15.86:0/3948403124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c01d040 con 0x561b7d92ed60 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd51affd640 1 -- 172.21.15.86:0/3948403124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd52c003610 con 0x561b7d92ed60 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd544beb740 1 -- 172.21.15.86:0/3948403124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd534032870 con 0x561b7d92ed60 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd51affd640 1 -- 172.21.15.86:0/3948403124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd52c09c4f0 con 0x561b7d92ed60 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd544beb740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd544beb740 1 -- 172.21.15.86:0/3948403124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b7d6716b0 con 0x561b7d92ed60 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd51affd640 1 -- 172.21.15.86:0/3948403124 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd534032870 con 0x561b7d92ed60 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd51affd640 1 -- 172.21.15.86:0/3948403124 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd52c09b940 con 0x561b7d92ed60 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd544beb740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.734+0000 7fd544beb740 10 client.15462 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-16T05:35:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.735+0000 7fd544beb740 20 client.15462 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.735+0000 7fd544beb740 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x561b7d945c30 0x561b7d9470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.735+0000 7fd544beb740 4 client.15462 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.735+0000 7fd544beb740 1 -- 172.21.15.86:0/3948403124 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b7d81b4c0 con 0x561b7d945c30 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.735+0000 7fd532ffd640 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x561b7d945c30 0x561b7d9470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.735+0000 7fd51affd640 10 client.15462 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.735+0000 7fd532ffd640 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x561b7d945c30 0x561b7d9470a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59eae129af1ef6f9 server_cookie=a6fcb0c51382a06c in_seq=0 out_seq=0 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd51affd640 1 -- 172.21.15.86:0/3948403124 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b7d81b4c0 con 0x561b7d945c30 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd51affd640 10 client.15462 handle_command_reply: tid=0 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 client.15462 shutdown 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 -- 172.21.15.86:0/3948403124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 msgr2=0x561b7d938f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d938f90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00c940 tx=0x7fd52c098160 comp rx=0 tx=0).stop 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 -- 172.21.15.86:0/3948403124 shutdown_connections 2024-04-16T05:35:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d940510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d9394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x561b7d945c30 0x561b7d9470a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 --2- 172.21.15.86:0/3948403124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d938f90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 -- 172.21.15.86:0/3948403124 >> 172.21.15.86:0/3948403124 conn(0x561b7d92a2d0 msgr2=0x561b7d929410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd51affd640 0 client.15462 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:35:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 -- 172.21.15.86:0/3948403124 shutdown_connections 2024-04-16T05:35:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 1 -- 172.21.15.86:0/3948403124 wait complete. 2024-04-16T05:35:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.736+0000 7fd544beb740 20 client.15462 trim_cache size 0 max 16384 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd544beb740 1 Processor -- start 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd544beb740 1 -- start start 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd544beb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd544beb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d930110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd544beb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d938ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd544beb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b7d9453e0 con 0x561b7d92ed60 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd544beb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561b7d945ab0 con 0x561b7d92f750 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd544beb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b7d946180 con 0x561b7d9348c0 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd53b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd53b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53268/0 (socket says 172.21.15.86:53268) 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd53b7fe640 1 -- 172.21.15.86:0/2956351568 learned_addr learned my addr 172.21.15.86:0/2956351568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.754+0000 7fd532ffd640 1 --2- 172.21.15.86:0/2956351568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d930110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd53b7fe640 1 -- 172.21.15.86:0/2956351568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 msgr2=0x561b7d938ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/2956351568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d938ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd53b7fe640 1 -- 172.21.15.86:0/2956351568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 msgr2=0x561b7d930110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/2956351568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d930110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd53b7fe640 1 -- 172.21.15.86:0/2956351568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7d92b040 con 0x561b7d92ed60 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd532ffd640 1 --2- 172.21.15.86:0/2956351568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d930110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/2956351568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92a6b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00afd0 tx=0x7fd52c087070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50285e43d41ba7a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd5317fa640 1 -- 172.21.15.86:0/2956351568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c01d040 con 0x561b7d92ed60 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd5317fa640 1 -- 172.21.15.86:0/2956351568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c0a5160 con 0x561b7d92ed60 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd5317fa640 1 -- 172.21.15.86:0/2956351568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c09b940 con 0x561b7d92ed60 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd544beb740 1 -- 172.21.15.86:0/2956351568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 msgr2=0x561b7d92a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd544beb740 1 --2- 172.21.15.86:0/2956351568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92a6b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00afd0 tx=0x7fd52c087070 comp rx=0 tx=0).stop 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd544beb740 1 -- 172.21.15.86:0/2956351568 shutdown_connections 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd544beb740 1 --2- 172.21.15.86:0/2956351568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d9348c0 0x561b7d938ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd544beb740 1 --2- 172.21.15.86:0/2956351568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92f750 0x561b7d930110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd544beb740 1 --2- 172.21.15.86:0/2956351568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d92ed60 0x561b7d92a6b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd544beb740 1 -- 172.21.15.86:0/2956351568 >> 172.21.15.86:0/2956351568 conn(0x561b7d92a2d0 msgr2=0x561b7d944c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.755+0000 7fd544beb740 1 -- 172.21.15.86:0/2956351568 shutdown_connections 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 1 -- 172.21.15.86:0/2956351568 wait complete. 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 1 -- start start 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 1 Processor -- start 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92ed60 0x561b7d9a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d92f750 0x561b7d9ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d9348c0 0x561b7d9ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b7d946f20 con 0x561b7d9348c0 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561b7d946850 con 0x561b7d92ed60 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b7d9475f0 con 0x561b7d92f750 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd544beb740 10 client.0 _ll_register_callbacks cb 0x7fd52c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:08.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd53bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d9348c0 0x561b7d9ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd53bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d9348c0 0x561b7d9ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53272/0 (socket says 172.21.15.86:53272) 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd53bfff640 1 -- 172.21.15.86:0/1704276784 learned_addr learned my addr 172.21.15.86:0/1704276784 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.756+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92ed60 0x561b7d9a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd53bfff640 1 -- 172.21.15.86:0/1704276784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d92f750 msgr2=0x561b7d9ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd532ffd640 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d92f750 0x561b7d9ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd53bfff640 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d92f750 0x561b7d9ac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd53bfff640 1 -- 172.21.15.86:0/1704276784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92ed60 msgr2=0x561b7d9a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd53bfff640 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92ed60 0x561b7d9a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd53bfff640 1 -- 172.21.15.86:0/1704276784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b7d9aef70 con 0x561b7d9348c0 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd53bfff640 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d9348c0 0x561b7d9ae740 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd534009ca0 tx=0x7fd534032d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606d274b3aaa67d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd51affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd51affd640 1 -- 172.21.15.86:0/1704276784 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53402cdc0 con 0x561b7d9348c0 2024-04-16T05:35:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd51affd640 1 -- 172.21.15.86:0/1704276784 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd534032870 con 0x561b7d9348c0 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd544beb740 1 -- 172.21.15.86:0/1704276784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd52c0a5160 con 0x561b7d9348c0 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd51affd640 1 -- 172.21.15.86:0/1704276784 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd534067050 con 0x561b7d9348c0 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd544beb740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd544beb740 1 -- 172.21.15.86:0/1704276784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b7d9b0270 con 0x561b7d9348c0 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd51affd640 1 -- 172.21.15.86:0/1704276784 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c0a5160 con 0x561b7d9348c0 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd51affd640 1 -- 172.21.15.86:0/1704276784 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd534004040 con 0x561b7d9348c0 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd544beb740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd544beb740 10 client.15468 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd544beb740 20 client.15468 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd544beb740 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x561b7d9b49e0 0x561b7d9b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd544beb740 4 client.15468 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.757+0000 7fd544beb740 1 -- 172.21.15.86:0/1704276784 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "status"}) -- 0x561b7d8a24e0 con 0x561b7d9b49e0 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x561b7d9b49e0 0x561b7d9b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd51affd640 10 client.15468 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd53b7fe640 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x561b7d9b49e0 0x561b7d9b36f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a9a222d100b44f2 server_cookie=90d509b7619015a6 in_seq=0 out_seq=0 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd51affd640 1 -- 172.21.15.86:0/1704276784 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561b7d8a24e0 con 0x561b7d9b49e0 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd51affd640 10 client.15468 handle_command_reply: tid=0 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd544beb740 1 client.15468 shutdown 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd544beb740 1 -- 172.21.15.86:0/1704276784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d9348c0 msgr2=0x561b7d9ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd544beb740 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d9348c0 0x561b7d9ae740 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd534009ca0 tx=0x7fd534032d20 comp rx=0 tx=0).stop 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd544beb740 1 -- 172.21.15.86:0/1704276784 shutdown_connections 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd544beb740 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x561b7d9b49e0 0x561b7d9b36f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd544beb740 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7d9348c0 0x561b7d9ae740 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd544beb740 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b7d92f750 0x561b7d9ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd51affd640 0 client.15468 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd544beb740 1 --2- 172.21.15.86:0/1704276784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b7d92ed60 0x561b7d9a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.758+0000 7fd544beb740 1 -- 172.21.15.86:0/1704276784 >> 172.21.15.86:0/1704276784 conn(0x561b7d92a2d0 msgr2=0x561b7d92a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd544beb740 1 -- 172.21.15.86:0/1704276784 shutdown_connections 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd544beb740 1 -- 172.21.15.86:0/1704276784 wait complete. 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd544beb740 20 client.15468 trim_cache size 0 max 16384 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24463, 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 113.16638706000001, 2024-04-16T05:35:08.765 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:35:08.766 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-16T05:35:08.766 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:35:08.766 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 153.91001597900001 2024-04-16T05:35:08.766 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 -- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd5280a3bb0 msgr2=0x7fd5280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd5280a3bb0 0x7fd5280a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd53404c000 tx=0x7fd534048000 comp rx=0 tx=0).stop 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 -- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c102050 msgr2=0x7fd53c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c102050 0x7fd53c198310 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00c970 tx=0x7fd52c00ce30 comp rx=0 tx=0).stop 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 -- 172.21.15.86:0/2209255137 shutdown_connections 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fd5280a3bb0 0x7fd5280a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd53c103c00 0x7fd53c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd53c103240 0x7fd53c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 --2- 172.21.15.86:0/2209255137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd53c102050 0x7fd53c198310 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 -- 172.21.15.86:0/2209255137 >> 172.21.15.86:0/2209255137 conn(0x7fd53c0fd5e0 msgr2=0x7fd53c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 -- 172.21.15.86:0/2209255137 shutdown_connections 2024-04-16T05:35:08.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.759+0000 7fd541f24640 1 -- 172.21.15.86:0/2209255137 wait complete. 2024-04-16T05:35:08.769 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-16T05:35:08.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 -- 172.21.15.86:0/3934691938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 msgr2=0x7fb24c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 --2- 172.21.15.86:0/3934691938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 0x7fb24c102510 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb234009900 tx=0x7fb23402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 -- 172.21.15.86:0/3934691938 shutdown_connections 2024-04-16T05:35:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 --2- 172.21.15.86:0/3934691938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb24c102a50 0x7fb24c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 --2- 172.21.15.86:0/3934691938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 0x7fb24c102510 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 --2- 172.21.15.86:0/3934691938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c100f40 0x7fb24c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 -- 172.21.15.86:0/3934691938 >> 172.21.15.86:0/3934691938 conn(0x7fb24c0fc4b0 msgr2=0x7fb24c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 -- 172.21.15.86:0/3934691938 shutdown_connections 2024-04-16T05:35:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 -- 172.21.15.86:0/3934691938 wait complete. 2024-04-16T05:35:08.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.875+0000 7fb250b8d640 1 Processor -- start 2024-04-16T05:35:08.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb250b8d640 1 -- start start 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb250b8d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb24c100f40 0x7fb24c069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb250b8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 0x7fb24c19d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb250b8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c102a50 0x7fb24c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb250b8d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c10cbd0 con 0x7fb24c102090 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb250b8d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c10ca50 con 0x7fb24c100f40 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb250b8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c10cd50 con 0x7fb24c102a50 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb241d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 0x7fb24c19d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb241d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 0x7fb24c19d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53290/0 (socket says 172.21.15.86:53290) 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb241d9b640 1 -- 172.21.15.86:0/33832768 learned_addr learned my addr 172.21.15.86:0/33832768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb24a59c640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb24c100f40 0x7fb24c069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c102a50 0x7fb24c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb241d9b640 1 -- 172.21.15.86:0/33832768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c102a50 msgr2=0x7fb24c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb241d9b640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c102a50 0x7fb24c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb241d9b640 1 -- 172.21.15.86:0/33832768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb24c100f40 msgr2=0x7fb24c069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb241d9b640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb24c100f40 0x7fb24c069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb241d9b640 1 -- 172.21.15.86:0/33832768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c1a0050 con 0x7fb24c102090 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c102a50 0x7fb24c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb24a59c640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb24c100f40 0x7fb24c069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb241d9b640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 0x7fb24c19d120 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb234000c00 tx=0x7fb234005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.876+0000 7fb243fff640 1 -- 172.21.15.86:0/33832768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234041070 con 0x7fb24c102090 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.877+0000 7fb243fff640 1 -- 172.21.15.86:0/33832768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2340044d0 con 0x7fb24c102090 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.877+0000 7fb250b8d640 1 -- 172.21.15.86:0/33832768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c1a0280 con 0x7fb24c102090 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.877+0000 7fb243fff640 1 -- 172.21.15.86:0/33832768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234034440 con 0x7fb24c102090 2024-04-16T05:35:08.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.877+0000 7fb250b8d640 1 -- 172.21.15.86:0/33832768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c1a0770 con 0x7fb24c102090 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb243fff640 1 -- 172.21.15.86:0/33832768 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb234044070 con 0x7fb24c102090 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb253772740 1 Processor -- start 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb253772740 1 -- start start 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb253772740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 0x564ad5bf8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb253772740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bfd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb253772740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 0x564ad5bffd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb253772740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ad5b0b240 con 0x564ad5bf8750 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb253772740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564ad5b689f0 con 0x564ad5bfd8c0 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb253772740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ad5b239e0 con 0x564ad5bf7d60 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.879+0000 7fb24a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bfd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bfd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53294/0 (socket says 172.21.15.86:53294) 2024-04-16T05:35:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24a59c640 1 -- 172.21.15.86:0/2715119121 learned_addr learned my addr 172.21.15.86:0/2715119121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb241d9b640 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 0x564ad5bffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 0x564ad5bf8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24a59c640 1 -- 172.21.15.86:0/2715119121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 msgr2=0x564ad5bf8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24a59c640 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 0x564ad5bf8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24a59c640 1 -- 172.21.15.86:0/2715119121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 msgr2=0x564ad5bffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24a59c640 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 0x564ad5bffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb243fff640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2380a3bb0 0x7fb2380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24a59c640 1 -- 172.21.15.86:0/2715119121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad5b16bc0 con 0x564ad5bf8750 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 0x564ad5bf8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2380a3bb0 0x7fb2380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb243fff640 1 -- 172.21.15.86:0/33832768 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb2340f2300 con 0x7fb24c102090 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb24a59c640 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bfd250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0104d0 tx=0x7fb23c010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d500b17ceeb361 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb240d99640 1 -- 172.21.15.86:0/2715119121 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c004220 con 0x564ad5bf8750 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb240d99640 1 -- 172.21.15.86:0/2715119121 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb23c0043c0 con 0x564ad5bf8750 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.880+0000 7fb240d99640 1 -- 172.21.15.86:0/2715119121 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c011a70 con 0x564ad5bf8750 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 -- 172.21.15.86:0/2715119121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 msgr2=0x564ad5bfd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2380a3bb0 0x7fb2380a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x564ad5bf8fb0 tx=0x7fb244009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bfd250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0104d0 tx=0x7fb23c010990 comp rx=0 tx=0).stop 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 -- 172.21.15.86:0/2715119121 shutdown_connections 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 0x564ad5bffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bfd250 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 --2- 172.21.15.86:0/2715119121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 0x564ad5bf8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 -- 172.21.15.86:0/2715119121 >> 172.21.15.86:0/2715119121 conn(0x564ad5bf32d0 msgr2=0x564ad5bf36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 -- 172.21.15.86:0/2715119121 shutdown_connections 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 -- 172.21.15.86:0/2715119121 wait complete. 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 -- start start 2024-04-16T05:35:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.881+0000 7fb253772740 1 Processor -- start 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb253772740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf7d60 0x564ad5c01fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb253772740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf8750 0x564ad5c02510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb253772740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 0x564ad5c09550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb253772740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ad5b620f0 con 0x564ad5bf7d60 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb253772740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564ad5b5cdd0 con 0x564ad5bfd8c0 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb253772740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ad5b6acd0 con 0x564ad5bf8750 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb253772740 10 client.0 _ll_register_callbacks cb 0x564ad5b53570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf7d60 0x564ad5c01fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf7d60 0x564ad5c01fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53308/0 (socket says 172.21.15.86:53308) 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24a59c640 1 -- 172.21.15.86:0/4196793892 learned_addr learned my addr 172.21.15.86:0/4196793892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb241d9b640 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf8750 0x564ad5c02510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 0x564ad5c09550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24a59c640 1 -- 172.21.15.86:0/4196793892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf8750 msgr2=0x564ad5c02510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24a59c640 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf8750 0x564ad5c02510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24a59c640 1 -- 172.21.15.86:0/4196793892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 msgr2=0x564ad5c09550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24a59c640 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 0x564ad5c09550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24a59c640 1 -- 172.21.15.86:0/4196793892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad58702d0 con 0x564ad5bf7d60 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb241d9b640 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf8750 0x564ad5c02510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 0x564ad5c09550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb24a59c640 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf7d60 0x564ad5c01fd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0104a0 tx=0x7fb23c004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526d9c42eaa6cf84 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb231ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb231ffb640 1 -- 172.21.15.86:0/4196793892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c029060 con 0x564ad5bf7d60 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb231ffb640 1 -- 172.21.15.86:0/4196793892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb23c003800 con 0x564ad5bf7d60 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.882+0000 7fb253772740 1 -- 172.21.15.86:0/4196793892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb23c0043c0 con 0x564ad5bf7d60 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb231ffb640 1 -- 172.21.15.86:0/4196793892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb23c030050 con 0x564ad5bf7d60 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb253772740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb253772740 1 -- 172.21.15.86:0/4196793892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ad593a6b0 con 0x564ad5bf7d60 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb231ffb640 1 -- 172.21.15.86:0/4196793892 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb23c0043c0 con 0x564ad5bf7d60 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb231ffb640 1 -- 172.21.15.86:0/4196793892 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb23c011a40 con 0x564ad5bf7d60 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb253772740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb253772740 10 client.15480 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb253772740 20 client.15480 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb253772740 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x564ad5c0dac0 0x564ad5c0fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb253772740 4 client.15480 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb253772740 1 -- 172.21.15.86:0/4196793892 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ad5ae44c0 con 0x564ad5c0dac0 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.883+0000 7fb241d9b640 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x564ad5c0dac0 0x564ad5c0fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.884+0000 7fb241d9b640 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x564ad5c0dac0 0x564ad5c0fc30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e933251d14ca52b9 server_cookie=d581684cefcaf8c0 in_seq=0 out_seq=0 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.884+0000 7fb231ffb640 10 client.15480 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb231ffb640 1 -- 172.21.15.86:0/4196793892 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ad5ae44c0 con 0x564ad5c0dac0 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb231ffb640 10 client.15480 handle_command_reply: tid=0 2024-04-16T05:35:08.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 client.15480 shutdown 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 -- 172.21.15.86:0/4196793892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf7d60 msgr2=0x564ad5c01fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf7d60 0x564ad5c01fd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0104a0 tx=0x7fb23c004230 comp rx=0 tx=0).stop 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 -- 172.21.15.86:0/4196793892 shutdown_connections 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x564ad5c0dac0 0x564ad5c0fc30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bfd8c0 0x564ad5c09550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb231ffb640 0 client.15480 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf8750 0x564ad5c02510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 --2- 172.21.15.86:0/4196793892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf7d60 0x564ad5c01fd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 -- 172.21.15.86:0/4196793892 >> 172.21.15.86:0/4196793892 conn(0x564ad5bf32d0 msgr2=0x564ad5bf2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 -- 172.21.15.86:0/4196793892 shutdown_connections 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 1 -- 172.21.15.86:0/4196793892 wait complete. 2024-04-16T05:35:08.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.885+0000 7fb253772740 20 client.15480 trim_cache size 0 max 16384 2024-04-16T05:35:08.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb253772740 1 Processor -- start 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb253772740 1 -- start start 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb253772740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf32d0 0x564ad5bffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb253772740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 0x564ad5bf1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb253772740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bf20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb253772740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ad5c0df70 con 0x564ad5bf8750 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb253772740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564ad5c0e640 con 0x564ad5bf32d0 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb253772740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ad5c0ed10 con 0x564ad5bf7d60 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bf20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bf20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53312/0 (socket says 172.21.15.86:53312) 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24ad9d640 1 -- 172.21.15.86:0/1849246115 learned_addr learned my addr 172.21.15.86:0/1849246115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb241d9b640 1 --2- 172.21.15.86:0/1849246115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 0x564ad5bf1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24ad9d640 1 -- 172.21.15.86:0/1849246115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 msgr2=0x564ad5bf1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/1849246115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 0x564ad5bf1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24ad9d640 1 -- 172.21.15.86:0/1849246115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf32d0 msgr2=0x564ad5bffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24a59c640 1 --2- 172.21.15.86:0/1849246115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf32d0 0x564ad5bffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/1849246115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf32d0 0x564ad5bffcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24ad9d640 1 -- 172.21.15.86:0/1849246115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad5bf8b30 con 0x564ad5bf8750 2024-04-16T05:35:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.903+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/1849246115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bf20c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb2440128c0 tx=0x7fb244009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a360d47446b77a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb233fff640 1 -- 172.21.15.86:0/1849246115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244011b60 con 0x564ad5bf8750 2024-04-16T05:35:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb233fff640 1 -- 172.21.15.86:0/1849246115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb244011d00 con 0x564ad5bf8750 2024-04-16T05:35:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb233fff640 1 -- 172.21.15.86:0/1849246115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244028620 con 0x564ad5bf8750 2024-04-16T05:35:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 -- 172.21.15.86:0/1849246115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 msgr2=0x564ad5bf20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 --2- 172.21.15.86:0/1849246115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bf20c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb2440128c0 tx=0x7fb244009f90 comp rx=0 tx=0).stop 2024-04-16T05:35:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 -- 172.21.15.86:0/1849246115 shutdown_connections 2024-04-16T05:35:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 --2- 172.21.15.86:0/1849246115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5bf20c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 --2- 172.21.15.86:0/1849246115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf7d60 0x564ad5bf1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 --2- 172.21.15.86:0/1849246115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf32d0 0x564ad5bffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 -- 172.21.15.86:0/1849246115 >> 172.21.15.86:0/1849246115 conn(0x564ad5bf9530 msgr2=0x564ad5bf36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 -- 172.21.15.86:0/1849246115 shutdown_connections 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 -- 172.21.15.86:0/1849246115 wait complete. 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.904+0000 7fb253772740 1 -- start start 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb253772740 1 Processor -- start 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb253772740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf32d0 0x564ad5c709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb253772740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf7d60 0x564ad5c70ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb253772740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5c778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb253772740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ad5c0fab0 con 0x564ad5bf8750 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb253772740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564ad5c0f3e0 con 0x564ad5bf7d60 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb253772740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ad5c10180 con 0x564ad5bf32d0 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb253772740 10 client.0 _ll_register_callbacks cb 0x7fb23c00bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5c778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5c778c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53322/0 (socket says 172.21.15.86:53322) 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24ad9d640 1 -- 172.21.15.86:0/592002202 learned_addr learned my addr 172.21.15.86:0/592002202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb241d9b640 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf7d60 0x564ad5c70ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24a59c640 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf32d0 0x564ad5c709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24ad9d640 1 -- 172.21.15.86:0/592002202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf32d0 msgr2=0x564ad5c709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf32d0 0x564ad5c709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24ad9d640 1 -- 172.21.15.86:0/592002202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf7d60 msgr2=0x564ad5c70ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf7d60 0x564ad5c70ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24ad9d640 1 -- 172.21.15.86:0/592002202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad5c77f00 con 0x564ad5bf8750 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb24ad9d640 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5c778c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb244005b00 tx=0x7fb244011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572a1c01be827e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb231ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb231ffb640 1 -- 172.21.15.86:0/592002202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244004a40 con 0x564ad5bf8750 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb231ffb640 1 -- 172.21.15.86:0/592002202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb244004be0 con 0x564ad5bf8750 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.905+0000 7fb253772740 1 -- 172.21.15.86:0/592002202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb244011d00 con 0x564ad5bf8750 2024-04-16T05:35:08.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb231ffb640 1 -- 172.21.15.86:0/592002202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb244004de0 con 0x564ad5bf8750 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb253772740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb253772740 1 -- 172.21.15.86:0/592002202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ad5c791a0 con 0x564ad5bf8750 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb231ffb640 1 -- 172.21.15.86:0/592002202 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb244011d00 con 0x564ad5bf8750 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb231ffb640 1 -- 172.21.15.86:0/592002202 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb244030030 con 0x564ad5bf8750 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb253772740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb253772740 10 client.15486 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb253772740 20 client.15486 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb253772740 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x564ad5bfd8c0 0x564ad5c7c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb253772740 4 client.15486 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb253772740 1 -- 172.21.15.86:0/592002202 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "status"}) -- 0x564ad5b6b4e0 con 0x564ad5bfd8c0 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.906+0000 7fb24a59c640 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x564ad5bfd8c0 0x564ad5c7c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb24a59c640 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x564ad5bfd8c0 0x564ad5c7c8f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7834faefdbf0323 server_cookie=20b9f3129d235fb7 in_seq=0 out_seq=0 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb231ffb640 10 client.15486 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb231ffb640 1 -- 172.21.15.86:0/592002202 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564ad5b6b4e0 con 0x564ad5bfd8c0 2024-04-16T05:35:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb231ffb640 10 client.15486 handle_command_reply: tid=0 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 client.15486 shutdown 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 -- 172.21.15.86:0/592002202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 msgr2=0x564ad5c778c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5c778c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb244005b00 tx=0x7fb244011960 comp rx=0 tx=0).stop 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 -- 172.21.15.86:0/592002202 shutdown_connections 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x564ad5bfd8c0 0x564ad5c7c8f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ad5bf8750 0x564ad5c778c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ad5bf7d60 0x564ad5c70ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb231ffb640 0 client.15486 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 --2- 172.21.15.86:0/592002202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ad5bf32d0 0x564ad5c709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 -- 172.21.15.86:0/592002202 >> 172.21.15.86:0/592002202 conn(0x564ad5bf9530 msgr2=0x564ad5bf1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 -- 172.21.15.86:0/592002202 shutdown_connections 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 1 -- 172.21.15.86:0/592002202 wait complete. 2024-04-16T05:35:08.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb253772740 20 client.15486 trim_cache size 0 max 16384 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24481, 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 114.328591809, 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:35:08.915 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 148.32820301999999 2024-04-16T05:35:08.916 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:08.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb250b8d640 1 -- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2380a3bb0 msgr2=0x7fb2380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb250b8d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2380a3bb0 0x7fb2380a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x564ad5bf8fb0 tx=0x7fb244009500 comp rx=0 tx=0).stop 2024-04-16T05:35:08.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb250b8d640 1 -- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 msgr2=0x7fb24c19d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:08.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.907+0000 7fb250b8d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 0x7fb24c19d120 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb234000c00 tx=0x7fb234005d50 comp rx=0 tx=0).stop 2024-04-16T05:35:08.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.908+0000 7fb250b8d640 1 -- 172.21.15.86:0/33832768 shutdown_connections 2024-04-16T05:35:08.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.908+0000 7fb250b8d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c102a50 0x7fb24c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.908+0000 7fb250b8d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24c102090 0x7fb24c19d120 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.908+0000 7fb250b8d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fb2380a3bb0 0x7fb2380a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.908+0000 7fb250b8d640 1 --2- 172.21.15.86:0/33832768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb24c100f40 0x7fb24c069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:08.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.908+0000 7fb250b8d640 1 -- 172.21.15.86:0/33832768 >> 172.21.15.86:0/33832768 conn(0x7fb24c0fc4b0 msgr2=0x7fb24c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:08.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.908+0000 7fb250b8d640 1 -- 172.21.15.86:0/33832768 shutdown_connections 2024-04-16T05:35:08.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:08.908+0000 7fb250b8d640 1 -- 172.21.15.86:0/33832768 wait complete. 2024-04-16T05:35:08.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:08 smithi086 ceph-mon[32028]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:35:08.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:08 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2240320856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:08.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:08 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1458172598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:08.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T05:35:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:08 smithi196 ceph-mon[36373]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:35:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:08 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2240320856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:08 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1458172598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.024+0000 7f6e41a60640 1 -- 172.21.15.86:0/4293313565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 msgr2=0x7f6e3c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.024+0000 7f6e41a60640 1 --2- 172.21.15.86:0/4293313565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 0x7f6e3c10a490 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009ca0 tx=0x7f6e3402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.024+0000 7f6e41a60640 1 -- 172.21.15.86:0/4293313565 shutdown_connections 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.024+0000 7f6e41a60640 1 --2- 172.21.15.86:0/4293313565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 0x7f6e3c10a490 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.024+0000 7f6e41a60640 1 --2- 172.21.15.86:0/4293313565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e3c103240 0x7f6e3c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.024+0000 7f6e41a60640 1 --2- 172.21.15.86:0/4293313565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e3c102050 0x7f6e3c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.024+0000 7f6e41a60640 1 -- 172.21.15.86:0/4293313565 >> 172.21.15.86:0/4293313565 conn(0x7f6e3c0fd5e0 msgr2=0x7f6e3c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.024+0000 7f6e41a60640 1 -- 172.21.15.86:0/4293313565 shutdown_connections 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.024+0000 7f6e41a60640 1 -- 172.21.15.86:0/4293313565 wait complete. 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e41a60640 1 Processor -- start 2024-04-16T05:35:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e41a60640 1 -- start start 2024-04-16T05:35:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e41a60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e3c102050 0x7f6e3c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e41a60640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e3c103240 0x7f6e3c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e41a60640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 0x7f6e3c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e41a60640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c10dce0 con 0x7f6e3c103c00 2024-04-16T05:35:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e41a60640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c10db60 con 0x7f6e3c103240 2024-04-16T05:35:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e41a60640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c10de60 con 0x7f6e3c102050 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 0x7f6e3c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 0x7f6e3c19f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53340/0 (socket says 172.21.15.86:53340) 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.025+0000 7f6e3b7fe640 1 -- 172.21.15.86:0/1071310585 learned_addr learned my addr 172.21.15.86:0/1071310585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e3affd640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e3c102050 0x7f6e3c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e327fc640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e3c103240 0x7f6e3c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e3b7fe640 1 -- 172.21.15.86:0/1071310585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e3c102050 msgr2=0x7f6e3c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e3b7fe640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e3c102050 0x7f6e3c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e3b7fe640 1 -- 172.21.15.86:0/1071310585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e3c103240 msgr2=0x7f6e3c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e3b7fe640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e3c103240 0x7f6e3c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e3b7fe640 1 -- 172.21.15.86:0/1071310585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c19ff50 con 0x7f6e3c103c00 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e3b7fe640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 0x7f6e3c19f8e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6e3402c700 tx=0x7f6e34032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e40a5e640 1 -- 172.21.15.86:0/1071310585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34039040 con 0x7f6e3c103c00 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e41a60640 1 -- 172.21.15.86:0/1071310585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c1a01e0 con 0x7f6e3c103c00 2024-04-16T05:35:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e40a5e640 1 -- 172.21.15.86:0/1071310585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e34044070 con 0x7f6e3c103c00 2024-04-16T05:35:09.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e40a5e640 1 -- 172.21.15.86:0/1071310585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e340042a0 con 0x7f6e3c103c00 2024-04-16T05:35:09.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.026+0000 7f6e41a60640 1 -- 172.21.15.86:0/1071310585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c1a06d0 con 0x7f6e3c103c00 2024-04-16T05:35:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.028+0000 7f6e40a5e640 1 -- 172.21.15.86:0/1071310585 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6e34003970 con 0x7f6e3c103c00 2024-04-16T05:35:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e44647740 1 Processor -- start 2024-04-16T05:35:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e44647740 1 -- start start 2024-04-16T05:35:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e44647740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221dd60 0x55b72221e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e44647740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221e750 0x55b722223250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e44647740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b722225d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e44647740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b722131240 con 0x55b72221e750 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e44647740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b72218e9f0 con 0x55b72221dd60 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e44647740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b7221499e0 con 0x55b7222238c0 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e40a5e640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e280a3bb0 0x7f6e280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e3affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221e750 0x55b722223250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e3affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221e750 0x55b722223250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53344/0 (socket says 172.21.15.86:53344) 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e3affd640 1 -- 172.21.15.86:0/2241169923 learned_addr learned my addr 172.21.15.86:0/2241169923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e327fc640 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221dd60 0x55b72221e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e3b7fe640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e280a3bb0 0x7f6e280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e40a5e640 1 -- 172.21.15.86:0/1071310585 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6e340f2ce0 con 0x7f6e3c103c00 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e327fc640 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b722225d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e3affd640 1 -- 172.21.15.86:0/2241169923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 msgr2=0x55b722225d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e3affd640 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b722225d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e3affd640 1 -- 172.21.15.86:0/2241169923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221dd60 msgr2=0x55b72221e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e3affd640 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221dd60 0x55b72221e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e3affd640 1 -- 172.21.15.86:0/2241169923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72213cbc0 con 0x55b72221e750 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.029+0000 7f6e327fc640 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b722225d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e3affd640 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221e750 0x55b722223250 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00bcb0 tx=0x7f6e2c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ddeae1f95841c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e3b7fe640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e280a3bb0 0x7f6e280a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e340bb120 tx=0x7f6e3402c0f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e31ffb640 1 -- 172.21.15.86:0/2241169923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c005020 con 0x55b72221e750 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e31ffb640 1 -- 172.21.15.86:0/2241169923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c0047a0 con 0x55b72221e750 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e31ffb640 1 -- 172.21.15.86:0/2241169923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c011760 con 0x55b72221e750 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e44647740 1 -- 172.21.15.86:0/2241169923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221e750 msgr2=0x55b722223250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e44647740 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221e750 0x55b722223250 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00bcb0 tx=0x7f6e2c0107a0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e44647740 1 -- 172.21.15.86:0/2241169923 shutdown_connections 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e44647740 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b722225d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e44647740 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221e750 0x55b722223250 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e44647740 1 --2- 172.21.15.86:0/2241169923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221dd60 0x55b72221e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e44647740 1 -- 172.21.15.86:0/2241169923 >> 172.21.15.86:0/2241169923 conn(0x55b7222192d0 msgr2=0x55b7222196d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e44647740 1 -- 172.21.15.86:0/2241169923 shutdown_connections 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.030+0000 7f6e44647740 1 -- 172.21.15.86:0/2241169923 wait complete. 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e44647740 1 -- start start 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e44647740 1 Processor -- start 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e44647740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b722227fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e44647740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b7222284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e44647740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72222f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e44647740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b7221880f0 con 0x55b72221dd60 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e44647740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b722182dd0 con 0x55b72221e750 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e44647740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b722190cd0 con 0x55b7222238c0 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e44647740 10 client.0 _ll_register_callbacks cb 0x55b722179570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b722227fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b722227fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53350/0 (socket says 172.21.15.86:53350) 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e327fc640 1 -- 172.21.15.86:0/4271801728 learned_addr learned my addr 172.21.15.86:0/4271801728 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.031+0000 7f6e3affd640 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b7222284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e327fc640 1 -- 172.21.15.86:0/4271801728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 msgr2=0x55b72222f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e327fc640 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72222f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e327fc640 1 -- 172.21.15.86:0/4271801728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 msgr2=0x55b7222284e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e327fc640 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b7222284e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e327fc640 1 -- 172.21.15.86:0/4271801728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b721e962d0 con 0x55b72221dd60 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e3affd640 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b7222284e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e327fc640 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b722227fa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6e24011b20 tx=0x7f6e240103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8340f5842e93129 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e1affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e1affd640 1 -- 172.21.15.86:0/4271801728 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24004020 con 0x55b72221dd60 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e1affd640 1 -- 172.21.15.86:0/4271801728 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6e240041c0 con 0x55b72221dd60 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e44647740 1 -- 172.21.15.86:0/4271801728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2c0047a0 con 0x55b72221dd60 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e1affd640 1 -- 172.21.15.86:0/4271801728 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e240043c0 con 0x55b72221dd60 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e44647740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e44647740 1 -- 172.21.15.86:0/4271801728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b721f606b0 con 0x55b72221dd60 2024-04-16T05:35:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e1affd640 1 -- 172.21.15.86:0/4271801728 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c0047a0 con 0x55b72221dd60 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e1affd640 1 -- 172.21.15.86:0/4271801728 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e24019030 con 0x55b72221dd60 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e44647740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e44647740 10 client.15498 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e44647740 20 client.15498 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e44647740 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b722234d10 0x55b722236180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e44647740 4 client.15498 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.032+0000 7f6e44647740 1 -- 172.21.15.86:0/4271801728 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b72210a4c0 con 0x55b722234d10 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.033+0000 7f6e3affd640 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b722234d10 0x55b722236180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.033+0000 7f6e1affd640 10 client.15498 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.033+0000 7f6e3affd640 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b722234d10 0x55b722236180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf9d31a61413aa server_cookie=7c23c770b168413f in_seq=0 out_seq=0 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e1affd640 1 -- 172.21.15.86:0/4271801728 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b72210a4c0 con 0x55b722234d10 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e1affd640 10 client.15498 handle_command_reply: tid=0 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 client.15498 shutdown 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 -- 172.21.15.86:0/4271801728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 msgr2=0x55b722227fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b722227fa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6e24011b20 tx=0x7f6e240103d0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 -- 172.21.15.86:0/4271801728 shutdown_connections 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b722234d10 0x55b722236180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72222f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b7222284e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e1affd640 0 client.15498 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 --2- 172.21.15.86:0/4271801728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b722227fa0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 -- 172.21.15.86:0/4271801728 >> 172.21.15.86:0/4271801728 conn(0x55b7222192d0 msgr2=0x55b722218410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 -- 172.21.15.86:0/4271801728 shutdown_connections 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 1 -- 172.21.15.86:0/4271801728 wait complete. 2024-04-16T05:35:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.034+0000 7f6e44647740 20 client.15498 trim_cache size 0 max 16384 2024-04-16T05:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e44647740 1 Processor -- start 2024-04-16T05:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e44647740 1 -- start start 2024-04-16T05:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e44647740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b72221f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e44647740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b7222196b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e44647740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72221fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e44647740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b7222344c0 con 0x55b72221dd60 2024-04-16T05:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e44647740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b722234b90 con 0x55b72221e750 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e44647740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b722235260 con 0x55b7222238c0 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b72221f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b72221f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53358/0 (socket says 172.21.15.86:53358) 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e327fc640 1 -- 172.21.15.86:0/3304228114 learned_addr learned my addr 172.21.15.86:0/3304228114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e3affd640 1 --2- 172.21.15.86:0/3304228114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b7222196b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e327fc640 1 -- 172.21.15.86:0/3304228114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 msgr2=0x55b72221fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e327fc640 1 --2- 172.21.15.86:0/3304228114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72221fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e327fc640 1 -- 172.21.15.86:0/3304228114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 msgr2=0x55b7222196b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e327fc640 1 --2- 172.21.15.86:0/3304228114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b7222196b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e327fc640 1 -- 172.21.15.86:0/3304228114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b722232740 con 0x55b72221dd60 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.052+0000 7f6e327fc640 1 --2- 172.21.15.86:0/3304228114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b72221f1f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005e60 tx=0x7f6e240117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebcbd27d75c8385 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e317fa640 1 -- 172.21.15.86:0/3304228114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24005c20 con 0x55b72221dd60 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e317fa640 1 -- 172.21.15.86:0/3304228114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2402d070 con 0x55b72221dd60 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e317fa640 1 -- 172.21.15.86:0/3304228114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24028450 con 0x55b72221dd60 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 -- 172.21.15.86:0/3304228114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 msgr2=0x55b72221f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 --2- 172.21.15.86:0/3304228114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b72221f1f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005e60 tx=0x7f6e240117b0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 -- 172.21.15.86:0/3304228114 shutdown_connections 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 --2- 172.21.15.86:0/3304228114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72221fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 --2- 172.21.15.86:0/3304228114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b7222196b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 --2- 172.21.15.86:0/3304228114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b72221f1f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 -- 172.21.15.86:0/3304228114 >> 172.21.15.86:0/3304228114 conn(0x55b7222192d0 msgr2=0x55b722226af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 -- 172.21.15.86:0/3304228114 shutdown_connections 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 -- 172.21.15.86:0/3304228114 wait complete. 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.053+0000 7f6e44647740 1 -- start start 2024-04-16T05:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e44647740 1 Processor -- start 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e44647740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b7222219c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e44647740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b722221f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e44647740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72229cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e44647740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b722236000 con 0x55b72221dd60 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e44647740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b722235930 con 0x55b72221e750 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e44647740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b7222366d0 con 0x55b7222238c0 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e44647740 10 client.0 _ll_register_callbacks cb 0x7f6e2400e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b7222219c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b7222219c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53362/0 (socket says 172.21.15.86:53362) 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e327fc640 1 -- 172.21.15.86:0/3753221116 learned_addr learned my addr 172.21.15.86:0/3753221116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e3affd640 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b722221f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e327fc640 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72229cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e327fc640 1 -- 172.21.15.86:0/3753221116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 msgr2=0x55b72229cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e327fc640 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72229cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e327fc640 1 -- 172.21.15.86:0/3753221116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 msgr2=0x55b722221f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e327fc640 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b722221f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.054+0000 7f6e327fc640 1 -- 172.21.15.86:0/3753221116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72229d580 con 0x55b72221dd60 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e327fc640 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b7222219c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6e24004e20 tx=0x7f6e24011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b9f08c41da37c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e1affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e1affd640 1 -- 172.21.15.86:0/3753221116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24039020 con 0x55b72221dd60 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e1affd640 1 -- 172.21.15.86:0/3753221116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6e2400e6a0 con 0x55b72221dd60 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e44647740 1 -- 172.21.15.86:0/3753221116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2402d070 con 0x55b72221dd60 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e1affd640 1 -- 172.21.15.86:0/3753221116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e24028450 con 0x55b72221dd60 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e44647740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e44647740 1 -- 172.21.15.86:0/3753221116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b72229e880 con 0x55b72221dd60 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e1affd640 1 -- 172.21.15.86:0/3753221116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2402d070 con 0x55b72221dd60 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e1affd640 1 -- 172.21.15.86:0/3753221116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e24019040 con 0x55b72221dd60 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e44647740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e44647740 10 client.15504 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:35:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e44647740 20 client.15504 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e44647740 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b7222a3230 0x55b7222a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e44647740 4 client.15504 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.055+0000 7f6e44647740 1 -- 172.21.15.86:0/3753221116 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "status"}) -- 0x55b7221914e0 con 0x55b7222a3230 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e3affd640 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b7222a3230 0x55b7222a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e1affd640 10 client.15504 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e3affd640 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b7222a3230 0x55b7222a1f40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248cfdee36b2d996 server_cookie=b6eaebb7168990b7 in_seq=0 out_seq=0 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e1affd640 1 -- 172.21.15.86:0/3753221116 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b7221914e0 con 0x55b7222a3230 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e1affd640 10 client.15504 handle_command_reply: tid=0 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 client.15504 shutdown 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 -- 172.21.15.86:0/3753221116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 msgr2=0x55b7222219c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b7222219c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6e24004e20 tx=0x7f6e24011880 comp rx=0 tx=0).stop 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 -- 172.21.15.86:0/3753221116 shutdown_connections 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b7222a3230 0x55b7222a1f40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7222238c0 0x55b72229cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b72221e750 0x55b722221f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e1affd640 0 client.15504 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 --2- 172.21.15.86:0/3753221116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72221dd60 0x55b7222219c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 -- 172.21.15.86:0/3753221116 >> 172.21.15.86:0/3753221116 conn(0x55b7222192d0 msgr2=0x55b72221f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 -- 172.21.15.86:0/3753221116 shutdown_connections 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 1 -- 172.21.15.86:0/3753221116 wait complete. 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.056+0000 7f6e44647740 20 client.15504 trim_cache size 0 max 16384 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "6a4a23f8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24482, 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-16T05:35:09.062 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-16T05:35:09.063 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-16T05:35:09.063 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 133.21081881200001, 2024-04-16T05:35:09.063 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-16T05:35:09.063 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-16T05:35:09.063 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T05:35:09.063 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 139.57837203400001 2024-04-16T05:35:09.063 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 -- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e280a3bb0 msgr2=0x7f6e280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e280a3bb0 0x7f6e280a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e340bb120 tx=0x7f6e3402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 -- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 msgr2=0x7f6e3c19f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 0x7f6e3c19f8e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6e3402c700 tx=0x7f6e34032f20 comp rx=0 tx=0).stop 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 -- 172.21.15.86:0/1071310585 shutdown_connections 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c103c00 0x7f6e3c19f8e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f6e280a3bb0 0x7f6e280a6070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e3c103240 0x7f6e3c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 --2- 172.21.15.86:0/1071310585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e3c102050 0x7f6e3c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 -- 172.21.15.86:0/1071310585 >> 172.21.15.86:0/1071310585 conn(0x7f6e3c0fd5e0 msgr2=0x7f6e3c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 -- 172.21.15.86:0/1071310585 shutdown_connections 2024-04-16T05:35:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.057+0000 7f6e41a60640 1 -- 172.21.15.86:0/1071310585 wait complete. 2024-04-16T05:35:09.067 INFO:teuthology.run_tasks:Running task sequential... 2024-04-16T05:35:09.076 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-16T05:35:09.076 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-16T05:35:09.076 INFO:teuthology.task.print:Enabling mgr modules 2024-04-16T05:35:09.076 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-16T05:35:09.077 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T05:35:09.077 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:35:09.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-16T05:35:09.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:08 smithi194 ceph-mon[35865]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-16T05:35:09.180 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:08 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2240320856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:09.180 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:08 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1458172598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 -- 172.21.15.86:0/1124227949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f01011b0 msgr2=0x7fe8f0101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 --2- 172.21.15.86:0/1124227949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f01011b0 0x7fe8f0101630 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009900 tx=0x7fe8e402bc80 comp rx=0 tx=0).stop 2024-04-16T05:35:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 -- 172.21.15.86:0/1124227949 shutdown_connections 2024-04-16T05:35:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 --2- 172.21.15.86:0/1124227949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f0101b70 0x7fe8f0109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 --2- 172.21.15.86:0/1124227949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f01011b0 0x7fe8f0101630 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 --2- 172.21.15.86:0/1124227949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f01025f0 0x7fe8f0102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 -- 172.21.15.86:0/1124227949 >> 172.21.15.86:0/1124227949 conn(0x7fe8f00fbc60 msgr2=0x7fe8f00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 -- 172.21.15.86:0/1124227949 shutdown_connections 2024-04-16T05:35:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 -- 172.21.15.86:0/1124227949 wait complete. 2024-04-16T05:35:09.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.182+0000 7fe8f567b640 1 Processor -- start 2024-04-16T05:35:09.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8f567b640 1 -- start start 2024-04-16T05:35:09.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8f567b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f01011b0 0x7fe8f0100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8f567b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f0101b70 0x7fe8f019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8f567b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f01025f0 0x7fe8f019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8f567b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010cc40 con 0x7fe8f0101b70 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8f567b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010cac0 con 0x7fe8f01025f0 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8f567b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010cdc0 con 0x7fe8f01011b0 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ee7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f0101b70 0x7fe8f019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ee7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f0101b70 0x7fe8f019d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53390/0 (socket says 172.21.15.86:53390) 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ee7fc640 1 -- 172.21.15.86:0/2868112868 learned_addr learned my addr 172.21.15.86:0/2868112868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ef7fe640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f01025f0 0x7fe8f019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8eeffd640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f01011b0 0x7fe8f0100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ee7fc640 1 -- 172.21.15.86:0/2868112868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f01011b0 msgr2=0x7fe8f0100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ee7fc640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f01011b0 0x7fe8f0100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ee7fc640 1 -- 172.21.15.86:0/2868112868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f01025f0 msgr2=0x7fe8f019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ee7fc640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f01025f0 0x7fe8f019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ee7fc640 1 -- 172.21.15.86:0/2868112868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f019ff10 con 0x7fe8f0101b70 2024-04-16T05:35:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.183+0000 7fe8ee7fc640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f0101b70 0x7fe8f019d010 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4000c00 tx=0x7fe8e40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.184+0000 7fe8d7fff640 1 -- 172.21.15.86:0/2868112868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4041070 con 0x7fe8f0101b70 2024-04-16T05:35:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.184+0000 7fe8d7fff640 1 -- 172.21.15.86:0/2868112868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e40044c0 con 0x7fe8f0101b70 2024-04-16T05:35:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.184+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f01a01a0 con 0x7fe8f0101b70 2024-04-16T05:35:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.184+0000 7fe8d7fff640 1 -- 172.21.15.86:0/2868112868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40346a0 con 0x7fe8f0101b70 2024-04-16T05:35:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.184+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a0690 con 0x7fe8f0101b70 2024-04-16T05:35:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.185+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f01082f0 con 0x7fe8f0101b70 2024-04-16T05:35:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.186+0000 7fe8d7fff640 1 -- 172.21.15.86:0/2868112868 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe8e4044070 con 0x7fe8f0101b70 2024-04-16T05:35:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.186+0000 7fe8d7fff640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3c50 0x7fe8d80a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.186+0000 7fe8eeffd640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3c50 0x7fe8d80a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.186+0000 7fe8d7fff640 1 -- 172.21.15.86:0/2868112868 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe8e40f2520 con 0x7fe8f0101b70 2024-04-16T05:35:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.187+0000 7fe8eeffd640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3c50 0x7fe8d80a6110 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00058c0 tx=0x7fe8e0005c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.191+0000 7fe8d7fff640 1 -- 172.21.15.86:0/2868112868 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8e40beb70 con 0x7fe8f0101b70 2024-04-16T05:35:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.418+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fe8f0062c00 con 0x7fe8f0101b70 2024-04-16T05:35:09.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.668+0000 7fe8d7fff640 1 -- 172.21.15.86:0/2868112868 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fe8e40be470 con 0x7fe8f0101b70 2024-04-16T05:35:09.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.670+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3c50 msgr2=0x7fe8d80a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.670+0000 7fe8f567b640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3c50 0x7fe8d80a6110 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00058c0 tx=0x7fe8e0005c60 comp rx=0 tx=0).stop 2024-04-16T05:35:09.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.670+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f0101b70 msgr2=0x7fe8f019d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.670+0000 7fe8f567b640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f0101b70 0x7fe8f019d010 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4000c00 tx=0x7fe8e40040c0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.671+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 shutdown_connections 2024-04-16T05:35:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.671+0000 7fe8f567b640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7fe8d80a3c50 0x7fe8d80a6110 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.671+0000 7fe8f567b640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f01025f0 0x7fe8f019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.671+0000 7fe8f567b640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f0101b70 0x7fe8f019d010 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.671+0000 7fe8f567b640 1 --2- 172.21.15.86:0/2868112868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f01011b0 0x7fe8f0100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.671+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 >> 172.21.15.86:0/2868112868 conn(0x7fe8f00fbc60 msgr2=0x7fe8f01054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.671+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 shutdown_connections 2024-04-16T05:35:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.672+0000 7fe8f567b640 1 -- 172.21.15.86:0/2868112868 wait complete. 2024-04-16T05:35:09.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-16T05:35:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.798+0000 7f408b349640 1 -- 172.21.15.86:0/2018975165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 msgr2=0x7f4084102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.798+0000 7f408b349640 1 --2- 172.21.15.86:0/2018975165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 0x7f4084102560 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4074009900 tx=0x7f407402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.798+0000 7f408b349640 1 -- 172.21.15.86:0/2018975165 shutdown_connections 2024-04-16T05:35:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.798+0000 7f408b349640 1 --2- 172.21.15.86:0/2018975165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4084102aa0 0x7f40841093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.798+0000 7f408b349640 1 --2- 172.21.15.86:0/2018975165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 0x7f4084102560 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.798+0000 7f408b349640 1 --2- 172.21.15.86:0/2018975165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4084100f10 0x7f4084101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.798+0000 7f408b349640 1 -- 172.21.15.86:0/2018975165 >> 172.21.15.86:0/2018975165 conn(0x7f40840fc4d0 msgr2=0x7f40840fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 -- 172.21.15.86:0/2018975165 shutdown_connections 2024-04-16T05:35:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 -- 172.21.15.86:0/2018975165 wait complete. 2024-04-16T05:35:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 Processor -- start 2024-04-16T05:35:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 -- start start 2024-04-16T05:35:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4084100f10 0x7f40841971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 0x7f40841976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4084102aa0 0x7f408419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f408410cbe0 con 0x7f40841020e0 2024-04-16T05:35:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f408410ca60 con 0x7f4084100f10 2024-04-16T05:35:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f408b349640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f408410cd60 con 0x7f4084102aa0 2024-04-16T05:35:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f40888e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 0x7f40841976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f40888e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 0x7f40841976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53422/0 (socket says 172.21.15.86:53422) 2024-04-16T05:35:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.799+0000 7f40888e4640 1 -- 172.21.15.86:0/1574440855 learned_addr learned my addr 172.21.15.86:0/1574440855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f40898e6640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4084102aa0 0x7f408419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f40890e5640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4084100f10 0x7f40841971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f40888e4640 1 -- 172.21.15.86:0/1574440855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4084102aa0 msgr2=0x7f408419e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f40888e4640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4084102aa0 0x7f408419e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f40888e4640 1 -- 172.21.15.86:0/1574440855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4084100f10 msgr2=0x7f40841971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f40888e4640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4084100f10 0x7f40841971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f40888e4640 1 -- 172.21.15.86:0/1574440855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408419edc0 con 0x7f40841020e0 2024-04-16T05:35:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f40888e4640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 0x7f40841976e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4074009230 tx=0x7f4074005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f407a7fc640 1 -- 172.21.15.86:0/1574440855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074041070 con 0x7f40841020e0 2024-04-16T05:35:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f407a7fc640 1 -- 172.21.15.86:0/1574440855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40740044d0 con 0x7f40841020e0 2024-04-16T05:35:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408419f050 con 0x7f40841020e0 2024-04-16T05:35:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f407a7fc640 1 -- 172.21.15.86:0/1574440855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074034440 con 0x7f40841020e0 2024-04-16T05:35:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.800+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408419f590 con 0x7f40841020e0 2024-04-16T05:35:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.801+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4084108320 con 0x7f40841020e0 2024-04-16T05:35:09.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.802+0000 7f407a7fc640 1 -- 172.21.15.86:0/1574440855 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4074044070 con 0x7f40841020e0 2024-04-16T05:35:09.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.803+0000 7f407a7fc640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f40700a3d20 0x7f40700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:09.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.803+0000 7f40890e5640 1 -- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f40700a3d20 msgr2=0x7f40700a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3194681946 2024-04-16T05:35:09.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.803+0000 7f407a7fc640 1 -- 172.21.15.86:0/1574440855 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40740f2310 con 0x7f40841020e0 2024-04-16T05:35:09.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.803+0000 7f40890e5640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f40700a3d20 0x7f40700a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:35:09.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.807+0000 7f407a7fc640 1 -- 172.21.15.86:0/1574440855 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40740be310 con 0x7f40841020e0 2024-04-16T05:35:09.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:09 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2868112868' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T05:35:09.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:09 smithi086 ceph-mon[32028]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:35:09.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:09 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: ignoring --setuser ceph since I am not root 2024-04-16T05:35:09.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:09 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: ignoring --setgroup ceph since I am not root 2024-04-16T05:35:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:09 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2868112868' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T05:35:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:09 smithi196 ceph-mon[36373]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:35:09.985 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: ignoring --setuser ceph since I am not root 2024-04-16T05:35:09.985 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: ignoring --setgroup ceph since I am not root 2024-04-16T05:35:09.985 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:09 smithi196 ceph-mgr[38124]: -- 172.21.15.196:0/1250726017 <== mon.2 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b0ec245a0 con 0x558b0ebf9000 2024-04-16T05:35:09.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.990+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f408402d620 con 0x7f40841020e0 2024-04-16T05:35:09.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.994+0000 7f407a7fc640 1 -- 172.21.15.86:0/1574440855 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f40740c2610 con 0x7f40841020e0 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f40700a3d20 msgr2=0x7f40700a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f40700a3d20 0x7f40700a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 msgr2=0x7f40841976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 0x7f40841976e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4074009230 tx=0x7f4074005d50 comp rx=0 tx=0).stop 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 shutdown_connections 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4084102aa0 0x7f408419e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40841020e0 0x7f40841976e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f40700a3d20 0x7f40700a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 --2- 172.21.15.86:0/1574440855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4084100f10 0x7f40841971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 >> 172.21.15.86:0/1574440855 conn(0x7f40840fc4d0 msgr2=0x7f40841054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:09.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 shutdown_connections 2024-04-16T05:35:09.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:09.997+0000 7f408b349640 1 -- 172.21.15.86:0/1574440855 wait complete. 2024-04-16T05:35:10.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.115+0000 7feb7be0d640 1 -- 172.21.15.86:0/1546972514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb74102aa0 msgr2=0x7feb741093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.115+0000 7feb7be0d640 1 --2- 172.21.15.86:0/1546972514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb74102aa0 0x7feb741093c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009ca0 tx=0x7feb6c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 -- 172.21.15.86:0/1546972514 shutdown_connections 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 --2- 172.21.15.86:0/1546972514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb74102aa0 0x7feb741093c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 --2- 172.21.15.86:0/1546972514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb741020e0 0x7feb74102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 --2- 172.21.15.86:0/1546972514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb74100f10 0x7feb74101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 -- 172.21.15.86:0/1546972514 >> 172.21.15.86:0/1546972514 conn(0x7feb740fc4d0 msgr2=0x7feb740fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 -- 172.21.15.86:0/1546972514 shutdown_connections 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 -- 172.21.15.86:0/1546972514 wait complete. 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 Processor -- start 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 -- start start 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb74100f10 0x7feb741982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb741020e0 0x7feb741987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb74102aa0 0x7feb7419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb7410dcc0 con 0x7feb741020e0 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb7410db40 con 0x7feb74100f10 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.116+0000 7feb7be0d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb7410de40 con 0x7feb74102aa0 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb793a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb741020e0 0x7feb741987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb793a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb741020e0 0x7feb741987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53442/0 (socket says 172.21.15.86:53442) 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb793a8640 1 -- 172.21.15.86:0/2229166542 learned_addr learned my addr 172.21.15.86:0/2229166542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb79ba9640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb74100f10 0x7feb741982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb7a3aa640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb74102aa0 0x7feb7419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb793a8640 1 -- 172.21.15.86:0/2229166542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb74102aa0 msgr2=0x7feb7419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb793a8640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb74102aa0 0x7feb7419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb793a8640 1 -- 172.21.15.86:0/2229166542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb74100f10 msgr2=0x7feb741982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb793a8640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb74100f10 0x7feb741982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb793a8640 1 -- 172.21.15.86:0/2229166542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7419fec0 con 0x7feb741020e0 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb79ba9640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb74100f10 0x7feb741982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb793a8640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb741020e0 0x7feb741987e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d950 tx=0x7feb6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb66ffd640 1 -- 172.21.15.86:0/2229166542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68017070 con 0x7feb741020e0 2024-04-16T05:35:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb741a01b0 con 0x7feb741020e0 2024-04-16T05:35:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb66ffd640 1 -- 172.21.15.86:0/2229166542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb68009c30 con 0x7feb741020e0 2024-04-16T05:35:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb66ffd640 1 -- 172.21.15.86:0/2229166542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800f6b0 con 0x7feb741020e0 2024-04-16T05:35:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.117+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb741a0750 con 0x7feb741020e0 2024-04-16T05:35:10.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.118+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb741082a0 con 0x7feb741020e0 2024-04-16T05:35:10.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.120+0000 7feb66ffd640 1 -- 172.21.15.86:0/2229166542 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7feb680109d0 con 0x7feb741020e0 2024-04-16T05:35:10.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.121+0000 7feb66ffd640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7feb600a3ca0 0x7feb600a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:10.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.121+0000 7feb79ba9640 1 -- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7feb600a3ca0 msgr2=0x7feb600a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3194681946 2024-04-16T05:35:10.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.121+0000 7feb66ffd640 1 -- 172.21.15.86:0/2229166542 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7feb680d02d0 con 0x7feb741020e0 2024-04-16T05:35:10.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.121+0000 7feb79ba9640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7feb600a3ca0 0x7feb600a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:35:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.124+0000 7feb66ffd640 1 -- 172.21.15.86:0/2229166542 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb68009de0 con 0x7feb741020e0 2024-04-16T05:35:10.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:09 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2868112868' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T05:35:10.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:09 smithi194 ceph-mon[35865]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:35:10.156 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:09 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: ignoring --setuser ceph since I am not root 2024-04-16T05:35:10.156 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:09 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: ignoring --setgroup ceph since I am not root 2024-04-16T05:35:10.156 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:09 smithi194 ceph-mgr[37266]: -- 172.21.15.194:0/2446407094 <== mon.2 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b97ae45a0 con 0x556b97ab8c00 2024-04-16T05:35:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.310+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7feb74062c70 con 0x7feb741020e0 2024-04-16T05:35:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.314+0000 7feb66ffd640 1 -- 172.21.15.86:0/2229166542 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7feb6809c2d0 con 0x7feb741020e0 2024-04-16T05:35:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7feb600a3ca0 msgr2=0x7feb600a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:35:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7feb600a3ca0 0x7feb600a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb741020e0 msgr2=0x7feb741987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb741020e0 0x7feb741987e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d950 tx=0x7feb6800de10 comp rx=0 tx=0).stop 2024-04-16T05:35:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 shutdown_connections 2024-04-16T05:35:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb74102aa0 0x7feb7419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb741020e0 0x7feb741987e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7feb600a3ca0 0x7feb600a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 --2- 172.21.15.86:0/2229166542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb74100f10 0x7feb741982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.316+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 >> 172.21.15.86:0/2229166542 conn(0x7feb740fc4d0 msgr2=0x7feb74105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.317+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 shutdown_connections 2024-04-16T05:35:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:10.317+0000 7feb7be0d640 1 -- 172.21.15.86:0/2229166542 wait complete. 2024-04-16T05:35:10.326 INFO:teuthology.run_tasks:Running task kclient... 2024-04-16T05:35:10.339 INFO:tasks.kclient:Mounting kernel clients... 2024-04-16T05:35:10.339 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-16T05:35:10.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:10.339 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-16T05:35:10.355 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-16T05:35:10.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:10.356 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:35:10.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:10.425 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-16T05:35:10.485 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:10.229+0000 7f361336e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:35:10.485 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:10.322+0000 7f361336e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:35:10.523 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:10 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:10.263+0000 7ff13933f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:35:10.523 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:10 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:10.362+0000 7ff13933f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:35:10.531 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-16T05:35:10.531 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T05:35:10.531 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:10.531 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph fs ls 2024-04-16T05:35:10.655 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:10 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:10.230+0000 7fd556185200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:35:10.656 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:10 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:10.322+0000 7fd556185200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:35:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:35:10.911 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:10 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2868112868' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T05:35:10.911 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:10 smithi086 ceph-mon[32028]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-16T05:35:10.911 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:10 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1574440855' entity='client.admin' 2024-04-16T05:35:10.911 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:10 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2229166542' entity='client.admin' 2024-04-16T05:35:10.958 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:10 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2868112868' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T05:35:10.958 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:10 smithi194 ceph-mon[35865]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-16T05:35:10.958 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:10 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1574440855' entity='client.admin' 2024-04-16T05:35:10.959 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:10 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2229166542' entity='client.admin' 2024-04-16T05:35:10.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:10 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2868112868' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T05:35:10.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:10 smithi196 ceph-mon[36373]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-16T05:35:10.960 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:10 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1574440855' entity='client.admin' 2024-04-16T05:35:10.960 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:10 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2229166542' entity='client.admin' 2024-04-16T05:35:11.177 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:11 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:11.059+0000 7ff13933f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:35:11.177 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:11 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:11.155+0000 7ff13933f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:35:11.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:10.960+0000 7f361336e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:35:11.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:11.052+0000 7f361336e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:35:11.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:11.193+0000 7f361336e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:35:11.283 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:10 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:10.959+0000 7fd556185200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:35:11.283 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:11.051+0000 7fd556185200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:35:11.283 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:11.191+0000 7fd556185200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:35:11.283 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:11.284+0000 7fd556185200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:35:11.467 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:11 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:11.303+0000 7ff13933f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:35:11.468 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:11 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:11.399+0000 7ff13933f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:35:11.628 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:11.367+0000 7fd556185200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:35:11.636 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:11.288+0000 7f361336e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:35:11.636 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:11.371+0000 7f361336e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:35:11.636 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:11.637+0000 7f361336e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:35:11.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:11 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:11.485+0000 7ff13933f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:35:11.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:11 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:11.759+0000 7ff13933f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:35:11.905 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:11.629+0000 7fd556185200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:35:11.906 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:11.737+0000 7fd556185200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:35:11.906 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:11.822+0000 7fd556185200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:35:11.926 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:11.747+0000 7f361336e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:35:11.927 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:11.833+0000 7f361336e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:35:11.927 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:11.928+0000 7f361336e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:35:12.065 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:11 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:11.873+0000 7ff13933f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:35:12.065 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:11 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:11.961+0000 7ff13933f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:35:12.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:12 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:12.104+0000 7f361336e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:35:12.377 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:12 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:12.066+0000 7ff13933f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:35:12.377 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:12 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:12.248+0000 7ff13933f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:35:12.405 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:11.916+0000 7fd556185200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:35:12.405 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:12 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:12.091+0000 7fd556185200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:35:12.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.480+0000 7f00c613f700 1 -- 172.21.15.86:0/3021311620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 msgr2=0x7f00c010f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:12.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.480+0000 7f00c613f700 1 --2- 172.21.15.86:0/3021311620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 0x7f00c010f9a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f00b800b0a0 tx=0x7f00b800b3b0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.480+0000 7f00c613f700 1 -- 172.21.15.86:0/3021311620 shutdown_connections 2024-04-16T05:35:12.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.480+0000 7f00c613f700 1 --2- 172.21.15.86:0/3021311620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 0x7f00c010f9a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.480+0000 7f00c613f700 1 --2- 172.21.15.86:0/3021311620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00c0108b60 0x7f00c0108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.480+0000 7f00c613f700 1 --2- 172.21.15.86:0/3021311620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c006b090 0x7f00c006b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.480+0000 7f00c613f700 1 -- 172.21.15.86:0/3021311620 >> 172.21.15.86:0/3021311620 conn(0x7f00c0078960 msgr2=0x7f00c0078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.480+0000 7f00c613f700 1 -- 172.21.15.86:0/3021311620 shutdown_connections 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.480+0000 7f00c613f700 1 -- 172.21.15.86:0/3021311620 wait complete. 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.481+0000 7f00c613f700 1 Processor -- start 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.481+0000 7f00c613f700 1 -- start start 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.481+0000 7f00c613f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00c006b090 0x7f00c0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.481+0000 7f00c613f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 0x7f00c0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.481+0000 7f00c613f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c0108b60 0x7f00c019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.481+0000 7f00c613f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0112db0 con 0x7f00c006ba30 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.481+0000 7f00c613f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0112c30 con 0x7f00c006b090 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.481+0000 7f00c613f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0112f30 con 0x7f00c0108b60 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.481+0000 7f00beffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 0x7f00c0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00beffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 0x7f00c0199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49306/0 (socket says 172.21.15.86:49306) 2024-04-16T05:35:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00bf7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00c006b090 0x7f00c0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00beffd700 1 -- 172.21.15.86:0/3752981000 learned_addr learned my addr 172.21.15.86:0/3752981000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00bffff700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c0108b60 0x7f00c019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00beffd700 1 -- 172.21.15.86:0/3752981000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c0108b60 msgr2=0x7f00c019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00beffd700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c0108b60 0x7f00c019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00beffd700 1 -- 172.21.15.86:0/3752981000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00c006b090 msgr2=0x7f00c0199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00beffd700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00c006b090 0x7f00c0199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00beffd700 1 -- 172.21.15.86:0/3752981000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c019e230 con 0x7f00c006ba30 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00beffd700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 0x7f00c0199760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f00b400d950 tx=0x7f00b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00bcff9700 1 -- 172.21.15.86:0/3752981000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4016070 con 0x7f00c006ba30 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00bcff9700 1 -- 172.21.15.86:0/3752981000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b400fd70 con 0x7f00c006ba30 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00bcff9700 1 -- 172.21.15.86:0/3752981000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b400e610 con 0x7f00c006ba30 2024-04-16T05:35:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c019e520 con 0x7f00c006ba30 2024-04-16T05:35:12.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.482+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c01a5da0 con 0x7f00c006ba30 2024-04-16T05:35:12.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.483+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c00697e0 con 0x7f00c006ba30 2024-04-16T05:35:12.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.485+0000 7f00bcff9700 1 -- 172.21.15.86:0/3752981000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f00b4019070 con 0x7f00c006ba30 2024-04-16T05:35:12.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.486+0000 7f00bcff9700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f00a80a3c90 0x7f00a80a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:12.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.486+0000 7f00bf7fe700 1 -- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f00a80a3c90 msgr2=0x7f00a80a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3194681946 2024-04-16T05:35:12.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.486+0000 7f00bcff9700 1 -- 172.21.15.86:0/3752981000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f00b40d0230 con 0x7f00c006ba30 2024-04-16T05:35:12.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.486+0000 7f00bf7fe700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f00a80a3c90 0x7f00a80a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:35:12.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.489+0000 7f00bcff9700 1 -- 172.21.15.86:0/3752981000 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00b409c880 con 0x7f00c006ba30 2024-04-16T05:35:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.687+0000 7f00bf7fe700 1 -- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f00a80a3c90 msgr2=0x7f00a80a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3194681946 2024-04-16T05:35:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.687+0000 7f00bf7fe700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f00a80a3c90 0x7f00a80a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:35:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.782+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f00c002dfe0 con 0x7f00c006ba30 2024-04-16T05:35:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.783+0000 7f00bcff9700 1 -- 172.21.15.86:0/3752981000 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f00b409c180 con 0x7f00c006ba30 2024-04-16T05:35:12.781 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T05:35:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f00a80a3c90 msgr2=0x7f00a80a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:35:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f00a80a3c90 0x7f00a80a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 msgr2=0x7f00c0199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 0x7f00c0199760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f00b400d950 tx=0x7f00b400dc60 comp rx=0 tx=0).stop 2024-04-16T05:35:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 shutdown_connections 2024-04-16T05:35:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f00a80a3c90 0x7f00a80a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c0108b60 0x7f00c019daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00c006b090 0x7f00c0199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 --2- 172.21.15.86:0/3752981000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c006ba30 0x7f00c0199760 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.785+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 >> 172.21.15.86:0/3752981000 conn(0x7f00c0078960 msgr2=0x7f00c00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.786+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 shutdown_connections 2024-04-16T05:35:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:12.786+0000 7f00c613f700 1 -- 172.21.15.86:0/3752981000 wait complete. 2024-04-16T05:35:12.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:12 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3752981000' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:35:13.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:12 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3752981000' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:35:13.156 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:13 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:13.049+0000 7fd556185200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:35:13.192 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:12 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3752981000' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:35:13.192 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:13.063+0000 7f361336e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:35:13.467 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:13 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:13.257+0000 7ff13933f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:35:13.468 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:13 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:13.390+0000 7ff13933f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:35:13.485 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:13.193+0000 7f361336e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:35:13.508 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:13 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:13.178+0000 7fd556185200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:35:13.508 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:13 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:13.509+0000 7fd556185200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:35:13.526 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-16T05:35:13.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T05:35:13.526 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-16T05:35:13.526 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-16T05:35:13.526 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:13.526 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-16T05:35:13.526 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T05:35:13.526 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T05:35:13.527 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-16T05:35:13.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:13.527 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-16T05:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T05:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T05:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-16T05:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41925sec preferred_lft 41925sec 2024-04-16T05:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-16T05:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:35:13.545 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T05:35:13.545 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:35:13.545 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:35:13.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-16T05:35:13.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-16T05:35:13.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-16T05:35:13.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T05:35:13.545 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:35:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:35:13.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:35:13.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:13.706 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T05:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-16T05:35:13.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:13.737 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-16T05:35:13.791 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:13 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:13.597+0000 7fd556185200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:35:13.791 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:13 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:13.792+0000 7fd556185200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-16T05:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T05:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-16T05:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-16T05:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T05:35:13.795 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:35:13.795 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:35:13.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-16T05:35:13.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-16T05:35:13.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-16T05:35:13.795 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:35:13.814 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:13.527+0000 7f361336e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:35:13.814 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:13.617+0000 7f361336e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:35:13.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:35:13.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:35:13.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:13.942 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-16T05:35:13.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:13.958 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-16T05:35:14.014 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:35:14.014 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:35:14.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T05:35:14.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-16T05:35:14.014 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:35:14.076 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:13 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:13.750+0000 7ff13933f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:35:14.076 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:13 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:13.849+0000 7ff13933f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:35:14.076 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:14 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:14.077+0000 7ff13933f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:35:14.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:35:14.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:35:14.141 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-16T05:35:14.141 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:35:14.141 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:35:14.141 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-16T05:35:14.141 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T05:35:14.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-16T05:35:14.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-16T05:35:14.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-16T05:35:14.142 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:35:14.155 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:13 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:13.975+0000 7fd556185200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:35:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:35:14.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:13.815+0000 7f361336e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:35:14.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:14.001+0000 7f361336e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:35:14.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:35:14.319 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:35:14.319 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-16T05:35:14.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-16T05:35:14.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T05:35:14.319 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-16T05:35:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:35:14.380 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:14 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:14.284+0000 7ff13933f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:35:14.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:35:14.407 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T05:35:14.407 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-16T05:35:14.428 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T05:35:14.428 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:14.484 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-16T05:35:14.485 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-16T05:35:14.485 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/304f66f3-2ffd-4c6a-8666-2669b193ec6a 2024-04-16T05:35:14.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:14.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/304f66f3-2ffd-4c6a-8666-2669b193ec6a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-16T05:35:14.515 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:14 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:14.256+0000 7fd556185200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:35:14.515 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:14 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:14.516+0000 7fd556185200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:35:14.548 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:14.287+0000 7f361336e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:35:14.548 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:14.549+0000 7f361336e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-16T05:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-16T05:35:14.635 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:14 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:14.595+0000 7ff13933f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-16T05:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-16T05:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.194:3300,172.21.15.196:3300" 2024-04-16T05:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 0@6a4a23f8-fbb1-11ee-bc8f-c7b262605968.cephfs=/volumes/qa/sv_1/304f66f3-2ffd-4c6a-8666-2669b193ec6a 2024-04-16T05:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.86:3300/172.21.15.194:3300/172.21.15.196:3300" will pass to kernel 2024-04-16T05:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-16T05:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: You just mounted a file system that supports labels which does not 2024-04-16T05:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-16T05:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: applications will generate AVC messages and not be allowed access to 2024-04-16T05:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-16T05:35:14.729 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-16T05:35:14.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:14.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:14.771 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T05:35:14.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T05:35:14.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:35:14.870 INFO:teuthology.orchestra.run.smithi086.stdout:client25048 2024-04-16T05:35:14.871 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:35:14.871 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:35:14.887 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T05:35:14.905 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:14 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:14.752+0000 7fd556185200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:35:14.967 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:14 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:14.873+0000 7ff13933f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:35:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T05:35:14.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:35:14.985 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:14.790+0000 7f361336e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:35:14.985 INFO:teuthology.orchestra.run.smithi086.stdout:client25048 2024-04-16T05:35:14.986 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T05:35:15.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T05:35:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:35:15.083 INFO:teuthology.orchestra.run.smithi086.stdout:client25048 2024-04-16T05:35:15.083 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968.client25048/status') 2024-04-16T05:35:15.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:35:15.146 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.25048 (3)172.21.15.86:0/1375645136 2024-04-16T05:35:15.146 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-04-16T05:35:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-04-16T05:35:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-04-16T05:35:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:71 bytes copied, 2.2179e-05 s, 3.2 MB/s 2024-04-16T05:35:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:35:15.154 INFO:teuthology.run_tasks:Running task exec... 2024-04-16T05:35:15.167 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T05:35:15.167 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:35:15.167 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-16T05:35:15.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 -- 172.21.15.86:0/269586053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218105410 msgr2=0x7f6218107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:15.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 --2- 172.21.15.86:0/269586053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218105410 0x7f6218107800 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f620c009900 tx=0x7f620c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 -- 172.21.15.86:0/269586053 shutdown_connections 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 --2- 172.21.15.86:0/269586053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6218107d40 0x7f621810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 --2- 172.21.15.86:0/269586053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218105410 0x7f6218107800 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 --2- 172.21.15.86:0/269586053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218102ae0 0x7f6218104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 -- 172.21.15.86:0/269586053 >> 172.21.15.86:0/269586053 conn(0x7f62180fc4d0 msgr2=0x7f62180fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 -- 172.21.15.86:0/269586053 shutdown_connections 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 -- 172.21.15.86:0/269586053 wait complete. 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 Processor -- start 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.276+0000 7f621f847640 1 -- start start 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621f847640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218102ae0 0x7f6218102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621f847640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6218105410 0x7f6218100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621f847640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218107d40 0x7f6218100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621f847640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f621810dcf0 con 0x7f6218102ae0 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621f847640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f621810db70 con 0x7f6218105410 2024-04-16T05:35:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621f847640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f621810de70 con 0x7f6218107d40 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621d5e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218102ae0 0x7f6218102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621d5e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218102ae0 0x7f6218102450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49326/0 (socket says 172.21.15.86:49326) 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621d5e3640 1 -- 172.21.15.86:0/1365594311 learned_addr learned my addr 172.21.15.86:0/1365594311 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621cde2640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6218105410 0x7f6218100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621dde4640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218107d40 0x7f6218100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621d5e3640 1 -- 172.21.15.86:0/1365594311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218107d40 msgr2=0x7f6218100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621d5e3640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218107d40 0x7f6218100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621d5e3640 1 -- 172.21.15.86:0/1365594311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6218105410 msgr2=0x7f6218100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621d5e3640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6218105410 0x7f6218100a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621d5e3640 1 -- 172.21.15.86:0/1365594311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6218101510 con 0x7f6218102ae0 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621dde4640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218107d40 0x7f6218100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621d5e3640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218102ae0 0x7f6218102450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f621000db80 tx=0x7f621000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f62167fc640 1 -- 172.21.15.86:0/1365594311 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210017070 con 0x7f6218102ae0 2024-04-16T05:35:15.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f62167fc640 1 -- 172.21.15.86:0/1365594311 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6210003b70 con 0x7f6218102ae0 2024-04-16T05:35:15.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6218101800 con 0x7f6218102ae0 2024-04-16T05:35:15.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.277+0000 7f62167fc640 1 -- 172.21.15.86:0/1365594311 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62100106c0 con 0x7f6218102ae0 2024-04-16T05:35:15.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.278+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62181a6da0 con 0x7f6218102ae0 2024-04-16T05:35:15.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.278+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62180fdfc0 con 0x7f6218102ae0 2024-04-16T05:35:15.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.280+0000 7f62167fc640 1 -- 172.21.15.86:0/1365594311 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f6210004020 con 0x7f6218102ae0 2024-04-16T05:35:15.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.280+0000 7f62167fc640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f62000a3ca0 0x7f62000a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:15.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.280+0000 7f621cde2640 1 -- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f62000a3ca0 msgr2=0x7f62000a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3194681946 2024-04-16T05:35:15.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.280+0000 7f62167fc640 1 -- 172.21.15.86:0/1365594311 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f62100d0660 con 0x7f6218102ae0 2024-04-16T05:35:15.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.280+0000 7f621cde2640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f62000a3ca0 0x7f62000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:35:15.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.285+0000 7f62167fc640 1 -- 172.21.15.86:0/1365594311 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6210020070 con 0x7f6218102ae0 2024-04-16T05:35:15.467 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:15 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:15.130+0000 7ff13933f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:35:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.481+0000 7f621cde2640 1 -- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f62000a3ca0 msgr2=0x7f62000a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3194681946 2024-04-16T05:35:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.481+0000 7f621cde2640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f62000a3ca0 0x7f62000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:35:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.511+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f621802d620 con 0x7f6218102ae0 2024-04-16T05:35:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.563+0000 7f62167fc640 1 -- 172.21.15.86:0/1365594311 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f621009ccb0 con 0x7f6218102ae0 2024-04-16T05:35:15.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f62000a3ca0 msgr2=0x7f62000a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:35:15.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f62000a3ca0 0x7f62000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218102ae0 msgr2=0x7f6218102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:15.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218102ae0 0x7f6218102450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f621000db80 tx=0x7f621000b630 comp rx=0 tx=0).stop 2024-04-16T05:35:15.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 shutdown_connections 2024-04-16T05:35:15.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218107d40 0x7f6218100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f62000a3ca0 0x7f62000a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6218105410 0x7f6218100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 --2- 172.21.15.86:0/1365594311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218102ae0 0x7f6218102450 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 >> 172.21.15.86:0/1365594311 conn(0x7f62180fc4d0 msgr2=0x7f6218103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:15.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 shutdown_connections 2024-04-16T05:35:15.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.565+0000 7f621f847640 1 -- 172.21.15.86:0/1365594311 wait complete. 2024-04-16T05:35:15.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-16T05:35:15.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 -- 172.21.15.86:0/2571888373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8168105470 msgr2=0x7f8168107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:15.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 --2- 172.21.15.86:0/2571888373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8168105470 0x7f8168107860 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f815c009900 tx=0x7f815c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:15.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 -- 172.21.15.86:0/2571888373 shutdown_connections 2024-04-16T05:35:15.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 --2- 172.21.15.86:0/2571888373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8168107da0 0x7f816810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 --2- 172.21.15.86:0/2571888373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8168105470 0x7f8168107860 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 --2- 172.21.15.86:0/2571888373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8168102b40 0x7f8168104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 -- 172.21.15.86:0/2571888373 >> 172.21.15.86:0/2571888373 conn(0x7f81680fc4d0 msgr2=0x7f81680fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:15.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 -- 172.21.15.86:0/2571888373 shutdown_connections 2024-04-16T05:35:15.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 -- 172.21.15.86:0/2571888373 wait complete. 2024-04-16T05:35:15.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.686+0000 7f816d088640 1 Processor -- start 2024-04-16T05:35:15.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816d088640 1 -- start start 2024-04-16T05:35:15.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816d088640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8168102b40 0x7f8168198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:15.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816d088640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8168105470 0x7f8168198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816d088640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8168107da0 0x7f816819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816d088640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f816810dcf0 con 0x7f8168105470 2024-04-16T05:35:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816d088640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f816810db70 con 0x7f8168102b40 2024-04-16T05:35:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816d088640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f816810de70 con 0x7f8168107da0 2024-04-16T05:35:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8168105470 0x7f8168198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:15.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8168105470 0x7f8168198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49350/0 (socket says 172.21.15.86:49350) 2024-04-16T05:35:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f8166d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8168102b40 0x7f8168198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816659c640 1 -- 172.21.15.86:0/2473579657 learned_addr learned my addr 172.21.15.86:0/2473579657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816759e640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8168107da0 0x7f816819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816659c640 1 -- 172.21.15.86:0/2473579657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8168107da0 msgr2=0x7f816819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816659c640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8168107da0 0x7f816819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816659c640 1 -- 172.21.15.86:0/2473579657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8168102b40 msgr2=0x7f8168198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:15.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816659c640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8168102b40 0x7f8168198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:15.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f816659c640 1 -- 172.21.15.86:0/2473579657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816819ff30 con 0x7f8168105470 2024-04-16T05:35:15.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.687+0000 7f8166d9d640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8168102b40 0x7f8168198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:15.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.688+0000 7f816659c640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8168105470 0x7f8168198850 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f815c000c00 tx=0x7f815c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:15.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.688+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c041070 con 0x7f8168105470 2024-04-16T05:35:15.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.688+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81681a01c0 con 0x7f8168105470 2024-04-16T05:35:15.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.688+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f815c0044d0 con 0x7f8168105470 2024-04-16T05:35:15.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.688+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c034440 con 0x7f8168105470 2024-04-16T05:35:15.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.688+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81681a0700 con 0x7f8168105470 2024-04-16T05:35:15.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.689+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81680fdfc0 con 0x7f8168105470 2024-04-16T05:35:15.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.690+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f815c044070 con 0x7f8168105470 2024-04-16T05:35:15.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.690+0000 7f814ffff640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f81500a3ca0 0x7f81500a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:15.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.690+0000 7f8166d9d640 1 -- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f81500a3ca0 msgr2=0x7f81500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3194681946 2024-04-16T05:35:15.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.691+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f815c0f22f0 con 0x7f8168105470 2024-04-16T05:35:15.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.691+0000 7f8166d9d640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f81500a3ca0 0x7f81500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:35:15.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.695+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f815c0be2f0 con 0x7f8168105470 2024-04-16T05:35:15.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.892+0000 7f8166d9d640 1 -- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f81500a3ca0 msgr2=0x7f81500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3194681946 2024-04-16T05:35:15.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.892+0000 7f8166d9d640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f81500a3ca0 0x7f81500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:35:15.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:15 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1365594311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-16T05:35:15.906 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:15 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:15.553+0000 7fd556185200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:35:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:15.938+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 --> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f81680624e0 con 0x7f81500a3ca0 2024-04-16T05:35:15.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:15 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1365594311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-16T05:35:15.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:15 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1365594311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-16T05:35:15.985 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:15 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:15.587+0000 7f361336e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:35:16.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:16.293+0000 7f8166d9d640 1 -- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f81500a3ca0 msgr2=0x7f81500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3194681946 2024-04-16T05:35:16.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:16.293+0000 7f8166d9d640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f81500a3ca0 0x7f81500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T05:35:16.405 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:16.124+0000 7fd556185200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:35:16.405 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:16.219+0000 7fd556185200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:35:16.465 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:16.155+0000 7f361336e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:35:16.465 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:16.251+0000 7f361336e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:35:16.466 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38120]: 2024-04-16T05:35:16.467+0000 7f361336e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:35:16.467 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:16.017+0000 7ff13933f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:35:16.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:16.581+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f815c0ba960 con 0x7f8168105470 2024-04-16T05:35:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1365594311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-16T05:35:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-16T05:35:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: Standby manager daemon y restarted 2024-04-16T05:35:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: Standby manager daemon y started 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: Standby manager daemon z restarted 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: Standby manager daemon z started 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:35:16.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:16 smithi196 ceph-mon[36373]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37262]: 2024-04-16T05:35:16.434+0000 7fd556185200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1365594311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: Standby manager daemon y restarted 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: Standby manager daemon y started 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:35:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: Standby manager daemon z restarted 2024-04-16T05:35:16.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: Standby manager daemon z started 2024-04-16T05:35:16.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:35:16.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:35:16.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:35:16.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:16 smithi194 ceph-mon[35865]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:35:16.926 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:16.598+0000 7ff13933f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:35:16.926 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:16.699+0000 7ff13933f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:35:16.927 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:16.927+0000 7ff13933f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:35:16.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1365594311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-16T05:35:16.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-16T05:35:16.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: Standby manager daemon y restarted 2024-04-16T05:35:16.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: Standby manager daemon y started 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.194:0/1940488619' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: Standby manager daemon z restarted 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: Standby manager daemon z started 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:35:16.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:16 smithi086 ceph-mon[32028]: from='mgr.? 172.21.15.196:0/3214659292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:35:16.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:16.939+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f815c0036f0 con 0x7f8168105470 2024-04-16T05:35:16.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:16.940+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f81500a3ca0 msgr2=0x7f81500a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:35:16.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:16.940+0000 7f814ffff640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/3194681946,v1:172.21.15.86:6801/3194681946] conn(0x7f81500a3ca0 0x7f81500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.172+0000 7ff09bd52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.172+0000 7ff09bd52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.172+0000 7ff09bd52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.172+0000 7ff09bd52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.172+0000 7ff09bd52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.173+0000 7ff09854b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.173+0000 7ff09854b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.173+0000 7ff09854b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.219 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.173+0000 7ff09854b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.219 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.173+0000 7ff09854b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:17.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-16T05:35:17.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: Active manager daemon x restarted 2024-04-16T05:35:17.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: Activating manager daemon x 2024-04-16T05:35:17.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T05:35:17.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: mgrmap e21: x(active, starting, since 0.0121275s), standbys: z, y 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:35:17.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:35:17.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:35:17.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:35:17.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:35:17.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:35:17.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: Manager daemon x is now available 2024-04-16T05:35:17.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:35:17.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:35:17.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:17 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:35:17.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.947+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mon.0 v2:172.21.15.86:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f815c0998e0 con 0x7f8168105470 2024-04-16T05:35:17.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.948+0000 7f814ffff640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8150035890 0x7f8150035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:17.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.948+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f81680624e0 con 0x7f8150035890 2024-04-16T05:35:17.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.949+0000 7f8166d9d640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8150035890 0x7f8150035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:17.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.952+0000 7f8166d9d640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8150035890 0x7f8150035c70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8158005d00 tx=0x7f815800b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:17.959 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: Active manager daemon x restarted 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: Activating manager daemon x 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: mgrmap e21: x(active, starting, since 0.0121275s), standbys: z, y 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:35:17.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:35:17.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: Manager daemon x is now available 2024-04-16T05:35:17.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:35:17.963 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:35:17.963 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:35:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.974+0000 7f814ffff640 1 -- 172.21.15.86:0/2473579657 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f81680624e0 con 0x7f8150035890 2024-04-16T05:35:17.973 INFO:teuthology.orchestra.run.smithi086.stdout:{} 2024-04-16T05:35:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.977+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8150035890 msgr2=0x7f8150035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.977+0000 7f816d088640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8150035890 0x7f8150035c70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8158005d00 tx=0x7f815800b410 comp rx=0 tx=0).stop 2024-04-16T05:35:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.977+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8168105470 msgr2=0x7f8168198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.977+0000 7f816d088640 1 --2- 172.21.15.86:0/2473579657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8168105470 0x7f8168198850 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f815c000c00 tx=0x7f815c005d50 comp rx=0 tx=0).stop 2024-04-16T05:35:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.977+0000 7f816759e640 1 -- 172.21.15.86:0/2473579657 reap_dead start 2024-04-16T05:35:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.977+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 shutdown_connections 2024-04-16T05:35:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.977+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 >> 172.21.15.86:0/2473579657 conn(0x7f81680fc4d0 msgr2=0x7f8168103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.977+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 shutdown_connections 2024-04-16T05:35:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:17.977+0000 7f816d088640 1 -- 172.21.15.86:0/2473579657 wait complete. 2024-04-16T05:35:17.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-16T05:35:17.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: Active manager daemon x restarted 2024-04-16T05:35:17.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: Activating manager daemon x 2024-04-16T05:35:17.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T05:35:17.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: mgrmap e21: x(active, starting, since 0.0121275s), standbys: z, y 2024-04-16T05:35:17.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:35:17.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:35:17.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:35:17.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:35:17.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:35:17.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:35:17.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:35:17.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:35:17.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: Manager daemon x is now available 2024-04-16T05:35:17.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:35:17.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:35:17.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:17 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:35:17.989 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-16T05:35:18.000 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-16T05:35:18.034 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:35:18.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.142+0000 7f4829093640 1 -- 172.21.15.86:0/3940522351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824107d40 msgr2=0x7f482410a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.142+0000 7f4829093640 1 --2- 172.21.15.86:0/3940522351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824107d40 0x7f482410a190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f481c009ca0 tx=0x7f481c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.142+0000 7f4829093640 1 -- 172.21.15.86:0/3940522351 shutdown_connections 2024-04-16T05:35:18.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.142+0000 7f4829093640 1 --2- 172.21.15.86:0/3940522351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824107d40 0x7f482410a190 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.142+0000 7f4829093640 1 --2- 172.21.15.86:0/3940522351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4824105410 0x7f4824107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.142+0000 7f4829093640 1 --2- 172.21.15.86:0/3940522351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824102ae0 0x7f4824104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.142+0000 7f4829093640 1 -- 172.21.15.86:0/3940522351 >> 172.21.15.86:0/3940522351 conn(0x7f48240fc4f0 msgr2=0x7f48240fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.142+0000 7f4829093640 1 -- 172.21.15.86:0/3940522351 shutdown_connections 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.142+0000 7f4829093640 1 -- 172.21.15.86:0/3940522351 wait complete. 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f4829093640 1 Processor -- start 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f4829093640 1 -- start start 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f4829093640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4824102ae0 0x7f4824198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f4829093640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824105410 0x7f4824198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f4829093640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824107d40 0x7f482419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f4829093640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f482410dcf0 con 0x7f4824105410 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f4829093640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f482410db70 con 0x7f4824102ae0 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f4829093640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f482410de70 con 0x7f4824107d40 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f482259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824105410 0x7f4824198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f482259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824105410 0x7f4824198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49514/0 (socket says 172.21.15.86:49514) 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f482259c640 1 -- 172.21.15.86:0/3664837162 learned_addr learned my addr 172.21.15.86:0/3664837162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f4822d9d640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4824102ae0 0x7f4824198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.143+0000 7f482359e640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824107d40 0x7f482419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f482259c640 1 -- 172.21.15.86:0/3664837162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824107d40 msgr2=0x7f482419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f482259c640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824107d40 0x7f482419f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f482259c640 1 -- 172.21.15.86:0/3664837162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4824102ae0 msgr2=0x7f4824198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f482259c640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4824102ae0 0x7f4824198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f482259c640 1 -- 172.21.15.86:0/3664837162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482419ff40 con 0x7f4824105410 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f482259c640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824105410 0x7f4824198860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f481800e9c0 tx=0x7f481800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f480bfff640 1 -- 172.21.15.86:0/3664837162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481800cd80 con 0x7f4824105410 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48241a0230 con 0x7f4824105410 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f480bfff640 1 -- 172.21.15.86:0/3664837162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4818005500 con 0x7f4824105410 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f480bfff640 1 -- 172.21.15.86:0/3664837162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818010690 con 0x7f4824105410 2024-04-16T05:35:18.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.144+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48241a07d0 con 0x7f4824105410 2024-04-16T05:35:18.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.145+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48240fdfe0 con 0x7f4824105410 2024-04-16T05:35:18.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.146+0000 7f480bfff640 1 -- 172.21.15.86:0/3664837162 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f48180046c0 con 0x7f4824105410 2024-04-16T05:35:18.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.147+0000 7f480bfff640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f480c0a39b0 0x7f480c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.147+0000 7f4822d9d640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f480c0a39b0 0x7f480c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.147+0000 7f480bfff640 1 -- 172.21.15.86:0/3664837162 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f48180d07f0 con 0x7f4824105410 2024-04-16T05:35:18.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.147+0000 7f4822d9d640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f480c0a39b0 0x7f480c0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4814005fd0 tx=0x7f481400b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:18.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.151+0000 7f480bfff640 1 -- 172.21.15.86:0/3664837162 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f481809d030 con 0x7f4824105410 2024-04-16T05:35:18.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.960+0000 7ff095d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:18.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.960+0000 7ff095d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:18.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.960+0000 7ff095d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:18.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.960+0000 7ff095d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:18.218 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:35:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:35:17.960+0000 7ff095d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:35:18.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.341+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4824062c00 con 0x7f4824105410 2024-04-16T05:35:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.343+0000 7f480bfff640 1 -- 172.21.15.86:0/3664837162 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25077 (secure 0 0 0) 0x7f481809c930 con 0x7f4824105410 2024-04-16T05:35:18.342 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:18.342 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":95,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:35:16.929832+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2330974702":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/3900523320":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2641972995":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/501397249":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f480c0a39b0 msgr2=0x7f480c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f480c0a39b0 0x7f480c0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4814005fd0 tx=0x7f481400b390 comp rx=0 tx=0).stop 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824105410 msgr2=0x7f4824198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824105410 0x7f4824198860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f481800e9c0 tx=0x7f481800ee80 comp rx=0 tx=0).stop 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 shutdown_connections 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824107d40 0x7f482419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4824105410 0x7f4824198860 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f480c0a39b0 0x7f480c0a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 --2- 172.21.15.86:0/3664837162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4824102ae0 0x7f4824198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 >> 172.21.15.86:0/3664837162 conn(0x7f48240fc4f0 msgr2=0x7f4824103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 shutdown_connections 2024-04-16T05:35:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.345+0000 7f4829093640 1 -- 172.21.15.86:0/3664837162 wait complete. 2024-04-16T05:35:18.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:35:18.356 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:35:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.467+0000 7f7b4ae88640 1 -- 172.21.15.86:0/546733196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 msgr2=0x7f7b44101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.467+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/546733196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 0x7f7b44101360 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009900 tx=0x7f7b4002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.467+0000 7f7b4ae88640 1 -- 172.21.15.86:0/546733196 shutdown_connections 2024-04-16T05:35:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.467+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/546733196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b44102a50 0x7f7b44109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.467+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/546733196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b44102090 0x7f7b44102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.467+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/546733196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 0x7f7b44101360 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.467+0000 7f7b4ae88640 1 -- 172.21.15.86:0/546733196 >> 172.21.15.86:0/546733196 conn(0x7f7b440fc4b0 msgr2=0x7f7b440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.467+0000 7f7b4ae88640 1 -- 172.21.15.86:0/546733196 shutdown_connections 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.467+0000 7f7b4ae88640 1 -- 172.21.15.86:0/546733196 wait complete. 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b4ae88640 1 Processor -- start 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b4ae88640 1 -- start start 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b4ae88640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 0x7f7b44069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b4ae88640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b44102090 0x7f7b4419d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b4ae88640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b44102a50 0x7f7b4419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b4ae88640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4410cbf0 con 0x7f7b44100f40 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b4ae88640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4410ca70 con 0x7f7b44102a50 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b4ae88640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4410cd70 con 0x7f7b44102090 2024-04-16T05:35:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b48c24640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 0x7f7b44069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b48c24640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 0x7f7b44069d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49530/0 (socket says 172.21.15.86:49530) 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b48c24640 1 -- 172.21.15.86:0/2592205202 learned_addr learned my addr 172.21.15.86:0/2592205202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b49425640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b44102a50 0x7f7b4419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b3bfff640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b44102090 0x7f7b4419d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b48c24640 1 -- 172.21.15.86:0/2592205202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b44102090 msgr2=0x7f7b4419d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b48c24640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b44102090 0x7f7b4419d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b48c24640 1 -- 172.21.15.86:0/2592205202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b44102a50 msgr2=0x7f7b4419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b48c24640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b44102a50 0x7f7b4419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b48c24640 1 -- 172.21.15.86:0/2592205202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b441a0040 con 0x7f7b44100f40 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.468+0000 7f7b3bfff640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b44102090 0x7f7b4419d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.469+0000 7f7b48c24640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 0x7f7b44069d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009230 tx=0x7f7b40005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.469+0000 7f7b39ffb640 1 -- 172.21.15.86:0/2592205202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40041070 con 0x7f7b44100f40 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.469+0000 7f7b39ffb640 1 -- 172.21.15.86:0/2592205202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b400054e0 con 0x7f7b44100f40 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.469+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b441a0270 con 0x7f7b44100f40 2024-04-16T05:35:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.469+0000 7f7b39ffb640 1 -- 172.21.15.86:0/2592205202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40034440 con 0x7f7b44100f40 2024-04-16T05:35:18.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.469+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b441a0760 con 0x7f7b44100f40 2024-04-16T05:35:18.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.470+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b44108320 con 0x7f7b44100f40 2024-04-16T05:35:18.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.471+0000 7f7b39ffb640 1 -- 172.21.15.86:0/2592205202 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f7b40044070 con 0x7f7b44100f40 2024-04-16T05:35:18.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.471+0000 7f7b39ffb640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7b2c0a39b0 0x7f7b2c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.472+0000 7f7b3bfff640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7b2c0a39b0 0x7f7b2c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.472+0000 7f7b39ffb640 1 -- 172.21.15.86:0/2592205202 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7b400f21b0 con 0x7f7b44100f40 2024-04-16T05:35:18.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.472+0000 7f7b3bfff640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7b2c0a39b0 0x7f7b2c0a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b44069a70 tx=0x7f7b34005e70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:18.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.476+0000 7f7b39ffb640 1 -- 172.21.15.86:0/2592205202 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b400be9f0 con 0x7f7b44100f40 2024-04-16T05:35:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.659+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b4402d840 con 0x7f7b44100f40 2024-04-16T05:35:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.660+0000 7f7b39ffb640 1 -- 172.21.15.86:0/2592205202 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f7b400be2f0 con 0x7f7b44100f40 2024-04-16T05:35:18.659 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:35:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7b2c0a39b0 msgr2=0x7f7b2c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7b2c0a39b0 0x7f7b2c0a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b44069a70 tx=0x7f7b34005e70 comp rx=0 tx=0).stop 2024-04-16T05:35:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 msgr2=0x7f7b44069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 0x7f7b44069d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009230 tx=0x7f7b40005dd0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 shutdown_connections 2024-04-16T05:35:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7b2c0a39b0 0x7f7b2c0a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b44102a50 0x7f7b4419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b44102090 0x7f7b4419d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 --2- 172.21.15.86:0/2592205202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b44100f40 0x7f7b44069d00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 >> 172.21.15.86:0/2592205202 conn(0x7f7b440fc4b0 msgr2=0x7f7b441054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 shutdown_connections 2024-04-16T05:35:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.662+0000 7f7b4ae88640 1 -- 172.21.15.86:0/2592205202 wait complete. 2024-04-16T05:35:18.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:35:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:18 smithi196 ceph-mon[36373]: [16/Apr/2024:05:35:17] ENGINE Bus STARTING 2024-04-16T05:35:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:18 smithi196 ceph-mon[36373]: [16/Apr/2024:05:35:17] ENGINE Serving on http://172.21.15.86:8765 2024-04-16T05:35:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:18 smithi196 ceph-mon[36373]: [16/Apr/2024:05:35:17] ENGINE Serving on https://172.21.15.86:7150 2024-04-16T05:35:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:18 smithi196 ceph-mon[36373]: [16/Apr/2024:05:35:17] ENGINE Bus STARTED 2024-04-16T05:35:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:18 smithi196 ceph-mon[36373]: mgrmap e22: x(active, since 1.01831s), standbys: z, y 2024-04-16T05:35:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:18 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3664837162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.791+0000 7f1bad2eb640 1 -- 172.21.15.86:0/4175739610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba8100f10 msgr2=0x7f1ba8101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.791+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/4175739610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba8100f10 0x7f1ba8101350 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009900 tx=0x7f1b9802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.791+0000 7f1bad2eb640 1 -- 172.21.15.86:0/4175739610 shutdown_connections 2024-04-16T05:35:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.791+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/4175739610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ba8102aa0 0x7f1ba81093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.791+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/4175739610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba81020e0 0x7f1ba8102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.791+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/4175739610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba8100f10 0x7f1ba8101350 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.791+0000 7f1bad2eb640 1 -- 172.21.15.86:0/4175739610 >> 172.21.15.86:0/4175739610 conn(0x7f1ba80fc4f0 msgr2=0x7f1ba80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.791+0000 7f1bad2eb640 1 -- 172.21.15.86:0/4175739610 shutdown_connections 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.791+0000 7f1bad2eb640 1 -- 172.21.15.86:0/4175739610 wait complete. 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.792+0000 7f1bad2eb640 1 Processor -- start 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.792+0000 7f1bad2eb640 1 -- start start 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.792+0000 7f1bad2eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ba8100f10 0x7f1ba81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.792+0000 7f1bad2eb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba81020e0 0x7f1ba8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.792+0000 7f1bad2eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba8102aa0 0x7f1ba819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.792+0000 7f1bad2eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba810cbe0 con 0x7f1ba8102aa0 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.792+0000 7f1bad2eb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba810ca60 con 0x7f1ba81020e0 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.792+0000 7f1bad2eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba810cd60 con 0x7f1ba8100f10 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba8102aa0 0x7f1ba819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ba8100f10 0x7f1ba81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ba8100f10 0x7f1ba81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:35402/0 (socket says 172.21.15.86:35402) 2024-04-16T05:35:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba6ffd640 1 -- 172.21.15.86:0/225386682 learned_addr learned my addr 172.21.15.86:0/225386682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba77fe640 1 -- 172.21.15.86:0/225386682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ba8100f10 msgr2=0x7f1ba81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba77fe640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ba8100f10 0x7f1ba81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba77fe640 1 -- 172.21.15.86:0/225386682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba81020e0 msgr2=0x7f1ba8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba77fe640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba81020e0 0x7f1ba8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba77fe640 1 -- 172.21.15.86:0/225386682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba819ffb0 con 0x7f1ba8102aa0 2024-04-16T05:35:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1ba77fe640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba8102aa0 0x7f1ba819f940 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009fd0 tx=0x7f1ba000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:18.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1b8ffff640 1 -- 172.21.15.86:0/225386682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000ce20 con 0x7f1ba8102aa0 2024-04-16T05:35:18.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1b8ffff640 1 -- 172.21.15.86:0/225386682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba0005020 con 0x7f1ba8102aa0 2024-04-16T05:35:18.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba81a02a0 con 0x7f1ba8102aa0 2024-04-16T05:35:18.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.793+0000 7f1b8ffff640 1 -- 172.21.15.86:0/225386682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0010720 con 0x7f1ba8102aa0 2024-04-16T05:35:18.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.794+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba81a07f0 con 0x7f1ba8102aa0 2024-04-16T05:35:18.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.795+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba81a0ab0 con 0x7f1ba8102aa0 2024-04-16T05:35:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.796+0000 7f1b8ffff640 1 -- 172.21.15.86:0/225386682 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f1ba0005540 con 0x7f1ba8102aa0 2024-04-16T05:35:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.796+0000 7f1b8ffff640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b900a39b0 0x7f1b900a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:18.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.796+0000 7f1ba6ffd640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b900a39b0 0x7f1b900a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:18.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.796+0000 7f1b8ffff640 1 -- 172.21.15.86:0/225386682 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1ba00d0b10 con 0x7f1ba8102aa0 2024-04-16T05:35:18.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.797+0000 7f1ba6ffd640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b900a39b0 0x7f1b900a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b9802c8e0 tx=0x7f1b98005b50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:18.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:18.802+0000 7f1b8ffff640 1 -- 172.21.15.86:0/225386682 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ba009d350 con 0x7f1ba8102aa0 2024-04-16T05:35:18.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:18 smithi194 ceph-mon[35865]: [16/Apr/2024:05:35:17] ENGINE Bus STARTING 2024-04-16T05:35:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:18 smithi194 ceph-mon[35865]: [16/Apr/2024:05:35:17] ENGINE Serving on http://172.21.15.86:8765 2024-04-16T05:35:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:18 smithi194 ceph-mon[35865]: [16/Apr/2024:05:35:17] ENGINE Serving on https://172.21.15.86:7150 2024-04-16T05:35:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:18 smithi194 ceph-mon[35865]: [16/Apr/2024:05:35:17] ENGINE Bus STARTED 2024-04-16T05:35:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:18 smithi194 ceph-mon[35865]: mgrmap e22: x(active, since 1.01831s), standbys: z, y 2024-04-16T05:35:18.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:18 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3664837162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:18 smithi086 ceph-mon[32028]: [16/Apr/2024:05:35:17] ENGINE Bus STARTING 2024-04-16T05:35:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:18 smithi086 ceph-mon[32028]: [16/Apr/2024:05:35:17] ENGINE Serving on http://172.21.15.86:8765 2024-04-16T05:35:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:18 smithi086 ceph-mon[32028]: [16/Apr/2024:05:35:17] ENGINE Serving on https://172.21.15.86:7150 2024-04-16T05:35:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:18 smithi086 ceph-mon[32028]: [16/Apr/2024:05:35:17] ENGINE Bus STARTED 2024-04-16T05:35:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:18 smithi086 ceph-mon[32028]: mgrmap e22: x(active, since 1.01831s), standbys: z, y 2024-04-16T05:35:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:18 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3664837162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:19.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.014+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ba81a0c80 con 0x7f1ba8102aa0 2024-04-16T05:35:19.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.014+0000 7f1b8ffff640 1 -- 172.21.15.86:0/225386682 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f1ba009cc50 con 0x7f1ba8102aa0 2024-04-16T05:35:19.013 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:19.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b900a39b0 msgr2=0x7f1b900a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b900a39b0 0x7f1b900a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b9802c8e0 tx=0x7f1b98005b50 comp rx=0 tx=0).stop 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba8102aa0 msgr2=0x7f1ba819f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba8102aa0 0x7f1ba819f940 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009fd0 tx=0x7f1ba000ef10 comp rx=0 tx=0).stop 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 shutdown_connections 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba8102aa0 0x7f1ba819f940 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b900a39b0 0x7f1b900a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba81020e0 0x7f1ba8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 --2- 172.21.15.86:0/225386682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ba8100f10 0x7f1ba81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.016+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 >> 172.21.15.86:0/225386682 conn(0x7f1ba80fc4f0 msgr2=0x7f1ba80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.017+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 shutdown_connections 2024-04-16T05:35:19.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.017+0000 7f1bad2eb640 1 -- 172.21.15.86:0/225386682 wait complete. 2024-04-16T05:35:19.028 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:35:19.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 -- 172.21.15.86:0/1841385990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c81020e0 msgr2=0x7f39c8102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1841385990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c81020e0 0x7f39c8102560 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009900 tx=0x7f39bc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 -- 172.21.15.86:0/1841385990 shutdown_connections 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1841385990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39c8102aa0 0x7f39c81093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1841385990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c81020e0 0x7f39c8102560 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1841385990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39c8100f10 0x7f39c8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 -- 172.21.15.86:0/1841385990 >> 172.21.15.86:0/1841385990 conn(0x7f39c80fc4f0 msgr2=0x7f39c80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 -- 172.21.15.86:0/1841385990 shutdown_connections 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 -- 172.21.15.86:0/1841385990 wait complete. 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.149+0000 7f39cd55f640 1 Processor -- start 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39cd55f640 1 -- start start 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39cd55f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39c8100f10 0x7f39c819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39cd55f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39c81020e0 0x7f39c819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39cd55f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c8102aa0 0x7f39c81a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39cd55f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39c810cbe0 con 0x7f39c8102aa0 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39cd55f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f39c810ca60 con 0x7f39c81020e0 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39cd55f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39c810cd60 con 0x7f39c8100f10 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c8102aa0 0x7f39c81a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c8102aa0 0x7f39c81a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49566/0 (socket says 172.21.15.86:49566) 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c77fe640 1 -- 172.21.15.86:0/1167494433 learned_addr learned my addr 172.21.15.86:0/1167494433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c67fc640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39c81020e0 0x7f39c819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c6ffd640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39c8100f10 0x7f39c819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c77fe640 1 -- 172.21.15.86:0/1167494433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39c8100f10 msgr2=0x7f39c819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c77fe640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39c8100f10 0x7f39c819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c77fe640 1 -- 172.21.15.86:0/1167494433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39c81020e0 msgr2=0x7f39c819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c77fe640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39c81020e0 0x7f39c819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c77fe640 1 -- 172.21.15.86:0/1167494433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c81a2240 con 0x7f39c8102aa0 2024-04-16T05:35:19.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c6ffd640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39c8100f10 0x7f39c819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:19.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c67fc640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39c81020e0 0x7f39c819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:19.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39c77fe640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c8102aa0 0x7f39c81a1bd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f39c000efd0 tx=0x7f39c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:19.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.150+0000 7f39affff640 1 -- 172.21.15.86:0/1167494433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0003b70 con 0x7f39c8102aa0 2024-04-16T05:35:19.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.151+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c81a2530 con 0x7f39c8102aa0 2024-04-16T05:35:19.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.151+0000 7f39affff640 1 -- 172.21.15.86:0/1167494433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c0003d10 con 0x7f39c8102aa0 2024-04-16T05:35:19.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.151+0000 7f39affff640 1 -- 172.21.15.86:0/1167494433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00109f0 con 0x7f39c8102aa0 2024-04-16T05:35:19.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.151+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81a2a50 con 0x7f39c8102aa0 2024-04-16T05:35:19.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.152+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c81a2d10 con 0x7f39c8102aa0 2024-04-16T05:35:19.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.153+0000 7f39affff640 1 -- 172.21.15.86:0/1167494433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f39c0010b90 con 0x7f39c8102aa0 2024-04-16T05:35:19.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.153+0000 7f39affff640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39b00a39b0 0x7f39b00a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.154+0000 7f39affff640 1 -- 172.21.15.86:0/1167494433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f39c00d0c90 con 0x7f39c8102aa0 2024-04-16T05:35:19.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.154+0000 7f39c6ffd640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39b00a39b0 0x7f39b00a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.154+0000 7f39c6ffd640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39b00a39b0 0x7f39b00a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f39c8100c30 tx=0x7f39b8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:19.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.158+0000 7f39affff640 1 -- 172.21.15.86:0/1167494433 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39c009d4b0 con 0x7f39c8102aa0 2024-04-16T05:35:19.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.343+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f39c81a2ee0 con 0x7f39c8102aa0 2024-04-16T05:35:19.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.344+0000 7f39affff640 1 -- 172.21.15.86:0/1167494433 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f39c009cdb0 con 0x7f39c8102aa0 2024-04-16T05:35:19.343 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39b00a39b0 msgr2=0x7f39b00a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39b00a39b0 0x7f39b00a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f39c8100c30 tx=0x7f39b8005f90 comp rx=0 tx=0).stop 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c8102aa0 msgr2=0x7f39c81a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c8102aa0 0x7f39c81a1bd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f39c000efd0 tx=0x7f39c000c6a0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 shutdown_connections 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c8102aa0 0x7f39c81a1bd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39b00a39b0 0x7f39b00a5e70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39c81020e0 0x7f39c819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 --2- 172.21.15.86:0/1167494433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39c8100f10 0x7f39c819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 >> 172.21.15.86:0/1167494433 conn(0x7f39c80fc4f0 msgr2=0x7f39c806a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 shutdown_connections 2024-04-16T05:35:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.346+0000 7f39cd55f640 1 -- 172.21.15.86:0/1167494433 wait complete. 2024-04-16T05:35:19.356 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:35:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.467+0000 7f9b55de9640 1 -- 172.21.15.86:0/1248669650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103240 msgr2=0x7f9b501036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.467+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1248669650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103240 0x7f9b501036c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9b44009900 tx=0x7f9b4402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.467+0000 7f9b55de9640 1 -- 172.21.15.86:0/1248669650 shutdown_connections 2024-04-16T05:35:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.467+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1248669650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b50103c00 0x7f9b5010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.467+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1248669650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103240 0x7f9b501036c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.467+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1248669650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b50102050 0x7f9b50102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.467+0000 7f9b55de9640 1 -- 172.21.15.86:0/1248669650 >> 172.21.15.86:0/1248669650 conn(0x7f9b500fd5e0 msgr2=0x7f9b500ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.467+0000 7f9b55de9640 1 -- 172.21.15.86:0/1248669650 shutdown_connections 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.467+0000 7f9b55de9640 1 -- 172.21.15.86:0/1248669650 wait complete. 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b55de9640 1 Processor -- start 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b55de9640 1 -- start start 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b55de9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b50102050 0x7f9b50075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b55de9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b50103240 0x7f9b500743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b55de9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103c00 0x7f9b50074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b55de9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5010dce0 con 0x7f9b50103c00 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b55de9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5010db60 con 0x7f9b50103240 2024-04-16T05:35:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b55de9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5010de60 con 0x7f9b50102050 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b4ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103c00 0x7f9b50074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b4ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103c00 0x7f9b50074930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49586/0 (socket says 172.21.15.86:49586) 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b4ffff640 1 -- 172.21.15.86:0/1915020326 learned_addr learned my addr 172.21.15.86:0/1915020326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b4effd640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b50103240 0x7f9b500743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.468+0000 7f9b4f7fe640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b50102050 0x7f9b50075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b4ffff640 1 -- 172.21.15.86:0/1915020326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b50102050 msgr2=0x7f9b50075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b4ffff640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b50102050 0x7f9b50075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b4ffff640 1 -- 172.21.15.86:0/1915020326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b50103240 msgr2=0x7f9b500743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b4ffff640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b50103240 0x7f9b500743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b4ffff640 1 -- 172.21.15.86:0/1915020326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b500751c0 con 0x7f9b50103c00 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b4ffff640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103c00 0x7f9b50074930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800eb10 tx=0x7f9b4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b4cff9640 1 -- 172.21.15.86:0/1915020326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48004b20 con 0x7f9b50103c00 2024-04-16T05:35:19.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b501a2470 con 0x7f9b50103c00 2024-04-16T05:35:19.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b4cff9640 1 -- 172.21.15.86:0/1915020326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b48004cc0 con 0x7f9b50103c00 2024-04-16T05:35:19.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b4cff9640 1 -- 172.21.15.86:0/1915020326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48010760 con 0x7f9b50103c00 2024-04-16T05:35:19.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.469+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b501a2990 con 0x7f9b50103c00 2024-04-16T05:35:19.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.470+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b50068e20 con 0x7f9b50103c00 2024-04-16T05:35:19.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.471+0000 7f9b4cff9640 1 -- 172.21.15.86:0/1915020326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f9b48005020 con 0x7f9b50103c00 2024-04-16T05:35:19.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.472+0000 7f9b4cff9640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9b380a39b0 0x7f9b380a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.472+0000 7f9b4f7fe640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9b380a39b0 0x7f9b380a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.473+0000 7f9b4cff9640 1 -- 172.21.15.86:0/1915020326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b480d0a80 con 0x7f9b50103c00 2024-04-16T05:35:19.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.473+0000 7f9b4f7fe640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9b380a39b0 0x7f9b380a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9b501030a0 tx=0x7f9b40005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:19.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.481+0000 7f9b4cff9640 1 -- 172.21.15.86:0/1915020326 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b4809d2c0 con 0x7f9b50103c00 2024-04-16T05:35:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.673+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9b50062c00 con 0x7f9b50103c00 2024-04-16T05:35:19.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.673+0000 7f9b4cff9640 1 -- 172.21.15.86:0/1915020326 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f9b4809cbc0 con 0x7f9b50103c00 2024-04-16T05:35:19.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9b380a39b0 msgr2=0x7f9b380a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9b380a39b0 0x7f9b380a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9b501030a0 tx=0x7f9b40005f90 comp rx=0 tx=0).stop 2024-04-16T05:35:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103c00 msgr2=0x7f9b50074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103c00 0x7f9b50074930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800eb10 tx=0x7f9b4800c2d0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 shutdown_connections 2024-04-16T05:35:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50103c00 0x7f9b50074930 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9b380a39b0 0x7f9b380a5e70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b50103240 0x7f9b500743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 --2- 172.21.15.86:0/1915020326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b50102050 0x7f9b50075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 >> 172.21.15.86:0/1915020326 conn(0x7f9b500fd5e0 msgr2=0x7f9b501065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 shutdown_connections 2024-04-16T05:35:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.675+0000 7f9b55de9640 1 -- 172.21.15.86:0/1915020326 wait complete. 2024-04-16T05:35:19.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:19.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:19.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2592205202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:19.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/225386682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:19.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:19 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1167494433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:19.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.796+0000 7fde72ed9640 1 -- 172.21.15.86:0/1374754843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c101070 msgr2=0x7fde6c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.796+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1374754843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c101070 0x7fde6c1014f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fde58009900 tx=0x7fde5802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.796+0000 7fde72ed9640 1 -- 172.21.15.86:0/1374754843 shutdown_connections 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.796+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1374754843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c101a30 0x7fde6c1062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.796+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1374754843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c101070 0x7fde6c1014f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.796+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1374754843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c100310 0x7fde6c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.796+0000 7fde72ed9640 1 -- 172.21.15.86:0/1374754843 >> 172.21.15.86:0/1374754843 conn(0x7fde6c0fbc40 msgr2=0x7fde6c0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.796+0000 7fde72ed9640 1 -- 172.21.15.86:0/1374754843 shutdown_connections 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.796+0000 7fde72ed9640 1 -- 172.21.15.86:0/1374754843 wait complete. 2024-04-16T05:35:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde72ed9640 1 Processor -- start 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde72ed9640 1 -- start start 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde72ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c100310 0x7fde6c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde72ed9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c101070 0x7fde6c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde72ed9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c101a30 0x7fde6c1a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde72ed9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1142f0 con 0x7fde6c100310 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde72ed9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c114170 con 0x7fde6c101a30 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde72ed9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c114470 con 0x7fde6c101070 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde70c75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c100310 0x7fde6c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde70c75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c100310 0x7fde6c19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49604/0 (socket says 172.21.15.86:49604) 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde70c75640 1 -- 172.21.15.86:0/1610865046 learned_addr learned my addr 172.21.15.86:0/1610865046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde71476640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c101a30 0x7fde6c1a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde6bfff640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c101070 0x7fde6c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde70c75640 1 -- 172.21.15.86:0/1610865046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c101070 msgr2=0x7fde6c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde70c75640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c101070 0x7fde6c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde70c75640 1 -- 172.21.15.86:0/1610865046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c101a30 msgr2=0x7fde6c1a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde70c75640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c101a30 0x7fde6c1a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde70c75640 1 -- 172.21.15.86:0/1610865046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c1a6580 con 0x7fde6c100310 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde70c75640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c100310 0x7fde6c19e960 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fde6000d950 tx=0x7fde6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.797+0000 7fde69ffb640 1 -- 172.21.15.86:0/1610865046 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60017030 con 0x7fde6c100310 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.798+0000 7fde69ffb640 1 -- 172.21.15.86:0/1610865046 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde60004020 con 0x7fde6c100310 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.798+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6c1a6870 con 0x7fde6c100310 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.798+0000 7fde69ffb640 1 -- 172.21.15.86:0/1610865046 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde600047a0 con 0x7fde6c100310 2024-04-16T05:35:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.798+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6c1a6d90 con 0x7fde6c100310 2024-04-16T05:35:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.799+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde6c1a6ff0 con 0x7fde6c100310 2024-04-16T05:35:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.800+0000 7fde69ffb640 1 -- 172.21.15.86:0/1610865046 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fde600041d0 con 0x7fde6c100310 2024-04-16T05:35:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.800+0000 7fde69ffb640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fde5c0a39b0 0x7fde5c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.800+0000 7fde6bfff640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fde5c0a39b0 0x7fde5c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.800+0000 7fde69ffb640 1 -- 172.21.15.86:0/1610865046 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fde600cfe90 con 0x7fde6c100310 2024-04-16T05:35:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.801+0000 7fde6bfff640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fde5c0a39b0 0x7fde5c0a5e70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde58000c00 tx=0x7fde58005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:19.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.805+0000 7fde69ffb640 1 -- 172.21.15.86:0/1610865046 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde6009c6d0 con 0x7fde6c100310 2024-04-16T05:35:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2592205202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/225386682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:19 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1167494433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:19.941+0000 7fde69ffb640 1 -- 172.21.15.86:0/1610865046 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fde60098640 con 0x7fde6c100310 2024-04-16T05:35:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2592205202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/225386682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:19.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:19 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1167494433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:20.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.009+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde6c1a71c0 con 0x7fde6c100310 2024-04-16T05:35:20.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.011+0000 7fde69ffb640 1 -- 172.21.15.86:0/1610865046 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25077 (secure 0 0 0) 0x7fde60057770 con 0x7fde6c100310 2024-04-16T05:35:20.010 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:20.010 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":95,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:35:16.929832+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2330974702":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/3900523320":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2641972995":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/501397249":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:35:20.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fde5c0a39b0 msgr2=0x7fde5c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fde5c0a39b0 0x7fde5c0a5e70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde58000c00 tx=0x7fde58005fd0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c100310 msgr2=0x7fde6c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c100310 0x7fde6c19e960 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fde6000d950 tx=0x7fde6000de10 comp rx=0 tx=0).stop 2024-04-16T05:35:20.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 shutdown_connections 2024-04-16T05:35:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fde5c0a39b0 0x7fde5c0a5e70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c101a30 0x7fde6c1a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c101070 0x7fde6c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 --2- 172.21.15.86:0/1610865046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c100310 0x7fde6c19e960 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 >> 172.21.15.86:0/1610865046 conn(0x7fde6c0fbc40 msgr2=0x7fde6c0fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 shutdown_connections 2024-04-16T05:35:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.013+0000 7fde72ed9640 1 -- 172.21.15.86:0/1610865046 wait complete. 2024-04-16T05:35:20.024 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:35:20.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:35:20.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 -- 172.21.15.86:0/2750349336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 msgr2=0x7fd2b8104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/2750349336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 0x7fd2b8104eb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009900 tx=0x7fd2a802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 -- 172.21.15.86:0/2750349336 shutdown_connections 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/2750349336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2b8107d20 0x7fd2b810a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/2750349336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b81053f0 0x7fd2b81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/2750349336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 0x7fd2b8104eb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 -- 172.21.15.86:0/2750349336 >> 172.21.15.86:0/2750349336 conn(0x7fd2b80fc4b0 msgr2=0x7fd2b80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 -- 172.21.15.86:0/2750349336 shutdown_connections 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 -- 172.21.15.86:0/2750349336 wait complete. 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.137+0000 7fd2bdce4640 1 Processor -- start 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2bdce4640 1 -- start start 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2bdce4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 0x7fd2b8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2bdce4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b81053f0 0x7fd2b8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2bdce4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2b8107d20 0x7fd2b819f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2bdce4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b810dd30 con 0x7fd2b8102ac0 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2bdce4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b810dbb0 con 0x7fd2b8107d20 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2bdce4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b810deb0 con 0x7fd2b81053f0 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 0x7fd2b8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 0x7fd2b8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49622/0 (socket says 172.21.15.86:49622) 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b77fe640 1 -- 172.21.15.86:0/4006167598 learned_addr learned my addr 172.21.15.86:0/4006167598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b6ffd640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b81053f0 0x7fd2b8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b7fff640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2b8107d20 0x7fd2b819f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b77fe640 1 -- 172.21.15.86:0/4006167598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b81053f0 msgr2=0x7fd2b8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b77fe640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b81053f0 0x7fd2b8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b77fe640 1 -- 172.21.15.86:0/4006167598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2b8107d20 msgr2=0x7fd2b819f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b77fe640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2b8107d20 0x7fd2b819f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b77fe640 1 -- 172.21.15.86:0/4006167598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b81a0050 con 0x7fd2b8102ac0 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.138+0000 7fd2b77fe640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 0x7fd2b8198430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009230 tx=0x7fd2a8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:20.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.139+0000 7fd2b4ff9640 1 -- 172.21.15.86:0/4006167598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8041070 con 0x7fd2b8102ac0 2024-04-16T05:35:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.139+0000 7fd2b4ff9640 1 -- 172.21.15.86:0/4006167598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a80054e0 con 0x7fd2b8102ac0 2024-04-16T05:35:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.139+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b81a02e0 con 0x7fd2b8102ac0 2024-04-16T05:35:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.139+0000 7fd2b4ff9640 1 -- 172.21.15.86:0/4006167598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8034440 con 0x7fd2b8102ac0 2024-04-16T05:35:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.139+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b81a07d0 con 0x7fd2b8102ac0 2024-04-16T05:35:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.140+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b80fdfa0 con 0x7fd2b8102ac0 2024-04-16T05:35:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.141+0000 7fd2b4ff9640 1 -- 172.21.15.86:0/4006167598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fd2a8044070 con 0x7fd2b8102ac0 2024-04-16T05:35:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.142+0000 7fd2b4ff9640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd2a00a3db0 0x7fd2a00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.142+0000 7fd2b6ffd640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd2a00a3db0 0x7fd2a00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.142+0000 7fd2b4ff9640 1 -- 172.21.15.86:0/4006167598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd2a80f2c30 con 0x7fd2b8102ac0 2024-04-16T05:35:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.143+0000 7fd2b6ffd640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd2a00a3db0 0x7fd2a00a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8199940 tx=0x7fd2ac00b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.146+0000 7fd2b4ff9640 1 -- 172.21.15.86:0/4006167598 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2a80be8b0 con 0x7fd2b8102ac0 2024-04-16T05:35:20.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.335+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd2b802d7b0 con 0x7fd2b8102ac0 2024-04-16T05:35:20.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.335+0000 7fd2b4ff9640 1 -- 172.21.15.86:0/4006167598 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fd2a80c2bb0 con 0x7fd2b8102ac0 2024-04-16T05:35:20.334 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:35:20.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd2a00a3db0 msgr2=0x7fd2a00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd2a00a3db0 0x7fd2a00a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8199940 tx=0x7fd2ac00b410 comp rx=0 tx=0).stop 2024-04-16T05:35:20.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 msgr2=0x7fd2b8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 0x7fd2b8198430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009230 tx=0x7fd2a8005dd0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 shutdown_connections 2024-04-16T05:35:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd2a00a3db0 0x7fd2a00a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2b8107d20 0x7fd2b819f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b81053f0 0x7fd2b8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 --2- 172.21.15.86:0/4006167598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b8102ac0 0x7fd2b8198430 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.337+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 >> 172.21.15.86:0/4006167598 conn(0x7fd2b80fc4b0 msgr2=0x7fd2b81034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.338+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 shutdown_connections 2024-04-16T05:35:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.338+0000 7fd2bdce4640 1 -- 172.21.15.86:0/4006167598 wait complete. 2024-04-16T05:35:20.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:35:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.459+0000 7fa5306c7640 1 -- 172.21.15.86:0/965374693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528102ac0 msgr2=0x7fa528104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.459+0000 7fa5306c7640 1 --2- 172.21.15.86:0/965374693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528102ac0 0x7fa528104eb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa524009900 tx=0x7fa52402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.459+0000 7fa5306c7640 1 -- 172.21.15.86:0/965374693 shutdown_connections 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.459+0000 7fa5306c7640 1 --2- 172.21.15.86:0/965374693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa528107d20 0x7fa52810a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.459+0000 7fa5306c7640 1 --2- 172.21.15.86:0/965374693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5281053f0 0x7fa5281077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.459+0000 7fa5306c7640 1 --2- 172.21.15.86:0/965374693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528102ac0 0x7fa528104eb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.459+0000 7fa5306c7640 1 -- 172.21.15.86:0/965374693 >> 172.21.15.86:0/965374693 conn(0x7fa5280fc4b0 msgr2=0x7fa5280fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.459+0000 7fa5306c7640 1 -- 172.21.15.86:0/965374693 shutdown_connections 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.459+0000 7fa5306c7640 1 -- 172.21.15.86:0/965374693 wait complete. 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa5306c7640 1 Processor -- start 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa5306c7640 1 -- start start 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa5306c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa528102ac0 0x7fa5281983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa5306c7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5281053f0 0x7fa528198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa5306c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528107d20 0x7fa52819f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa5306c7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa52810dcd0 con 0x7fa528107d20 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa5306c7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa52810db50 con 0x7fa5281053f0 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa5306c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa52810de50 con 0x7fa528102ac0 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52ec64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528107d20 0x7fa52819f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52ec64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528107d20 0x7fa52819f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49626/0 (socket says 172.21.15.86:49626) 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52e463640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa528102ac0 0x7fa5281983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52ec64640 1 -- 172.21.15.86:0/2454033491 learned_addr learned my addr 172.21.15.86:0/2454033491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52dc62640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5281053f0 0x7fa528198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52ec64640 1 -- 172.21.15.86:0/2454033491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa528102ac0 msgr2=0x7fa5281983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52ec64640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa528102ac0 0x7fa5281983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52ec64640 1 -- 172.21.15.86:0/2454033491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5281053f0 msgr2=0x7fa528198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52ec64640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5281053f0 0x7fa528198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.460+0000 7fa52ec64640 1 -- 172.21.15.86:0/2454033491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5281a0000 con 0x7fa528107d20 2024-04-16T05:35:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.461+0000 7fa52ec64640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528107d20 0x7fa52819f990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa52000eb10 tx=0x7fa52000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.461+0000 7fa51f7fe640 1 -- 172.21.15.86:0/2454033491 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520004b20 con 0x7fa528107d20 2024-04-16T05:35:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.461+0000 7fa51f7fe640 1 -- 172.21.15.86:0/2454033491 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa520004cc0 con 0x7fa528107d20 2024-04-16T05:35:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.461+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5281a02f0 con 0x7fa528107d20 2024-04-16T05:35:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.461+0000 7fa51f7fe640 1 -- 172.21.15.86:0/2454033491 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520010760 con 0x7fa528107d20 2024-04-16T05:35:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.461+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5281a07f0 con 0x7fa528107d20 2024-04-16T05:35:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.462+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5280fdfa0 con 0x7fa528107d20 2024-04-16T05:35:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.463+0000 7fa51f7fe640 1 -- 172.21.15.86:0/2454033491 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fa520005020 con 0x7fa528107d20 2024-04-16T05:35:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.464+0000 7fa51f7fe640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5140a3db0 0x7fa5140a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.464+0000 7fa52e463640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5140a3db0 0x7fa5140a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.464+0000 7fa51f7fe640 1 -- 172.21.15.86:0/2454033491 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa5200d1290 con 0x7fa528107d20 2024-04-16T05:35:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.464+0000 7fa52e463640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5140a3db0 0x7fa5140a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa52402c7b0 tx=0x7fa524005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.468+0000 7fa51f7fe640 1 -- 172.21.15.86:0/2454033491 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa52009d450 con 0x7fa528107d20 2024-04-16T05:35:20.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.673+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa528062c00 con 0x7fa528107d20 2024-04-16T05:35:20.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.673+0000 7fa51f7fe640 1 -- 172.21.15.86:0/2454033491 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fa52009cd50 con 0x7fa528107d20 2024-04-16T05:35:20.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:20.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5140a3db0 msgr2=0x7fa5140a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5140a3db0 0x7fa5140a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa52402c7b0 tx=0x7fa524005fd0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528107d20 msgr2=0x7fa52819f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528107d20 0x7fa52819f990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa52000eb10 tx=0x7fa52000c2d0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 shutdown_connections 2024-04-16T05:35:20.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa528107d20 0x7fa52819f990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5140a3db0 0x7fa5140a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5281053f0 0x7fa528198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 --2- 172.21.15.86:0/2454033491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa528102ac0 0x7fa5281983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 >> 172.21.15.86:0/2454033491 conn(0x7fa5280fc4b0 msgr2=0x7fa528103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:20.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 shutdown_connections 2024-04-16T05:35:20.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.675+0000 7fa5306c7640 1 -- 172.21.15.86:0/2454033491 wait complete. 2024-04-16T05:35:20.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:20.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1915020326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:20.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1610865046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4006167598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:20.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:35:20.687 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:20 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:35:20.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:35:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.801+0000 7fe14953d640 1 -- 172.21.15.86:0/187514590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 msgr2=0x7fe144102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.801+0000 7fe14953d640 1 --2- 172.21.15.86:0/187514590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 0x7fe144102510 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe138009900 tx=0x7fe13802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.801+0000 7fe14953d640 1 -- 172.21.15.86:0/187514590 shutdown_connections 2024-04-16T05:35:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.801+0000 7fe14953d640 1 --2- 172.21.15.86:0/187514590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe144102a50 0x7fe144109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.801+0000 7fe14953d640 1 --2- 172.21.15.86:0/187514590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 0x7fe144102510 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.801+0000 7fe14953d640 1 --2- 172.21.15.86:0/187514590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144100f40 0x7fe144101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.801+0000 7fe14953d640 1 -- 172.21.15.86:0/187514590 >> 172.21.15.86:0/187514590 conn(0x7fe1440fc4b0 msgr2=0x7fe1440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.801+0000 7fe14953d640 1 -- 172.21.15.86:0/187514590 shutdown_connections 2024-04-16T05:35:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.801+0000 7fe14953d640 1 -- 172.21.15.86:0/187514590 wait complete. 2024-04-16T05:35:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe14953d640 1 Processor -- start 2024-04-16T05:35:20.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe14953d640 1 -- start start 2024-04-16T05:35:20.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe14953d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe144100f40 0x7fe1441983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe14953d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 0x7fe144198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe14953d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144102a50 0x7fe14419f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe14953d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe14410cbf0 con 0x7fe144102090 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe14953d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe14410ca70 con 0x7fe144100f40 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe14953d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe14410cd70 con 0x7fe144102a50 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe1427fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 0x7fe144198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe142ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe144100f40 0x7fe1441983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe1427fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 0x7fe144198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59444/0 (socket says 172.21.15.86:59444) 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe1427fc640 1 -- 172.21.15.86:0/3515787936 learned_addr learned my addr 172.21.15.86:0/3515787936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe1437fe640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144102a50 0x7fe14419f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe1427fc640 1 -- 172.21.15.86:0/3515787936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144102a50 msgr2=0x7fe14419f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe1427fc640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144102a50 0x7fe14419f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe1427fc640 1 -- 172.21.15.86:0/3515787936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe144100f40 msgr2=0x7fe1441983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe1427fc640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe144100f40 0x7fe1441983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:20.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe1427fc640 1 -- 172.21.15.86:0/3515787936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14419ffe0 con 0x7fe144102090 2024-04-16T05:35:20.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.802+0000 7fe142ffd640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe144100f40 0x7fe1441983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:20.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.803+0000 7fe1427fc640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 0x7fe144198900 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe138000c00 tx=0x7fe13802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:20.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.803+0000 7fe12bfff640 1 -- 172.21.15.86:0/3515787936 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138041070 con 0x7fe144102090 2024-04-16T05:35:20.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.803+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1441a0270 con 0x7fe144102090 2024-04-16T05:35:20.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.803+0000 7fe12bfff640 1 -- 172.21.15.86:0/3515787936 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe138004450 con 0x7fe144102090 2024-04-16T05:35:20.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.803+0000 7fe12bfff640 1 -- 172.21.15.86:0/3515787936 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138034440 con 0x7fe144102090 2024-04-16T05:35:20.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.803+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1441a0760 con 0x7fe144102090 2024-04-16T05:35:20.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.804+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1441a09c0 con 0x7fe144102090 2024-04-16T05:35:20.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.805+0000 7fe12bfff640 1 -- 172.21.15.86:0/3515787936 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fe138044070 con 0x7fe144102090 2024-04-16T05:35:20.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.806+0000 7fe12bfff640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe12c0a3db0 0x7fe12c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:20.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.806+0000 7fe142ffd640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe12c0a3db0 0x7fe12c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:20.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.806+0000 7fe12bfff640 1 -- 172.21.15.86:0/3515787936 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe1380f28d0 con 0x7fe144102090 2024-04-16T05:35:20.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.807+0000 7fe142ffd640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe12c0a3db0 0x7fe12c0a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe134009fc0 tx=0x7fe134005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:20.810+0000 7fe12bfff640 1 -- 172.21.15.86:0/3515787936 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1380be5e0 con 0x7fe144102090 2024-04-16T05:35:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1915020326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-04-16T05:35:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1610865046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4006167598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:35:20.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:20 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:35:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1915020326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-04-16T05:35:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1610865046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4006167598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:35:20.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:35:20.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:20 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:35:21.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.016+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe1441a0b90 con 0x7fe144102090 2024-04-16T05:35:21.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.016+0000 7fe12bfff640 1 -- 172.21.15.86:0/3515787936 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fe1380c28e0 con 0x7fe144102090 2024-04-16T05:35:21.015 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:35:21.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe12c0a3db0 msgr2=0x7fe12c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe12c0a3db0 0x7fe12c0a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe134009fc0 tx=0x7fe134005f90 comp rx=0 tx=0).stop 2024-04-16T05:35:21.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 msgr2=0x7fe144198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 0x7fe144198900 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe138000c00 tx=0x7fe13802c640 comp rx=0 tx=0).stop 2024-04-16T05:35:21.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 shutdown_connections 2024-04-16T05:35:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144102a50 0x7fe14419f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe144102090 0x7fe144198900 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe12c0a3db0 0x7fe12c0a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 --2- 172.21.15.86:0/3515787936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe144100f40 0x7fe1441983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 >> 172.21.15.86:0/3515787936 conn(0x7fe1440fc4b0 msgr2=0x7fe1440fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 shutdown_connections 2024-04-16T05:35:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.018+0000 7fe14953d640 1 -- 172.21.15.86:0/3515787936 wait complete. 2024-04-16T05:35:21.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 -- 172.21.15.86:0/888954669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103ba0 msgr2=0x7fefd010c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 --2- 172.21.15.86:0/888954669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103ba0 0x7fefd010c080 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009900 tx=0x7fefc402bd60 comp rx=0 tx=0).stop 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 -- 172.21.15.86:0/888954669 shutdown_connections 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 --2- 172.21.15.86:0/888954669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefd010c5c0 0x7fefd010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 --2- 172.21.15.86:0/888954669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103ba0 0x7fefd010c080 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 --2- 172.21.15.86:0/888954669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0103280 0x7fefd0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 -- 172.21.15.86:0/888954669 >> 172.21.15.86:0/888954669 conn(0x7fefd0078000 msgr2=0x7fefd0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 -- 172.21.15.86:0/888954669 shutdown_connections 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 -- 172.21.15.86:0/888954669 wait complete. 2024-04-16T05:35:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.142+0000 7fefd6093640 1 Processor -- start 2024-04-16T05:35:21.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefd6093640 1 -- start start 2024-04-16T05:35:21.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefd6093640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103280 0x7fefd019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefd6093640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0103ba0 0x7fefd019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefd6093640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefd010c5c0 0x7fefd0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefd6093640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefd01123d0 con 0x7fefd0103280 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefd6093640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0112250 con 0x7fefd010c5c0 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefd6093640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0112550 con 0x7fefd0103ba0 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103280 0x7fefd019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103280 0x7fefd019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59466/0 (socket says 172.21.15.86:59466) 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcf7fe640 1 -- 172.21.15.86:0/3334810571 learned_addr learned my addr 172.21.15.86:0/3334810571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefceffd640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0103ba0 0x7fefd019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcffff640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefd010c5c0 0x7fefd0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcf7fe640 1 -- 172.21.15.86:0/3334810571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0103ba0 msgr2=0x7fefd019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcf7fe640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0103ba0 0x7fefd019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcf7fe640 1 -- 172.21.15.86:0/3334810571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefd010c5c0 msgr2=0x7fefd0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcf7fe640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefd010c5c0 0x7fefd0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcf7fe640 1 -- 172.21.15.86:0/3334810571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd0198fd0 con 0x7fefd0103280 2024-04-16T05:35:21.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.143+0000 7fefcf7fe640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103280 0x7fefd019e650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fefc000d950 tx=0x7fefc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:21.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.144+0000 7fefccff9640 1 -- 172.21.15.86:0/3334810571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0017030 con 0x7fefd0103280 2024-04-16T05:35:21.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.144+0000 7fefccff9640 1 -- 172.21.15.86:0/3334810571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc0004020 con 0x7fefd0103280 2024-04-16T05:35:21.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.144+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd01992c0 con 0x7fefd0103280 2024-04-16T05:35:21.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.144+0000 7fefccff9640 1 -- 172.21.15.86:0/3334810571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc00047a0 con 0x7fefd0103280 2024-04-16T05:35:21.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.144+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd01a55d0 con 0x7fefd0103280 2024-04-16T05:35:21.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.145+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd0199600 con 0x7fefd0103280 2024-04-16T05:35:21.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.146+0000 7fefccff9640 1 -- 172.21.15.86:0/3334810571 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fefc00041d0 con 0x7fefd0103280 2024-04-16T05:35:21.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.146+0000 7fefccff9640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fefb80a3db0 0x7fefb80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.147+0000 7fefceffd640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fefb80a3db0 0x7fefb80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.147+0000 7fefccff9640 1 -- 172.21.15.86:0/3334810571 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fefc00d0670 con 0x7fefd0103280 2024-04-16T05:35:21.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.147+0000 7fefceffd640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fefb80a3db0 0x7fefb80a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fefc4000c00 tx=0x7fefc40058d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.151+0000 7fefccff9640 1 -- 172.21.15.86:0/3334810571 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefc009c830 con 0x7fefd0103280 2024-04-16T05:35:21.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.362+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fefd0199c40 con 0x7fefd0103280 2024-04-16T05:35:21.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.362+0000 7fefccff9640 1 -- 172.21.15.86:0/3334810571 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7fefc009c130 con 0x7fefd0103280 2024-04-16T05:35:21.361 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:21.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.364+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fefb80a3db0 msgr2=0x7fefb80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.364+0000 7fefd6093640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fefb80a3db0 0x7fefb80a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fefc4000c00 tx=0x7fefc40058d0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.364+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103280 msgr2=0x7fefd019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.364+0000 7fefd6093640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103280 0x7fefd019e650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fefc000d950 tx=0x7fefc000de10 comp rx=0 tx=0).stop 2024-04-16T05:35:21.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.365+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 shutdown_connections 2024-04-16T05:35:21.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.365+0000 7fefd6093640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fefb80a3db0 0x7fefb80a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.365+0000 7fefd6093640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefd010c5c0 0x7fefd0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.365+0000 7fefd6093640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0103ba0 0x7fefd019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.365+0000 7fefd6093640 1 --2- 172.21.15.86:0/3334810571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefd0103280 0x7fefd019e650 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.365+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 >> 172.21.15.86:0/3334810571 conn(0x7fefd0078000 msgr2=0x7fefd00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:21.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.365+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 shutdown_connections 2024-04-16T05:35:21.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.365+0000 7fefd6093640 1 -- 172.21.15.86:0/3334810571 wait complete. 2024-04-16T05:35:21.375 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-16T05:35:21.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:35:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.484+0000 7ffb33ed3640 1 -- 172.21.15.86:0/2322087251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c103c00 msgr2=0x7ffb2c10c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.484+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/2322087251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c103c00 0x7ffb2c10c0e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009900 tx=0x7ffb1c02bcd0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.484+0000 7ffb33ed3640 1 -- 172.21.15.86:0/2322087251 shutdown_connections 2024-04-16T05:35:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.484+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/2322087251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb2c10c620 0x7ffb2c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.484+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/2322087251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c103c00 0x7ffb2c10c0e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.484+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/2322087251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2c1032e0 0x7ffb2c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.484+0000 7ffb33ed3640 1 -- 172.21.15.86:0/2322087251 >> 172.21.15.86:0/2322087251 conn(0x7ffb2c078060 msgr2=0x7ffb2c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.484+0000 7ffb33ed3640 1 -- 172.21.15.86:0/2322087251 shutdown_connections 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.484+0000 7ffb33ed3640 1 -- 172.21.15.86:0/2322087251 wait complete. 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb33ed3640 1 Processor -- start 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb33ed3640 1 -- start start 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb33ed3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c1032e0 0x7ffb2c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb33ed3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb2c103c00 0x7ffb2c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb33ed3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2c10c620 0x7ffb2c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb33ed3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c1123d0 con 0x7ffb2c1032e0 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb33ed3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c112250 con 0x7ffb2c10c620 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb33ed3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c112550 con 0x7ffb2c103c00 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb31c6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c1032e0 0x7ffb2c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb31c6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c1032e0 0x7ffb2c19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59486/0 (socket says 172.21.15.86:59486) 2024-04-16T05:35:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb31c6f640 1 -- 172.21.15.86:0/311761915 learned_addr learned my addr 172.21.15.86:0/311761915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.485+0000 7ffb3146e640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb2c103c00 0x7ffb2c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb32470640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2c10c620 0x7ffb2c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb31c6f640 1 -- 172.21.15.86:0/311761915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb2c103c00 msgr2=0x7ffb2c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb31c6f640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb2c103c00 0x7ffb2c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb31c6f640 1 -- 172.21.15.86:0/311761915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2c10c620 msgr2=0x7ffb2c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb31c6f640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2c10c620 0x7ffb2c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb31c6f640 1 -- 172.21.15.86:0/311761915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2c198fd0 con 0x7ffb2c1032e0 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb31c6f640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c1032e0 0x7ffb2c19e600 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800d950 tx=0x7ffb2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb22ffd640 1 -- 172.21.15.86:0/311761915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28017030 con 0x7ffb2c1032e0 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb22ffd640 1 -- 172.21.15.86:0/311761915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb28004020 con 0x7ffb2c1032e0 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2c1992c0 con 0x7ffb2c1032e0 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb22ffd640 1 -- 172.21.15.86:0/311761915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb280047a0 con 0x7ffb2c1032e0 2024-04-16T05:35:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.486+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2c1a5580 con 0x7ffb2c1032e0 2024-04-16T05:35:21.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.487+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2c199600 con 0x7ffb2c1032e0 2024-04-16T05:35:21.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.488+0000 7ffb22ffd640 1 -- 172.21.15.86:0/311761915 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7ffb280041d0 con 0x7ffb2c1032e0 2024-04-16T05:35:21.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.489+0000 7ffb22ffd640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ffb180a3db0 0x7ffb180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.489+0000 7ffb3146e640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ffb180a3db0 0x7ffb180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.489+0000 7ffb22ffd640 1 -- 172.21.15.86:0/311761915 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ffb280d0d70 con 0x7ffb2c1032e0 2024-04-16T05:35:21.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.489+0000 7ffb3146e640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ffb180a3db0 0x7ffb180a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c02efd0 tx=0x7ffb1c02cf90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:21.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.492+0000 7ffb22ffd640 1 -- 172.21.15.86:0/311761915 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb2809cf30 con 0x7ffb2c1032e0 2024-04-16T05:35:21.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.595+0000 7ffb22ffd640 1 -- 172.21.15.86:0/311761915 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ffb28098ea0 con 0x7ffb2c1032e0 2024-04-16T05:35:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-04-16T05:35:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:35:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-04-16T05:35:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Unable to set osd_memory_target on smithi196 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-16T05:35:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-16T05:35:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-16T05:35:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-16T05:35:21.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2454033491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3515787936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3334810571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:35:21.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:35:21.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:35:21.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.722+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb2c06ae30 con 0x7ffb2c1032e0 2024-04-16T05:35:21.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.723+0000 7ffb22ffd640 1 -- 172.21.15.86:0/311761915 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7ffb280c72b0 con 0x7ffb2c1032e0 2024-04-16T05:35:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:21.722 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:35:21.722 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:35:21.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ffb180a3db0 msgr2=0x7ffb180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ffb180a3db0 0x7ffb180a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c02efd0 tx=0x7ffb1c02cf90 comp rx=0 tx=0).stop 2024-04-16T05:35:21.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c1032e0 msgr2=0x7ffb2c19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c1032e0 0x7ffb2c19e600 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800d950 tx=0x7ffb2800de10 comp rx=0 tx=0).stop 2024-04-16T05:35:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 shutdown_connections 2024-04-16T05:35:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ffb180a3db0 0x7ffb180a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2c10c620 0x7ffb2c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb2c103c00 0x7ffb2c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 --2- 172.21.15.86:0/311761915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2c1032e0 0x7ffb2c19e600 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.725+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 >> 172.21.15.86:0/311761915 conn(0x7ffb2c078060 msgr2=0x7ffb2c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.726+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 shutdown_connections 2024-04-16T05:35:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.726+0000 7ffb33ed3640 1 -- 172.21.15.86:0/311761915 wait complete. 2024-04-16T05:35:21.735 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-16T05:35:21.736 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-16T05:35:21.736 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:35:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 -- 172.21.15.86:0/3467181747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 msgr2=0x7f7510102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 --2- 172.21.15.86:0/3467181747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 0x7f7510102650 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f750c009900 tx=0x7f750c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 -- 172.21.15.86:0/3467181747 shutdown_connections 2024-04-16T05:35:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 --2- 172.21.15.86:0/3467181747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7510107340 0x7f751010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 --2- 172.21.15.86:0/3467181747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7510102c20 0x7f7510106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 --2- 172.21.15.86:0/3467181747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 0x7f7510102650 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 -- 172.21.15.86:0/3467181747 >> 172.21.15.86:0/3467181747 conn(0x7f7510079490 msgr2=0x7f75100798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 -- 172.21.15.86:0/3467181747 shutdown_connections 2024-04-16T05:35:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 -- 172.21.15.86:0/3467181747 wait complete. 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 Processor -- start 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.842+0000 7f7517677640 1 -- start start 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7517677640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 0x7f751019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7517677640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7510107340 0x7f751019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7517677640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75101986a0 0x7f7510198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7517677640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7510112400 con 0x7f7510102270 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7517677640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7510112280 con 0x7f7510107340 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7517677640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7510112580 con 0x7f75101986a0 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515413640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 0x7f751019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515413640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 0x7f751019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59502/0 (socket says 172.21.15.86:59502) 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515413640 1 -- 172.21.15.86:0/1587021855 learned_addr learned my addr 172.21.15.86:0/1587021855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515c14640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75101986a0 0x7f7510198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7514c12640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7510107340 0x7f751019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515413640 1 -- 172.21.15.86:0/1587021855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75101986a0 msgr2=0x7f7510198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515413640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75101986a0 0x7f7510198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515413640 1 -- 172.21.15.86:0/1587021855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7510107340 msgr2=0x7f751019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515413640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7510107340 0x7f751019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515413640 1 -- 172.21.15.86:0/1587021855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75101992a0 con 0x7f7510102270 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7514c12640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7510107340 0x7f751019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515c14640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75101986a0 0x7f7510198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7515413640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 0x7f751019e590 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f750c006010 tx=0x7f750c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f75067fc640 1 -- 172.21.15.86:0/1587021855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c041070 con 0x7f7510102270 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f75067fc640 1 -- 172.21.15.86:0/1587021855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750c004450 con 0x7f7510102270 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7510199530 con 0x7f7510102270 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f75067fc640 1 -- 172.21.15.86:0/1587021855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c003e00 con 0x7f7510102270 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.843+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75101a5670 con 0x7f7510102270 2024-04-16T05:35:21.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.844+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7510103740 con 0x7f7510102270 2024-04-16T05:35:21.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.845+0000 7f75067fc640 1 -- 172.21.15.86:0/1587021855 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f750c044070 con 0x7f7510102270 2024-04-16T05:35:21.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.846+0000 7f75067fc640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74fc0a3ec0 0x7f74fc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:21.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.846+0000 7f7514c12640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74fc0a3ec0 0x7f74fc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:21.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.847+0000 7f75067fc640 1 -- 172.21.15.86:0/1587021855 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f750c0f28a0 con 0x7f7510102270 2024-04-16T05:35:21.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.847+0000 7f7514c12640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74fc0a3ec0 0x7f74fc0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7510199ea0 tx=0x7f750000a5c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:21.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:21.851+0000 7f75067fc640 1 -- 172.21.15.86:0/1587021855 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f750c0bea90 con 0x7f7510102270 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Unable to set osd_memory_target on smithi196 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-16T05:35:21.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2454033491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3515787936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3334810571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:35:21.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:35:21.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:35:21.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-04-16T05:35:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:35:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-04-16T05:35:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Unable to set osd_memory_target on smithi196 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-16T05:35:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-16T05:35:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-16T05:35:21.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2454033491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3515787936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3334810571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:21.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:35:21.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:35:21.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:35:21.987 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:35:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.026+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7510102c20 con 0x7f74fc0a3ec0 2024-04-16T05:35:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.036+0000 7f75067fc640 1 -- 172.21.15.86:0/1587021855 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339713 (secure 0 0 0) 0x7f7510102c20 con 0x7f74fc0a3ec0 2024-04-16T05:35:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:22.040 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74fc0a3ec0 msgr2=0x7f74fc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74fc0a3ec0 0x7f74fc0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7510199ea0 tx=0x7f750000a5c0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 msgr2=0x7f751019e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 0x7f751019e590 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f750c006010 tx=0x7f750c02cb30 comp rx=0 tx=0).stop 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 shutdown_connections 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75101986a0 0x7f7510198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74fc0a3ec0 0x7f74fc0a6380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7510107340 0x7f751019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 --2- 172.21.15.86:0/1587021855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7510102270 0x7f751019e590 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 >> 172.21.15.86:0/1587021855 conn(0x7f7510079490 msgr2=0x7f75100ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 shutdown_connections 2024-04-16T05:35:22.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.039+0000 7f7517677640 1 -- 172.21.15.86:0/1587021855 wait complete. 2024-04-16T05:35:22.049 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-16T05:35:20.947179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191369,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1472,"num_read_kb":903,"num_write":562,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370484,"kb_used_data":12972,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571340,"statfs":{"total":1151940427776,"available":1151561052160,"internally_reserved":0,"allocated":13283328,"data_stored":9798980,"data_compressed":560711,"data_compressed_allocated":757760,"data_compressed_original":4206592,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987345"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140922+0000","last_change":"2024-04-16T05:32:55.507548+0000","last_active":"2024-04-16T05:35:17.140922+0000","last_peered":"2024-04-16T05:35:17.140922+0000","last_clean":"2024-04-16T05:35:17.140922+0000","last_became_active":"2024-04-16T05:32:55.507200+0000","last_became_peered":"2024-04-16T05:32:55.507200+0000","last_unstale":"2024-04-16T05:35:17.140922+0000","last_undegraded":"2024-04-16T05:35:17.140922+0000","last_fullsized":"2024-04-16T05:35:17.140922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:58:07.842226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139986+0000","last_change":"2024-04-16T05:32:55.509792+0000","last_active":"2024-04-16T05:35:17.139986+0000","last_peered":"2024-04-16T05:35:17.139986+0000","last_clean":"2024-04-16T05:35:17.139986+0000","last_became_active":"2024-04-16T05:32:55.509442+0000","last_became_peered":"2024-04-16T05:32:55.509442+0000","last_unstale":"2024-04-16T05:35:17.139986+0000","last_undegraded":"2024-04-16T05:35:17.139986+0000","last_fullsized":"2024-04-16T05:35:17.139986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:08:42.040875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140224+0000","last_change":"2024-04-16T05:32:55.505664+0000","last_active":"2024-04-16T05:35:17.140224+0000","last_peered":"2024-04-16T05:35:17.140224+0000","last_clean":"2024-04-16T05:35:17.140224+0000","last_became_active":"2024-04-16T05:32:55.505281+0000","last_became_peered":"2024-04-16T05:32:55.505281+0000","last_unstale":"2024-04-16T05:35:17.140224+0000","last_undegraded":"2024-04-16T05:35:17.140224+0000","last_fullsized":"2024-04-16T05:35:17.140224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:33.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139993+0000","last_change":"2024-04-16T05:32:55.517072+0000","last_active":"2024-04-16T05:35:17.139993+0000","last_peered":"2024-04-16T05:35:17.139993+0000","last_clean":"2024-04-16T05:35:17.139993+0000","last_became_active":"2024-04-16T05:32:55.516648+0000","last_became_peered":"2024-04-16T05:32:55.516648+0000","last_unstale":"2024-04-16T05:35:17.139993+0000","last_undegraded":"2024-04-16T05:35:17.139993+0000","last_fullsized":"2024-04-16T05:35:17.139993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:41.735720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935557+0000","last_change":"2024-04-16T05:32:55.506773+0000","last_active":"2024-04-16T05:35:16.935557+0000","last_peered":"2024-04-16T05:35:16.935557+0000","last_clean":"2024-04-16T05:35:16.935557+0000","last_became_active":"2024-04-16T05:32:55.506414+0000","last_became_peered":"2024-04-16T05:32:55.506414+0000","last_unstale":"2024-04-16T05:35:16.935557+0000","last_undegraded":"2024-04-16T05:35:16.935557+0000","last_fullsized":"2024-04-16T05:35:16.935557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:59.254334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935461+0000","last_change":"2024-04-16T05:32:55.517875+0000","last_active":"2024-04-16T05:35:16.935461+0000","last_peered":"2024-04-16T05:35:16.935461+0000","last_clean":"2024-04-16T05:35:16.935461+0000","last_became_active":"2024-04-16T05:32:55.517546+0000","last_became_peered":"2024-04-16T05:32:55.517546+0000","last_unstale":"2024-04-16T05:35:16.935461+0000","last_undegraded":"2024-04-16T05:35:16.935461+0000","last_fullsized":"2024-04-16T05:35:16.935461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:12:15.098439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139958+0000","last_change":"2024-04-16T05:32:55.509036+0000","last_active":"2024-04-16T05:35:17.139958+0000","last_peered":"2024-04-16T05:35:17.139958+0000","last_clean":"2024-04-16T05:35:17.139958+0000","last_became_active":"2024-04-16T05:32:55.508690+0000","last_became_peered":"2024-04-16T05:32:55.508690+0000","last_unstale":"2024-04-16T05:35:17.139958+0000","last_undegraded":"2024-04-16T05:35:17.139958+0000","last_fullsized":"2024-04-16T05:35:17.139958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:26.724281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934682+0000","last_change":"2024-04-16T05:32:55.510146+0000","last_active":"2024-04-16T05:35:16.934682+0000","last_peered":"2024-04-16T05:35:16.934682+0000","last_clean":"2024-04-16T05:35:16.934682+0000","last_became_active":"2024-04-16T05:32:55.508665+0000","last_became_peered":"2024-04-16T05:32:55.508665+0000","last_unstale":"2024-04-16T05:35:16.934682+0000","last_undegraded":"2024-04-16T05:35:16.934682+0000","last_fullsized":"2024-04-16T05:35:16.934682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:14.158221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140557+0000","last_change":"2024-04-16T05:32:55.513738+0000","last_active":"2024-04-16T05:35:17.140557+0000","last_peered":"2024-04-16T05:35:17.140557+0000","last_clean":"2024-04-16T05:35:17.140557+0000","last_became_active":"2024-04-16T05:32:55.513374+0000","last_became_peered":"2024-04-16T05:32:55.513374+0000","last_unstale":"2024-04-16T05:35:17.140557+0000","last_undegraded":"2024-04-16T05:35:17.140557+0000","last_fullsized":"2024-04-16T05:35:17.140557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:06:23.390908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935707+0000","last_change":"2024-04-16T05:32:55.519276+0000","last_active":"2024-04-16T05:35:16.935707+0000","last_peered":"2024-04-16T05:35:16.935707+0000","last_clean":"2024-04-16T05:35:16.935707+0000","last_became_active":"2024-04-16T05:32:55.518729+0000","last_became_peered":"2024-04-16T05:32:55.518729+0000","last_unstale":"2024-04-16T05:35:16.935707+0000","last_undegraded":"2024-04-16T05:35:16.935707+0000","last_fullsized":"2024-04-16T05:35:16.935707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:44:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140646+0000","last_change":"2024-04-16T05:32:55.516379+0000","last_active":"2024-04-16T05:35:17.140646+0000","last_peered":"2024-04-16T05:35:17.140646+0000","last_clean":"2024-04-16T05:35:17.140646+0000","last_became_active":"2024-04-16T05:32:55.515574+0000","last_became_peered":"2024-04-16T05:32:55.515574+0000","last_unstale":"2024-04-16T05:35:17.140646+0000","last_undegraded":"2024-04-16T05:35:17.140646+0000","last_fullsized":"2024-04-16T05:35:17.140646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:16:01.744380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032219+0000","last_change":"2024-04-16T05:33:52.674399+0000","last_active":"2024-04-16T05:35:17.032219+0000","last_peered":"2024-04-16T05:35:17.032219+0000","last_clean":"2024-04-16T05:35:17.032219+0000","last_became_active":"2024-04-16T05:33:52.673964+0000","last_became_peered":"2024-04-16T05:33:52.673964+0000","last_unstale":"2024-04-16T05:35:17.032219+0000","last_undegraded":"2024-04-16T05:35:17.032219+0000","last_fullsized":"2024-04-16T05:35:17.032219+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.166747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141154+0000","last_change":"2024-04-16T05:32:55.509682+0000","last_active":"2024-04-16T05:35:17.141154+0000","last_peered":"2024-04-16T05:35:17.141154+0000","last_clean":"2024-04-16T05:35:17.141154+0000","last_became_active":"2024-04-16T05:32:55.509383+0000","last_became_peered":"2024-04-16T05:32:55.509383+0000","last_unstale":"2024-04-16T05:35:17.141154+0000","last_undegraded":"2024-04-16T05:35:17.141154+0000","last_fullsized":"2024-04-16T05:35:17.141154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:25:23.487003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139784+0000","last_change":"2024-04-16T05:32:55.506338+0000","last_active":"2024-04-16T05:35:17.139784+0000","last_peered":"2024-04-16T05:35:17.139784+0000","last_clean":"2024-04-16T05:35:17.139784+0000","last_became_active":"2024-04-16T05:32:55.506000+0000","last_became_peered":"2024-04-16T05:32:55.506000+0000","last_unstale":"2024-04-16T05:35:17.139784+0000","last_undegraded":"2024-04-16T05:35:17.139784+0000","last_fullsized":"2024-04-16T05:35:17.139784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:37:02.296515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139777+0000","last_change":"2024-04-16T05:32:55.516897+0000","last_active":"2024-04-16T05:35:17.139777+0000","last_peered":"2024-04-16T05:35:17.139777+0000","last_clean":"2024-04-16T05:35:17.139777+0000","last_became_active":"2024-04-16T05:32:55.516489+0000","last_became_peered":"2024-04-16T05:32:55.516489+0000","last_unstale":"2024-04-16T05:35:17.139777+0000","last_undegraded":"2024-04-16T05:35:17.139777+0000","last_fullsized":"2024-04-16T05:35:17.139777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:26:08.289501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139046+0000","last_change":"2024-04-16T05:32:55.507374+0000","last_active":"2024-04-16T05:35:17.139046+0000","last_peered":"2024-04-16T05:35:17.139046+0000","last_clean":"2024-04-16T05:35:17.139046+0000","last_became_active":"2024-04-16T05:32:55.507045+0000","last_became_peered":"2024-04-16T05:32:55.507045+0000","last_unstale":"2024-04-16T05:35:17.139046+0000","last_undegraded":"2024-04-16T05:35:17.139046+0000","last_fullsized":"2024-04-16T05:35:17.139046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:35.968704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140697+0000","last_change":"2024-04-16T05:32:55.508845+0000","last_active":"2024-04-16T05:35:17.140697+0000","last_peered":"2024-04-16T05:35:17.140697+0000","last_clean":"2024-04-16T05:35:17.140697+0000","last_became_active":"2024-04-16T05:32:55.508527+0000","last_became_peered":"2024-04-16T05:32:55.508527+0000","last_unstale":"2024-04-16T05:35:17.140697+0000","last_undegraded":"2024-04-16T05:35:17.140697+0000","last_fullsized":"2024-04-16T05:35:17.140697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:20:15.033043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032367+0000","last_change":"2024-04-16T05:32:55.506059+0000","last_active":"2024-04-16T05:35:17.032367+0000","last_peered":"2024-04-16T05:35:17.032367+0000","last_clean":"2024-04-16T05:35:17.032367+0000","last_became_active":"2024-04-16T05:32:55.505728+0000","last_became_peered":"2024-04-16T05:32:55.505728+0000","last_unstale":"2024-04-16T05:35:17.032367+0000","last_undegraded":"2024-04-16T05:35:17.032367+0000","last_fullsized":"2024-04-16T05:35:17.032367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:19.253543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140417+0000","last_change":"2024-04-16T05:32:55.506791+0000","last_active":"2024-04-16T05:35:17.140417+0000","last_peered":"2024-04-16T05:35:17.140417+0000","last_clean":"2024-04-16T05:35:17.140417+0000","last_became_active":"2024-04-16T05:32:55.506379+0000","last_became_peered":"2024-04-16T05:32:55.506379+0000","last_unstale":"2024-04-16T05:35:17.140417+0000","last_undegraded":"2024-04-16T05:35:17.140417+0000","last_fullsized":"2024-04-16T05:35:17.140417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:09:11.457144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139494+0000","last_change":"2024-04-16T05:33:52.582503+0000","last_active":"2024-04-16T05:35:17.139494+0000","last_peered":"2024-04-16T05:35:17.139494+0000","last_clean":"2024-04-16T05:35:17.139494+0000","last_became_active":"2024-04-16T05:33:52.581996+0000","last_became_peered":"2024-04-16T05:33:52.581996+0000","last_unstale":"2024-04-16T05:35:17.139494+0000","last_undegraded":"2024-04-16T05:35:17.139494+0000","last_fullsized":"2024-04-16T05:35:17.139494+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:34:40.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032559+0000","last_change":"2024-04-16T05:32:54.504826+0000","last_active":"2024-04-16T05:35:17.032559+0000","last_peered":"2024-04-16T05:35:17.032559+0000","last_clean":"2024-04-16T05:35:17.032559+0000","last_became_active":"2024-04-16T05:32:54.504526+0000","last_became_peered":"2024-04-16T05:32:54.504526+0000","last_unstale":"2024-04-16T05:35:17.032559+0000","last_undegraded":"2024-04-16T05:35:17.032559+0000","last_fullsized":"2024-04-16T05:35:17.032559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:21:28.434692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935279+0000","last_change":"2024-04-16T05:33:52.580931+0000","last_active":"2024-04-16T05:35:16.935279+0000","last_peered":"2024-04-16T05:35:16.935279+0000","last_clean":"2024-04-16T05:35:16.935279+0000","last_became_active":"2024-04-16T05:33:52.580685+0000","last_became_peered":"2024-04-16T05:33:52.580685+0000","last_unstale":"2024-04-16T05:35:16.935279+0000","last_undegraded":"2024-04-16T05:35:16.935279+0000","last_fullsized":"2024-04-16T05:35:16.935279+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:35:34.548838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139517+0000","last_change":"2024-04-16T05:32:59.524687+0000","last_active":"2024-04-16T05:35:17.139517+0000","last_peered":"2024-04-16T05:35:17.139517+0000","last_clean":"2024-04-16T05:35:17.139517+0000","last_became_active":"2024-04-16T05:32:59.524228+0000","last_became_peered":"2024-04-16T05:32:59.524228+0000","last_unstale":"2024-04-16T05:35:17.139517+0000","last_undegraded":"2024-04-16T05:35:17.139517+0000","last_fullsized":"2024-04-16T05:35:17.139517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:23:15.244359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935307+0000","last_change":"2024-04-16T05:32:54.504062+0000","last_active":"2024-04-16T05:35:16.935307+0000","last_peered":"2024-04-16T05:35:16.935307+0000","last_clean":"2024-04-16T05:35:16.935307+0000","last_became_active":"2024-04-16T05:32:54.503745+0000","last_became_peered":"2024-04-16T05:32:54.503745+0000","last_unstale":"2024-04-16T05:35:16.935307+0000","last_undegraded":"2024-04-16T05:35:16.935307+0000","last_fullsized":"2024-04-16T05:35:16.935307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:29:34.157937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140473+0000","last_change":"2024-04-16T05:32:55.509820+0000","last_active":"2024-04-16T05:35:17.140473+0000","last_peered":"2024-04-16T05:35:17.140473+0000","last_clean":"2024-04-16T05:35:17.140473+0000","last_became_active":"2024-04-16T05:32:55.509473+0000","last_became_peered":"2024-04-16T05:32:55.509473+0000","last_unstale":"2024-04-16T05:35:17.140473+0000","last_undegraded":"2024-04-16T05:35:17.140473+0000","last_fullsized":"2024-04-16T05:35:17.140473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:12:22.433189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140278+0000","last_change":"2024-04-16T05:33:52.678903+0000","last_active":"2024-04-16T05:35:17.140278+0000","last_peered":"2024-04-16T05:35:17.140278+0000","last_clean":"2024-04-16T05:35:17.140278+0000","last_became_active":"2024-04-16T05:33:52.678388+0000","last_became_peered":"2024-04-16T05:33:52.678388+0000","last_unstale":"2024-04-16T05:35:17.140278+0000","last_undegraded":"2024-04-16T05:35:17.140278+0000","last_fullsized":"2024-04-16T05:35:17.140278+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:07:01.060645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935984+0000","last_change":"2024-04-16T05:32:54.511469+0000","last_active":"2024-04-16T05:35:16.935984+0000","last_peered":"2024-04-16T05:35:16.935984+0000","last_clean":"2024-04-16T05:35:16.935984+0000","last_became_active":"2024-04-16T05:32:54.511120+0000","last_became_peered":"2024-04-16T05:32:54.511120+0000","last_unstale":"2024-04-16T05:35:16.935984+0000","last_undegraded":"2024-04-16T05:35:16.935984+0000","last_fullsized":"2024-04-16T05:35:16.935984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139428+0000","last_change":"2024-04-16T05:32:55.512308+0000","last_active":"2024-04-16T05:35:17.139428+0000","last_peered":"2024-04-16T05:35:17.139428+0000","last_clean":"2024-04-16T05:35:17.139428+0000","last_became_active":"2024-04-16T05:32:55.511625+0000","last_became_peered":"2024-04-16T05:32:55.511625+0000","last_unstale":"2024-04-16T05:35:17.139428+0000","last_undegraded":"2024-04-16T05:35:17.139428+0000","last_fullsized":"2024-04-16T05:35:17.139428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:29:49.323092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139445+0000","last_change":"2024-04-16T05:32:59.519306+0000","last_active":"2024-04-16T05:35:17.139445+0000","last_peered":"2024-04-16T05:35:17.139445+0000","last_clean":"2024-04-16T05:35:17.139445+0000","last_became_active":"2024-04-16T05:32:59.518797+0000","last_became_peered":"2024-04-16T05:32:59.518797+0000","last_unstale":"2024-04-16T05:35:17.139445+0000","last_undegraded":"2024-04-16T05:35:17.139445+0000","last_fullsized":"2024-04-16T05:35:17.139445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:04:12.389004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935592+0000","last_change":"2024-04-16T05:32:54.505102+0000","last_active":"2024-04-16T05:35:16.935592+0000","last_peered":"2024-04-16T05:35:16.935592+0000","last_clean":"2024-04-16T05:35:16.935592+0000","last_became_active":"2024-04-16T05:32:54.504806+0000","last_became_peered":"2024-04-16T05:32:54.504806+0000","last_unstale":"2024-04-16T05:35:16.935592+0000","last_undegraded":"2024-04-16T05:35:16.935592+0000","last_fullsized":"2024-04-16T05:35:16.935592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:19.467690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.041059+0000","last_change":"2024-04-16T05:32:55.517097+0000","last_active":"2024-04-16T05:35:17.041059+0000","last_peered":"2024-04-16T05:35:17.041059+0000","last_clean":"2024-04-16T05:35:17.041059+0000","last_became_active":"2024-04-16T05:32:55.516717+0000","last_became_peered":"2024-04-16T05:32:55.516717+0000","last_unstale":"2024-04-16T05:35:17.041059+0000","last_undegraded":"2024-04-16T05:35:17.041059+0000","last_fullsized":"2024-04-16T05:35:17.041059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:10:44.544935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140140+0000","last_change":"2024-04-16T05:32:59.521429+0000","last_active":"2024-04-16T05:35:17.140140+0000","last_peered":"2024-04-16T05:35:17.140140+0000","last_clean":"2024-04-16T05:35:17.140140+0000","last_became_active":"2024-04-16T05:32:59.520789+0000","last_became_peered":"2024-04-16T05:32:59.520789+0000","last_unstale":"2024-04-16T05:35:17.140140+0000","last_undegraded":"2024-04-16T05:35:17.140140+0000","last_fullsized":"2024-04-16T05:35:17.140140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:40:14.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":167,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787507+0000","last_change":"2024-04-16T05:32:54.507374+0000","last_active":"2024-04-16T05:35:17.787507+0000","last_peered":"2024-04-16T05:35:17.787507+0000","last_clean":"2024-04-16T05:35:17.787507+0000","last_became_active":"2024-04-16T05:32:54.507055+0000","last_became_peered":"2024-04-16T05:32:54.507055+0000","last_unstale":"2024-04-16T05:35:17.787507+0000","last_undegraded":"2024-04-16T05:35:17.787507+0000","last_fullsized":"2024-04-16T05:35:17.787507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:00:57.761971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935533+0000","last_change":"2024-04-16T05:32:55.513980+0000","last_active":"2024-04-16T05:35:16.935533+0000","last_peered":"2024-04-16T05:35:16.935533+0000","last_clean":"2024-04-16T05:35:16.935533+0000","last_became_active":"2024-04-16T05:32:55.513622+0000","last_became_peered":"2024-04-16T05:32:55.513622+0000","last_unstale":"2024-04-16T05:35:16.935533+0000","last_undegraded":"2024-04-16T05:35:16.935533+0000","last_fullsized":"2024-04-16T05:35:16.935533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:16:10.018381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139704+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:35:17.139704+0000","last_peered":"2024-04-16T05:35:17.139704+0000","last_clean":"2024-04-16T05:35:17.139704+0000","last_became_active":"2024-04-16T05:32:59.523032+0000","last_became_peered":"2024-04-16T05:32:59.523032+0000","last_unstale":"2024-04-16T05:35:17.139704+0000","last_undegraded":"2024-04-16T05:35:17.139704+0000","last_fullsized":"2024-04-16T05:35:17.139704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:33:17.897241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935663+0000","last_change":"2024-04-16T05:32:54.504527+0000","last_active":"2024-04-16T05:35:16.935663+0000","last_peered":"2024-04-16T05:35:16.935663+0000","last_clean":"2024-04-16T05:35:16.935663+0000","last_became_active":"2024-04-16T05:32:54.504187+0000","last_became_peered":"2024-04-16T05:32:54.504187+0000","last_unstale":"2024-04-16T05:35:16.935663+0000","last_undegraded":"2024-04-16T05:35:16.935663+0000","last_fullsized":"2024-04-16T05:35:16.935663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.414362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040983+0000","last_change":"2024-04-16T05:32:55.520270+0000","last_active":"2024-04-16T05:35:17.040983+0000","last_peered":"2024-04-16T05:35:17.040983+0000","last_clean":"2024-04-16T05:35:17.040983+0000","last_became_active":"2024-04-16T05:32:55.519895+0000","last_became_peered":"2024-04-16T05:32:55.519895+0000","last_unstale":"2024-04-16T05:35:17.040983+0000","last_undegraded":"2024-04-16T05:35:17.040983+0000","last_fullsized":"2024-04-16T05:35:17.040983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:09.967836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140531+0000","last_change":"2024-04-16T05:32:59.518997+0000","last_active":"2024-04-16T05:35:17.140531+0000","last_peered":"2024-04-16T05:35:17.140531+0000","last_clean":"2024-04-16T05:35:17.140531+0000","last_became_active":"2024-04-16T05:32:59.518558+0000","last_became_peered":"2024-04-16T05:32:59.518558+0000","last_unstale":"2024-04-16T05:35:17.140531+0000","last_undegraded":"2024-04-16T05:35:17.140531+0000","last_fullsized":"2024-04-16T05:35:17.140531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:04:30.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140063+0000","last_change":"2024-04-16T05:32:54.502074+0000","last_active":"2024-04-16T05:35:17.140063+0000","last_peered":"2024-04-16T05:35:17.140063+0000","last_clean":"2024-04-16T05:35:17.140063+0000","last_became_active":"2024-04-16T05:32:54.501730+0000","last_became_peered":"2024-04-16T05:32:54.501730+0000","last_unstale":"2024-04-16T05:35:17.140063+0000","last_undegraded":"2024-04-16T05:35:17.140063+0000","last_fullsized":"2024-04-16T05:35:17.140063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:05:02.929799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934611+0000","last_change":"2024-04-16T05:32:55.504557+0000","last_active":"2024-04-16T05:35:16.934611+0000","last_peered":"2024-04-16T05:35:16.934611+0000","last_clean":"2024-04-16T05:35:16.934611+0000","last_became_active":"2024-04-16T05:32:55.504244+0000","last_became_peered":"2024-04-16T05:32:55.504244+0000","last_unstale":"2024-04-16T05:35:16.934611+0000","last_undegraded":"2024-04-16T05:35:16.934611+0000","last_fullsized":"2024-04-16T05:35:16.934611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:33:40.500436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139880+0000","last_change":"2024-04-16T05:32:59.524681+0000","last_active":"2024-04-16T05:35:17.139880+0000","last_peered":"2024-04-16T05:35:17.139880+0000","last_clean":"2024-04-16T05:35:17.139880+0000","last_became_active":"2024-04-16T05:32:59.524217+0000","last_became_peered":"2024-04-16T05:32:59.524217+0000","last_unstale":"2024-04-16T05:35:17.139880+0000","last_undegraded":"2024-04-16T05:35:17.139880+0000","last_fullsized":"2024-04-16T05:35:17.139880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:28.314442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040615+0000","last_change":"2024-04-16T05:32:54.508242+0000","last_active":"2024-04-16T05:35:17.040615+0000","last_peered":"2024-04-16T05:35:17.040615+0000","last_clean":"2024-04-16T05:35:17.040615+0000","last_became_active":"2024-04-16T05:32:54.507782+0000","last_became_peered":"2024-04-16T05:32:54.507782+0000","last_unstale":"2024-04-16T05:35:17.040615+0000","last_undegraded":"2024-04-16T05:35:17.040615+0000","last_fullsized":"2024-04-16T05:35:17.040615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:27:26.805716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.031911+0000","last_change":"2024-04-16T05:32:55.508985+0000","last_active":"2024-04-16T05:35:17.031911+0000","last_peered":"2024-04-16T05:35:17.031911+0000","last_clean":"2024-04-16T05:35:17.031911+0000","last_became_active":"2024-04-16T05:32:55.508659+0000","last_became_peered":"2024-04-16T05:32:55.508659+0000","last_unstale":"2024-04-16T05:35:17.031911+0000","last_undegraded":"2024-04-16T05:35:17.031911+0000","last_fullsized":"2024-04-16T05:35:17.031911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:43:41.534155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934714+0000","last_change":"2024-04-16T05:32:59.521257+0000","last_active":"2024-04-16T05:35:16.934714+0000","last_peered":"2024-04-16T05:35:16.934714+0000","last_clean":"2024-04-16T05:35:16.934714+0000","last_became_active":"2024-04-16T05:32:59.520848+0000","last_became_peered":"2024-04-16T05:32:59.520848+0000","last_unstale":"2024-04-16T05:35:16.934714+0000","last_undegraded":"2024-04-16T05:35:16.934714+0000","last_fullsized":"2024-04-16T05:35:16.934714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:14.219378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'8","reported_seq":284,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.786952+0000","last_change":"2024-04-16T05:32:54.500684+0000","last_active":"2024-04-16T05:35:17.786952+0000","last_peered":"2024-04-16T05:35:17.786952+0000","last_clean":"2024-04-16T05:35:17.786952+0000","last_became_active":"2024-04-16T05:32:54.500290+0000","last_became_peered":"2024-04-16T05:32:54.500290+0000","last_unstale":"2024-04-16T05:35:17.786952+0000","last_undegraded":"2024-04-16T05:35:17.786952+0000","last_fullsized":"2024-04-16T05:35:17.786952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:20:55.263313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140189+0000","last_change":"2024-04-16T05:32:55.511224+0000","last_active":"2024-04-16T05:35:17.140189+0000","last_peered":"2024-04-16T05:35:17.140189+0000","last_clean":"2024-04-16T05:35:17.140189+0000","last_became_active":"2024-04-16T05:32:55.510891+0000","last_became_peered":"2024-04-16T05:32:55.510891+0000","last_unstale":"2024-04-16T05:35:17.140189+0000","last_undegraded":"2024-04-16T05:35:17.140189+0000","last_fullsized":"2024-04-16T05:35:17.140189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:27:24.894658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.031803+0000","last_change":"2024-04-16T05:32:59.521695+0000","last_active":"2024-04-16T05:35:17.031803+0000","last_peered":"2024-04-16T05:35:17.031803+0000","last_clean":"2024-04-16T05:35:17.031803+0000","last_became_active":"2024-04-16T05:32:59.520569+0000","last_became_peered":"2024-04-16T05:32:59.520569+0000","last_unstale":"2024-04-16T05:35:17.031803+0000","last_undegraded":"2024-04-16T05:35:17.031803+0000","last_fullsized":"2024-04-16T05:35:17.031803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:01:14.726043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141002+0000","last_change":"2024-04-16T05:32:54.505058+0000","last_active":"2024-04-16T05:35:17.141002+0000","last_peered":"2024-04-16T05:35:17.141002+0000","last_clean":"2024-04-16T05:35:17.141002+0000","last_became_active":"2024-04-16T05:32:54.504636+0000","last_became_peered":"2024-04-16T05:32:54.504636+0000","last_unstale":"2024-04-16T05:35:17.141002+0000","last_undegraded":"2024-04-16T05:35:17.141002+0000","last_fullsized":"2024-04-16T05:35:17.141002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:49:14.429468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140474+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:35:17.140474+0000","last_peered":"2024-04-16T05:35:17.140474+0000","last_clean":"2024-04-16T05:35:17.140474+0000","last_became_active":"2024-04-16T05:33:52.678570+0000","last_became_peered":"2024-04-16T05:33:52.678570+0000","last_unstale":"2024-04-16T05:35:17.140474+0000","last_undegraded":"2024-04-16T05:35:17.140474+0000","last_fullsized":"2024-04-16T05:35:17.140474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:56:32.405491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140508+0000","last_change":"2024-04-16T05:32:59.520394+0000","last_active":"2024-04-16T05:35:17.140508+0000","last_peered":"2024-04-16T05:35:17.140508+0000","last_clean":"2024-04-16T05:35:17.140508+0000","last_became_active":"2024-04-16T05:32:59.519947+0000","last_became_peered":"2024-04-16T05:32:59.519947+0000","last_unstale":"2024-04-16T05:35:17.140508+0000","last_undegraded":"2024-04-16T05:35:17.140508+0000","last_fullsized":"2024-04-16T05:35:17.140508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:33:27.647919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.041043+0000","last_change":"2024-04-16T05:32:54.505350+0000","last_active":"2024-04-16T05:35:17.041043+0000","last_peered":"2024-04-16T05:35:17.041043+0000","last_clean":"2024-04-16T05:35:17.041043+0000","last_became_active":"2024-04-16T05:32:54.505011+0000","last_became_peered":"2024-04-16T05:32:54.505011+0000","last_unstale":"2024-04-16T05:35:17.041043+0000","last_undegraded":"2024-04-16T05:35:17.041043+0000","last_fullsized":"2024-04-16T05:35:17.041043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:03:00.276862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139699+0000","last_change":"2024-04-16T05:33:52.673799+0000","last_active":"2024-04-16T05:35:17.139699+0000","last_peered":"2024-04-16T05:35:17.139699+0000","last_clean":"2024-04-16T05:35:17.139699+0000","last_became_active":"2024-04-16T05:33:52.673344+0000","last_became_peered":"2024-04-16T05:33:52.673344+0000","last_unstale":"2024-04-16T05:35:17.139699+0000","last_undegraded":"2024-04-16T05:35:17.139699+0000","last_fullsized":"2024-04-16T05:35:17.139699+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:07.387912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.031930+0000","last_change":"2024-04-16T05:32:59.519660+0000","last_active":"2024-04-16T05:35:17.031930+0000","last_peered":"2024-04-16T05:35:17.031930+0000","last_clean":"2024-04-16T05:35:17.031930+0000","last_became_active":"2024-04-16T05:32:59.518198+0000","last_became_peered":"2024-04-16T05:32:59.518198+0000","last_unstale":"2024-04-16T05:35:17.031930+0000","last_undegraded":"2024-04-16T05:35:17.031930+0000","last_fullsized":"2024-04-16T05:35:17.031930+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:03:10.096252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'40","reported_seq":325,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787683+0000","last_change":"2024-04-16T05:32:54.501530+0000","last_active":"2024-04-16T05:35:17.787683+0000","last_peered":"2024-04-16T05:35:17.787683+0000","last_clean":"2024-04-16T05:35:17.787683+0000","last_became_active":"2024-04-16T05:32:54.501159+0000","last_became_peered":"2024-04-16T05:32:54.501159+0000","last_unstale":"2024-04-16T05:35:17.787683+0000","last_undegraded":"2024-04-16T05:35:17.787683+0000","last_fullsized":"2024-04-16T05:35:17.787683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:26:22.240137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66452,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":190,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935749+0000","last_change":"2024-04-16T05:32:55.512210+0000","last_active":"2024-04-16T05:35:16.935749+0000","last_peered":"2024-04-16T05:35:16.935749+0000","last_clean":"2024-04-16T05:35:16.935749+0000","last_became_active":"2024-04-16T05:32:55.511672+0000","last_became_peered":"2024-04-16T05:32:55.511672+0000","last_unstale":"2024-04-16T05:35:16.935749+0000","last_undegraded":"2024-04-16T05:35:16.935749+0000","last_fullsized":"2024-04-16T05:35:16.935749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:10:53.755803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.041267+0000","last_change":"2024-04-16T05:32:59.527552+0000","last_active":"2024-04-16T05:35:17.041267+0000","last_peered":"2024-04-16T05:35:17.041267+0000","last_clean":"2024-04-16T05:35:17.041267+0000","last_became_active":"2024-04-16T05:32:59.526919+0000","last_became_peered":"2024-04-16T05:32:59.526919+0000","last_unstale":"2024-04-16T05:35:17.041267+0000","last_undegraded":"2024-04-16T05:35:17.041267+0000","last_fullsized":"2024-04-16T05:35:17.041267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:00:46.342383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139675+0000","last_change":"2024-04-16T05:32:54.494589+0000","last_active":"2024-04-16T05:35:17.139675+0000","last_peered":"2024-04-16T05:35:17.139675+0000","last_clean":"2024-04-16T05:35:17.139675+0000","last_became_active":"2024-04-16T05:32:54.494221+0000","last_became_peered":"2024-04-16T05:32:54.494221+0000","last_unstale":"2024-04-16T05:35:17.139675+0000","last_undegraded":"2024-04-16T05:35:17.139675+0000","last_fullsized":"2024-04-16T05:35:17.139675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:17:53.245359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140562+0000","last_change":"2024-04-16T05:32:55.502724+0000","last_active":"2024-04-16T05:35:17.140562+0000","last_peered":"2024-04-16T05:35:17.140562+0000","last_clean":"2024-04-16T05:35:17.140562+0000","last_became_active":"2024-04-16T05:32:55.502415+0000","last_became_peered":"2024-04-16T05:32:55.502415+0000","last_unstale":"2024-04-16T05:35:17.140562+0000","last_undegraded":"2024-04-16T05:35:17.140562+0000","last_fullsized":"2024-04-16T05:35:17.140562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:20:44.443926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140552+0000","last_change":"2024-04-16T05:32:59.522243+0000","last_active":"2024-04-16T05:35:17.140552+0000","last_peered":"2024-04-16T05:35:17.140552+0000","last_clean":"2024-04-16T05:35:17.140552+0000","last_became_active":"2024-04-16T05:32:59.521823+0000","last_became_peered":"2024-04-16T05:32:59.521823+0000","last_unstale":"2024-04-16T05:35:17.140552+0000","last_undegraded":"2024-04-16T05:35:17.140552+0000","last_fullsized":"2024-04-16T05:35:17.140552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:09:11.158434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040872+0000","last_change":"2024-04-16T05:32:54.507505+0000","last_active":"2024-04-16T05:35:17.040872+0000","last_peered":"2024-04-16T05:35:17.040872+0000","last_clean":"2024-04-16T05:35:17.040872+0000","last_became_active":"2024-04-16T05:32:54.506558+0000","last_became_peered":"2024-04-16T05:32:54.506558+0000","last_unstale":"2024-04-16T05:35:17.040872+0000","last_undegraded":"2024-04-16T05:35:17.040872+0000","last_fullsized":"2024-04-16T05:35:17.040872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:36:17.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140421+0000","last_change":"2024-04-16T05:32:55.518998+0000","last_active":"2024-04-16T05:35:17.140421+0000","last_peered":"2024-04-16T05:35:17.140421+0000","last_clean":"2024-04-16T05:35:17.140421+0000","last_became_active":"2024-04-16T05:32:55.518640+0000","last_became_peered":"2024-04-16T05:32:55.518640+0000","last_unstale":"2024-04-16T05:35:17.140421+0000","last_undegraded":"2024-04-16T05:35:17.140421+0000","last_fullsized":"2024-04-16T05:35:17.140421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:42:44.667504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032258+0000","last_change":"2024-04-16T05:32:59.519050+0000","last_active":"2024-04-16T05:35:17.032258+0000","last_peered":"2024-04-16T05:35:17.032258+0000","last_clean":"2024-04-16T05:35:17.032258+0000","last_became_active":"2024-04-16T05:32:59.518663+0000","last_became_peered":"2024-04-16T05:32:59.518663+0000","last_unstale":"2024-04-16T05:35:17.032258+0000","last_undegraded":"2024-04-16T05:35:17.032258+0000","last_fullsized":"2024-04-16T05:35:17.032258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:36:19.649605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787652+0000","last_change":"2024-04-16T05:32:54.500940+0000","last_active":"2024-04-16T05:35:17.787652+0000","last_peered":"2024-04-16T05:35:17.787652+0000","last_clean":"2024-04-16T05:35:17.787652+0000","last_became_active":"2024-04-16T05:32:54.500542+0000","last_became_peered":"2024-04-16T05:32:54.500542+0000","last_unstale":"2024-04-16T05:35:17.787652+0000","last_undegraded":"2024-04-16T05:35:17.787652+0000","last_fullsized":"2024-04-16T05:35:17.787652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:38:00.232244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139964+0000","last_change":"2024-04-16T05:32:55.509889+0000","last_active":"2024-04-16T05:35:17.139964+0000","last_peered":"2024-04-16T05:35:17.139964+0000","last_clean":"2024-04-16T05:35:17.139964+0000","last_became_active":"2024-04-16T05:32:55.508911+0000","last_became_peered":"2024-04-16T05:32:55.508911+0000","last_unstale":"2024-04-16T05:35:17.139964+0000","last_undegraded":"2024-04-16T05:35:17.139964+0000","last_fullsized":"2024-04-16T05:35:17.139964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:14:46.609127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141707+0000","last_change":"2024-04-16T05:32:59.523555+0000","last_active":"2024-04-16T05:35:17.141707+0000","last_peered":"2024-04-16T05:35:17.141707+0000","last_clean":"2024-04-16T05:35:17.141707+0000","last_became_active":"2024-04-16T05:32:59.523187+0000","last_became_peered":"2024-04-16T05:32:59.523187+0000","last_unstale":"2024-04-16T05:35:17.141707+0000","last_undegraded":"2024-04-16T05:35:17.141707+0000","last_fullsized":"2024-04-16T05:35:17.141707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:35.049355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935450+0000","last_change":"2024-04-16T05:32:54.503247+0000","last_active":"2024-04-16T05:35:16.935450+0000","last_peered":"2024-04-16T05:35:16.935450+0000","last_clean":"2024-04-16T05:35:16.935450+0000","last_became_active":"2024-04-16T05:32:54.502927+0000","last_became_peered":"2024-04-16T05:32:54.502927+0000","last_unstale":"2024-04-16T05:35:16.935450+0000","last_undegraded":"2024-04-16T05:35:16.935450+0000","last_fullsized":"2024-04-16T05:35:16.935450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:36:17.585827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040774+0000","last_change":"2024-04-16T05:32:55.519265+0000","last_active":"2024-04-16T05:35:17.040774+0000","last_peered":"2024-04-16T05:35:17.040774+0000","last_clean":"2024-04-16T05:35:17.040774+0000","last_became_active":"2024-04-16T05:32:55.518840+0000","last_became_peered":"2024-04-16T05:32:55.518840+0000","last_unstale":"2024-04-16T05:35:17.040774+0000","last_undegraded":"2024-04-16T05:35:17.040774+0000","last_fullsized":"2024-04-16T05:35:17.040774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:56.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040636+0000","last_change":"2024-04-16T05:32:59.525275+0000","last_active":"2024-04-16T05:35:17.040636+0000","last_peered":"2024-04-16T05:35:17.040636+0000","last_clean":"2024-04-16T05:35:17.040636+0000","last_became_active":"2024-04-16T05:32:59.524546+0000","last_became_peered":"2024-04-16T05:32:59.524546+0000","last_unstale":"2024-04-16T05:35:17.040636+0000","last_undegraded":"2024-04-16T05:35:17.040636+0000","last_fullsized":"2024-04-16T05:35:17.040636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:07.561898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140459+0000","last_change":"2024-04-16T05:34:52.635664+0000","last_active":"2024-04-16T05:35:17.140459+0000","last_peered":"2024-04-16T05:35:17.140459+0000","last_clean":"2024-04-16T05:35:17.140459+0000","last_became_active":"2024-04-16T05:34:52.627471+0000","last_became_peered":"2024-04-16T05:34:52.627471+0000","last_unstale":"2024-04-16T05:35:17.140459+0000","last_undegraded":"2024-04-16T05:35:17.140459+0000","last_fullsized":"2024-04-16T05:35:17.140459+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:48:28.009122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140884+0000","last_change":"2024-04-16T05:32:55.515685+0000","last_active":"2024-04-16T05:35:17.140884+0000","last_peered":"2024-04-16T05:35:17.140884+0000","last_clean":"2024-04-16T05:35:17.140884+0000","last_became_active":"2024-04-16T05:32:55.515367+0000","last_became_peered":"2024-04-16T05:32:55.515367+0000","last_unstale":"2024-04-16T05:35:17.140884+0000","last_undegraded":"2024-04-16T05:35:17.140884+0000","last_fullsized":"2024-04-16T05:35:17.140884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:46.670143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935508+0000","last_change":"2024-04-16T05:33:00.419828+0000","last_active":"2024-04-16T05:35:16.935508+0000","last_peered":"2024-04-16T05:35:16.935508+0000","last_clean":"2024-04-16T05:35:16.935508+0000","last_became_active":"2024-04-16T05:33:00.419321+0000","last_became_peered":"2024-04-16T05:33:00.419321+0000","last_unstale":"2024-04-16T05:35:16.935508+0000","last_undegraded":"2024-04-16T05:35:16.935508+0000","last_fullsized":"2024-04-16T05:35:16.935508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:56.694914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032165+0000","last_change":"2024-04-16T05:32:54.505369+0000","last_active":"2024-04-16T05:35:17.032165+0000","last_peered":"2024-04-16T05:35:17.032165+0000","last_clean":"2024-04-16T05:35:17.032165+0000","last_became_active":"2024-04-16T05:32:54.505043+0000","last_became_peered":"2024-04-16T05:32:54.505043+0000","last_unstale":"2024-04-16T05:35:17.032165+0000","last_undegraded":"2024-04-16T05:35:17.032165+0000","last_fullsized":"2024-04-16T05:35:17.032165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:34.242117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935671+0000","last_change":"2024-04-16T05:32:55.509970+0000","last_active":"2024-04-16T05:35:16.935671+0000","last_peered":"2024-04-16T05:35:16.935671+0000","last_clean":"2024-04-16T05:35:16.935671+0000","last_became_active":"2024-04-16T05:32:55.509632+0000","last_became_peered":"2024-04-16T05:32:55.509632+0000","last_unstale":"2024-04-16T05:35:16.935671+0000","last_undegraded":"2024-04-16T05:35:16.935671+0000","last_fullsized":"2024-04-16T05:35:16.935671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:33:55.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141399+0000","last_change":"2024-04-16T05:32:59.527007+0000","last_active":"2024-04-16T05:35:17.141399+0000","last_peered":"2024-04-16T05:35:17.141399+0000","last_clean":"2024-04-16T05:35:17.141399+0000","last_became_active":"2024-04-16T05:32:59.526645+0000","last_became_peered":"2024-04-16T05:32:59.526645+0000","last_unstale":"2024-04-16T05:35:17.141399+0000","last_undegraded":"2024-04-16T05:35:17.141399+0000","last_fullsized":"2024-04-16T05:35:17.141399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:46:06.698912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":161,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139866+0000","last_change":"2024-04-16T05:32:54.503409+0000","last_active":"2024-04-16T05:35:17.139866+0000","last_peered":"2024-04-16T05:35:17.139866+0000","last_clean":"2024-04-16T05:35:17.139866+0000","last_became_active":"2024-04-16T05:32:54.503069+0000","last_became_peered":"2024-04-16T05:32:54.503069+0000","last_unstale":"2024-04-16T05:35:17.139866+0000","last_undegraded":"2024-04-16T05:35:17.139866+0000","last_fullsized":"2024-04-16T05:35:17.139866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:45:10.535225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935458+0000","last_change":"2024-04-16T05:32:55.519744+0000","last_active":"2024-04-16T05:35:16.935458+0000","last_peered":"2024-04-16T05:35:16.935458+0000","last_clean":"2024-04-16T05:35:16.935458+0000","last_became_active":"2024-04-16T05:32:55.519091+0000","last_became_peered":"2024-04-16T05:32:55.519091+0000","last_unstale":"2024-04-16T05:35:16.935458+0000","last_undegraded":"2024-04-16T05:35:16.935458+0000","last_fullsized":"2024-04-16T05:35:16.935458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:19.417691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141700+0000","last_change":"2024-04-16T05:33:00.419494+0000","last_active":"2024-04-16T05:35:17.141700+0000","last_peered":"2024-04-16T05:35:17.141700+0000","last_clean":"2024-04-16T05:35:17.141700+0000","last_became_active":"2024-04-16T05:33:00.418954+0000","last_became_peered":"2024-04-16T05:33:00.418954+0000","last_unstale":"2024-04-16T05:35:17.141700+0000","last_undegraded":"2024-04-16T05:35:17.141700+0000","last_fullsized":"2024-04-16T05:35:17.141700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:03:01.835304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040794+0000","last_change":"2024-04-16T05:32:55.518272+0000","last_active":"2024-04-16T05:35:17.040794+0000","last_peered":"2024-04-16T05:35:17.040794+0000","last_clean":"2024-04-16T05:35:17.040794+0000","last_became_active":"2024-04-16T05:32:55.517928+0000","last_became_peered":"2024-04-16T05:32:55.517928+0000","last_unstale":"2024-04-16T05:35:17.040794+0000","last_undegraded":"2024-04-16T05:35:17.040794+0000","last_fullsized":"2024-04-16T05:35:17.040794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:44:47.739994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":164,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.178665+0000","last_change":"2024-04-16T05:32:54.503923+0000","last_active":"2024-04-16T05:35:17.178665+0000","last_peered":"2024-04-16T05:35:17.178665+0000","last_clean":"2024-04-16T05:35:17.178665+0000","last_became_active":"2024-04-16T05:32:54.503576+0000","last_became_peered":"2024-04-16T05:32:54.503576+0000","last_unstale":"2024-04-16T05:35:17.178665+0000","last_undegraded":"2024-04-16T05:35:17.178665+0000","last_fullsized":"2024-04-16T05:35:17.178665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:49:36.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140426+0000","last_change":"2024-04-16T05:32:59.517757+0000","last_active":"2024-04-16T05:35:17.140426+0000","last_peered":"2024-04-16T05:35:17.140426+0000","last_clean":"2024-04-16T05:35:17.140426+0000","last_became_active":"2024-04-16T05:32:59.517357+0000","last_became_peered":"2024-04-16T05:32:59.517357+0000","last_unstale":"2024-04-16T05:35:17.140426+0000","last_undegraded":"2024-04-16T05:35:17.140426+0000","last_fullsized":"2024-04-16T05:35:17.140426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:21.767071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934854+0000","last_change":"2024-04-16T05:32:55.514617+0000","last_active":"2024-04-16T05:35:16.934854+0000","last_peered":"2024-04-16T05:35:16.934854+0000","last_clean":"2024-04-16T05:35:16.934854+0000","last_became_active":"2024-04-16T05:32:55.512660+0000","last_became_peered":"2024-04-16T05:32:55.512660+0000","last_unstale":"2024-04-16T05:35:16.934854+0000","last_undegraded":"2024-04-16T05:35:16.934854+0000","last_fullsized":"2024-04-16T05:35:16.934854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:31:45.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140378+0000","last_change":"2024-04-16T05:32:54.498650+0000","last_active":"2024-04-16T05:35:17.140378+0000","last_peered":"2024-04-16T05:35:17.140378+0000","last_clean":"2024-04-16T05:35:17.140378+0000","last_became_active":"2024-04-16T05:32:54.497672+0000","last_became_peered":"2024-04-16T05:32:54.497672+0000","last_unstale":"2024-04-16T05:35:17.140378+0000","last_undegraded":"2024-04-16T05:35:17.140378+0000","last_fullsized":"2024-04-16T05:35:17.140378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:36.908809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":369,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139698+0000","last_change":"2024-04-16T05:31:42.831800+0000","last_active":"2024-04-16T05:35:17.139698+0000","last_peered":"2024-04-16T05:35:17.139698+0000","last_clean":"2024-04-16T05:35:17.139698+0000","last_became_active":"2024-04-16T05:31:42.831391+0000","last_became_peered":"2024-04-16T05:31:42.831391+0000","last_unstale":"2024-04-16T05:35:17.139698+0000","last_undegraded":"2024-04-16T05:35:17.139698+0000","last_fullsized":"2024-04-16T05:35:17.139698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_clean_scrub_stamp":"2024-04-16T05:28:15.969690+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.152803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935901+0000","last_change":"2024-04-16T05:33:52.675549+0000","last_active":"2024-04-16T05:35:16.935901+0000","last_peered":"2024-04-16T05:35:16.935901+0000","last_clean":"2024-04-16T05:35:16.935901+0000","last_became_active":"2024-04-16T05:33:52.674969+0000","last_became_peered":"2024-04-16T05:33:52.674969+0000","last_unstale":"2024-04-16T05:35:16.935901+0000","last_undegraded":"2024-04-16T05:35:16.935901+0000","last_fullsized":"2024-04-16T05:35:16.935901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:58:33.137841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139786+0000","last_change":"2024-04-16T05:32:55.512324+0000","last_active":"2024-04-16T05:35:17.139786+0000","last_peered":"2024-04-16T05:35:17.139786+0000","last_clean":"2024-04-16T05:35:17.139786+0000","last_became_active":"2024-04-16T05:32:55.511958+0000","last_became_peered":"2024-04-16T05:32:55.511958+0000","last_unstale":"2024-04-16T05:35:17.139786+0000","last_undegraded":"2024-04-16T05:35:17.139786+0000","last_fullsized":"2024-04-16T05:35:17.139786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:38.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139804+0000","last_change":"2024-04-16T05:32:54.508021+0000","last_active":"2024-04-16T05:35:17.139804+0000","last_peered":"2024-04-16T05:35:17.139804+0000","last_clean":"2024-04-16T05:35:17.139804+0000","last_became_active":"2024-04-16T05:32:54.507645+0000","last_became_peered":"2024-04-16T05:32:54.507645+0000","last_unstale":"2024-04-16T05:35:17.139804+0000","last_undegraded":"2024-04-16T05:35:17.139804+0000","last_fullsized":"2024-04-16T05:35:17.139804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:21:50.721260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140448+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:35:17.140448+0000","last_peered":"2024-04-16T05:35:17.140448+0000","last_clean":"2024-04-16T05:35:17.140448+0000","last_became_active":"2024-04-16T05:32:59.523159+0000","last_became_peered":"2024-04-16T05:32:59.523159+0000","last_unstale":"2024-04-16T05:35:17.140448+0000","last_undegraded":"2024-04-16T05:35:17.140448+0000","last_fullsized":"2024-04-16T05:35:17.140448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:48:37.696713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935875+0000","last_change":"2024-04-16T05:32:54.503262+0000","last_active":"2024-04-16T05:35:16.935875+0000","last_peered":"2024-04-16T05:35:16.935875+0000","last_clean":"2024-04-16T05:35:16.935875+0000","last_became_active":"2024-04-16T05:32:54.502909+0000","last_became_peered":"2024-04-16T05:32:54.502909+0000","last_unstale":"2024-04-16T05:35:16.935875+0000","last_undegraded":"2024-04-16T05:35:16.935875+0000","last_fullsized":"2024-04-16T05:35:16.935875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:48:32.841921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141260+0000","last_change":"2024-04-16T05:32:55.507126+0000","last_active":"2024-04-16T05:35:17.141260+0000","last_peered":"2024-04-16T05:35:17.141260+0000","last_clean":"2024-04-16T05:35:17.141260+0000","last_became_active":"2024-04-16T05:32:55.506743+0000","last_became_peered":"2024-04-16T05:32:55.506743+0000","last_unstale":"2024-04-16T05:35:17.141260+0000","last_undegraded":"2024-04-16T05:35:17.141260+0000","last_fullsized":"2024-04-16T05:35:17.141260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:10:58.697649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934575+0000","last_change":"2024-04-16T05:33:00.419435+0000","last_active":"2024-04-16T05:35:16.934575+0000","last_peered":"2024-04-16T05:35:16.934575+0000","last_clean":"2024-04-16T05:35:16.934575+0000","last_became_active":"2024-04-16T05:33:00.418985+0000","last_became_peered":"2024-04-16T05:33:00.418985+0000","last_unstale":"2024-04-16T05:35:16.934575+0000","last_undegraded":"2024-04-16T05:35:16.934575+0000","last_fullsized":"2024-04-16T05:35:16.934575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:08:31.188911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'48","reported_seq":107,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.144790+0000","last_change":"2024-04-16T05:32:54.503349+0000","last_active":"2024-04-16T05:35:17.144790+0000","last_peered":"2024-04-16T05:35:17.144790+0000","last_clean":"2024-04-16T05:35:17.144790+0000","last_became_active":"2024-04-16T05:32:54.503018+0000","last_became_peered":"2024-04-16T05:32:54.503018+0000","last_unstale":"2024-04-16T05:35:17.144790+0000","last_undegraded":"2024-04-16T05:35:17.144790+0000","last_fullsized":"2024-04-16T05:35:17.144790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:34:20.853543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935262+0000","last_change":"2024-04-16T05:32:55.511201+0000","last_active":"2024-04-16T05:35:16.935262+0000","last_peered":"2024-04-16T05:35:16.935262+0000","last_clean":"2024-04-16T05:35:16.935262+0000","last_became_active":"2024-04-16T05:32:55.510906+0000","last_became_peered":"2024-04-16T05:32:55.510906+0000","last_unstale":"2024-04-16T05:35:16.935262+0000","last_undegraded":"2024-04-16T05:35:16.935262+0000","last_fullsized":"2024-04-16T05:35:16.935262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:37:37.147628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139081+0000","last_change":"2024-04-16T05:32:59.518440+0000","last_active":"2024-04-16T05:35:17.139081+0000","last_peered":"2024-04-16T05:35:17.139081+0000","last_clean":"2024-04-16T05:35:17.139081+0000","last_became_active":"2024-04-16T05:32:59.517870+0000","last_became_peered":"2024-04-16T05:32:59.517870+0000","last_unstale":"2024-04-16T05:35:17.139081+0000","last_undegraded":"2024-04-16T05:35:17.139081+0000","last_fullsized":"2024-04-16T05:35:17.139081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:51:31.157418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140365+0000","last_change":"2024-04-16T05:32:54.503983+0000","last_active":"2024-04-16T05:35:17.140365+0000","last_peered":"2024-04-16T05:35:17.140365+0000","last_clean":"2024-04-16T05:35:17.140365+0000","last_became_active":"2024-04-16T05:32:54.502367+0000","last_became_peered":"2024-04-16T05:32:54.502367+0000","last_unstale":"2024-04-16T05:35:17.140365+0000","last_undegraded":"2024-04-16T05:35:17.140365+0000","last_fullsized":"2024-04-16T05:35:17.140365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:49:36.997692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140112+0000","last_change":"2024-04-16T05:32:55.516812+0000","last_active":"2024-04-16T05:35:17.140112+0000","last_peered":"2024-04-16T05:35:17.140112+0000","last_clean":"2024-04-16T05:35:17.140112+0000","last_became_active":"2024-04-16T05:32:55.516477+0000","last_became_peered":"2024-04-16T05:32:55.516477+0000","last_unstale":"2024-04-16T05:35:17.140112+0000","last_undegraded":"2024-04-16T05:35:17.140112+0000","last_fullsized":"2024-04-16T05:35:17.140112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:24:29.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140029+0000","last_change":"2024-04-16T05:33:00.419622+0000","last_active":"2024-04-16T05:35:17.140029+0000","last_peered":"2024-04-16T05:35:17.140029+0000","last_clean":"2024-04-16T05:35:17.140029+0000","last_became_active":"2024-04-16T05:33:00.419135+0000","last_became_peered":"2024-04-16T05:33:00.419135+0000","last_unstale":"2024-04-16T05:35:17.140029+0000","last_undegraded":"2024-04-16T05:35:17.140029+0000","last_fullsized":"2024-04-16T05:35:17.140029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:38:02.529192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.138987+0000","last_change":"2024-04-16T05:32:54.503652+0000","last_active":"2024-04-16T05:35:17.138987+0000","last_peered":"2024-04-16T05:35:17.138987+0000","last_clean":"2024-04-16T05:35:17.138987+0000","last_became_active":"2024-04-16T05:32:54.503311+0000","last_became_peered":"2024-04-16T05:32:54.503311+0000","last_unstale":"2024-04-16T05:35:17.138987+0000","last_undegraded":"2024-04-16T05:35:17.138987+0000","last_fullsized":"2024-04-16T05:35:17.138987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:49:21.584029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935388+0000","last_change":"2024-04-16T05:32:55.508145+0000","last_active":"2024-04-16T05:35:16.935388+0000","last_peered":"2024-04-16T05:35:16.935388+0000","last_clean":"2024-04-16T05:35:16.935388+0000","last_became_active":"2024-04-16T05:32:55.507794+0000","last_became_peered":"2024-04-16T05:32:55.507794+0000","last_unstale":"2024-04-16T05:35:16.935388+0000","last_undegraded":"2024-04-16T05:35:16.935388+0000","last_fullsized":"2024-04-16T05:35:16.935388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:11:57.925733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.936185+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:35:16.936185+0000","last_peered":"2024-04-16T05:35:16.936185+0000","last_clean":"2024-04-16T05:35:16.936185+0000","last_became_active":"2024-04-16T05:33:52.678330+0000","last_became_peered":"2024-04-16T05:33:52.678330+0000","last_unstale":"2024-04-16T05:35:16.936185+0000","last_undegraded":"2024-04-16T05:35:16.936185+0000","last_fullsized":"2024-04-16T05:35:16.936185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:33.323513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934764+0000","last_change":"2024-04-16T05:32:54.500909+0000","last_active":"2024-04-16T05:35:16.934764+0000","last_peered":"2024-04-16T05:35:16.934764+0000","last_clean":"2024-04-16T05:35:16.934764+0000","last_became_active":"2024-04-16T05:32:54.499348+0000","last_became_peered":"2024-04-16T05:32:54.499348+0000","last_unstale":"2024-04-16T05:35:16.934764+0000","last_undegraded":"2024-04-16T05:35:16.934764+0000","last_fullsized":"2024-04-16T05:35:16.934764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:51:49.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140151+0000","last_change":"2024-04-16T05:32:55.510922+0000","last_active":"2024-04-16T05:35:17.140151+0000","last_peered":"2024-04-16T05:35:17.140151+0000","last_clean":"2024-04-16T05:35:17.140151+0000","last_became_active":"2024-04-16T05:32:55.510546+0000","last_became_peered":"2024-04-16T05:32:55.510546+0000","last_unstale":"2024-04-16T05:35:17.140151+0000","last_undegraded":"2024-04-16T05:35:17.140151+0000","last_fullsized":"2024-04-16T05:35:17.140151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:27.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139878+0000","last_change":"2024-04-16T05:32:59.524257+0000","last_active":"2024-04-16T05:35:17.139878+0000","last_peered":"2024-04-16T05:35:17.139878+0000","last_clean":"2024-04-16T05:35:17.139878+0000","last_became_active":"2024-04-16T05:32:59.523768+0000","last_became_peered":"2024-04-16T05:32:59.523768+0000","last_unstale":"2024-04-16T05:35:17.139878+0000","last_undegraded":"2024-04-16T05:35:17.139878+0000","last_fullsized":"2024-04-16T05:35:17.139878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:38:27.594048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'5","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140663+0000","last_change":"2024-04-16T05:34:52.631768+0000","last_active":"2024-04-16T05:35:17.140663+0000","last_peered":"2024-04-16T05:35:17.140663+0000","last_clean":"2024-04-16T05:35:17.140663+0000","last_became_active":"2024-04-16T05:34:52.627168+0000","last_became_peered":"2024-04-16T05:34:52.627168+0000","last_unstale":"2024-04-16T05:35:17.140663+0000","last_undegraded":"2024-04-16T05:35:17.140663+0000","last_fullsized":"2024-04-16T05:35:17.140663+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:46.866673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935117+0000","last_change":"2024-04-16T05:32:55.508005+0000","last_active":"2024-04-16T05:35:16.935117+0000","last_peered":"2024-04-16T05:35:16.935117+0000","last_clean":"2024-04-16T05:35:16.935117+0000","last_became_active":"2024-04-16T05:32:55.507698+0000","last_became_peered":"2024-04-16T05:32:55.507698+0000","last_unstale":"2024-04-16T05:35:16.935117+0000","last_undegraded":"2024-04-16T05:35:16.935117+0000","last_fullsized":"2024-04-16T05:35:16.935117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:11:12.423294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140202+0000","last_change":"2024-04-16T05:33:00.419442+0000","last_active":"2024-04-16T05:35:17.140202+0000","last_peered":"2024-04-16T05:35:17.140202+0000","last_clean":"2024-04-16T05:35:17.140202+0000","last_became_active":"2024-04-16T05:33:00.418906+0000","last_became_peered":"2024-04-16T05:33:00.418906+0000","last_unstale":"2024-04-16T05:35:17.140202+0000","last_undegraded":"2024-04-16T05:35:17.140202+0000","last_fullsized":"2024-04-16T05:35:17.140202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:21:22.124879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935760+0000","last_change":"2024-04-16T05:32:54.509090+0000","last_active":"2024-04-16T05:35:16.935760+0000","last_peered":"2024-04-16T05:35:16.935760+0000","last_clean":"2024-04-16T05:35:16.935760+0000","last_became_active":"2024-04-16T05:32:54.508700+0000","last_became_peered":"2024-04-16T05:32:54.508700+0000","last_unstale":"2024-04-16T05:35:16.935760+0000","last_undegraded":"2024-04-16T05:35:16.935760+0000","last_fullsized":"2024-04-16T05:35:16.935760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:53:34.344073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139999+0000","last_change":"2024-04-16T05:32:55.509658+0000","last_active":"2024-04-16T05:35:17.139999+0000","last_peered":"2024-04-16T05:35:17.139999+0000","last_clean":"2024-04-16T05:35:17.139999+0000","last_became_active":"2024-04-16T05:32:55.509258+0000","last_became_peered":"2024-04-16T05:32:55.509258+0000","last_unstale":"2024-04-16T05:35:17.139999+0000","last_undegraded":"2024-04-16T05:35:17.139999+0000","last_fullsized":"2024-04-16T05:35:17.139999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:17:03.490544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139041+0000","last_change":"2024-04-16T05:32:59.519260+0000","last_active":"2024-04-16T05:35:17.139041+0000","last_peered":"2024-04-16T05:35:17.139041+0000","last_clean":"2024-04-16T05:35:17.139041+0000","last_became_active":"2024-04-16T05:32:59.518839+0000","last_became_peered":"2024-04-16T05:32:59.518839+0000","last_unstale":"2024-04-16T05:35:17.139041+0000","last_undegraded":"2024-04-16T05:35:17.139041+0000","last_fullsized":"2024-04-16T05:35:17.139041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:54:50.555214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787320+0000","last_change":"2024-04-16T05:32:54.503226+0000","last_active":"2024-04-16T05:35:17.787320+0000","last_peered":"2024-04-16T05:35:17.787320+0000","last_clean":"2024-04-16T05:35:17.787320+0000","last_became_active":"2024-04-16T05:32:54.502687+0000","last_became_peered":"2024-04-16T05:32:54.502687+0000","last_unstale":"2024-04-16T05:35:17.787320+0000","last_undegraded":"2024-04-16T05:35:17.787320+0000","last_fullsized":"2024-04-16T05:35:17.787320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:34:24.228464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140830+0000","last_change":"2024-04-16T05:32:55.515203+0000","last_active":"2024-04-16T05:35:17.140830+0000","last_peered":"2024-04-16T05:35:17.140830+0000","last_clean":"2024-04-16T05:35:17.140830+0000","last_became_active":"2024-04-16T05:32:55.514815+0000","last_became_peered":"2024-04-16T05:32:55.514815+0000","last_unstale":"2024-04-16T05:35:17.140830+0000","last_undegraded":"2024-04-16T05:35:17.140830+0000","last_fullsized":"2024-04-16T05:35:17.140830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:11:18.464532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139998+0000","last_change":"2024-04-16T05:32:59.520442+0000","last_active":"2024-04-16T05:35:17.139998+0000","last_peered":"2024-04-16T05:35:17.139998+0000","last_clean":"2024-04-16T05:35:17.139998+0000","last_became_active":"2024-04-16T05:32:59.520071+0000","last_became_peered":"2024-04-16T05:32:59.520071+0000","last_unstale":"2024-04-16T05:35:17.139998+0000","last_undegraded":"2024-04-16T05:35:17.139998+0000","last_fullsized":"2024-04-16T05:35:17.139998+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:19:16.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":84,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.142330+0000","last_change":"2024-04-16T05:32:54.510098+0000","last_active":"2024-04-16T05:35:17.142330+0000","last_peered":"2024-04-16T05:35:17.142330+0000","last_clean":"2024-04-16T05:35:17.142330+0000","last_became_active":"2024-04-16T05:32:54.509750+0000","last_became_peered":"2024-04-16T05:32:54.509750+0000","last_unstale":"2024-04-16T05:35:17.142330+0000","last_undegraded":"2024-04-16T05:35:17.142330+0000","last_fullsized":"2024-04-16T05:35:17.142330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:00:09.528958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140257+0000","last_change":"2024-04-16T05:33:52.678823+0000","last_active":"2024-04-16T05:35:17.140257+0000","last_peered":"2024-04-16T05:35:17.140257+0000","last_clean":"2024-04-16T05:35:17.140257+0000","last_became_active":"2024-04-16T05:33:52.678276+0000","last_became_peered":"2024-04-16T05:33:52.678276+0000","last_unstale":"2024-04-16T05:35:17.140257+0000","last_undegraded":"2024-04-16T05:35:17.140257+0000","last_fullsized":"2024-04-16T05:35:17.140257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:58:33.186382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040472+0000","last_change":"2024-04-16T05:32:59.521334+0000","last_active":"2024-04-16T05:35:17.040472+0000","last_peered":"2024-04-16T05:35:17.040472+0000","last_clean":"2024-04-16T05:35:17.040472+0000","last_became_active":"2024-04-16T05:32:59.520348+0000","last_became_peered":"2024-04-16T05:32:59.520348+0000","last_unstale":"2024-04-16T05:35:17.040472+0000","last_undegraded":"2024-04-16T05:35:17.040472+0000","last_fullsized":"2024-04-16T05:35:17.040472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:40.628552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141843+0000","last_change":"2024-04-16T05:32:54.503062+0000","last_active":"2024-04-16T05:35:17.141843+0000","last_peered":"2024-04-16T05:35:17.141843+0000","last_clean":"2024-04-16T05:35:17.141843+0000","last_became_active":"2024-04-16T05:32:54.502762+0000","last_became_peered":"2024-04-16T05:32:54.502762+0000","last_unstale":"2024-04-16T05:35:17.141843+0000","last_undegraded":"2024-04-16T05:35:17.141843+0000","last_fullsized":"2024-04-16T05:35:17.141843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:43:57.911296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141355+0000","last_change":"2024-04-16T05:32:55.510461+0000","last_active":"2024-04-16T05:35:17.141355+0000","last_peered":"2024-04-16T05:35:17.141355+0000","last_clean":"2024-04-16T05:35:17.141355+0000","last_became_active":"2024-04-16T05:32:55.510142+0000","last_became_peered":"2024-04-16T05:32:55.510142+0000","last_unstale":"2024-04-16T05:35:17.141355+0000","last_undegraded":"2024-04-16T05:35:17.141355+0000","last_fullsized":"2024-04-16T05:35:17.141355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:41.346078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140171+0000","last_change":"2024-04-16T05:32:55.512600+0000","last_active":"2024-04-16T05:35:17.140171+0000","last_peered":"2024-04-16T05:35:17.140171+0000","last_clean":"2024-04-16T05:35:17.140171+0000","last_became_active":"2024-04-16T05:32:55.512265+0000","last_became_peered":"2024-04-16T05:32:55.512265+0000","last_unstale":"2024-04-16T05:35:17.140171+0000","last_undegraded":"2024-04-16T05:35:17.140171+0000","last_fullsized":"2024-04-16T05:35:17.140171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:04:46.112644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139670+0000","last_change":"2024-04-16T05:32:55.514481+0000","last_active":"2024-04-16T05:35:17.139670+0000","last_peered":"2024-04-16T05:35:17.139670+0000","last_clean":"2024-04-16T05:35:17.139670+0000","last_became_active":"2024-04-16T05:32:55.512485+0000","last_became_peered":"2024-04-16T05:32:55.512485+0000","last_unstale":"2024-04-16T05:35:17.139670+0000","last_undegraded":"2024-04-16T05:35:17.139670+0000","last_fullsized":"2024-04-16T05:35:17.139670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:33:02.179774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139422+0000","last_change":"2024-04-16T05:32:55.509849+0000","last_active":"2024-04-16T05:35:17.139422+0000","last_peered":"2024-04-16T05:35:17.139422+0000","last_clean":"2024-04-16T05:35:17.139422+0000","last_became_active":"2024-04-16T05:32:55.509338+0000","last_became_peered":"2024-04-16T05:32:55.509338+0000","last_unstale":"2024-04-16T05:35:17.139422+0000","last_undegraded":"2024-04-16T05:35:17.139422+0000","last_fullsized":"2024-04-16T05:35:17.139422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:32:38.900455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140163+0000","last_change":"2024-04-16T05:32:55.518556+0000","last_active":"2024-04-16T05:35:17.140163+0000","last_peered":"2024-04-16T05:35:17.140163+0000","last_clean":"2024-04-16T05:35:17.140163+0000","last_became_active":"2024-04-16T05:32:55.518240+0000","last_became_peered":"2024-04-16T05:32:55.518240+0000","last_unstale":"2024-04-16T05:35:17.140163+0000","last_undegraded":"2024-04-16T05:35:17.140163+0000","last_fullsized":"2024-04-16T05:35:17.140163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:25:46.002516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141244+0000","last_change":"2024-04-16T05:32:55.518106+0000","last_active":"2024-04-16T05:35:17.141244+0000","last_peered":"2024-04-16T05:35:17.141244+0000","last_clean":"2024-04-16T05:35:17.141244+0000","last_became_active":"2024-04-16T05:32:55.517403+0000","last_became_peered":"2024-04-16T05:32:55.517403+0000","last_unstale":"2024-04-16T05:35:17.141244+0000","last_undegraded":"2024-04-16T05:35:17.141244+0000","last_fullsized":"2024-04-16T05:35:17.141244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:36:34.731644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140630+0000","last_change":"2024-04-16T05:33:52.581236+0000","last_active":"2024-04-16T05:35:17.140630+0000","last_peered":"2024-04-16T05:35:17.140630+0000","last_clean":"2024-04-16T05:35:17.140630+0000","last_became_active":"2024-04-16T05:33:52.580748+0000","last_became_peered":"2024-04-16T05:33:52.580748+0000","last_unstale":"2024-04-16T05:35:17.140630+0000","last_undegraded":"2024-04-16T05:35:17.140630+0000","last_fullsized":"2024-04-16T05:35:17.140630+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:13:33.382776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139982+0000","last_change":"2024-04-16T05:32:55.507670+0000","last_active":"2024-04-16T05:35:17.139982+0000","last_peered":"2024-04-16T05:35:17.139982+0000","last_clean":"2024-04-16T05:35:17.139982+0000","last_became_active":"2024-04-16T05:32:55.506672+0000","last_became_peered":"2024-04-16T05:32:55.506672+0000","last_unstale":"2024-04-16T05:35:17.139982+0000","last_undegraded":"2024-04-16T05:35:17.139982+0000","last_fullsized":"2024-04-16T05:35:17.139982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:54.579672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935321+0000","last_change":"2024-04-16T05:32:55.516987+0000","last_active":"2024-04-16T05:35:16.935321+0000","last_peered":"2024-04-16T05:35:16.935321+0000","last_clean":"2024-04-16T05:35:16.935321+0000","last_became_active":"2024-04-16T05:32:55.516654+0000","last_became_peered":"2024-04-16T05:32:55.516654+0000","last_unstale":"2024-04-16T05:35:16.935321+0000","last_undegraded":"2024-04-16T05:35:16.935321+0000","last_fullsized":"2024-04-16T05:35:16.935321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:32:42.957658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040684+0000","last_change":"2024-04-16T05:32:55.506539+0000","last_active":"2024-04-16T05:35:17.040684+0000","last_peered":"2024-04-16T05:35:17.040684+0000","last_clean":"2024-04-16T05:35:17.040684+0000","last_became_active":"2024-04-16T05:32:55.503658+0000","last_became_peered":"2024-04-16T05:32:55.503658+0000","last_unstale":"2024-04-16T05:35:17.040684+0000","last_undegraded":"2024-04-16T05:35:17.040684+0000","last_fullsized":"2024-04-16T05:35:17.040684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:22.116213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934725+0000","last_change":"2024-04-16T05:32:55.507363+0000","last_active":"2024-04-16T05:35:16.934725+0000","last_peered":"2024-04-16T05:35:16.934725+0000","last_clean":"2024-04-16T05:35:16.934725+0000","last_became_active":"2024-04-16T05:32:55.506923+0000","last_became_peered":"2024-04-16T05:32:55.506923+0000","last_unstale":"2024-04-16T05:35:16.934725+0000","last_undegraded":"2024-04-16T05:35:16.934725+0000","last_fullsized":"2024-04-16T05:35:16.934725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:19:34.356834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139209+0000","last_change":"2024-04-16T05:32:55.508531+0000","last_active":"2024-04-16T05:35:17.139209+0000","last_peered":"2024-04-16T05:35:17.139209+0000","last_clean":"2024-04-16T05:35:17.139209+0000","last_became_active":"2024-04-16T05:32:55.508177+0000","last_became_peered":"2024-04-16T05:32:55.508177+0000","last_unstale":"2024-04-16T05:35:17.139209+0000","last_undegraded":"2024-04-16T05:35:17.139209+0000","last_fullsized":"2024-04-16T05:35:17.139209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:49:59.352572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140924+0000","last_change":"2024-04-16T05:32:55.507793+0000","last_active":"2024-04-16T05:35:17.140924+0000","last_peered":"2024-04-16T05:35:17.140924+0000","last_clean":"2024-04-16T05:35:17.140924+0000","last_became_active":"2024-04-16T05:32:55.507443+0000","last_became_peered":"2024-04-16T05:32:55.507443+0000","last_unstale":"2024-04-16T05:35:17.140924+0000","last_undegraded":"2024-04-16T05:35:17.140924+0000","last_fullsized":"2024-04-16T05:35:17.140924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:35.982879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140787+0000","last_change":"2024-04-16T05:32:55.517057+0000","last_active":"2024-04-16T05:35:17.140787+0000","last_peered":"2024-04-16T05:35:17.140787+0000","last_clean":"2024-04-16T05:35:17.140787+0000","last_became_active":"2024-04-16T05:32:55.516696+0000","last_became_peered":"2024-04-16T05:32:55.516696+0000","last_unstale":"2024-04-16T05:35:17.140787+0000","last_undegraded":"2024-04-16T05:35:17.140787+0000","last_fullsized":"2024-04-16T05:35:17.140787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:43:10.391125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106974,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600767,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1336,"num_read_kb":784,"num_write":379,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1802301,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383720,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":540350,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":1188583,"data_compressed":68167,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1105920,"data_stored":936730,"data_compressed":51657,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":932990,"data_compressed":51655,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":1138139,"data_compressed":68167,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":534922,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1126400,"data_stored":602404,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":537218,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":589425,"data_compressed":29531,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":993902,"data_compressed":54666,"data_compressed_allocated":73728,"data_compressed_original":544768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":539919,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1264398,"data_compressed":74323,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135734,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200711,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:35:22.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:35:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 -- 172.21.15.86:0/1078346438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1790103b80 msgr2=0x7f179010c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 --2- 172.21.15.86:0/1078346438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1790103b80 0x7f179010c060 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f178402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 -- 172.21.15.86:0/1078346438 shutdown_connections 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 --2- 172.21.15.86:0/1078346438 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f179010c630 0x7f179010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 --2- 172.21.15.86:0/1078346438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1790103b80 0x7f179010c060 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 --2- 172.21.15.86:0/1078346438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1790103260 0x7f1790103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 -- 172.21.15.86:0/1078346438 >> 172.21.15.86:0/1078346438 conn(0x7f1790077280 msgr2=0x7f1790077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 -- 172.21.15.86:0/1078346438 shutdown_connections 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 -- 172.21.15.86:0/1078346438 wait complete. 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 Processor -- start 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.196+0000 7f1795e93640 1 -- start start 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f1795e93640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1790103260 0x7f179019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f1795e93640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1790103b80 0x7f179019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f1795e93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f179010c630 0x7f17901987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f1795e93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1790112420 con 0x7f179010c630 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f1795e93640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f17901122a0 con 0x7f1790103b80 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f1795e93640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17901125a0 con 0x7f1790103260 2024-04-16T05:35:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f179010c630 0x7f17901987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f179010c630 0x7f17901987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59510/0 (socket says 172.21.15.86:59510) 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178ffff640 1 -- 172.21.15.86:0/2029006571 learned_addr learned my addr 172.21.15.86:0/2029006571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178f7fe640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1790103260 0x7f179019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178effd640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1790103b80 0x7f179019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178ffff640 1 -- 172.21.15.86:0/2029006571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1790103260 msgr2=0x7f179019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178ffff640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1790103260 0x7f179019e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178ffff640 1 -- 172.21.15.86:0/2029006571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1790103b80 msgr2=0x7f179019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178ffff640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1790103b80 0x7f179019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178ffff640 1 -- 172.21.15.86:0/2029006571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1790198fb0 con 0x7f179010c630 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178effd640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1790103b80 0x7f179019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178ffff640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f179010c630 0x7f17901987e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f178800e9d0 tx=0x7f178800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178cff9640 1 -- 172.21.15.86:0/2029006571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178800cd80 con 0x7f179010c630 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178cff9640 1 -- 172.21.15.86:0/2029006571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788005020 con 0x7f179010c630 2024-04-16T05:35:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901992a0 con 0x7f179010c630 2024-04-16T05:35:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f178cff9640 1 -- 172.21.15.86:0/2029006571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880116d0 con 0x7f179010c630 2024-04-16T05:35:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.197+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a54c0 con 0x7f179010c630 2024-04-16T05:35:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.198+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179006ae80 con 0x7f179010c630 2024-04-16T05:35:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.199+0000 7f178cff9640 1 -- 172.21.15.86:0/2029006571 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f17880051d0 con 0x7f179010c630 2024-04-16T05:35:22.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.200+0000 7f178cff9640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f17780a3e40 0x7f17780a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.200+0000 7f178f7fe640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f17780a3e40 0x7f17780a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.200+0000 7f178cff9640 1 -- 172.21.15.86:0/2029006571 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f17880d1480 con 0x7f179010c630 2024-04-16T05:35:22.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.200+0000 7f178f7fe640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f17780a3e40 0x7f17780a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1780009890 tx=0x7f1780008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:22.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.204+0000 7f178cff9640 1 -- 172.21.15.86:0/2029006571 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1788016030 con 0x7f179010c630 2024-04-16T05:35:22.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.379+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1790062440 con 0x7f17780a3e40 2024-04-16T05:35:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.388+0000 7f178cff9640 1 -- 172.21.15.86:0/2029006571 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339713 (secure 0 0 0) 0x7f1790062440 con 0x7f17780a3e40 2024-04-16T05:35:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:22.392 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-16T05:35:22.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.391+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f17780a3e40 msgr2=0x7f17780a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.391+0000 7f1795e93640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f17780a3e40 0x7f17780a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1780009890 tx=0x7f1780008040 comp rx=0 tx=0).stop 2024-04-16T05:35:22.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.391+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f179010c630 msgr2=0x7f17901987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.391+0000 7f1795e93640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f179010c630 0x7f17901987e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f178800e9d0 tx=0x7f178800ee90 comp rx=0 tx=0).stop 2024-04-16T05:35:22.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.392+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 shutdown_connections 2024-04-16T05:35:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.392+0000 7f1795e93640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f179010c630 0x7f17901987e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.392+0000 7f1795e93640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f17780a3e40 0x7f17780a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.392+0000 7f1795e93640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1790103b80 0x7f179019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.392+0000 7f1795e93640 1 --2- 172.21.15.86:0/2029006571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1790103260 0x7f179019e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.392+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 >> 172.21.15.86:0/2029006571 conn(0x7f1790077280 msgr2=0x7f17900ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.392+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 shutdown_connections 2024-04-16T05:35:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.392+0000 7f1795e93640 1 -- 172.21.15.86:0/2029006571 wait complete. 2024-04-16T05:35:22.402 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-16T05:35:20.947179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191369,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1472,"num_read_kb":903,"num_write":562,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370484,"kb_used_data":12972,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571340,"statfs":{"total":1151940427776,"available":1151561052160,"internally_reserved":0,"allocated":13283328,"data_stored":9798980,"data_compressed":560711,"data_compressed_allocated":757760,"data_compressed_original":4206592,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987345"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140922+0000","last_change":"2024-04-16T05:32:55.507548+0000","last_active":"2024-04-16T05:35:17.140922+0000","last_peered":"2024-04-16T05:35:17.140922+0000","last_clean":"2024-04-16T05:35:17.140922+0000","last_became_active":"2024-04-16T05:32:55.507200+0000","last_became_peered":"2024-04-16T05:32:55.507200+0000","last_unstale":"2024-04-16T05:35:17.140922+0000","last_undegraded":"2024-04-16T05:35:17.140922+0000","last_fullsized":"2024-04-16T05:35:17.140922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:58:07.842226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139986+0000","last_change":"2024-04-16T05:32:55.509792+0000","last_active":"2024-04-16T05:35:17.139986+0000","last_peered":"2024-04-16T05:35:17.139986+0000","last_clean":"2024-04-16T05:35:17.139986+0000","last_became_active":"2024-04-16T05:32:55.509442+0000","last_became_peered":"2024-04-16T05:32:55.509442+0000","last_unstale":"2024-04-16T05:35:17.139986+0000","last_undegraded":"2024-04-16T05:35:17.139986+0000","last_fullsized":"2024-04-16T05:35:17.139986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:08:42.040875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140224+0000","last_change":"2024-04-16T05:32:55.505664+0000","last_active":"2024-04-16T05:35:17.140224+0000","last_peered":"2024-04-16T05:35:17.140224+0000","last_clean":"2024-04-16T05:35:17.140224+0000","last_became_active":"2024-04-16T05:32:55.505281+0000","last_became_peered":"2024-04-16T05:32:55.505281+0000","last_unstale":"2024-04-16T05:35:17.140224+0000","last_undegraded":"2024-04-16T05:35:17.140224+0000","last_fullsized":"2024-04-16T05:35:17.140224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:33.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139993+0000","last_change":"2024-04-16T05:32:55.517072+0000","last_active":"2024-04-16T05:35:17.139993+0000","last_peered":"2024-04-16T05:35:17.139993+0000","last_clean":"2024-04-16T05:35:17.139993+0000","last_became_active":"2024-04-16T05:32:55.516648+0000","last_became_peered":"2024-04-16T05:32:55.516648+0000","last_unstale":"2024-04-16T05:35:17.139993+0000","last_undegraded":"2024-04-16T05:35:17.139993+0000","last_fullsized":"2024-04-16T05:35:17.139993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:41.735720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935557+0000","last_change":"2024-04-16T05:32:55.506773+0000","last_active":"2024-04-16T05:35:16.935557+0000","last_peered":"2024-04-16T05:35:16.935557+0000","last_clean":"2024-04-16T05:35:16.935557+0000","last_became_active":"2024-04-16T05:32:55.506414+0000","last_became_peered":"2024-04-16T05:32:55.506414+0000","last_unstale":"2024-04-16T05:35:16.935557+0000","last_undegraded":"2024-04-16T05:35:16.935557+0000","last_fullsized":"2024-04-16T05:35:16.935557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:59.254334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935461+0000","last_change":"2024-04-16T05:32:55.517875+0000","last_active":"2024-04-16T05:35:16.935461+0000","last_peered":"2024-04-16T05:35:16.935461+0000","last_clean":"2024-04-16T05:35:16.935461+0000","last_became_active":"2024-04-16T05:32:55.517546+0000","last_became_peered":"2024-04-16T05:32:55.517546+0000","last_unstale":"2024-04-16T05:35:16.935461+0000","last_undegraded":"2024-04-16T05:35:16.935461+0000","last_fullsized":"2024-04-16T05:35:16.935461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:12:15.098439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139958+0000","last_change":"2024-04-16T05:32:55.509036+0000","last_active":"2024-04-16T05:35:17.139958+0000","last_peered":"2024-04-16T05:35:17.139958+0000","last_clean":"2024-04-16T05:35:17.139958+0000","last_became_active":"2024-04-16T05:32:55.508690+0000","last_became_peered":"2024-04-16T05:32:55.508690+0000","last_unstale":"2024-04-16T05:35:17.139958+0000","last_undegraded":"2024-04-16T05:35:17.139958+0000","last_fullsized":"2024-04-16T05:35:17.139958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:26.724281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934682+0000","last_change":"2024-04-16T05:32:55.510146+0000","last_active":"2024-04-16T05:35:16.934682+0000","last_peered":"2024-04-16T05:35:16.934682+0000","last_clean":"2024-04-16T05:35:16.934682+0000","last_became_active":"2024-04-16T05:32:55.508665+0000","last_became_peered":"2024-04-16T05:32:55.508665+0000","last_unstale":"2024-04-16T05:35:16.934682+0000","last_undegraded":"2024-04-16T05:35:16.934682+0000","last_fullsized":"2024-04-16T05:35:16.934682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:14.158221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140557+0000","last_change":"2024-04-16T05:32:55.513738+0000","last_active":"2024-04-16T05:35:17.140557+0000","last_peered":"2024-04-16T05:35:17.140557+0000","last_clean":"2024-04-16T05:35:17.140557+0000","last_became_active":"2024-04-16T05:32:55.513374+0000","last_became_peered":"2024-04-16T05:32:55.513374+0000","last_unstale":"2024-04-16T05:35:17.140557+0000","last_undegraded":"2024-04-16T05:35:17.140557+0000","last_fullsized":"2024-04-16T05:35:17.140557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:06:23.390908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935707+0000","last_change":"2024-04-16T05:32:55.519276+0000","last_active":"2024-04-16T05:35:16.935707+0000","last_peered":"2024-04-16T05:35:16.935707+0000","last_clean":"2024-04-16T05:35:16.935707+0000","last_became_active":"2024-04-16T05:32:55.518729+0000","last_became_peered":"2024-04-16T05:32:55.518729+0000","last_unstale":"2024-04-16T05:35:16.935707+0000","last_undegraded":"2024-04-16T05:35:16.935707+0000","last_fullsized":"2024-04-16T05:35:16.935707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:44:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140646+0000","last_change":"2024-04-16T05:32:55.516379+0000","last_active":"2024-04-16T05:35:17.140646+0000","last_peered":"2024-04-16T05:35:17.140646+0000","last_clean":"2024-04-16T05:35:17.140646+0000","last_became_active":"2024-04-16T05:32:55.515574+0000","last_became_peered":"2024-04-16T05:32:55.515574+0000","last_unstale":"2024-04-16T05:35:17.140646+0000","last_undegraded":"2024-04-16T05:35:17.140646+0000","last_fullsized":"2024-04-16T05:35:17.140646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:16:01.744380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032219+0000","last_change":"2024-04-16T05:33:52.674399+0000","last_active":"2024-04-16T05:35:17.032219+0000","last_peered":"2024-04-16T05:35:17.032219+0000","last_clean":"2024-04-16T05:35:17.032219+0000","last_became_active":"2024-04-16T05:33:52.673964+0000","last_became_peered":"2024-04-16T05:33:52.673964+0000","last_unstale":"2024-04-16T05:35:17.032219+0000","last_undegraded":"2024-04-16T05:35:17.032219+0000","last_fullsized":"2024-04-16T05:35:17.032219+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.166747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141154+0000","last_change":"2024-04-16T05:32:55.509682+0000","last_active":"2024-04-16T05:35:17.141154+0000","last_peered":"2024-04-16T05:35:17.141154+0000","last_clean":"2024-04-16T05:35:17.141154+0000","last_became_active":"2024-04-16T05:32:55.509383+0000","last_became_peered":"2024-04-16T05:32:55.509383+0000","last_unstale":"2024-04-16T05:35:17.141154+0000","last_undegraded":"2024-04-16T05:35:17.141154+0000","last_fullsized":"2024-04-16T05:35:17.141154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:25:23.487003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139784+0000","last_change":"2024-04-16T05:32:55.506338+0000","last_active":"2024-04-16T05:35:17.139784+0000","last_peered":"2024-04-16T05:35:17.139784+0000","last_clean":"2024-04-16T05:35:17.139784+0000","last_became_active":"2024-04-16T05:32:55.506000+0000","last_became_peered":"2024-04-16T05:32:55.506000+0000","last_unstale":"2024-04-16T05:35:17.139784+0000","last_undegraded":"2024-04-16T05:35:17.139784+0000","last_fullsized":"2024-04-16T05:35:17.139784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:37:02.296515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139777+0000","last_change":"2024-04-16T05:32:55.516897+0000","last_active":"2024-04-16T05:35:17.139777+0000","last_peered":"2024-04-16T05:35:17.139777+0000","last_clean":"2024-04-16T05:35:17.139777+0000","last_became_active":"2024-04-16T05:32:55.516489+0000","last_became_peered":"2024-04-16T05:32:55.516489+0000","last_unstale":"2024-04-16T05:35:17.139777+0000","last_undegraded":"2024-04-16T05:35:17.139777+0000","last_fullsized":"2024-04-16T05:35:17.139777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:26:08.289501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139046+0000","last_change":"2024-04-16T05:32:55.507374+0000","last_active":"2024-04-16T05:35:17.139046+0000","last_peered":"2024-04-16T05:35:17.139046+0000","last_clean":"2024-04-16T05:35:17.139046+0000","last_became_active":"2024-04-16T05:32:55.507045+0000","last_became_peered":"2024-04-16T05:32:55.507045+0000","last_unstale":"2024-04-16T05:35:17.139046+0000","last_undegraded":"2024-04-16T05:35:17.139046+0000","last_fullsized":"2024-04-16T05:35:17.139046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:35.968704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140697+0000","last_change":"2024-04-16T05:32:55.508845+0000","last_active":"2024-04-16T05:35:17.140697+0000","last_peered":"2024-04-16T05:35:17.140697+0000","last_clean":"2024-04-16T05:35:17.140697+0000","last_became_active":"2024-04-16T05:32:55.508527+0000","last_became_peered":"2024-04-16T05:32:55.508527+0000","last_unstale":"2024-04-16T05:35:17.140697+0000","last_undegraded":"2024-04-16T05:35:17.140697+0000","last_fullsized":"2024-04-16T05:35:17.140697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:20:15.033043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032367+0000","last_change":"2024-04-16T05:32:55.506059+0000","last_active":"2024-04-16T05:35:17.032367+0000","last_peered":"2024-04-16T05:35:17.032367+0000","last_clean":"2024-04-16T05:35:17.032367+0000","last_became_active":"2024-04-16T05:32:55.505728+0000","last_became_peered":"2024-04-16T05:32:55.505728+0000","last_unstale":"2024-04-16T05:35:17.032367+0000","last_undegraded":"2024-04-16T05:35:17.032367+0000","last_fullsized":"2024-04-16T05:35:17.032367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:19.253543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140417+0000","last_change":"2024-04-16T05:32:55.506791+0000","last_active":"2024-04-16T05:35:17.140417+0000","last_peered":"2024-04-16T05:35:17.140417+0000","last_clean":"2024-04-16T05:35:17.140417+0000","last_became_active":"2024-04-16T05:32:55.506379+0000","last_became_peered":"2024-04-16T05:32:55.506379+0000","last_unstale":"2024-04-16T05:35:17.140417+0000","last_undegraded":"2024-04-16T05:35:17.140417+0000","last_fullsized":"2024-04-16T05:35:17.140417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:09:11.457144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139494+0000","last_change":"2024-04-16T05:33:52.582503+0000","last_active":"2024-04-16T05:35:17.139494+0000","last_peered":"2024-04-16T05:35:17.139494+0000","last_clean":"2024-04-16T05:35:17.139494+0000","last_became_active":"2024-04-16T05:33:52.581996+0000","last_became_peered":"2024-04-16T05:33:52.581996+0000","last_unstale":"2024-04-16T05:35:17.139494+0000","last_undegraded":"2024-04-16T05:35:17.139494+0000","last_fullsized":"2024-04-16T05:35:17.139494+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:34:40.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032559+0000","last_change":"2024-04-16T05:32:54.504826+0000","last_active":"2024-04-16T05:35:17.032559+0000","last_peered":"2024-04-16T05:35:17.032559+0000","last_clean":"2024-04-16T05:35:17.032559+0000","last_became_active":"2024-04-16T05:32:54.504526+0000","last_became_peered":"2024-04-16T05:32:54.504526+0000","last_unstale":"2024-04-16T05:35:17.032559+0000","last_undegraded":"2024-04-16T05:35:17.032559+0000","last_fullsized":"2024-04-16T05:35:17.032559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:21:28.434692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935279+0000","last_change":"2024-04-16T05:33:52.580931+0000","last_active":"2024-04-16T05:35:16.935279+0000","last_peered":"2024-04-16T05:35:16.935279+0000","last_clean":"2024-04-16T05:35:16.935279+0000","last_became_active":"2024-04-16T05:33:52.580685+0000","last_became_peered":"2024-04-16T05:33:52.580685+0000","last_unstale":"2024-04-16T05:35:16.935279+0000","last_undegraded":"2024-04-16T05:35:16.935279+0000","last_fullsized":"2024-04-16T05:35:16.935279+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:35:34.548838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139517+0000","last_change":"2024-04-16T05:32:59.524687+0000","last_active":"2024-04-16T05:35:17.139517+0000","last_peered":"2024-04-16T05:35:17.139517+0000","last_clean":"2024-04-16T05:35:17.139517+0000","last_became_active":"2024-04-16T05:32:59.524228+0000","last_became_peered":"2024-04-16T05:32:59.524228+0000","last_unstale":"2024-04-16T05:35:17.139517+0000","last_undegraded":"2024-04-16T05:35:17.139517+0000","last_fullsized":"2024-04-16T05:35:17.139517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:23:15.244359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935307+0000","last_change":"2024-04-16T05:32:54.504062+0000","last_active":"2024-04-16T05:35:16.935307+0000","last_peered":"2024-04-16T05:35:16.935307+0000","last_clean":"2024-04-16T05:35:16.935307+0000","last_became_active":"2024-04-16T05:32:54.503745+0000","last_became_peered":"2024-04-16T05:32:54.503745+0000","last_unstale":"2024-04-16T05:35:16.935307+0000","last_undegraded":"2024-04-16T05:35:16.935307+0000","last_fullsized":"2024-04-16T05:35:16.935307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:29:34.157937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140473+0000","last_change":"2024-04-16T05:32:55.509820+0000","last_active":"2024-04-16T05:35:17.140473+0000","last_peered":"2024-04-16T05:35:17.140473+0000","last_clean":"2024-04-16T05:35:17.140473+0000","last_became_active":"2024-04-16T05:32:55.509473+0000","last_became_peered":"2024-04-16T05:32:55.509473+0000","last_unstale":"2024-04-16T05:35:17.140473+0000","last_undegraded":"2024-04-16T05:35:17.140473+0000","last_fullsized":"2024-04-16T05:35:17.140473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:12:22.433189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140278+0000","last_change":"2024-04-16T05:33:52.678903+0000","last_active":"2024-04-16T05:35:17.140278+0000","last_peered":"2024-04-16T05:35:17.140278+0000","last_clean":"2024-04-16T05:35:17.140278+0000","last_became_active":"2024-04-16T05:33:52.678388+0000","last_became_peered":"2024-04-16T05:33:52.678388+0000","last_unstale":"2024-04-16T05:35:17.140278+0000","last_undegraded":"2024-04-16T05:35:17.140278+0000","last_fullsized":"2024-04-16T05:35:17.140278+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:07:01.060645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935984+0000","last_change":"2024-04-16T05:32:54.511469+0000","last_active":"2024-04-16T05:35:16.935984+0000","last_peered":"2024-04-16T05:35:16.935984+0000","last_clean":"2024-04-16T05:35:16.935984+0000","last_became_active":"2024-04-16T05:32:54.511120+0000","last_became_peered":"2024-04-16T05:32:54.511120+0000","last_unstale":"2024-04-16T05:35:16.935984+0000","last_undegraded":"2024-04-16T05:35:16.935984+0000","last_fullsized":"2024-04-16T05:35:16.935984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139428+0000","last_change":"2024-04-16T05:32:55.512308+0000","last_active":"2024-04-16T05:35:17.139428+0000","last_peered":"2024-04-16T05:35:17.139428+0000","last_clean":"2024-04-16T05:35:17.139428+0000","last_became_active":"2024-04-16T05:32:55.511625+0000","last_became_peered":"2024-04-16T05:32:55.511625+0000","last_unstale":"2024-04-16T05:35:17.139428+0000","last_undegraded":"2024-04-16T05:35:17.139428+0000","last_fullsized":"2024-04-16T05:35:17.139428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:29:49.323092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139445+0000","last_change":"2024-04-16T05:32:59.519306+0000","last_active":"2024-04-16T05:35:17.139445+0000","last_peered":"2024-04-16T05:35:17.139445+0000","last_clean":"2024-04-16T05:35:17.139445+0000","last_became_active":"2024-04-16T05:32:59.518797+0000","last_became_peered":"2024-04-16T05:32:59.518797+0000","last_unstale":"2024-04-16T05:35:17.139445+0000","last_undegraded":"2024-04-16T05:35:17.139445+0000","last_fullsized":"2024-04-16T05:35:17.139445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:04:12.389004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935592+0000","last_change":"2024-04-16T05:32:54.505102+0000","last_active":"2024-04-16T05:35:16.935592+0000","last_peered":"2024-04-16T05:35:16.935592+0000","last_clean":"2024-04-16T05:35:16.935592+0000","last_became_active":"2024-04-16T05:32:54.504806+0000","last_became_peered":"2024-04-16T05:32:54.504806+0000","last_unstale":"2024-04-16T05:35:16.935592+0000","last_undegraded":"2024-04-16T05:35:16.935592+0000","last_fullsized":"2024-04-16T05:35:16.935592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:19.467690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.041059+0000","last_change":"2024-04-16T05:32:55.517097+0000","last_active":"2024-04-16T05:35:17.041059+0000","last_peered":"2024-04-16T05:35:17.041059+0000","last_clean":"2024-04-16T05:35:17.041059+0000","last_became_active":"2024-04-16T05:32:55.516717+0000","last_became_peered":"2024-04-16T05:32:55.516717+0000","last_unstale":"2024-04-16T05:35:17.041059+0000","last_undegraded":"2024-04-16T05:35:17.041059+0000","last_fullsized":"2024-04-16T05:35:17.041059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:10:44.544935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140140+0000","last_change":"2024-04-16T05:32:59.521429+0000","last_active":"2024-04-16T05:35:17.140140+0000","last_peered":"2024-04-16T05:35:17.140140+0000","last_clean":"2024-04-16T05:35:17.140140+0000","last_became_active":"2024-04-16T05:32:59.520789+0000","last_became_peered":"2024-04-16T05:32:59.520789+0000","last_unstale":"2024-04-16T05:35:17.140140+0000","last_undegraded":"2024-04-16T05:35:17.140140+0000","last_fullsized":"2024-04-16T05:35:17.140140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:40:14.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":167,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787507+0000","last_change":"2024-04-16T05:32:54.507374+0000","last_active":"2024-04-16T05:35:17.787507+0000","last_peered":"2024-04-16T05:35:17.787507+0000","last_clean":"2024-04-16T05:35:17.787507+0000","last_became_active":"2024-04-16T05:32:54.507055+0000","last_became_peered":"2024-04-16T05:32:54.507055+0000","last_unstale":"2024-04-16T05:35:17.787507+0000","last_undegraded":"2024-04-16T05:35:17.787507+0000","last_fullsized":"2024-04-16T05:35:17.787507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:00:57.761971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935533+0000","last_change":"2024-04-16T05:32:55.513980+0000","last_active":"2024-04-16T05:35:16.935533+0000","last_peered":"2024-04-16T05:35:16.935533+0000","last_clean":"2024-04-16T05:35:16.935533+0000","last_became_active":"2024-04-16T05:32:55.513622+0000","last_became_peered":"2024-04-16T05:32:55.513622+0000","last_unstale":"2024-04-16T05:35:16.935533+0000","last_undegraded":"2024-04-16T05:35:16.935533+0000","last_fullsized":"2024-04-16T05:35:16.935533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:16:10.018381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139704+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:35:17.139704+0000","last_peered":"2024-04-16T05:35:17.139704+0000","last_clean":"2024-04-16T05:35:17.139704+0000","last_became_active":"2024-04-16T05:32:59.523032+0000","last_became_peered":"2024-04-16T05:32:59.523032+0000","last_unstale":"2024-04-16T05:35:17.139704+0000","last_undegraded":"2024-04-16T05:35:17.139704+0000","last_fullsized":"2024-04-16T05:35:17.139704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:33:17.897241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935663+0000","last_change":"2024-04-16T05:32:54.504527+0000","last_active":"2024-04-16T05:35:16.935663+0000","last_peered":"2024-04-16T05:35:16.935663+0000","last_clean":"2024-04-16T05:35:16.935663+0000","last_became_active":"2024-04-16T05:32:54.504187+0000","last_became_peered":"2024-04-16T05:32:54.504187+0000","last_unstale":"2024-04-16T05:35:16.935663+0000","last_undegraded":"2024-04-16T05:35:16.935663+0000","last_fullsized":"2024-04-16T05:35:16.935663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.414362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040983+0000","last_change":"2024-04-16T05:32:55.520270+0000","last_active":"2024-04-16T05:35:17.040983+0000","last_peered":"2024-04-16T05:35:17.040983+0000","last_clean":"2024-04-16T05:35:17.040983+0000","last_became_active":"2024-04-16T05:32:55.519895+0000","last_became_peered":"2024-04-16T05:32:55.519895+0000","last_unstale":"2024-04-16T05:35:17.040983+0000","last_undegraded":"2024-04-16T05:35:17.040983+0000","last_fullsized":"2024-04-16T05:35:17.040983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:09.967836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140531+0000","last_change":"2024-04-16T05:32:59.518997+0000","last_active":"2024-04-16T05:35:17.140531+0000","last_peered":"2024-04-16T05:35:17.140531+0000","last_clean":"2024-04-16T05:35:17.140531+0000","last_became_active":"2024-04-16T05:32:59.518558+0000","last_became_peered":"2024-04-16T05:32:59.518558+0000","last_unstale":"2024-04-16T05:35:17.140531+0000","last_undegraded":"2024-04-16T05:35:17.140531+0000","last_fullsized":"2024-04-16T05:35:17.140531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:04:30.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140063+0000","last_change":"2024-04-16T05:32:54.502074+0000","last_active":"2024-04-16T05:35:17.140063+0000","last_peered":"2024-04-16T05:35:17.140063+0000","last_clean":"2024-04-16T05:35:17.140063+0000","last_became_active":"2024-04-16T05:32:54.501730+0000","last_became_peered":"2024-04-16T05:32:54.501730+0000","last_unstale":"2024-04-16T05:35:17.140063+0000","last_undegraded":"2024-04-16T05:35:17.140063+0000","last_fullsized":"2024-04-16T05:35:17.140063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:05:02.929799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934611+0000","last_change":"2024-04-16T05:32:55.504557+0000","last_active":"2024-04-16T05:35:16.934611+0000","last_peered":"2024-04-16T05:35:16.934611+0000","last_clean":"2024-04-16T05:35:16.934611+0000","last_became_active":"2024-04-16T05:32:55.504244+0000","last_became_peered":"2024-04-16T05:32:55.504244+0000","last_unstale":"2024-04-16T05:35:16.934611+0000","last_undegraded":"2024-04-16T05:35:16.934611+0000","last_fullsized":"2024-04-16T05:35:16.934611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:33:40.500436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139880+0000","last_change":"2024-04-16T05:32:59.524681+0000","last_active":"2024-04-16T05:35:17.139880+0000","last_peered":"2024-04-16T05:35:17.139880+0000","last_clean":"2024-04-16T05:35:17.139880+0000","last_became_active":"2024-04-16T05:32:59.524217+0000","last_became_peered":"2024-04-16T05:32:59.524217+0000","last_unstale":"2024-04-16T05:35:17.139880+0000","last_undegraded":"2024-04-16T05:35:17.139880+0000","last_fullsized":"2024-04-16T05:35:17.139880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:28.314442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040615+0000","last_change":"2024-04-16T05:32:54.508242+0000","last_active":"2024-04-16T05:35:17.040615+0000","last_peered":"2024-04-16T05:35:17.040615+0000","last_clean":"2024-04-16T05:35:17.040615+0000","last_became_active":"2024-04-16T05:32:54.507782+0000","last_became_peered":"2024-04-16T05:32:54.507782+0000","last_unstale":"2024-04-16T05:35:17.040615+0000","last_undegraded":"2024-04-16T05:35:17.040615+0000","last_fullsized":"2024-04-16T05:35:17.040615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:27:26.805716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.031911+0000","last_change":"2024-04-16T05:32:55.508985+0000","last_active":"2024-04-16T05:35:17.031911+0000","last_peered":"2024-04-16T05:35:17.031911+0000","last_clean":"2024-04-16T05:35:17.031911+0000","last_became_active":"2024-04-16T05:32:55.508659+0000","last_became_peered":"2024-04-16T05:32:55.508659+0000","last_unstale":"2024-04-16T05:35:17.031911+0000","last_undegraded":"2024-04-16T05:35:17.031911+0000","last_fullsized":"2024-04-16T05:35:17.031911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:43:41.534155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934714+0000","last_change":"2024-04-16T05:32:59.521257+0000","last_active":"2024-04-16T05:35:16.934714+0000","last_peered":"2024-04-16T05:35:16.934714+0000","last_clean":"2024-04-16T05:35:16.934714+0000","last_became_active":"2024-04-16T05:32:59.520848+0000","last_became_peered":"2024-04-16T05:32:59.520848+0000","last_unstale":"2024-04-16T05:35:16.934714+0000","last_undegraded":"2024-04-16T05:35:16.934714+0000","last_fullsized":"2024-04-16T05:35:16.934714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:14.219378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'8","reported_seq":284,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.786952+0000","last_change":"2024-04-16T05:32:54.500684+0000","last_active":"2024-04-16T05:35:17.786952+0000","last_peered":"2024-04-16T05:35:17.786952+0000","last_clean":"2024-04-16T05:35:17.786952+0000","last_became_active":"2024-04-16T05:32:54.500290+0000","last_became_peered":"2024-04-16T05:32:54.500290+0000","last_unstale":"2024-04-16T05:35:17.786952+0000","last_undegraded":"2024-04-16T05:35:17.786952+0000","last_fullsized":"2024-04-16T05:35:17.786952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:20:55.263313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140189+0000","last_change":"2024-04-16T05:32:55.511224+0000","last_active":"2024-04-16T05:35:17.140189+0000","last_peered":"2024-04-16T05:35:17.140189+0000","last_clean":"2024-04-16T05:35:17.140189+0000","last_became_active":"2024-04-16T05:32:55.510891+0000","last_became_peered":"2024-04-16T05:32:55.510891+0000","last_unstale":"2024-04-16T05:35:17.140189+0000","last_undegraded":"2024-04-16T05:35:17.140189+0000","last_fullsized":"2024-04-16T05:35:17.140189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:27:24.894658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.031803+0000","last_change":"2024-04-16T05:32:59.521695+0000","last_active":"2024-04-16T05:35:17.031803+0000","last_peered":"2024-04-16T05:35:17.031803+0000","last_clean":"2024-04-16T05:35:17.031803+0000","last_became_active":"2024-04-16T05:32:59.520569+0000","last_became_peered":"2024-04-16T05:32:59.520569+0000","last_unstale":"2024-04-16T05:35:17.031803+0000","last_undegraded":"2024-04-16T05:35:17.031803+0000","last_fullsized":"2024-04-16T05:35:17.031803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:01:14.726043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141002+0000","last_change":"2024-04-16T05:32:54.505058+0000","last_active":"2024-04-16T05:35:17.141002+0000","last_peered":"2024-04-16T05:35:17.141002+0000","last_clean":"2024-04-16T05:35:17.141002+0000","last_became_active":"2024-04-16T05:32:54.504636+0000","last_became_peered":"2024-04-16T05:32:54.504636+0000","last_unstale":"2024-04-16T05:35:17.141002+0000","last_undegraded":"2024-04-16T05:35:17.141002+0000","last_fullsized":"2024-04-16T05:35:17.141002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:49:14.429468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140474+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:35:17.140474+0000","last_peered":"2024-04-16T05:35:17.140474+0000","last_clean":"2024-04-16T05:35:17.140474+0000","last_became_active":"2024-04-16T05:33:52.678570+0000","last_became_peered":"2024-04-16T05:33:52.678570+0000","last_unstale":"2024-04-16T05:35:17.140474+0000","last_undegraded":"2024-04-16T05:35:17.140474+0000","last_fullsized":"2024-04-16T05:35:17.140474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:56:32.405491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140508+0000","last_change":"2024-04-16T05:32:59.520394+0000","last_active":"2024-04-16T05:35:17.140508+0000","last_peered":"2024-04-16T05:35:17.140508+0000","last_clean":"2024-04-16T05:35:17.140508+0000","last_became_active":"2024-04-16T05:32:59.519947+0000","last_became_peered":"2024-04-16T05:32:59.519947+0000","last_unstale":"2024-04-16T05:35:17.140508+0000","last_undegraded":"2024-04-16T05:35:17.140508+0000","last_fullsized":"2024-04-16T05:35:17.140508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:33:27.647919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.041043+0000","last_change":"2024-04-16T05:32:54.505350+0000","last_active":"2024-04-16T05:35:17.041043+0000","last_peered":"2024-04-16T05:35:17.041043+0000","last_clean":"2024-04-16T05:35:17.041043+0000","last_became_active":"2024-04-16T05:32:54.505011+0000","last_became_peered":"2024-04-16T05:32:54.505011+0000","last_unstale":"2024-04-16T05:35:17.041043+0000","last_undegraded":"2024-04-16T05:35:17.041043+0000","last_fullsized":"2024-04-16T05:35:17.041043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:03:00.276862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139699+0000","last_change":"2024-04-16T05:33:52.673799+0000","last_active":"2024-04-16T05:35:17.139699+0000","last_peered":"2024-04-16T05:35:17.139699+0000","last_clean":"2024-04-16T05:35:17.139699+0000","last_became_active":"2024-04-16T05:33:52.673344+0000","last_became_peered":"2024-04-16T05:33:52.673344+0000","last_unstale":"2024-04-16T05:35:17.139699+0000","last_undegraded":"2024-04-16T05:35:17.139699+0000","last_fullsized":"2024-04-16T05:35:17.139699+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:07.387912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.031930+0000","last_change":"2024-04-16T05:32:59.519660+0000","last_active":"2024-04-16T05:35:17.031930+0000","last_peered":"2024-04-16T05:35:17.031930+0000","last_clean":"2024-04-16T05:35:17.031930+0000","last_became_active":"2024-04-16T05:32:59.518198+0000","last_became_peered":"2024-04-16T05:32:59.518198+0000","last_unstale":"2024-04-16T05:35:17.031930+0000","last_undegraded":"2024-04-16T05:35:17.031930+0000","last_fullsized":"2024-04-16T05:35:17.031930+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:03:10.096252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'40","reported_seq":325,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787683+0000","last_change":"2024-04-16T05:32:54.501530+0000","last_active":"2024-04-16T05:35:17.787683+0000","last_peered":"2024-04-16T05:35:17.787683+0000","last_clean":"2024-04-16T05:35:17.787683+0000","last_became_active":"2024-04-16T05:32:54.501159+0000","last_became_peered":"2024-04-16T05:32:54.501159+0000","last_unstale":"2024-04-16T05:35:17.787683+0000","last_undegraded":"2024-04-16T05:35:17.787683+0000","last_fullsized":"2024-04-16T05:35:17.787683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:26:22.240137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66452,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":190,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935749+0000","last_change":"2024-04-16T05:32:55.512210+0000","last_active":"2024-04-16T05:35:16.935749+0000","last_peered":"2024-04-16T05:35:16.935749+0000","last_clean":"2024-04-16T05:35:16.935749+0000","last_became_active":"2024-04-16T05:32:55.511672+0000","last_became_peered":"2024-04-16T05:32:55.511672+0000","last_unstale":"2024-04-16T05:35:16.935749+0000","last_undegraded":"2024-04-16T05:35:16.935749+0000","last_fullsized":"2024-04-16T05:35:16.935749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:10:53.755803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.041267+0000","last_change":"2024-04-16T05:32:59.527552+0000","last_active":"2024-04-16T05:35:17.041267+0000","last_peered":"2024-04-16T05:35:17.041267+0000","last_clean":"2024-04-16T05:35:17.041267+0000","last_became_active":"2024-04-16T05:32:59.526919+0000","last_became_peered":"2024-04-16T05:32:59.526919+0000","last_unstale":"2024-04-16T05:35:17.041267+0000","last_undegraded":"2024-04-16T05:35:17.041267+0000","last_fullsized":"2024-04-16T05:35:17.041267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:00:46.342383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139675+0000","last_change":"2024-04-16T05:32:54.494589+0000","last_active":"2024-04-16T05:35:17.139675+0000","last_peered":"2024-04-16T05:35:17.139675+0000","last_clean":"2024-04-16T05:35:17.139675+0000","last_became_active":"2024-04-16T05:32:54.494221+0000","last_became_peered":"2024-04-16T05:32:54.494221+0000","last_unstale":"2024-04-16T05:35:17.139675+0000","last_undegraded":"2024-04-16T05:35:17.139675+0000","last_fullsized":"2024-04-16T05:35:17.139675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:17:53.245359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140562+0000","last_change":"2024-04-16T05:32:55.502724+0000","last_active":"2024-04-16T05:35:17.140562+0000","last_peered":"2024-04-16T05:35:17.140562+0000","last_clean":"2024-04-16T05:35:17.140562+0000","last_became_active":"2024-04-16T05:32:55.502415+0000","last_became_peered":"2024-04-16T05:32:55.502415+0000","last_unstale":"2024-04-16T05:35:17.140562+0000","last_undegraded":"2024-04-16T05:35:17.140562+0000","last_fullsized":"2024-04-16T05:35:17.140562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:20:44.443926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140552+0000","last_change":"2024-04-16T05:32:59.522243+0000","last_active":"2024-04-16T05:35:17.140552+0000","last_peered":"2024-04-16T05:35:17.140552+0000","last_clean":"2024-04-16T05:35:17.140552+0000","last_became_active":"2024-04-16T05:32:59.521823+0000","last_became_peered":"2024-04-16T05:32:59.521823+0000","last_unstale":"2024-04-16T05:35:17.140552+0000","last_undegraded":"2024-04-16T05:35:17.140552+0000","last_fullsized":"2024-04-16T05:35:17.140552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:09:11.158434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040872+0000","last_change":"2024-04-16T05:32:54.507505+0000","last_active":"2024-04-16T05:35:17.040872+0000","last_peered":"2024-04-16T05:35:17.040872+0000","last_clean":"2024-04-16T05:35:17.040872+0000","last_became_active":"2024-04-16T05:32:54.506558+0000","last_became_peered":"2024-04-16T05:32:54.506558+0000","last_unstale":"2024-04-16T05:35:17.040872+0000","last_undegraded":"2024-04-16T05:35:17.040872+0000","last_fullsized":"2024-04-16T05:35:17.040872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:36:17.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140421+0000","last_change":"2024-04-16T05:32:55.518998+0000","last_active":"2024-04-16T05:35:17.140421+0000","last_peered":"2024-04-16T05:35:17.140421+0000","last_clean":"2024-04-16T05:35:17.140421+0000","last_became_active":"2024-04-16T05:32:55.518640+0000","last_became_peered":"2024-04-16T05:32:55.518640+0000","last_unstale":"2024-04-16T05:35:17.140421+0000","last_undegraded":"2024-04-16T05:35:17.140421+0000","last_fullsized":"2024-04-16T05:35:17.140421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:42:44.667504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032258+0000","last_change":"2024-04-16T05:32:59.519050+0000","last_active":"2024-04-16T05:35:17.032258+0000","last_peered":"2024-04-16T05:35:17.032258+0000","last_clean":"2024-04-16T05:35:17.032258+0000","last_became_active":"2024-04-16T05:32:59.518663+0000","last_became_peered":"2024-04-16T05:32:59.518663+0000","last_unstale":"2024-04-16T05:35:17.032258+0000","last_undegraded":"2024-04-16T05:35:17.032258+0000","last_fullsized":"2024-04-16T05:35:17.032258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:36:19.649605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787652+0000","last_change":"2024-04-16T05:32:54.500940+0000","last_active":"2024-04-16T05:35:17.787652+0000","last_peered":"2024-04-16T05:35:17.787652+0000","last_clean":"2024-04-16T05:35:17.787652+0000","last_became_active":"2024-04-16T05:32:54.500542+0000","last_became_peered":"2024-04-16T05:32:54.500542+0000","last_unstale":"2024-04-16T05:35:17.787652+0000","last_undegraded":"2024-04-16T05:35:17.787652+0000","last_fullsized":"2024-04-16T05:35:17.787652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:38:00.232244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139964+0000","last_change":"2024-04-16T05:32:55.509889+0000","last_active":"2024-04-16T05:35:17.139964+0000","last_peered":"2024-04-16T05:35:17.139964+0000","last_clean":"2024-04-16T05:35:17.139964+0000","last_became_active":"2024-04-16T05:32:55.508911+0000","last_became_peered":"2024-04-16T05:32:55.508911+0000","last_unstale":"2024-04-16T05:35:17.139964+0000","last_undegraded":"2024-04-16T05:35:17.139964+0000","last_fullsized":"2024-04-16T05:35:17.139964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:14:46.609127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141707+0000","last_change":"2024-04-16T05:32:59.523555+0000","last_active":"2024-04-16T05:35:17.141707+0000","last_peered":"2024-04-16T05:35:17.141707+0000","last_clean":"2024-04-16T05:35:17.141707+0000","last_became_active":"2024-04-16T05:32:59.523187+0000","last_became_peered":"2024-04-16T05:32:59.523187+0000","last_unstale":"2024-04-16T05:35:17.141707+0000","last_undegraded":"2024-04-16T05:35:17.141707+0000","last_fullsized":"2024-04-16T05:35:17.141707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:35.049355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935450+0000","last_change":"2024-04-16T05:32:54.503247+0000","last_active":"2024-04-16T05:35:16.935450+0000","last_peered":"2024-04-16T05:35:16.935450+0000","last_clean":"2024-04-16T05:35:16.935450+0000","last_became_active":"2024-04-16T05:32:54.502927+0000","last_became_peered":"2024-04-16T05:32:54.502927+0000","last_unstale":"2024-04-16T05:35:16.935450+0000","last_undegraded":"2024-04-16T05:35:16.935450+0000","last_fullsized":"2024-04-16T05:35:16.935450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:36:17.585827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040774+0000","last_change":"2024-04-16T05:32:55.519265+0000","last_active":"2024-04-16T05:35:17.040774+0000","last_peered":"2024-04-16T05:35:17.040774+0000","last_clean":"2024-04-16T05:35:17.040774+0000","last_became_active":"2024-04-16T05:32:55.518840+0000","last_became_peered":"2024-04-16T05:32:55.518840+0000","last_unstale":"2024-04-16T05:35:17.040774+0000","last_undegraded":"2024-04-16T05:35:17.040774+0000","last_fullsized":"2024-04-16T05:35:17.040774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:56.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040636+0000","last_change":"2024-04-16T05:32:59.525275+0000","last_active":"2024-04-16T05:35:17.040636+0000","last_peered":"2024-04-16T05:35:17.040636+0000","last_clean":"2024-04-16T05:35:17.040636+0000","last_became_active":"2024-04-16T05:32:59.524546+0000","last_became_peered":"2024-04-16T05:32:59.524546+0000","last_unstale":"2024-04-16T05:35:17.040636+0000","last_undegraded":"2024-04-16T05:35:17.040636+0000","last_fullsized":"2024-04-16T05:35:17.040636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:07.561898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140459+0000","last_change":"2024-04-16T05:34:52.635664+0000","last_active":"2024-04-16T05:35:17.140459+0000","last_peered":"2024-04-16T05:35:17.140459+0000","last_clean":"2024-04-16T05:35:17.140459+0000","last_became_active":"2024-04-16T05:34:52.627471+0000","last_became_peered":"2024-04-16T05:34:52.627471+0000","last_unstale":"2024-04-16T05:35:17.140459+0000","last_undegraded":"2024-04-16T05:35:17.140459+0000","last_fullsized":"2024-04-16T05:35:17.140459+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:48:28.009122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140884+0000","last_change":"2024-04-16T05:32:55.515685+0000","last_active":"2024-04-16T05:35:17.140884+0000","last_peered":"2024-04-16T05:35:17.140884+0000","last_clean":"2024-04-16T05:35:17.140884+0000","last_became_active":"2024-04-16T05:32:55.515367+0000","last_became_peered":"2024-04-16T05:32:55.515367+0000","last_unstale":"2024-04-16T05:35:17.140884+0000","last_undegraded":"2024-04-16T05:35:17.140884+0000","last_fullsized":"2024-04-16T05:35:17.140884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:46.670143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935508+0000","last_change":"2024-04-16T05:33:00.419828+0000","last_active":"2024-04-16T05:35:16.935508+0000","last_peered":"2024-04-16T05:35:16.935508+0000","last_clean":"2024-04-16T05:35:16.935508+0000","last_became_active":"2024-04-16T05:33:00.419321+0000","last_became_peered":"2024-04-16T05:33:00.419321+0000","last_unstale":"2024-04-16T05:35:16.935508+0000","last_undegraded":"2024-04-16T05:35:16.935508+0000","last_fullsized":"2024-04-16T05:35:16.935508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:56.694914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032165+0000","last_change":"2024-04-16T05:32:54.505369+0000","last_active":"2024-04-16T05:35:17.032165+0000","last_peered":"2024-04-16T05:35:17.032165+0000","last_clean":"2024-04-16T05:35:17.032165+0000","last_became_active":"2024-04-16T05:32:54.505043+0000","last_became_peered":"2024-04-16T05:32:54.505043+0000","last_unstale":"2024-04-16T05:35:17.032165+0000","last_undegraded":"2024-04-16T05:35:17.032165+0000","last_fullsized":"2024-04-16T05:35:17.032165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:34.242117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935671+0000","last_change":"2024-04-16T05:32:55.509970+0000","last_active":"2024-04-16T05:35:16.935671+0000","last_peered":"2024-04-16T05:35:16.935671+0000","last_clean":"2024-04-16T05:35:16.935671+0000","last_became_active":"2024-04-16T05:32:55.509632+0000","last_became_peered":"2024-04-16T05:32:55.509632+0000","last_unstale":"2024-04-16T05:35:16.935671+0000","last_undegraded":"2024-04-16T05:35:16.935671+0000","last_fullsized":"2024-04-16T05:35:16.935671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:33:55.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141399+0000","last_change":"2024-04-16T05:32:59.527007+0000","last_active":"2024-04-16T05:35:17.141399+0000","last_peered":"2024-04-16T05:35:17.141399+0000","last_clean":"2024-04-16T05:35:17.141399+0000","last_became_active":"2024-04-16T05:32:59.526645+0000","last_became_peered":"2024-04-16T05:32:59.526645+0000","last_unstale":"2024-04-16T05:35:17.141399+0000","last_undegraded":"2024-04-16T05:35:17.141399+0000","last_fullsized":"2024-04-16T05:35:17.141399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:46:06.698912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":161,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139866+0000","last_change":"2024-04-16T05:32:54.503409+0000","last_active":"2024-04-16T05:35:17.139866+0000","last_peered":"2024-04-16T05:35:17.139866+0000","last_clean":"2024-04-16T05:35:17.139866+0000","last_became_active":"2024-04-16T05:32:54.503069+0000","last_became_peered":"2024-04-16T05:32:54.503069+0000","last_unstale":"2024-04-16T05:35:17.139866+0000","last_undegraded":"2024-04-16T05:35:17.139866+0000","last_fullsized":"2024-04-16T05:35:17.139866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:45:10.535225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935458+0000","last_change":"2024-04-16T05:32:55.519744+0000","last_active":"2024-04-16T05:35:16.935458+0000","last_peered":"2024-04-16T05:35:16.935458+0000","last_clean":"2024-04-16T05:35:16.935458+0000","last_became_active":"2024-04-16T05:32:55.519091+0000","last_became_peered":"2024-04-16T05:32:55.519091+0000","last_unstale":"2024-04-16T05:35:16.935458+0000","last_undegraded":"2024-04-16T05:35:16.935458+0000","last_fullsized":"2024-04-16T05:35:16.935458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:19.417691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141700+0000","last_change":"2024-04-16T05:33:00.419494+0000","last_active":"2024-04-16T05:35:17.141700+0000","last_peered":"2024-04-16T05:35:17.141700+0000","last_clean":"2024-04-16T05:35:17.141700+0000","last_became_active":"2024-04-16T05:33:00.418954+0000","last_became_peered":"2024-04-16T05:33:00.418954+0000","last_unstale":"2024-04-16T05:35:17.141700+0000","last_undegraded":"2024-04-16T05:35:17.141700+0000","last_fullsized":"2024-04-16T05:35:17.141700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:03:01.835304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040794+0000","last_change":"2024-04-16T05:32:55.518272+0000","last_active":"2024-04-16T05:35:17.040794+0000","last_peered":"2024-04-16T05:35:17.040794+0000","last_clean":"2024-04-16T05:35:17.040794+0000","last_became_active":"2024-04-16T05:32:55.517928+0000","last_became_peered":"2024-04-16T05:32:55.517928+0000","last_unstale":"2024-04-16T05:35:17.040794+0000","last_undegraded":"2024-04-16T05:35:17.040794+0000","last_fullsized":"2024-04-16T05:35:17.040794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:44:47.739994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":164,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.178665+0000","last_change":"2024-04-16T05:32:54.503923+0000","last_active":"2024-04-16T05:35:17.178665+0000","last_peered":"2024-04-16T05:35:17.178665+0000","last_clean":"2024-04-16T05:35:17.178665+0000","last_became_active":"2024-04-16T05:32:54.503576+0000","last_became_peered":"2024-04-16T05:32:54.503576+0000","last_unstale":"2024-04-16T05:35:17.178665+0000","last_undegraded":"2024-04-16T05:35:17.178665+0000","last_fullsized":"2024-04-16T05:35:17.178665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:49:36.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140426+0000","last_change":"2024-04-16T05:32:59.517757+0000","last_active":"2024-04-16T05:35:17.140426+0000","last_peered":"2024-04-16T05:35:17.140426+0000","last_clean":"2024-04-16T05:35:17.140426+0000","last_became_active":"2024-04-16T05:32:59.517357+0000","last_became_peered":"2024-04-16T05:32:59.517357+0000","last_unstale":"2024-04-16T05:35:17.140426+0000","last_undegraded":"2024-04-16T05:35:17.140426+0000","last_fullsized":"2024-04-16T05:35:17.140426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:21.767071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934854+0000","last_change":"2024-04-16T05:32:55.514617+0000","last_active":"2024-04-16T05:35:16.934854+0000","last_peered":"2024-04-16T05:35:16.934854+0000","last_clean":"2024-04-16T05:35:16.934854+0000","last_became_active":"2024-04-16T05:32:55.512660+0000","last_became_peered":"2024-04-16T05:32:55.512660+0000","last_unstale":"2024-04-16T05:35:16.934854+0000","last_undegraded":"2024-04-16T05:35:16.934854+0000","last_fullsized":"2024-04-16T05:35:16.934854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:31:45.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140378+0000","last_change":"2024-04-16T05:32:54.498650+0000","last_active":"2024-04-16T05:35:17.140378+0000","last_peered":"2024-04-16T05:35:17.140378+0000","last_clean":"2024-04-16T05:35:17.140378+0000","last_became_active":"2024-04-16T05:32:54.497672+0000","last_became_peered":"2024-04-16T05:32:54.497672+0000","last_unstale":"2024-04-16T05:35:17.140378+0000","last_undegraded":"2024-04-16T05:35:17.140378+0000","last_fullsized":"2024-04-16T05:35:17.140378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:36.908809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":369,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139698+0000","last_change":"2024-04-16T05:31:42.831800+0000","last_active":"2024-04-16T05:35:17.139698+0000","last_peered":"2024-04-16T05:35:17.139698+0000","last_clean":"2024-04-16T05:35:17.139698+0000","last_became_active":"2024-04-16T05:31:42.831391+0000","last_became_peered":"2024-04-16T05:31:42.831391+0000","last_unstale":"2024-04-16T05:35:17.139698+0000","last_undegraded":"2024-04-16T05:35:17.139698+0000","last_fullsized":"2024-04-16T05:35:17.139698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_clean_scrub_stamp":"2024-04-16T05:28:15.969690+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.152803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935901+0000","last_change":"2024-04-16T05:33:52.675549+0000","last_active":"2024-04-16T05:35:16.935901+0000","last_peered":"2024-04-16T05:35:16.935901+0000","last_clean":"2024-04-16T05:35:16.935901+0000","last_became_active":"2024-04-16T05:33:52.674969+0000","last_became_peered":"2024-04-16T05:33:52.674969+0000","last_unstale":"2024-04-16T05:35:16.935901+0000","last_undegraded":"2024-04-16T05:35:16.935901+0000","last_fullsized":"2024-04-16T05:35:16.935901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:58:33.137841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139786+0000","last_change":"2024-04-16T05:32:55.512324+0000","last_active":"2024-04-16T05:35:17.139786+0000","last_peered":"2024-04-16T05:35:17.139786+0000","last_clean":"2024-04-16T05:35:17.139786+0000","last_became_active":"2024-04-16T05:32:55.511958+0000","last_became_peered":"2024-04-16T05:32:55.511958+0000","last_unstale":"2024-04-16T05:35:17.139786+0000","last_undegraded":"2024-04-16T05:35:17.139786+0000","last_fullsized":"2024-04-16T05:35:17.139786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:38.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139804+0000","last_change":"2024-04-16T05:32:54.508021+0000","last_active":"2024-04-16T05:35:17.139804+0000","last_peered":"2024-04-16T05:35:17.139804+0000","last_clean":"2024-04-16T05:35:17.139804+0000","last_became_active":"2024-04-16T05:32:54.507645+0000","last_became_peered":"2024-04-16T05:32:54.507645+0000","last_unstale":"2024-04-16T05:35:17.139804+0000","last_undegraded":"2024-04-16T05:35:17.139804+0000","last_fullsized":"2024-04-16T05:35:17.139804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:21:50.721260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140448+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:35:17.140448+0000","last_peered":"2024-04-16T05:35:17.140448+0000","last_clean":"2024-04-16T05:35:17.140448+0000","last_became_active":"2024-04-16T05:32:59.523159+0000","last_became_peered":"2024-04-16T05:32:59.523159+0000","last_unstale":"2024-04-16T05:35:17.140448+0000","last_undegraded":"2024-04-16T05:35:17.140448+0000","last_fullsized":"2024-04-16T05:35:17.140448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:48:37.696713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935875+0000","last_change":"2024-04-16T05:32:54.503262+0000","last_active":"2024-04-16T05:35:16.935875+0000","last_peered":"2024-04-16T05:35:16.935875+0000","last_clean":"2024-04-16T05:35:16.935875+0000","last_became_active":"2024-04-16T05:32:54.502909+0000","last_became_peered":"2024-04-16T05:32:54.502909+0000","last_unstale":"2024-04-16T05:35:16.935875+0000","last_undegraded":"2024-04-16T05:35:16.935875+0000","last_fullsized":"2024-04-16T05:35:16.935875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:48:32.841921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141260+0000","last_change":"2024-04-16T05:32:55.507126+0000","last_active":"2024-04-16T05:35:17.141260+0000","last_peered":"2024-04-16T05:35:17.141260+0000","last_clean":"2024-04-16T05:35:17.141260+0000","last_became_active":"2024-04-16T05:32:55.506743+0000","last_became_peered":"2024-04-16T05:32:55.506743+0000","last_unstale":"2024-04-16T05:35:17.141260+0000","last_undegraded":"2024-04-16T05:35:17.141260+0000","last_fullsized":"2024-04-16T05:35:17.141260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:10:58.697649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934575+0000","last_change":"2024-04-16T05:33:00.419435+0000","last_active":"2024-04-16T05:35:16.934575+0000","last_peered":"2024-04-16T05:35:16.934575+0000","last_clean":"2024-04-16T05:35:16.934575+0000","last_became_active":"2024-04-16T05:33:00.418985+0000","last_became_peered":"2024-04-16T05:33:00.418985+0000","last_unstale":"2024-04-16T05:35:16.934575+0000","last_undegraded":"2024-04-16T05:35:16.934575+0000","last_fullsized":"2024-04-16T05:35:16.934575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:08:31.188911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'48","reported_seq":107,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.144790+0000","last_change":"2024-04-16T05:32:54.503349+0000","last_active":"2024-04-16T05:35:17.144790+0000","last_peered":"2024-04-16T05:35:17.144790+0000","last_clean":"2024-04-16T05:35:17.144790+0000","last_became_active":"2024-04-16T05:32:54.503018+0000","last_became_peered":"2024-04-16T05:32:54.503018+0000","last_unstale":"2024-04-16T05:35:17.144790+0000","last_undegraded":"2024-04-16T05:35:17.144790+0000","last_fullsized":"2024-04-16T05:35:17.144790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:34:20.853543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935262+0000","last_change":"2024-04-16T05:32:55.511201+0000","last_active":"2024-04-16T05:35:16.935262+0000","last_peered":"2024-04-16T05:35:16.935262+0000","last_clean":"2024-04-16T05:35:16.935262+0000","last_became_active":"2024-04-16T05:32:55.510906+0000","last_became_peered":"2024-04-16T05:32:55.510906+0000","last_unstale":"2024-04-16T05:35:16.935262+0000","last_undegraded":"2024-04-16T05:35:16.935262+0000","last_fullsized":"2024-04-16T05:35:16.935262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:37:37.147628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139081+0000","last_change":"2024-04-16T05:32:59.518440+0000","last_active":"2024-04-16T05:35:17.139081+0000","last_peered":"2024-04-16T05:35:17.139081+0000","last_clean":"2024-04-16T05:35:17.139081+0000","last_became_active":"2024-04-16T05:32:59.517870+0000","last_became_peered":"2024-04-16T05:32:59.517870+0000","last_unstale":"2024-04-16T05:35:17.139081+0000","last_undegraded":"2024-04-16T05:35:17.139081+0000","last_fullsized":"2024-04-16T05:35:17.139081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:51:31.157418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140365+0000","last_change":"2024-04-16T05:32:54.503983+0000","last_active":"2024-04-16T05:35:17.140365+0000","last_peered":"2024-04-16T05:35:17.140365+0000","last_clean":"2024-04-16T05:35:17.140365+0000","last_became_active":"2024-04-16T05:32:54.502367+0000","last_became_peered":"2024-04-16T05:32:54.502367+0000","last_unstale":"2024-04-16T05:35:17.140365+0000","last_undegraded":"2024-04-16T05:35:17.140365+0000","last_fullsized":"2024-04-16T05:35:17.140365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:49:36.997692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140112+0000","last_change":"2024-04-16T05:32:55.516812+0000","last_active":"2024-04-16T05:35:17.140112+0000","last_peered":"2024-04-16T05:35:17.140112+0000","last_clean":"2024-04-16T05:35:17.140112+0000","last_became_active":"2024-04-16T05:32:55.516477+0000","last_became_peered":"2024-04-16T05:32:55.516477+0000","last_unstale":"2024-04-16T05:35:17.140112+0000","last_undegraded":"2024-04-16T05:35:17.140112+0000","last_fullsized":"2024-04-16T05:35:17.140112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:24:29.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140029+0000","last_change":"2024-04-16T05:33:00.419622+0000","last_active":"2024-04-16T05:35:17.140029+0000","last_peered":"2024-04-16T05:35:17.140029+0000","last_clean":"2024-04-16T05:35:17.140029+0000","last_became_active":"2024-04-16T05:33:00.419135+0000","last_became_peered":"2024-04-16T05:33:00.419135+0000","last_unstale":"2024-04-16T05:35:17.140029+0000","last_undegraded":"2024-04-16T05:35:17.140029+0000","last_fullsized":"2024-04-16T05:35:17.140029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:38:02.529192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.138987+0000","last_change":"2024-04-16T05:32:54.503652+0000","last_active":"2024-04-16T05:35:17.138987+0000","last_peered":"2024-04-16T05:35:17.138987+0000","last_clean":"2024-04-16T05:35:17.138987+0000","last_became_active":"2024-04-16T05:32:54.503311+0000","last_became_peered":"2024-04-16T05:32:54.503311+0000","last_unstale":"2024-04-16T05:35:17.138987+0000","last_undegraded":"2024-04-16T05:35:17.138987+0000","last_fullsized":"2024-04-16T05:35:17.138987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:49:21.584029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935388+0000","last_change":"2024-04-16T05:32:55.508145+0000","last_active":"2024-04-16T05:35:16.935388+0000","last_peered":"2024-04-16T05:35:16.935388+0000","last_clean":"2024-04-16T05:35:16.935388+0000","last_became_active":"2024-04-16T05:32:55.507794+0000","last_became_peered":"2024-04-16T05:32:55.507794+0000","last_unstale":"2024-04-16T05:35:16.935388+0000","last_undegraded":"2024-04-16T05:35:16.935388+0000","last_fullsized":"2024-04-16T05:35:16.935388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:11:57.925733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.936185+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:35:16.936185+0000","last_peered":"2024-04-16T05:35:16.936185+0000","last_clean":"2024-04-16T05:35:16.936185+0000","last_became_active":"2024-04-16T05:33:52.678330+0000","last_became_peered":"2024-04-16T05:33:52.678330+0000","last_unstale":"2024-04-16T05:35:16.936185+0000","last_undegraded":"2024-04-16T05:35:16.936185+0000","last_fullsized":"2024-04-16T05:35:16.936185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:33.323513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934764+0000","last_change":"2024-04-16T05:32:54.500909+0000","last_active":"2024-04-16T05:35:16.934764+0000","last_peered":"2024-04-16T05:35:16.934764+0000","last_clean":"2024-04-16T05:35:16.934764+0000","last_became_active":"2024-04-16T05:32:54.499348+0000","last_became_peered":"2024-04-16T05:32:54.499348+0000","last_unstale":"2024-04-16T05:35:16.934764+0000","last_undegraded":"2024-04-16T05:35:16.934764+0000","last_fullsized":"2024-04-16T05:35:16.934764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:51:49.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140151+0000","last_change":"2024-04-16T05:32:55.510922+0000","last_active":"2024-04-16T05:35:17.140151+0000","last_peered":"2024-04-16T05:35:17.140151+0000","last_clean":"2024-04-16T05:35:17.140151+0000","last_became_active":"2024-04-16T05:32:55.510546+0000","last_became_peered":"2024-04-16T05:32:55.510546+0000","last_unstale":"2024-04-16T05:35:17.140151+0000","last_undegraded":"2024-04-16T05:35:17.140151+0000","last_fullsized":"2024-04-16T05:35:17.140151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:27.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139878+0000","last_change":"2024-04-16T05:32:59.524257+0000","last_active":"2024-04-16T05:35:17.139878+0000","last_peered":"2024-04-16T05:35:17.139878+0000","last_clean":"2024-04-16T05:35:17.139878+0000","last_became_active":"2024-04-16T05:32:59.523768+0000","last_became_peered":"2024-04-16T05:32:59.523768+0000","last_unstale":"2024-04-16T05:35:17.139878+0000","last_undegraded":"2024-04-16T05:35:17.139878+0000","last_fullsized":"2024-04-16T05:35:17.139878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:38:27.594048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'5","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140663+0000","last_change":"2024-04-16T05:34:52.631768+0000","last_active":"2024-04-16T05:35:17.140663+0000","last_peered":"2024-04-16T05:35:17.140663+0000","last_clean":"2024-04-16T05:35:17.140663+0000","last_became_active":"2024-04-16T05:34:52.627168+0000","last_became_peered":"2024-04-16T05:34:52.627168+0000","last_unstale":"2024-04-16T05:35:17.140663+0000","last_undegraded":"2024-04-16T05:35:17.140663+0000","last_fullsized":"2024-04-16T05:35:17.140663+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:46.866673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935117+0000","last_change":"2024-04-16T05:32:55.508005+0000","last_active":"2024-04-16T05:35:16.935117+0000","last_peered":"2024-04-16T05:35:16.935117+0000","last_clean":"2024-04-16T05:35:16.935117+0000","last_became_active":"2024-04-16T05:32:55.507698+0000","last_became_peered":"2024-04-16T05:32:55.507698+0000","last_unstale":"2024-04-16T05:35:16.935117+0000","last_undegraded":"2024-04-16T05:35:16.935117+0000","last_fullsized":"2024-04-16T05:35:16.935117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:11:12.423294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140202+0000","last_change":"2024-04-16T05:33:00.419442+0000","last_active":"2024-04-16T05:35:17.140202+0000","last_peered":"2024-04-16T05:35:17.140202+0000","last_clean":"2024-04-16T05:35:17.140202+0000","last_became_active":"2024-04-16T05:33:00.418906+0000","last_became_peered":"2024-04-16T05:33:00.418906+0000","last_unstale":"2024-04-16T05:35:17.140202+0000","last_undegraded":"2024-04-16T05:35:17.140202+0000","last_fullsized":"2024-04-16T05:35:17.140202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:21:22.124879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935760+0000","last_change":"2024-04-16T05:32:54.509090+0000","last_active":"2024-04-16T05:35:16.935760+0000","last_peered":"2024-04-16T05:35:16.935760+0000","last_clean":"2024-04-16T05:35:16.935760+0000","last_became_active":"2024-04-16T05:32:54.508700+0000","last_became_peered":"2024-04-16T05:32:54.508700+0000","last_unstale":"2024-04-16T05:35:16.935760+0000","last_undegraded":"2024-04-16T05:35:16.935760+0000","last_fullsized":"2024-04-16T05:35:16.935760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:53:34.344073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139999+0000","last_change":"2024-04-16T05:32:55.509658+0000","last_active":"2024-04-16T05:35:17.139999+0000","last_peered":"2024-04-16T05:35:17.139999+0000","last_clean":"2024-04-16T05:35:17.139999+0000","last_became_active":"2024-04-16T05:32:55.509258+0000","last_became_peered":"2024-04-16T05:32:55.509258+0000","last_unstale":"2024-04-16T05:35:17.139999+0000","last_undegraded":"2024-04-16T05:35:17.139999+0000","last_fullsized":"2024-04-16T05:35:17.139999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:17:03.490544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139041+0000","last_change":"2024-04-16T05:32:59.519260+0000","last_active":"2024-04-16T05:35:17.139041+0000","last_peered":"2024-04-16T05:35:17.139041+0000","last_clean":"2024-04-16T05:35:17.139041+0000","last_became_active":"2024-04-16T05:32:59.518839+0000","last_became_peered":"2024-04-16T05:32:59.518839+0000","last_unstale":"2024-04-16T05:35:17.139041+0000","last_undegraded":"2024-04-16T05:35:17.139041+0000","last_fullsized":"2024-04-16T05:35:17.139041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:54:50.555214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787320+0000","last_change":"2024-04-16T05:32:54.503226+0000","last_active":"2024-04-16T05:35:17.787320+0000","last_peered":"2024-04-16T05:35:17.787320+0000","last_clean":"2024-04-16T05:35:17.787320+0000","last_became_active":"2024-04-16T05:32:54.502687+0000","last_became_peered":"2024-04-16T05:32:54.502687+0000","last_unstale":"2024-04-16T05:35:17.787320+0000","last_undegraded":"2024-04-16T05:35:17.787320+0000","last_fullsized":"2024-04-16T05:35:17.787320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:34:24.228464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140830+0000","last_change":"2024-04-16T05:32:55.515203+0000","last_active":"2024-04-16T05:35:17.140830+0000","last_peered":"2024-04-16T05:35:17.140830+0000","last_clean":"2024-04-16T05:35:17.140830+0000","last_became_active":"2024-04-16T05:32:55.514815+0000","last_became_peered":"2024-04-16T05:32:55.514815+0000","last_unstale":"2024-04-16T05:35:17.140830+0000","last_undegraded":"2024-04-16T05:35:17.140830+0000","last_fullsized":"2024-04-16T05:35:17.140830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:11:18.464532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139998+0000","last_change":"2024-04-16T05:32:59.520442+0000","last_active":"2024-04-16T05:35:17.139998+0000","last_peered":"2024-04-16T05:35:17.139998+0000","last_clean":"2024-04-16T05:35:17.139998+0000","last_became_active":"2024-04-16T05:32:59.520071+0000","last_became_peered":"2024-04-16T05:32:59.520071+0000","last_unstale":"2024-04-16T05:35:17.139998+0000","last_undegraded":"2024-04-16T05:35:17.139998+0000","last_fullsized":"2024-04-16T05:35:17.139998+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:19:16.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":84,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.142330+0000","last_change":"2024-04-16T05:32:54.510098+0000","last_active":"2024-04-16T05:35:17.142330+0000","last_peered":"2024-04-16T05:35:17.142330+0000","last_clean":"2024-04-16T05:35:17.142330+0000","last_became_active":"2024-04-16T05:32:54.509750+0000","last_became_peered":"2024-04-16T05:32:54.509750+0000","last_unstale":"2024-04-16T05:35:17.142330+0000","last_undegraded":"2024-04-16T05:35:17.142330+0000","last_fullsized":"2024-04-16T05:35:17.142330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:00:09.528958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140257+0000","last_change":"2024-04-16T05:33:52.678823+0000","last_active":"2024-04-16T05:35:17.140257+0000","last_peered":"2024-04-16T05:35:17.140257+0000","last_clean":"2024-04-16T05:35:17.140257+0000","last_became_active":"2024-04-16T05:33:52.678276+0000","last_became_peered":"2024-04-16T05:33:52.678276+0000","last_unstale":"2024-04-16T05:35:17.140257+0000","last_undegraded":"2024-04-16T05:35:17.140257+0000","last_fullsized":"2024-04-16T05:35:17.140257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:58:33.186382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040472+0000","last_change":"2024-04-16T05:32:59.521334+0000","last_active":"2024-04-16T05:35:17.040472+0000","last_peered":"2024-04-16T05:35:17.040472+0000","last_clean":"2024-04-16T05:35:17.040472+0000","last_became_active":"2024-04-16T05:32:59.520348+0000","last_became_peered":"2024-04-16T05:32:59.520348+0000","last_unstale":"2024-04-16T05:35:17.040472+0000","last_undegraded":"2024-04-16T05:35:17.040472+0000","last_fullsized":"2024-04-16T05:35:17.040472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:40.628552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141843+0000","last_change":"2024-04-16T05:32:54.503062+0000","last_active":"2024-04-16T05:35:17.141843+0000","last_peered":"2024-04-16T05:35:17.141843+0000","last_clean":"2024-04-16T05:35:17.141843+0000","last_became_active":"2024-04-16T05:32:54.502762+0000","last_became_peered":"2024-04-16T05:32:54.502762+0000","last_unstale":"2024-04-16T05:35:17.141843+0000","last_undegraded":"2024-04-16T05:35:17.141843+0000","last_fullsized":"2024-04-16T05:35:17.141843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:43:57.911296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141355+0000","last_change":"2024-04-16T05:32:55.510461+0000","last_active":"2024-04-16T05:35:17.141355+0000","last_peered":"2024-04-16T05:35:17.141355+0000","last_clean":"2024-04-16T05:35:17.141355+0000","last_became_active":"2024-04-16T05:32:55.510142+0000","last_became_peered":"2024-04-16T05:32:55.510142+0000","last_unstale":"2024-04-16T05:35:17.141355+0000","last_undegraded":"2024-04-16T05:35:17.141355+0000","last_fullsized":"2024-04-16T05:35:17.141355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:41.346078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140171+0000","last_change":"2024-04-16T05:32:55.512600+0000","last_active":"2024-04-16T05:35:17.140171+0000","last_peered":"2024-04-16T05:35:17.140171+0000","last_clean":"2024-04-16T05:35:17.140171+0000","last_became_active":"2024-04-16T05:32:55.512265+0000","last_became_peered":"2024-04-16T05:32:55.512265+0000","last_unstale":"2024-04-16T05:35:17.140171+0000","last_undegraded":"2024-04-16T05:35:17.140171+0000","last_fullsized":"2024-04-16T05:35:17.140171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:04:46.112644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139670+0000","last_change":"2024-04-16T05:32:55.514481+0000","last_active":"2024-04-16T05:35:17.139670+0000","last_peered":"2024-04-16T05:35:17.139670+0000","last_clean":"2024-04-16T05:35:17.139670+0000","last_became_active":"2024-04-16T05:32:55.512485+0000","last_became_peered":"2024-04-16T05:32:55.512485+0000","last_unstale":"2024-04-16T05:35:17.139670+0000","last_undegraded":"2024-04-16T05:35:17.139670+0000","last_fullsized":"2024-04-16T05:35:17.139670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:33:02.179774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139422+0000","last_change":"2024-04-16T05:32:55.509849+0000","last_active":"2024-04-16T05:35:17.139422+0000","last_peered":"2024-04-16T05:35:17.139422+0000","last_clean":"2024-04-16T05:35:17.139422+0000","last_became_active":"2024-04-16T05:32:55.509338+0000","last_became_peered":"2024-04-16T05:32:55.509338+0000","last_unstale":"2024-04-16T05:35:17.139422+0000","last_undegraded":"2024-04-16T05:35:17.139422+0000","last_fullsized":"2024-04-16T05:35:17.139422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:32:38.900455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140163+0000","last_change":"2024-04-16T05:32:55.518556+0000","last_active":"2024-04-16T05:35:17.140163+0000","last_peered":"2024-04-16T05:35:17.140163+0000","last_clean":"2024-04-16T05:35:17.140163+0000","last_became_active":"2024-04-16T05:32:55.518240+0000","last_became_peered":"2024-04-16T05:32:55.518240+0000","last_unstale":"2024-04-16T05:35:17.140163+0000","last_undegraded":"2024-04-16T05:35:17.140163+0000","last_fullsized":"2024-04-16T05:35:17.140163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:25:46.002516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141244+0000","last_change":"2024-04-16T05:32:55.518106+0000","last_active":"2024-04-16T05:35:17.141244+0000","last_peered":"2024-04-16T05:35:17.141244+0000","last_clean":"2024-04-16T05:35:17.141244+0000","last_became_active":"2024-04-16T05:32:55.517403+0000","last_became_peered":"2024-04-16T05:32:55.517403+0000","last_unstale":"2024-04-16T05:35:17.141244+0000","last_undegraded":"2024-04-16T05:35:17.141244+0000","last_fullsized":"2024-04-16T05:35:17.141244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:36:34.731644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140630+0000","last_change":"2024-04-16T05:33:52.581236+0000","last_active":"2024-04-16T05:35:17.140630+0000","last_peered":"2024-04-16T05:35:17.140630+0000","last_clean":"2024-04-16T05:35:17.140630+0000","last_became_active":"2024-04-16T05:33:52.580748+0000","last_became_peered":"2024-04-16T05:33:52.580748+0000","last_unstale":"2024-04-16T05:35:17.140630+0000","last_undegraded":"2024-04-16T05:35:17.140630+0000","last_fullsized":"2024-04-16T05:35:17.140630+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:13:33.382776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139982+0000","last_change":"2024-04-16T05:32:55.507670+0000","last_active":"2024-04-16T05:35:17.139982+0000","last_peered":"2024-04-16T05:35:17.139982+0000","last_clean":"2024-04-16T05:35:17.139982+0000","last_became_active":"2024-04-16T05:32:55.506672+0000","last_became_peered":"2024-04-16T05:32:55.506672+0000","last_unstale":"2024-04-16T05:35:17.139982+0000","last_undegraded":"2024-04-16T05:35:17.139982+0000","last_fullsized":"2024-04-16T05:35:17.139982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:54.579672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935321+0000","last_change":"2024-04-16T05:32:55.516987+0000","last_active":"2024-04-16T05:35:16.935321+0000","last_peered":"2024-04-16T05:35:16.935321+0000","last_clean":"2024-04-16T05:35:16.935321+0000","last_became_active":"2024-04-16T05:32:55.516654+0000","last_became_peered":"2024-04-16T05:32:55.516654+0000","last_unstale":"2024-04-16T05:35:16.935321+0000","last_undegraded":"2024-04-16T05:35:16.935321+0000","last_fullsized":"2024-04-16T05:35:16.935321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:32:42.957658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040684+0000","last_change":"2024-04-16T05:32:55.506539+0000","last_active":"2024-04-16T05:35:17.040684+0000","last_peered":"2024-04-16T05:35:17.040684+0000","last_clean":"2024-04-16T05:35:17.040684+0000","last_became_active":"2024-04-16T05:32:55.503658+0000","last_became_peered":"2024-04-16T05:32:55.503658+0000","last_unstale":"2024-04-16T05:35:17.040684+0000","last_undegraded":"2024-04-16T05:35:17.040684+0000","last_fullsized":"2024-04-16T05:35:17.040684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:22.116213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934725+0000","last_change":"2024-04-16T05:32:55.507363+0000","last_active":"2024-04-16T05:35:16.934725+0000","last_peered":"2024-04-16T05:35:16.934725+0000","last_clean":"2024-04-16T05:35:16.934725+0000","last_became_active":"2024-04-16T05:32:55.506923+0000","last_became_peered":"2024-04-16T05:32:55.506923+0000","last_unstale":"2024-04-16T05:35:16.934725+0000","last_undegraded":"2024-04-16T05:35:16.934725+0000","last_fullsized":"2024-04-16T05:35:16.934725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:19:34.356834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139209+0000","last_change":"2024-04-16T05:32:55.508531+0000","last_active":"2024-04-16T05:35:17.139209+0000","last_peered":"2024-04-16T05:35:17.139209+0000","last_clean":"2024-04-16T05:35:17.139209+0000","last_became_active":"2024-04-16T05:32:55.508177+0000","last_became_peered":"2024-04-16T05:32:55.508177+0000","last_unstale":"2024-04-16T05:35:17.139209+0000","last_undegraded":"2024-04-16T05:35:17.139209+0000","last_fullsized":"2024-04-16T05:35:17.139209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:49:59.352572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140924+0000","last_change":"2024-04-16T05:32:55.507793+0000","last_active":"2024-04-16T05:35:17.140924+0000","last_peered":"2024-04-16T05:35:17.140924+0000","last_clean":"2024-04-16T05:35:17.140924+0000","last_became_active":"2024-04-16T05:32:55.507443+0000","last_became_peered":"2024-04-16T05:32:55.507443+0000","last_unstale":"2024-04-16T05:35:17.140924+0000","last_undegraded":"2024-04-16T05:35:17.140924+0000","last_fullsized":"2024-04-16T05:35:17.140924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:35.982879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140787+0000","last_change":"2024-04-16T05:32:55.517057+0000","last_active":"2024-04-16T05:35:17.140787+0000","last_peered":"2024-04-16T05:35:17.140787+0000","last_clean":"2024-04-16T05:35:17.140787+0000","last_became_active":"2024-04-16T05:32:55.516696+0000","last_became_peered":"2024-04-16T05:32:55.516696+0000","last_unstale":"2024-04-16T05:35:17.140787+0000","last_undegraded":"2024-04-16T05:35:17.140787+0000","last_fullsized":"2024-04-16T05:35:17.140787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:43:10.391125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106974,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600767,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1336,"num_read_kb":784,"num_write":379,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1802301,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383720,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":540350,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":1188583,"data_compressed":68167,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1105920,"data_stored":936730,"data_compressed":51657,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":932990,"data_compressed":51655,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":1138139,"data_compressed":68167,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":534922,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1126400,"data_stored":602404,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":537218,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":589425,"data_compressed":29531,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":993902,"data_compressed":54666,"data_compressed_allocated":73728,"data_compressed_original":544768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":539919,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1264398,"data_compressed":74323,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135734,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200711,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:35:22.406 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-16T05:35:22.406 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:35:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.548+0000 7f1f99150640 1 -- 172.21.15.86:0/1454555882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101af0 msgr2=0x7f1f94105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.548+0000 7f1f99150640 1 --2- 172.21.15.86:0/1454555882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101af0 0x7f1f94105b30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009900 tx=0x7f1f8802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.548+0000 7f1f99150640 1 -- 172.21.15.86:0/1454555882 shutdown_connections 2024-04-16T05:35:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.548+0000 7f1f99150640 1 --2- 172.21.15.86:0/1454555882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94106210 0x7f1f9410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.548+0000 7f1f99150640 1 --2- 172.21.15.86:0/1454555882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101af0 0x7f1f94105b30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.548+0000 7f1f99150640 1 --2- 172.21.15.86:0/1454555882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94101140 0x7f1f94101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.548+0000 7f1f99150640 1 -- 172.21.15.86:0/1454555882 >> 172.21.15.86:0/1454555882 conn(0x7f1f94077280 msgr2=0x7f1f94077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.549+0000 7f1f99150640 1 -- 172.21.15.86:0/1454555882 shutdown_connections 2024-04-16T05:35:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.549+0000 7f1f99150640 1 -- 172.21.15.86:0/1454555882 wait complete. 2024-04-16T05:35:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.549+0000 7f1f99150640 1 Processor -- start 2024-04-16T05:35:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.549+0000 7f1f99150640 1 -- start start 2024-04-16T05:35:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f99150640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101140 0x7f1f9419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f99150640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94106210 0x7f1f9419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f99150640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94198780 0x7f1f94198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f99150640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94111320 con 0x7f1f94101140 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f99150640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941111a0 con 0x7f1f94198780 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f99150640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941114a0 con 0x7f1f94106210 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f92d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101140 0x7f1f9419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f92d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101140 0x7f1f9419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59522/0 (socket says 172.21.15.86:59522) 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f92d9d640 1 -- 172.21.15.86:0/2820156079 learned_addr learned my addr 172.21.15.86:0/2820156079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f9259c640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94106210 0x7f1f9419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f9359e640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94198780 0x7f1f94198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f92d9d640 1 -- 172.21.15.86:0/2820156079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94106210 msgr2=0x7f1f9419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f92d9d640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94106210 0x7f1f9419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f92d9d640 1 -- 172.21.15.86:0/2820156079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94198780 msgr2=0x7f1f94198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f92d9d640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94198780 0x7f1f94198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f92d9d640 1 -- 172.21.15.86:0/2820156079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f94199310 con 0x7f1f94101140 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f9259c640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94106210 0x7f1f9419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f92d9d640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101140 0x7f1f9419e610 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00cce0 tx=0x7f1f8c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f7bfff640 1 -- 172.21.15.86:0/2820156079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c004b20 con 0x7f1f94101140 2024-04-16T05:35:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.550+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f94199600 con 0x7f1f94101140 2024-04-16T05:35:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.551+0000 7f1f7bfff640 1 -- 172.21.15.86:0/2820156079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8c004cc0 con 0x7f1f94101140 2024-04-16T05:35:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.551+0000 7f1f7bfff640 1 -- 172.21.15.86:0/2820156079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00f8c0 con 0x7f1f94101140 2024-04-16T05:35:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.551+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f941a57e0 con 0x7f1f94101140 2024-04-16T05:35:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.551+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f94102610 con 0x7f1f94101140 2024-04-16T05:35:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.553+0000 7f1f7bfff640 1 -- 172.21.15.86:0/2820156079 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f1f8c00fa60 con 0x7f1f94101140 2024-04-16T05:35:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.553+0000 7f1f7bfff640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1f7c0a3e40 0x7f1f7c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.553+0000 7f1f9259c640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1f7c0a3e40 0x7f1f7c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.553+0000 7f1f7bfff640 1 -- 172.21.15.86:0/2820156079 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1f8c0d0fd0 con 0x7f1f94101140 2024-04-16T05:35:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.554+0000 7f1f9259c640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1f7c0a3e40 0x7f1f7c0a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1f88006010 tx=0x7f1f88000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:22.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.558+0000 7f1f7bfff640 1 -- 172.21.15.86:0/2820156079 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f94199600 con 0x7f1f94101140 2024-04-16T05:35:22.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:22 smithi086 ceph-mon[32028]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:22 smithi086 ceph-mon[32028]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:22 smithi086 ceph-mon[32028]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:22 smithi086 ceph-mon[32028]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:22 smithi086 ceph-mon[32028]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:22 smithi086 ceph-mon[32028]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:22 smithi086 ceph-mon[32028]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:22 smithi086 ceph-mon[32028]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T05:35:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:22 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/311761915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:22.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.735+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f94101bc0 con 0x7f1f7c0a3e40 2024-04-16T05:35:22.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.744+0000 7f1f7bfff640 1 -- 172.21.15.86:0/2820156079 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339713 (secure 0 0 0) 0x7f1f94101bc0 con 0x7f1f7c0a3e40 2024-04-16T05:35:22.746 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-16T05:35:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1f7c0a3e40 msgr2=0x7f1f7c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1f7c0a3e40 0x7f1f7c0a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1f88006010 tx=0x7f1f88000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101140 msgr2=0x7f1f9419e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101140 0x7f1f9419e610 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00cce0 tx=0x7f1f8c00a620 comp rx=0 tx=0).stop 2024-04-16T05:35:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 shutdown_connections 2024-04-16T05:35:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1f7c0a3e40 0x7f1f7c0a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94198780 0x7f1f94198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94106210 0x7f1f9419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 --2- 172.21.15.86:0/2820156079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f94101140 0x7f1f9419e610 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 >> 172.21.15.86:0/2820156079 conn(0x7f1f94077280 msgr2=0x7f1f94104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.747+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 shutdown_connections 2024-04-16T05:35:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.748+0000 7f1f99150640 1 -- 172.21.15.86:0/2820156079 wait complete. 2024-04-16T05:35:22.757 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-16T05:35:20.947179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191369,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1472,"num_read_kb":903,"num_write":562,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370484,"kb_used_data":12972,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571340,"statfs":{"total":1151940427776,"available":1151561052160,"internally_reserved":0,"allocated":13283328,"data_stored":9798980,"data_compressed":560711,"data_compressed_allocated":757760,"data_compressed_original":4206592,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987345"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140922+0000","last_change":"2024-04-16T05:32:55.507548+0000","last_active":"2024-04-16T05:35:17.140922+0000","last_peered":"2024-04-16T05:35:17.140922+0000","last_clean":"2024-04-16T05:35:17.140922+0000","last_became_active":"2024-04-16T05:32:55.507200+0000","last_became_peered":"2024-04-16T05:32:55.507200+0000","last_unstale":"2024-04-16T05:35:17.140922+0000","last_undegraded":"2024-04-16T05:35:17.140922+0000","last_fullsized":"2024-04-16T05:35:17.140922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:58:07.842226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139986+0000","last_change":"2024-04-16T05:32:55.509792+0000","last_active":"2024-04-16T05:35:17.139986+0000","last_peered":"2024-04-16T05:35:17.139986+0000","last_clean":"2024-04-16T05:35:17.139986+0000","last_became_active":"2024-04-16T05:32:55.509442+0000","last_became_peered":"2024-04-16T05:32:55.509442+0000","last_unstale":"2024-04-16T05:35:17.139986+0000","last_undegraded":"2024-04-16T05:35:17.139986+0000","last_fullsized":"2024-04-16T05:35:17.139986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:08:42.040875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140224+0000","last_change":"2024-04-16T05:32:55.505664+0000","last_active":"2024-04-16T05:35:17.140224+0000","last_peered":"2024-04-16T05:35:17.140224+0000","last_clean":"2024-04-16T05:35:17.140224+0000","last_became_active":"2024-04-16T05:32:55.505281+0000","last_became_peered":"2024-04-16T05:32:55.505281+0000","last_unstale":"2024-04-16T05:35:17.140224+0000","last_undegraded":"2024-04-16T05:35:17.140224+0000","last_fullsized":"2024-04-16T05:35:17.140224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:33.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139993+0000","last_change":"2024-04-16T05:32:55.517072+0000","last_active":"2024-04-16T05:35:17.139993+0000","last_peered":"2024-04-16T05:35:17.139993+0000","last_clean":"2024-04-16T05:35:17.139993+0000","last_became_active":"2024-04-16T05:32:55.516648+0000","last_became_peered":"2024-04-16T05:32:55.516648+0000","last_unstale":"2024-04-16T05:35:17.139993+0000","last_undegraded":"2024-04-16T05:35:17.139993+0000","last_fullsized":"2024-04-16T05:35:17.139993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:41.735720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935557+0000","last_change":"2024-04-16T05:32:55.506773+0000","last_active":"2024-04-16T05:35:16.935557+0000","last_peered":"2024-04-16T05:35:16.935557+0000","last_clean":"2024-04-16T05:35:16.935557+0000","last_became_active":"2024-04-16T05:32:55.506414+0000","last_became_peered":"2024-04-16T05:32:55.506414+0000","last_unstale":"2024-04-16T05:35:16.935557+0000","last_undegraded":"2024-04-16T05:35:16.935557+0000","last_fullsized":"2024-04-16T05:35:16.935557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:41:59.254334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935461+0000","last_change":"2024-04-16T05:32:55.517875+0000","last_active":"2024-04-16T05:35:16.935461+0000","last_peered":"2024-04-16T05:35:16.935461+0000","last_clean":"2024-04-16T05:35:16.935461+0000","last_became_active":"2024-04-16T05:32:55.517546+0000","last_became_peered":"2024-04-16T05:32:55.517546+0000","last_unstale":"2024-04-16T05:35:16.935461+0000","last_undegraded":"2024-04-16T05:35:16.935461+0000","last_fullsized":"2024-04-16T05:35:16.935461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:12:15.098439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139958+0000","last_change":"2024-04-16T05:32:55.509036+0000","last_active":"2024-04-16T05:35:17.139958+0000","last_peered":"2024-04-16T05:35:17.139958+0000","last_clean":"2024-04-16T05:35:17.139958+0000","last_became_active":"2024-04-16T05:32:55.508690+0000","last_became_peered":"2024-04-16T05:32:55.508690+0000","last_unstale":"2024-04-16T05:35:17.139958+0000","last_undegraded":"2024-04-16T05:35:17.139958+0000","last_fullsized":"2024-04-16T05:35:17.139958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:26.724281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934682+0000","last_change":"2024-04-16T05:32:55.510146+0000","last_active":"2024-04-16T05:35:16.934682+0000","last_peered":"2024-04-16T05:35:16.934682+0000","last_clean":"2024-04-16T05:35:16.934682+0000","last_became_active":"2024-04-16T05:32:55.508665+0000","last_became_peered":"2024-04-16T05:32:55.508665+0000","last_unstale":"2024-04-16T05:35:16.934682+0000","last_undegraded":"2024-04-16T05:35:16.934682+0000","last_fullsized":"2024-04-16T05:35:16.934682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:14.158221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140557+0000","last_change":"2024-04-16T05:32:55.513738+0000","last_active":"2024-04-16T05:35:17.140557+0000","last_peered":"2024-04-16T05:35:17.140557+0000","last_clean":"2024-04-16T05:35:17.140557+0000","last_became_active":"2024-04-16T05:32:55.513374+0000","last_became_peered":"2024-04-16T05:32:55.513374+0000","last_unstale":"2024-04-16T05:35:17.140557+0000","last_undegraded":"2024-04-16T05:35:17.140557+0000","last_fullsized":"2024-04-16T05:35:17.140557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:06:23.390908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935707+0000","last_change":"2024-04-16T05:32:55.519276+0000","last_active":"2024-04-16T05:35:16.935707+0000","last_peered":"2024-04-16T05:35:16.935707+0000","last_clean":"2024-04-16T05:35:16.935707+0000","last_became_active":"2024-04-16T05:32:55.518729+0000","last_became_peered":"2024-04-16T05:32:55.518729+0000","last_unstale":"2024-04-16T05:35:16.935707+0000","last_undegraded":"2024-04-16T05:35:16.935707+0000","last_fullsized":"2024-04-16T05:35:16.935707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:44:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140646+0000","last_change":"2024-04-16T05:32:55.516379+0000","last_active":"2024-04-16T05:35:17.140646+0000","last_peered":"2024-04-16T05:35:17.140646+0000","last_clean":"2024-04-16T05:35:17.140646+0000","last_became_active":"2024-04-16T05:32:55.515574+0000","last_became_peered":"2024-04-16T05:32:55.515574+0000","last_unstale":"2024-04-16T05:35:17.140646+0000","last_undegraded":"2024-04-16T05:35:17.140646+0000","last_fullsized":"2024-04-16T05:35:17.140646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:16:01.744380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032219+0000","last_change":"2024-04-16T05:33:52.674399+0000","last_active":"2024-04-16T05:35:17.032219+0000","last_peered":"2024-04-16T05:35:17.032219+0000","last_clean":"2024-04-16T05:35:17.032219+0000","last_became_active":"2024-04-16T05:33:52.673964+0000","last_became_peered":"2024-04-16T05:33:52.673964+0000","last_unstale":"2024-04-16T05:35:17.032219+0000","last_undegraded":"2024-04-16T05:35:17.032219+0000","last_fullsized":"2024-04-16T05:35:17.032219+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.166747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141154+0000","last_change":"2024-04-16T05:32:55.509682+0000","last_active":"2024-04-16T05:35:17.141154+0000","last_peered":"2024-04-16T05:35:17.141154+0000","last_clean":"2024-04-16T05:35:17.141154+0000","last_became_active":"2024-04-16T05:32:55.509383+0000","last_became_peered":"2024-04-16T05:32:55.509383+0000","last_unstale":"2024-04-16T05:35:17.141154+0000","last_undegraded":"2024-04-16T05:35:17.141154+0000","last_fullsized":"2024-04-16T05:35:17.141154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:25:23.487003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139784+0000","last_change":"2024-04-16T05:32:55.506338+0000","last_active":"2024-04-16T05:35:17.139784+0000","last_peered":"2024-04-16T05:35:17.139784+0000","last_clean":"2024-04-16T05:35:17.139784+0000","last_became_active":"2024-04-16T05:32:55.506000+0000","last_became_peered":"2024-04-16T05:32:55.506000+0000","last_unstale":"2024-04-16T05:35:17.139784+0000","last_undegraded":"2024-04-16T05:35:17.139784+0000","last_fullsized":"2024-04-16T05:35:17.139784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:37:02.296515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139777+0000","last_change":"2024-04-16T05:32:55.516897+0000","last_active":"2024-04-16T05:35:17.139777+0000","last_peered":"2024-04-16T05:35:17.139777+0000","last_clean":"2024-04-16T05:35:17.139777+0000","last_became_active":"2024-04-16T05:32:55.516489+0000","last_became_peered":"2024-04-16T05:32:55.516489+0000","last_unstale":"2024-04-16T05:35:17.139777+0000","last_undegraded":"2024-04-16T05:35:17.139777+0000","last_fullsized":"2024-04-16T05:35:17.139777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:26:08.289501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139046+0000","last_change":"2024-04-16T05:32:55.507374+0000","last_active":"2024-04-16T05:35:17.139046+0000","last_peered":"2024-04-16T05:35:17.139046+0000","last_clean":"2024-04-16T05:35:17.139046+0000","last_became_active":"2024-04-16T05:32:55.507045+0000","last_became_peered":"2024-04-16T05:32:55.507045+0000","last_unstale":"2024-04-16T05:35:17.139046+0000","last_undegraded":"2024-04-16T05:35:17.139046+0000","last_fullsized":"2024-04-16T05:35:17.139046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:35.968704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140697+0000","last_change":"2024-04-16T05:32:55.508845+0000","last_active":"2024-04-16T05:35:17.140697+0000","last_peered":"2024-04-16T05:35:17.140697+0000","last_clean":"2024-04-16T05:35:17.140697+0000","last_became_active":"2024-04-16T05:32:55.508527+0000","last_became_peered":"2024-04-16T05:32:55.508527+0000","last_unstale":"2024-04-16T05:35:17.140697+0000","last_undegraded":"2024-04-16T05:35:17.140697+0000","last_fullsized":"2024-04-16T05:35:17.140697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:20:15.033043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032367+0000","last_change":"2024-04-16T05:32:55.506059+0000","last_active":"2024-04-16T05:35:17.032367+0000","last_peered":"2024-04-16T05:35:17.032367+0000","last_clean":"2024-04-16T05:35:17.032367+0000","last_became_active":"2024-04-16T05:32:55.505728+0000","last_became_peered":"2024-04-16T05:32:55.505728+0000","last_unstale":"2024-04-16T05:35:17.032367+0000","last_undegraded":"2024-04-16T05:35:17.032367+0000","last_fullsized":"2024-04-16T05:35:17.032367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:19.253543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140417+0000","last_change":"2024-04-16T05:32:55.506791+0000","last_active":"2024-04-16T05:35:17.140417+0000","last_peered":"2024-04-16T05:35:17.140417+0000","last_clean":"2024-04-16T05:35:17.140417+0000","last_became_active":"2024-04-16T05:32:55.506379+0000","last_became_peered":"2024-04-16T05:32:55.506379+0000","last_unstale":"2024-04-16T05:35:17.140417+0000","last_undegraded":"2024-04-16T05:35:17.140417+0000","last_fullsized":"2024-04-16T05:35:17.140417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:09:11.457144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139494+0000","last_change":"2024-04-16T05:33:52.582503+0000","last_active":"2024-04-16T05:35:17.139494+0000","last_peered":"2024-04-16T05:35:17.139494+0000","last_clean":"2024-04-16T05:35:17.139494+0000","last_became_active":"2024-04-16T05:33:52.581996+0000","last_became_peered":"2024-04-16T05:33:52.581996+0000","last_unstale":"2024-04-16T05:35:17.139494+0000","last_undegraded":"2024-04-16T05:35:17.139494+0000","last_fullsized":"2024-04-16T05:35:17.139494+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:34:40.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032559+0000","last_change":"2024-04-16T05:32:54.504826+0000","last_active":"2024-04-16T05:35:17.032559+0000","last_peered":"2024-04-16T05:35:17.032559+0000","last_clean":"2024-04-16T05:35:17.032559+0000","last_became_active":"2024-04-16T05:32:54.504526+0000","last_became_peered":"2024-04-16T05:32:54.504526+0000","last_unstale":"2024-04-16T05:35:17.032559+0000","last_undegraded":"2024-04-16T05:35:17.032559+0000","last_fullsized":"2024-04-16T05:35:17.032559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:21:28.434692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935279+0000","last_change":"2024-04-16T05:33:52.580931+0000","last_active":"2024-04-16T05:35:16.935279+0000","last_peered":"2024-04-16T05:35:16.935279+0000","last_clean":"2024-04-16T05:35:16.935279+0000","last_became_active":"2024-04-16T05:33:52.580685+0000","last_became_peered":"2024-04-16T05:33:52.580685+0000","last_unstale":"2024-04-16T05:35:16.935279+0000","last_undegraded":"2024-04-16T05:35:16.935279+0000","last_fullsized":"2024-04-16T05:35:16.935279+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:35:34.548838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139517+0000","last_change":"2024-04-16T05:32:59.524687+0000","last_active":"2024-04-16T05:35:17.139517+0000","last_peered":"2024-04-16T05:35:17.139517+0000","last_clean":"2024-04-16T05:35:17.139517+0000","last_became_active":"2024-04-16T05:32:59.524228+0000","last_became_peered":"2024-04-16T05:32:59.524228+0000","last_unstale":"2024-04-16T05:35:17.139517+0000","last_undegraded":"2024-04-16T05:35:17.139517+0000","last_fullsized":"2024-04-16T05:35:17.139517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:23:15.244359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935307+0000","last_change":"2024-04-16T05:32:54.504062+0000","last_active":"2024-04-16T05:35:16.935307+0000","last_peered":"2024-04-16T05:35:16.935307+0000","last_clean":"2024-04-16T05:35:16.935307+0000","last_became_active":"2024-04-16T05:32:54.503745+0000","last_became_peered":"2024-04-16T05:32:54.503745+0000","last_unstale":"2024-04-16T05:35:16.935307+0000","last_undegraded":"2024-04-16T05:35:16.935307+0000","last_fullsized":"2024-04-16T05:35:16.935307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:29:34.157937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140473+0000","last_change":"2024-04-16T05:32:55.509820+0000","last_active":"2024-04-16T05:35:17.140473+0000","last_peered":"2024-04-16T05:35:17.140473+0000","last_clean":"2024-04-16T05:35:17.140473+0000","last_became_active":"2024-04-16T05:32:55.509473+0000","last_became_peered":"2024-04-16T05:32:55.509473+0000","last_unstale":"2024-04-16T05:35:17.140473+0000","last_undegraded":"2024-04-16T05:35:17.140473+0000","last_fullsized":"2024-04-16T05:35:17.140473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:12:22.433189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140278+0000","last_change":"2024-04-16T05:33:52.678903+0000","last_active":"2024-04-16T05:35:17.140278+0000","last_peered":"2024-04-16T05:35:17.140278+0000","last_clean":"2024-04-16T05:35:17.140278+0000","last_became_active":"2024-04-16T05:33:52.678388+0000","last_became_peered":"2024-04-16T05:33:52.678388+0000","last_unstale":"2024-04-16T05:35:17.140278+0000","last_undegraded":"2024-04-16T05:35:17.140278+0000","last_fullsized":"2024-04-16T05:35:17.140278+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:07:01.060645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935984+0000","last_change":"2024-04-16T05:32:54.511469+0000","last_active":"2024-04-16T05:35:16.935984+0000","last_peered":"2024-04-16T05:35:16.935984+0000","last_clean":"2024-04-16T05:35:16.935984+0000","last_became_active":"2024-04-16T05:32:54.511120+0000","last_became_peered":"2024-04-16T05:32:54.511120+0000","last_unstale":"2024-04-16T05:35:16.935984+0000","last_undegraded":"2024-04-16T05:35:16.935984+0000","last_fullsized":"2024-04-16T05:35:16.935984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139428+0000","last_change":"2024-04-16T05:32:55.512308+0000","last_active":"2024-04-16T05:35:17.139428+0000","last_peered":"2024-04-16T05:35:17.139428+0000","last_clean":"2024-04-16T05:35:17.139428+0000","last_became_active":"2024-04-16T05:32:55.511625+0000","last_became_peered":"2024-04-16T05:32:55.511625+0000","last_unstale":"2024-04-16T05:35:17.139428+0000","last_undegraded":"2024-04-16T05:35:17.139428+0000","last_fullsized":"2024-04-16T05:35:17.139428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:29:49.323092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139445+0000","last_change":"2024-04-16T05:32:59.519306+0000","last_active":"2024-04-16T05:35:17.139445+0000","last_peered":"2024-04-16T05:35:17.139445+0000","last_clean":"2024-04-16T05:35:17.139445+0000","last_became_active":"2024-04-16T05:32:59.518797+0000","last_became_peered":"2024-04-16T05:32:59.518797+0000","last_unstale":"2024-04-16T05:35:17.139445+0000","last_undegraded":"2024-04-16T05:35:17.139445+0000","last_fullsized":"2024-04-16T05:35:17.139445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:04:12.389004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935592+0000","last_change":"2024-04-16T05:32:54.505102+0000","last_active":"2024-04-16T05:35:16.935592+0000","last_peered":"2024-04-16T05:35:16.935592+0000","last_clean":"2024-04-16T05:35:16.935592+0000","last_became_active":"2024-04-16T05:32:54.504806+0000","last_became_peered":"2024-04-16T05:32:54.504806+0000","last_unstale":"2024-04-16T05:35:16.935592+0000","last_undegraded":"2024-04-16T05:35:16.935592+0000","last_fullsized":"2024-04-16T05:35:16.935592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:19.467690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.041059+0000","last_change":"2024-04-16T05:32:55.517097+0000","last_active":"2024-04-16T05:35:17.041059+0000","last_peered":"2024-04-16T05:35:17.041059+0000","last_clean":"2024-04-16T05:35:17.041059+0000","last_became_active":"2024-04-16T05:32:55.516717+0000","last_became_peered":"2024-04-16T05:32:55.516717+0000","last_unstale":"2024-04-16T05:35:17.041059+0000","last_undegraded":"2024-04-16T05:35:17.041059+0000","last_fullsized":"2024-04-16T05:35:17.041059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:10:44.544935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140140+0000","last_change":"2024-04-16T05:32:59.521429+0000","last_active":"2024-04-16T05:35:17.140140+0000","last_peered":"2024-04-16T05:35:17.140140+0000","last_clean":"2024-04-16T05:35:17.140140+0000","last_became_active":"2024-04-16T05:32:59.520789+0000","last_became_peered":"2024-04-16T05:32:59.520789+0000","last_unstale":"2024-04-16T05:35:17.140140+0000","last_undegraded":"2024-04-16T05:35:17.140140+0000","last_fullsized":"2024-04-16T05:35:17.140140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:40:14.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":167,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787507+0000","last_change":"2024-04-16T05:32:54.507374+0000","last_active":"2024-04-16T05:35:17.787507+0000","last_peered":"2024-04-16T05:35:17.787507+0000","last_clean":"2024-04-16T05:35:17.787507+0000","last_became_active":"2024-04-16T05:32:54.507055+0000","last_became_peered":"2024-04-16T05:32:54.507055+0000","last_unstale":"2024-04-16T05:35:17.787507+0000","last_undegraded":"2024-04-16T05:35:17.787507+0000","last_fullsized":"2024-04-16T05:35:17.787507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:00:57.761971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935533+0000","last_change":"2024-04-16T05:32:55.513980+0000","last_active":"2024-04-16T05:35:16.935533+0000","last_peered":"2024-04-16T05:35:16.935533+0000","last_clean":"2024-04-16T05:35:16.935533+0000","last_became_active":"2024-04-16T05:32:55.513622+0000","last_became_peered":"2024-04-16T05:32:55.513622+0000","last_unstale":"2024-04-16T05:35:16.935533+0000","last_undegraded":"2024-04-16T05:35:16.935533+0000","last_fullsized":"2024-04-16T05:35:16.935533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:16:10.018381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139704+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:35:17.139704+0000","last_peered":"2024-04-16T05:35:17.139704+0000","last_clean":"2024-04-16T05:35:17.139704+0000","last_became_active":"2024-04-16T05:32:59.523032+0000","last_became_peered":"2024-04-16T05:32:59.523032+0000","last_unstale":"2024-04-16T05:35:17.139704+0000","last_undegraded":"2024-04-16T05:35:17.139704+0000","last_fullsized":"2024-04-16T05:35:17.139704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:33:17.897241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935663+0000","last_change":"2024-04-16T05:32:54.504527+0000","last_active":"2024-04-16T05:35:16.935663+0000","last_peered":"2024-04-16T05:35:16.935663+0000","last_clean":"2024-04-16T05:35:16.935663+0000","last_became_active":"2024-04-16T05:32:54.504187+0000","last_became_peered":"2024-04-16T05:32:54.504187+0000","last_unstale":"2024-04-16T05:35:16.935663+0000","last_undegraded":"2024-04-16T05:35:16.935663+0000","last_fullsized":"2024-04-16T05:35:16.935663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:28:33.414362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040983+0000","last_change":"2024-04-16T05:32:55.520270+0000","last_active":"2024-04-16T05:35:17.040983+0000","last_peered":"2024-04-16T05:35:17.040983+0000","last_clean":"2024-04-16T05:35:17.040983+0000","last_became_active":"2024-04-16T05:32:55.519895+0000","last_became_peered":"2024-04-16T05:32:55.519895+0000","last_unstale":"2024-04-16T05:35:17.040983+0000","last_undegraded":"2024-04-16T05:35:17.040983+0000","last_fullsized":"2024-04-16T05:35:17.040983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:09.967836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140531+0000","last_change":"2024-04-16T05:32:59.518997+0000","last_active":"2024-04-16T05:35:17.140531+0000","last_peered":"2024-04-16T05:35:17.140531+0000","last_clean":"2024-04-16T05:35:17.140531+0000","last_became_active":"2024-04-16T05:32:59.518558+0000","last_became_peered":"2024-04-16T05:32:59.518558+0000","last_unstale":"2024-04-16T05:35:17.140531+0000","last_undegraded":"2024-04-16T05:35:17.140531+0000","last_fullsized":"2024-04-16T05:35:17.140531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:04:30.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140063+0000","last_change":"2024-04-16T05:32:54.502074+0000","last_active":"2024-04-16T05:35:17.140063+0000","last_peered":"2024-04-16T05:35:17.140063+0000","last_clean":"2024-04-16T05:35:17.140063+0000","last_became_active":"2024-04-16T05:32:54.501730+0000","last_became_peered":"2024-04-16T05:32:54.501730+0000","last_unstale":"2024-04-16T05:35:17.140063+0000","last_undegraded":"2024-04-16T05:35:17.140063+0000","last_fullsized":"2024-04-16T05:35:17.140063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:05:02.929799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934611+0000","last_change":"2024-04-16T05:32:55.504557+0000","last_active":"2024-04-16T05:35:16.934611+0000","last_peered":"2024-04-16T05:35:16.934611+0000","last_clean":"2024-04-16T05:35:16.934611+0000","last_became_active":"2024-04-16T05:32:55.504244+0000","last_became_peered":"2024-04-16T05:32:55.504244+0000","last_unstale":"2024-04-16T05:35:16.934611+0000","last_undegraded":"2024-04-16T05:35:16.934611+0000","last_fullsized":"2024-04-16T05:35:16.934611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:33:40.500436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139880+0000","last_change":"2024-04-16T05:32:59.524681+0000","last_active":"2024-04-16T05:35:17.139880+0000","last_peered":"2024-04-16T05:35:17.139880+0000","last_clean":"2024-04-16T05:35:17.139880+0000","last_became_active":"2024-04-16T05:32:59.524217+0000","last_became_peered":"2024-04-16T05:32:59.524217+0000","last_unstale":"2024-04-16T05:35:17.139880+0000","last_undegraded":"2024-04-16T05:35:17.139880+0000","last_fullsized":"2024-04-16T05:35:17.139880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:28.314442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040615+0000","last_change":"2024-04-16T05:32:54.508242+0000","last_active":"2024-04-16T05:35:17.040615+0000","last_peered":"2024-04-16T05:35:17.040615+0000","last_clean":"2024-04-16T05:35:17.040615+0000","last_became_active":"2024-04-16T05:32:54.507782+0000","last_became_peered":"2024-04-16T05:32:54.507782+0000","last_unstale":"2024-04-16T05:35:17.040615+0000","last_undegraded":"2024-04-16T05:35:17.040615+0000","last_fullsized":"2024-04-16T05:35:17.040615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:27:26.805716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.031911+0000","last_change":"2024-04-16T05:32:55.508985+0000","last_active":"2024-04-16T05:35:17.031911+0000","last_peered":"2024-04-16T05:35:17.031911+0000","last_clean":"2024-04-16T05:35:17.031911+0000","last_became_active":"2024-04-16T05:32:55.508659+0000","last_became_peered":"2024-04-16T05:32:55.508659+0000","last_unstale":"2024-04-16T05:35:17.031911+0000","last_undegraded":"2024-04-16T05:35:17.031911+0000","last_fullsized":"2024-04-16T05:35:17.031911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:43:41.534155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934714+0000","last_change":"2024-04-16T05:32:59.521257+0000","last_active":"2024-04-16T05:35:16.934714+0000","last_peered":"2024-04-16T05:35:16.934714+0000","last_clean":"2024-04-16T05:35:16.934714+0000","last_became_active":"2024-04-16T05:32:59.520848+0000","last_became_peered":"2024-04-16T05:32:59.520848+0000","last_unstale":"2024-04-16T05:35:16.934714+0000","last_undegraded":"2024-04-16T05:35:16.934714+0000","last_fullsized":"2024-04-16T05:35:16.934714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:14.219378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'8","reported_seq":284,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.786952+0000","last_change":"2024-04-16T05:32:54.500684+0000","last_active":"2024-04-16T05:35:17.786952+0000","last_peered":"2024-04-16T05:35:17.786952+0000","last_clean":"2024-04-16T05:35:17.786952+0000","last_became_active":"2024-04-16T05:32:54.500290+0000","last_became_peered":"2024-04-16T05:32:54.500290+0000","last_unstale":"2024-04-16T05:35:17.786952+0000","last_undegraded":"2024-04-16T05:35:17.786952+0000","last_fullsized":"2024-04-16T05:35:17.786952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:20:55.263313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140189+0000","last_change":"2024-04-16T05:32:55.511224+0000","last_active":"2024-04-16T05:35:17.140189+0000","last_peered":"2024-04-16T05:35:17.140189+0000","last_clean":"2024-04-16T05:35:17.140189+0000","last_became_active":"2024-04-16T05:32:55.510891+0000","last_became_peered":"2024-04-16T05:32:55.510891+0000","last_unstale":"2024-04-16T05:35:17.140189+0000","last_undegraded":"2024-04-16T05:35:17.140189+0000","last_fullsized":"2024-04-16T05:35:17.140189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:27:24.894658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.031803+0000","last_change":"2024-04-16T05:32:59.521695+0000","last_active":"2024-04-16T05:35:17.031803+0000","last_peered":"2024-04-16T05:35:17.031803+0000","last_clean":"2024-04-16T05:35:17.031803+0000","last_became_active":"2024-04-16T05:32:59.520569+0000","last_became_peered":"2024-04-16T05:32:59.520569+0000","last_unstale":"2024-04-16T05:35:17.031803+0000","last_undegraded":"2024-04-16T05:35:17.031803+0000","last_fullsized":"2024-04-16T05:35:17.031803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:01:14.726043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141002+0000","last_change":"2024-04-16T05:32:54.505058+0000","last_active":"2024-04-16T05:35:17.141002+0000","last_peered":"2024-04-16T05:35:17.141002+0000","last_clean":"2024-04-16T05:35:17.141002+0000","last_became_active":"2024-04-16T05:32:54.504636+0000","last_became_peered":"2024-04-16T05:32:54.504636+0000","last_unstale":"2024-04-16T05:35:17.141002+0000","last_undegraded":"2024-04-16T05:35:17.141002+0000","last_fullsized":"2024-04-16T05:35:17.141002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:49:14.429468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140474+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:35:17.140474+0000","last_peered":"2024-04-16T05:35:17.140474+0000","last_clean":"2024-04-16T05:35:17.140474+0000","last_became_active":"2024-04-16T05:33:52.678570+0000","last_became_peered":"2024-04-16T05:33:52.678570+0000","last_unstale":"2024-04-16T05:35:17.140474+0000","last_undegraded":"2024-04-16T05:35:17.140474+0000","last_fullsized":"2024-04-16T05:35:17.140474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:56:32.405491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140508+0000","last_change":"2024-04-16T05:32:59.520394+0000","last_active":"2024-04-16T05:35:17.140508+0000","last_peered":"2024-04-16T05:35:17.140508+0000","last_clean":"2024-04-16T05:35:17.140508+0000","last_became_active":"2024-04-16T05:32:59.519947+0000","last_became_peered":"2024-04-16T05:32:59.519947+0000","last_unstale":"2024-04-16T05:35:17.140508+0000","last_undegraded":"2024-04-16T05:35:17.140508+0000","last_fullsized":"2024-04-16T05:35:17.140508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:33:27.647919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.041043+0000","last_change":"2024-04-16T05:32:54.505350+0000","last_active":"2024-04-16T05:35:17.041043+0000","last_peered":"2024-04-16T05:35:17.041043+0000","last_clean":"2024-04-16T05:35:17.041043+0000","last_became_active":"2024-04-16T05:32:54.505011+0000","last_became_peered":"2024-04-16T05:32:54.505011+0000","last_unstale":"2024-04-16T05:35:17.041043+0000","last_undegraded":"2024-04-16T05:35:17.041043+0000","last_fullsized":"2024-04-16T05:35:17.041043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:03:00.276862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139699+0000","last_change":"2024-04-16T05:33:52.673799+0000","last_active":"2024-04-16T05:35:17.139699+0000","last_peered":"2024-04-16T05:35:17.139699+0000","last_clean":"2024-04-16T05:35:17.139699+0000","last_became_active":"2024-04-16T05:33:52.673344+0000","last_became_peered":"2024-04-16T05:33:52.673344+0000","last_unstale":"2024-04-16T05:35:17.139699+0000","last_undegraded":"2024-04-16T05:35:17.139699+0000","last_fullsized":"2024-04-16T05:35:17.139699+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:07.387912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.031930+0000","last_change":"2024-04-16T05:32:59.519660+0000","last_active":"2024-04-16T05:35:17.031930+0000","last_peered":"2024-04-16T05:35:17.031930+0000","last_clean":"2024-04-16T05:35:17.031930+0000","last_became_active":"2024-04-16T05:32:59.518198+0000","last_became_peered":"2024-04-16T05:32:59.518198+0000","last_unstale":"2024-04-16T05:35:17.031930+0000","last_undegraded":"2024-04-16T05:35:17.031930+0000","last_fullsized":"2024-04-16T05:35:17.031930+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:03:10.096252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'40","reported_seq":325,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787683+0000","last_change":"2024-04-16T05:32:54.501530+0000","last_active":"2024-04-16T05:35:17.787683+0000","last_peered":"2024-04-16T05:35:17.787683+0000","last_clean":"2024-04-16T05:35:17.787683+0000","last_became_active":"2024-04-16T05:32:54.501159+0000","last_became_peered":"2024-04-16T05:32:54.501159+0000","last_unstale":"2024-04-16T05:35:17.787683+0000","last_undegraded":"2024-04-16T05:35:17.787683+0000","last_fullsized":"2024-04-16T05:35:17.787683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:26:22.240137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66452,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":190,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935749+0000","last_change":"2024-04-16T05:32:55.512210+0000","last_active":"2024-04-16T05:35:16.935749+0000","last_peered":"2024-04-16T05:35:16.935749+0000","last_clean":"2024-04-16T05:35:16.935749+0000","last_became_active":"2024-04-16T05:32:55.511672+0000","last_became_peered":"2024-04-16T05:32:55.511672+0000","last_unstale":"2024-04-16T05:35:16.935749+0000","last_undegraded":"2024-04-16T05:35:16.935749+0000","last_fullsized":"2024-04-16T05:35:16.935749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:10:53.755803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.041267+0000","last_change":"2024-04-16T05:32:59.527552+0000","last_active":"2024-04-16T05:35:17.041267+0000","last_peered":"2024-04-16T05:35:17.041267+0000","last_clean":"2024-04-16T05:35:17.041267+0000","last_became_active":"2024-04-16T05:32:59.526919+0000","last_became_peered":"2024-04-16T05:32:59.526919+0000","last_unstale":"2024-04-16T05:35:17.041267+0000","last_undegraded":"2024-04-16T05:35:17.041267+0000","last_fullsized":"2024-04-16T05:35:17.041267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:00:46.342383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139675+0000","last_change":"2024-04-16T05:32:54.494589+0000","last_active":"2024-04-16T05:35:17.139675+0000","last_peered":"2024-04-16T05:35:17.139675+0000","last_clean":"2024-04-16T05:35:17.139675+0000","last_became_active":"2024-04-16T05:32:54.494221+0000","last_became_peered":"2024-04-16T05:32:54.494221+0000","last_unstale":"2024-04-16T05:35:17.139675+0000","last_undegraded":"2024-04-16T05:35:17.139675+0000","last_fullsized":"2024-04-16T05:35:17.139675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:17:53.245359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140562+0000","last_change":"2024-04-16T05:32:55.502724+0000","last_active":"2024-04-16T05:35:17.140562+0000","last_peered":"2024-04-16T05:35:17.140562+0000","last_clean":"2024-04-16T05:35:17.140562+0000","last_became_active":"2024-04-16T05:32:55.502415+0000","last_became_peered":"2024-04-16T05:32:55.502415+0000","last_unstale":"2024-04-16T05:35:17.140562+0000","last_undegraded":"2024-04-16T05:35:17.140562+0000","last_fullsized":"2024-04-16T05:35:17.140562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:20:44.443926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140552+0000","last_change":"2024-04-16T05:32:59.522243+0000","last_active":"2024-04-16T05:35:17.140552+0000","last_peered":"2024-04-16T05:35:17.140552+0000","last_clean":"2024-04-16T05:35:17.140552+0000","last_became_active":"2024-04-16T05:32:59.521823+0000","last_became_peered":"2024-04-16T05:32:59.521823+0000","last_unstale":"2024-04-16T05:35:17.140552+0000","last_undegraded":"2024-04-16T05:35:17.140552+0000","last_fullsized":"2024-04-16T05:35:17.140552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:09:11.158434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040872+0000","last_change":"2024-04-16T05:32:54.507505+0000","last_active":"2024-04-16T05:35:17.040872+0000","last_peered":"2024-04-16T05:35:17.040872+0000","last_clean":"2024-04-16T05:35:17.040872+0000","last_became_active":"2024-04-16T05:32:54.506558+0000","last_became_peered":"2024-04-16T05:32:54.506558+0000","last_unstale":"2024-04-16T05:35:17.040872+0000","last_undegraded":"2024-04-16T05:35:17.040872+0000","last_fullsized":"2024-04-16T05:35:17.040872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:36:17.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140421+0000","last_change":"2024-04-16T05:32:55.518998+0000","last_active":"2024-04-16T05:35:17.140421+0000","last_peered":"2024-04-16T05:35:17.140421+0000","last_clean":"2024-04-16T05:35:17.140421+0000","last_became_active":"2024-04-16T05:32:55.518640+0000","last_became_peered":"2024-04-16T05:32:55.518640+0000","last_unstale":"2024-04-16T05:35:17.140421+0000","last_undegraded":"2024-04-16T05:35:17.140421+0000","last_fullsized":"2024-04-16T05:35:17.140421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:42:44.667504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032258+0000","last_change":"2024-04-16T05:32:59.519050+0000","last_active":"2024-04-16T05:35:17.032258+0000","last_peered":"2024-04-16T05:35:17.032258+0000","last_clean":"2024-04-16T05:35:17.032258+0000","last_became_active":"2024-04-16T05:32:59.518663+0000","last_became_peered":"2024-04-16T05:32:59.518663+0000","last_unstale":"2024-04-16T05:35:17.032258+0000","last_undegraded":"2024-04-16T05:35:17.032258+0000","last_fullsized":"2024-04-16T05:35:17.032258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:36:19.649605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787652+0000","last_change":"2024-04-16T05:32:54.500940+0000","last_active":"2024-04-16T05:35:17.787652+0000","last_peered":"2024-04-16T05:35:17.787652+0000","last_clean":"2024-04-16T05:35:17.787652+0000","last_became_active":"2024-04-16T05:32:54.500542+0000","last_became_peered":"2024-04-16T05:32:54.500542+0000","last_unstale":"2024-04-16T05:35:17.787652+0000","last_undegraded":"2024-04-16T05:35:17.787652+0000","last_fullsized":"2024-04-16T05:35:17.787652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:38:00.232244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139964+0000","last_change":"2024-04-16T05:32:55.509889+0000","last_active":"2024-04-16T05:35:17.139964+0000","last_peered":"2024-04-16T05:35:17.139964+0000","last_clean":"2024-04-16T05:35:17.139964+0000","last_became_active":"2024-04-16T05:32:55.508911+0000","last_became_peered":"2024-04-16T05:32:55.508911+0000","last_unstale":"2024-04-16T05:35:17.139964+0000","last_undegraded":"2024-04-16T05:35:17.139964+0000","last_fullsized":"2024-04-16T05:35:17.139964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:14:46.609127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141707+0000","last_change":"2024-04-16T05:32:59.523555+0000","last_active":"2024-04-16T05:35:17.141707+0000","last_peered":"2024-04-16T05:35:17.141707+0000","last_clean":"2024-04-16T05:35:17.141707+0000","last_became_active":"2024-04-16T05:32:59.523187+0000","last_became_peered":"2024-04-16T05:32:59.523187+0000","last_unstale":"2024-04-16T05:35:17.141707+0000","last_undegraded":"2024-04-16T05:35:17.141707+0000","last_fullsized":"2024-04-16T05:35:17.141707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:35.049355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935450+0000","last_change":"2024-04-16T05:32:54.503247+0000","last_active":"2024-04-16T05:35:16.935450+0000","last_peered":"2024-04-16T05:35:16.935450+0000","last_clean":"2024-04-16T05:35:16.935450+0000","last_became_active":"2024-04-16T05:32:54.502927+0000","last_became_peered":"2024-04-16T05:32:54.502927+0000","last_unstale":"2024-04-16T05:35:16.935450+0000","last_undegraded":"2024-04-16T05:35:16.935450+0000","last_fullsized":"2024-04-16T05:35:16.935450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:36:17.585827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040774+0000","last_change":"2024-04-16T05:32:55.519265+0000","last_active":"2024-04-16T05:35:17.040774+0000","last_peered":"2024-04-16T05:35:17.040774+0000","last_clean":"2024-04-16T05:35:17.040774+0000","last_became_active":"2024-04-16T05:32:55.518840+0000","last_became_peered":"2024-04-16T05:32:55.518840+0000","last_unstale":"2024-04-16T05:35:17.040774+0000","last_undegraded":"2024-04-16T05:35:17.040774+0000","last_fullsized":"2024-04-16T05:35:17.040774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:37:56.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040636+0000","last_change":"2024-04-16T05:32:59.525275+0000","last_active":"2024-04-16T05:35:17.040636+0000","last_peered":"2024-04-16T05:35:17.040636+0000","last_clean":"2024-04-16T05:35:17.040636+0000","last_became_active":"2024-04-16T05:32:59.524546+0000","last_became_peered":"2024-04-16T05:32:59.524546+0000","last_unstale":"2024-04-16T05:35:17.040636+0000","last_undegraded":"2024-04-16T05:35:17.040636+0000","last_fullsized":"2024-04-16T05:35:17.040636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:07.561898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140459+0000","last_change":"2024-04-16T05:34:52.635664+0000","last_active":"2024-04-16T05:35:17.140459+0000","last_peered":"2024-04-16T05:35:17.140459+0000","last_clean":"2024-04-16T05:35:17.140459+0000","last_became_active":"2024-04-16T05:34:52.627471+0000","last_became_peered":"2024-04-16T05:34:52.627471+0000","last_unstale":"2024-04-16T05:35:17.140459+0000","last_undegraded":"2024-04-16T05:35:17.140459+0000","last_fullsized":"2024-04-16T05:35:17.140459+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:48:28.009122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140884+0000","last_change":"2024-04-16T05:32:55.515685+0000","last_active":"2024-04-16T05:35:17.140884+0000","last_peered":"2024-04-16T05:35:17.140884+0000","last_clean":"2024-04-16T05:35:17.140884+0000","last_became_active":"2024-04-16T05:32:55.515367+0000","last_became_peered":"2024-04-16T05:32:55.515367+0000","last_unstale":"2024-04-16T05:35:17.140884+0000","last_undegraded":"2024-04-16T05:35:17.140884+0000","last_fullsized":"2024-04-16T05:35:17.140884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:46.670143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935508+0000","last_change":"2024-04-16T05:33:00.419828+0000","last_active":"2024-04-16T05:35:16.935508+0000","last_peered":"2024-04-16T05:35:16.935508+0000","last_clean":"2024-04-16T05:35:16.935508+0000","last_became_active":"2024-04-16T05:33:00.419321+0000","last_became_peered":"2024-04-16T05:33:00.419321+0000","last_unstale":"2024-04-16T05:35:16.935508+0000","last_undegraded":"2024-04-16T05:35:16.935508+0000","last_fullsized":"2024-04-16T05:35:16.935508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:53:56.694914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.032165+0000","last_change":"2024-04-16T05:32:54.505369+0000","last_active":"2024-04-16T05:35:17.032165+0000","last_peered":"2024-04-16T05:35:17.032165+0000","last_clean":"2024-04-16T05:35:17.032165+0000","last_became_active":"2024-04-16T05:32:54.505043+0000","last_became_peered":"2024-04-16T05:32:54.505043+0000","last_unstale":"2024-04-16T05:35:17.032165+0000","last_undegraded":"2024-04-16T05:35:17.032165+0000","last_fullsized":"2024-04-16T05:35:17.032165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:34.242117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935671+0000","last_change":"2024-04-16T05:32:55.509970+0000","last_active":"2024-04-16T05:35:16.935671+0000","last_peered":"2024-04-16T05:35:16.935671+0000","last_clean":"2024-04-16T05:35:16.935671+0000","last_became_active":"2024-04-16T05:32:55.509632+0000","last_became_peered":"2024-04-16T05:32:55.509632+0000","last_unstale":"2024-04-16T05:35:16.935671+0000","last_undegraded":"2024-04-16T05:35:16.935671+0000","last_fullsized":"2024-04-16T05:35:16.935671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:33:55.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141399+0000","last_change":"2024-04-16T05:32:59.527007+0000","last_active":"2024-04-16T05:35:17.141399+0000","last_peered":"2024-04-16T05:35:17.141399+0000","last_clean":"2024-04-16T05:35:17.141399+0000","last_became_active":"2024-04-16T05:32:59.526645+0000","last_became_peered":"2024-04-16T05:32:59.526645+0000","last_unstale":"2024-04-16T05:35:17.141399+0000","last_undegraded":"2024-04-16T05:35:17.141399+0000","last_fullsized":"2024-04-16T05:35:17.141399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:46:06.698912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":161,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139866+0000","last_change":"2024-04-16T05:32:54.503409+0000","last_active":"2024-04-16T05:35:17.139866+0000","last_peered":"2024-04-16T05:35:17.139866+0000","last_clean":"2024-04-16T05:35:17.139866+0000","last_became_active":"2024-04-16T05:32:54.503069+0000","last_became_peered":"2024-04-16T05:32:54.503069+0000","last_unstale":"2024-04-16T05:35:17.139866+0000","last_undegraded":"2024-04-16T05:35:17.139866+0000","last_fullsized":"2024-04-16T05:35:17.139866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:45:10.535225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935458+0000","last_change":"2024-04-16T05:32:55.519744+0000","last_active":"2024-04-16T05:35:16.935458+0000","last_peered":"2024-04-16T05:35:16.935458+0000","last_clean":"2024-04-16T05:35:16.935458+0000","last_became_active":"2024-04-16T05:32:55.519091+0000","last_became_peered":"2024-04-16T05:32:55.519091+0000","last_unstale":"2024-04-16T05:35:16.935458+0000","last_undegraded":"2024-04-16T05:35:16.935458+0000","last_fullsized":"2024-04-16T05:35:16.935458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:19.417691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141700+0000","last_change":"2024-04-16T05:33:00.419494+0000","last_active":"2024-04-16T05:35:17.141700+0000","last_peered":"2024-04-16T05:35:17.141700+0000","last_clean":"2024-04-16T05:35:17.141700+0000","last_became_active":"2024-04-16T05:33:00.418954+0000","last_became_peered":"2024-04-16T05:33:00.418954+0000","last_unstale":"2024-04-16T05:35:17.141700+0000","last_undegraded":"2024-04-16T05:35:17.141700+0000","last_fullsized":"2024-04-16T05:35:17.141700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:03:01.835304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040794+0000","last_change":"2024-04-16T05:32:55.518272+0000","last_active":"2024-04-16T05:35:17.040794+0000","last_peered":"2024-04-16T05:35:17.040794+0000","last_clean":"2024-04-16T05:35:17.040794+0000","last_became_active":"2024-04-16T05:32:55.517928+0000","last_became_peered":"2024-04-16T05:32:55.517928+0000","last_unstale":"2024-04-16T05:35:17.040794+0000","last_undegraded":"2024-04-16T05:35:17.040794+0000","last_fullsized":"2024-04-16T05:35:17.040794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:44:47.739994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":164,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.178665+0000","last_change":"2024-04-16T05:32:54.503923+0000","last_active":"2024-04-16T05:35:17.178665+0000","last_peered":"2024-04-16T05:35:17.178665+0000","last_clean":"2024-04-16T05:35:17.178665+0000","last_became_active":"2024-04-16T05:32:54.503576+0000","last_became_peered":"2024-04-16T05:32:54.503576+0000","last_unstale":"2024-04-16T05:35:17.178665+0000","last_undegraded":"2024-04-16T05:35:17.178665+0000","last_fullsized":"2024-04-16T05:35:17.178665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:49:36.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140426+0000","last_change":"2024-04-16T05:32:59.517757+0000","last_active":"2024-04-16T05:35:17.140426+0000","last_peered":"2024-04-16T05:35:17.140426+0000","last_clean":"2024-04-16T05:35:17.140426+0000","last_became_active":"2024-04-16T05:32:59.517357+0000","last_became_peered":"2024-04-16T05:32:59.517357+0000","last_unstale":"2024-04-16T05:35:17.140426+0000","last_undegraded":"2024-04-16T05:35:17.140426+0000","last_fullsized":"2024-04-16T05:35:17.140426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:21.767071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934854+0000","last_change":"2024-04-16T05:32:55.514617+0000","last_active":"2024-04-16T05:35:16.934854+0000","last_peered":"2024-04-16T05:35:16.934854+0000","last_clean":"2024-04-16T05:35:16.934854+0000","last_became_active":"2024-04-16T05:32:55.512660+0000","last_became_peered":"2024-04-16T05:32:55.512660+0000","last_unstale":"2024-04-16T05:35:16.934854+0000","last_undegraded":"2024-04-16T05:35:16.934854+0000","last_fullsized":"2024-04-16T05:35:16.934854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:31:45.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140378+0000","last_change":"2024-04-16T05:32:54.498650+0000","last_active":"2024-04-16T05:35:17.140378+0000","last_peered":"2024-04-16T05:35:17.140378+0000","last_clean":"2024-04-16T05:35:17.140378+0000","last_became_active":"2024-04-16T05:32:54.497672+0000","last_became_peered":"2024-04-16T05:32:54.497672+0000","last_unstale":"2024-04-16T05:35:17.140378+0000","last_undegraded":"2024-04-16T05:35:17.140378+0000","last_fullsized":"2024-04-16T05:35:17.140378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:47:36.908809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":369,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139698+0000","last_change":"2024-04-16T05:31:42.831800+0000","last_active":"2024-04-16T05:35:17.139698+0000","last_peered":"2024-04-16T05:35:17.139698+0000","last_clean":"2024-04-16T05:35:17.139698+0000","last_became_active":"2024-04-16T05:31:42.831391+0000","last_became_peered":"2024-04-16T05:31:42.831391+0000","last_unstale":"2024-04-16T05:35:17.139698+0000","last_undegraded":"2024-04-16T05:35:17.139698+0000","last_fullsized":"2024-04-16T05:35:17.139698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:28:15.969690+0000","last_clean_scrub_stamp":"2024-04-16T05:28:15.969690+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:11:40.152803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935901+0000","last_change":"2024-04-16T05:33:52.675549+0000","last_active":"2024-04-16T05:35:16.935901+0000","last_peered":"2024-04-16T05:35:16.935901+0000","last_clean":"2024-04-16T05:35:16.935901+0000","last_became_active":"2024-04-16T05:33:52.674969+0000","last_became_peered":"2024-04-16T05:33:52.674969+0000","last_unstale":"2024-04-16T05:35:16.935901+0000","last_undegraded":"2024-04-16T05:35:16.935901+0000","last_fullsized":"2024-04-16T05:35:16.935901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:58:33.137841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139786+0000","last_change":"2024-04-16T05:32:55.512324+0000","last_active":"2024-04-16T05:35:17.139786+0000","last_peered":"2024-04-16T05:35:17.139786+0000","last_clean":"2024-04-16T05:35:17.139786+0000","last_became_active":"2024-04-16T05:32:55.511958+0000","last_became_peered":"2024-04-16T05:32:55.511958+0000","last_unstale":"2024-04-16T05:35:17.139786+0000","last_undegraded":"2024-04-16T05:35:17.139786+0000","last_fullsized":"2024-04-16T05:35:17.139786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:38.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139804+0000","last_change":"2024-04-16T05:32:54.508021+0000","last_active":"2024-04-16T05:35:17.139804+0000","last_peered":"2024-04-16T05:35:17.139804+0000","last_clean":"2024-04-16T05:35:17.139804+0000","last_became_active":"2024-04-16T05:32:54.507645+0000","last_became_peered":"2024-04-16T05:32:54.507645+0000","last_unstale":"2024-04-16T05:35:17.139804+0000","last_undegraded":"2024-04-16T05:35:17.139804+0000","last_fullsized":"2024-04-16T05:35:17.139804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:21:50.721260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140448+0000","last_change":"2024-04-16T05:32:59.523574+0000","last_active":"2024-04-16T05:35:17.140448+0000","last_peered":"2024-04-16T05:35:17.140448+0000","last_clean":"2024-04-16T05:35:17.140448+0000","last_became_active":"2024-04-16T05:32:59.523159+0000","last_became_peered":"2024-04-16T05:32:59.523159+0000","last_unstale":"2024-04-16T05:35:17.140448+0000","last_undegraded":"2024-04-16T05:35:17.140448+0000","last_fullsized":"2024-04-16T05:35:17.140448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:48:37.696713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935875+0000","last_change":"2024-04-16T05:32:54.503262+0000","last_active":"2024-04-16T05:35:16.935875+0000","last_peered":"2024-04-16T05:35:16.935875+0000","last_clean":"2024-04-16T05:35:16.935875+0000","last_became_active":"2024-04-16T05:32:54.502909+0000","last_became_peered":"2024-04-16T05:32:54.502909+0000","last_unstale":"2024-04-16T05:35:16.935875+0000","last_undegraded":"2024-04-16T05:35:16.935875+0000","last_fullsized":"2024-04-16T05:35:16.935875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:48:32.841921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141260+0000","last_change":"2024-04-16T05:32:55.507126+0000","last_active":"2024-04-16T05:35:17.141260+0000","last_peered":"2024-04-16T05:35:17.141260+0000","last_clean":"2024-04-16T05:35:17.141260+0000","last_became_active":"2024-04-16T05:32:55.506743+0000","last_became_peered":"2024-04-16T05:32:55.506743+0000","last_unstale":"2024-04-16T05:35:17.141260+0000","last_undegraded":"2024-04-16T05:35:17.141260+0000","last_fullsized":"2024-04-16T05:35:17.141260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:10:58.697649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934575+0000","last_change":"2024-04-16T05:33:00.419435+0000","last_active":"2024-04-16T05:35:16.934575+0000","last_peered":"2024-04-16T05:35:16.934575+0000","last_clean":"2024-04-16T05:35:16.934575+0000","last_became_active":"2024-04-16T05:33:00.418985+0000","last_became_peered":"2024-04-16T05:33:00.418985+0000","last_unstale":"2024-04-16T05:35:16.934575+0000","last_undegraded":"2024-04-16T05:35:16.934575+0000","last_fullsized":"2024-04-16T05:35:16.934575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:08:31.188911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'48","reported_seq":107,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.144790+0000","last_change":"2024-04-16T05:32:54.503349+0000","last_active":"2024-04-16T05:35:17.144790+0000","last_peered":"2024-04-16T05:35:17.144790+0000","last_clean":"2024-04-16T05:35:17.144790+0000","last_became_active":"2024-04-16T05:32:54.503018+0000","last_became_peered":"2024-04-16T05:32:54.503018+0000","last_unstale":"2024-04-16T05:35:17.144790+0000","last_undegraded":"2024-04-16T05:35:17.144790+0000","last_fullsized":"2024-04-16T05:35:17.144790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:34:20.853543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935262+0000","last_change":"2024-04-16T05:32:55.511201+0000","last_active":"2024-04-16T05:35:16.935262+0000","last_peered":"2024-04-16T05:35:16.935262+0000","last_clean":"2024-04-16T05:35:16.935262+0000","last_became_active":"2024-04-16T05:32:55.510906+0000","last_became_peered":"2024-04-16T05:32:55.510906+0000","last_unstale":"2024-04-16T05:35:16.935262+0000","last_undegraded":"2024-04-16T05:35:16.935262+0000","last_fullsized":"2024-04-16T05:35:16.935262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:37:37.147628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139081+0000","last_change":"2024-04-16T05:32:59.518440+0000","last_active":"2024-04-16T05:35:17.139081+0000","last_peered":"2024-04-16T05:35:17.139081+0000","last_clean":"2024-04-16T05:35:17.139081+0000","last_became_active":"2024-04-16T05:32:59.517870+0000","last_became_peered":"2024-04-16T05:32:59.517870+0000","last_unstale":"2024-04-16T05:35:17.139081+0000","last_undegraded":"2024-04-16T05:35:17.139081+0000","last_fullsized":"2024-04-16T05:35:17.139081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:51:31.157418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140365+0000","last_change":"2024-04-16T05:32:54.503983+0000","last_active":"2024-04-16T05:35:17.140365+0000","last_peered":"2024-04-16T05:35:17.140365+0000","last_clean":"2024-04-16T05:35:17.140365+0000","last_became_active":"2024-04-16T05:32:54.502367+0000","last_became_peered":"2024-04-16T05:32:54.502367+0000","last_unstale":"2024-04-16T05:35:17.140365+0000","last_undegraded":"2024-04-16T05:35:17.140365+0000","last_fullsized":"2024-04-16T05:35:17.140365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:49:36.997692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140112+0000","last_change":"2024-04-16T05:32:55.516812+0000","last_active":"2024-04-16T05:35:17.140112+0000","last_peered":"2024-04-16T05:35:17.140112+0000","last_clean":"2024-04-16T05:35:17.140112+0000","last_became_active":"2024-04-16T05:32:55.516477+0000","last_became_peered":"2024-04-16T05:32:55.516477+0000","last_unstale":"2024-04-16T05:35:17.140112+0000","last_undegraded":"2024-04-16T05:35:17.140112+0000","last_fullsized":"2024-04-16T05:35:17.140112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:24:29.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140029+0000","last_change":"2024-04-16T05:33:00.419622+0000","last_active":"2024-04-16T05:35:17.140029+0000","last_peered":"2024-04-16T05:35:17.140029+0000","last_clean":"2024-04-16T05:35:17.140029+0000","last_became_active":"2024-04-16T05:33:00.419135+0000","last_became_peered":"2024-04-16T05:33:00.419135+0000","last_unstale":"2024-04-16T05:35:17.140029+0000","last_undegraded":"2024-04-16T05:35:17.140029+0000","last_fullsized":"2024-04-16T05:35:17.140029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:38:02.529192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.138987+0000","last_change":"2024-04-16T05:32:54.503652+0000","last_active":"2024-04-16T05:35:17.138987+0000","last_peered":"2024-04-16T05:35:17.138987+0000","last_clean":"2024-04-16T05:35:17.138987+0000","last_became_active":"2024-04-16T05:32:54.503311+0000","last_became_peered":"2024-04-16T05:32:54.503311+0000","last_unstale":"2024-04-16T05:35:17.138987+0000","last_undegraded":"2024-04-16T05:35:17.138987+0000","last_fullsized":"2024-04-16T05:35:17.138987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:49:21.584029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935388+0000","last_change":"2024-04-16T05:32:55.508145+0000","last_active":"2024-04-16T05:35:16.935388+0000","last_peered":"2024-04-16T05:35:16.935388+0000","last_clean":"2024-04-16T05:35:16.935388+0000","last_became_active":"2024-04-16T05:32:55.507794+0000","last_became_peered":"2024-04-16T05:32:55.507794+0000","last_unstale":"2024-04-16T05:35:16.935388+0000","last_undegraded":"2024-04-16T05:35:16.935388+0000","last_fullsized":"2024-04-16T05:35:16.935388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:11:57.925733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.936185+0000","last_change":"2024-04-16T05:33:52.678920+0000","last_active":"2024-04-16T05:35:16.936185+0000","last_peered":"2024-04-16T05:35:16.936185+0000","last_clean":"2024-04-16T05:35:16.936185+0000","last_became_active":"2024-04-16T05:33:52.678330+0000","last_became_peered":"2024-04-16T05:33:52.678330+0000","last_unstale":"2024-04-16T05:35:16.936185+0000","last_undegraded":"2024-04-16T05:35:16.936185+0000","last_fullsized":"2024-04-16T05:35:16.936185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:03:33.323513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934764+0000","last_change":"2024-04-16T05:32:54.500909+0000","last_active":"2024-04-16T05:35:16.934764+0000","last_peered":"2024-04-16T05:35:16.934764+0000","last_clean":"2024-04-16T05:35:16.934764+0000","last_became_active":"2024-04-16T05:32:54.499348+0000","last_became_peered":"2024-04-16T05:32:54.499348+0000","last_unstale":"2024-04-16T05:35:16.934764+0000","last_undegraded":"2024-04-16T05:35:16.934764+0000","last_fullsized":"2024-04-16T05:35:16.934764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:51:49.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140151+0000","last_change":"2024-04-16T05:32:55.510922+0000","last_active":"2024-04-16T05:35:17.140151+0000","last_peered":"2024-04-16T05:35:17.140151+0000","last_clean":"2024-04-16T05:35:17.140151+0000","last_became_active":"2024-04-16T05:32:55.510546+0000","last_became_peered":"2024-04-16T05:32:55.510546+0000","last_unstale":"2024-04-16T05:35:17.140151+0000","last_undegraded":"2024-04-16T05:35:17.140151+0000","last_fullsized":"2024-04-16T05:35:17.140151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:07:27.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139878+0000","last_change":"2024-04-16T05:32:59.524257+0000","last_active":"2024-04-16T05:35:17.139878+0000","last_peered":"2024-04-16T05:35:17.139878+0000","last_clean":"2024-04-16T05:35:17.139878+0000","last_became_active":"2024-04-16T05:32:59.523768+0000","last_became_peered":"2024-04-16T05:32:59.523768+0000","last_unstale":"2024-04-16T05:35:17.139878+0000","last_undegraded":"2024-04-16T05:35:17.139878+0000","last_fullsized":"2024-04-16T05:35:17.139878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:38:27.594048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'5","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140663+0000","last_change":"2024-04-16T05:34:52.631768+0000","last_active":"2024-04-16T05:35:17.140663+0000","last_peered":"2024-04-16T05:35:17.140663+0000","last_clean":"2024-04-16T05:35:17.140663+0000","last_became_active":"2024-04-16T05:34:52.627168+0000","last_became_peered":"2024-04-16T05:34:52.627168+0000","last_unstale":"2024-04-16T05:35:17.140663+0000","last_undegraded":"2024-04-16T05:35:17.140663+0000","last_fullsized":"2024-04-16T05:35:17.140663+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:29:46.866673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935117+0000","last_change":"2024-04-16T05:32:55.508005+0000","last_active":"2024-04-16T05:35:16.935117+0000","last_peered":"2024-04-16T05:35:16.935117+0000","last_clean":"2024-04-16T05:35:16.935117+0000","last_became_active":"2024-04-16T05:32:55.507698+0000","last_became_peered":"2024-04-16T05:32:55.507698+0000","last_unstale":"2024-04-16T05:35:16.935117+0000","last_undegraded":"2024-04-16T05:35:16.935117+0000","last_fullsized":"2024-04-16T05:35:16.935117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:11:12.423294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140202+0000","last_change":"2024-04-16T05:33:00.419442+0000","last_active":"2024-04-16T05:35:17.140202+0000","last_peered":"2024-04-16T05:35:17.140202+0000","last_clean":"2024-04-16T05:35:17.140202+0000","last_became_active":"2024-04-16T05:33:00.418906+0000","last_became_peered":"2024-04-16T05:33:00.418906+0000","last_unstale":"2024-04-16T05:35:17.140202+0000","last_undegraded":"2024-04-16T05:35:17.140202+0000","last_fullsized":"2024-04-16T05:35:17.140202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:21:22.124879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935760+0000","last_change":"2024-04-16T05:32:54.509090+0000","last_active":"2024-04-16T05:35:16.935760+0000","last_peered":"2024-04-16T05:35:16.935760+0000","last_clean":"2024-04-16T05:35:16.935760+0000","last_became_active":"2024-04-16T05:32:54.508700+0000","last_became_peered":"2024-04-16T05:32:54.508700+0000","last_unstale":"2024-04-16T05:35:16.935760+0000","last_undegraded":"2024-04-16T05:35:16.935760+0000","last_fullsized":"2024-04-16T05:35:16.935760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:53:34.344073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139999+0000","last_change":"2024-04-16T05:32:55.509658+0000","last_active":"2024-04-16T05:35:17.139999+0000","last_peered":"2024-04-16T05:35:17.139999+0000","last_clean":"2024-04-16T05:35:17.139999+0000","last_became_active":"2024-04-16T05:32:55.509258+0000","last_became_peered":"2024-04-16T05:32:55.509258+0000","last_unstale":"2024-04-16T05:35:17.139999+0000","last_undegraded":"2024-04-16T05:35:17.139999+0000","last_fullsized":"2024-04-16T05:35:17.139999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:17:03.490544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139041+0000","last_change":"2024-04-16T05:32:59.519260+0000","last_active":"2024-04-16T05:35:17.139041+0000","last_peered":"2024-04-16T05:35:17.139041+0000","last_clean":"2024-04-16T05:35:17.139041+0000","last_became_active":"2024-04-16T05:32:59.518839+0000","last_became_peered":"2024-04-16T05:32:59.518839+0000","last_unstale":"2024-04-16T05:35:17.139041+0000","last_undegraded":"2024-04-16T05:35:17.139041+0000","last_fullsized":"2024-04-16T05:35:17.139041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:54:50.555214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.787320+0000","last_change":"2024-04-16T05:32:54.503226+0000","last_active":"2024-04-16T05:35:17.787320+0000","last_peered":"2024-04-16T05:35:17.787320+0000","last_clean":"2024-04-16T05:35:17.787320+0000","last_became_active":"2024-04-16T05:32:54.502687+0000","last_became_peered":"2024-04-16T05:32:54.502687+0000","last_unstale":"2024-04-16T05:35:17.787320+0000","last_undegraded":"2024-04-16T05:35:17.787320+0000","last_fullsized":"2024-04-16T05:35:17.787320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:34:24.228464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140830+0000","last_change":"2024-04-16T05:32:55.515203+0000","last_active":"2024-04-16T05:35:17.140830+0000","last_peered":"2024-04-16T05:35:17.140830+0000","last_clean":"2024-04-16T05:35:17.140830+0000","last_became_active":"2024-04-16T05:32:55.514815+0000","last_became_peered":"2024-04-16T05:32:55.514815+0000","last_unstale":"2024-04-16T05:35:17.140830+0000","last_undegraded":"2024-04-16T05:35:17.140830+0000","last_fullsized":"2024-04-16T05:35:17.140830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:11:18.464532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139998+0000","last_change":"2024-04-16T05:32:59.520442+0000","last_active":"2024-04-16T05:35:17.139998+0000","last_peered":"2024-04-16T05:35:17.139998+0000","last_clean":"2024-04-16T05:35:17.139998+0000","last_became_active":"2024-04-16T05:32:59.520071+0000","last_became_peered":"2024-04-16T05:32:59.520071+0000","last_unstale":"2024-04-16T05:35:17.139998+0000","last_undegraded":"2024-04-16T05:35:17.139998+0000","last_fullsized":"2024-04-16T05:35:17.139998+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:19:16.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":84,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.142330+0000","last_change":"2024-04-16T05:32:54.510098+0000","last_active":"2024-04-16T05:35:17.142330+0000","last_peered":"2024-04-16T05:35:17.142330+0000","last_clean":"2024-04-16T05:35:17.142330+0000","last_became_active":"2024-04-16T05:32:54.509750+0000","last_became_peered":"2024-04-16T05:32:54.509750+0000","last_unstale":"2024-04-16T05:35:17.142330+0000","last_undegraded":"2024-04-16T05:35:17.142330+0000","last_fullsized":"2024-04-16T05:35:17.142330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:00:09.528958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140257+0000","last_change":"2024-04-16T05:33:52.678823+0000","last_active":"2024-04-16T05:35:17.140257+0000","last_peered":"2024-04-16T05:35:17.140257+0000","last_clean":"2024-04-16T05:35:17.140257+0000","last_became_active":"2024-04-16T05:33:52.678276+0000","last_became_peered":"2024-04-16T05:33:52.678276+0000","last_unstale":"2024-04-16T05:35:17.140257+0000","last_undegraded":"2024-04-16T05:35:17.140257+0000","last_fullsized":"2024-04-16T05:35:17.140257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:58:33.186382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040472+0000","last_change":"2024-04-16T05:32:59.521334+0000","last_active":"2024-04-16T05:35:17.040472+0000","last_peered":"2024-04-16T05:35:17.040472+0000","last_clean":"2024-04-16T05:35:17.040472+0000","last_became_active":"2024-04-16T05:32:59.520348+0000","last_became_peered":"2024-04-16T05:32:59.520348+0000","last_unstale":"2024-04-16T05:35:17.040472+0000","last_undegraded":"2024-04-16T05:35:17.040472+0000","last_fullsized":"2024-04-16T05:35:17.040472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:58.497004+0000","last_clean_scrub_stamp":"2024-04-16T05:32:58.497004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:40.628552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141843+0000","last_change":"2024-04-16T05:32:54.503062+0000","last_active":"2024-04-16T05:35:17.141843+0000","last_peered":"2024-04-16T05:35:17.141843+0000","last_clean":"2024-04-16T05:35:17.141843+0000","last_became_active":"2024-04-16T05:32:54.502762+0000","last_became_peered":"2024-04-16T05:32:54.502762+0000","last_unstale":"2024-04-16T05:35:17.141843+0000","last_undegraded":"2024-04-16T05:35:17.141843+0000","last_fullsized":"2024-04-16T05:35:17.141843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:53.474929+0000","last_clean_scrub_stamp":"2024-04-16T05:32:53.474929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:43:57.911296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141355+0000","last_change":"2024-04-16T05:32:55.510461+0000","last_active":"2024-04-16T05:35:17.141355+0000","last_peered":"2024-04-16T05:35:17.141355+0000","last_clean":"2024-04-16T05:35:17.141355+0000","last_became_active":"2024-04-16T05:32:55.510142+0000","last_became_peered":"2024-04-16T05:32:55.510142+0000","last_unstale":"2024-04-16T05:35:17.141355+0000","last_undegraded":"2024-04-16T05:35:17.141355+0000","last_fullsized":"2024-04-16T05:35:17.141355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:41.346078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140171+0000","last_change":"2024-04-16T05:32:55.512600+0000","last_active":"2024-04-16T05:35:17.140171+0000","last_peered":"2024-04-16T05:35:17.140171+0000","last_clean":"2024-04-16T05:35:17.140171+0000","last_became_active":"2024-04-16T05:32:55.512265+0000","last_became_peered":"2024-04-16T05:32:55.512265+0000","last_unstale":"2024-04-16T05:35:17.140171+0000","last_undegraded":"2024-04-16T05:35:17.140171+0000","last_fullsized":"2024-04-16T05:35:17.140171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:04:46.112644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139670+0000","last_change":"2024-04-16T05:32:55.514481+0000","last_active":"2024-04-16T05:35:17.139670+0000","last_peered":"2024-04-16T05:35:17.139670+0000","last_clean":"2024-04-16T05:35:17.139670+0000","last_became_active":"2024-04-16T05:32:55.512485+0000","last_became_peered":"2024-04-16T05:32:55.512485+0000","last_unstale":"2024-04-16T05:35:17.139670+0000","last_undegraded":"2024-04-16T05:35:17.139670+0000","last_fullsized":"2024-04-16T05:35:17.139670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:33:02.179774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139422+0000","last_change":"2024-04-16T05:32:55.509849+0000","last_active":"2024-04-16T05:35:17.139422+0000","last_peered":"2024-04-16T05:35:17.139422+0000","last_clean":"2024-04-16T05:35:17.139422+0000","last_became_active":"2024-04-16T05:32:55.509338+0000","last_became_peered":"2024-04-16T05:32:55.509338+0000","last_unstale":"2024-04-16T05:35:17.139422+0000","last_undegraded":"2024-04-16T05:35:17.139422+0000","last_fullsized":"2024-04-16T05:35:17.139422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:32:38.900455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140163+0000","last_change":"2024-04-16T05:32:55.518556+0000","last_active":"2024-04-16T05:35:17.140163+0000","last_peered":"2024-04-16T05:35:17.140163+0000","last_clean":"2024-04-16T05:35:17.140163+0000","last_became_active":"2024-04-16T05:32:55.518240+0000","last_became_peered":"2024-04-16T05:32:55.518240+0000","last_unstale":"2024-04-16T05:35:17.140163+0000","last_undegraded":"2024-04-16T05:35:17.140163+0000","last_fullsized":"2024-04-16T05:35:17.140163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:25:46.002516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.141244+0000","last_change":"2024-04-16T05:32:55.518106+0000","last_active":"2024-04-16T05:35:17.141244+0000","last_peered":"2024-04-16T05:35:17.141244+0000","last_clean":"2024-04-16T05:35:17.141244+0000","last_became_active":"2024-04-16T05:32:55.517403+0000","last_became_peered":"2024-04-16T05:32:55.517403+0000","last_unstale":"2024-04-16T05:35:17.141244+0000","last_undegraded":"2024-04-16T05:35:17.141244+0000","last_fullsized":"2024-04-16T05:35:17.141244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:36:34.731644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140630+0000","last_change":"2024-04-16T05:33:52.581236+0000","last_active":"2024-04-16T05:35:17.140630+0000","last_peered":"2024-04-16T05:35:17.140630+0000","last_clean":"2024-04-16T05:35:17.140630+0000","last_became_active":"2024-04-16T05:33:52.580748+0000","last_became_peered":"2024-04-16T05:33:52.580748+0000","last_unstale":"2024-04-16T05:35:17.140630+0000","last_undegraded":"2024-04-16T05:35:17.140630+0000","last_fullsized":"2024-04-16T05:35:17.140630+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:13:33.382776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139982+0000","last_change":"2024-04-16T05:32:55.507670+0000","last_active":"2024-04-16T05:35:17.139982+0000","last_peered":"2024-04-16T05:35:17.139982+0000","last_clean":"2024-04-16T05:35:17.139982+0000","last_became_active":"2024-04-16T05:32:55.506672+0000","last_became_peered":"2024-04-16T05:32:55.506672+0000","last_unstale":"2024-04-16T05:35:17.139982+0000","last_undegraded":"2024-04-16T05:35:17.139982+0000","last_fullsized":"2024-04-16T05:35:17.139982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:30:54.579672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.935321+0000","last_change":"2024-04-16T05:32:55.516987+0000","last_active":"2024-04-16T05:35:16.935321+0000","last_peered":"2024-04-16T05:35:16.935321+0000","last_clean":"2024-04-16T05:35:16.935321+0000","last_became_active":"2024-04-16T05:32:55.516654+0000","last_became_peered":"2024-04-16T05:32:55.516654+0000","last_unstale":"2024-04-16T05:35:16.935321+0000","last_undegraded":"2024-04-16T05:35:16.935321+0000","last_fullsized":"2024-04-16T05:35:16.935321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:32:42.957658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.040684+0000","last_change":"2024-04-16T05:32:55.506539+0000","last_active":"2024-04-16T05:35:17.040684+0000","last_peered":"2024-04-16T05:35:17.040684+0000","last_clean":"2024-04-16T05:35:17.040684+0000","last_became_active":"2024-04-16T05:32:55.503658+0000","last_became_peered":"2024-04-16T05:32:55.503658+0000","last_unstale":"2024-04-16T05:35:17.040684+0000","last_undegraded":"2024-04-16T05:35:17.040684+0000","last_fullsized":"2024-04-16T05:35:17.040684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:22.116213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:16.934725+0000","last_change":"2024-04-16T05:32:55.507363+0000","last_active":"2024-04-16T05:35:16.934725+0000","last_peered":"2024-04-16T05:35:16.934725+0000","last_clean":"2024-04-16T05:35:16.934725+0000","last_became_active":"2024-04-16T05:32:55.506923+0000","last_became_peered":"2024-04-16T05:32:55.506923+0000","last_unstale":"2024-04-16T05:35:16.934725+0000","last_undegraded":"2024-04-16T05:35:16.934725+0000","last_fullsized":"2024-04-16T05:35:16.934725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:19:34.356834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.139209+0000","last_change":"2024-04-16T05:32:55.508531+0000","last_active":"2024-04-16T05:35:17.139209+0000","last_peered":"2024-04-16T05:35:17.139209+0000","last_clean":"2024-04-16T05:35:17.139209+0000","last_became_active":"2024-04-16T05:32:55.508177+0000","last_became_peered":"2024-04-16T05:32:55.508177+0000","last_unstale":"2024-04-16T05:35:17.139209+0000","last_undegraded":"2024-04-16T05:35:17.139209+0000","last_fullsized":"2024-04-16T05:35:17.139209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:49:59.352572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140924+0000","last_change":"2024-04-16T05:32:55.507793+0000","last_active":"2024-04-16T05:35:17.140924+0000","last_peered":"2024-04-16T05:35:17.140924+0000","last_clean":"2024-04-16T05:35:17.140924+0000","last_became_active":"2024-04-16T05:32:55.507443+0000","last_became_peered":"2024-04-16T05:32:55.507443+0000","last_unstale":"2024-04-16T05:35:17.140924+0000","last_undegraded":"2024-04-16T05:35:17.140924+0000","last_fullsized":"2024-04-16T05:35:17.140924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:35.982879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T05:35:17.140787+0000","last_change":"2024-04-16T05:32:55.517057+0000","last_active":"2024-04-16T05:35:17.140787+0000","last_peered":"2024-04-16T05:35:17.140787+0000","last_clean":"2024-04-16T05:35:17.140787+0000","last_became_active":"2024-04-16T05:32:55.516696+0000","last_became_peered":"2024-04-16T05:32:55.516696+0000","last_unstale":"2024-04-16T05:35:17.140787+0000","last_undegraded":"2024-04-16T05:35:17.140787+0000","last_fullsized":"2024-04-16T05:35:17.140787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:54.477298+0000","last_clean_scrub_stamp":"2024-04-16T05:32:54.477298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:43:10.391125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106974,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600767,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1336,"num_read_kb":784,"num_write":379,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1802301,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383720,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":540350,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":1188583,"data_compressed":68167,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1105920,"data_stored":936730,"data_compressed":51657,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":932990,"data_compressed":51655,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":1138139,"data_compressed":68167,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":534922,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1126400,"data_stored":602404,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":537218,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":589425,"data_compressed":29531,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":993902,"data_compressed":54666,"data_compressed_allocated":73728,"data_compressed_original":544768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":539919,"data_compressed":32509,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1264398,"data_compressed":74323,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35658,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135734,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200711,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:35:22.762 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.903+0000 7f3112d00640 1 -- 172.21.15.86:0/2740698118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101af0 msgr2=0x7f310c105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.903+0000 7f3112d00640 1 --2- 172.21.15.86:0/2740698118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101af0 0x7f310c105b30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009900 tx=0x7f30f402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.903+0000 7f3112d00640 1 -- 172.21.15.86:0/2740698118 shutdown_connections 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.903+0000 7f3112d00640 1 --2- 172.21.15.86:0/2740698118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f310c106210 0x7f310c10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.903+0000 7f3112d00640 1 --2- 172.21.15.86:0/2740698118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101af0 0x7f310c105b30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.903+0000 7f3112d00640 1 --2- 172.21.15.86:0/2740698118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f310c101140 0x7f310c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.903+0000 7f3112d00640 1 -- 172.21.15.86:0/2740698118 >> 172.21.15.86:0/2740698118 conn(0x7f310c077280 msgr2=0x7f310c077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.903+0000 7f3112d00640 1 -- 172.21.15.86:0/2740698118 shutdown_connections 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.903+0000 7f3112d00640 1 -- 172.21.15.86:0/2740698118 wait complete. 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3112d00640 1 Processor -- start 2024-04-16T05:35:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3112d00640 1 -- start start 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3112d00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101140 0x7f310c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3112d00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f310c106210 0x7f310c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3112d00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f310c19a2a0 0x7f310c1a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3112d00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f310c111320 con 0x7f310c101140 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3112d00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f310c1111a0 con 0x7f310c19a2a0 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3112d00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f310c1114a0 con 0x7f310c106210 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3110a9c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101140 0x7f310c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3110a9c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101140 0x7f310c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59532/0 (socket says 172.21.15.86:59532) 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3110a9c640 1 -- 172.21.15.86:0/4154646362 learned_addr learned my addr 172.21.15.86:0/4154646362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f311129d640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f310c19a2a0 0x7f310c1a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f310bfff640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f310c106210 0x7f310c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3110a9c640 1 -- 172.21.15.86:0/4154646362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f310c106210 msgr2=0x7f310c199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3110a9c640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f310c106210 0x7f310c199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3110a9c640 1 -- 172.21.15.86:0/4154646362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f310c19a2a0 msgr2=0x7f310c1a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3110a9c640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f310c19a2a0 0x7f310c1a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3110a9c640 1 -- 172.21.15.86:0/4154646362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c1a5020 con 0x7f310c101140 2024-04-16T05:35:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f311129d640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f310c19a2a0 0x7f310c1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.904+0000 7f3110a9c640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101140 0x7f310c199780 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00d770 tx=0x7f30fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.905+0000 7f3109ffb640 1 -- 172.21.15.86:0/4154646362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc017070 con 0x7f310c101140 2024-04-16T05:35:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.905+0000 7f3109ffb640 1 -- 172.21.15.86:0/4154646362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30fc005500 con 0x7f310c101140 2024-04-16T05:35:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.905+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310c1a5310 con 0x7f310c101140 2024-04-16T05:35:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.905+0000 7f3109ffb640 1 -- 172.21.15.86:0/4154646362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc004e00 con 0x7f310c101140 2024-04-16T05:35:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.905+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310c1a57e0 con 0x7f310c101140 2024-04-16T05:35:22.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:22 smithi194 ceph-mon[35865]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:22.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:22 smithi194 ceph-mon[35865]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:22.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:22 smithi194 ceph-mon[35865]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:22.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:22 smithi194 ceph-mon[35865]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:22.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:22 smithi194 ceph-mon[35865]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:22.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:22 smithi194 ceph-mon[35865]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:22.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:22 smithi194 ceph-mon[35865]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:22.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:22 smithi194 ceph-mon[35865]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T05:35:22.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:22 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/311761915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.906+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f310c102610 con 0x7f310c101140 2024-04-16T05:35:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.907+0000 7f3109ffb640 1 -- 172.21.15.86:0/4154646362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f30fc004fa0 con 0x7f310c101140 2024-04-16T05:35:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.907+0000 7f3109ffb640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30f80a3e40 0x7f30f80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.908+0000 7f310bfff640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30f80a3e40 0x7f30f80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.908+0000 7f3109ffb640 1 -- 172.21.15.86:0/4154646362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f30fc0d0950 con 0x7f310c101140 2024-04-16T05:35:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.908+0000 7f310bfff640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30f80a3e40 0x7f30f80a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30f4006010 tx=0x7f30f4000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:22.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:22.912+0000 7f3109ffb640 1 -- 172.21.15.86:0/4154646362 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30fc09ca30 con 0x7f310c101140 2024-04-16T05:35:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:22 smithi196 ceph-mon[36373]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:22 smithi196 ceph-mon[36373]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:22 smithi196 ceph-mon[36373]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:22 smithi196 ceph-mon[36373]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:22 smithi196 ceph-mon[36373]: Updating smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:22 smithi196 ceph-mon[36373]: Updating smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:22 smithi196 ceph-mon[36373]: Updating smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:22.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:22 smithi196 ceph-mon[36373]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T05:35:22.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:22 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/311761915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:23.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.094+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f310c1094a0 con 0x7f310c101140 2024-04-16T05:35:23.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.096+0000 7f3109ffb640 1 -- 172.21.15.86:0/4154646362 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25077 (secure 0 0 0) 0x7f30fc09c330 con 0x7f310c101140 2024-04-16T05:35:23.094 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:23.095 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":95,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:35:16.929832+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2330974702":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/3900523320":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2641972995":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/501397249":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30f80a3e40 msgr2=0x7f30f80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30f80a3e40 0x7f30f80a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30f4006010 tx=0x7f30f4000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101140 msgr2=0x7f310c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101140 0x7f310c199780 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00d770 tx=0x7f30fc00dc30 comp rx=0 tx=0).stop 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 shutdown_connections 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30f80a3e40 0x7f30f80a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f310c19a2a0 0x7f310c1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f310c106210 0x7f310c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 --2- 172.21.15.86:0/4154646362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310c101140 0x7f310c199780 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 >> 172.21.15.86:0/4154646362 conn(0x7f310c077280 msgr2=0x7f310c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 shutdown_connections 2024-04-16T05:35:23.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.098+0000 7f3112d00640 1 -- 172.21.15.86:0/4154646362 wait complete. 2024-04-16T05:35:23.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:35:23.108 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.217+0000 7f57f7035640 1 -- 172.21.15.86:0/1609918978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0101140 msgr2=0x7f57f0101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.217+0000 7f57f7035640 1 --2- 172.21.15.86:0/1609918978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0101140 0x7f57f0101520 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009900 tx=0x7f57e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.217+0000 7f57f7035640 1 -- 172.21.15.86:0/1609918978 shutdown_connections 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.217+0000 7f57f7035640 1 --2- 172.21.15.86:0/1609918978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f0106210 0x7f57f010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.217+0000 7f57f7035640 1 --2- 172.21.15.86:0/1609918978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f0101af0 0x7f57f0105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.217+0000 7f57f7035640 1 --2- 172.21.15.86:0/1609918978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0101140 0x7f57f0101520 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.217+0000 7f57f7035640 1 -- 172.21.15.86:0/1609918978 >> 172.21.15.86:0/1609918978 conn(0x7f57f0078000 msgr2=0x7f57f0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.217+0000 7f57f7035640 1 -- 172.21.15.86:0/1609918978 shutdown_connections 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.217+0000 7f57f7035640 1 -- 172.21.15.86:0/1609918978 wait complete. 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f7035640 1 Processor -- start 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f7035640 1 -- start start 2024-04-16T05:35:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f7035640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f0101140 0x7f57f0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f7035640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0106210 0x7f57f0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f7035640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f019a2f0 0x7f57f01a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f7035640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01113a0 con 0x7f57f0106210 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f7035640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0111220 con 0x7f57f019a2f0 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f7035640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0111520 con 0x7f57f0101140 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0106210 0x7f57f0199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0106210 0x7f57f0199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59562/0 (socket says 172.21.15.86:59562) 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57effff640 1 -- 172.21.15.86:0/1745415966 learned_addr learned my addr 172.21.15.86:0/1745415966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f4dd1640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f0101140 0x7f57f0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f55d2640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f019a2f0 0x7f57f01a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57effff640 1 -- 172.21.15.86:0/1745415966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f0101140 msgr2=0x7f57f0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57effff640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f0101140 0x7f57f0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57effff640 1 -- 172.21.15.86:0/1745415966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f019a2f0 msgr2=0x7f57f01a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57effff640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f019a2f0 0x7f57f01a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57effff640 1 -- 172.21.15.86:0/1745415966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f01a5070 con 0x7f57f0106210 2024-04-16T05:35:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57f55d2640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f019a2f0 0x7f57f01a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57effff640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0106210 0x7f57f0199d60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00b450 tx=0x7f57dc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.218+0000 7f57edffb640 1 -- 172.21.15.86:0/1745415966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc015020 con 0x7f57f0106210 2024-04-16T05:35:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.219+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f01a5360 con 0x7f57f0106210 2024-04-16T05:35:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.219+0000 7f57edffb640 1 -- 172.21.15.86:0/1745415966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57dc011d70 con 0x7f57f0106210 2024-04-16T05:35:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.219+0000 7f57edffb640 1 -- 172.21.15.86:0/1745415966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00f340 con 0x7f57f0106210 2024-04-16T05:35:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.219+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f01a5900 con 0x7f57f0106210 2024-04-16T05:35:23.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.220+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f0068e20 con 0x7f57f0106210 2024-04-16T05:35:23.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.221+0000 7f57edffb640 1 -- 172.21.15.86:0/1745415966 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f57dc018020 con 0x7f57f0106210 2024-04-16T05:35:23.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.221+0000 7f57edffb640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f57e00a3e40 0x7f57e00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.222+0000 7f57edffb640 1 -- 172.21.15.86:0/1745415966 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f57dc0d0850 con 0x7f57f0106210 2024-04-16T05:35:23.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.222+0000 7f57f4dd1640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f57e00a3e40 0x7f57e00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.222+0000 7f57f4dd1640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f57e00a3e40 0x7f57e00a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57e402cfd0 tx=0x7f57e4004f10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:23.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.226+0000 7f57edffb640 1 -- 172.21.15.86:0/1745415966 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57dc01f020 con 0x7f57f0106210 2024-04-16T05:35:23.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.403+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57f0109440 con 0x7f57f0106210 2024-04-16T05:35:23.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.403+0000 7f57edffb640 1 -- 172.21.15.86:0/1745415966 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f57dc00d040 con 0x7f57f0106210 2024-04-16T05:35:23.402 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:35:23.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f57e00a3e40 msgr2=0x7f57e00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f57e00a3e40 0x7f57e00a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57e402cfd0 tx=0x7f57e4004f10 comp rx=0 tx=0).stop 2024-04-16T05:35:23.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0106210 msgr2=0x7f57f0199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0106210 0x7f57f0199d60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00b450 tx=0x7f57dc00b910 comp rx=0 tx=0).stop 2024-04-16T05:35:23.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 shutdown_connections 2024-04-16T05:35:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f57e00a3e40 0x7f57e00a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f019a2f0 0x7f57f01a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0106210 0x7f57f0199d60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 --2- 172.21.15.86:0/1745415966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f0101140 0x7f57f0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 >> 172.21.15.86:0/1745415966 conn(0x7f57f0078000 msgr2=0x7f57f00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 shutdown_connections 2024-04-16T05:35:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.405+0000 7f57f7035640 1 -- 172.21.15.86:0/1745415966 wait complete. 2024-04-16T05:35:23.415 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:35:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.524+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2041677527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 msgr2=0x7fd42406aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.524+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2041677527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 0x7fd42406aaa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd418009930 tx=0x7fd41802bc20 comp rx=0 tx=0).stop 2024-04-16T05:35:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.524+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2041677527 shutdown_connections 2024-04-16T05:35:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.524+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2041677527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd42406afe0 0x7fd42410ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.524+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2041677527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 0x7fd42406aaa0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.524+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2041677527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd424107040 0x7fd424107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.524+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2041677527 >> 172.21.15.86:0/2041677527 conn(0x7fd424077ee0 msgr2=0x7fd4240782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.524+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2041677527 shutdown_connections 2024-04-16T05:35:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.524+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2041677527 wait complete. 2024-04-16T05:35:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd42bf8d640 1 Processor -- start 2024-04-16T05:35:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd42bf8d640 1 -- start start 2024-04-16T05:35:23.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd42bf8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 0x7fd4241a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd42bf8d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd42406afe0 0x7fd4241a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd42bf8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd424107040 0x7fd42419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd42bf8d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241123f0 con 0x7fd42406a620 2024-04-16T05:35:23.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd42bf8d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd424112270 con 0x7fd42406afe0 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd42bf8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd424112570 con 0x7fd424107040 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd429d29640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 0x7fd4241a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd429d29640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 0x7fd4241a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59576/0 (socket says 172.21.15.86:59576) 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd429d29640 1 -- 172.21.15.86:0/2305456248 learned_addr learned my addr 172.21.15.86:0/2305456248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd429528640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd42406afe0 0x7fd4241a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd42a52a640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd424107040 0x7fd42419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd429d29640 1 -- 172.21.15.86:0/2305456248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd424107040 msgr2=0x7fd42419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd429d29640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd424107040 0x7fd42419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd429d29640 1 -- 172.21.15.86:0/2305456248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd42406afe0 msgr2=0x7fd4241a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd429d29640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd42406afe0 0x7fd4241a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.525+0000 7fd429d29640 1 -- 172.21.15.86:0/2305456248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42419b0a0 con 0x7fd42406a620 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.526+0000 7fd429d29640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 0x7fd4241a0770 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd42000d950 tx=0x7fd42000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.526+0000 7fd416ffd640 1 -- 172.21.15.86:0/2305456248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420017070 con 0x7fd42406a620 2024-04-16T05:35:23.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.526+0000 7fd416ffd640 1 -- 172.21.15.86:0/2305456248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd420009c30 con 0x7fd42406a620 2024-04-16T05:35:23.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.526+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42419b390 con 0x7fd42406a620 2024-04-16T05:35:23.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.526+0000 7fd416ffd640 1 -- 172.21.15.86:0/2305456248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42000f6b0 con 0x7fd42406a620 2024-04-16T05:35:23.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.526+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4241a7630 con 0x7fd42406a620 2024-04-16T05:35:23.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.527+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd424069270 con 0x7fd42406a620 2024-04-16T05:35:23.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.528+0000 7fd416ffd640 1 -- 172.21.15.86:0/2305456248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd4200109d0 con 0x7fd42406a620 2024-04-16T05:35:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.529+0000 7fd416ffd640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4100a3e40 0x7fd4100a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.529+0000 7fd429528640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4100a3e40 0x7fd4100a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.529+0000 7fd416ffd640 1 -- 172.21.15.86:0/2305456248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd4200d09c0 con 0x7fd42406a620 2024-04-16T05:35:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.530+0000 7fd429528640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4100a3e40 0x7fd4100a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd418009900 tx=0x7fd418000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.534+0000 7fd416ffd640 1 -- 172.21.15.86:0/2305456248 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd420009de0 con 0x7fd42406a620 2024-04-16T05:35:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.717+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd4241a7800 con 0x7fd42406a620 2024-04-16T05:35:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.718+0000 7fd416ffd640 1 -- 172.21.15.86:0/2305456248 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fd42009c450 con 0x7fd42406a620 2024-04-16T05:35:23.717 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:23.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:23 smithi086 ceph-mon[32028]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:23.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:23 smithi086 ceph-mon[32028]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:23.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:23 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4154646362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:23.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:23 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1745415966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4100a3e40 msgr2=0x7fd4100a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4100a3e40 0x7fd4100a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd418009900 tx=0x7fd418000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 msgr2=0x7fd4241a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 0x7fd4241a0770 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd42000d950 tx=0x7fd42000de10 comp rx=0 tx=0).stop 2024-04-16T05:35:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 shutdown_connections 2024-04-16T05:35:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd424107040 0x7fd42419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4100a3e40 0x7fd4100a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd42406afe0 0x7fd4241a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 --2- 172.21.15.86:0/2305456248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42406a620 0x7fd4241a0770 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 >> 172.21.15.86:0/2305456248 conn(0x7fd424077ee0 msgr2=0x7fd42410d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 shutdown_connections 2024-04-16T05:35:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.720+0000 7fd42bf8d640 1 -- 172.21.15.86:0/2305456248 wait complete. 2024-04-16T05:35:23.730 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:35:23.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.839+0000 7fc15a00d640 1 -- 172.21.15.86:0/3757970978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 msgr2=0x7fc1541074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.839+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3757970978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 0x7fc1541074c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc144009930 tx=0x7fc14402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.839+0000 7fc15a00d640 1 -- 172.21.15.86:0/3757970978 shutdown_connections 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.839+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3757970978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc154101bf0 0x7fc15410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.839+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3757970978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc154101250 0x7fc1541016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.839+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3757970978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 0x7fc1541074c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.839+0000 7fc15a00d640 1 -- 172.21.15.86:0/3757970978 >> 172.21.15.86:0/3757970978 conn(0x7fc154076d00 msgr2=0x7fc154077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.839+0000 7fc15a00d640 1 -- 172.21.15.86:0/3757970978 shutdown_connections 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.839+0000 7fc15a00d640 1 -- 172.21.15.86:0/3757970978 wait complete. 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc15a00d640 1 Processor -- start 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc15a00d640 1 -- start start 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc15a00d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc154101250 0x7fc15419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc15a00d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc154101bf0 0x7fc15419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc15a00d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 0x7fc154198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc15a00d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc154112460 con 0x7fc1541070e0 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc15a00d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc1541122e0 con 0x7fc154101bf0 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc15a00d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1541125e0 con 0x7fc154101250 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc153fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 0x7fc154198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc153fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 0x7fc154198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59594/0 (socket says 172.21.15.86:59594) 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc153fff640 1 -- 172.21.15.86:0/3454985168 learned_addr learned my addr 172.21.15.86:0/3454985168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc14bfff640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc154101bf0 0x7fc15419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc153fff640 1 -- 172.21.15.86:0/3454985168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc154101250 msgr2=0x7fc15419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc153fff640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc154101250 0x7fc15419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc153fff640 1 -- 172.21.15.86:0/3454985168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc154101bf0 msgr2=0x7fc15419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc153fff640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc154101bf0 0x7fc15419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc153fff640 1 -- 172.21.15.86:0/3454985168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc154198fc0 con 0x7fc1541070e0 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc153fff640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 0x7fc154198700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00eaa0 tx=0x7fc14c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.840+0000 7fc1517fa640 1 -- 172.21.15.86:0/3454985168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00cdb0 con 0x7fc1541070e0 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.841+0000 7fc1517fa640 1 -- 172.21.15.86:0/3454985168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc14c005020 con 0x7fc1541070e0 2024-04-16T05:35:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.841+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1541992b0 con 0x7fc1541070e0 2024-04-16T05:35:23.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.841+0000 7fc1517fa640 1 -- 172.21.15.86:0/3454985168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0116b0 con 0x7fc1541070e0 2024-04-16T05:35:23.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.841+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1541a55c0 con 0x7fc1541070e0 2024-04-16T05:35:23.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.841+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc154069e50 con 0x7fc1541070e0 2024-04-16T05:35:23.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.843+0000 7fc1517fa640 1 -- 172.21.15.86:0/3454985168 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc14c005560 con 0x7fc1541070e0 2024-04-16T05:35:23.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.843+0000 7fc1517fa640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1400a3ec0 0x7fc1400a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:23.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.844+0000 7fc1537fe640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1400a3ec0 0x7fc1400a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:23.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.844+0000 7fc1517fa640 1 -- 172.21.15.86:0/3454985168 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc14c0d1380 con 0x7fc1541070e0 2024-04-16T05:35:23.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.844+0000 7fc1537fe640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1400a3ec0 0x7fc1400a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1440096c0 tx=0x7fc144000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:23.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:23.847+0000 7fc1517fa640 1 -- 172.21.15.86:0/3454985168 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc14c016030 con 0x7fc1541070e0 2024-04-16T05:35:23.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:23 smithi194 ceph-mon[35865]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:23.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:23 smithi194 ceph-mon[35865]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:23 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4154646362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:23 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1745415966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:23 smithi196 ceph-mon[36373]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:23 smithi196 ceph-mon[36373]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:23 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4154646362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:23 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1745415966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.023+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc154102640 con 0x7fc1541070e0 2024-04-16T05:35:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.024+0000 7fc1517fa640 1 -- 172.21.15.86:0/3454985168 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fc14c09d4e0 con 0x7fc1541070e0 2024-04-16T05:35:24.022 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:35:24.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1400a3ec0 msgr2=0x7fc1400a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1400a3ec0 0x7fc1400a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1440096c0 tx=0x7fc144000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 msgr2=0x7fc154198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 0x7fc154198700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00eaa0 tx=0x7fc14c00ef60 comp rx=0 tx=0).stop 2024-04-16T05:35:24.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 shutdown_connections 2024-04-16T05:35:24.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1541070e0 0x7fc154198700 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1400a3ec0 0x7fc1400a6380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc154101bf0 0x7fc15419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 --2- 172.21.15.86:0/3454985168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc154101250 0x7fc15419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 >> 172.21.15.86:0/3454985168 conn(0x7fc154076d00 msgr2=0x7fc1540ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:24.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.025+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 shutdown_connections 2024-04-16T05:35:24.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.026+0000 7fc15a00d640 1 -- 172.21.15.86:0/3454985168 wait complete. 2024-04-16T05:35:24.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:35:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.143+0000 7f99fb975640 1 -- 172.21.15.86:0/82014893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4102bf0 msgr2=0x7f99f4106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.143+0000 7f99fb975640 1 --2- 172.21.15.86:0/82014893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4102bf0 0x7f99f4106c30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009900 tx=0x7f99e802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.143+0000 7f99fb975640 1 -- 172.21.15.86:0/82014893 shutdown_connections 2024-04-16T05:35:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.143+0000 7f99fb975640 1 --2- 172.21.15.86:0/82014893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99f4107310 0x7f99f410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.143+0000 7f99fb975640 1 --2- 172.21.15.86:0/82014893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4102bf0 0x7f99f4106c30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.143+0000 7f99fb975640 1 --2- 172.21.15.86:0/82014893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99f4102240 0x7f99f4102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.143+0000 7f99fb975640 1 -- 172.21.15.86:0/82014893 >> 172.21.15.86:0/82014893 conn(0x7f99f4079450 msgr2=0x7f99f4079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 -- 172.21.15.86:0/82014893 shutdown_connections 2024-04-16T05:35:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 -- 172.21.15.86:0/82014893 wait complete. 2024-04-16T05:35:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 Processor -- start 2024-04-16T05:35:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 -- start start 2024-04-16T05:35:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99f4102240 0x7f99f419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4107310 0x7f99f419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99f41986e0 0x7f99f4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41123a0 con 0x7f99f4107310 2024-04-16T05:35:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4112220 con 0x7f99f41986e0 2024-04-16T05:35:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99fb975640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4112520 con 0x7f99f4102240 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99f8f10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4107310 0x7f99f419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99f8f10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4107310 0x7f99f419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59608/0 (socket says 172.21.15.86:59608) 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.144+0000 7f99f8f10640 1 -- 172.21.15.86:0/1661424843 learned_addr learned my addr 172.21.15.86:0/1661424843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f9f12640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99f41986e0 0x7f99f4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f9711640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99f4102240 0x7f99f419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f8f10640 1 -- 172.21.15.86:0/1661424843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99f4102240 msgr2=0x7f99f419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f8f10640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99f4102240 0x7f99f419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f8f10640 1 -- 172.21.15.86:0/1661424843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99f41986e0 msgr2=0x7f99f4198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f8f10640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99f41986e0 0x7f99f4198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f8f10640 1 -- 172.21.15.86:0/1661424843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f41992e0 con 0x7f99f4107310 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f8f10640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4107310 0x7f99f419eb60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f99e8036040 tx=0x7f99e802ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f27fc640 1 -- 172.21.15.86:0/1661424843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8041070 con 0x7f99f4107310 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f27fc640 1 -- 172.21.15.86:0/1661424843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e80054e0 con 0x7f99f4107310 2024-04-16T05:35:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f4199570 con 0x7f99f4107310 2024-04-16T05:35:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99f27fc640 1 -- 172.21.15.86:0/1661424843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8034440 con 0x7f99f4107310 2024-04-16T05:35:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.145+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f41a56b0 con 0x7f99f4107310 2024-04-16T05:35:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.146+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f4103710 con 0x7f99f4107310 2024-04-16T05:35:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.148+0000 7f99f27fc640 1 -- 172.21.15.86:0/1661424843 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f99e8044070 con 0x7f99f4107310 2024-04-16T05:35:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.149+0000 7f99f27fc640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f99dc0a3e40 0x7f99dc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.149+0000 7f99f9711640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f99dc0a3e40 0x7f99dc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.149+0000 7f99f27fc640 1 -- 172.21.15.86:0/1661424843 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f99e80f2b30 con 0x7f99f4107310 2024-04-16T05:35:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.149+0000 7f99f9711640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f99dc0a3e40 0x7f99dc0a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99e4005fd0 tx=0x7f99e4009870 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.152+0000 7f99f27fc640 1 -- 172.21.15.86:0/1661424843 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99e80bedb0 con 0x7f99f4107310 2024-04-16T05:35:24.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.330+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f99f4102bf0 con 0x7f99f4107310 2024-04-16T05:35:24.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.331+0000 7f99f27fc640 1 -- 172.21.15.86:0/1661424843 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f99e8004c50 con 0x7f99f4107310 2024-04-16T05:35:24.330 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:35:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f99dc0a3e40 msgr2=0x7f99dc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f99dc0a3e40 0x7f99dc0a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99e4005fd0 tx=0x7f99e4009870 comp rx=0 tx=0).stop 2024-04-16T05:35:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4107310 msgr2=0x7f99f419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4107310 0x7f99f419eb60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f99e8036040 tx=0x7f99e802ce20 comp rx=0 tx=0).stop 2024-04-16T05:35:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 shutdown_connections 2024-04-16T05:35:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f99dc0a3e40 0x7f99dc0a6300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99f41986e0 0x7f99f4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4107310 0x7f99f419eb60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 --2- 172.21.15.86:0/1661424843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99f4102240 0x7f99f419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 >> 172.21.15.86:0/1661424843 conn(0x7f99f4079450 msgr2=0x7f99f40fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 shutdown_connections 2024-04-16T05:35:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.333+0000 7f99fb975640 1 -- 172.21.15.86:0/1661424843 wait complete. 2024-04-16T05:35:24.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:35:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/2962016245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc101b00 msgr2=0x7fe4dc105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/2962016245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc101b00 0x7fe4dc105b40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009900 tx=0x7fe4c402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/2962016245 shutdown_connections 2024-04-16T05:35:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/2962016245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4dc106220 0x7fe4dc10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/2962016245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc101b00 0x7fe4dc105b40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/2962016245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4dc101150 0x7fe4dc101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/2962016245 >> 172.21.15.86:0/2962016245 conn(0x7fe4dc078000 msgr2=0x7fe4dc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/2962016245 shutdown_connections 2024-04-16T05:35:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/2962016245 wait complete. 2024-04-16T05:35:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 Processor -- start 2024-04-16T05:35:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.450+0000 7fe4e0eeb640 1 -- start start 2024-04-16T05:35:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4e0eeb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4dc101150 0x7fe4dc1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4e0eeb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4dc106220 0x7fe4dc199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4e0eeb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc19a1a0 0x7fe4dc1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4e0eeb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc111290 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4e0eeb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc111110 con 0x7fe4dc106220 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4e0eeb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc111410 con 0x7fe4dc101150 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4dad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc19a1a0 0x7fe4dc1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4dad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc19a1a0 0x7fe4dc1a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59624/0 (socket says 172.21.15.86:59624) 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4dad9d640 1 -- 172.21.15.86:0/3772010140 learned_addr learned my addr 172.21.15.86:0/3772010140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4d9d9b640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4dc106220 0x7fe4dc199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4da59c640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4dc101150 0x7fe4dc1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4dad9d640 1 -- 172.21.15.86:0/3772010140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4dc101150 msgr2=0x7fe4dc1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4dad9d640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4dc101150 0x7fe4dc1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4dad9d640 1 -- 172.21.15.86:0/3772010140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4dc106220 msgr2=0x7fe4dc199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4dad9d640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4dc106220 0x7fe4dc199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4dad9d640 1 -- 172.21.15.86:0/3772010140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc1a4f50 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4dad9d640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc19a1a0 0x7fe4dc1a4750 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c970 tx=0x7fe4cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4d37fe640 1 -- 172.21.15.86:0/3772010140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc007bb0 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4d37fe640 1 -- 172.21.15.86:0/3772010140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc007d50 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc1a5240 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.451+0000 7fe4d37fe640 1 -- 172.21.15.86:0/3772010140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc018800 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.452+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc1a5710 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.452+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4dc068e20 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.454+0000 7fe4d37fe640 1 -- 172.21.15.86:0/3772010140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe4cc00fa50 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.454+0000 7fe4d37fe640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4c80a3e40 0x7fe4c80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.454+0000 7fe4da59c640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4c80a3e40 0x7fe4c80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.454+0000 7fe4d37fe640 1 -- 172.21.15.86:0/3772010140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe4cc0d0d70 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.455+0000 7fe4da59c640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4c80a3e40 0x7fe4c80a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005fd0 tx=0x7fe4d4009680 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.458+0000 7fe4d37fe640 1 -- 172.21.15.86:0/3772010140 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4cc0d5070 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.675+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4dc101b00 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.676+0000 7fe4d37fe640 1 -- 172.21.15.86:0/3772010140 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fe4cc014030 con 0x7fe4dc19a1a0 2024-04-16T05:35:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:24.675 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:35:24.675 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:35:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4c80a3e40 msgr2=0x7fe4c80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4c80a3e40 0x7fe4c80a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005fd0 tx=0x7fe4d4009680 comp rx=0 tx=0).stop 2024-04-16T05:35:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc19a1a0 msgr2=0x7fe4dc1a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc19a1a0 0x7fe4dc1a4750 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c970 tx=0x7fe4cc00ce30 comp rx=0 tx=0).stop 2024-04-16T05:35:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 shutdown_connections 2024-04-16T05:35:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc19a1a0 0x7fe4dc1a4750 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4c80a3e40 0x7fe4c80a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4dc106220 0x7fe4dc199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 --2- 172.21.15.86:0/3772010140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4dc101150 0x7fe4dc1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 >> 172.21.15.86:0/3772010140 conn(0x7fe4dc078000 msgr2=0x7fe4dc0fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 shutdown_connections 2024-04-16T05:35:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.678+0000 7fe4e0eeb640 1 -- 172.21.15.86:0/3772010140 wait complete. 2024-04-16T05:35:24.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:35:24.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.794+0000 7f6ee8515640 1 -- 172.21.15.86:0/2692522834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee0101140 msgr2=0x7f6ee0101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.794+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2692522834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee0101140 0x7f6ee0101520 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009900 tx=0x7f6ed402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:24.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.794+0000 7f6ee8515640 1 -- 172.21.15.86:0/2692522834 shutdown_connections 2024-04-16T05:35:24.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.794+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2692522834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee0106210 0x7f6ee010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.794+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2692522834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee0101af0 0x7f6ee0105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.794+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2692522834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee0101140 0x7f6ee0101520 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.794+0000 7f6ee8515640 1 -- 172.21.15.86:0/2692522834 >> 172.21.15.86:0/2692522834 conn(0x7f6ee0077280 msgr2=0x7f6ee0077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:24.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.794+0000 7f6ee8515640 1 -- 172.21.15.86:0/2692522834 shutdown_connections 2024-04-16T05:35:24.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.794+0000 7f6ee8515640 1 -- 172.21.15.86:0/2692522834 wait complete. 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee8515640 1 Processor -- start 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee8515640 1 -- start start 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee8515640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee0101140 0x7f6ee01a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee8515640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee0106210 0x7f6ee01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee8515640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee019a980 0x7f6ee019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee8515640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee01112d0 con 0x7f6ee019a980 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee8515640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0111150 con 0x7f6ee0106210 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee8515640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0111450 con 0x7f6ee0101140 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee6ab2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee019a980 0x7f6ee019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee6ab2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee019a980 0x7f6ee019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59644/0 (socket says 172.21.15.86:59644) 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee5ab0640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee0106210 0x7f6ee01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee6ab2640 1 -- 172.21.15.86:0/2316973977 learned_addr learned my addr 172.21.15.86:0/2316973977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.795+0000 7f6ee62b1640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee0101140 0x7f6ee01a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ee6ab2640 1 -- 172.21.15.86:0/2316973977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee0101140 msgr2=0x7f6ee01a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ee6ab2640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee0101140 0x7f6ee01a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ee6ab2640 1 -- 172.21.15.86:0/2316973977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee0106210 msgr2=0x7f6ee01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ee6ab2640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee0106210 0x7f6ee01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ee6ab2640 1 -- 172.21.15.86:0/2316973977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee019b4a0 con 0x7f6ee019a980 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ee6ab2640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee019a980 0x7f6ee019ae30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800cad0 tx=0x7f6ed800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ed37fe640 1 -- 172.21.15.86:0/2316973977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed800ae20 con 0x7f6ee019a980 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee019b790 con 0x7f6ee019a980 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ed37fe640 1 -- 172.21.15.86:0/2316973977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed8005020 con 0x7f6ee019a980 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ed37fe640 1 -- 172.21.15.86:0/2316973977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed800f6f0 con 0x7f6ee019a980 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.796+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee01a79e0 con 0x7f6ee019a980 2024-04-16T05:35:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.797+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee0102610 con 0x7f6ee019a980 2024-04-16T05:35:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.798+0000 7f6ed37fe640 1 -- 172.21.15.86:0/2316973977 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6ed80051d0 con 0x7f6ee019a980 2024-04-16T05:35:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.799+0000 7f6ed37fe640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6ecc0a3e40 0x7f6ecc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.799+0000 7f6ee62b1640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6ecc0a3e40 0x7f6ecc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.799+0000 7f6ed37fe640 1 -- 172.21.15.86:0/2316973977 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6ed80d10a0 con 0x7f6ee019a980 2024-04-16T05:35:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.799+0000 7f6ee62b1640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6ecc0a3e40 0x7f6ecc0a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009230 tx=0x7f6ed4005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:24.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.803+0000 7f6ed37fe640 1 -- 172.21.15.86:0/2316973977 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ed8014030 con 0x7f6ee019a980 2024-04-16T05:35:24.885 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:24 smithi086 ceph-mon[32028]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:24.885 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:24 smithi086 ceph-mon[32028]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:24.885 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:24 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2305456248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.885 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:24 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3454985168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.885 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:24 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1661424843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:24 smithi194 ceph-mon[35865]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:24.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:24 smithi194 ceph-mon[35865]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:24.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:24 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2305456248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:24 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3454985168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:24 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1661424843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:24 smithi196 ceph-mon[36373]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:24 smithi196 ceph-mon[36373]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:24 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2305456248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:24 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3454985168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:24 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1661424843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.986+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ee019beb0 con 0x7f6ee019a980 2024-04-16T05:35:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.988+0000 7f6ed37fe640 1 -- 172.21.15.86:0/2316973977 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25077 (secure 0 0 0) 0x7f6ed809cb30 con 0x7f6ee019a980 2024-04-16T05:35:24.987 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:24.987 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":95,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:35:16.929832+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2330974702":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/3900523320":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2641972995":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/501397249":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:35:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6ecc0a3e40 msgr2=0x7f6ecc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6ecc0a3e40 0x7f6ecc0a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009230 tx=0x7f6ed4005fd0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee019a980 msgr2=0x7f6ee019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee019a980 0x7f6ee019ae30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800cad0 tx=0x7f6ed800cf90 comp rx=0 tx=0).stop 2024-04-16T05:35:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 shutdown_connections 2024-04-16T05:35:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee019a980 0x7f6ee019ae30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6ecc0a3e40 0x7f6ecc0a6300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee0106210 0x7f6ee01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 --2- 172.21.15.86:0/2316973977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee0101140 0x7f6ee01a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 >> 172.21.15.86:0/2316973977 conn(0x7f6ee0077280 msgr2=0x7f6ee00fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 shutdown_connections 2024-04-16T05:35:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:24.990+0000 7f6ee8515640 1 -- 172.21.15.86:0/2316973977 wait complete. 2024-04-16T05:35:25.001 DEBUG:tasks.fwd_scrub:Yielding 2024-04-16T05:35:25.001 INFO:teuthology.run_tasks:Running task exec... 2024-04-16T05:35:25.010 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T05:35:25.010 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:35:25.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-16T05:35:25.011 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-16T05:35:25.011 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:35:25.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:25.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:35:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f66b413d640 1 -- 172.21.15.86:0/2042454365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac100e00 msgr2=0x7f66ac1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f66b413d640 1 --2- 172.21.15.86:0/2042454365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac100e00 0x7f66ac1011e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009900 tx=0x7f66a802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f7917e60640 1 -- 172.21.15.86:0/4153094517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910107350 msgr2=0x7f791010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f7917e60640 1 --2- 172.21.15.86:0/4153094517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910107350 0x7f791010ef20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7908009ca0 tx=0x7f790802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f66b413d640 1 -- 172.21.15.86:0/2042454365 shutdown_connections 2024-04-16T05:35:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f66b413d640 1 --2- 172.21.15.86:0/2042454365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66ac10e9b0 0x7f66ac110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f66b413d640 1 --2- 172.21.15.86:0/2042454365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66ac101720 0x7f66ac10e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f66b413d640 1 --2- 172.21.15.86:0/2042454365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac100e00 0x7f66ac1011e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f66b413d640 1 -- 172.21.15.86:0/2042454365 >> 172.21.15.86:0/2042454365 conn(0x7f66ac078000 msgr2=0x7f66ac078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f7917e60640 1 -- 172.21.15.86:0/4153094517 shutdown_connections 2024-04-16T05:35:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f7917e60640 1 --2- 172.21.15.86:0/4153094517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910107350 0x7f791010ef20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f7917e60640 1 --2- 172.21.15.86:0/4153094517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7910102c30 0x7f7910106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f7917e60640 1 --2- 172.21.15.86:0/4153094517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7910102280 0x7f7910102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f7917e60640 1 -- 172.21.15.86:0/4153094517 >> 172.21.15.86:0/4153094517 conn(0x7f79100fdac0 msgr2=0x7f79100ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f66b413d640 1 -- 172.21.15.86:0/2042454365 shutdown_connections 2024-04-16T05:35:25.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f66b413d640 1 -- 172.21.15.86:0/2042454365 wait complete. 2024-04-16T05:35:25.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f7917e60640 1 -- 172.21.15.86:0/4153094517 shutdown_connections 2024-04-16T05:35:25.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.121+0000 7f7917e60640 1 -- 172.21.15.86:0/4153094517 wait complete. 2024-04-16T05:35:25.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7917e60640 1 Processor -- start 2024-04-16T05:35:25.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b413d640 1 Processor -- start 2024-04-16T05:35:25.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7917e60640 1 -- start start 2024-04-16T05:35:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b413d640 1 -- start start 2024-04-16T05:35:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7917e60640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910102280 0x7f791019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7917e60640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7910107350 0x7f791019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b413d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66ac100e00 0x7f66ac1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7917e60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7910198670 0x7f7910198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b413d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66ac101720 0x7f66ac1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b413d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac10e9b0 0x7f66ac19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b413d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1145d0 con 0x7f66ac10e9b0 2024-04-16T05:35:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b413d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac114450 con 0x7f66ac101720 2024-04-16T05:35:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b413d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac114750 con 0x7f66ac100e00 2024-04-16T05:35:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7917e60640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79101123f0 con 0x7f7910102280 2024-04-16T05:35:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7917e60640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7910112270 con 0x7f7910107350 2024-04-16T05:35:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7917e60640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7910112570 con 0x7f7910198670 2024-04-16T05:35:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7915bfc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910102280 0x7f791019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7915bfc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910102280 0x7f791019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59668/0 (socket says 172.21.15.86:59668) 2024-04-16T05:35:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f7915bfc640 1 -- 172.21.15.86:0/623051738 learned_addr learned my addr 172.21.15.86:0/623051738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b26da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac10e9b0 0x7f66ac19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b26da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac10e9b0 0x7f66ac19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59670/0 (socket says 172.21.15.86:59670) 2024-04-16T05:35:25.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b26da640 1 -- 172.21.15.86:0/2101149062 learned_addr learned my addr 172.21.15.86:0/2101149062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f79153fb640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7910107350 0x7f791019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.122+0000 7f66b16d8640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66ac101720 0x7f66ac1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7915bfc640 1 -- 172.21.15.86:0/623051738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7910198670 msgr2=0x7f7910198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f79163fd640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7910198670 0x7f7910198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7915bfc640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7910198670 0x7f7910198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7915bfc640 1 -- 172.21.15.86:0/623051738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7910107350 msgr2=0x7f791019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7915bfc640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7910107350 0x7f791019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7915bfc640 1 -- 172.21.15.86:0/623051738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7910199190 con 0x7f7910102280 2024-04-16T05:35:25.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66ac100e00 0x7f66ac1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f79153fb640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7910107350 0x7f791019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:25.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f79163fd640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7910198670 0x7f7910198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:25.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b26da640 1 -- 172.21.15.86:0/2101149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66ac100e00 msgr2=0x7f66ac1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b26da640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66ac100e00 0x7f66ac1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b26da640 1 -- 172.21.15.86:0/2101149062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66ac101720 msgr2=0x7f66ac1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b26da640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66ac101720 0x7f66ac1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b26da640 1 -- 172.21.15.86:0/2101149062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ac19b090 con 0x7f66ac10e9b0 2024-04-16T05:35:25.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66ac100e00 0x7f66ac1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:25.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b16d8640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66ac101720 0x7f66ac1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:25.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7915bfc640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910102280 0x7f791019e470 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f790c00d950 tx=0x7f790c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b26da640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac10e9b0 0x7f66ac19a910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f66a400e9d0 tx=0x7f66a400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7906ffd640 1 -- 172.21.15.86:0/623051738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c017070 con 0x7f7910102280 2024-04-16T05:35:25.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7910199480 con 0x7f7910102280 2024-04-16T05:35:25.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7906ffd640 1 -- 172.21.15.86:0/623051738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790c009c30 con 0x7f7910102280 2024-04-16T05:35:25.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7906ffd640 1 -- 172.21.15.86:0/623051738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c00f6b0 con 0x7f7910102280 2024-04-16T05:35:25.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66a2ffd640 1 -- 172.21.15.86:0/2101149062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a400cd80 con 0x7f66ac10e9b0 2024-04-16T05:35:25.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a5680 con 0x7f7910102280 2024-04-16T05:35:25.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b413d640 1 -- 172.21.15.86:0/2101149062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ac19b380 con 0x7f66ac10e9b0 2024-04-16T05:35:25.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66a2ffd640 1 -- 172.21.15.86:0/2101149062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a4005020 con 0x7f66ac10e9b0 2024-04-16T05:35:25.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66a2ffd640 1 -- 172.21.15.86:0/2101149062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a40116d0 con 0x7f66ac10e9b0 2024-04-16T05:35:25.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.123+0000 7f66b413d640 1 -- 172.21.15.86:0/2101149062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ac1a7620 con 0x7f66ac10e9b0 2024-04-16T05:35:25.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.124+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7910103750 con 0x7f7910102280 2024-04-16T05:35:25.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.125+0000 7f7906ffd640 1 -- 172.21.15.86:0/623051738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f790c0109d0 con 0x7f7910102280 2024-04-16T05:35:25.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.126+0000 7f7906ffd640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f78fc0a3ee0 0x7f78fc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.126+0000 7f79153fb640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f78fc0a3ee0 0x7f78fc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.126+0000 7f7906ffd640 1 -- 172.21.15.86:0/623051738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f790c0d0b00 con 0x7f7910102280 2024-04-16T05:35:25.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.126+0000 7f66b6e02740 1 Processor -- start 2024-04-16T05:35:25.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.126+0000 7f66a2ffd640 1 -- 172.21.15.86:0/2101149062 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f66a40051d0 con 0x7f66ac10e9b0 2024-04-16T05:35:25.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.126+0000 7f79153fb640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f78fc0a3ee0 0x7f78fc0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79101088a0 tx=0x7f7900005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b6e02740 1 -- start start 2024-04-16T05:35:25.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b1312be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13134a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b13134fb0 0x564b131373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b6e02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564b13090e90 con 0x564b1312c3d0 2024-04-16T05:35:25.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b6e02740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564b1302e4c0 con 0x564b13134fb0 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b6e02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b1308ce50 con 0x564b1312bab0 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b1ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b1ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13134a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59676/0 (socket says 172.21.15.86:59676) 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b1ed9640 1 -- 172.21.15.86:0/1423700537 learned_addr learned my addr 172.21.15.86:0/1423700537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66a2ffd640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66980a3e40 0x7f66980a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b16d8640 1 --2- 172.21.15.86:0/1423700537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b13134fb0 0x564b131373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b1ed9640 1 -- 172.21.15.86:0/1423700537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 msgr2=0x564b1312be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/1423700537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b1312be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b1ed9640 1 -- 172.21.15.86:0/1423700537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b13134fb0 msgr2=0x564b131373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/1423700537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b13134fb0 0x564b131373a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b1ed9640 1 -- 172.21.15.86:0/1423700537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b12e6b7e0 con 0x564b1312c3d0 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b26da640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66980a3e40 0x7f66980a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.127+0000 7f66b16d8640 1 --2- 172.21.15.86:0/1423700537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b13134fb0 0x564b131373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.128+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/1423700537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13134a70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f66a8006010 tx=0x7f66a802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef492d479d893b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.128+0000 7f66a2ffd640 1 -- 172.21.15.86:0/2101149062 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f66a40d1420 con 0x7f66ac10e9b0 2024-04-16T05:35:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.128+0000 7f66b26da640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66980a3e40 0x7f66980a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66a40ce6e0 tx=0x7f66a406c6e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.131+0000 7f6697fff640 1 -- 172.21.15.86:0/1423700537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a802cb40 con 0x564b1312c3d0 2024-04-16T05:35:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.131+0000 7f6697fff640 1 -- 172.21.15.86:0/1423700537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a8005040 con 0x564b1312c3d0 2024-04-16T05:35:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.131+0000 7f6697fff640 1 -- 172.21.15.86:0/1423700537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8004a80 con 0x564b1312c3d0 2024-04-16T05:35:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.131+0000 7f7906ffd640 1 -- 172.21.15.86:0/623051738 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f790c009de0 con 0x7f7910102280 2024-04-16T05:35:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 -- 172.21.15.86:0/1423700537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 msgr2=0x564b13134a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1423700537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13134a70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f66a8006010 tx=0x7f66a802c5d0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 -- 172.21.15.86:0/1423700537 shutdown_connections 2024-04-16T05:35:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1423700537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b13134fb0 0x564b131373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1423700537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13134a70 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1423700537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b1312be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 -- 172.21.15.86:0/1423700537 >> 172.21.15.86:0/1423700537 conn(0x564b131261e0 msgr2=0x564b131265c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 -- 172.21.15.86:0/1423700537 shutdown_connections 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 -- 172.21.15.86:0/1423700537 wait complete. 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 -- start start 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.132+0000 7f66b6e02740 1 Processor -- start 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312bab0 0x564b13131ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13132530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b13134fb0 0x564b13132a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b6e02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564b13137d40 con 0x564b1312c3d0 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b6e02740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564b13019410 con 0x564b1312bab0 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b6e02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b13137ec0 con 0x564b13134fb0 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b6e02740 10 client.0 _ll_register_callbacks cb 0x564b13014b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b1ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13132530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b1ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13132530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59684/0 (socket says 172.21.15.86:59684) 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b1ed9640 1 -- 172.21.15.86:0/158350569 learned_addr learned my addr 172.21.15.86:0/158350569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b16d8640 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312bab0 0x564b13131ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b16d8640 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b13134fb0 0x564b13132a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b1ed9640 1 -- 172.21.15.86:0/158350569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b13134fb0 msgr2=0x564b13132a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b13134fb0 0x564b13132a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b1ed9640 1 -- 172.21.15.86:0/158350569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312bab0 msgr2=0x564b13131ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312bab0 0x564b13131ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b1ed9640 1 -- 172.21.15.86:0/158350569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b12ecbf70 con 0x564b1312c3d0 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b16d8640 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b13134fb0 0x564b13132a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13132530 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f66a8005450 tx=0x7f66a8005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e17dad7e7ecae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66957fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.133+0000 7f66957fa640 1 -- 172.21.15.86:0/158350569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a804d060 con 0x564b1312c3d0 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66957fa640 1 -- 172.21.15.86:0/158350569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f66a8004bd0 con 0x564b1312c3d0 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b6e02740 1 -- 172.21.15.86:0/158350569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a8005040 con 0x564b1312c3d0 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66957fa640 1 -- 172.21.15.86:0/158350569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66a8054050 con 0x564b1312c3d0 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b6e02740 10 client.15750 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b6e02740 1 -- 172.21.15.86:0/158350569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b12d63170 con 0x564b1312c3d0 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66957fa640 1 -- 172.21.15.86:0/158350569 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a8005040 con 0x564b1312c3d0 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66957fa640 1 -- 172.21.15.86:0/158350569 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f66a803d430 con 0x564b1312c3d0 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b6e02740 10 client.15750 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b6e02740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b6e02740 20 client.15750 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b6e02740 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564b13145150 0x564b13147540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b6e02740 4 client.15750 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b6e02740 1 -- 172.21.15.86:0/158350569 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b1309e8b0 con 0x564b13145150 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.134+0000 7f66b16d8640 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564b13145150 0x564b13147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.135+0000 7f66957fa640 10 client.15750 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.135+0000 7f66b16d8640 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564b13145150 0x564b13147540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeccce54d4cbe410 server_cookie=ebd39b7673bc57ca in_seq=0 out_seq=0 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66957fa640 1 -- 172.21.15.86:0/158350569 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b1309e8b0 con 0x564b13145150 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66957fa640 10 client.15750 handle_command_reply: tid=0 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 client.15750 shutdown 2024-04-16T05:35:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 -- 172.21.15.86:0/158350569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 msgr2=0x564b13132530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13132530 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f66a8005450 tx=0x7f66a8005480 comp rx=0 tx=0).stop 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 -- 172.21.15.86:0/158350569 shutdown_connections 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564b13145150 0x564b13147540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b13134fb0 0x564b13132a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b1312c3d0 0x564b13132530 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 --2- 172.21.15.86:0/158350569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312bab0 0x564b13131ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66957fa640 0 client.15750 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 -- 172.21.15.86:0/158350569 >> 172.21.15.86:0/158350569 conn(0x564b131261e0 msgr2=0x564b13127f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 -- 172.21.15.86:0/158350569 shutdown_connections 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 1 -- 172.21.15.86:0/158350569 wait complete. 2024-04-16T05:35:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.136+0000 7f66b6e02740 20 client.15750 trim_cache size 0 max 16384 2024-04-16T05:35:25.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.153+0000 7f66b6e02740 1 Processor -- start 2024-04-16T05:35:25.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.153+0000 7f66b6e02740 1 -- start start 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b13134720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 0x564b13140c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1314cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b6e02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1b1620 con 0x564b13134fb0 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b6e02740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1b3160 con 0x564b1312c3d0 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b6e02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1b1cf0 con 0x564b1312bab0 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1314cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1314cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59692/0 (socket says 172.21.15.86:59692) 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 -- 172.21.15.86:0/2427810442 learned_addr learned my addr 172.21.15.86:0/2427810442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/2427810442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 0x564b13140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 --2- 172.21.15.86:0/2427810442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b13134720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 -- 172.21.15.86:0/2427810442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 msgr2=0x564b13134720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 --2- 172.21.15.86:0/2427810442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b13134720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 -- 172.21.15.86:0/2427810442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 msgr2=0x564b13140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 --2- 172.21.15.86:0/2427810442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 0x564b13140c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 -- 172.21.15.86:0/2427810442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b1312ae00 con 0x564b13134fb0 2024-04-16T05:35:25.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.154+0000 7f66b16d8640 1 --2- 172.21.15.86:0/2427810442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1314cb60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f669c0058c0 tx=0x7f669c00d590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01529043a1eaf8e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66977fe640 1 -- 172.21.15.86:0/2427810442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c01e940 con 0x564b13134fb0 2024-04-16T05:35:25.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66977fe640 1 -- 172.21.15.86:0/2427810442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c023070 con 0x564b13134fb0 2024-04-16T05:35:25.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66977fe640 1 -- 172.21.15.86:0/2427810442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c00f420 con 0x564b13134fb0 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66b6e02740 1 -- 172.21.15.86:0/2427810442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 msgr2=0x564b1314cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66b6e02740 1 --2- 172.21.15.86:0/2427810442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1314cb60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f669c0058c0 tx=0x7f669c00d590 comp rx=0 tx=0).stop 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66b6e02740 1 -- 172.21.15.86:0/2427810442 shutdown_connections 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66b6e02740 1 --2- 172.21.15.86:0/2427810442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1314cb60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66b6e02740 1 --2- 172.21.15.86:0/2427810442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 0x564b13140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66b6e02740 1 --2- 172.21.15.86:0/2427810442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b13134720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66b6e02740 1 -- 172.21.15.86:0/2427810442 >> 172.21.15.86:0/2427810442 conn(0x564b131261e0 msgr2=0x564b13125250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66b6e02740 1 -- 172.21.15.86:0/2427810442 shutdown_connections 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.155+0000 7f66b6e02740 1 -- 172.21.15.86:0/2427810442 wait complete. 2024-04-16T05:35:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b6e02740 1 -- start start 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b6e02740 1 Processor -- start 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b1313d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 0x564b1313d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b6e02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1313a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b6e02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1b2a90 con 0x564b13134fb0 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b6e02740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1b23c0 con 0x564b1312c3d0 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b6e02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1af5e0 con 0x564b1312bab0 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b6e02740 10 client.0 _ll_register_callbacks cb 0x7f66a8005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b16d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1313a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b16d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1313a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59700/0 (socket says 172.21.15.86:59700) 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b16d8640 1 -- 172.21.15.86:0/1454708690 learned_addr learned my addr 172.21.15.86:0/1454708690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.156+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 0x564b1313d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b16d8640 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b1313d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b16d8640 1 -- 172.21.15.86:0/1454708690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 msgr2=0x564b1313d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b16d8640 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b1313d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b16d8640 1 -- 172.21.15.86:0/1454708690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 msgr2=0x564b1313d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b16d8640 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 0x564b1313d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b16d8640 1 -- 172.21.15.86:0/1454708690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b1313a7b0 con 0x564b13134fb0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b16d8640 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1313a140 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f669c00dc90 tx=0x7f669c010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca44a2b9171f1471 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66957fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66957fa640 1 -- 172.21.15.86:0/1454708690 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c011da0 con 0x564b13134fb0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66957fa640 1 -- 172.21.15.86:0/1454708690 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f669c010430 con 0x564b13134fb0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b6e02740 1 -- 172.21.15.86:0/1454708690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669c023070 con 0x564b13134fb0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66957fa640 1 -- 172.21.15.86:0/1454708690 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f669c010630 con 0x564b13134fb0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b6e02740 10 client.15756 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66b6e02740 1 -- 172.21.15.86:0/1454708690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b131abd60 con 0x564b13134fb0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.157+0000 7f66957fa640 1 -- 172.21.15.86:0/1454708690 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c023070 con 0x564b13134fb0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66957fa640 1 -- 172.21.15.86:0/1454708690 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f669c004a00 con 0x564b13134fb0 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66b6e02740 10 client.15756 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:25.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66b6e02740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66b6e02740 20 client.15756 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564b131ac380 0x564b131ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66b6e02740 4 client.15756 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66b6e02740 1 -- 172.21.15.86:0/1454708690 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564b13098720 con 0x564b131ac380 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564b131ac380 0x564b131ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66957fa640 10 client.15756 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.158+0000 7f66b1ed9640 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564b131ac380 0x564b131ae770 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab8a133bbdc3bed8 server_cookie=2aab37b57300aa08 in_seq=0 out_seq=0 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.159+0000 7f66957fa640 1 -- 172.21.15.86:0/1454708690 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564b13098720 con 0x564b131ac380 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.159+0000 7f66957fa640 10 client.15756 handle_command_reply: tid=0 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.159+0000 7f66b6e02740 1 client.15756 shutdown 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.159+0000 7f66b6e02740 1 -- 172.21.15.86:0/1454708690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 msgr2=0x564b1313a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.159+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1313a140 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f669c00dc90 tx=0x7f669c010c30 comp rx=0 tx=0).stop 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b6e02740 1 -- 172.21.15.86:0/1454708690 shutdown_connections 2024-04-16T05:35:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564b131ac380 0x564b131ae770 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b13134fb0 0x564b1313a140 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b1312c3d0 0x564b1313d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b6e02740 1 --2- 172.21.15.86:0/1454708690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b1312bab0 0x564b1313d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66957fa640 0 client.15756 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b6e02740 1 -- 172.21.15.86:0/1454708690 >> 172.21.15.86:0/1454708690 conn(0x564b131261e0 msgr2=0x564b13131fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b6e02740 1 -- 172.21.15.86:0/1454708690 shutdown_connections 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b6e02740 1 -- 172.21.15.86:0/1454708690 wait complete. 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b6e02740 20 client.15756 trim_cache size 0 max 16384 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:25.163 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:35:25.164 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "f6bd3337-0902-404b-9c82-9cb535e69495", 2024-04-16T05:35:25.164 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:35:25.164 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 -- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66980a3e40 msgr2=0x7f66980a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66980a3e40 0x7f66980a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66a40ce6e0 tx=0x7f66a406c6e0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 -- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac10e9b0 msgr2=0x7f66ac19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac10e9b0 0x7f66ac19a910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f66a400e9d0 tx=0x7f66a400ee90 comp rx=0 tx=0).stop 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 -- 172.21.15.86:0/2101149062 shutdown_connections 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66ac10e9b0 0x7f66ac19a910 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66980a3e40 0x7f66980a6300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66ac101720 0x7f66ac1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 --2- 172.21.15.86:0/2101149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66ac100e00 0x7f66ac1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.160+0000 7f66b413d640 1 -- 172.21.15.86:0/2101149062 >> 172.21.15.86:0/2101149062 conn(0x7f66ac078000 msgr2=0x7f66ac110010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.161+0000 7f66b413d640 1 -- 172.21.15.86:0/2101149062 shutdown_connections 2024-04-16T05:35:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.161+0000 7f66b413d640 1 -- 172.21.15.86:0/2101149062 wait complete. 2024-04-16T05:35:25.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:35:25.170 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.279+0000 7f6d230c5640 1 -- 172.21.15.86:0/2155033311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c103ba0 msgr2=0x7f6d1c10c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.279+0000 7f6d230c5640 1 --2- 172.21.15.86:0/2155033311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c103ba0 0x7f6d1c10c080 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009900 tx=0x7f6d0802bd60 comp rx=0 tx=0).stop 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.279+0000 7f6d230c5640 1 -- 172.21.15.86:0/2155033311 shutdown_connections 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.279+0000 7f6d230c5640 1 --2- 172.21.15.86:0/2155033311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d1c10c5c0 0x7f6d1c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.279+0000 7f6d230c5640 1 --2- 172.21.15.86:0/2155033311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c103ba0 0x7f6d1c10c080 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.279+0000 7f6d230c5640 1 --2- 172.21.15.86:0/2155033311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d1c103280 0x7f6d1c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.279+0000 7f6d230c5640 1 -- 172.21.15.86:0/2155033311 >> 172.21.15.86:0/2155033311 conn(0x7f6d1c078000 msgr2=0x7f6d1c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.279+0000 7f6d230c5640 1 -- 172.21.15.86:0/2155033311 shutdown_connections 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.279+0000 7f6d230c5640 1 -- 172.21.15.86:0/2155033311 wait complete. 2024-04-16T05:35:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d230c5640 1 Processor -- start 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d230c5640 1 -- start start 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d230c5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d1c103280 0x7f6d1c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d230c5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d1c103ba0 0x7f6d1c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d230c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c10c5c0 0x7f6d1c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d230c5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1123d0 con 0x7f6d1c10c5c0 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d230c5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c112250 con 0x7f6d1c103ba0 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d230c5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c112550 con 0x7f6d1c103280 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d21662640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c10c5c0 0x7f6d1c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d21662640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c10c5c0 0x7f6d1c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59714/0 (socket says 172.21.15.86:59714) 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d21662640 1 -- 172.21.15.86:0/21361000 learned_addr learned my addr 172.21.15.86:0/21361000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d20e61640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d1c103280 0x7f6d1c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d1c103ba0 0x7f6d1c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d21662640 1 -- 172.21.15.86:0/21361000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d1c103280 msgr2=0x7f6d1c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d21662640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d1c103280 0x7f6d1c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d21662640 1 -- 172.21.15.86:0/21361000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d1c103ba0 msgr2=0x7f6d1c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d21662640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d1c103ba0 0x7f6d1c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.280+0000 7f6d21662640 1 -- 172.21.15.86:0/21361000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1c198fd0 con 0x7f6d1c10c5c0 2024-04-16T05:35:25.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.281+0000 7f6d21662640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c10c5c0 0x7f6d1c198710 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400eaa0 tx=0x7f6d1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.281+0000 7f6d19ffb640 1 -- 172.21.15.86:0/21361000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400cdb0 con 0x7f6d1c10c5c0 2024-04-16T05:35:25.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.281+0000 7f6d19ffb640 1 -- 172.21.15.86:0/21361000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d14005020 con 0x7f6d1c10c5c0 2024-04-16T05:35:25.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.281+0000 7f6d230c5640 1 -- 172.21.15.86:0/21361000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1c1992c0 con 0x7f6d1c10c5c0 2024-04-16T05:35:25.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.281+0000 7f6d19ffb640 1 -- 172.21.15.86:0/21361000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d140116b0 con 0x7f6d1c10c5c0 2024-04-16T05:35:25.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.281+0000 7f6d230c5640 1 -- 172.21.15.86:0/21361000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d1c1a55d0 con 0x7f6d1c10c5c0 2024-04-16T05:35:25.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.283+0000 7f6d19ffb640 1 -- 172.21.15.86:0/21361000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6d14005560 con 0x7f6d1c10c5c0 2024-04-16T05:35:25.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d25d83740 1 Processor -- start 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d25d83740 1 -- start start 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d19ffb640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6d0c0a3e40 0x7f6d0c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 0x55fea2e1c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1c770 0x55fea2e24e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e27740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6d0c0a3e40 0x7f6d0c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d19ffb640 1 -- 172.21.15.86:0/21361000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6d140d0f70 con 0x7f6d1c10c5c0 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d25d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fea2d81180 con 0x55fea2e1c770 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d25d83740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fea2d1e360 con 0x55fea2e25350 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d25d83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fea2d7d140 con 0x55fea2e1be50 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1c770 0x55fea2e24e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1c770 0x55fea2e24e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59726/0 (socket says 172.21.15.86:59726) 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 -- 172.21.15.86:0/4187864027 learned_addr learned my addr 172.21.15.86:0/4187864027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d20e61640 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 0x55fea2e1c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d21662640 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e27740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 -- 172.21.15.86:0/4187864027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 msgr2=0x55fea2e1c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 0x55fea2e1c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 -- 172.21.15.86:0/4187864027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 msgr2=0x55fea2e27740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e27740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 -- 172.21.15.86:0/4187864027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea2b5d7d0 con 0x55fea2e1c770 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d20e61640 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 0x55fea2e1c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.284+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6d0c0a3e40 0x7f6d0c0a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d080096c0 tx=0x7f6d0802c810 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1c770 0x55fea2e24e10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6d08036040 tx=0x7f6d08048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92d316ae75d7930 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d06ffd640 1 -- 172.21.15.86:0/4187864027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0802cd10 con 0x55fea2e1c770 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d06ffd640 1 -- 172.21.15.86:0/4187864027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d08004a90 con 0x55fea2e1c770 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d06ffd640 1 -- 172.21.15.86:0/4187864027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08003650 con 0x55fea2e1c770 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d25d83740 1 -- 172.21.15.86:0/4187864027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1c770 msgr2=0x55fea2e24e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d25d83740 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1c770 0x55fea2e24e10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6d08036040 tx=0x7f6d08048000 comp rx=0 tx=0).stop 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d25d83740 1 -- 172.21.15.86:0/4187864027 shutdown_connections 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d25d83740 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e27740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d25d83740 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1c770 0x55fea2e24e10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d25d83740 1 --2- 172.21.15.86:0/4187864027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 0x55fea2e1c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.285+0000 7f6d25d83740 1 -- 172.21.15.86:0/4187864027 >> 172.21.15.86:0/4187864027 conn(0x55fea2e16580 msgr2=0x55fea2e16960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 -- 172.21.15.86:0/4187864027 shutdown_connections 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 -- 172.21.15.86:0/4187864027 wait complete. 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 -- start start 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 Processor -- start 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 0x55fea2e223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e1c770 0x55fea2e22910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e25350 0x55fea2e22e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fea2e2c4f0 con 0x55fea2e25350 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fea2d3a2c0 con 0x55fea2e1c770 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fea2e2c670 con 0x55fea2e1be50 2024-04-16T05:35:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d25d83740 10 client.0 _ll_register_callbacks cb 0x55fea2d21090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d21662640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e25350 0x55fea2e22e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d21662640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e25350 0x55fea2e22e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59738/0 (socket says 172.21.15.86:59738) 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.286+0000 7f6d21662640 1 -- 172.21.15.86:0/411292733 learned_addr learned my addr 172.21.15.86:0/411292733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d20e61640 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 0x55fea2e223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e1c770 0x55fea2e22910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d21662640 1 -- 172.21.15.86:0/411292733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 msgr2=0x55fea2e223d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d21662640 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 0x55fea2e223d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d21662640 1 -- 172.21.15.86:0/411292733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e1c770 msgr2=0x55fea2e22910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d21662640 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e1c770 0x55fea2e22910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d21662640 1 -- 172.21.15.86:0/411292733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea2a933f0 con 0x55fea2e25350 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e1c770 0x55fea2e22910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d21662640 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e25350 0x55fea2e22e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6d1409c990 tx=0x7f6d1409cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df28decfebd84d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6cfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6cfbfff640 1 -- 172.21.15.86:0/411292733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1401f040 con 0x55fea2e25350 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6cfbfff640 1 -- 172.21.15.86:0/411292733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6d140ce6b0 con 0x55fea2e25350 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d25d83740 1 -- 172.21.15.86:0/411292733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d08004a90 con 0x55fea2e25350 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6cfbfff640 1 -- 172.21.15.86:0/411292733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d1409be50 con 0x55fea2e25350 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d25d83740 10 client.15768 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6d25d83740 1 -- 172.21.15.86:0/411292733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fea2a530f0 con 0x55fea2e25350 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.287+0000 7f6cfbfff640 1 -- 172.21.15.86:0/411292733 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d08004a90 con 0x55fea2e25350 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6cfbfff640 1 -- 172.21.15.86:0/411292733 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6d1400f040 con 0x55fea2e25350 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6d25d83740 10 client.15768 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6d25d83740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6d25d83740 20 client.15768 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6d25d83740 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fea2e39b80 0x55fea2e3bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6d25d83740 4 client.15768 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6d25d83740 1 -- 172.21.15.86:0/411292733 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fea2d8eba0 con 0x55fea2e39b80 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6d20e61640 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fea2e39b80 0x55fea2e3bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6cfbfff640 10 client.15768 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:25.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.288+0000 7f6d20e61640 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fea2e39b80 0x55fea2e3bf70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c28dd76c523fe145 server_cookie=4a6fc87b37e83781 in_seq=0 out_seq=0 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6cfbfff640 1 -- 172.21.15.86:0/411292733 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fea2d8eba0 con 0x55fea2e39b80 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6cfbfff640 10 client.15768 handle_command_reply: tid=0 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 client.15768 shutdown 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 -- 172.21.15.86:0/411292733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e25350 msgr2=0x55fea2e22e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e25350 0x55fea2e22e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6d1409c990 tx=0x7f6d1409cb40 comp rx=0 tx=0).stop 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 -- 172.21.15.86:0/411292733 shutdown_connections 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fea2e39b80 0x55fea2e3bf70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e25350 0x55fea2e22e50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e1c770 0x55fea2e22910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6cfbfff640 0 client.15768 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 --2- 172.21.15.86:0/411292733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1be50 0x55fea2e223d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 -- 172.21.15.86:0/411292733 >> 172.21.15.86:0/411292733 conn(0x55fea2e16580 msgr2=0x55fea2e182d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 -- 172.21.15.86:0/411292733 shutdown_connections 2024-04-16T05:35:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 1 -- 172.21.15.86:0/411292733 wait complete. 2024-04-16T05:35:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.289+0000 7f6d25d83740 20 client.15768 trim_cache size 0 max 16384 2024-04-16T05:35:25.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.327+0000 7f6d25d83740 1 Processor -- start 2024-04-16T05:35:25.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.327+0000 7f6d25d83740 1 -- start start 2024-04-16T05:35:25.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.327+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.327+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 0x55fea2e18240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.327+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e22c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.327+0000 7f6d25d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1ad310 con 0x55fea2e1be50 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.327+0000 7f6d25d83740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1aee50 con 0x55fea2e25350 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.327+0000 7f6d25d83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1ad9e0 con 0x55fea2e1c770 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d20e61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d20e61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e24b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59750/0 (socket says 172.21.15.86:59750) 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d20e61640 1 -- 172.21.15.86:0/3144506059 learned_addr learned my addr 172.21.15.86:0/3144506059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/3144506059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 0x55fea2e18240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d20e61640 1 -- 172.21.15.86:0/3144506059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 msgr2=0x55fea2e18240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d21662640 1 --2- 172.21.15.86:0/3144506059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e22c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d20e61640 1 --2- 172.21.15.86:0/3144506059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 0x55fea2e18240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d20e61640 1 -- 172.21.15.86:0/3144506059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 msgr2=0x55fea2e22c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d20e61640 1 --2- 172.21.15.86:0/3144506059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e22c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d20e61640 1 -- 172.21.15.86:0/3144506059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea2e27ab0 con 0x55fea2e1be50 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d20e61640 1 --2- 172.21.15.86:0/3144506059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e24b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6d10000c00 tx=0x7f6d10011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b85fc9604fd29d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d067fc640 1 -- 172.21.15.86:0/3144506059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10021030 con 0x55fea2e1be50 2024-04-16T05:35:25.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d067fc640 1 -- 172.21.15.86:0/3144506059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d1001d900 con 0x55fea2e1be50 2024-04-16T05:35:25.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.328+0000 7f6d067fc640 1 -- 172.21.15.86:0/3144506059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100049c0 con 0x55fea2e1be50 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 -- 172.21.15.86:0/3144506059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 msgr2=0x55fea2e24b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 --2- 172.21.15.86:0/3144506059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e24b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6d10000c00 tx=0x7f6d10011810 comp rx=0 tx=0).stop 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 -- 172.21.15.86:0/3144506059 shutdown_connections 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 --2- 172.21.15.86:0/3144506059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e22c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 --2- 172.21.15.86:0/3144506059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 0x55fea2e18240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 --2- 172.21.15.86:0/3144506059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e24b00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 -- 172.21.15.86:0/3144506059 >> 172.21.15.86:0/3144506059 conn(0x55fea2e16580 msgr2=0x55fea2e14ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 -- 172.21.15.86:0/3144506059 shutdown_connections 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 -- 172.21.15.86:0/3144506059 wait complete. 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 -- start start 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.329+0000 7f6d25d83740 1 Processor -- start 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e2fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 0x55fea2e2ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d25d83740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e9f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d25d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1ae780 con 0x55fea2e1be50 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d25d83740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1ae0b0 con 0x55fea2e25350 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d25d83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1ab2d0 con 0x55fea2e1c770 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d25d83740 10 client.0 _ll_register_callbacks cb 0x7f6d14098ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d20e61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e2fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d20e61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e2fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59764/0 (socket says 172.21.15.86:59764) 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d20e61640 1 -- 172.21.15.86:0/738129899 learned_addr learned my addr 172.21.15.86:0/738129899 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 0x55fea2e2ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d21662640 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e9f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d20e61640 1 -- 172.21.15.86:0/738129899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 msgr2=0x55fea2e2ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d20e61640 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 0x55fea2e2ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d20e61640 1 -- 172.21.15.86:0/738129899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 msgr2=0x55fea2e9f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d20e61640 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e9f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d20e61640 1 -- 172.21.15.86:0/738129899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea2e9f5b0 con 0x55fea2e1be50 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6d20e61640 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e2fa40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000b010 tx=0x7f6d1001d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43da69a4e133033f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.330+0000 7f6cfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6cfbfff640 1 -- 172.21.15.86:0/738129899 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1002b070 con 0x55fea2e1be50 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6cfbfff640 1 -- 172.21.15.86:0/738129899 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6d10005320 con 0x55fea2e1be50 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6d25d83740 1 -- 172.21.15.86:0/738129899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d1001d900 con 0x55fea2e1be50 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6cfbfff640 1 -- 172.21.15.86:0/738129899 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d10037050 con 0x55fea2e1be50 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6d25d83740 10 client.15774 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6d25d83740 1 -- 172.21.15.86:0/738129899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fea2ea0870 con 0x55fea2e1be50 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6cfbfff640 1 -- 172.21.15.86:0/738129899 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d1001d900 con 0x55fea2e1be50 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6cfbfff640 1 -- 172.21.15.86:0/738129899 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6d1000f7d0 con 0x55fea2e1be50 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6d25d83740 10 client.15774 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:25.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6d25d83740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6d25d83740 20 client.15774 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6d25d83740 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fea2ea0f60 0x55fea2ea3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6d25d83740 4 client.15774 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.331+0000 7f6d25d83740 1 -- 172.21.15.86:0/738129899 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fea2d88a10 con 0x55fea2ea0f60 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fea2ea0f60 0x55fea2ea3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d1bfff640 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fea2ea0f60 0x55fea2ea3380 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f9583a1e394b759 server_cookie=5d0e94a55c490e64 in_seq=0 out_seq=0 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6cfbfff640 10 client.15774 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6cfbfff640 1 -- 172.21.15.86:0/738129899 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fea2d88a10 con 0x55fea2ea0f60 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6cfbfff640 10 client.15774 handle_command_reply: tid=0 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 client.15774 shutdown 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 -- 172.21.15.86:0/738129899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 msgr2=0x55fea2e2fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e2fa40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000b010 tx=0x7f6d1001d810 comp rx=0 tx=0).stop 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 -- 172.21.15.86:0/738129899 shutdown_connections 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fea2ea0f60 0x55fea2ea3380 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fea2e25350 0x55fea2e9f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6cfbfff640 0 client.15774 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea2e1c770 0x55fea2e2ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 --2- 172.21.15.86:0/738129899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea2e1be50 0x55fea2e2fa40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 -- 172.21.15.86:0/738129899 >> 172.21.15.86:0/738129899 conn(0x55fea2e16580 msgr2=0x55fea2e15760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 -- 172.21.15.86:0/738129899 shutdown_connections 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.332+0000 7f6d25d83740 1 -- 172.21.15.86:0/738129899 wait complete. 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d25d83740 20 client.15774 trim_cache size 0 max 16384 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:35:25.337 INFO:teuthology.orchestra.run.smithi086.stdout: "f6bd3337-0902-404b-9c82-9cb535e69495": { 2024-04-16T05:35:25.338 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:35:25.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "f6bd3337-0902-404b-9c82-9cb535e69495", 2024-04-16T05:35:25.338 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:35:25.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:35:25.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:35:25.338 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 -- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6d0c0a3e40 msgr2=0x7f6d0c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6d0c0a3e40 0x7f6d0c0a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d080096c0 tx=0x7f6d0802c810 comp rx=0 tx=0).stop 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 -- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c10c5c0 msgr2=0x7f6d1c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c10c5c0 0x7f6d1c198710 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400eaa0 tx=0x7f6d1400ef60 comp rx=0 tx=0).stop 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 -- 172.21.15.86:0/21361000 shutdown_connections 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c10c5c0 0x7f6d1c198710 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6d0c0a3e40 0x7f6d0c0a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d1c103ba0 0x7f6d1c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 --2- 172.21.15.86:0/21361000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d1c103280 0x7f6d1c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 -- 172.21.15.86:0/21361000 >> 172.21.15.86:0/21361000 conn(0x7f6d1c078000 msgr2=0x7f6d1c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 -- 172.21.15.86:0/21361000 shutdown_connections 2024-04-16T05:35:25.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.333+0000 7f6d230c5640 1 -- 172.21.15.86:0/21361000 wait complete. 2024-04-16T05:35:25.342 INFO:tasks.cephfs.filesystem:scrub status for tag:f6bd3337-0902-404b-9c82-9cb535e69495 - {'path': '/', 'tag': 'f6bd3337-0902-404b-9c82-9cb535e69495', 'options': 'recursive,force'} 2024-04-16T05:35:25.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.366+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f791010a5f0 con 0x7f78fc0a3ee0 2024-04-16T05:35:25.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.431+0000 7f7906ffd640 1 -- 172.21.15.86:0/623051738 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7900010a80 con 0x7f78fc0a3ee0 2024-04-16T05:35:25.430 INFO:teuthology.orchestra.run.smithi086.stdout:Schedule set for path / 2024-04-16T05:35:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f78fc0a3ee0 msgr2=0x7f78fc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f78fc0a3ee0 0x7f78fc0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79101088a0 tx=0x7f7900005f90 comp rx=0 tx=0).stop 2024-04-16T05:35:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910102280 msgr2=0x7f791019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910102280 0x7f791019e470 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f790c00d950 tx=0x7f790c00de10 comp rx=0 tx=0).stop 2024-04-16T05:35:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 shutdown_connections 2024-04-16T05:35:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7910198670 0x7f7910198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f78fc0a3ee0 0x7f78fc0a63a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7910107350 0x7f791019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 --2- 172.21.15.86:0/623051738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910102280 0x7f791019e470 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.433+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 >> 172.21.15.86:0/623051738 conn(0x7f79100fdac0 msgr2=0x7f79100ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.434+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 shutdown_connections 2024-04-16T05:35:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.434+0000 7f7917e60640 1 -- 172.21.15.86:0/623051738 wait complete. 2024-04-16T05:35:25.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-16T05:35:25.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.591+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3995411965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8100670 msgr2=0x7fd7c8100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.591+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3995411965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8100670 0x7fd7c8100af0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009930 tx=0x7fd7b002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:25.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.591+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3995411965 shutdown_connections 2024-04-16T05:35:25.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.591+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3995411965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c8101030 0x7fd7c810d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.591+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3995411965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8100670 0x7fd7c8100af0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.591+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3995411965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c8106680 0x7fd7c8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.591+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3995411965 >> 172.21.15.86:0/3995411965 conn(0x7fd7c8077420 msgr2=0x7fd7c80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.591+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3995411965 shutdown_connections 2024-04-16T05:35:25.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.591+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3995411965 wait complete. 2024-04-16T05:35:25.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7ccfa8640 1 Processor -- start 2024-04-16T05:35:25.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7ccfa8640 1 -- start start 2024-04-16T05:35:25.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7ccfa8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c8100670 0x7fd7c8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7ccfa8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c8101030 0x7fd7c8199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7ccfa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8106680 0x7fd7c81a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7ccfa8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8111310 con 0x7fd7c8106680 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7ccfa8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8111190 con 0x7fd7c8100670 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7ccfa8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8111490 con 0x7fd7c8101030 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7c6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8106680 0x7fd7c81a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7c6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8106680 0x7fd7c81a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59784/0 (socket says 172.21.15.86:59784) 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7c6d9d640 1 -- 172.21.15.86:0/3970482214 learned_addr learned my addr 172.21.15.86:0/3970482214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.592+0000 7fd7c659c640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c8100670 0x7fd7c8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7bffff640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c8101030 0x7fd7c8199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7c6d9d640 1 -- 172.21.15.86:0/3970482214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c8101030 msgr2=0x7fd7c8199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7c6d9d640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c8101030 0x7fd7c8199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7c6d9d640 1 -- 172.21.15.86:0/3970482214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c8100670 msgr2=0x7fd7c8199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7c6d9d640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c8100670 0x7fd7c8199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7c6d9d640 1 -- 172.21.15.86:0/3970482214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c81a4c50 con 0x7fd7c8106680 2024-04-16T05:35:25.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7c6d9d640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8106680 0x7fd7c81a4710 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000eaa0 tx=0x7fd7c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7bf7fe640 1 -- 172.21.15.86:0/3970482214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000cdb0 con 0x7fd7c8106680 2024-04-16T05:35:25.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7bf7fe640 1 -- 172.21.15.86:0/3970482214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c0005020 con 0x7fd7c8106680 2024-04-16T05:35:25.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c81a4e20 con 0x7fd7c8106680 2024-04-16T05:35:25.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7bf7fe640 1 -- 172.21.15.86:0/3970482214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c00116b0 con 0x7fd7c8106680 2024-04-16T05:35:25.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.593+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c81a52f0 con 0x7fd7c8106680 2024-04-16T05:35:25.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.594+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c804fd90 con 0x7fd7c8106680 2024-04-16T05:35:25.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.596+0000 7fd7bf7fe640 1 -- 172.21.15.86:0/3970482214 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd7c0005560 con 0x7fd7c8106680 2024-04-16T05:35:25.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.596+0000 7fd7bf7fe640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7b40a3ee0 0x7fd7b40a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.597+0000 7fd7bf7fe640 1 -- 172.21.15.86:0/3970482214 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd7c00d0fc0 con 0x7fd7c8106680 2024-04-16T05:35:25.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.597+0000 7fd7c659c640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7b40a3ee0 0x7fd7b40a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.597+0000 7fd7c659c640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7b40a3ee0 0x7fd7b40a63a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800bb60 tx=0x7fd7b800b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.603+0000 7fd7bf7fe640 1 -- 172.21.15.86:0/3970482214 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7c0016030 con 0x7fd7c8106680 2024-04-16T05:35:25.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:25 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3772010140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:25.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:25 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2316973977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:25.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:25 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:35:25.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:25 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:25.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:25 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:35:25.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:25 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:25.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.805+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fd7c8062440 con 0x7fd7b40a3ee0 2024-04-16T05:35:25.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.867+0000 7fd7bf7fe640 1 -- 172.21.15.86:0/3970482214 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fd7c8062440 con 0x7fd7b40a3ee0 2024-04-16T05:35:25.866 INFO:teuthology.orchestra.run.smithi086.stdout:Retention added to path / 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7b40a3ee0 msgr2=0x7fd7b40a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7b40a3ee0 0x7fd7b40a63a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800bb60 tx=0x7fd7b800b390 comp rx=0 tx=0).stop 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8106680 msgr2=0x7fd7c81a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8106680 0x7fd7c81a4710 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000eaa0 tx=0x7fd7c000ef60 comp rx=0 tx=0).stop 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 shutdown_connections 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c8106680 0x7fd7c81a4710 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c8101030 0x7fd7c8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7b40a3ee0 0x7fd7b40a63a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 --2- 172.21.15.86:0/3970482214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c8100670 0x7fd7c8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 >> 172.21.15.86:0/3970482214 conn(0x7fd7c8077420 msgr2=0x7fd7c8103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 shutdown_connections 2024-04-16T05:35:25.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.870+0000 7fd7ccfa8640 1 -- 172.21.15.86:0/3970482214 wait complete. 2024-04-16T05:35:25.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-16T05:35:25.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:25 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3772010140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:25.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:25 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2316973977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:25.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:25 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:35:25.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:25 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:25.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:25 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:35:25.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:25 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:25 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3772010140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:25 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2316973977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:25 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:35:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:25 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:25 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:35:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:25 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.986+0000 7f6a56bbd640 1 -- 172.21.15.86:0/3040370251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a501011a0 msgr2=0x7f6a50101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.986+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/3040370251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a501011a0 0x7f6a50101580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009900 tx=0x7f6a4c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.986+0000 7f6a56bbd640 1 -- 172.21.15.86:0/3040370251 shutdown_connections 2024-04-16T05:35:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.986+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/3040370251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50106220 0x7f6a5010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.986+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/3040370251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50101b50 0x7f6a50105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.986+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/3040370251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a501011a0 0x7f6a50101580 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.986+0000 7f6a56bbd640 1 -- 172.21.15.86:0/3040370251 >> 172.21.15.86:0/3040370251 conn(0x7f6a50078080 msgr2=0x7f6a50078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.986+0000 7f6a56bbd640 1 -- 172.21.15.86:0/3040370251 shutdown_connections 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.986+0000 7f6a56bbd640 1 -- 172.21.15.86:0/3040370251 wait complete. 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a56bbd640 1 Processor -- start 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a56bbd640 1 -- start start 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a56bbd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a501011a0 0x7f6a5010e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a56bbd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50106220 0x7f6a5010ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a56bbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a5010f200 0x7f6a501a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a56bbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501123f0 con 0x7f6a5010f200 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a56bbd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6a50112270 con 0x7f6a50106220 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a56bbd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a50112570 con 0x7f6a501011a0 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a5515a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a5010f200 0x7f6a501a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a5515a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a5010f200 0x7f6a501a6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59800/0 (socket says 172.21.15.86:59800) 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a5515a640 1 -- 172.21.15.86:0/2167592456 learned_addr learned my addr 172.21.15.86:0/2167592456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a54959640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a501011a0 0x7f6a5010e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.987+0000 7f6a47fff640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50106220 0x7f6a5010ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a5515a640 1 -- 172.21.15.86:0/2167592456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a501011a0 msgr2=0x7f6a5010e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a5515a640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a501011a0 0x7f6a5010e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a5515a640 1 -- 172.21.15.86:0/2167592456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50106220 msgr2=0x7f6a5010ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a5515a640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50106220 0x7f6a5010ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a5515a640 1 -- 172.21.15.86:0/2167592456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a501a70a0 con 0x7f6a5010f200 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a5515a640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a5010f200 0x7f6a501a6a00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800eaa0 tx=0x7f6a4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a45ffb640 1 -- 172.21.15.86:0/2167592456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800cdb0 con 0x7f6a5010f200 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a45ffb640 1 -- 172.21.15.86:0/2167592456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a48005020 con 0x7f6a5010f200 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a501a7390 con 0x7f6a5010f200 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a45ffb640 1 -- 172.21.15.86:0/2167592456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a480116b0 con 0x7f6a5010f200 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.988+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a501a78e0 con 0x7f6a5010f200 2024-04-16T05:35:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.989+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5004fd90 con 0x7f6a5010f200 2024-04-16T05:35:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.990+0000 7f6a45ffb640 1 -- 172.21.15.86:0/2167592456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6a48005560 con 0x7f6a5010f200 2024-04-16T05:35:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.991+0000 7f6a45ffb640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6a380a3ee0 0x7f6a380a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.991+0000 7f6a54959640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6a380a3ee0 0x7f6a380a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.991+0000 7f6a45ffb640 1 -- 172.21.15.86:0/2167592456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6a480d0f40 con 0x7f6a5010f200 2024-04-16T05:35:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.991+0000 7f6a54959640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6a380a3ee0 0x7f6a380a63a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c006010 tx=0x7f6a4c005dd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:25.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:25.996+0000 7f6a45ffb640 1 -- 172.21.15.86:0/2167592456 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a48016030 con 0x7f6a5010f200 2024-04-16T05:35:26.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.194+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6a501094c0 con 0x7f6a380a3ee0 2024-04-16T05:35:26.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.201+0000 7f6a45ffb640 1 -- 172.21.15.86:0/2167592456 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f6a4c02c880 con 0x7f6a380a3ee0 2024-04-16T05:35:26.200 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-16T00:00:00", "created": "2024-04-16T05:35:25", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-16T05:35:26.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6a380a3ee0 msgr2=0x7f6a380a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:26.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6a380a3ee0 0x7f6a380a63a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c006010 tx=0x7f6a4c005dd0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a5010f200 msgr2=0x7f6a501a6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:26.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a5010f200 0x7f6a501a6a00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800eaa0 tx=0x7f6a4800ef60 comp rx=0 tx=0).stop 2024-04-16T05:35:26.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 shutdown_connections 2024-04-16T05:35:26.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a5010f200 0x7f6a501a6a00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6a380a3ee0 0x7f6a380a63a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a50106220 0x7f6a5010ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 --2- 172.21.15.86:0/2167592456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a501011a0 0x7f6a5010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 >> 172.21.15.86:0/2167592456 conn(0x7f6a50078080 msgr2=0x7f6a500feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:26.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 shutdown_connections 2024-04-16T05:35:26.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.203+0000 7f6a56bbd640 1 -- 172.21.15.86:0/2167592456 wait complete. 2024-04-16T05:35:26.213 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-16T05:35:26.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.320+0000 7f3d0b532640 1 -- 172.21.15.86:0/2769070406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d04103bd0 msgr2=0x7f3d0410c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:26.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.320+0000 7f3d0b532640 1 --2- 172.21.15.86:0/2769070406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d04103bd0 0x7f3d0410c0b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009ca0 tx=0x7f3cf802c180 comp rx=0 tx=0).stop 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.320+0000 7f3d0b532640 1 -- 172.21.15.86:0/2769070406 shutdown_connections 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.320+0000 7f3d0b532640 1 --2- 172.21.15.86:0/2769070406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d0410c680 0x7f3d0410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.320+0000 7f3d0b532640 1 --2- 172.21.15.86:0/2769070406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d04103bd0 0x7f3d0410c0b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.320+0000 7f3d0b532640 1 --2- 172.21.15.86:0/2769070406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d041032b0 0x7f3d04103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.320+0000 7f3d0b532640 1 -- 172.21.15.86:0/2769070406 >> 172.21.15.86:0/2769070406 conn(0x7f3d04077280 msgr2=0x7f3d04077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.320+0000 7f3d0b532640 1 -- 172.21.15.86:0/2769070406 shutdown_connections 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.320+0000 7f3d0b532640 1 -- 172.21.15.86:0/2769070406 wait complete. 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d0b532640 1 Processor -- start 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d0b532640 1 -- start start 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d0b532640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d041032b0 0x7f3d0419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d0b532640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d04103bd0 0x7f3d0419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d0b532640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d0410c680 0x7f3d041986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d0b532640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04112410 con 0x7f3d041032b0 2024-04-16T05:35:26.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d0b532640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04112290 con 0x7f3d04103bd0 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d0b532640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04112590 con 0x7f3d0410c680 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d092ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d041032b0 0x7f3d0419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d092ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d041032b0 0x7f3d0419e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59816/0 (socket says 172.21.15.86:59816) 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d092ce640 1 -- 172.21.15.86:0/4283591925 learned_addr learned my addr 172.21.15.86:0/4283591925 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d09acf640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d0410c680 0x7f3d041986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d08acd640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d04103bd0 0x7f3d0419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d092ce640 1 -- 172.21.15.86:0/4283591925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d0410c680 msgr2=0x7f3d041986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d092ce640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d0410c680 0x7f3d041986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d092ce640 1 -- 172.21.15.86:0/4283591925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d04103bd0 msgr2=0x7f3d0419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d092ce640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d04103bd0 0x7f3d0419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d092ce640 1 -- 172.21.15.86:0/4283591925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d04198f70 con 0x7f3d041032b0 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d092ce640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d041032b0 0x7f3d0419e5a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00d770 tx=0x7f3cfc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:26.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.321+0000 7f3d027fc640 1 -- 172.21.15.86:0/4283591925 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc017070 con 0x7f3d041032b0 2024-04-16T05:35:26.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.322+0000 7f3d027fc640 1 -- 172.21.15.86:0/4283591925 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cfc005500 con 0x7f3d041032b0 2024-04-16T05:35:26.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.322+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d04199260 con 0x7f3d041032b0 2024-04-16T05:35:26.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.322+0000 7f3d027fc640 1 -- 172.21.15.86:0/4283591925 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc004e00 con 0x7f3d041032b0 2024-04-16T05:35:26.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.322+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d041a5520 con 0x7f3d041032b0 2024-04-16T05:35:26.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.322+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0406b050 con 0x7f3d041032b0 2024-04-16T05:35:26.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.324+0000 7f3d027fc640 1 -- 172.21.15.86:0/4283591925 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3cfc004fa0 con 0x7f3d041032b0 2024-04-16T05:35:26.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.324+0000 7f3d027fc640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cec0a3ee0 0x7f3cec0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:26.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.324+0000 7f3d08acd640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cec0a3ee0 0x7f3cec0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:26.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.324+0000 7f3d027fc640 1 -- 172.21.15.86:0/4283591925 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3cfc0d0ce0 con 0x7f3d041032b0 2024-04-16T05:35:26.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.325+0000 7f3d08acd640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cec0a3ee0 0x7f3cec0a63a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8000f80 tx=0x7f3cf8005c30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:26.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.328+0000 7f3d027fc640 1 -- 172.21.15.86:0/4283591925 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cfc09cdc0 con 0x7f3d041032b0 2024-04-16T05:35:26.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.530+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3d040fe480 con 0x7f3cec0a3ee0 2024-04-16T05:35:26.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.536+0000 7f3d027fc640 1 -- 172.21.15.86:0/4283591925 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3d040fe480 con 0x7f3cec0a3ee0 2024-04-16T05:35:26.535 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-04-16T05:35:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cec0a3ee0 msgr2=0x7f3cec0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cec0a3ee0 0x7f3cec0a63a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8000f80 tx=0x7f3cf8005c30 comp rx=0 tx=0).stop 2024-04-16T05:35:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d041032b0 msgr2=0x7f3d0419e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d041032b0 0x7f3d0419e5a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00d770 tx=0x7f3cfc00dc30 comp rx=0 tx=0).stop 2024-04-16T05:35:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 shutdown_connections 2024-04-16T05:35:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d0410c680 0x7f3d041986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cec0a3ee0 0x7f3cec0a63a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d04103bd0 0x7f3d0419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 --2- 172.21.15.86:0/4283591925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d041032b0 0x7f3d0419e5a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 >> 172.21.15.86:0/4283591925 conn(0x7f3d04077280 msgr2=0x7f3d040fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 shutdown_connections 2024-04-16T05:35:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:26.538+0000 7f3d0b532640 1 -- 172.21.15.86:0/4283591925 wait complete. 2024-04-16T05:35:26.548 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-16T05:35:26.578 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-16T05:35:26.588 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-16T05:35:26.616 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:35:26.616 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-16T05:35:26.621 INFO:teuthology.run_tasks:Running task workunit... 2024-04-16T05:35:26.652 INFO:tasks.workunit:Pulling workunits from ref a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:35:26.652 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-16T05:35:26.652 INFO:tasks.workunit:timeout=3h 2024-04-16T05:35:26.653 INFO:tasks.workunit:cleanup=True 2024-04-16T05:35:26.653 DEBUG:teuthology.orchestra.run.smithi086:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:26.668 INFO:teuthology.orchestra.run.smithi086.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:26.668 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-16T05:35:26.669 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-04-16T05:35:26.669 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:35:26.669 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-16T05:35:26.669 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-16 05:33:18.677461697 +0000 2024-04-16T05:35:26.669 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-16 05:33:18.677461697 +0000 2024-04-16T05:35:26.669 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-16 05:35:14.766608665 +0000 2024-04-16T05:35:26.669 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-16 05:33:18.677461697 +0000 2024-04-16T05:35:26.669 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:26.669 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-16T05:35:26.746 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:35:26.802 INFO:tasks.workunit.client.0.smithi086.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-16T05:35:26.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:26 smithi194 ceph-mon[35865]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-04-16T05:35:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:26 smithi194 ceph-mon[35865]: from='client.15741 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:26.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:26 smithi086 ceph-mon[32028]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-04-16T05:35:26.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:26 smithi086 ceph-mon[32028]: from='client.15741 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:26 smithi196 ceph-mon[36373]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-04-16T05:35:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:26 smithi196 ceph-mon[36373]: from='client.15741 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:27 smithi194 ceph-mon[35865]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:27 smithi194 ceph-mon[35865]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:27.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:27 smithi194 ceph-mon[35865]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:27.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:27 smithi086 ceph-mon[32028]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:27.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:27 smithi086 ceph-mon[32028]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:27.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:27 smithi086 ceph-mon[32028]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:27.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:27 smithi196 ceph-mon[36373]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:27.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:27 smithi196 ceph-mon[36373]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:27.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:27 smithi196 ceph-mon[36373]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:28.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:28 smithi194 ceph-mon[35865]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-04-16T05:35:28.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:28 smithi086 ceph-mon[32028]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-04-16T05:35:28.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:28 smithi196 ceph-mon[36373]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-04-16T05:35:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:30 smithi194 ceph-mon[35865]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-16T05:35:30.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:30 smithi086 ceph-mon[32028]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-16T05:35:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:30 smithi196 ceph-mon[36373]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-16T05:35:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:32 smithi194 ceph-mon[35865]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-16T05:35:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:32 smithi086 ceph-mon[32028]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-16T05:35:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:32 smithi196 ceph-mon[36373]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-16T05:35:34.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:34 smithi194 ceph-mon[35865]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-16T05:35:34.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:34 smithi086 ceph-mon[32028]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-16T05:35:34.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:34 smithi196 ceph-mon[36373]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-16T05:35:36.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:36 smithi194 ceph-mon[35865]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-16T05:35:36.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:36 smithi086 ceph-mon[32028]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-16T05:35:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:36 smithi196 ceph-mon[36373]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-16T05:35:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:38 smithi194 ceph-mon[35865]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-16T05:35:38.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:38 smithi086 ceph-mon[32028]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-16T05:35:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:38 smithi196 ceph-mon[36373]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-16T05:35:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.765+0000 7f088e58f640 1 -- 172.21.15.86:0/442933127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 msgr2=0x7f088810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.765+0000 7f088e58f640 1 --2- 172.21.15.86:0/442933127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 0x7f088810dd20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0880009ca0 tx=0x7f088002c140 comp rx=0 tx=0).stop 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.765+0000 7f088e58f640 1 -- 172.21.15.86:0/442933127 shutdown_connections 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.765+0000 7f088e58f640 1 --2- 172.21.15.86:0/442933127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 0x7f088810dd20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.765+0000 7f088e58f640 1 --2- 172.21.15.86:0/442933127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0888101250 0x7f08881016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.765+0000 7f088e58f640 1 --2- 172.21.15.86:0/442933127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888107060 0x7f0888107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.765+0000 7f088e58f640 1 -- 172.21.15.86:0/442933127 >> 172.21.15.86:0/442933127 conn(0x7f0888076d00 msgr2=0x7f0888077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.765+0000 7f088e58f640 1 -- 172.21.15.86:0/442933127 shutdown_connections 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.765+0000 7f088e58f640 1 -- 172.21.15.86:0/442933127 wait complete. 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f088e58f640 1 Processor -- start 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f088e58f640 1 -- start start 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f088e58f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0888101250 0x7f0888198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f088e58f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 0x7f0888198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f088e58f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888107060 0x7f088819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f088e58f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08881123e0 con 0x7f0888101c10 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f088e58f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0888112260 con 0x7f0888107060 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f088e58f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0888112560 con 0x7f0888101250 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f087ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 0x7f0888198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f087ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 0x7f0888198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49182/0 (socket says 172.21.15.86:49182) 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f087ffff640 1 -- 172.21.15.86:0/724218832 learned_addr learned my addr 172.21.15.86:0/724218832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f088cb2c640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888107060 0x7f088819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f0887fff640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0888101250 0x7f0888198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f087ffff640 1 -- 172.21.15.86:0/724218832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0888101250 msgr2=0x7f0888198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f087ffff640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0888101250 0x7f0888198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f087ffff640 1 -- 172.21.15.86:0/724218832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888107060 msgr2=0x7f088819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f087ffff640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888107060 0x7f088819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f087ffff640 1 -- 172.21.15.86:0/724218832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088819d750 con 0x7f0888101c10 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.766+0000 7f087ffff640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 0x7f0888198ca0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f087000d770 tx=0x7f087000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.767+0000 7f0885ffb640 1 -- 172.21.15.86:0/724218832 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870017070 con 0x7f0888101c10 2024-04-16T05:35:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.767+0000 7f0885ffb640 1 -- 172.21.15.86:0/724218832 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870005500 con 0x7f0888101c10 2024-04-16T05:35:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.767+0000 7f088e58f640 1 -- 172.21.15.86:0/724218832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088819da40 con 0x7f0888101c10 2024-04-16T05:35:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.767+0000 7f0885ffb640 1 -- 172.21.15.86:0/724218832 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870004e00 con 0x7f0888101c10 2024-04-16T05:35:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.767+0000 7f088e58f640 1 -- 172.21.15.86:0/724218832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881a52d0 con 0x7f0888101c10 2024-04-16T05:35:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f089117b740 1 Processor -- start 2024-04-16T05:35:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f0885ffb640 1 -- 172.21.15.86:0/724218832 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0870004fa0 con 0x7f0888101c10 2024-04-16T05:35:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f089117b740 1 -- start start 2024-04-16T05:35:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f089117b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61464c10 0x558c61464ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f089117b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61465530 0x558c6146dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f089117b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61470500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f089117b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c613c97d0 con 0x558c6146e110 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f089117b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558c61366f70 con 0x558c61465530 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f089117b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c613c5790 con 0x558c61464c10 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f087ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61470500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f087ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61470500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49194/0 (socket says 172.21.15.86:49194) 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f087ffff640 1 -- 172.21.15.86:0/4177470454 learned_addr learned my addr 172.21.15.86:0/4177470454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f087ffff640 1 -- 172.21.15.86:0/4177470454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61464c10 msgr2=0x558c61464ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f088cb2c640 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61464c10 0x558c61464ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f0887fff640 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61465530 0x558c6146dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.770+0000 7f087ffff640 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61464c10 0x558c61464ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f0885ffb640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08740a3f60 0x7f08740a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f087ffff640 1 -- 172.21.15.86:0/4177470454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61465530 msgr2=0x558c6146dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f087ffff640 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61465530 0x558c6146dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f087ffff640 1 -- 172.21.15.86:0/4177470454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c610dc210 con 0x558c6146e110 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f0887fff640 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61465530 0x558c6146dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f088cb2c640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08740a3f60 0x7f08740a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f087ffff640 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61470500 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f08700cd9f0 tx=0x7f08700d6f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ac5fba0cd240e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f0885ffb640 1 -- 172.21.15.86:0/724218832 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f08700d07f0 con 0x7f0888101c10 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f087e7fc640 1 -- 172.21.15.86:0/4177470454 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870020030 con 0x558c6146e110 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f087e7fc640 1 -- 172.21.15.86:0/4177470454 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08700046c0 con 0x558c6146e110 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.771+0000 7f087e7fc640 1 -- 172.21.15.86:0/4177470454 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08700056b0 con 0x558c6146e110 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 -- 172.21.15.86:0/4177470454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 msgr2=0x558c61470500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f088cb2c640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08740a3f60 0x7f08740a6420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0880004af0 tx=0x7f088002e040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61470500 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f08700cd9f0 tx=0x7f08700d6f60 comp rx=0 tx=0).stop 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 -- 172.21.15.86:0/4177470454 shutdown_connections 2024-04-16T05:35:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61470500 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61465530 0x558c6146dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 --2- 172.21.15.86:0/4177470454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61464c10 0x558c61464ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 -- 172.21.15.86:0/4177470454 >> 172.21.15.86:0/4177470454 conn(0x558c6145f340 msgr2=0x558c6145f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 -- 172.21.15.86:0/4177470454 shutdown_connections 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 -- 172.21.15.86:0/4177470454 wait complete. 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.772+0000 7f089117b740 1 -- start start 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f089117b740 1 Processor -- start 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f089117b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c6146b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f089117b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c6146b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f089117b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 0x558c6146bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f089117b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c61470f00 con 0x558c61464c10 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f089117b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558c61351e90 con 0x558c6146e110 2024-04-16T05:35:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f089117b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c61471080 con 0x558c61465530 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f089117b740 10 client.0 _ll_register_callbacks cb 0x558c61385cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c6146b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c6146b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49196/0 (socket says 172.21.15.86:49196) 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f0887fff640 1 -- 172.21.15.86:0/713996326 learned_addr learned my addr 172.21.15.86:0/713996326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f088cb2c640 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 0x558c6146bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f087ffff640 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c6146b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f0887fff640 1 -- 172.21.15.86:0/713996326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 msgr2=0x558c6146b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f0887fff640 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c6146b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f0887fff640 1 -- 172.21.15.86:0/713996326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 msgr2=0x558c6146bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f0887fff640 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 0x558c6146bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f0887fff640 1 -- 172.21.15.86:0/713996326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c611a8620 con 0x558c61464c10 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f087ffff640 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c6146b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f0887fff640 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c6146b180 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0878005d00 tx=0x7f087800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bac350b1eeea30d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.773+0000 7f08677fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f08677fe640 1 -- 172.21.15.86:0/713996326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878005220 con 0x558c61464c10 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f08677fe640 1 -- 172.21.15.86:0/713996326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f08780053c0 con 0x558c61464c10 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f089117b740 1 -- 172.21.15.86:0/713996326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c61369380 con 0x558c61464c10 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f08677fe640 1 -- 172.21.15.86:0/713996326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f08780055c0 con 0x558c61464c10 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f089117b740 10 client.15804 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f089117b740 1 -- 172.21.15.86:0/713996326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c6147d270 con 0x558c61464c10 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f08677fe640 1 -- 172.21.15.86:0/713996326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558c61369380 con 0x558c61464c10 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f08677fe640 1 -- 172.21.15.86:0/713996326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0878011860 con 0x558c61464c10 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f089117b740 10 client.15804 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f089117b740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f089117b740 20 client.15804 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f089117b740 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c6147e780 0x558c61480bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f089117b740 4 client.15804 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.774+0000 7f089117b740 1 -- 172.21.15.86:0/713996326 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c613d71f0 con 0x558c6147e780 2024-04-16T05:35:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.775+0000 7f087ffff640 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c6147e780 0x558c61480bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.775+0000 7f08677fe640 10 client.15804 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.775+0000 7f087ffff640 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c6147e780 0x558c61480bf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1481e6b16699da7f server_cookie=c74983f539c9e0ba in_seq=0 out_seq=0 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f08677fe640 1 -- 172.21.15.86:0/713996326 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c613d71f0 con 0x558c6147e780 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f08677fe640 10 client.15804 handle_command_reply: tid=0 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 client.15804 shutdown 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 -- 172.21.15.86:0/713996326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 msgr2=0x558c6146b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c6146b180 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0878005d00 tx=0x7f087800f730 comp rx=0 tx=0).stop 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 -- 172.21.15.86:0/713996326 shutdown_connections 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c6147e780 0x558c61480bf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 0x558c6146bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c6146b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f08677fe640 0 client.15804 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 --2- 172.21.15.86:0/713996326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c6146b180 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 -- 172.21.15.86:0/713996326 >> 172.21.15.86:0/713996326 conn(0x558c6145f340 msgr2=0x558c61461090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 -- 172.21.15.86:0/713996326 shutdown_connections 2024-04-16T05:35:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 1 -- 172.21.15.86:0/713996326 wait complete. 2024-04-16T05:35:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.776+0000 7f089117b740 20 client.15804 trim_cache size 0 max 16384 2024-04-16T05:35:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f089117b740 1 Processor -- start 2024-04-16T05:35:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f089117b740 1 -- start start 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f089117b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61464c10 0x558c6146d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f089117b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c61479e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f089117b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61486500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f089117b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08881af310 con 0x558c6146e110 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f089117b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b0e50 con 0x558c61464c10 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f089117b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08881af9e0 con 0x558c61465530 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f088cb2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61486500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f088cb2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61486500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49208/0 (socket says 172.21.15.86:49208) 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f088cb2c640 1 -- 172.21.15.86:0/1377914504 learned_addr learned my addr 172.21.15.86:0/1377914504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.791+0000 7f0887fff640 1 --2- 172.21.15.86:0/1377914504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61464c10 0x558c6146d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f088cb2c640 1 -- 172.21.15.86:0/1377914504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 msgr2=0x558c61479e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f088cb2c640 1 --2- 172.21.15.86:0/1377914504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c61479e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f088cb2c640 1 -- 172.21.15.86:0/1377914504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61464c10 msgr2=0x558c6146d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f088cb2c640 1 --2- 172.21.15.86:0/1377914504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61464c10 0x558c6146d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f088cb2c640 1 -- 172.21.15.86:0/1377914504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c61459870 con 0x558c6146e110 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f0887fff640 1 --2- 172.21.15.86:0/1377914504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61464c10 0x558c6146d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f088cb2c640 1 --2- 172.21.15.86:0/1377914504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61486500 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0880009ca0 tx=0x7f088003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47df277477b3557f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f087dffb640 1 -- 172.21.15.86:0/1377914504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880039040 con 0x558c6146e110 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f087dffb640 1 -- 172.21.15.86:0/1377914504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0880051070 con 0x558c6146e110 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f087dffb640 1 -- 172.21.15.86:0/1377914504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088004c6a0 con 0x558c6146e110 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f089117b740 1 -- 172.21.15.86:0/1377914504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 msgr2=0x558c61486500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f089117b740 1 --2- 172.21.15.86:0/1377914504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61486500 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0880009ca0 tx=0x7f088003f000 comp rx=0 tx=0).stop 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f089117b740 1 -- 172.21.15.86:0/1377914504 shutdown_connections 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f089117b740 1 --2- 172.21.15.86:0/1377914504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c6146e110 0x558c61486500 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f089117b740 1 --2- 172.21.15.86:0/1377914504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c61479e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.792+0000 7f089117b740 1 --2- 172.21.15.86:0/1377914504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c61464c10 0x558c6146d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 -- 172.21.15.86:0/1377914504 >> 172.21.15.86:0/1377914504 conn(0x558c6145f340 msgr2=0x558c61461020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 -- 172.21.15.86:0/1377914504 shutdown_connections 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 -- 172.21.15.86:0/1377914504 wait complete. 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 -- start start 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 Processor -- start 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c61471f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c61472480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 0x558c614729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b0780 con 0x558c61464c10 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b00b0 con 0x558c6146e110 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08881ad2d0 con 0x558c61465530 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f089117b740 10 client.0 _ll_register_callbacks cb 0x7f087800d2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c61471f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c61471f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49224/0 (socket says 172.21.15.86:49224) 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.793+0000 7f0887fff640 1 -- 172.21.15.86:0/2520404698 learned_addr learned my addr 172.21.15.86:0/2520404698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f087ffff640 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c61472480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f088cb2c640 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 0x558c614729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f0887fff640 1 -- 172.21.15.86:0/2520404698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 msgr2=0x558c61472480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f0887fff640 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c61472480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f0887fff640 1 -- 172.21.15.86:0/2520404698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 msgr2=0x558c614729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f0887fff640 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 0x558c614729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f0887fff640 1 -- 172.21.15.86:0/2520404698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c614e3f70 con 0x558c61464c10 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f088cb2c640 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 0x558c614729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f0887fff640 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c61471f40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f0878011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88283b75f210e951 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f08677fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f08677fe640 1 -- 172.21.15.86:0/2520404698 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878019430 con 0x558c61464c10 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f08677fe640 1 -- 172.21.15.86:0/2520404698 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0878019750 con 0x558c61464c10 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f089117b740 1 -- 172.21.15.86:0/2520404698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0880051070 con 0x558c61464c10 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f08677fe640 1 -- 172.21.15.86:0/2520404698 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0878019950 con 0x558c61464c10 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f089117b740 10 client.15810 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f089117b740 1 -- 172.21.15.86:0/2520404698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c614e52c0 con 0x558c61464c10 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.794+0000 7f08677fe640 1 -- 172.21.15.86:0/2520404698 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0880051070 con 0x558c61464c10 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f08677fe640 1 -- 172.21.15.86:0/2520404698 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f08780318c0 con 0x558c61464c10 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f089117b740 10 client.15810 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f089117b740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f089117b740 20 client.15810 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f089117b740 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c614e5900 0x558c614e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f089117b740 4 client.15810 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f089117b740 1 -- 172.21.15.86:0/2520404698 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c613d1060 con 0x558c614e5900 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f087ffff640 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c614e5900 0x558c614e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f08677fe640 10 client.15810 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.795+0000 7f087ffff640 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c614e5900 0x558c614e7d20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0df5e0d0db52d16 server_cookie=4047f21be5b156fb in_seq=0 out_seq=0 2024-04-16T05:35:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f08677fe640 1 -- 172.21.15.86:0/2520404698 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c613d1060 con 0x558c614e5900 2024-04-16T05:35:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f08677fe640 10 client.15810 handle_command_reply: tid=0 2024-04-16T05:35:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 client.15810 shutdown 2024-04-16T05:35:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 -- 172.21.15.86:0/2520404698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 msgr2=0x558c61471f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c61471f40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f0878011b10 comp rx=0 tx=0).stop 2024-04-16T05:35:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 -- 172.21.15.86:0/2520404698 shutdown_connections 2024-04-16T05:35:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c614e5900 0x558c614e7d20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c6146e110 0x558c614729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c61465530 0x558c61472480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f08677fe640 0 client.15810 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 --2- 172.21.15.86:0/2520404698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c61464c10 0x558c61471f40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 -- 172.21.15.86:0/2520404698 >> 172.21.15.86:0/2520404698 conn(0x558c6145f340 msgr2=0x558c61460500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 -- 172.21.15.86:0/2520404698 shutdown_connections 2024-04-16T05:35:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 1 -- 172.21.15.86:0/2520404698 wait complete. 2024-04-16T05:35:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.808+0000 7f089117b740 20 client.15810 trim_cache size 0 max 16384 2024-04-16T05:35:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-16T05:35:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-16T05:35:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-16T05:35:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-16T05:35:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 -- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08740a3f60 msgr2=0x7f08740a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08740a3f60 0x7f08740a6420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0880004af0 tx=0x7f088002e040 comp rx=0 tx=0).stop 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 -- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 msgr2=0x7f0888198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 0x7f0888198ca0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f087000d770 tx=0x7f087000dc30 comp rx=0 tx=0).stop 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 -- 172.21.15.86:0/724218832 shutdown_connections 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08740a3f60 0x7f08740a6420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888107060 0x7f088819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888101c10 0x7f0888198ca0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 --2- 172.21.15.86:0/724218832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0888101250 0x7f0888198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 -- 172.21.15.86:0/724218832 >> 172.21.15.86:0/724218832 conn(0x7f0888076d00 msgr2=0x7f08880ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 -- 172.21.15.86:0/724218832 shutdown_connections 2024-04-16T05:35:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:39.809+0000 7f088e58f640 1 -- 172.21.15.86:0/724218832 wait complete. 2024-04-16T05:35:39.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:39 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:35:39.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:39 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:35:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:39 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:35:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:40 smithi194 ceph-mon[35865]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-16T05:35:40.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:40 smithi086 ceph-mon[32028]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-16T05:35:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:40 smithi196 ceph-mon[36373]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-16T05:35:42.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:42 smithi194 ceph-mon[35865]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:35:42.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:42 smithi086 ceph-mon[32028]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:35:42.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:42 smithi196 ceph-mon[36373]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:35:44.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:44 smithi194 ceph-mon[35865]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:35:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:44 smithi086 ceph-mon[32028]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:35:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:44 smithi196 ceph-mon[36373]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:35:46.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:46 smithi194 ceph-mon[35865]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-04-16T05:35:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:46 smithi086 ceph-mon[32028]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-04-16T05:35:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:46 smithi196 ceph-mon[36373]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-04-16T05:35:48.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:48 smithi194 ceph-mon[35865]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-16T05:35:48.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:48 smithi086 ceph-mon[32028]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-16T05:35:48.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:48 smithi196 ceph-mon[36373]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-16T05:35:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:50 smithi194 ceph-mon[35865]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-04-16T05:35:50.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:50 smithi086 ceph-mon[32028]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-04-16T05:35:50.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:50 smithi196 ceph-mon[36373]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-04-16T05:35:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:52 smithi194 ceph-mon[35865]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-16T05:35:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.907+0000 7f2450b5f640 1 -- 172.21.15.86:0/1814015152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100670 msgr2=0x7f244c100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.907+0000 7f2450b5f640 1 --2- 172.21.15.86:0/1814015152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100670 0x7f244c100af0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2440009930 tx=0x7f244002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.907+0000 7f2450b5f640 1 -- 172.21.15.86:0/1814015152 shutdown_connections 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.907+0000 7f2450b5f640 1 --2- 172.21.15.86:0/1814015152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c101030 0x7f244c10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.907+0000 7f2450b5f640 1 --2- 172.21.15.86:0/1814015152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100670 0x7f244c100af0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.907+0000 7f2450b5f640 1 --2- 172.21.15.86:0/1814015152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f244c106680 0x7f244c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.907+0000 7f2450b5f640 1 -- 172.21.15.86:0/1814015152 >> 172.21.15.86:0/1814015152 conn(0x7f244c077420 msgr2=0x7f244c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.907+0000 7f2450b5f640 1 -- 172.21.15.86:0/1814015152 shutdown_connections 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.907+0000 7f2450b5f640 1 -- 172.21.15.86:0/1814015152 wait complete. 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.908+0000 7f2450b5f640 1 Processor -- start 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.908+0000 7f2450b5f640 1 -- start start 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.908+0000 7f2450b5f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f244c100670 0x7f244c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.908+0000 7f2450b5f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c101030 0x7f244c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.908+0000 7f2450b5f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106680 0x7f244c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.908+0000 7f2450b5f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f244c110a90 con 0x7f244c106680 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.908+0000 7f2450b5f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f244c110910 con 0x7f244c100670 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.908+0000 7f2450b5f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f244c110c10 con 0x7f244c101030 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106680 0x7f244c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106680 0x7f244c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54916/0 (socket says 172.21.15.86:54916) 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244ad9d640 1 -- 172.21.15.86:0/3994320114 learned_addr learned my addr 172.21.15.86:0/3994320114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f2449d9b640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c101030 0x7f244c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244a59c640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f244c100670 0x7f244c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244ad9d640 1 -- 172.21.15.86:0/3994320114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c101030 msgr2=0x7f244c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244ad9d640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c101030 0x7f244c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244ad9d640 1 -- 172.21.15.86:0/3994320114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f244c100670 msgr2=0x7f244c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244ad9d640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f244c100670 0x7f244c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244ad9d640 1 -- 172.21.15.86:0/3994320114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c198fa0 con 0x7f244c106680 2024-04-16T05:35:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f244ad9d640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106680 0x7f244c1986e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f244400cad0 tx=0x7f244400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f243b7fe640 1 -- 172.21.15.86:0/3994320114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244400ae20 con 0x7f244c106680 2024-04-16T05:35:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f2450b5f640 1 -- 172.21.15.86:0/3994320114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244c199290 con 0x7f244c106680 2024-04-16T05:35:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f243b7fe640 1 -- 172.21.15.86:0/3994320114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2444005020 con 0x7f244c106680 2024-04-16T05:35:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f243b7fe640 1 -- 172.21.15.86:0/3994320114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244400f6f0 con 0x7f244c106680 2024-04-16T05:35:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.909+0000 7f2450b5f640 1 -- 172.21.15.86:0/3994320114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244c1a55a0 con 0x7f244c106680 2024-04-16T05:35:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.911+0000 7f243b7fe640 1 -- 172.21.15.86:0/3994320114 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f24440051d0 con 0x7f244c106680 2024-04-16T05:35:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.912+0000 7f2453820740 1 Processor -- start 2024-04-16T05:35:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.912+0000 7f2453820740 1 -- start start 2024-04-16T05:35:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.912+0000 7f243b7fe640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f24340a3ee0 0x7f24340a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2453820740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df41ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f24340a3ee0 0x7f24340a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2453820740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df42220 0x56446df4efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2453820740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df4f520 0x56446df51910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f243b7fe640 1 -- 172.21.15.86:0/3994320114 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f24440d0fa0 con 0x7f244c106680 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2453820740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56446dea67d0 con 0x56446df42220 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2453820740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56446de43f70 con 0x56446df4f520 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2453820740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56446dea2790 con 0x56446df41900 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df42220 0x56446df4efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df42220 0x56446df4efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54932/0 (socket says 172.21.15.86:54932) 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 -- 172.21.15.86:0/3682596134 learned_addr learned my addr 172.21.15.86:0/3682596134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f244a59c640 1 --2- 172.21.15.86:0/3682596134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df41ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f244ad9d640 1 --2- 172.21.15.86:0/3682596134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df4f520 0x56446df51910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 -- 172.21.15.86:0/3682596134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 msgr2=0x56446df41ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 --2- 172.21.15.86:0/3682596134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df41ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 -- 172.21.15.86:0/3682596134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df4f520 msgr2=0x56446df51910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 --2- 172.21.15.86:0/3682596134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df4f520 0x56446df51910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 -- 172.21.15.86:0/3682596134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56446dbb9210 con 0x56446df42220 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f24340a3ee0 0x7f24340a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f244002cfd0 tx=0x7f244002c6e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.913+0000 7f2449d9b640 1 --2- 172.21.15.86:0/3682596134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df42220 0x56446df4efe0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2440009930 tx=0x7f2440047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b77d50e22ac1aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2438ff9640 1 -- 172.21.15.86:0/3682596134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440004980 con 0x56446df42220 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2438ff9640 1 -- 172.21.15.86:0/3682596134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2440004b20 con 0x56446df42220 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2438ff9640 1 -- 172.21.15.86:0/3682596134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440003bf0 con 0x56446df42220 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2453820740 1 -- 172.21.15.86:0/3682596134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df42220 msgr2=0x56446df4efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2453820740 1 --2- 172.21.15.86:0/3682596134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df42220 0x56446df4efe0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2440009930 tx=0x7f2440047000 comp rx=0 tx=0).stop 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2453820740 1 -- 172.21.15.86:0/3682596134 shutdown_connections 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2453820740 1 --2- 172.21.15.86:0/3682596134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df4f520 0x56446df51910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2453820740 1 --2- 172.21.15.86:0/3682596134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df42220 0x56446df4efe0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2453820740 1 --2- 172.21.15.86:0/3682596134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df41ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.914+0000 7f2453820740 1 -- 172.21.15.86:0/3682596134 >> 172.21.15.86:0/3682596134 conn(0x56446df3c340 msgr2=0x56446df3c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 -- 172.21.15.86:0/3682596134 shutdown_connections 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 -- 172.21.15.86:0/3682596134 wait complete. 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 -- start start 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 Processor -- start 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df41900 0x56446df4c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df42220 0x56446df4ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df4cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56446df522b0 con 0x56446df4f520 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56446de2ee90 con 0x56446df41900 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56446df52430 con 0x56446df42220 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2453820740 10 client.0 _ll_register_callbacks cb 0x56446de62cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f244ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df4cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f244ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df4cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54940/0 (socket says 172.21.15.86:54940) 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f244ad9d640 1 -- 172.21.15.86:0/2792611744 learned_addr learned my addr 172.21.15.86:0/2792611744 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f244a59c640 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df41900 0x56446df4c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.915+0000 7f2449d9b640 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df42220 0x56446df4ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f244ad9d640 1 -- 172.21.15.86:0/2792611744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df42220 msgr2=0x56446df4ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f244ad9d640 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df42220 0x56446df4ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f244ad9d640 1 -- 172.21.15.86:0/2792611744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df41900 msgr2=0x56446df4c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f244ad9d640 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df41900 0x56446df4c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f244ad9d640 1 -- 172.21.15.86:0/2792611744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446dc85620 con 0x56446df4f520 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f244ad9d640 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df4cfc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f244400caa0 tx=0x7f244409ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf0d5fe05219508 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2431ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2431ffb640 1 -- 172.21.15.86:0/2792611744 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244401d040 con 0x56446df4f520 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2431ffb640 1 -- 172.21.15.86:0/2792611744 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f24440ce6e0 con 0x56446df4f520 2024-04-16T05:35:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2453820740 1 -- 172.21.15.86:0/2792611744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2440004b20 con 0x56446df4f520 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2431ffb640 1 -- 172.21.15.86:0/2792611744 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24440a55f0 con 0x56446df4f520 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2453820740 10 client.15822 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2453820740 1 -- 172.21.15.86:0/2792611744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56446df5e620 con 0x56446df4f520 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2431ffb640 1 -- 172.21.15.86:0/2792611744 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2440004b20 con 0x56446df4f520 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2431ffb640 1 -- 172.21.15.86:0/2792611744 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2444067260 con 0x56446df4f520 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2453820740 10 client.15822 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.916+0000 7f2453820740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.917+0000 7f2453820740 20 client.15822 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.917+0000 7f2453820740 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56446df5fae0 0x56446df61ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.917+0000 7f2453820740 4 client.15822 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.917+0000 7f2453820740 1 -- 172.21.15.86:0/2792611744 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56446de46380 con 0x56446df5fae0 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.917+0000 7f244a59c640 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56446df5fae0 0x56446df61ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.917+0000 7f2431ffb640 10 client.15822 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.917+0000 7f244a59c640 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56446df5fae0 0x56446df61ed0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7033bc586857766b server_cookie=6750c03eafb93fc9 in_seq=0 out_seq=0 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2431ffb640 1 -- 172.21.15.86:0/2792611744 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56446de46380 con 0x56446df5fae0 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2431ffb640 10 client.15822 handle_command_reply: tid=0 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 client.15822 shutdown 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 -- 172.21.15.86:0/2792611744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 msgr2=0x56446df4cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df4cfc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f244400caa0 tx=0x7f244409ca40 comp rx=0 tx=0).stop 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 -- 172.21.15.86:0/2792611744 shutdown_connections 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56446df5fae0 0x56446df61ed0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df4cfc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df42220 0x56446df4ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 --2- 172.21.15.86:0/2792611744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df41900 0x56446df4c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2431ffb640 0 client.15822 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 -- 172.21.15.86:0/2792611744 >> 172.21.15.86:0/2792611744 conn(0x56446df3c340 msgr2=0x56446df3e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 -- 172.21.15.86:0/2792611744 shutdown_connections 2024-04-16T05:35:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 1 -- 172.21.15.86:0/2792611744 wait complete. 2024-04-16T05:35:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.918+0000 7f2453820740 20 client.15822 trim_cache size 0 max 16384 2024-04-16T05:35:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f2453820740 1 Processor -- start 2024-04-16T05:35:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f2453820740 1 -- start start 2024-04-16T05:35:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f2453820740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df5b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f2453820740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 0x56446df4cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f2453820740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f2453820740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ad260 con 0x56446df4f520 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f2453820740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1aeda0 con 0x56446df42220 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f2453820740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ad930 con 0x56446df41900 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f244ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f244ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df677a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54944/0 (socket says 172.21.15.86:54944) 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.933+0000 7f244ad9d640 1 -- 172.21.15.86:0/695650877 learned_addr learned my addr 172.21.15.86:0/695650877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f244ad9d640 1 -- 172.21.15.86:0/695650877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 msgr2=0x56446df5b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f244ad9d640 1 --2- 172.21.15.86:0/695650877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df5b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2449d9b640 1 --2- 172.21.15.86:0/695650877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 0x56446df4cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f244ad9d640 1 -- 172.21.15.86:0/695650877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 msgr2=0x56446df4cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f244ad9d640 1 --2- 172.21.15.86:0/695650877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 0x56446df4cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f244ad9d640 1 -- 172.21.15.86:0/695650877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56446df62240 con 0x56446df4f520 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f244ad9d640 1 --2- 172.21.15.86:0/695650877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df677a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f24440a0ef0 tx=0x7f2444005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c86e4f5c629769e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2433fff640 1 -- 172.21.15.86:0/695650877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244401d040 con 0x56446df4f520 2024-04-16T05:35:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2433fff640 1 -- 172.21.15.86:0/695650877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24440672c0 con 0x56446df4f520 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2433fff640 1 -- 172.21.15.86:0/695650877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24440bbda0 con 0x56446df4f520 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2453820740 1 -- 172.21.15.86:0/695650877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 msgr2=0x56446df677a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2453820740 1 --2- 172.21.15.86:0/695650877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df677a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f24440a0ef0 tx=0x7f2444005770 comp rx=0 tx=0).stop 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2453820740 1 -- 172.21.15.86:0/695650877 shutdown_connections 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2453820740 1 --2- 172.21.15.86:0/695650877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446df677a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2453820740 1 --2- 172.21.15.86:0/695650877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 0x56446df4cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2453820740 1 --2- 172.21.15.86:0/695650877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df5b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.934+0000 7f2453820740 1 -- 172.21.15.86:0/695650877 >> 172.21.15.86:0/695650877 conn(0x56446df3c340 msgr2=0x56446df4c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 -- 172.21.15.86:0/695650877 shutdown_connections 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 -- 172.21.15.86:0/695650877 wait complete. 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 -- start start 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 Processor -- start 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 0x56446df49bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446dfc5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ae6d0 con 0x56446df4f520 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ae000 con 0x56446df42220 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ab220 con 0x56446df41900 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2453820740 10 client.0 _ll_register_callbacks cb 0x7f244409c520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f244ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446dfc5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f244ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446dfc5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54948/0 (socket says 172.21.15.86:54948) 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f244ad9d640 1 -- 172.21.15.86:0/3306334092 learned_addr learned my addr 172.21.15.86:0/3306334092 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f244a59c640 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f2449d9b640 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 0x56446df49bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f244ad9d640 1 -- 172.21.15.86:0/3306334092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 msgr2=0x56446df49670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f244ad9d640 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df49670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f244ad9d640 1 -- 172.21.15.86:0/3306334092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 msgr2=0x56446df49bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f244ad9d640 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 0x56446df49bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.935+0000 7f244ad9d640 1 -- 172.21.15.86:0/3306334092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446dfc5550 con 0x56446df4f520 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f244ad9d640 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446dfc5010 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2444098de0 tx=0x7f2444005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a70884bfa480fce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2431ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2431ffb640 1 -- 172.21.15.86:0/3306334092 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244407f070 con 0x56446df4f520 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2431ffb640 1 -- 172.21.15.86:0/3306334092 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f244409c080 con 0x56446df4f520 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2453820740 1 -- 172.21.15.86:0/3306334092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24440672c0 con 0x56446df4f520 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2431ffb640 1 -- 172.21.15.86:0/3306334092 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f244401d070 con 0x56446df4f520 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2453820740 10 client.15828 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2453820740 1 -- 172.21.15.86:0/3306334092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56446dfc5550 con 0x56446df4f520 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2431ffb640 1 -- 172.21.15.86:0/3306334092 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24440672c0 con 0x56446df4f520 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2431ffb640 1 -- 172.21.15.86:0/3306334092 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f24440cead0 con 0x56446df4f520 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2453820740 10 client.15828 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2453820740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2453820740 20 client.15828 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2453820740 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56446dfc6a00 0x56446dfc8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2453820740 4 client.15828 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:35:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.936+0000 7f2453820740 1 -- 172.21.15.86:0/3306334092 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56446deb41f0 con 0x56446dfc6a00 2024-04-16T05:35:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.937+0000 7f244a59c640 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56446dfc6a00 0x56446dfc8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.937+0000 7f2431ffb640 10 client.15828 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.937+0000 7f244a59c640 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56446dfc6a00 0x56446dfc8df0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383c9be1454a30ab server_cookie=b126743b629ace9a in_seq=0 out_seq=0 2024-04-16T05:35:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.940+0000 7f2431ffb640 1 -- 172.21.15.86:0/3306334092 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56446deb41f0 con 0x56446dfc6a00 2024-04-16T05:35:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.940+0000 7f2431ffb640 10 client.15828 handle_command_reply: tid=0 2024-04-16T05:35:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.940+0000 7f2453820740 1 client.15828 shutdown 2024-04-16T05:35:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 -- 172.21.15.86:0/3306334092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 msgr2=0x56446dfc5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446dfc5010 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2444098de0 tx=0x7f2444005b40 comp rx=0 tx=0).stop 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 -- 172.21.15.86:0/3306334092 shutdown_connections 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56446dfc6a00 0x56446dfc8df0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446df4f520 0x56446dfc5010 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2431ffb640 0 client.15828 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56446df42220 0x56446df49bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 --2- 172.21.15.86:0/3306334092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446df41900 0x56446df49670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 -- 172.21.15.86:0/3306334092 >> 172.21.15.86:0/3306334092 conn(0x56446df3c340 msgr2=0x56446df4cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 -- 172.21.15.86:0/3306334092 shutdown_connections 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 1 -- 172.21.15.86:0/3306334092 wait complete. 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2453820740 20 client.15828 trim_cache size 0 max 16384 2024-04-16T05:35:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-16T05:35:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-16T05:35:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-16T05:35:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-16T05:35:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 -- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f24340a3ee0 msgr2=0x7f24340a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f24340a3ee0 0x7f24340a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f244002cfd0 tx=0x7f244002c6e0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 -- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106680 msgr2=0x7f244c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106680 0x7f244c1986e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f244400cad0 tx=0x7f244400cf90 comp rx=0 tx=0).stop 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 -- 172.21.15.86:0/3994320114 shutdown_connections 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106680 0x7f244c1986e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c101030 0x7f244c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f24340a3ee0 0x7f24340a63a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 --2- 172.21.15.86:0/3994320114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f244c100670 0x7f244c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 -- 172.21.15.86:0/3994320114 >> 172.21.15.86:0/3994320114 conn(0x7f244c077420 msgr2=0x7f244c0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 -- 172.21.15.86:0/3994320114 shutdown_connections 2024-04-16T05:35:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:35:52.941+0000 7f2450b5f640 1 -- 172.21.15.86:0/3994320114 wait complete. 2024-04-16T05:35:52.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:52 smithi086 ceph-mon[32028]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-16T05:35:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:52 smithi196 ceph-mon[36373]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-16T05:35:54.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:54 smithi086 ceph-mon[32028]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-16T05:35:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:54 smithi196 ceph-mon[36373]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-16T05:35:55.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:54 smithi194 ceph-mon[35865]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-16T05:35:55.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:35:55.344 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:35:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.465+0000 7f8d0b304640 1 -- 172.21.15.86:0/2370111781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 msgr2=0x7f8d0406ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.465+0000 7f8d0b304640 1 --2- 172.21.15.86:0/2370111781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 0x7f8d0406ac50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009900 tx=0x7f8d0002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.465+0000 7f8d0b304640 1 -- 172.21.15.86:0/2370111781 shutdown_connections 2024-04-16T05:35:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.465+0000 7f8d0b304640 1 --2- 172.21.15.86:0/2370111781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d04107330 0x7f8d0410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.465+0000 7f8d0b304640 1 --2- 172.21.15.86:0/2370111781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d0406b220 0x7f8d04106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.465+0000 7f8d0b304640 1 --2- 172.21.15.86:0/2370111781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 0x7f8d0406ac50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.465+0000 7f8d0b304640 1 -- 172.21.15.86:0/2370111781 >> 172.21.15.86:0/2370111781 conn(0x7f8d04076d00 msgr2=0x7f8d04077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.465+0000 7f8d0b304640 1 -- 172.21.15.86:0/2370111781 shutdown_connections 2024-04-16T05:35:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.465+0000 7f8d0b304640 1 -- 172.21.15.86:0/2370111781 wait complete. 2024-04-16T05:35:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d0b304640 1 Processor -- start 2024-04-16T05:35:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d0b304640 1 -- start start 2024-04-16T05:35:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d0b304640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 0x7f8d0419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d0b304640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d04107330 0x7f8d0419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d0b304640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d04198700 0x7f8d04198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d0b304640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041123c0 con 0x7f8d0406a870 2024-04-16T05:35:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d0b304640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04112240 con 0x7f8d04107330 2024-04-16T05:35:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d0b304640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04112540 con 0x7f8d04198700 2024-04-16T05:35:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d090a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 0x7f8d0419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d090a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 0x7f8d0419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54962/0 (socket says 172.21.15.86:54962) 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d090a0640 1 -- 172.21.15.86:0/3407346160 learned_addr learned my addr 172.21.15.86:0/3407346160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d098a1640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d04198700 0x7f8d04198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d090a0640 1 -- 172.21.15.86:0/3407346160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d04198700 msgr2=0x7f8d04198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.466+0000 7f8d090a0640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d04198700 0x7f8d04198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8d090a0640 1 -- 172.21.15.86:0/3407346160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d04107330 msgr2=0x7f8d0419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8d0889f640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d04107330 0x7f8d0419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8d090a0640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d04107330 0x7f8d0419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8d090a0640 1 -- 172.21.15.86:0/3407346160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d04199300 con 0x7f8d0406a870 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8d098a1640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d04198700 0x7f8d04198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8d0889f640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d04107330 0x7f8d0419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8d090a0640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 0x7f8d0419e5f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d0002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8cf67fc640 1 -- 172.21.15.86:0/3407346160 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00041070 con 0x7f8d0406a870 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8cf67fc640 1 -- 172.21.15.86:0/3407346160 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d00004450 con 0x7f8d0406a870 2024-04-16T05:35:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8d0b304640 1 -- 172.21.15.86:0/3407346160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d04199590 con 0x7f8d0406a870 2024-04-16T05:35:55.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8cf67fc640 1 -- 172.21.15.86:0/3407346160 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00034440 con 0x7f8d0406a870 2024-04-16T05:35:55.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.467+0000 7f8d0b304640 1 -- 172.21.15.86:0/3407346160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d041a5680 con 0x7f8d0406a870 2024-04-16T05:35:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.469+0000 7f8cf67fc640 1 -- 172.21.15.86:0/3407346160 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8d00044070 con 0x7f8d0406a870 2024-04-16T05:35:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d0dfc9740 1 Processor -- start 2024-04-16T05:35:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d0dfc9740 1 -- start start 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f632735230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f632740740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f63269a180 con 0x55f632735770 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f632637360 con 0x55f632734e50 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f632696140 con 0x55f63273e350 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d090a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d090a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54968/0 (socket says 172.21.15.86:54968) 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8cf67fc640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8cf00a3e40 0x7f8cf00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d090a0640 1 -- 172.21.15.86:0/2259267237 learned_addr learned my addr 172.21.15.86:0/2259267237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d098a1640 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f632735230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d098a1640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8cf00a3e40 0x7f8cf00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.470+0000 7f8d0889f640 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f632740740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8cf67fc640 1 -- 172.21.15.86:0/3407346160 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8d000f29d0 con 0x7f8d0406a870 2024-04-16T05:35:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8d090a0640 1 -- 172.21.15.86:0/2259267237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 msgr2=0x55f632740740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8d090a0640 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f632740740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8d090a0640 1 -- 172.21.15.86:0/2259267237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 msgr2=0x55f632735230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8d090a0640 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f632735230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8d090a0640 1 -- 172.21.15.86:0/2259267237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6324767d0 con 0x55f632735770 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8d0889f640 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f632740740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8d090a0640 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273de10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8d000bc130 tx=0x7f8d0002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281e2e9a8a93349c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8d098a1640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8cf00a3e40 0x7f8cf00a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005c40 tx=0x7f8cfc009280 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8cef7fe640 1 -- 172.21.15.86:0/2259267237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0004b070 con 0x55f632735770 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8cef7fe640 1 -- 172.21.15.86:0/2259267237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d000f9040 con 0x55f632735770 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.471+0000 7f8cef7fe640 1 -- 172.21.15.86:0/2259267237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d000c7130 con 0x55f632735770 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/2259267237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 msgr2=0x55f63273de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273de10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8d000bc130 tx=0x7f8d0002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/2259267237 shutdown_connections 2024-04-16T05:35:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f632740740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273de10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/2259267237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f632735230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/2259267237 >> 172.21.15.86:0/2259267237 conn(0x55f63272f580 msgr2=0x55f63272f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/2259267237 shutdown_connections 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/2259267237 wait complete. 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 -- start start 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.472+0000 7f8d0dfc9740 1 Processor -- start 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f63273b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f63273bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f63269a180 con 0x55f632735770 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f6326532c0 con 0x55f632734e50 2024-04-16T05:35:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f632740f00 con 0x55f63273e350 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 10 client.0 _ll_register_callbacks cb 0x55f63263a090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0889f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0889f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54980/0 (socket says 172.21.15.86:54980) 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0889f640 1 -- 172.21.15.86:0/3938512656 learned_addr learned my addr 172.21.15.86:0/3938512656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d090a0640 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f63273b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d098a1640 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f63273bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0889f640 1 -- 172.21.15.86:0/3938512656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 msgr2=0x55f63273bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0889f640 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f63273bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0889f640 1 -- 172.21.15.86:0/3938512656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 msgr2=0x55f63273b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0889f640 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f63273b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0889f640 1 -- 172.21.15.86:0/3938512656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6323ac3f0 con 0x55f632735770 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d098a1640 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f63273bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d090a0640 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f63273b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0889f640 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273b9b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8d0406a410 tx=0x7f8cf8010b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b29c7cdfb8b149 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8cecff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8cecff9640 1 -- 172.21.15.86:0/3938512656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8005220 con 0x55f632735770 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8cecff9640 1 -- 172.21.15.86:0/3938512656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8cf80053c0 con 0x55f632735770 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/3938512656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d000f9040 con 0x55f632735770 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8cecff9640 1 -- 172.21.15.86:0/3938512656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cf8024050 con 0x55f632735770 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 10 client.15840 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.473+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/3938512656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f63236c0f0 con 0x55f632735770 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.474+0000 7f8cecff9640 1 -- 172.21.15.86:0/3938512656 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d000f9040 con 0x55f632735770 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.474+0000 7f8cecff9640 1 -- 172.21.15.86:0/3938512656 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8cf800f040 con 0x55f632735770 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.474+0000 7f8d0dfc9740 10 client.15840 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.474+0000 7f8d0dfc9740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.474+0000 7f8d0dfc9740 20 client.15840 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.474+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f63274e550 0x55f632750940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.474+0000 7f8d0dfc9740 4 client.15840 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.474+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/3938512656 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6326a7ba0 con 0x55f63274e550 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.474+0000 7f8d090a0640 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f63274e550 0x55f632750940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.475+0000 7f8cecff9640 10 client.15840 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.475+0000 7f8d090a0640 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f63274e550 0x55f632750940 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=485eb014df7af5e7 server_cookie=2f292931b1faf07b in_seq=0 out_seq=0 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.475+0000 7f8cecff9640 1 -- 172.21.15.86:0/3938512656 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6326a7ba0 con 0x55f63274e550 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.475+0000 7f8cecff9640 10 client.15840 handle_command_reply: tid=0 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.475+0000 7f8d0dfc9740 1 client.15840 shutdown 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/3938512656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 msgr2=0x55f63273b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273b9b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8d0406a410 tx=0x7f8cf8010b00 comp rx=0 tx=0).stop 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/3938512656 shutdown_connections 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f63274e550 0x55f632750940 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63273e350 0x55f63273bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f632735770 0x55f63273b9b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/3938512656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f63273b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8cecff9640 0 client.15840 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/3938512656 >> 172.21.15.86:0/3938512656 conn(0x55f63272f580 msgr2=0x55f6327312d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/3938512656 shutdown_connections 2024-04-16T05:35:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/3938512656 wait complete. 2024-04-16T05:35:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.476+0000 7f8d0dfc9740 20 client.15840 trim_cache size 0 max 16384 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.493+0000 7f8d0dfc9740 1 Processor -- start 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.493+0000 7f8d0dfc9740 1 -- start start 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.493+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f63273dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.493+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 0x55f632749c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.493+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632756230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.493+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041af470 con 0x55f63273e350 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.493+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041b0fb0 con 0x55f632734e50 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.493+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041afb40 con 0x55f632735770 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d098a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632756230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d098a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632756230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54984/0 (socket says 172.21.15.86:54984) 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d098a1640 1 -- 172.21.15.86:0/299101748 learned_addr learned my addr 172.21.15.86:0/299101748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d090a0640 1 --2- 172.21.15.86:0/299101748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f63273dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d098a1640 1 -- 172.21.15.86:0/299101748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 msgr2=0x55f632749c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d098a1640 1 --2- 172.21.15.86:0/299101748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 0x55f632749c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d098a1640 1 -- 172.21.15.86:0/299101748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 msgr2=0x55f63273dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d098a1640 1 --2- 172.21.15.86:0/299101748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f63273dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d098a1640 1 -- 172.21.15.86:0/299101748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f632734170 con 0x55f63273e350 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8d098a1640 1 --2- 172.21.15.86:0/299101748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632756230 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009d20 tx=0x7f8cfc009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712debdb36fa42f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8ceeffd640 1 -- 172.21.15.86:0/299101748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc012d00 con 0x55f63273e350 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8ceeffd640 1 -- 172.21.15.86:0/299101748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cfc021450 con 0x55f63273e350 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.494+0000 7f8ceeffd640 1 -- 172.21.15.86:0/299101748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc012ea0 con 0x55f63273e350 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/299101748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 msgr2=0x55f632756230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/299101748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632756230 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009d20 tx=0x7f8cfc009700 comp rx=0 tx=0).stop 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/299101748 shutdown_connections 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/299101748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632756230 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/299101748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 0x55f632749c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/299101748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f63273dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/299101748 >> 172.21.15.86:0/299101748 conn(0x55f63272f580 msgr2=0x55f63272f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/299101748 shutdown_connections 2024-04-16T05:35:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/299101748 wait complete. 2024-04-16T05:35:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 -- start start 2024-04-16T05:35:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.495+0000 7f8d0dfc9740 1 Processor -- start 2024-04-16T05:35:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f6327462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 0x55f632743740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0dfc9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632743c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041b08e0 con 0x55f63273e350 2024-04-16T05:35:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041b0210 con 0x55f632734e50 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0dfc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041ad430 con 0x55f632735770 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0dfc9740 10 client.0 _ll_register_callbacks cb 0x7f8cf80094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d098a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632743c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d098a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632743c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54990/0 (socket says 172.21.15.86:54990) 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d098a1640 1 -- 172.21.15.86:0/1811978951 learned_addr learned my addr 172.21.15.86:0/1811978951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d090a0640 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f6327462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0889f640 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 0x55f632743740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d098a1640 1 -- 172.21.15.86:0/1811978951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 msgr2=0x55f632743740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d098a1640 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 0x55f632743740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d098a1640 1 -- 172.21.15.86:0/1811978951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 msgr2=0x55f6327462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d098a1640 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f6327462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d098a1640 1 -- 172.21.15.86:0/1811978951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f632744270 con 0x55f63273e350 2024-04-16T05:35:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0889f640 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 0x55f632743740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d098a1640 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632743c80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc021e90 tx=0x7f8cfc021f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6c521c9157c246 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8cecff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8cecff9640 1 -- 172.21.15.86:0/1811978951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc01b040 con 0x55f63273e350 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8cecff9640 1 -- 172.21.15.86:0/1811978951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8cfc021600 con 0x55f63273e350 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.496+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/1811978951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cfc021450 con 0x55f63273e350 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8cecff9640 1 -- 172.21.15.86:0/1811978951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cfc03f050 con 0x55f63273e350 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8d0dfc9740 10 client.15846 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/1811978951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6327b48c0 con 0x55f63273e350 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8cecff9640 1 -- 172.21.15.86:0/1811978951 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cfc021450 con 0x55f63273e350 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8cecff9640 1 -- 172.21.15.86:0/1811978951 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8cfc03a030 con 0x55f63273e350 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8d0dfc9740 10 client.15846 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8d0dfc9740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8d0dfc9740 20 client.15846 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f6327b5000 0x55f6327b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8d0dfc9740 4 client.15846 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/1811978951 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6326a1a10 con 0x55f6327b5000 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.497+0000 7f8d090a0640 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f6327b5000 0x55f6327b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d090a0640 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f6327b5000 0x55f6327b7420 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e04fbe98a598d9ce server_cookie=684de53982d2625b in_seq=0 out_seq=0 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8cecff9640 10 client.15846 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8cecff9640 1 -- 172.21.15.86:0/1811978951 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f6326a1a10 con 0x55f6327b5000 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8cecff9640 10 client.15846 handle_command_reply: tid=0 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 client.15846 shutdown 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/1811978951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 msgr2=0x55f632743c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632743c80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc021e90 tx=0x7f8cfc021f80 comp rx=0 tx=0).stop 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/1811978951 shutdown_connections 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f6327b5000 0x55f6327b7420 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f63273e350 0x55f632743c80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f632735770 0x55f632743740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8cecff9640 0 client.15846 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 --2- 172.21.15.86:0/1811978951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f632734e50 0x55f6327462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/1811978951 >> 172.21.15.86:0/1811978951 conn(0x55f63272f580 msgr2=0x55f6327309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/1811978951 shutdown_connections 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 1 -- 172.21.15.86:0/1811978951 wait complete. 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.498+0000 7f8d0dfc9740 20 client.15846 trim_cache size 0 max 16384 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:35:55.503 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:55.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 -- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8cf00a3e40 msgr2=0x7f8cf00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8cf00a3e40 0x7f8cf00a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005c40 tx=0x7f8cfc009280 comp rx=0 tx=0).stop 2024-04-16T05:35:55.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 -- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 msgr2=0x7f8d0419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 0x7f8d0419e5f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d0002cc70 comp rx=0 tx=0).stop 2024-04-16T05:35:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 -- 172.21.15.86:0/3407346160 shutdown_connections 2024-04-16T05:35:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d04198700 0x7f8d04198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8cf00a3e40 0x7f8cf00a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d04107330 0x7f8d0419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 --2- 172.21.15.86:0/3407346160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0406a870 0x7f8d0419e5f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 -- 172.21.15.86:0/3407346160 >> 172.21.15.86:0/3407346160 conn(0x7f8d04076d00 msgr2=0x7f8d040ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 -- 172.21.15.86:0/3407346160 shutdown_connections 2024-04-16T05:35:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.499+0000 7f8d0b304640 1 -- 172.21.15.86:0/3407346160 wait complete. 2024-04-16T05:35:55.508 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:35:55.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:35:55.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 -- 172.21.15.86:0/875515422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c103260 msgr2=0x7f322c103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 --2- 172.21.15.86:0/875515422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c103260 0x7f322c103640 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c02bcd0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 -- 172.21.15.86:0/875515422 shutdown_connections 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 --2- 172.21.15.86:0/875515422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f322c10c630 0x7f322c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 --2- 172.21.15.86:0/875515422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f322c103b80 0x7f322c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 --2- 172.21.15.86:0/875515422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c103260 0x7f322c103640 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 -- 172.21.15.86:0/875515422 >> 172.21.15.86:0/875515422 conn(0x7f322c077280 msgr2=0x7f322c077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 -- 172.21.15.86:0/875515422 shutdown_connections 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 -- 172.21.15.86:0/875515422 wait complete. 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 Processor -- start 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 -- start start 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.616+0000 7f323380c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f322c103260 0x7f322c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f323380c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f322c103b80 0x7f322c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f323380c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c10c630 0x7f322c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f323380c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f322c112440 con 0x7f322c10c630 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f323380c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1122c0 con 0x7f322c103260 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f323380c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1125c0 con 0x7f322c103b80 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3231da9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c10c630 0x7f322c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3231da9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c10c630 0x7f322c198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55010/0 (socket says 172.21.15.86:55010) 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3231da9640 1 -- 172.21.15.86:0/4232641439 learned_addr learned my addr 172.21.15.86:0/4232641439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3230da7640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f322c103b80 0x7f322c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f32315a8640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f322c103260 0x7f322c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3231da9640 1 -- 172.21.15.86:0/4232641439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f322c103b80 msgr2=0x7f322c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3231da9640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f322c103b80 0x7f322c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3231da9640 1 -- 172.21.15.86:0/4232641439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f322c103260 msgr2=0x7f322c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3231da9640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f322c103260 0x7f322c19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3231da9640 1 -- 172.21.15.86:0/4232641439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c198f50 con 0x7f322c10c630 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3230da7640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f322c103b80 0x7f322c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:55.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f3231da9640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c10c630 0x7f322c198780 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f322400b3f0 tx=0x7f322400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f32227fc640 1 -- 172.21.15.86:0/4232641439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224015020 con 0x7f322c10c630 2024-04-16T05:35:55.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f32227fc640 1 -- 172.21.15.86:0/4232641439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3224011d10 con 0x7f322c10c630 2024-04-16T05:35:55.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f32227fc640 1 -- 172.21.15.86:0/4232641439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322400f2e0 con 0x7f322c10c630 2024-04-16T05:35:55.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c199240 con 0x7f322c10c630 2024-04-16T05:35:55.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.617+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c1a5460 con 0x7f322c10c630 2024-04-16T05:35:55.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.618+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c06ae20 con 0x7f322c10c630 2024-04-16T05:35:55.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.619+0000 7f32227fc640 1 -- 172.21.15.86:0/4232641439 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3224018020 con 0x7f322c10c630 2024-04-16T05:35:55.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.620+0000 7f32227fc640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32180a3e40 0x7f32180a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.620+0000 7f32315a8640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32180a3e40 0x7f32180a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.620+0000 7f32227fc640 1 -- 172.21.15.86:0/4232641439 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f32240d0600 con 0x7f322c10c630 2024-04-16T05:35:55.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.620+0000 7f32315a8640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32180a3e40 0x7f32180a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f321c02efd0 tx=0x7f321c000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.625+0000 7f32227fc640 1 -- 172.21.15.86:0/4232641439 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f322401f020 con 0x7f322c10c630 2024-04-16T05:35:55.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.846+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f322c199cb0 con 0x7f322c10c630 2024-04-16T05:35:55.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.847+0000 7f32227fc640 1 -- 172.21.15.86:0/4232641439 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f322409c7f0 con 0x7f322c10c630 2024-04-16T05:35:55.846 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:35:55.846 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:35:55.846 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:35:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32180a3e40 msgr2=0x7f32180a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32180a3e40 0x7f32180a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f321c02efd0 tx=0x7f321c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c10c630 msgr2=0x7f322c198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c10c630 0x7f322c198780 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f322400b3f0 tx=0x7f322400b8b0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 shutdown_connections 2024-04-16T05:35:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c10c630 0x7f322c198780 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f322c103b80 0x7f322c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32180a3e40 0x7f32180a6300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 --2- 172.21.15.86:0/4232641439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f322c103260 0x7f322c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 >> 172.21.15.86:0/4232641439 conn(0x7f322c077280 msgr2=0x7f322c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 shutdown_connections 2024-04-16T05:35:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.849+0000 7f323380c640 1 -- 172.21.15.86:0/4232641439 wait complete. 2024-04-16T05:35:55.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 -- 172.21.15.86:0/149342426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298101af0 msgr2=0x7f329810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 --2- 172.21.15.86:0/149342426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298101af0 0x7f329810dd50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3290009ca0 tx=0x7f329002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 -- 172.21.15.86:0/149342426 shutdown_connections 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 --2- 172.21.15.86:0/149342426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298101af0 0x7f329810dd50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 --2- 172.21.15.86:0/149342426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3298101150 0x7f32981015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 --2- 172.21.15.86:0/149342426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3298105e80 0x7f3298106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 -- 172.21.15.86:0/149342426 >> 172.21.15.86:0/149342426 conn(0x7f3298077cf0 msgr2=0x7f3298078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 -- 172.21.15.86:0/149342426 shutdown_connections 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 -- 172.21.15.86:0/149342426 wait complete. 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 Processor -- start 2024-04-16T05:35:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 -- start start 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3298101150 0x7f3298197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3298101af0 0x7f3298197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298105e80 0x7f329819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3298111360 con 0x7f3298105e80 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f32981111e0 con 0x7f3298101af0 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.967+0000 7f329f5d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32981114e0 con 0x7f3298101150 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329db71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298105e80 0x7f329819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329db71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298105e80 0x7f329819bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55022/0 (socket says 172.21.15.86:55022) 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329db71640 1 -- 172.21.15.86:0/2870419847 learned_addr learned my addr 172.21.15.86:0/2870419847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329d370640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3298101150 0x7f3298197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329cb6f640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3298101af0 0x7f3298197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329db71640 1 -- 172.21.15.86:0/2870419847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3298101150 msgr2=0x7f3298197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329db71640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3298101150 0x7f3298197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329db71640 1 -- 172.21.15.86:0/2870419847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3298101af0 msgr2=0x7f3298197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329db71640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3298101af0 0x7f3298197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329db71640 1 -- 172.21.15.86:0/2870419847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329819c6f0 con 0x7f3298105e80 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329d370640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3298101150 0x7f3298197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329cb6f640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3298101af0 0x7f3298197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329db71640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298105e80 0x7f329819bfc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f329002c700 tx=0x7f3290032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f328e7fc640 1 -- 172.21.15.86:0/2870419847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290039040 con 0x7f3298105e80 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f328e7fc640 1 -- 172.21.15.86:0/2870419847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3290044070 con 0x7f3298105e80 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329f5d4640 1 -- 172.21.15.86:0/2870419847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329819c980 con 0x7f3298105e80 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f328e7fc640 1 -- 172.21.15.86:0/2870419847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32900042a0 con 0x7f3298105e80 2024-04-16T05:35:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.968+0000 7f329f5d4640 1 -- 172.21.15.86:0/2870419847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32981a4270 con 0x7f3298105e80 2024-04-16T05:35:55.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.970+0000 7f328e7fc640 1 -- 172.21.15.86:0/2870419847 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3290004440 con 0x7f3298105e80 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f32a21c0740 1 Processor -- start 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f32a21c0740 1 -- start start 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de273e60 0x5628de274240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de27ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de27d360 0x5628de27f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f32a21c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5628de1d90e0 con 0x5628de27d360 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f32a21c0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5628de176520 con 0x5628de274780 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f32a21c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5628de1d50a0 con 0x5628de273e60 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f329cb6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de27d360 0x5628de27f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f328e7fc640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32840a3ec0 0x7f32840a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f329cb6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de27d360 0x5628de27f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55032/0 (socket says 172.21.15.86:55032) 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f329cb6f640 1 -- 172.21.15.86:0/1830442214 learned_addr learned my addr 172.21.15.86:0/1830442214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.971+0000 7f329db71640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32840a3ec0 0x7f32840a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de273e60 0x5628de274240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329d370640 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de27ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329cb6f640 1 -- 172.21.15.86:0/1830442214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de273e60 msgr2=0x5628de274240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de273e60 0x5628de274240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f328e7fc640 1 -- 172.21.15.86:0/2870419847 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f32900f30e0 con 0x7f3298105e80 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329cb6f640 1 -- 172.21.15.86:0/1830442214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 msgr2=0x5628de27ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de27ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329cb6f640 1 -- 172.21.15.86:0/1830442214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628de19e8d0 con 0x5628de27d360 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329d370640 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de27ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de27d360 0x5628de27f750 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32940145c0 tx=0x7f329400d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c541a0799a1aaf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f329db71640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32840a3ec0 0x7f32840a6380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32900f04b0 tx=0x7f32900916c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f32837fe640 1 -- 172.21.15.86:0/1830442214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294005220 con 0x5628de27d360 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f32837fe640 1 -- 172.21.15.86:0/1830442214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3294004c80 con 0x5628de27d360 2024-04-16T05:35:55.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.972+0000 7f32837fe640 1 -- 172.21.15.86:0/1830442214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400f880 con 0x5628de27d360 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 -- 172.21.15.86:0/1830442214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de27d360 msgr2=0x5628de27f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de27d360 0x5628de27f750 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32940145c0 tx=0x7f329400d620 comp rx=0 tx=0).stop 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 -- 172.21.15.86:0/1830442214 shutdown_connections 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de27d360 0x5628de27f750 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de27ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1830442214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de273e60 0x5628de274240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 -- 172.21.15.86:0/1830442214 >> 172.21.15.86:0/1830442214 conn(0x5628de26e590 msgr2=0x5628de26e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 -- 172.21.15.86:0/1830442214 shutdown_connections 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 -- 172.21.15.86:0/1830442214 wait complete. 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 -- start start 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.973+0000 7f32a21c0740 1 Processor -- start 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de27a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 0x5628de27a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de27d360 0x5628de27aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f32a21c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5628de280060 con 0x5628de273e60 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f32a21c0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5628de1924d0 con 0x5628de27d360 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f32a21c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5628de2801e0 con 0x5628de274780 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f32a21c0740 10 client.0 _ll_register_callbacks cb 0x5628de1772c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329cb6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de27a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329cb6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de27a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55048/0 (socket says 172.21.15.86:55048) 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329cb6f640 1 -- 172.21.15.86:0/2440944457 learned_addr learned my addr 172.21.15.86:0/2440944457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329d370640 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 0x5628de27a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329cb6f640 1 -- 172.21.15.86:0/2440944457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 msgr2=0x5628de27a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329cb6f640 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 0x5628de27a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329cb6f640 1 -- 172.21.15.86:0/2440944457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de27d360 msgr2=0x5628de27aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329cb6f640 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de27d360 0x5628de27aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329cb6f640 1 -- 172.21.15.86:0/2440944457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628ddeeb3f0 con 0x5628de273e60 2024-04-16T05:35:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329d370640 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 0x5628de27a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f329cb6f640 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de27a460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f329400d8c0 tx=0x7f3294014300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec234f3853cee35 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f3280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f3280ff9640 1 -- 172.21.15.86:0/2440944457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940048c0 con 0x5628de273e60 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f3280ff9640 1 -- 172.21.15.86:0/2440944457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3294004a60 con 0x5628de273e60 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f32a21c0740 1 -- 172.21.15.86:0/2440944457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3294004c80 con 0x5628de273e60 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.974+0000 7f3280ff9640 1 -- 172.21.15.86:0/2440944457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3294005000 con 0x5628de273e60 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f32a21c0740 10 client.15864 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f32a21c0740 1 -- 172.21.15.86:0/2440944457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5628ddfb57d0 con 0x5628de273e60 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f3280ff9640 1 -- 172.21.15.86:0/2440944457 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3294004c80 con 0x5628de273e60 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f3280ff9640 1 -- 172.21.15.86:0/2440944457 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f329400f520 con 0x5628de273e60 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f32a21c0740 10 client.15864 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f32a21c0740 10 client.15864 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f32a21c0740 20 client.15864 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f32a21c0740 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5628de28d230 0x5628de28f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f32a21c0740 4 client.15864 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f32a21c0740 1 -- 172.21.15.86:0/2440944457 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628de1e6b00 con 0x5628de28d230 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.975+0000 7f329d370640 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5628de28d230 0x5628de28f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.976+0000 7f329d370640 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5628de28d230 0x5628de28f620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8b35667910e1e450 server_cookie=5f3ed929314c6acb in_seq=0 out_seq=0 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.976+0000 7f3280ff9640 10 client.15864 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f3280ff9640 1 -- 172.21.15.86:0/2440944457 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628de1e6b00 con 0x5628de28d230 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f3280ff9640 10 client.15864 handle_command_reply: tid=0 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 client.15864 shutdown 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 -- 172.21.15.86:0/2440944457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 msgr2=0x5628de27a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de27a460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f329400d8c0 tx=0x7f3294014300 comp rx=0 tx=0).stop 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 -- 172.21.15.86:0/2440944457 shutdown_connections 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5628de28d230 0x5628de28f620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de27d360 0x5628de27aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f3280ff9640 0 client.15864 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:35:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 0x5628de27a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 --2- 172.21.15.86:0/2440944457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de27a460 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 -- 172.21.15.86:0/2440944457 >> 172.21.15.86:0/2440944457 conn(0x5628de26e590 msgr2=0x5628de2702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 -- 172.21.15.86:0/2440944457 shutdown_connections 2024-04-16T05:35:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 1 -- 172.21.15.86:0/2440944457 wait complete. 2024-04-16T05:35:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.977+0000 7f32a21c0740 20 client.15864 trim_cache size 0 max 16384 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.990+0000 7f32a21c0740 1 Processor -- start 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.990+0000 7f32a21c0740 1 -- start start 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.990+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de289060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.990+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 0x5628de294e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.990+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de27d360 0x5628de295390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.990+0000 7f32a21c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32981ad0c0 con 0x5628de273e60 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.990+0000 7f32a21c0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f32981aec00 con 0x5628de27d360 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.990+0000 7f32a21c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32981ad790 con 0x5628de274780 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f329cb6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de289060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f329cb6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de289060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55058/0 (socket says 172.21.15.86:55058) 2024-04-16T05:35:55.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f329cb6f640 1 -- 172.21.15.86:0/1654205767 learned_addr learned my addr 172.21.15.86:0/1654205767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f329cb6f640 1 -- 172.21.15.86:0/1654205767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 msgr2=0x5628de294e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1654205767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 0x5628de294e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f329cb6f640 1 -- 172.21.15.86:0/1654205767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de27d360 msgr2=0x5628de295390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1654205767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de27d360 0x5628de295390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f329cb6f640 1 -- 172.21.15.86:0/1654205767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628de273060 con 0x5628de273e60 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1654205767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de289060 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f329400f430 tx=0x7f329400df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd95cecbb6c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f3282ffd640 1 -- 172.21.15.86:0/1654205767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400f520 con 0x5628de273e60 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f3282ffd640 1 -- 172.21.15.86:0/1654205767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329400f6c0 con 0x5628de273e60 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.991+0000 7f3282ffd640 1 -- 172.21.15.86:0/1654205767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294004c30 con 0x5628de273e60 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 -- 172.21.15.86:0/1654205767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 msgr2=0x5628de289060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1654205767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de289060 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f329400f430 tx=0x7f329400df30 comp rx=0 tx=0).stop 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 -- 172.21.15.86:0/1654205767 shutdown_connections 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1654205767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de27d360 0x5628de295390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1654205767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de274780 0x5628de294e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1654205767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de289060 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 -- 172.21.15.86:0/1654205767 >> 172.21.15.86:0/1654205767 conn(0x5628de26e590 msgr2=0x5628de2695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:55.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 -- 172.21.15.86:0/1654205767 shutdown_connections 2024-04-16T05:35:55.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 -- 172.21.15.86:0/1654205767 wait complete. 2024-04-16T05:35:55.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 -- start start 2024-04-16T05:35:55.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 Processor -- start 2024-04-16T05:35:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.992+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de285a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de277b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f32a21c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de27d360 0x5628de278110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f32a21c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32981ae530 con 0x5628de273e60 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f32a21c0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f32981ade60 con 0x5628de274780 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f32a21c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32981ab080 con 0x5628de27d360 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f32a21c0740 10 client.0 _ll_register_callbacks cb 0x7f3294014470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329cb6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de285a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329cb6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de285a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55062/0 (socket says 172.21.15.86:55062) 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329cb6f640 1 -- 172.21.15.86:0/1860789905 learned_addr learned my addr 172.21.15.86:0/1860789905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329d370640 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de277b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329cb6f640 1 -- 172.21.15.86:0/1860789905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de27d360 msgr2=0x5628de278110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de27d360 0x5628de278110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329cb6f640 1 -- 172.21.15.86:0/1860789905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 msgr2=0x5628de277b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de277b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329cb6f640 1 -- 172.21.15.86:0/1860789905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628de276920 con 0x5628de273e60 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329d370640 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de277b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f329cb6f640 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de285a40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f32940047b0 tx=0x7f329400f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2a0e5b0983129f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f3280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f3280ff9640 1 -- 172.21.15.86:0/1860789905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400fd40 con 0x5628de273e60 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f3280ff9640 1 -- 172.21.15.86:0/1860789905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f32940049b0 con 0x5628de273e60 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f32a21c0740 1 -- 172.21.15.86:0/1860789905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329400f6c0 con 0x5628de273e60 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f3280ff9640 1 -- 172.21.15.86:0/1860789905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3294048050 con 0x5628de273e60 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f32a21c0740 10 client.15870 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f32a21c0740 1 -- 172.21.15.86:0/1860789905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5628de2f2bf0 con 0x5628de273e60 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.993+0000 7f3280ff9640 1 -- 172.21.15.86:0/1860789905 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329400f6c0 con 0x5628de273e60 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f3280ff9640 1 -- 172.21.15.86:0/1860789905 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3294035020 con 0x5628de273e60 2024-04-16T05:35:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f32a21c0740 10 client.15870 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f32a21c0740 10 client.15870 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f32a21c0740 20 client.15870 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5628de2f3250 0x5628de2f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f32a21c0740 4 client.15870 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f32a21c0740 1 -- 172.21.15.86:0/1860789905 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628de1e0970 con 0x5628de2f3250 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f329d370640 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5628de2f3250 0x5628de2f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f3280ff9640 10 client.15870 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.994+0000 7f329d370640 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5628de2f3250 0x5628de2f5670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd4ed4b30fa52c5b server_cookie=9d3a428d42e81239 in_seq=0 out_seq=0 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f3280ff9640 1 -- 172.21.15.86:0/1860789905 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628de1e0970 con 0x5628de2f3250 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f3280ff9640 10 client.15870 handle_command_reply: tid=0 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 client.15870 shutdown 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 -- 172.21.15.86:0/1860789905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 msgr2=0x5628de285a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de285a40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f32940047b0 tx=0x7f329400f5b0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 -- 172.21.15.86:0/1860789905 shutdown_connections 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5628de2f3250 0x5628de2f5670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628de27d360 0x5628de278110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628de274780 0x5628de277b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 --2- 172.21.15.86:0/1860789905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628de273e60 0x5628de285a40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f3280ff9640 0 client.15870 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 -- 172.21.15.86:0/1860789905 >> 172.21.15.86:0/1860789905 conn(0x5628de26e590 msgr2=0x5628de278d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 -- 172.21.15.86:0/1860789905 shutdown_connections 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 1 -- 172.21.15.86:0/1860789905 wait complete. 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f32a21c0740 20 client.15870 trim_cache size 0 max 16384 2024-04-16T05:35:56.000 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:35:56.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 -- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32840a3ec0 msgr2=0x7f32840a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32840a3ec0 0x7f32840a6380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32900f04b0 tx=0x7f32900916c0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 -- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298105e80 msgr2=0x7f329819bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298105e80 0x7f329819bfc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f329002c700 tx=0x7f3290032f20 comp rx=0 tx=0).stop 2024-04-16T05:35:56.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 -- 172.21.15.86:0/2870419847 shutdown_connections 2024-04-16T05:35:56.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3298105e80 0x7f329819bfc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32840a3ec0 0x7f32840a6380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3298101af0 0x7f3298197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 --2- 172.21.15.86:0/2870419847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3298101150 0x7f3298197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 -- 172.21.15.86:0/2870419847 >> 172.21.15.86:0/2870419847 conn(0x7f3298077cf0 msgr2=0x7f32980febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.995+0000 7f329f5d4640 1 -- 172.21.15.86:0/2870419847 shutdown_connections 2024-04-16T05:35:56.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:55.996+0000 7f329f5d4640 1 -- 172.21.15.86:0/2870419847 wait complete. 2024-04-16T05:35:56.005 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.112+0000 7f559bc55640 1 -- 172.21.15.86:0/623912673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406b1f0 msgr2=0x7f5594104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.112+0000 7f559bc55640 1 --2- 172.21.15.86:0/623912673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406b1f0 0x7f5594104a50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f558c009ca0 tx=0x7f558c02c180 comp rx=0 tx=0).stop 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.112+0000 7f559bc55640 1 -- 172.21.15.86:0/623912673 shutdown_connections 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.112+0000 7f559bc55640 1 --2- 172.21.15.86:0/623912673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406b1f0 0x7f5594104a50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.112+0000 7f559bc55640 1 --2- 172.21.15.86:0/623912673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559406a680 0x7f559406aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.112+0000 7f559bc55640 1 --2- 172.21.15.86:0/623912673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5594106f50 0x7f5594107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.112+0000 7f559bc55640 1 -- 172.21.15.86:0/623912673 >> 172.21.15.86:0/623912673 conn(0x7f5594076d00 msgr2=0x7f5594077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.112+0000 7f559bc55640 1 -- 172.21.15.86:0/623912673 shutdown_connections 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.112+0000 7f559bc55640 1 -- 172.21.15.86:0/623912673 wait complete. 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f559bc55640 1 Processor -- start 2024-04-16T05:35:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f559bc55640 1 -- start start 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f559bc55640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406a680 0x7f559419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f559bc55640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559406b1f0 0x7f559419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f559bc55640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5594106f50 0x7f55941a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f559bc55640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55941189b0 con 0x7f559406a680 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f559bc55640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5594118830 con 0x7f5594106f50 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f559bc55640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5594118b30 con 0x7f559406b1f0 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f55999f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406a680 0x7f559419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f55999f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406a680 0x7f559419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55080/0 (socket says 172.21.15.86:55080) 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f55999f1640 1 -- 172.21.15.86:0/247213234 learned_addr learned my addr 172.21.15.86:0/247213234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f55991f0640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559406b1f0 0x7f559419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f55999f1640 1 -- 172.21.15.86:0/247213234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559406b1f0 msgr2=0x7f559419f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f55999f1640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559406b1f0 0x7f559419f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f55999f1640 1 -- 172.21.15.86:0/247213234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5594106f50 msgr2=0x7f55941a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f55999f1640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5594106f50 0x7f55941a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.113+0000 7f55999f1640 1 -- 172.21.15.86:0/247213234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55941a3e00 con 0x7f559406a680 2024-04-16T05:35:56.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.114+0000 7f55999f1640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406a680 0x7f559419ee10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f559000c920 tx=0x7f559000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.114+0000 7f558affd640 1 -- 172.21.15.86:0/247213234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590007c70 con 0x7f559406a680 2024-04-16T05:35:56.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.114+0000 7f558affd640 1 -- 172.21.15.86:0/247213234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5590007e10 con 0x7f559406a680 2024-04-16T05:35:56.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.114+0000 7f559bc55640 1 -- 172.21.15.86:0/247213234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55941a40f0 con 0x7f559406a680 2024-04-16T05:35:56.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.114+0000 7f558affd640 1 -- 172.21.15.86:0/247213234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590018890 con 0x7f559406a680 2024-04-16T05:35:56.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.114+0000 7f559bc55640 1 -- 172.21.15.86:0/247213234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55941ab980 con 0x7f559406a680 2024-04-16T05:35:56.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.116+0000 7f558affd640 1 -- 172.21.15.86:0/247213234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5590018a70 con 0x7f559406a680 2024-04-16T05:35:56.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.116+0000 7f559e840740 1 Processor -- start 2024-04-16T05:35:56.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559e840740 1 -- start start 2024-04-16T05:35:56.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559e840740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ec7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559e840740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559e840740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 0x55b728ed2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559e840740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b728e2c0e0 con 0x55b728ec7780 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559e840740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b728dc9520 con 0x55b728ed0360 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559e840740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b728e280a0 con 0x55b728ec6e60 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f55999f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f55999f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55092/0 (socket says 172.21.15.86:55092) 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f55999f1640 1 -- 172.21.15.86:0/565130730 learned_addr learned my addr 172.21.15.86:0/565130730 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f55991f0640 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 0x55b728ed2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559a1f2640 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ec7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f558affd640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55800a3e40 0x7f55800a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f55999f1640 1 -- 172.21.15.86:0/565130730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 msgr2=0x55b728ec7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f55999f1640 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ec7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f55999f1640 1 -- 172.21.15.86:0/565130730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 msgr2=0x55b728ed2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f55999f1640 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 0x55b728ed2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f55999f1640 1 -- 172.21.15.86:0/565130730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b728df18d0 con 0x55b728ec7780 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559a1f2640 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ec7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.117+0000 7f559a1f2640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55800a3e40 0x7f55800a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.118+0000 7f558affd640 1 -- 172.21.15.86:0/247213234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f55900d0ef0 con 0x7f559406a680 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.118+0000 7f55999f1640 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecfe20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f55900ce0f0 tx=0x7f5590020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f150b7820f616ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.118+0000 7f557ffff640 1 -- 172.21.15.86:0/565130730 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559009c230 con 0x55b728ec7780 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.118+0000 7f557ffff640 1 -- 172.21.15.86:0/565130730 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5590085e10 con 0x55b728ec7780 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.118+0000 7f557ffff640 1 -- 172.21.15.86:0/565130730 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559009c8d0 con 0x55b728ec7780 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.118+0000 7f559a1f2640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55800a3e40 0x7f55800a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f558c009430 tx=0x7f558c009360 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 -- 172.21.15.86:0/565130730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 msgr2=0x55b728ecfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecfe20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f55900ce0f0 tx=0x7f5590020000 comp rx=0 tx=0).stop 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 -- 172.21.15.86:0/565130730 shutdown_connections 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 0x55b728ed2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecfe20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 --2- 172.21.15.86:0/565130730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ec7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 -- 172.21.15.86:0/565130730 >> 172.21.15.86:0/565130730 conn(0x55b728ec1590 msgr2=0x55b728ec1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 -- 172.21.15.86:0/565130730 shutdown_connections 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 -- 172.21.15.86:0/565130730 wait complete. 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 -- start start 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 Processor -- start 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.119+0000 7f559e840740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ecd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559e840740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 0x55b728ecd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559e840740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ed0360 0x55b728ecdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559e840740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b728ed2fa0 con 0x55b728ed0360 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559e840740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b728de54d0 con 0x55b728ec7780 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559e840740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b728ed3120 con 0x55b728ec6e60 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559e840740 10 client.0 _ll_register_callbacks cb 0x55b728dca2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559a1f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ed0360 0x55b728ecdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559a1f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ed0360 0x55b728ecdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55106/0 (socket says 172.21.15.86:55106) 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559a1f2640 1 -- 172.21.15.86:0/26426480 learned_addr learned my addr 172.21.15.86:0/26426480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f55991f0640 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 0x55b728ecd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f55999f1640 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ecd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559a1f2640 1 -- 172.21.15.86:0/26426480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 msgr2=0x55b728ecd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559a1f2640 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ecd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559a1f2640 1 -- 172.21.15.86:0/26426480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 msgr2=0x55b728ecd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559a1f2640 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 0x55b728ecd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559a1f2640 1 -- 172.21.15.86:0/26426480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b728b3e3f0 con 0x55b728ed0360 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f55991f0640 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 0x55b728ecd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559a1f2640 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ed0360 0x55b728ecdef0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f558c036000 tx=0x7f558c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faafaeb3dd326f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f557d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f557d7fa640 1 -- 172.21.15.86:0/26426480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c04f060 con 0x55b728ed0360 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f557d7fa640 1 -- 172.21.15.86:0/26426480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f558c032a80 con 0x55b728ed0360 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559e840740 1 -- 172.21.15.86:0/26426480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5590085e10 con 0x55b728ed0360 2024-04-16T05:35:56.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f557d7fa640 1 -- 172.21.15.86:0/26426480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f558c056050 con 0x55b728ed0360 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559e840740 10 client.15882 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.120+0000 7f559e840740 1 -- 172.21.15.86:0/26426480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b728c087d0 con 0x55b728ed0360 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f557d7fa640 1 -- 172.21.15.86:0/26426480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5590085e10 con 0x55b728ed0360 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f557d7fa640 1 -- 172.21.15.86:0/26426480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f558c03fbf0 con 0x55b728ed0360 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f559e840740 10 client.15882 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f559e840740 10 client.15882 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f559e840740 20 client.15882 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f559e840740 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b728ee0290 0x55b728ee26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f559e840740 4 client.15882 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f559e840740 1 -- 172.21.15.86:0/26426480 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b728e39b00 con 0x55b728ee0290 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f55999f1640 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b728ee0290 0x55b728ee26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f557d7fa640 10 client.15882 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:35:56.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.121+0000 7f55999f1640 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b728ee0290 0x55b728ee26b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9df34f952a10ef47 server_cookie=6c4b8078f092c54e in_seq=0 out_seq=0 2024-04-16T05:35:56.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f557d7fa640 1 -- 172.21.15.86:0/26426480 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b728e39b00 con 0x55b728ee0290 2024-04-16T05:35:56.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f557d7fa640 10 client.15882 handle_command_reply: tid=0 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 client.15882 shutdown 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 -- 172.21.15.86:0/26426480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ed0360 msgr2=0x55b728ecdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ed0360 0x55b728ecdef0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f558c036000 tx=0x7f558c0095c0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 -- 172.21.15.86:0/26426480 shutdown_connections 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ed0360 0x55b728ecdef0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 0x55b728ecd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b728ee0290 0x55b728ee26b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 --2- 172.21.15.86:0/26426480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ecd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f557d7fa640 0 client.15882 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 -- 172.21.15.86:0/26426480 >> 172.21.15.86:0/26426480 conn(0x55b728ec1590 msgr2=0x55b728ec32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 -- 172.21.15.86:0/26426480 shutdown_connections 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.122+0000 7f559e840740 1 -- 172.21.15.86:0/26426480 wait complete. 2024-04-16T05:35:56.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.123+0000 7f559e840740 20 client.15882 trim_cache size 0 max 16384 2024-04-16T05:35:56.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f559e840740 1 Processor -- start 2024-04-16T05:35:56.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f559e840740 1 -- start start 2024-04-16T05:35:56.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f559e840740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ecf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f559e840740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f559e840740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 0x55b728ee7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f559e840740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b5a40 con 0x55b728ec7780 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f559e840740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b7580 con 0x55b728ed0360 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f559e840740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b6110 con 0x55b728ec6e60 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55991f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55991f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55116/0 (socket says 172.21.15.86:55116) 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55991f0640 1 -- 172.21.15.86:0/3323283596 learned_addr learned my addr 172.21.15.86:0/3323283596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f559a1f2640 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 0x55b728ee7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55999f1640 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ecf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55991f0640 1 -- 172.21.15.86:0/3323283596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 msgr2=0x55b728ecf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55991f0640 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ecf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55991f0640 1 -- 172.21.15.86:0/3323283596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 msgr2=0x55b728ee7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55991f0640 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 0x55b728ee7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55991f0640 1 -- 172.21.15.86:0/3323283596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b728ecc460 con 0x55b728ec7780 2024-04-16T05:35:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55999f1640 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ecf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.136+0000 7f55991f0640 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecdd10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f55840045a0 tx=0x7f558400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe6b511bf5b0913 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f557f7fe640 1 -- 172.21.15.86:0/3323283596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584020070 con 0x55b728ec7780 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f557f7fe640 1 -- 172.21.15.86:0/3323283596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5584003cf0 con 0x55b728ec7780 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f557f7fe640 1 -- 172.21.15.86:0/3323283596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558401b650 con 0x55b728ec7780 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 -- 172.21.15.86:0/3323283596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 msgr2=0x55b728ecdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecdd10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f55840045a0 tx=0x7f558400bcb0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 -- 172.21.15.86:0/3323283596 shutdown_connections 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ed0360 0x55b728ee7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec7780 0x55b728ecdd10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 --2- 172.21.15.86:0/3323283596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ec6e60 0x55b728ecf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 -- 172.21.15.86:0/3323283596 >> 172.21.15.86:0/3323283596 conn(0x55b728ec1590 msgr2=0x55b728ec1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 -- 172.21.15.86:0/3323283596 shutdown_connections 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 -- 172.21.15.86:0/3323283596 wait complete. 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.137+0000 7f559e840740 1 -- start start 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559e840740 1 Processor -- start 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559e840740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec6e60 0x55b728eca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559e840740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 0x55b728eca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559e840740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ed0360 0x55b728ecada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559e840740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b6eb0 con 0x55b728ec6e60 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559e840740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b67e0 con 0x55b728ec7780 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559e840740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b3a00 con 0x55b728ed0360 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559e840740 10 client.0 _ll_register_callbacks cb 0x7f559407a330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55999f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec6e60 0x55b728eca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55999f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec6e60 0x55b728eca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55122/0 (socket says 172.21.15.86:55122) 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55999f1640 1 -- 172.21.15.86:0/1696164186 learned_addr learned my addr 172.21.15.86:0/1696164186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55991f0640 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 0x55b728eca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559a1f2640 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ed0360 0x55b728ecada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55999f1640 1 -- 172.21.15.86:0/1696164186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ed0360 msgr2=0x55b728ecada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55999f1640 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ed0360 0x55b728ecada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55999f1640 1 -- 172.21.15.86:0/1696164186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 msgr2=0x55b728eca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55999f1640 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 0x55b728eca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55999f1640 1 -- 172.21.15.86:0/1696164186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b728f459f0 con 0x55b728ec6e60 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f559a1f2640 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ed0360 0x55b728ecada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f55999f1640 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec6e60 0x55b728eca320 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f559001b000 tx=0x7f559000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540a284ffd91f3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.138+0000 7f557d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f557d7fa640 1 -- 172.21.15.86:0/1696164186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559009c410 con 0x55b728ec6e60 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f557d7fa640 1 -- 172.21.15.86:0/1696164186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f559009abd0 con 0x55b728ec6e60 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f559e840740 1 -- 172.21.15.86:0/1696164186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5584003cf0 con 0x55b728ec6e60 2024-04-16T05:35:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f557d7fa640 1 -- 172.21.15.86:0/1696164186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55900ce090 con 0x55b728ec6e60 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f559e840740 10 client.15888 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f559e840740 1 -- 172.21.15.86:0/1696164186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b728f46c60 con 0x55b728ec6e60 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f557d7fa640 1 -- 172.21.15.86:0/1696164186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5584003cf0 con 0x55b728ec6e60 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f557d7fa640 1 -- 172.21.15.86:0/1696164186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5590098f40 con 0x55b728ec6e60 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f559e840740 10 client.15888 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f559e840740 10 client.15888 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f559e840740 20 client.15888 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f559e840740 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b728f46fa0 0x55b728f49390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f559e840740 4 client.15888 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f559e840740 1 -- 172.21.15.86:0/1696164186 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b728e33970 con 0x55b728f46fa0 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.139+0000 7f55991f0640 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b728f46fa0 0x55b728f49390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f557d7fa640 10 client.15888 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f55991f0640 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b728f46fa0 0x55b728f49390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc7c24fb82d33852 server_cookie=5685339f2bece9f4 in_seq=0 out_seq=0 2024-04-16T05:35:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f557d7fa640 1 -- 172.21.15.86:0/1696164186 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b728e33970 con 0x55b728f46fa0 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f557d7fa640 10 client.15888 handle_command_reply: tid=0 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 client.15888 shutdown 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 -- 172.21.15.86:0/1696164186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec6e60 msgr2=0x55b728eca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec6e60 0x55b728eca320 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f559001b000 tx=0x7f559000cf10 comp rx=0 tx=0).stop 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 -- 172.21.15.86:0/1696164186 shutdown_connections 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b728f46fa0 0x55b728f49390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b728ed0360 0x55b728ecada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b728ec7780 0x55b728eca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f557d7fa640 0 client.15888 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 --2- 172.21.15.86:0/1696164186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b728ec6e60 0x55b728eca320 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 -- 172.21.15.86:0/1696164186 >> 172.21.15.86:0/1696164186 conn(0x55b728ec1590 msgr2=0x55b728ec0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 -- 172.21.15.86:0/1696164186 shutdown_connections 2024-04-16T05:35:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 1 -- 172.21.15.86:0/1696164186 wait complete. 2024-04-16T05:35:56.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.140+0000 7f559e840740 20 client.15888 trim_cache size 0 max 16384 2024-04-16T05:35:56.147 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 -- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55800a3e40 msgr2=0x7f55800a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55800a3e40 0x7f55800a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f558c009430 tx=0x7f558c009360 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 -- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406a680 msgr2=0x7f559419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406a680 0x7f559419ee10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f559000c920 tx=0x7f559000cde0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 -- 172.21.15.86:0/247213234 shutdown_connections 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55800a3e40 0x7f55800a6300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5594106f50 0x7f55941a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559406b1f0 0x7f559419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 --2- 172.21.15.86:0/247213234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559406a680 0x7f559419ee10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 -- 172.21.15.86:0/247213234 >> 172.21.15.86:0/247213234 conn(0x7f5594076d00 msgr2=0x7f55941037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 -- 172.21.15.86:0/247213234 shutdown_connections 2024-04-16T05:35:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.141+0000 7f559bc55640 1 -- 172.21.15.86:0/247213234 wait complete. 2024-04-16T05:35:56.150 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 -- 172.21.15.86:0/64984613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a4106200 msgr2=0x7f77a410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 --2- 172.21.15.86:0/64984613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a4106200 0x7f77a410ddd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f779c009ca0 tx=0x7f779c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 -- 172.21.15.86:0/64984613 shutdown_connections 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 --2- 172.21.15.86:0/64984613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a4106200 0x7f77a410ddd0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 --2- 172.21.15.86:0/64984613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101ae0 0x7f77a4105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 --2- 172.21.15.86:0/64984613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4101130 0x7f77a4101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 -- 172.21.15.86:0/64984613 >> 172.21.15.86:0/64984613 conn(0x7f77a40780b0 msgr2=0x7f77a40784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 -- 172.21.15.86:0/64984613 shutdown_connections 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 -- 172.21.15.86:0/64984613 wait complete. 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 Processor -- start 2024-04-16T05:35:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.259+0000 7f77abc7b640 1 -- start start 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77abc7b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101130 0x7f77a41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77abc7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4106200 0x7f77a4199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77abc7b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a419a170 0x7f77a41a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77abc7b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41112c0 con 0x7f77a419a170 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77abc7b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4111140 con 0x7f77a4101130 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77abc7b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4111440 con 0x7f77a4106200 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77aa218640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a419a170 0x7f77a41a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77aa218640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a419a170 0x7f77a41a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55136/0 (socket says 172.21.15.86:55136) 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77aa218640 1 -- 172.21.15.86:0/4005293314 learned_addr learned my addr 172.21.15.86:0/4005293314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77a9a17640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101130 0x7f77a41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77a9216640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4106200 0x7f77a4199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77aa218640 1 -- 172.21.15.86:0/4005293314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4106200 msgr2=0x7f77a4199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77aa218640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4106200 0x7f77a4199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77aa218640 1 -- 172.21.15.86:0/4005293314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101130 msgr2=0x7f77a41996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77aa218640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101130 0x7f77a41996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77aa218640 1 -- 172.21.15.86:0/4005293314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a41a4f20 con 0x7f77a419a170 2024-04-16T05:35:56.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77a9216640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4106200 0x7f77a4199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:56.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77aa218640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a419a170 0x7f77a41a4720 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f779c02c700 tx=0x7f779c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f7796ffd640 1 -- 172.21.15.86:0/4005293314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c039040 con 0x7f77a419a170 2024-04-16T05:35:56.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f7796ffd640 1 -- 172.21.15.86:0/4005293314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c044070 con 0x7f77a419a170 2024-04-16T05:35:56.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77abc7b640 1 -- 172.21.15.86:0/4005293314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a41a51b0 con 0x7f77a419a170 2024-04-16T05:35:56.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f7796ffd640 1 -- 172.21.15.86:0/4005293314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0042a0 con 0x7f77a419a170 2024-04-16T05:35:56.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.260+0000 7f77abc7b640 1 -- 172.21.15.86:0/4005293314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a56a0 con 0x7f77a419a170 2024-04-16T05:35:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.263+0000 7f7796ffd640 1 -- 172.21.15.86:0/4005293314 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f779c004440 con 0x7f77a419a170 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.263+0000 7f77ae86e740 1 Processor -- start 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.263+0000 7f77ae86e740 1 -- start start 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a7e60 0x55ca874a8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 0x55ca874b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77ae86e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca8740d0e0 con 0x55ca874a8780 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77ae86e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ca873aa520 con 0x55ca874a7e60 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77ae86e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ca874090a0 con 0x55ca874b1360 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f7796ffd640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77900a3e40 0x7f77900a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9a17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9a17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55140/0 (socket says 172.21.15.86:55140) 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9a17640 1 -- 172.21.15.86:0/1491305217 learned_addr learned my addr 172.21.15.86:0/1491305217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77aa218640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77900a3e40 0x7f77900a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9216640 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a7e60 0x55ca874a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f7796ffd640 1 -- 172.21.15.86:0/4005293314 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f779c0f3100 con 0x7f77a419a170 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9216640 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 0x55ca874b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9a17640 1 -- 172.21.15.86:0/1491305217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 msgr2=0x55ca874b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9a17640 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 0x55ca874b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9a17640 1 -- 172.21.15.86:0/1491305217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a7e60 msgr2=0x55ca874a8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9a17640 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a7e60 0x55ca874a8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9a17640 1 -- 172.21.15.86:0/1491305217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca873d28d0 con 0x55ca874a8780 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9216640 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 0x55ca874b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77aa218640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77900a3e40 0x7f77900a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f779c0f04d0 tx=0x7f779c0b4080 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f77a9a17640 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874b0e20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f779800b760 tx=0x7f779800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d1786d572e32ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.264+0000 7f778ffff640 1 -- 172.21.15.86:0/1491305217 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798003dd0 con 0x55ca874a8780 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f778ffff640 1 -- 172.21.15.86:0/1491305217 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798015070 con 0x55ca874a8780 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f778ffff640 1 -- 172.21.15.86:0/1491305217 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798004e10 con 0x55ca874a8780 2024-04-16T05:35:56.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f77ae86e740 1 -- 172.21.15.86:0/1491305217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 msgr2=0x55ca874b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874b0e20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f779800b760 tx=0x7f779800bc20 comp rx=0 tx=0).stop 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f77ae86e740 1 -- 172.21.15.86:0/1491305217 shutdown_connections 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 0x55ca874b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874b0e20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1491305217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a7e60 0x55ca874a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f77ae86e740 1 -- 172.21.15.86:0/1491305217 >> 172.21.15.86:0/1491305217 conn(0x55ca874a2590 msgr2=0x55ca874a2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f77ae86e740 1 -- 172.21.15.86:0/1491305217 shutdown_connections 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.265+0000 7f77ae86e740 1 -- 172.21.15.86:0/1491305217 wait complete. 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77ae86e740 1 -- start start 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77ae86e740 1 Processor -- start 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a8780 0x55ca874ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 0x55ca874aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77ae86e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca874b4060 con 0x55ca874a7e60 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77ae86e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ca873c64d0 con 0x55ca874a8780 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77ae86e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ca874b41e0 con 0x55ca874b1360 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77ae86e740 10 client.0 _ll_register_callbacks cb 0x55ca873ab2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77a9216640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77a9216640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55144/0 (socket says 172.21.15.86:55144) 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.266+0000 7f77a9216640 1 -- 172.21.15.86:0/1440446990 learned_addr learned my addr 172.21.15.86:0/1440446990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77a9216640 1 -- 172.21.15.86:0/1440446990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 msgr2=0x55ca874aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77a9216640 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 0x55ca874aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77a9a17640 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a8780 0x55ca874ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77a9216640 1 -- 172.21.15.86:0/1440446990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a8780 msgr2=0x55ca874ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77a9216640 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a8780 0x55ca874ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77a9216640 1 -- 172.21.15.86:0/1440446990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8711f3f0 con 0x55ca874a7e60 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77a9216640 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874ae350 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f77a000e330 tx=0x7f77a0013f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83e30d874fca871 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f778d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f778d7fa640 1 -- 172.21.15.86:0/1440446990 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0010ca0 con 0x55ca874a7e60 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f778d7fa640 1 -- 172.21.15.86:0/1440446990 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f77a001d450 con 0x55ca874a7e60 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77ae86e740 1 -- 172.21.15.86:0/1440446990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7798015070 con 0x55ca874a7e60 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f778d7fa640 1 -- 172.21.15.86:0/1440446990 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77a0026050 con 0x55ca874a7e60 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77ae86e740 10 client.15900 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77ae86e740 1 -- 172.21.15.86:0/1440446990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca871e97d0 con 0x55ca874a7e60 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f778d7fa640 1 -- 172.21.15.86:0/1440446990 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798015070 con 0x55ca874a7e60 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f778d7fa640 1 -- 172.21.15.86:0/1440446990 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f77a000f040 con 0x55ca874a7e60 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77ae86e740 10 client.15900 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77ae86e740 10 client.15900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77ae86e740 20 client.15900 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ca874c1330 0x55ca874c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77ae86e740 4 client.15900 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.267+0000 7f77ae86e740 1 -- 172.21.15.86:0/1440446990 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca8741ab00 con 0x55ca874c1330 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.268+0000 7f77a9a17640 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ca874c1330 0x55ca874c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.268+0000 7f778d7fa640 10 client.15900 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.268+0000 7f77a9a17640 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ca874c1330 0x55ca874c3790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd0102c626ba3689 server_cookie=13bc6a379ed17070 in_seq=0 out_seq=0 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f778d7fa640 1 -- 172.21.15.86:0/1440446990 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca8741ab00 con 0x55ca874c1330 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f778d7fa640 10 client.15900 handle_command_reply: tid=0 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 client.15900 shutdown 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 -- 172.21.15.86:0/1440446990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 msgr2=0x55ca874ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874ae350 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f77a000e330 tx=0x7f77a0013f70 comp rx=0 tx=0).stop 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 -- 172.21.15.86:0/1440446990 shutdown_connections 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874b1360 0x55ca874aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874a8780 0x55ca874ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ca874c1330 0x55ca874c3790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 --2- 172.21.15.86:0/1440446990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874ae350 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f778d7fa640 0 client.15900 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:35:56.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 -- 172.21.15.86:0/1440446990 >> 172.21.15.86:0/1440446990 conn(0x55ca874a2590 msgr2=0x55ca874a42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 -- 172.21.15.86:0/1440446990 shutdown_connections 2024-04-16T05:35:56.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 1 -- 172.21.15.86:0/1440446990 wait complete. 2024-04-16T05:35:56.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.269+0000 7f77ae86e740 20 client.15900 trim_cache size 0 max 16384 2024-04-16T05:35:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.282+0000 7f77ae86e740 1 Processor -- start 2024-04-16T05:35:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.282+0000 7f77ae86e740 1 -- start start 2024-04-16T05:35:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.282+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a7e60 0x55ca874ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.282+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.282+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874b1360 0x55ca874c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77ae86e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41af3f0 con 0x55ca874a8780 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77ae86e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41b0f30 con 0x55ca874b1360 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77ae86e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41afac0 con 0x55ca874a7e60 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9a17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9a17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55150/0 (socket says 172.21.15.86:55150) 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9a17640 1 -- 172.21.15.86:0/3162395515 learned_addr learned my addr 172.21.15.86:0/3162395515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9216640 1 --2- 172.21.15.86:0/3162395515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a7e60 0x55ca874ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9a17640 1 -- 172.21.15.86:0/3162395515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a7e60 msgr2=0x55ca874ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9a17640 1 --2- 172.21.15.86:0/3162395515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a7e60 0x55ca874ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9a17640 1 -- 172.21.15.86:0/3162395515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874b1360 msgr2=0x55ca874c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9a17640 1 --2- 172.21.15.86:0/3162395515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874b1360 0x55ca874c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9a17640 1 -- 172.21.15.86:0/3162395515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca874b3ac0 con 0x55ca874a8780 2024-04-16T05:35:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f77a9a17640 1 --2- 172.21.15.86:0/3162395515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874c9060 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f779800b760 tx=0x7f779800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45757d2f333b5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f778f7fe640 1 -- 172.21.15.86:0/3162395515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779802d070 con 0x55ca874a8780 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f778f7fe640 1 -- 172.21.15.86:0/3162395515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798015040 con 0x55ca874a8780 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.283+0000 7f778f7fe640 1 -- 172.21.15.86:0/3162395515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798011440 con 0x55ca874a8780 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 -- 172.21.15.86:0/3162395515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 msgr2=0x55ca874c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3162395515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874c9060 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f779800b760 tx=0x7f779800b420 comp rx=0 tx=0).stop 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 -- 172.21.15.86:0/3162395515 shutdown_connections 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3162395515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874b1360 0x55ca874c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3162395515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a8780 0x55ca874c9060 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3162395515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a7e60 0x55ca874ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 -- 172.21.15.86:0/3162395515 >> 172.21.15.86:0/3162395515 conn(0x55ca874a2590 msgr2=0x55ca874a8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 -- 172.21.15.86:0/3162395515 shutdown_connections 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 -- 172.21.15.86:0/3162395515 wait complete. 2024-04-16T05:35:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 -- start start 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 Processor -- start 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874b9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a8780 0x55ca874aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874b1360 0x55ca874ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41b0860 con 0x55ca874a7e60 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41b0190 con 0x55ca874b1360 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.284+0000 7f77ae86e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41ad3b0 con 0x55ca874a8780 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77ae86e740 10 client.0 _ll_register_callbacks cb 0x7f77a000ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9216640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9216640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874b9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55160/0 (socket says 172.21.15.86:55160) 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9216640 1 -- 172.21.15.86:0/3095014691 learned_addr learned my addr 172.21.15.86:0/3095014691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9a17640 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a8780 0x55ca874aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9216640 1 -- 172.21.15.86:0/3095014691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a8780 msgr2=0x55ca874aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9216640 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a8780 0x55ca874aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9216640 1 -- 172.21.15.86:0/3095014691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874b1360 msgr2=0x55ca874ab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9216640 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874b1360 0x55ca874ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9216640 1 -- 172.21.15.86:0/3095014691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca874ab920 con 0x55ca874a7e60 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9a17640 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a8780 0x55ca874aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77a9216640 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874b9860 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f77a0001eb0 tx=0x7f77a000e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64143f9b4a9ae78 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f778d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f778d7fa640 1 -- 172.21.15.86:0/3095014691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0011e90 con 0x55ca874a7e60 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f778d7fa640 1 -- 172.21.15.86:0/3095014691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f77a0004020 con 0x55ca874a7e60 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77ae86e740 1 -- 172.21.15.86:0/3095014691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7798015040 con 0x55ca874a7e60 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f778d7fa640 1 -- 172.21.15.86:0/3095014691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77a0004220 con 0x55ca874a7e60 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77ae86e740 10 client.15906 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f77ae86e740 1 -- 172.21.15.86:0/3095014691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca87527820 con 0x55ca874a7e60 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.285+0000 7f778d7fa640 1 -- 172.21.15.86:0/3095014691 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798015040 con 0x55ca874a7e60 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f778d7fa640 1 -- 172.21.15.86:0/3095014691 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f77a0004930 con 0x55ca874a7e60 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f77ae86e740 10 client.15906 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f77ae86e740 10 client.15906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f77ae86e740 20 client.15906 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ca87527f70 0x55ca8752a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f77ae86e740 4 client.15906 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f77ae86e740 1 -- 172.21.15.86:0/3095014691 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca87414970 con 0x55ca87527f70 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f77a9a17640 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ca87527f70 0x55ca8752a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f778d7fa640 10 client.15906 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f77a9a17640 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ca87527f70 0x55ca8752a390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fcd1d5f575284eee server_cookie=4f054b260cfd82a3 in_seq=0 out_seq=0 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f778d7fa640 1 -- 172.21.15.86:0/3095014691 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca87414970 con 0x55ca87527f70 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.286+0000 7f778d7fa640 10 client.15906 handle_command_reply: tid=0 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 client.15906 shutdown 2024-04-16T05:35:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 -- 172.21.15.86:0/3095014691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 msgr2=0x55ca874b9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874b9860 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f77a0001eb0 tx=0x7f77a000e790 comp rx=0 tx=0).stop 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 -- 172.21.15.86:0/3095014691 shutdown_connections 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca874b1360 0x55ca874ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca874a8780 0x55ca874aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ca87527f70 0x55ca8752a390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 --2- 172.21.15.86:0/3095014691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca874a7e60 0x55ca874b9860 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f778d7fa640 0 client.15906 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 -- 172.21.15.86:0/3095014691 >> 172.21.15.86:0/3095014691 conn(0x55ca874a2590 msgr2=0x55ca874ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 -- 172.21.15.86:0/3095014691 shutdown_connections 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 1 -- 172.21.15.86:0/3095014691 wait complete. 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77ae86e740 20 client.15906 trim_cache size 0 max 16384 2024-04-16T05:35:56.292 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:35:56.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 -- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77900a3e40 msgr2=0x7f77900a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77900a3e40 0x7f77900a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f779c0f04d0 tx=0x7f779c0b4080 comp rx=0 tx=0).stop 2024-04-16T05:35:56.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 -- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a419a170 msgr2=0x7f77a41a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a419a170 0x7f77a41a4720 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f779c02c700 tx=0x7f779c032f20 comp rx=0 tx=0).stop 2024-04-16T05:35:56.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 -- 172.21.15.86:0/4005293314 shutdown_connections 2024-04-16T05:35:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a419a170 0x7f77a41a4720 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4106200 0x7f77a4199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77900a3e40 0x7f77900a6300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 --2- 172.21.15.86:0/4005293314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101130 0x7f77a41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.287+0000 7f77abc7b640 1 -- 172.21.15.86:0/4005293314 >> 172.21.15.86:0/4005293314 conn(0x7f77a40780b0 msgr2=0x7f77a40fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.288+0000 7f77abc7b640 1 -- 172.21.15.86:0/4005293314 shutdown_connections 2024-04-16T05:35:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.288+0000 7f77abc7b640 1 -- 172.21.15.86:0/4005293314 wait complete. 2024-04-16T05:35:56.297 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:35:56.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.404+0000 7f7d5558f640 1 -- 172.21.15.86:0/3356227572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50102220 msgr2=0x7f7d50102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.404+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3356227572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50102220 0x7f7d50102600 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009ca0 tx=0x7f7d4002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 -- 172.21.15.86:0/3356227572 shutdown_connections 2024-04-16T05:35:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3356227572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d501072f0 0x7f7d5010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3356227572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d50102bd0 0x7f7d50106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3356227572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50102220 0x7f7d50102600 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 -- 172.21.15.86:0/3356227572 >> 172.21.15.86:0/3356227572 conn(0x7f7d500fdac0 msgr2=0x7f7d500ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 -- 172.21.15.86:0/3356227572 shutdown_connections 2024-04-16T05:35:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 -- 172.21.15.86:0/3356227572 wait complete. 2024-04-16T05:35:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 Processor -- start 2024-04-16T05:35:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 -- start start 2024-04-16T05:35:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d50102220 0x7f7d50074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d501072f0 0x7f7d50074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d50074d80 0x7f7d501a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50112380 con 0x7f7d501072f0 2024-04-16T05:35:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.405+0000 7f7d5558f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50112200 con 0x7f7d50102220 2024-04-16T05:35:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d5558f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50112500 con 0x7f7d50074d80 2024-04-16T05:35:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d501072f0 0x7f7d50074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d501072f0 0x7f7d50074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55184/0 (socket says 172.21.15.86:55184) 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4e7fc640 1 -- 172.21.15.86:0/3889584497 learned_addr learned my addr 172.21.15.86:0/3889584497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4effd640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d50102220 0x7f7d50074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d50074d80 0x7f7d501a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4e7fc640 1 -- 172.21.15.86:0/3889584497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d50074d80 msgr2=0x7f7d501a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d50074d80 0x7f7d501a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4e7fc640 1 -- 172.21.15.86:0/3889584497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d50102220 msgr2=0x7f7d50074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d50102220 0x7f7d50074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4e7fc640 1 -- 172.21.15.86:0/3889584497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501a70d0 con 0x7f7d501072f0 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d50074d80 0x7f7d501a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d501072f0 0x7f7d50074840 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c970 tx=0x7f7d3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d47fff640 1 -- 172.21.15.86:0/3889584497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38014070 con 0x7f7d501072f0 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d47fff640 1 -- 172.21.15.86:0/3889584497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d38004440 con 0x7f7d501072f0 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d5558f640 1 -- 172.21.15.86:0/3889584497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d501a73c0 con 0x7f7d501072f0 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d47fff640 1 -- 172.21.15.86:0/3889584497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800f450 con 0x7f7d501072f0 2024-04-16T05:35:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.406+0000 7f7d5558f640 1 -- 172.21.15.86:0/3889584497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d501a7910 con 0x7f7d501072f0 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.409+0000 7f7d47fff640 1 -- 172.21.15.86:0/3889584497 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7d38017070 con 0x7f7d501072f0 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.409+0000 7f7d5817b740 1 Processor -- start 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.409+0000 7f7d5817b740 1 -- start start 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.409+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 0x55979c67f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.409+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 0x55979c687e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.409+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c68a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.409+0000 7f7d5817b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55979c5e40e0 con 0x55979c688360 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.409+0000 7f7d5817b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55979c581520 con 0x55979c67ee60 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.409+0000 7f7d5817b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55979c5e00a0 con 0x55979c67f780 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d47fff640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7d3c0a3e70 0x7f7d3c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c68a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c68a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55192/0 (socket says 172.21.15.86:55192) 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4e7fc640 1 -- 172.21.15.86:0/834632496 learned_addr learned my addr 172.21.15.86:0/834632496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7d3c0a3e70 0x7f7d3c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4effd640 1 --2- 172.21.15.86:0/834632496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 0x55979c687e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/834632496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 0x55979c67f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d47fff640 1 -- 172.21.15.86:0/3889584497 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7d380d0940 con 0x7f7d501072f0 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4e7fc640 1 -- 172.21.15.86:0/834632496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 msgr2=0x55979c687e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/834632496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 0x55979c687e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4e7fc640 1 -- 172.21.15.86:0/834632496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 msgr2=0x55979c67f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/834632496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 0x55979c67f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4e7fc640 1 -- 172.21.15.86:0/834632496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979c5a98d0 con 0x55979c688360 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7d3c0a3e70 0x7f7d3c0a6330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d480058c0 tx=0x7f7d4800c7a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/834632496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c68a750 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7d380cdb40 tx=0x7f7d38003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fe717c2f9fec1d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d457fa640 1 -- 172.21.15.86:0/834632496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d380a06d0 con 0x55979c688360 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d457fa640 1 -- 172.21.15.86:0/834632496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d3809c320 con 0x55979c688360 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.410+0000 7f7d457fa640 1 -- 172.21.15.86:0/834632496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d380cde80 con 0x55979c688360 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.411+0000 7f7d5817b740 1 -- 172.21.15.86:0/834632496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 msgr2=0x55979c68a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.411+0000 7f7d5817b740 1 --2- 172.21.15.86:0/834632496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c68a750 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7d380cdb40 tx=0x7f7d38003920 comp rx=0 tx=0).stop 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.411+0000 7f7d5817b740 1 -- 172.21.15.86:0/834632496 shutdown_connections 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.411+0000 7f7d5817b740 1 --2- 172.21.15.86:0/834632496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c68a750 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.411+0000 7f7d5817b740 1 --2- 172.21.15.86:0/834632496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 0x55979c687e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.411+0000 7f7d5817b740 1 --2- 172.21.15.86:0/834632496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 0x55979c67f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.411+0000 7f7d5817b740 1 -- 172.21.15.86:0/834632496 >> 172.21.15.86:0/834632496 conn(0x55979c679590 msgr2=0x55979c679970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.411+0000 7f7d5817b740 1 -- 172.21.15.86:0/834632496 shutdown_connections 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.411+0000 7f7d5817b740 1 -- 172.21.15.86:0/834632496 wait complete. 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d5817b740 1 -- start start 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d5817b740 1 Processor -- start 2024-04-16T05:35:56.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67ee60 0x55979c685410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 0x55979c685950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c685e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d5817b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55979c68afa0 con 0x55979c688360 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d5817b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55979c59d4d0 con 0x55979c67f780 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d5817b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55979c68b120 con 0x55979c67ee60 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d5817b740 10 client.0 _ll_register_callbacks cb 0x55979c5822c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c685e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c685e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55202/0 (socket says 172.21.15.86:55202) 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4f7fe640 1 -- 172.21.15.86:0/497476610 learned_addr learned my addr 172.21.15.86:0/497476610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 0x55979c685950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4effd640 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67ee60 0x55979c685410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4f7fe640 1 -- 172.21.15.86:0/497476610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67ee60 msgr2=0x55979c685410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67ee60 0x55979c685410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4f7fe640 1 -- 172.21.15.86:0/497476610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 msgr2=0x55979c685950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 0x55979c685950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4f7fe640 1 -- 172.21.15.86:0/497476610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55979c2f63f0 con 0x55979c688360 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c685e90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7d48011a50 tx=0x7f7d48005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7306753c0ddd5593 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.412+0000 7f7d367fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d367fc640 1 -- 172.21.15.86:0/497476610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48024070 con 0x55979c688360 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d367fc640 1 -- 172.21.15.86:0/497476610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7d4800fd60 con 0x55979c688360 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d5817b740 1 -- 172.21.15.86:0/497476610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d3809c320 con 0x55979c688360 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d367fc640 1 -- 172.21.15.86:0/497476610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d4802f050 con 0x55979c688360 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d5817b740 10 client.15918 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d5817b740 1 -- 172.21.15.86:0/497476610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55979c3c07d0 con 0x55979c688360 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d367fc640 1 -- 172.21.15.86:0/497476610 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d3809c320 con 0x55979c688360 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d367fc640 1 -- 172.21.15.86:0/497476610 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d4801e500 con 0x55979c688360 2024-04-16T05:35:56.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d5817b740 10 client.15918 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d5817b740 10 client.15918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d5817b740 20 client.15918 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d5817b740 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55979c6982e0 0x55979c69a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d5817b740 4 client.15918 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.413+0000 7f7d5817b740 1 -- 172.21.15.86:0/497476610 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55979c5f1b00 con 0x55979c6982e0 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.414+0000 7f7d4effd640 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55979c6982e0 0x55979c69a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.414+0000 7f7d367fc640 10 client.15918 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.414+0000 7f7d4effd640 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55979c6982e0 0x55979c69a720 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61e3be90d65ab3ec server_cookie=baab1571817013f9 in_seq=0 out_seq=0 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d367fc640 1 -- 172.21.15.86:0/497476610 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55979c5f1b00 con 0x55979c6982e0 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d367fc640 10 client.15918 handle_command_reply: tid=0 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 client.15918 shutdown 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 -- 172.21.15.86:0/497476610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 msgr2=0x55979c685e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c685e90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7d48011a50 tx=0x7f7d48005850 comp rx=0 tx=0).stop 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 -- 172.21.15.86:0/497476610 shutdown_connections 2024-04-16T05:35:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55979c6982e0 0x55979c69a720 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c685e90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 0x55979c685950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 --2- 172.21.15.86:0/497476610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67ee60 0x55979c685410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d367fc640 0 client.15918 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:35:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 -- 172.21.15.86:0/497476610 >> 172.21.15.86:0/497476610 conn(0x55979c679590 msgr2=0x55979c67b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 -- 172.21.15.86:0/497476610 shutdown_connections 2024-04-16T05:35:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 1 -- 172.21.15.86:0/497476610 wait complete. 2024-04-16T05:35:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.415+0000 7f7d5817b740 20 client.15918 trim_cache size 0 max 16384 2024-04-16T05:35:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.428+0000 7f7d5817b740 1 Processor -- start 2024-04-16T05:35:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.428+0000 7f7d5817b740 1 -- start start 2024-04-16T05:35:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 0x55979c693f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 0x55979c684d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c687240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d5817b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b1620 con 0x55979c688360 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d5817b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b3160 con 0x55979c67ee60 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d5817b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b1cf0 con 0x55979c67f780 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c687240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c687240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55210/0 (socket says 172.21.15.86:55210) 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4f7fe640 1 -- 172.21.15.86:0/2922024531 learned_addr learned my addr 172.21.15.86:0/2922024531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4effd640 1 --2- 172.21.15.86:0/2922024531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 0x55979c693f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/2922024531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 0x55979c684d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4f7fe640 1 -- 172.21.15.86:0/2922024531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 msgr2=0x55979c684d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/2922024531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 0x55979c684d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4f7fe640 1 -- 172.21.15.86:0/2922024531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 msgr2=0x55979c693f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/2922024531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 0x55979c693f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4f7fe640 1 -- 172.21.15.86:0/2922024531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979c681030 con 0x55979c688360 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/2922024531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c687240 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7d48006010 tx=0x7f7d4800c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8862fcba1425180 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d44ff9640 1 -- 172.21.15.86:0/2922024531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48038070 con 0x55979c688360 2024-04-16T05:35:56.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d44ff9640 1 -- 172.21.15.86:0/2922024531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4802c030 con 0x55979c688360 2024-04-16T05:35:56.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.429+0000 7f7d44ff9640 1 -- 172.21.15.86:0/2922024531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48024050 con 0x55979c688360 2024-04-16T05:35:56.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 -- 172.21.15.86:0/2922024531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 msgr2=0x55979c687240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 --2- 172.21.15.86:0/2922024531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c687240 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7d48006010 tx=0x7f7d4800c670 comp rx=0 tx=0).stop 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 -- 172.21.15.86:0/2922024531 shutdown_connections 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 --2- 172.21.15.86:0/2922024531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c688360 0x55979c687240 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 --2- 172.21.15.86:0/2922024531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c67f780 0x55979c684d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 --2- 172.21.15.86:0/2922024531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67ee60 0x55979c693f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 -- 172.21.15.86:0/2922024531 >> 172.21.15.86:0/2922024531 conn(0x55979c679590 msgr2=0x55979c686b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 -- 172.21.15.86:0/2922024531 shutdown_connections 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 -- 172.21.15.86:0/2922024531 wait complete. 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.430+0000 7f7d5817b740 1 -- start start 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d5817b740 1 Processor -- start 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c67ee60 0x55979c690690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 0x55979c681c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d5817b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c688360 0x55979c682180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d5817b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b2a90 con 0x55979c67ee60 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d5817b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b23c0 con 0x55979c67f780 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d5817b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501af5e0 con 0x55979c688360 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d5817b740 10 client.0 _ll_register_callbacks cb 0x7f7d4800cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c67ee60 0x55979c690690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c67ee60 0x55979c690690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55226/0 (socket says 172.21.15.86:55226) 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4effd640 1 -- 172.21.15.86:0/1924977096 learned_addr learned my addr 172.21.15.86:0/1924977096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 0x55979c681c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4effd640 1 -- 172.21.15.86:0/1924977096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c688360 msgr2=0x55979c682180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c688360 0x55979c682180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4effd640 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c688360 0x55979c682180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4effd640 1 -- 172.21.15.86:0/1924977096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 msgr2=0x55979c681c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4effd640 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 0x55979c681c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4effd640 1 -- 172.21.15.86:0/1924977096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55979c6826c0 con 0x55979c67ee60 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 0x55979c681c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4f7fe640 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c688360 0x55979c682180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d4effd640 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c67ee60 0x55979c690690 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7d40004af0 tx=0x7f7d40005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ce6138c76bf61 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d367fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d367fc640 1 -- 172.21.15.86:0/1924977096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40039040 con 0x55979c67ee60 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d367fc640 1 -- 172.21.15.86:0/1924977096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7d40048d50 con 0x55979c67ee60 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.431+0000 7f7d5817b740 1 -- 172.21.15.86:0/1924977096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d4802c030 con 0x55979c67ee60 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d367fc640 1 -- 172.21.15.86:0/1924977096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d40057050 con 0x55979c67ee60 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d5817b740 10 client.15924 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d5817b740 1 -- 172.21.15.86:0/1924977096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55979c6fe5e0 con 0x55979c67ee60 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d367fc640 1 -- 172.21.15.86:0/1924977096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4802c030 con 0x55979c67ee60 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d367fc640 1 -- 172.21.15.86:0/1924977096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d400481d0 con 0x55979c67ee60 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d5817b740 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d5817b740 10 client.15924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d5817b740 20 client.15924 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d5817b740 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55979c6fed30 0x55979c701170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d5817b740 4 client.15924 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d5817b740 1 -- 172.21.15.86:0/1924977096 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55979c5eb970 con 0x55979c6fed30 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.432+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55979c6fed30 0x55979c701170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d367fc640 10 client.15924 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d4e7fc640 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55979c6fed30 0x55979c701170 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8accd51cf0820eb server_cookie=a17acf58859cebe9 in_seq=0 out_seq=0 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d367fc640 1 -- 172.21.15.86:0/1924977096 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55979c5eb970 con 0x55979c6fed30 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d367fc640 10 client.15924 handle_command_reply: tid=0 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 client.15924 shutdown 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 -- 172.21.15.86:0/1924977096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c67ee60 msgr2=0x55979c690690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c67ee60 0x55979c690690 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7d40004af0 tx=0x7f7d40005c50 comp rx=0 tx=0).stop 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 -- 172.21.15.86:0/1924977096 shutdown_connections 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55979c6fed30 0x55979c701170 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55979c688360 0x55979c682180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d367fc640 0 client.15924 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979c67f780 0x55979c681c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 --2- 172.21.15.86:0/1924977096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979c67ee60 0x55979c690690 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 -- 172.21.15.86:0/1924977096 >> 172.21.15.86:0/1924977096 conn(0x55979c679590 msgr2=0x55979c678730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 -- 172.21.15.86:0/1924977096 shutdown_connections 2024-04-16T05:35:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 1 -- 172.21.15.86:0/1924977096 wait complete. 2024-04-16T05:35:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.433+0000 7f7d5817b740 20 client.15924 trim_cache size 0 max 16384 2024-04-16T05:35:56.438 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 -- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7d3c0a3e70 msgr2=0x7f7d3c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7d3c0a3e70 0x7f7d3c0a6330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d480058c0 tx=0x7f7d4800c7a0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 -- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d501072f0 msgr2=0x7f7d50074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d501072f0 0x7f7d50074840 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c970 tx=0x7f7d3800ce30 comp rx=0 tx=0).stop 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 -- 172.21.15.86:0/3889584497 shutdown_connections 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d50074d80 0x7f7d501a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d501072f0 0x7f7d50074840 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7d3c0a3e70 0x7f7d3c0a6330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 --2- 172.21.15.86:0/3889584497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d50102220 0x7f7d50074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 -- 172.21.15.86:0/3889584497 >> 172.21.15.86:0/3889584497 conn(0x7f7d500fdac0 msgr2=0x7f7d500fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 -- 172.21.15.86:0/3889584497 shutdown_connections 2024-04-16T05:35:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.434+0000 7f7d5558f640 1 -- 172.21.15.86:0/3889584497 wait complete. 2024-04-16T05:35:56.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 -- 172.21.15.86:0/2528218468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0101120 msgr2=0x7fa5d0106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 --2- 172.21.15.86:0/2528218468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0101120 0x7fa5d0106c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009900 tx=0x7fa5b802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 -- 172.21.15.86:0/2528218468 shutdown_connections 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 --2- 172.21.15.86:0/2528218468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5d0107330 0x7fa5d010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 --2- 172.21.15.86:0/2528218468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0101120 0x7fa5d0106c50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 --2- 172.21.15.86:0/2528218468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5d0100770 0x7fa5d0100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 -- 172.21.15.86:0/2528218468 >> 172.21.15.86:0/2528218468 conn(0x7fa5d0077450 msgr2=0x7fa5d00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 -- 172.21.15.86:0/2528218468 shutdown_connections 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 -- 172.21.15.86:0/2528218468 wait complete. 2024-04-16T05:35:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.551+0000 7fa5d5959640 1 Processor -- start 2024-04-16T05:35:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d5959640 1 -- start start 2024-04-16T05:35:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d5959640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0100770 0x7fa5d01a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d5959640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5d0107330 0x7fa5d01a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d5959640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5d019a940 0x7fa5d019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d5959640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01123c0 con 0x7fa5d0100770 2024-04-16T05:35:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d5959640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0112240 con 0x7fa5d019a940 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d5959640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0112540 con 0x7fa5d0107330 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ceffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0100770 0x7fa5d01a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ceffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0100770 0x7fa5d01a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55248/0 (socket says 172.21.15.86:55248) 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ceffd640 1 -- 172.21.15.86:0/1200317456 learned_addr learned my addr 172.21.15.86:0/1200317456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5d019a940 0x7fa5d019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ce7fc640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5d0107330 0x7fa5d01a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ceffd640 1 -- 172.21.15.86:0/1200317456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5d0107330 msgr2=0x7fa5d01a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5d0107330 0x7fa5d01a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ceffd640 1 -- 172.21.15.86:0/1200317456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5d019a940 msgr2=0x7fa5d019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5d019a940 0x7fa5d019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ceffd640 1 -- 172.21.15.86:0/1200317456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d019b460 con 0x7fa5d0100770 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0100770 0x7fa5d01a0790 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800cce0 tx=0x7fa5c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d4957640 1 -- 172.21.15.86:0/1200317456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8004b20 con 0x7fa5d0100770 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d4957640 1 -- 172.21.15.86:0/1200317456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c8004cc0 con 0x7fa5d0100770 2024-04-16T05:35:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.552+0000 7fa5d4957640 1 -- 172.21.15.86:0/1200317456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c800f8c0 con 0x7fa5d0100770 2024-04-16T05:35:56.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.553+0000 7fa5d5959640 1 -- 172.21.15.86:0/1200317456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d019b750 con 0x7fa5d0100770 2024-04-16T05:35:56.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.553+0000 7fa5d5959640 1 -- 172.21.15.86:0/1200317456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d01a7950 con 0x7fa5d0100770 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.555+0000 7fa5d4957640 1 -- 172.21.15.86:0/1200317456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa5c800fa60 con 0x7fa5d0100770 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.555+0000 7fa5d8624740 1 Processor -- start 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.555+0000 7fa5d8624740 1 -- start start 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.555+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.555+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087ece20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.555+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 0x55fe087ef750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.555+0000 7fa5d8624740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe087490e0 con 0x55fe087e3e60 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.555+0000 7fa5d8624740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fe086e6520 con 0x55fe087ed360 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.555+0000 7fa5d8624740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fe087450a0 con 0x55fe087e4780 2024-04-16T05:35:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087e4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55258/0 (socket says 172.21.15.86:55258) 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/1870056172 learned_addr learned my addr 172.21.15.86:0/1870056172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5d4957640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5bc0a3e40 0x7fa5bc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087ece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5bc0a3e40 0x7fa5bc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5d4957640 1 -- 172.21.15.86:0/1200317456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa5c80d0f60 con 0x7fa5d0100770 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/1870056172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 msgr2=0x55fe087ece20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087ece20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5ce7fc640 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 0x55fe087ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/1870056172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 msgr2=0x55fe087ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 0x55fe087ef750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/1870056172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0870e8d0 con 0x55fe087e3e60 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087ece20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087e4240 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000bd20 tx=0x7fa5c00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58ae1d7cbba3bf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5c5ffb640 1 -- 172.21.15.86:0/1870056172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000b740 con 0x55fe087e3e60 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5c5ffb640 1 -- 172.21.15.86:0/1870056172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c0005570 con 0x55fe087e3e60 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5c5ffb640 1 -- 172.21.15.86:0/1870056172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c001ee20 con 0x55fe087e3e60 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.556+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5bc0a3e40 0x7fa5bc0a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0021000 tx=0x7fa5c0020000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 -- 172.21.15.86:0/1870056172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 msgr2=0x55fe087e4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087e4240 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000bd20 tx=0x7fa5c00039c0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 -- 172.21.15.86:0/1870056172 shutdown_connections 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 0x55fe087ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087ece20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1870056172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087e4240 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 -- 172.21.15.86:0/1870056172 >> 172.21.15.86:0/1870056172 conn(0x55fe087de590 msgr2=0x55fe087de970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 -- 172.21.15.86:0/1870056172 shutdown_connections 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 -- 172.21.15.86:0/1870056172 wait complete. 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.557+0000 7fa5d8624740 1 -- start start 2024-04-16T05:35:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5d8624740 1 Processor -- start 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087ea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087ea9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 0x55fe087eaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5d8624740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe087effa0 con 0x55fe087e3e60 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5d8624740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fe087024d0 con 0x55fe087ed360 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5d8624740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fe087f0120 con 0x55fe087e4780 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5d8624740 10 client.0 _ll_register_callbacks cb 0x55fe086e72c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ceffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ceffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087ea480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55270/0 (socket says 172.21.15.86:55270) 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ceffd640 1 -- 172.21.15.86:0/3840551090 learned_addr learned my addr 172.21.15.86:0/3840551090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 0x55fe087eaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ce7fc640 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087ea9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ceffd640 1 -- 172.21.15.86:0/3840551090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 msgr2=0x55fe087ea9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087ea9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ceffd640 1 -- 172.21.15.86:0/3840551090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 msgr2=0x55fe087eaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 0x55fe087eaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ceffd640 1 -- 172.21.15.86:0/3840551090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0845b3f0 con 0x55fe087e3e60 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087ea480 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800ab20 tx=0x7fa5c801d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c2b355040d0be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.558+0000 7fa5aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5aeffd640 1 -- 172.21.15.86:0/3840551090 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c801d040 con 0x55fe087e3e60 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5aeffd640 1 -- 172.21.15.86:0/3840551090 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa5c8014d00 con 0x55fe087e3e60 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5d8624740 1 -- 172.21.15.86:0/3840551090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5c0005570 con 0x55fe087e3e60 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5aeffd640 1 -- 172.21.15.86:0/3840551090 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5c809d000 con 0x55fe087e3e60 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5d8624740 10 client.15936 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5d8624740 1 -- 172.21.15.86:0/3840551090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe085257d0 con 0x55fe087e3e60 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5aeffd640 1 -- 172.21.15.86:0/3840551090 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c0005570 con 0x55fe087e3e60 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5aeffd640 1 -- 172.21.15.86:0/3840551090 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa5c8012110 con 0x55fe087e3e60 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5d8624740 10 client.15936 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5d8624740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5d8624740 20 client.15936 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fe087fd2a0 0x55fe087ff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5d8624740 4 client.15936 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5d8624740 1 -- 172.21.15.86:0/3840551090 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe08756b00 con 0x55fe087fd2a0 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.559+0000 7fa5ce7fc640 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fe087fd2a0 0x55fe087ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.560+0000 7fa5aeffd640 10 client.15936 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.560+0000 7fa5ce7fc640 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fe087fd2a0 0x55fe087ff690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0ba86ab28ddec3 server_cookie=2cbfb7ff8a939f8e in_seq=0 out_seq=0 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.560+0000 7fa5aeffd640 1 -- 172.21.15.86:0/3840551090 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe08756b00 con 0x55fe087fd2a0 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.560+0000 7fa5aeffd640 10 client.15936 handle_command_reply: tid=0 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 client.15936 shutdown 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 -- 172.21.15.86:0/3840551090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 msgr2=0x55fe087ea480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087ea480 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800ab20 tx=0x7fa5c801d9d0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 -- 172.21.15.86:0/3840551090 shutdown_connections 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fe087fd2a0 0x55fe087ff690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087ed360 0x55fe087eaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087ea9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5aeffd640 0 client.15936 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3840551090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087e3e60 0x55fe087ea480 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 -- 172.21.15.86:0/3840551090 >> 172.21.15.86:0/3840551090 conn(0x55fe087de590 msgr2=0x55fe087e02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 -- 172.21.15.86:0/3840551090 shutdown_connections 2024-04-16T05:35:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 1 -- 172.21.15.86:0/3840551090 wait complete. 2024-04-16T05:35:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.561+0000 7fa5d8624740 20 client.15936 trim_cache size 0 max 16384 2024-04-16T05:35:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5d8624740 1 Processor -- start 2024-04-16T05:35:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5d8624740 1 -- start start 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087e3e60 0x55fe087f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5d8624740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01b16c0 con 0x55fe087ed360 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5d8624740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01b3200 con 0x55fe087e3e60 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5d8624740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01b1d90 con 0x55fe087e4780 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087dcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55274/0 (socket says 172.21.15.86:55274) 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/3744169455 learned_addr learned my addr 172.21.15.86:0/3744169455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/3744169455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 msgr2=0x55fe087e9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/3744169455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/3744169455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087e3e60 msgr2=0x55fe087f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/3744169455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087e3e60 0x55fe087f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.574+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/3744169455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe087ffa00 con 0x55fe087ed360 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/3744169455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087dcea0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00042c0 tx=0x7fa5c0004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6549283fb8438e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5c57fa640 1 -- 172.21.15.86:0/3744169455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0005570 con 0x55fe087ed360 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5c57fa640 1 -- 172.21.15.86:0/3744169455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c000b740 con 0x55fe087ed360 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5c57fa640 1 -- 172.21.15.86:0/3744169455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0012ba0 con 0x55fe087ed360 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5d8624740 1 -- 172.21.15.86:0/3744169455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 msgr2=0x55fe087dcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3744169455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087dcea0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00042c0 tx=0x7fa5c0004680 comp rx=0 tx=0).stop 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5d8624740 1 -- 172.21.15.86:0/3744169455 shutdown_connections 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3744169455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087dcea0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3744169455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087e9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5d8624740 1 --2- 172.21.15.86:0/3744169455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087e3e60 0x55fe087f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5d8624740 1 -- 172.21.15.86:0/3744169455 >> 172.21.15.86:0/3744169455 conn(0x55fe087de590 msgr2=0x55fe087eacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5d8624740 1 -- 172.21.15.86:0/3744169455 shutdown_connections 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.575+0000 7fa5d8624740 1 -- 172.21.15.86:0/3744169455 wait complete. 2024-04-16T05:35:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5d8624740 1 -- start start 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5d8624740 1 Processor -- start 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087e3e60 0x55fe087e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087e7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5d8624740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5d8624740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01b2b30 con 0x55fe087ed360 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5d8624740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01b2460 con 0x55fe087e3e60 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5d8624740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01af680 con 0x55fe087e4780 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5d8624740 10 client.0 _ll_register_callbacks cb 0x7fa5c800af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087e7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55288/0 (socket says 172.21.15.86:55288) 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/1808232214 learned_addr learned my addr 172.21.15.86:0/1808232214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/1808232214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 msgr2=0x55fe087e7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087e7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/1808232214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087e3e60 msgr2=0x55fe087e73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087e3e60 0x55fe087e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.576+0000 7fa5cf7fe640 1 -- 172.21.15.86:0/1808232214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe087e45b0 con 0x55fe087ed360 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5cf7fe640 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087e7e40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0003a00 tx=0x7fa5c000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1871672b98936fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5aeffd640 1 -- 172.21.15.86:0/1808232214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0005630 con 0x55fe087ed360 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5aeffd640 1 -- 172.21.15.86:0/1808232214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa5c0023040 con 0x55fe087ed360 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5d8624740 1 -- 172.21.15.86:0/1808232214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5c000b740 con 0x55fe087ed360 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5aeffd640 1 -- 172.21.15.86:0/1808232214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5c00125d0 con 0x55fe087ed360 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5d8624740 10 client.15942 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5d8624740 1 -- 172.21.15.86:0/1808232214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe088639f0 con 0x55fe087ed360 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5aeffd640 1 -- 172.21.15.86:0/1808232214 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c000b740 con 0x55fe087ed360 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5aeffd640 1 -- 172.21.15.86:0/1808232214 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa5c0028450 con 0x55fe087ed360 2024-04-16T05:35:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5d8624740 10 client.15942 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5d8624740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5d8624740 20 client.15942 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fe08863eb0 0x55fe088662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5d8624740 4 client.15942 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.577+0000 7fa5d8624740 1 -- 172.21.15.86:0/1808232214 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe08750970 con 0x55fe08863eb0 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fe08863eb0 0x55fe088662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5aeffd640 10 client.15942 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5ceffd640 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fe08863eb0 0x55fe088662c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=169b39f477873704 server_cookie=d84eceecdaae85e0 in_seq=0 out_seq=0 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5aeffd640 1 -- 172.21.15.86:0/1808232214 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe08750970 con 0x55fe08863eb0 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5aeffd640 10 client.15942 handle_command_reply: tid=0 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 client.15942 shutdown 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 -- 172.21.15.86:0/1808232214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 msgr2=0x55fe087e7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087e7e40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0003a00 tx=0x7fa5c000b8f0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 -- 172.21.15.86:0/1808232214 shutdown_connections 2024-04-16T05:35:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fe08863eb0 0x55fe088662c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe087ed360 0x55fe087e7e40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe087e4780 0x55fe087e7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5aeffd640 0 client.15942 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 --2- 172.21.15.86:0/1808232214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe087e3e60 0x55fe087e73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 -- 172.21.15.86:0/1808232214 >> 172.21.15.86:0/1808232214 conn(0x55fe087de590 msgr2=0x55fe087df870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 -- 172.21.15.86:0/1808232214 shutdown_connections 2024-04-16T05:35:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.578+0000 7fa5d8624740 1 -- 172.21.15.86:0/1808232214 wait complete. 2024-04-16T05:35:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d8624740 20 client.15942 trim_cache size 0 max 16384 2024-04-16T05:35:56.583 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:35:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 -- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5bc0a3e40 msgr2=0x7fa5bc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5bc0a3e40 0x7fa5bc0a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0021000 tx=0x7fa5c0020000 comp rx=0 tx=0).stop 2024-04-16T05:35:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 -- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0100770 msgr2=0x7fa5d01a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0100770 0x7fa5d01a0790 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800cce0 tx=0x7fa5c800a620 comp rx=0 tx=0).stop 2024-04-16T05:35:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 -- 172.21.15.86:0/1200317456 shutdown_connections 2024-04-16T05:35:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa5bc0a3e40 0x7fa5bc0a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5d019a940 0x7fa5d019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5d0107330 0x7fa5d01a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 --2- 172.21.15.86:0/1200317456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5d0100770 0x7fa5d01a0790 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 -- 172.21.15.86:0/1200317456 >> 172.21.15.86:0/1200317456 conn(0x7fa5d0077450 msgr2=0x7fa5d01057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 -- 172.21.15.86:0/1200317456 shutdown_connections 2024-04-16T05:35:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:56.579+0000 7fa5d5959640 1 -- 172.21.15.86:0/1200317456 wait complete. 2024-04-16T05:35:56.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:56 smithi086 ceph-mon[32028]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 17 op/s 2024-04-16T05:35:56.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:56 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4232641439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:56 smithi196 ceph-mon[36373]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 17 op/s 2024-04-16T05:35:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:56 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4232641439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:57.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:56 smithi194 ceph-mon[35865]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 17 op/s 2024-04-16T05:35:57.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:56 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4232641439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:57.589 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:35:57.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:57.589 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:35:57.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 -- 172.21.15.86:0/674569832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100500 msgr2=0x7f72f0100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 --2- 172.21.15.86:0/674569832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100500 0x7f72f0100980 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009930 tx=0x7f72e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:57.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 -- 172.21.15.86:0/674569832 shutdown_connections 2024-04-16T05:35:57.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 --2- 172.21.15.86:0/674569832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0100ec0 0x7f72f010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 --2- 172.21.15.86:0/674569832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100500 0x7f72f0100980 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 --2- 172.21.15.86:0/674569832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72f0106fe0 0x7f72f01073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 -- 172.21.15.86:0/674569832 >> 172.21.15.86:0/674569832 conn(0x7f72f0077490 msgr2=0x7f72f00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 -- 172.21.15.86:0/674569832 shutdown_connections 2024-04-16T05:35:57.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 -- 172.21.15.86:0/674569832 wait complete. 2024-04-16T05:35:57.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.710+0000 7f72f51f0640 1 Processor -- start 2024-04-16T05:35:57.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72f51f0640 1 -- start start 2024-04-16T05:35:57.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72f51f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0100500 0x7f72f019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72f51f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100ec0 0x7f72f019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72f51f0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72f0106fe0 0x7f72f0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72f51f0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0112510 con 0x7f72f0100ec0 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72f51f0640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0112390 con 0x7f72f0106fe0 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72f51f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0112690 con 0x7f72f0100500 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ee59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100ec0 0x7f72f019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ee59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100ec0 0x7f72f019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55310/0 (socket says 172.21.15.86:55310) 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ee59c640 1 -- 172.21.15.86:0/1445116752 learned_addr learned my addr 172.21.15.86:0/1445116752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72eed9d640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0100500 0x7f72f019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72f0106fe0 0x7f72f0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ee59c640 1 -- 172.21.15.86:0/1445116752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0100500 msgr2=0x7f72f019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ee59c640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0100500 0x7f72f019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ee59c640 1 -- 172.21.15.86:0/1445116752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72f0106fe0 msgr2=0x7f72f0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ee59c640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72f0106fe0 0x7f72f0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ee59c640 1 -- 172.21.15.86:0/1445116752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f0198fe0 con 0x7f72f0100ec0 2024-04-16T05:35:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72eed9d640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0100500 0x7f72f019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72f0106fe0 0x7f72f0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.711+0000 7f72ee59c640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100ec0 0x7f72f019ebe0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009230 tx=0x7f72e402cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.712+0000 7f72d7fff640 1 -- 172.21.15.86:0/1445116752 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4041070 con 0x7f72f0100ec0 2024-04-16T05:35:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.712+0000 7f72d7fff640 1 -- 172.21.15.86:0/1445116752 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e4004450 con 0x7f72f0100ec0 2024-04-16T05:35:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.712+0000 7f72f51f0640 1 -- 172.21.15.86:0/1445116752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f0199270 con 0x7f72f0100ec0 2024-04-16T05:35:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.712+0000 7f72d7fff640 1 -- 172.21.15.86:0/1445116752 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4034440 con 0x7f72f0100ec0 2024-04-16T05:35:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.712+0000 7f72f51f0640 1 -- 172.21.15.86:0/1445116752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f01a54f0 con 0x7f72f0100ec0 2024-04-16T05:35:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.714+0000 7f72d7fff640 1 -- 172.21.15.86:0/1445116752 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f72e4044070 con 0x7f72f0100ec0 2024-04-16T05:35:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.714+0000 7f72f7dd8740 1 Processor -- start 2024-04-16T05:35:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.714+0000 7f72f7dd8740 1 -- start start 2024-04-16T05:35:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.714+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d237ab0 0x56071d237e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.714+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d2383d0 0x56071d240a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.714+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d240fb0 0x56071d2433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56071d19ce90 con 0x56071d2383d0 2024-04-16T05:35:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56071d13a4c0 con 0x56071d237ab0 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56071d198e50 con 0x56071d240fb0 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72eed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d2383d0 0x56071d240a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72eed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d2383d0 0x56071d240a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55312/0 (socket says 172.21.15.86:55312) 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72eed9d640 1 -- 172.21.15.86:0/3418305057 learned_addr learned my addr 172.21.15.86:0/3418305057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72d7fff640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f72d80a3e40 0x7f72d80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72ee59c640 1 --2- 172.21.15.86:0/3418305057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d240fb0 0x56071d2433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f72d80a3e40 0x7f72d80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72eed9d640 1 -- 172.21.15.86:0/3418305057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d240fb0 msgr2=0x56071d2433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72eed9d640 1 --2- 172.21.15.86:0/3418305057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d240fb0 0x56071d2433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72eed9d640 1 -- 172.21.15.86:0/3418305057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d237ab0 msgr2=0x56071d237e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72eed9d640 1 --2- 172.21.15.86:0/3418305057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d237ab0 0x56071d237e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72eed9d640 1 -- 172.21.15.86:0/3418305057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56071cf777e0 con 0x56071d2383d0 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72ee59c640 1 --2- 172.21.15.86:0/3418305057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d240fb0 0x56071d2433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72d7fff640 1 -- 172.21.15.86:0/1445116752 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f72e40f2880 con 0x7f72f0100ec0 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72eed9d640 1 --2- 172.21.15.86:0/3418305057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d2383d0 0x56071d240a70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f72e0007850 tx=0x7f72e0007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de24567fcec7f55a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72d57fa640 1 -- 172.21.15.86:0/3418305057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e000d950 con 0x56071d2383d0 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f72d80a3e40 0x7f72d80a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f72e8005c40 tx=0x7f72e80098e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72d57fa640 1 -- 172.21.15.86:0/3418305057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e0018070 con 0x56071d2383d0 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.715+0000 7f72d57fa640 1 -- 172.21.15.86:0/3418305057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0013e60 con 0x56071d2383d0 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 -- 172.21.15.86:0/3418305057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d2383d0 msgr2=0x56071d240a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/3418305057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d2383d0 0x56071d240a70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f72e0007850 tx=0x7f72e0007d10 comp rx=0 tx=0).stop 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 -- 172.21.15.86:0/3418305057 shutdown_connections 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/3418305057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d240fb0 0x56071d2433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/3418305057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d2383d0 0x56071d240a70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/3418305057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d237ab0 0x56071d237e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 -- 172.21.15.86:0/3418305057 >> 172.21.15.86:0/3418305057 conn(0x56071d2321e0 msgr2=0x56071d2325c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 -- 172.21.15.86:0/3418305057 shutdown_connections 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 -- 172.21.15.86:0/3418305057 wait complete. 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.716+0000 7f72f7dd8740 1 -- start start 2024-04-16T05:35:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72f7dd8740 1 Processor -- start 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d237ab0 0x56071d23e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d2383d0 0x56071d23e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d23eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56071d243da0 con 0x56071d240fb0 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56071d125410 con 0x56071d237ab0 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56071d243f20 con 0x56071d2383d0 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72f7dd8740 10 client.0 _ll_register_callbacks cb 0x56071d120b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ef59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d23eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ef59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d23eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55328/0 (socket says 172.21.15.86:55328) 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ef59e640 1 -- 172.21.15.86:0/809433644 learned_addr learned my addr 172.21.15.86:0/809433644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72eed9d640 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d237ab0 0x56071d23e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ee59c640 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d2383d0 0x56071d23e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ef59e640 1 -- 172.21.15.86:0/809433644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d2383d0 msgr2=0x56071d23e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ef59e640 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d2383d0 0x56071d23e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ef59e640 1 -- 172.21.15.86:0/809433644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d237ab0 msgr2=0x56071d23e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ef59e640 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d237ab0 0x56071d23e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ef59e640 1 -- 172.21.15.86:0/809433644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56071cfd7f70 con 0x56071d240fb0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ee59c640 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d2383d0 0x56071d23e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72ef59e640 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d23eb00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f72e8004a80 tx=0x7f72e8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3c80791d3dceea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.717+0000 7f72d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72d2ff5640 1 -- 172.21.15.86:0/809433644 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e802a060 con 0x56071d240fb0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72d2ff5640 1 -- 172.21.15.86:0/809433644 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f72e8016070 con 0x56071d240fb0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72f7dd8740 1 -- 172.21.15.86:0/809433644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e0018070 con 0x56071d240fb0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72d2ff5640 1 -- 172.21.15.86:0/809433644 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72e8031050 con 0x56071d240fb0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72f7dd8740 10 client.15954 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72f7dd8740 1 -- 172.21.15.86:0/809433644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56071ce6f170 con 0x56071d240fb0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72d2ff5640 1 -- 172.21.15.86:0/809433644 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e0018070 con 0x56071d240fb0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72d2ff5640 1 -- 172.21.15.86:0/809433644 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f72e8023320 con 0x56071d240fb0 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72f7dd8740 10 client.15954 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72f7dd8740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72f7dd8740 20 client.15954 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56071d251420 0x56071d253850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72f7dd8740 4 client.15954 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72f7dd8740 1 -- 172.21.15.86:0/809433644 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56071d1aa8b0 con 0x56071d251420 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.718+0000 7f72eed9d640 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56071d251420 0x56071d253850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.719+0000 7f72d2ff5640 10 client.15954 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.719+0000 7f72eed9d640 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56071d251420 0x56071d253850 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d2b236581b7651 server_cookie=ba2fee5658d3f4ad in_seq=0 out_seq=0 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72d2ff5640 1 -- 172.21.15.86:0/809433644 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56071d1aa8b0 con 0x56071d251420 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72d2ff5640 10 client.15954 handle_command_reply: tid=0 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 client.15954 shutdown 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 -- 172.21.15.86:0/809433644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 msgr2=0x56071d23eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d23eb00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f72e8004a80 tx=0x7f72e8009500 comp rx=0 tx=0).stop 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 -- 172.21.15.86:0/809433644 shutdown_connections 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56071d251420 0x56071d253850 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d23eb00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72d2ff5640 0 client.15954 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d2383d0 0x56071d23e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/809433644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d237ab0 0x56071d23e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 -- 172.21.15.86:0/809433644 >> 172.21.15.86:0/809433644 conn(0x56071d2321e0 msgr2=0x56071d233f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 -- 172.21.15.86:0/809433644 shutdown_connections 2024-04-16T05:35:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 1 -- 172.21.15.86:0/809433644 wait complete. 2024-04-16T05:35:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.720+0000 7f72f7dd8740 20 client.15954 trim_cache size 0 max 16384 2024-04-16T05:35:57.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.737+0000 7f72f7dd8740 1 Processor -- start 2024-04-16T05:35:57.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.737+0000 7f72f7dd8740 1 -- start start 2024-04-16T05:35:57.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 0x56071d2407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 0x56071d24cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d259120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01af460 con 0x56071d240fb0 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01b0fa0 con 0x56071d2383d0 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01afb30 con 0x56071d237ab0 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72ef59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d259120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72ef59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d259120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55338/0 (socket says 172.21.15.86:55338) 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72ef59e640 1 -- 172.21.15.86:0/1796344068 learned_addr learned my addr 172.21.15.86:0/1796344068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72ef59e640 1 -- 172.21.15.86:0/1796344068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 msgr2=0x56071d2407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72eed9d640 1 --2- 172.21.15.86:0/1796344068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 0x56071d2407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1796344068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 0x56071d2407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72ef59e640 1 -- 172.21.15.86:0/1796344068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 msgr2=0x56071d24cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1796344068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 0x56071d24cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72ef59e640 1 -- 172.21.15.86:0/1796344068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56071d238c10 con 0x56071d240fb0 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72eed9d640 1 --2- 172.21.15.86:0/1796344068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 0x56071d2407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1796344068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d259120 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f72e801ad30 tx=0x7f72e8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3531741baedf227 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72d4ff9640 1 -- 172.21.15.86:0/1796344068 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e803a070 con 0x56071d240fb0 2024-04-16T05:35:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72d4ff9640 1 -- 172.21.15.86:0/1796344068 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e8010cf0 con 0x56071d240fb0 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.738+0000 7f72d4ff9640 1 -- 172.21.15.86:0/1796344068 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8005040 con 0x56071d240fb0 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1796344068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 msgr2=0x56071d259120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1796344068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d259120 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f72e801ad30 tx=0x7f72e8011db0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1796344068 shutdown_connections 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1796344068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d259120 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1796344068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 0x56071d24cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1796344068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 0x56071d2407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1796344068 >> 172.21.15.86:0/1796344068 conn(0x56071d2321e0 msgr2=0x56071d23e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1796344068 shutdown_connections 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1796344068 wait complete. 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 -- start start 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 Processor -- start 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.739+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 0x56071d2495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 0x56071d245970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72f7dd8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d245f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01b08d0 con 0x56071d240fb0 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01b0200 con 0x56071d2383d0 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72f7dd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01ad420 con 0x56071d237ab0 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72f7dd8740 10 client.0 _ll_register_callbacks cb 0x7f72e8009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ef59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d245f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ef59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d245f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55342/0 (socket says 172.21.15.86:55342) 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ef59e640 1 -- 172.21.15.86:0/1638671836 learned_addr learned my addr 172.21.15.86:0/1638671836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72eed9d640 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 0x56071d2495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ee59c640 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 0x56071d245970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ef59e640 1 -- 172.21.15.86:0/1638671836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 msgr2=0x56071d2495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 0x56071d2495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ef59e640 1 -- 172.21.15.86:0/1638671836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 msgr2=0x56071d245970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 0x56071d245970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ef59e640 1 -- 172.21.15.86:0/1638671836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56071d246740 con 0x56071d240fb0 2024-04-16T05:35:57.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ee59c640 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 0x56071d245970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72ef59e640 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d245f10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f72e8023810 tx=0x7f72e8023a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114e1b7f78a684c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72d2ff5640 1 -- 172.21.15.86:0/1638671836 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e803a030 con 0x56071d240fb0 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72d2ff5640 1 -- 172.21.15.86:0/1638671836 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f72e8020040 con 0x56071d240fb0 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1638671836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e8010cf0 con 0x56071d240fb0 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72d2ff5640 1 -- 172.21.15.86:0/1638671836 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72e802a040 con 0x56071d240fb0 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72f7dd8740 10 client.15960 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.740+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1638671836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56071d2b8a20 con 0x56071d240fb0 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72d2ff5640 1 -- 172.21.15.86:0/1638671836 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e8010cf0 con 0x56071d240fb0 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72d2ff5640 1 -- 172.21.15.86:0/1638671836 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f72e8005530 con 0x56071d240fb0 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72f7dd8740 10 client.15960 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72f7dd8740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72f7dd8740 20 client.15960 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56071d2b9070 0x56071d2bb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72f7dd8740 4 client.15960 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:35:57.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1638671836 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56071d1a4720 con 0x56071d2b9070 2024-04-16T05:35:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72eed9d640 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56071d2b9070 0x56071d2bb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72eed9d640 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56071d2b9070 0x56071d2bb490 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec263ea755219749 server_cookie=a6267769363bf6f8 in_seq=0 out_seq=0 2024-04-16T05:35:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.741+0000 7f72d2ff5640 10 client.15960 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72d2ff5640 1 -- 172.21.15.86:0/1638671836 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56071d1a4720 con 0x56071d2b9070 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72d2ff5640 10 client.15960 handle_command_reply: tid=0 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72f7dd8740 1 client.15960 shutdown 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1638671836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 msgr2=0x56071d245f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d245f10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f72e8023810 tx=0x7f72e8023a00 comp rx=0 tx=0).stop 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1638671836 shutdown_connections 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56071d2b9070 0x56071d2bb490 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56071d240fb0 0x56071d245f10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56071d2383d0 0x56071d245970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72d2ff5640 0 client.15960 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72f7dd8740 1 --2- 172.21.15.86:0/1638671836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56071d237ab0 0x56071d2495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.742+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1638671836 >> 172.21.15.86:0/1638671836 conn(0x56071d2321e0 msgr2=0x56071d2309c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1638671836 shutdown_connections 2024-04-16T05:35:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f7dd8740 1 -- 172.21.15.86:0/1638671836 wait complete. 2024-04-16T05:35:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f7dd8740 20 client.15960 trim_cache size 0 max 16384 2024-04-16T05:35:57.747 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:57.747 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:35:57.747 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "73864a0e-3b36-493d-97e7-6dd5417ba5a8", 2024-04-16T05:35:57.747 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:35:57.747 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 -- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f72d80a3e40 msgr2=0x7f72d80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f72d80a3e40 0x7f72d80a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f72e8005c40 tx=0x7f72e80098e0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 -- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100ec0 msgr2=0x7f72f019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100ec0 0x7f72f019ebe0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009230 tx=0x7f72e402cba0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 -- 172.21.15.86:0/1445116752 shutdown_connections 2024-04-16T05:35:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f72d80a3e40 0x7f72d80a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72f0106fe0 0x7f72f0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0100ec0 0x7f72f019ebe0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 --2- 172.21.15.86:0/1445116752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0100500 0x7f72f019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 -- 172.21.15.86:0/1445116752 >> 172.21.15.86:0/1445116752 conn(0x7f72f0077490 msgr2=0x7f72f01050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 -- 172.21.15.86:0/1445116752 shutdown_connections 2024-04-16T05:35:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.743+0000 7f72f51f0640 1 -- 172.21.15.86:0/1445116752 wait complete. 2024-04-16T05:35:57.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:35:57.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:35:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 -- 172.21.15.86:0/2215139899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8101150 msgr2=0x7f3cc8101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/2215139899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8101150 0x7f3cc8101530 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009900 tx=0x7f3cb802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 -- 172.21.15.86:0/2215139899 shutdown_connections 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/2215139899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cc8106220 0x7f3cc810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/2215139899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3cc8101b00 0x7f3cc8105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/2215139899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8101150 0x7f3cc8101530 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 -- 172.21.15.86:0/2215139899 >> 172.21.15.86:0/2215139899 conn(0x7f3cc8077280 msgr2=0x7f3cc8077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 -- 172.21.15.86:0/2215139899 shutdown_connections 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 -- 172.21.15.86:0/2215139899 wait complete. 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 Processor -- start 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.861+0000 7f3cce5cd640 1 -- start start 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cce5cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cc8101150 0x7f3cc819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cce5cd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3cc8106220 0x7f3cc819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cce5cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8198710 0x7f3cc8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cce5cd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81123f0 con 0x7f3cc8198710 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cce5cd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc8112270 con 0x7f3cc8106220 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cce5cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc8112570 con 0x7f3cc8101150 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cccb6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8198710 0x7f3cc8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cccb6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8198710 0x7f3cc8198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55368/0 (socket says 172.21.15.86:55368) 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cccb6a640 1 -- 172.21.15.86:0/3296205175 learned_addr learned my addr 172.21.15.86:0/3296205175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3cc8106220 0x7f3cc819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cc8101150 0x7f3cc819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cccb6a640 1 -- 172.21.15.86:0/3296205175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cc8101150 msgr2=0x7f3cc819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cccb6a640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cc8101150 0x7f3cc819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cccb6a640 1 -- 172.21.15.86:0/3296205175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3cc8106220 msgr2=0x7f3cc819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cccb6a640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3cc8106220 0x7f3cc819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cccb6a640 1 -- 172.21.15.86:0/3296205175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc8199310 con 0x7f3cc8198710 2024-04-16T05:35:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cc8101150 0x7f3cc819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cccb6a640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8198710 0x7f3cc8198b90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000cad0 tx=0x7f3cc000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cc57fa640 1 -- 172.21.15.86:0/3296205175 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc000ae20 con 0x7f3cc8198710 2024-04-16T05:35:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cc57fa640 1 -- 172.21.15.86:0/3296205175 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc0005020 con 0x7f3cc8198710 2024-04-16T05:35:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cce5cd640 1 -- 172.21.15.86:0/3296205175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc81a53f0 con 0x7f3cc8198710 2024-04-16T05:35:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.862+0000 7f3cc57fa640 1 -- 172.21.15.86:0/3296205175 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc000f6f0 con 0x7f3cc8198710 2024-04-16T05:35:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.863+0000 7f3cce5cd640 1 -- 172.21.15.86:0/3296205175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc81a57f0 con 0x7f3cc8198710 2024-04-16T05:35:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cc57fa640 1 -- 172.21.15.86:0/3296205175 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3cc00051d0 con 0x7f3cc8198710 2024-04-16T05:35:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cd11ba740 1 Processor -- start 2024-04-16T05:35:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cd11ba740 1 -- start start 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b66230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b6ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b71740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f51acb180 con 0x564f51b66770 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564f51a68360 con 0x564f51b65e50 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f51ac7140 con 0x564f51b6f350 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cc7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b6ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cc57fa640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cb00a3ec0 0x7f3cb00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cc7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b6ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55380/0 (socket says 172.21.15.86:55380) 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.865+0000 7f3cc7fff640 1 -- 172.21.15.86:0/1156459839 learned_addr learned my addr 172.21.15.86:0/1156459839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cccb6a640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cb00a3ec0 0x7f3cb00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b71740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b66230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc7fff640 1 -- 172.21.15.86:0/1156459839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 msgr2=0x564f51b71740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b71740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc7fff640 1 -- 172.21.15.86:0/1156459839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 msgr2=0x564f51b66230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b66230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc7fff640 1 -- 172.21.15.86:0/1156459839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f518a77d0 con 0x564f51b66770 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b71740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc57fa640 1 -- 172.21.15.86:0/3296205175 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3cc00d0de0 con 0x7f3cc8198710 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b6ee10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009900 tx=0x7f3cb802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544bd7e5ad6835e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cccb6a640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cb00a3ec0 0x7f3cb00a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00ce0a0 tx=0x7f3cc00983c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cae7fc640 1 -- 172.21.15.86:0/1156459839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb802cbc0 con 0x564f51b66770 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cae7fc640 1 -- 172.21.15.86:0/1156459839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb8005040 con 0x564f51b66770 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.866+0000 7f3cae7fc640 1 -- 172.21.15.86:0/1156459839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80049e0 con 0x564f51b66770 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1156459839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 msgr2=0x564f51b6ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b6ee10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009900 tx=0x7f3cb802c650 comp rx=0 tx=0).stop 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1156459839 shutdown_connections 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b71740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b6ee10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1156459839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b66230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1156459839 >> 172.21.15.86:0/1156459839 conn(0x564f51b60580 msgr2=0x564f51b60960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1156459839 shutdown_connections 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1156459839 wait complete. 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.867+0000 7f3cd11ba740 1 -- start start 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cd11ba740 1 Processor -- start 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b6c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b66770 0x564f51b6c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b6f350 0x564f51b6cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f51b720e0 con 0x564f51b6f350 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564f51a842c0 con 0x564f51b65e50 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f51b72260 con 0x564f51b66770 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cd11ba740 10 client.0 _ll_register_callbacks cb 0x564f51a6b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b6f350 0x564f51b6cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b6f350 0x564f51b6cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55390/0 (socket says 172.21.15.86:55390) 2024-04-16T05:35:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 -- 172.21.15.86:0/1068600612 learned_addr learned my addr 172.21.15.86:0/1068600612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b6c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b66770 0x564f51b6c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 -- 172.21.15.86:0/1068600612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b66770 msgr2=0x564f51b6c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b66770 0x564f51b6c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 -- 172.21.15.86:0/1068600612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 msgr2=0x564f51b6c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b6c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 -- 172.21.15.86:0/1068600612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f517dd3f0 con 0x564f51b6f350 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b6f350 0x564f51b6cd80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc012c20 tx=0x7f3cbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1560ee47c7f41065 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.868+0000 7f3ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3ca77fe640 1 -- 172.21.15.86:0/1068600612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc01b040 con 0x564f51b6f350 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3ca77fe640 1 -- 172.21.15.86:0/1068600612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3cbc005020 con 0x564f51b6f350 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1068600612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cb8005040 con 0x564f51b6f350 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3ca77fe640 1 -- 172.21.15.86:0/1068600612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3cbc010e60 con 0x564f51b6f350 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cd11ba740 10 client.15972 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1068600612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f5179d0f0 con 0x564f51b6f350 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3ca77fe640 1 -- 172.21.15.86:0/1068600612 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb8005040 con 0x564f51b6f350 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3ca77fe640 1 -- 172.21.15.86:0/1068600612 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3cbc020400 con 0x564f51b6f350 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cd11ba740 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cd11ba740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cd11ba740 20 client.15972 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564f51b7f680 0x564f51b81a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cd11ba740 4 client.15972 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1068600612 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f51ad8ba0 con 0x564f51b7f680 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.869+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564f51b7f680 0x564f51b81a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.870+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564f51b7f680 0x564f51b81a70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87e56c87af2872 server_cookie=ab4ebc4ecf215719 in_seq=0 out_seq=0 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.870+0000 7f3ca77fe640 10 client.15972 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.870+0000 7f3ca77fe640 1 -- 172.21.15.86:0/1068600612 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f51ad8ba0 con 0x564f51b7f680 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.870+0000 7f3ca77fe640 10 client.15972 handle_command_reply: tid=0 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.870+0000 7f3cd11ba740 1 client.15972 shutdown 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1068600612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b6f350 msgr2=0x564f51b6cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b6f350 0x564f51b6cd80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc012c20 tx=0x7f3cbc004730 comp rx=0 tx=0).stop 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1068600612 shutdown_connections 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564f51b7f680 0x564f51b81a70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b6f350 0x564f51b6cd80 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b66770 0x564f51b6c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3ca77fe640 0 client.15972 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/1068600612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b6c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1068600612 >> 172.21.15.86:0/1068600612 conn(0x564f51b60580 msgr2=0x564f51b622d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1068600612 shutdown_connections 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 1 -- 172.21.15.86:0/1068600612 wait complete. 2024-04-16T05:35:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.871+0000 7f3cd11ba740 20 client.15972 trim_cache size 0 max 16384 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cd11ba740 1 Processor -- start 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cd11ba740 1 -- start start 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b65e50 0x564f51b6ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b66770 0x564f51b7afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b60960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81af460 con 0x564f51b65e50 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81b0fa0 con 0x564f51b66770 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81afb30 con 0x564f51b6f350 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b65e50 0x564f51b6ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b65e50 0x564f51b6ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55394/0 (socket says 172.21.15.86:55394) 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.888+0000 7f3cc77fe640 1 -- 172.21.15.86:0/2468736858 learned_addr learned my addr 172.21.15.86:0/2468736858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cc77fe640 1 -- 172.21.15.86:0/2468736858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 msgr2=0x564f51b60960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/2468736858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b66770 0x564f51b7afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/2468736858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b60960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cc77fe640 1 -- 172.21.15.86:0/2468736858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b66770 msgr2=0x564f51b7afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/2468736858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b66770 0x564f51b7afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cc77fe640 1 -- 172.21.15.86:0/2468736858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f51b603b0 con 0x564f51b65e50 2024-04-16T05:35:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/2468736858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b66770 0x564f51b7afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/2468736858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b65e50 0x564f51b6ea30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc000c00 tx=0x7f3cbc012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47053ed736ad7e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cadffb640 1 -- 172.21.15.86:0/2468736858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc012500 con 0x564f51b65e50 2024-04-16T05:35:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cadffb640 1 -- 172.21.15.86:0/2468736858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cbc032070 con 0x564f51b65e50 2024-04-16T05:35:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cadffb640 1 -- 172.21.15.86:0/2468736858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0035f0 con 0x564f51b65e50 2024-04-16T05:35:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cd11ba740 1 -- 172.21.15.86:0/2468736858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b65e50 msgr2=0x564f51b6ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/2468736858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b65e50 0x564f51b6ea30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc000c00 tx=0x7f3cbc012760 comp rx=0 tx=0).stop 2024-04-16T05:35:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cd11ba740 1 -- 172.21.15.86:0/2468736858 shutdown_connections 2024-04-16T05:35:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/2468736858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b60960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/2468736858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b66770 0x564f51b7afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/2468736858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b65e50 0x564f51b6ea30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.889+0000 7f3cd11ba740 1 -- 172.21.15.86:0/2468736858 >> 172.21.15.86:0/2468736858 conn(0x564f51b60580 msgr2=0x564f51b5eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 -- 172.21.15.86:0/2468736858 shutdown_connections 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 -- 172.21.15.86:0/2468736858 wait complete. 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 -- start start 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 Processor -- start 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b77a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b74c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81b08d0 con 0x564f51b66770 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81b0200 con 0x564f51b65e50 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81ad420 con 0x564f51b6f350 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cd11ba740 10 client.0 _ll_register_callbacks cb 0x7f3cbc00bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cc7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b77a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cc7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b77a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55402/0 (socket says 172.21.15.86:55402) 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.890+0000 7f3cc7fff640 1 -- 172.21.15.86:0/3109107015 learned_addr learned my addr 172.21.15.86:0/3109107015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:35:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cc7fff640 1 -- 172.21.15.86:0/3109107015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 msgr2=0x564f51b74c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b74c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cc7fff640 1 -- 172.21.15.86:0/3109107015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 msgr2=0x564f51b774d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b774d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cc7fff640 1 -- 172.21.15.86:0/3109107015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f51b75490 con 0x564f51b66770 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cc7fff640 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b77a10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8004620 tx=0x7f3cb802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7e9e3add95b1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3ca77fe640 1 -- 172.21.15.86:0/3109107015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8005450 con 0x564f51b66770 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3ca77fe640 1 -- 172.21.15.86:0/3109107015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3cb804f420 con 0x564f51b66770 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cd11ba740 1 -- 172.21.15.86:0/3109107015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cbc032070 con 0x564f51b66770 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3ca77fe640 1 -- 172.21.15.86:0/3109107015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3cb802cb20 con 0x564f51b66770 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cd11ba740 10 client.15978 fetch_fsmap learned FSMap version 29 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cd11ba740 1 -- 172.21.15.86:0/3109107015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f51be5800 con 0x564f51b66770 2024-04-16T05:35:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3ca77fe640 1 -- 172.21.15.86:0/3109107015 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cbc032070 con 0x564f51b66770 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3ca77fe640 1 -- 172.21.15.86:0/3109107015 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3cb804f740 con 0x564f51b66770 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cd11ba740 10 client.15978 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cd11ba740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cd11ba740 20 client.15978 populate_metadata read hostname 'smithi086' 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564f51be5ae0 0x564f51be7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cd11ba740 4 client.15978 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.891+0000 7f3cd11ba740 1 -- 172.21.15.86:0/3109107015 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f51ad2a10 con 0x564f51be5ae0 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564f51be5ae0 0x564f51be7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3ca77fe640 10 client.15978 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cc77fe640 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564f51be5ae0 0x564f51be7ed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc74ea73cdc8f6a0 server_cookie=2caf6f11b08cc0b1 in_seq=0 out_seq=0 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3ca77fe640 1 -- 172.21.15.86:0/3109107015 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564f51ad2a10 con 0x564f51be5ae0 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3ca77fe640 10 client.15978 handle_command_reply: tid=0 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cd11ba740 1 client.15978 shutdown 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cd11ba740 1 -- 172.21.15.86:0/3109107015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 msgr2=0x564f51b77a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b77a10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8004620 tx=0x7f3cb802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cd11ba740 1 -- 172.21.15.86:0/3109107015 shutdown_connections 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564f51be5ae0 0x564f51be7ed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f51b6f350 0x564f51b74c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f51b66770 0x564f51b77a10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3ca77fe640 0 client.15978 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cd11ba740 1 --2- 172.21.15.86:0/3109107015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f51b65e50 0x564f51b774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.892+0000 7f3cd11ba740 1 -- 172.21.15.86:0/3109107015 >> 172.21.15.86:0/3109107015 conn(0x564f51b60580 msgr2=0x564f51b6cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cd11ba740 1 -- 172.21.15.86:0/3109107015 shutdown_connections 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cd11ba740 1 -- 172.21.15.86:0/3109107015 wait complete. 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cd11ba740 20 client.15978 trim_cache size 0 max 16384 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:35:57.897 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:35:57.898 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:35:57.898 INFO:teuthology.orchestra.run.smithi086.stdout: "73864a0e-3b36-493d-97e7-6dd5417ba5a8": { 2024-04-16T05:35:57.898 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:35:57.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "73864a0e-3b36-493d-97e7-6dd5417ba5a8", 2024-04-16T05:35:57.898 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:35:57.898 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:35:57.898 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:35:57.898 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 -- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cb00a3ec0 msgr2=0x7f3cb00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cb00a3ec0 0x7f3cb00a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00ce0a0 tx=0x7f3cc00983c0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 -- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8198710 msgr2=0x7f3cc8198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8198710 0x7f3cc8198b90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000cad0 tx=0x7f3cc000cf90 comp rx=0 tx=0).stop 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 -- 172.21.15.86:0/3296205175 shutdown_connections 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc8198710 0x7f3cc8198b90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3cb00a3ec0 0x7f3cb00a6380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3cc8106220 0x7f3cc819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 --2- 172.21.15.86:0/3296205175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cc8101150 0x7f3cc819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:57.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 -- 172.21.15.86:0/3296205175 >> 172.21.15.86:0/3296205175 conn(0x7f3cc8077280 msgr2=0x7f3cc80fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:57.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 -- 172.21.15.86:0/3296205175 shutdown_connections 2024-04-16T05:35:57.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:35:57.893+0000 7f3cce5cd640 1 -- 172.21.15.86:0/3296205175 wait complete. 2024-04-16T05:35:57.903 INFO:tasks.cephfs.filesystem:scrub status for tag:73864a0e-3b36-493d-97e7-6dd5417ba5a8 - {'path': '/', 'tag': '73864a0e-3b36-493d-97e7-6dd5417ba5a8', 'options': 'recursive,force'} 2024-04-16T05:35:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:58 smithi086 ceph-mon[32028]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:35:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:58 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:35:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:58 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:58 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:35:58.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:58 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:58 smithi196 ceph-mon[36373]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:35:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:58 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:35:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:58 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:58 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:35:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:58 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:59.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:58 smithi194 ceph-mon[35865]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:35:59.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:58 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:35:59.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:58 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:59.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:58 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:35:59.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:58 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:59.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:35:59 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:35:59.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:35:59 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:36:00.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:35:59 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:36:00.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:00 smithi086 ceph-mon[32028]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-16T05:36:00.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:36:00 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:36:00.878+0000 7ff070fbd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:00.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:36:00 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:36:00.878+0000 7ff070fbd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:00.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:36:00 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:36:00.878+0000 7ff070fbd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:00.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:36:00 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:36:00.878+0000 7ff070fbd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:00.968 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:36:00 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32258]: 2024-04-16T05:36:00.878+0000 7ff070fbd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:00 smithi196 ceph-mon[36373]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-16T05:36:01.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:00 smithi194 ceph-mon[35865]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-16T05:36:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:02 smithi086 ceph-mon[32028]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:02 smithi086 ceph-mon[32028]: mgrmap e25: x(active, since 44s), standbys: z, y 2024-04-16T05:36:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:02 smithi196 ceph-mon[36373]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:02 smithi196 ceph-mon[36373]: mgrmap e25: x(active, since 44s), standbys: z, y 2024-04-16T05:36:03.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:02 smithi194 ceph-mon[35865]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:03.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:02 smithi194 ceph-mon[35865]: mgrmap e25: x(active, since 44s), standbys: z, y 2024-04-16T05:36:04.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:04 smithi086 ceph-mon[32028]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-16T05:36:04.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:04 smithi196 ceph-mon[36373]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-16T05:36:05.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:04 smithi194 ceph-mon[35865]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-16T05:36:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.037+0000 7f30837cd640 1 -- 172.21.15.86:0/3394688090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 msgr2=0x7f307c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.037+0000 7f30837cd640 1 --2- 172.21.15.86:0/3394688090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 0x7f307c10dd20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3074009ca0 tx=0x7f307402c140 comp rx=0 tx=0).stop 2024-04-16T05:36:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.037+0000 7f30837cd640 1 -- 172.21.15.86:0/3394688090 shutdown_connections 2024-04-16T05:36:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.037+0000 7f30837cd640 1 --2- 172.21.15.86:0/3394688090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 0x7f307c10dd20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.037+0000 7f30837cd640 1 --2- 172.21.15.86:0/3394688090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f307c101250 0x7f307c1016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.037+0000 7f30837cd640 1 --2- 172.21.15.86:0/3394688090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c107060 0x7f307c107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.037+0000 7f30837cd640 1 -- 172.21.15.86:0/3394688090 >> 172.21.15.86:0/3394688090 conn(0x7f307c076d00 msgr2=0x7f307c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 -- 172.21.15.86:0/3394688090 shutdown_connections 2024-04-16T05:36:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 -- 172.21.15.86:0/3394688090 wait complete. 2024-04-16T05:36:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 Processor -- start 2024-04-16T05:36:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 -- start start 2024-04-16T05:36:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f307c101250 0x7f307c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 0x7f307c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c107060 0x7f307c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1123e0 con 0x7f307c101c10 2024-04-16T05:36:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f307c112260 con 0x7f307c101250 2024-04-16T05:36:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.038+0000 7f30837cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f307c112560 con 0x7f307c107060 2024-04-16T05:36:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3080d68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 0x7f307c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3080d68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 0x7f307c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36196/0 (socket says 172.21.15.86:36196) 2024-04-16T05:36:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3080d68640 1 -- 172.21.15.86:0/4142126778 learned_addr learned my addr 172.21.15.86:0/4142126778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3081d6a640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c107060 0x7f307c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3081569640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f307c101250 0x7f307c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3080d68640 1 -- 172.21.15.86:0/4142126778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c107060 msgr2=0x7f307c19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3080d68640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c107060 0x7f307c19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3080d68640 1 -- 172.21.15.86:0/4142126778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f307c101250 msgr2=0x7f307c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3080d68640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f307c101250 0x7f307c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3080d68640 1 -- 172.21.15.86:0/4142126778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c19d6e0 con 0x7f307c101c10 2024-04-16T05:36:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f3080d68640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 0x7f307c198c80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f306c00c890 tx=0x7f306c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f30727fc640 1 -- 172.21.15.86:0/4142126778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c004220 con 0x7f307c101c10 2024-04-16T05:36:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f30837cd640 1 -- 172.21.15.86:0/4142126778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307c19d9d0 con 0x7f307c101c10 2024-04-16T05:36:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f30727fc640 1 -- 172.21.15.86:0/4142126778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f306c0043c0 con 0x7f307c101c10 2024-04-16T05:36:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f30727fc640 1 -- 172.21.15.86:0/4142126778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c004a00 con 0x7f307c101c10 2024-04-16T05:36:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.039+0000 7f30837cd640 1 -- 172.21.15.86:0/4142126778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307c1a5260 con 0x7f307c101c10 2024-04-16T05:36:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.042+0000 7f30727fc640 1 -- 172.21.15.86:0/4142126778 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f306c0079b0 con 0x7f307c101c10 2024-04-16T05:36:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.042+0000 7f30863bb740 1 Processor -- start 2024-04-16T05:36:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.042+0000 7f30863bb740 1 -- start start 2024-04-16T05:36:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f30863bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29ac10 0x55cdbc29aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f30863bb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29b530 0x55cdbc2a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f30863bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f30863bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc1ff7d0 con 0x55cdbc29ac10 2024-04-16T05:36:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f30863bb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc19cf70 con 0x55cdbc29b530 2024-04-16T05:36:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f30863bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc1fb790 con 0x55cdbc2a4110 2024-04-16T05:36:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f30727fc640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30680a3f70 0x7f30680a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081d6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29ac10 0x55cdbc29aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081d6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29ac10 0x55cdbc29aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36202/0 (socket says 172.21.15.86:36202) 2024-04-16T05:36:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081d6a640 1 -- 172.21.15.86:0/1914622364 learned_addr learned my addr 172.21.15.86:0/1914622364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081d6a640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30680a3f70 0x7f30680a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081569640 1 --2- 172.21.15.86:0/1914622364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29b530 0x55cdbc2a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f30727fc640 1 -- 172.21.15.86:0/4142126778 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f306c0d0a50 con 0x7f307c101c10 2024-04-16T05:36:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3080d68640 1 --2- 172.21.15.86:0/1914622364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081d6a640 1 -- 172.21.15.86:0/1914622364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 msgr2=0x55cdbc2a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081d6a640 1 --2- 172.21.15.86:0/1914622364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081d6a640 1 -- 172.21.15.86:0/1914622364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29b530 msgr2=0x55cdbc2a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081d6a640 1 --2- 172.21.15.86:0/1914622364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29b530 0x55cdbc2a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.043+0000 7f3081d6a640 1 -- 172.21.15.86:0/1914622364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdbbf12210 con 0x55cdbc29ac10 2024-04-16T05:36:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.044+0000 7f3081d6a640 1 --2- 172.21.15.86:0/1914622364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29ac10 0x55cdbc29aff0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3074009a60 tx=0x7f3074032610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e718e8098bf3ae6e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.044+0000 7f30677fe640 1 -- 172.21.15.86:0/1914622364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074039040 con 0x55cdbc29ac10 2024-04-16T05:36:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.044+0000 7f30677fe640 1 -- 172.21.15.86:0/1914622364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3074047070 con 0x55cdbc29ac10 2024-04-16T05:36:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.044+0000 7f30677fe640 1 -- 172.21.15.86:0/1914622364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074034680 con 0x55cdbc29ac10 2024-04-16T05:36:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 -- 172.21.15.86:0/1914622364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29ac10 msgr2=0x55cdbc29aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f3081d6a640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30680a3f70 0x7f30680a6430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3074032530 tx=0x7f3074048000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 --2- 172.21.15.86:0/1914622364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29ac10 0x55cdbc29aff0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3074009a60 tx=0x7f3074032610 comp rx=0 tx=0).stop 2024-04-16T05:36:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 -- 172.21.15.86:0/1914622364 shutdown_connections 2024-04-16T05:36:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 --2- 172.21.15.86:0/1914622364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 --2- 172.21.15.86:0/1914622364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29b530 0x55cdbc2a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 --2- 172.21.15.86:0/1914622364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29ac10 0x55cdbc29aff0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 -- 172.21.15.86:0/1914622364 >> 172.21.15.86:0/1914622364 conn(0x55cdbc295340 msgr2=0x55cdbc295720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 -- 172.21.15.86:0/1914622364 shutdown_connections 2024-04-16T05:36:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 -- 172.21.15.86:0/1914622364 wait complete. 2024-04-16T05:36:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 -- start start 2024-04-16T05:36:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.045+0000 7f30863bb740 1 Processor -- start 2024-04-16T05:36:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f30863bb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc2a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f30863bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 0x55cdbc2a1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f30863bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f30863bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc2a6f00 con 0x55cdbc2a4110 2024-04-16T05:36:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f30863bb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc187e90 con 0x55cdbc29ac10 2024-04-16T05:36:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f30863bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc2a7080 con 0x55cdbc29b530 2024-04-16T05:36:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f30863bb740 10 client.0 _ll_register_callbacks cb 0x55cdbc1bbcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081d6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081d6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36210/0 (socket says 172.21.15.86:36210) 2024-04-16T05:36:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081d6a640 1 -- 172.21.15.86:0/2024645301 learned_addr learned my addr 172.21.15.86:0/2024645301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3080d68640 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 0x55cdbc2a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081569640 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc2a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081d6a640 1 -- 172.21.15.86:0/2024645301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 msgr2=0x55cdbc2a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081d6a640 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 0x55cdbc2a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081d6a640 1 -- 172.21.15.86:0/2024645301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 msgr2=0x55cdbc2a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081d6a640 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc2a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081d6a640 1 -- 172.21.15.86:0/2024645301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdbbfde620 con 0x55cdbc2a4110 2024-04-16T05:36:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3081d6a640 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a1bc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3074032ad0 tx=0x7f3074005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef5b5ab0f006935 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3064ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3064ff9640 1 -- 172.21.15.86:0/2024645301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074057030 con 0x55cdbc2a4110 2024-04-16T05:36:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f3064ff9640 1 -- 172.21.15.86:0/2024645301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3074034920 con 0x55cdbc2a4110 2024-04-16T05:36:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.046+0000 7f30863bb740 1 -- 172.21.15.86:0/2024645301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3074047070 con 0x55cdbc2a4110 2024-04-16T05:36:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f3064ff9640 1 -- 172.21.15.86:0/2024645301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3074064050 con 0x55cdbc2a4110 2024-04-16T05:36:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f30863bb740 10 client.15996 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f30863bb740 1 -- 172.21.15.86:0/2024645301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cdbc2b31a0 con 0x55cdbc2a4110 2024-04-16T05:36:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f3064ff9640 1 -- 172.21.15.86:0/2024645301 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3074047070 con 0x55cdbc2a4110 2024-04-16T05:36:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f3064ff9640 1 -- 172.21.15.86:0/2024645301 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f307403f050 con 0x55cdbc2a4110 2024-04-16T05:36:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f30863bb740 10 client.15996 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f30863bb740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f30863bb740 20 client.15996 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f30863bb740 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cdbc2b44f0 0x55cdbc2b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f30863bb740 4 client.15996 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f30863bb740 1 -- 172.21.15.86:0/2024645301 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdbc19f380 con 0x55cdbc2b44f0 2024-04-16T05:36:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.047+0000 7f3081569640 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cdbc2b44f0 0x55cdbc2b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.048+0000 7f3064ff9640 10 client.15996 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.048+0000 7f3081569640 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cdbc2b44f0 0x55cdbc2b68e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4865529e46920c87 server_cookie=9f6a8de79b6d2c21 in_seq=0 out_seq=0 2024-04-16T05:36:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.048+0000 7f3064ff9640 1 -- 172.21.15.86:0/2024645301 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdbc19f380 con 0x55cdbc2b44f0 2024-04-16T05:36:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.048+0000 7f3064ff9640 10 client.15996 handle_command_reply: tid=0 2024-04-16T05:36:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.048+0000 7f30863bb740 1 client.15996 shutdown 2024-04-16T05:36:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 -- 172.21.15.86:0/2024645301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 msgr2=0x55cdbc2a1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a1bc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3074032ad0 tx=0x7f3074005ef0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 -- 172.21.15.86:0/2024645301 shutdown_connections 2024-04-16T05:36:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cdbc2b44f0 0x55cdbc2b68e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a1bc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f3064ff9640 0 client.15996 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 0x55cdbc2a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 --2- 172.21.15.86:0/2024645301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc2a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 -- 172.21.15.86:0/2024645301 >> 172.21.15.86:0/2024645301 conn(0x55cdbc295340 msgr2=0x55cdbc297090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 -- 172.21.15.86:0/2024645301 shutdown_connections 2024-04-16T05:36:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 1 -- 172.21.15.86:0/2024645301 wait complete. 2024-04-16T05:36:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.049+0000 7f30863bb740 20 client.15996 trim_cache size 0 max 16384 2024-04-16T05:36:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.063+0000 7f30863bb740 1 Processor -- start 2024-04-16T05:36:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f30863bb740 1 -- start start 2024-04-16T05:36:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f30863bb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc2b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f30863bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 0x55cdbc2afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f30863bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f30863bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1af2d0 con 0x55cdbc2a4110 2024-04-16T05:36:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f30863bb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1b0e10 con 0x55cdbc29ac10 2024-04-16T05:36:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f30863bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1af9a0 con 0x55cdbc29b530 2024-04-16T05:36:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081d6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081d6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36222/0 (socket says 172.21.15.86:36222) 2024-04-16T05:36:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081d6a640 1 -- 172.21.15.86:0/3215605759 learned_addr learned my addr 172.21.15.86:0/3215605759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081569640 1 --2- 172.21.15.86:0/3215605759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc2b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3080d68640 1 --2- 172.21.15.86:0/3215605759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 0x55cdbc2afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081d6a640 1 -- 172.21.15.86:0/3215605759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 msgr2=0x55cdbc2afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081d6a640 1 --2- 172.21.15.86:0/3215605759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 0x55cdbc2afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081d6a640 1 -- 172.21.15.86:0/3215605759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 msgr2=0x55cdbc2b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081d6a640 1 --2- 172.21.15.86:0/3215605759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc2b9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081d6a640 1 -- 172.21.15.86:0/3215605759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdbc29b360 con 0x55cdbc2a4110 2024-04-16T05:36:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3081d6a640 1 --2- 172.21.15.86:0/3215605759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a19e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3074037040 tx=0x7f307402c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1cde6d1a2cd24b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3066ffd640 1 -- 172.21.15.86:0/3215605759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074057070 con 0x55cdbc2a4110 2024-04-16T05:36:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3066ffd640 1 -- 172.21.15.86:0/3215605759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3074047070 con 0x55cdbc2a4110 2024-04-16T05:36:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.064+0000 7f3066ffd640 1 -- 172.21.15.86:0/3215605759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074039040 con 0x55cdbc2a4110 2024-04-16T05:36:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 -- 172.21.15.86:0/3215605759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 msgr2=0x55cdbc2a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 --2- 172.21.15.86:0/3215605759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a19e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3074037040 tx=0x7f307402c140 comp rx=0 tx=0).stop 2024-04-16T05:36:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 -- 172.21.15.86:0/3215605759 shutdown_connections 2024-04-16T05:36:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 --2- 172.21.15.86:0/3215605759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc2a4110 0x55cdbc2a19e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 --2- 172.21.15.86:0/3215605759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc29b530 0x55cdbc2afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 --2- 172.21.15.86:0/3215605759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc2b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 -- 172.21.15.86:0/3215605759 >> 172.21.15.86:0/3215605759 conn(0x55cdbc295340 msgr2=0x55cdbc2966f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 -- 172.21.15.86:0/3215605759 shutdown_connections 2024-04-16T05:36:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 -- 172.21.15.86:0/3215605759 wait complete. 2024-04-16T05:36:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 -- start start 2024-04-16T05:36:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.065+0000 7f30863bb740 1 Processor -- start 2024-04-16T05:36:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f30863bb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc3179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f30863bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29b530 0x55cdbc317f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f30863bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 0x55cdbc311df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f30863bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1b0740 con 0x55cdbc29b530 2024-04-16T05:36:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f30863bb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1b0070 con 0x55cdbc29ac10 2024-04-16T05:36:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f30863bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1ad290 con 0x55cdbc2a4110 2024-04-16T05:36:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f30863bb740 10 client.0 _ll_register_callbacks cb 0x7f307800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3080d68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29b530 0x55cdbc317f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3080d68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29b530 0x55cdbc317f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36236/0 (socket says 172.21.15.86:36236) 2024-04-16T05:36:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3080d68640 1 -- 172.21.15.86:0/1200396284 learned_addr learned my addr 172.21.15.86:0/1200396284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3081569640 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc3179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3081d6a640 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 0x55cdbc311df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3080d68640 1 -- 172.21.15.86:0/1200396284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 msgr2=0x55cdbc311df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3080d68640 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 0x55cdbc311df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3080d68640 1 -- 172.21.15.86:0/1200396284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 msgr2=0x55cdbc3179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3080d68640 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc3179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3080d68640 1 -- 172.21.15.86:0/1200396284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdbc3124e0 con 0x55cdbc29b530 2024-04-16T05:36:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.066+0000 7f3081569640 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc3179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f3080d68640 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29b530 0x55cdbc317f20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f306c0058c0 tx=0x7f306c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc692ab0c76f9fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f3064ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f3064ff9640 1 -- 172.21.15.86:0/1200396284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c01d040 con 0x55cdbc29b530 2024-04-16T05:36:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f3064ff9640 1 -- 172.21.15.86:0/1200396284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f306c0a4e20 con 0x55cdbc29b530 2024-04-16T05:36:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f30863bb740 1 -- 172.21.15.86:0/1200396284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3074047070 con 0x55cdbc29b530 2024-04-16T05:36:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f3064ff9640 1 -- 172.21.15.86:0/1200396284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f306c0a07f0 con 0x55cdbc29b530 2024-04-16T05:36:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f30863bb740 10 client.16002 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f30863bb740 1 -- 172.21.15.86:0/1200396284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cdbc31f1f0 con 0x55cdbc29b530 2024-04-16T05:36:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f3064ff9640 1 -- 172.21.15.86:0/1200396284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3074047070 con 0x55cdbc29b530 2024-04-16T05:36:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f3064ff9640 1 -- 172.21.15.86:0/1200396284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f306c0bb260 con 0x55cdbc29b530 2024-04-16T05:36:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f30863bb740 10 client.16002 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f30863bb740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f30863bb740 20 client.16002 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f30863bb740 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cdbc31f990 0x55cdbc321db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f30863bb740 4 client.16002 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:36:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.067+0000 7f30863bb740 1 -- 172.21.15.86:0/1200396284 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cdbc20d1f0 con 0x55cdbc31f990 2024-04-16T05:36:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.068+0000 7f3081569640 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cdbc31f990 0x55cdbc321db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.068+0000 7f3064ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.068+0000 7f3081569640 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cdbc31f990 0x55cdbc321db0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b647b20b64904b server_cookie=f479b2a5f19c62e4 in_seq=0 out_seq=0 2024-04-16T05:36:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.075+0000 7f3064ff9640 1 -- 172.21.15.86:0/1200396284 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cdbc20d1f0 con 0x55cdbc31f990 2024-04-16T05:36:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.075+0000 7f3064ff9640 10 client.16002 handle_command_reply: tid=0 2024-04-16T05:36:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 client.16002 shutdown 2024-04-16T05:36:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 -- 172.21.15.86:0/1200396284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29b530 msgr2=0x55cdbc317f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29b530 0x55cdbc317f20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f306c0058c0 tx=0x7f306c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 -- 172.21.15.86:0/1200396284 shutdown_connections 2024-04-16T05:36:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cdbc31f990 0x55cdbc321db0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdbc2a4110 0x55cdbc311df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc29b530 0x55cdbc317f20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f3064ff9640 0 client.16002 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 --2- 172.21.15.86:0/1200396284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cdbc29ac10 0x55cdbc3179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 -- 172.21.15.86:0/1200396284 >> 172.21.15.86:0/1200396284 conn(0x55cdbc295340 msgr2=0x55cdbc294460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 -- 172.21.15.86:0/1200396284 shutdown_connections 2024-04-16T05:36:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 1 -- 172.21.15.86:0/1200396284 wait complete. 2024-04-16T05:36:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30863bb740 20 client.16002 trim_cache size 0 max 16384 2024-04-16T05:36:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-16T05:36:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-16T05:36:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-16T05:36:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-16T05:36:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30837cd640 1 -- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30680a3f70 msgr2=0x7f30680a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30837cd640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30680a3f70 0x7f30680a6430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3074032530 tx=0x7f3074048000 comp rx=0 tx=0).stop 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30837cd640 1 -- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 msgr2=0x7f307c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30837cd640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 0x7f307c198c80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f306c00c890 tx=0x7f306c00cd50 comp rx=0 tx=0).stop 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.076+0000 7f30837cd640 1 -- 172.21.15.86:0/4142126778 shutdown_connections 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.077+0000 7f30837cd640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c107060 0x7f307c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.077+0000 7f30837cd640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c101c10 0x7f307c198c80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.077+0000 7f30837cd640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30680a3f70 0x7f30680a6430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.077+0000 7f30837cd640 1 --2- 172.21.15.86:0/4142126778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f307c101250 0x7f307c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.077+0000 7f30837cd640 1 -- 172.21.15.86:0/4142126778 >> 172.21.15.86:0/4142126778 conn(0x7f307c076d00 msgr2=0x7f307c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.077+0000 7f30837cd640 1 -- 172.21.15.86:0/4142126778 shutdown_connections 2024-04-16T05:36:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:06.077+0000 7f30837cd640 1 -- 172.21.15.86:0/4142126778 wait complete. 2024-04-16T05:36:06.958 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:06 smithi194 ceph-mon[35865]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-16T05:36:06.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:06 smithi086 ceph-mon[32028]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-16T05:36:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:06 smithi196 ceph-mon[36373]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-16T05:36:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:08 smithi086 ceph-mon[32028]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-16T05:36:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:08 smithi196 ceph-mon[36373]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-16T05:36:09.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:08 smithi194 ceph-mon[35865]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-16T05:36:10.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:10 smithi086 ceph-mon[32028]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-04-16T05:36:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:10 smithi196 ceph-mon[36373]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-04-16T05:36:11.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:10 smithi194 ceph-mon[35865]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-04-16T05:36:12.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:12 smithi086 ceph-mon[32028]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-16T05:36:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:12 smithi196 ceph-mon[36373]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-16T05:36:13.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:12 smithi194 ceph-mon[35865]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-16T05:36:14.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:14 smithi086 ceph-mon[32028]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-16T05:36:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:14 smithi196 ceph-mon[36373]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-16T05:36:15.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:14 smithi194 ceph-mon[35865]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-16T05:36:16.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:16 smithi086 ceph-mon[32028]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-16T05:36:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:16 smithi196 ceph-mon[36373]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-16T05:36:17.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:16 smithi194 ceph-mon[35865]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-16T05:36:18.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:18 smithi086 ceph-mon[32028]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-16T05:36:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:18 smithi196 ceph-mon[36373]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-16T05:36:19.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:18 smithi194 ceph-mon[35865]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-16T05:36:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.166+0000 7f002d17c640 1 -- 172.21.15.86:0/3508543089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028102260 msgr2=0x7f0028102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.166+0000 7f002d17c640 1 --2- 172.21.15.86:0/3508543089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028102260 0x7f0028102640 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0018009900 tx=0x7f001802bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.166+0000 7f002d17c640 1 -- 172.21.15.86:0/3508543089 shutdown_connections 2024-04-16T05:36:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.166+0000 7f002d17c640 1 --2- 172.21.15.86:0/3508543089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028107330 0x7f002810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.166+0000 7f002d17c640 1 --2- 172.21.15.86:0/3508543089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0028102c10 0x7f0028106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.166+0000 7f002d17c640 1 --2- 172.21.15.86:0/3508543089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028102260 0x7f0028102640 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.166+0000 7f002d17c640 1 -- 172.21.15.86:0/3508543089 >> 172.21.15.86:0/3508543089 conn(0x7f00280fdac0 msgr2=0x7f00280ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.166+0000 7f002d17c640 1 -- 172.21.15.86:0/3508543089 shutdown_connections 2024-04-16T05:36:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.166+0000 7f002d17c640 1 -- 172.21.15.86:0/3508543089 wait complete. 2024-04-16T05:36:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f002d17c640 1 Processor -- start 2024-04-16T05:36:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f002d17c640 1 -- start start 2024-04-16T05:36:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f002d17c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028102260 0x7f002819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f002d17c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028107330 0x7f002819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f002d17c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00281986f0 0x7f0028198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f002d17c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00281123f0 con 0x7f0028107330 2024-04-16T05:36:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f002d17c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0028112270 con 0x7f00281986f0 2024-04-16T05:36:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f002d17c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0028112570 con 0x7f0028102260 2024-04-16T05:36:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f001e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028107330 0x7f002819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f001e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028107330 0x7f002819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38318/0 (socket says 172.21.15.86:38318) 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f001e59c640 1 -- 172.21.15.86:0/2478538791 learned_addr learned my addr 172.21.15.86:0/2478538791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.167+0000 7f0026d9d640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028102260 0x7f002819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f002759e640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00281986f0 0x7f0028198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f001e59c640 1 -- 172.21.15.86:0/2478538791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028102260 msgr2=0x7f002819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f001e59c640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028102260 0x7f002819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f001e59c640 1 -- 172.21.15.86:0/2478538791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00281986f0 msgr2=0x7f0028198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f001e59c640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00281986f0 0x7f0028198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f001e59c640 1 -- 172.21.15.86:0/2478538791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00281992f0 con 0x7f0028107330 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f002759e640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00281986f0 0x7f0028198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f001e59c640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028107330 0x7f002819eb20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f001000d770 tx=0x7f001000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f0024d99640 1 -- 172.21.15.86:0/2478538791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010017070 con 0x7f0028107330 2024-04-16T05:36:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f0024d99640 1 -- 172.21.15.86:0/2478538791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0010009c30 con 0x7f0028107330 2024-04-16T05:36:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f002d17c640 1 -- 172.21.15.86:0/2478538791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00281a5380 con 0x7f0028107330 2024-04-16T05:36:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f0024d99640 1 -- 172.21.15.86:0/2478538791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001000f620 con 0x7f0028107330 2024-04-16T05:36:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.168+0000 7f002d17c640 1 -- 172.21.15.86:0/2478538791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00281a57e0 con 0x7f0028107330 2024-04-16T05:36:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.170+0000 7f0024d99640 1 -- 172.21.15.86:0/2478538791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f001001a070 con 0x7f0028107330 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.171+0000 7f002fd64740 1 Processor -- start 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.171+0000 7f002fd64740 1 -- start start 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.171+0000 7f0024d99640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f00140a3ff0 0x7f00140a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.171+0000 7f0024d99640 1 -- 172.21.15.86:0/2478538791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f00100d09a0 con 0x7f0028107330 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.171+0000 7f002759e640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f00140a3ff0 0x7f00140a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.171+0000 7f002fd64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.171+0000 7f002fd64740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.171+0000 7f002fd64740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f002fd64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55befb44b7d0 con 0x55befb4e6c10 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f002fd64740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55befb3e8f70 con 0x55befb4e7530 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f002fd64740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55befb447790 con 0x55befb4f0110 2024-04-16T05:36:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f0026d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f0026d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4e6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38320/0 (socket says 172.21.15.86:38320) 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f0026d9d640 1 -- 172.21.15.86:0/4131400657 learned_addr learned my addr 172.21.15.86:0/4131400657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f001e59c640 1 --2- 172.21.15.86:0/4131400657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f0026d9d640 1 -- 172.21.15.86:0/4131400657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 msgr2=0x55befb4f2500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f0026d9d640 1 --2- 172.21.15.86:0/4131400657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4f2500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f0026d9d640 1 -- 172.21.15.86:0/4131400657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 msgr2=0x55befb4efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f0026d9d640 1 --2- 172.21.15.86:0/4131400657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4efbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f0026d9d640 1 -- 172.21.15.86:0/4131400657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55befb15e210 con 0x55befb4e6c10 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f002759e640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f00140a3ff0 0x7f00140a64b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0020008ca0 tx=0x7f0020005f70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f0026d9d640 1 --2- 172.21.15.86:0/4131400657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4e6ff0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0018009900 tx=0x7f001802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811d75a665ba43f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f001d59a640 1 -- 172.21.15.86:0/4131400657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001802cb00 con 0x55befb4e6c10 2024-04-16T05:36:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f001d59a640 1 -- 172.21.15.86:0/4131400657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f001802cca0 con 0x55befb4e6c10 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.172+0000 7f001d59a640 1 -- 172.21.15.86:0/4131400657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00180048e0 con 0x55befb4e6c10 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 -- 172.21.15.86:0/4131400657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 msgr2=0x55befb4e6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 --2- 172.21.15.86:0/4131400657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4e6ff0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0018009900 tx=0x7f001802c650 comp rx=0 tx=0).stop 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 -- 172.21.15.86:0/4131400657 shutdown_connections 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 --2- 172.21.15.86:0/4131400657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 --2- 172.21.15.86:0/4131400657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 --2- 172.21.15.86:0/4131400657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4e6ff0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 -- 172.21.15.86:0/4131400657 >> 172.21.15.86:0/4131400657 conn(0x55befb4e1340 msgr2=0x55befb4e1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 -- 172.21.15.86:0/4131400657 shutdown_connections 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 -- 172.21.15.86:0/4131400657 wait complete. 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.173+0000 7f002fd64740 1 -- start start 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f002fd64740 1 Processor -- start 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f002fd64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f002fd64740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4ed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f002fd64740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4edb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f002fd64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55befb4f2ea0 con 0x55befb4e6c10 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f002fd64740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55befb3d3e90 con 0x55befb4e7530 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f002fd64740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55befb4f3020 con 0x55befb4f0110 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f002fd64740 10 client.0 _ll_register_callbacks cb 0x55befb407cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f0026d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f0026d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ed110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38322/0 (socket says 172.21.15.86:38322) 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f0026d9d640 1 -- 172.21.15.86:0/3230705293 learned_addr learned my addr 172.21.15.86:0/3230705293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f001e59c640 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f002759e640 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4edb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f0026d9d640 1 -- 172.21.15.86:0/3230705293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 msgr2=0x55befb4edb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f0026d9d640 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4edb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f0026d9d640 1 -- 172.21.15.86:0/3230705293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 msgr2=0x55befb4ed650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f0026d9d640 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4ed650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f0026d9d640 1 -- 172.21.15.86:0/3230705293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55befb22a620 con 0x55befb4e6c10 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f0026d9d640 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ed110 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f001802c170 tx=0x7f0018004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da407812142e6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.174+0000 7f000e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f000e7fc640 1 -- 172.21.15.86:0/3230705293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001804d020 con 0x55befb4e6c10 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f000e7fc640 1 -- 172.21.15.86:0/3230705293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0018003a00 con 0x55befb4e6c10 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f002fd64740 1 -- 172.21.15.86:0/3230705293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f001802cca0 con 0x55befb4e6c10 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f000e7fc640 1 -- 172.21.15.86:0/3230705293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0018058050 con 0x55befb4e6c10 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f002fd64740 10 client.16014 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f002fd64740 1 -- 172.21.15.86:0/3230705293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55befb4ff130 con 0x55befb4e6c10 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f000e7fc640 1 -- 172.21.15.86:0/3230705293 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f001802cca0 con 0x55befb4e6c10 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f000e7fc640 1 -- 172.21.15.86:0/3230705293 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0018034b50 con 0x55befb4e6c10 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f002fd64740 10 client.16014 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f002fd64740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f002fd64740 20 client.16014 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f002fd64740 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55befb500480 0x55befb502870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f002fd64740 4 client.16014 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.175+0000 7f002fd64740 1 -- 172.21.15.86:0/3230705293 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55befb3eb380 con 0x55befb500480 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.176+0000 7f001e59c640 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55befb500480 0x55befb502870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.176+0000 7f001e59c640 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55befb500480 0x55befb502870 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896c185489b49e6e server_cookie=c086437adb5a86e1 in_seq=0 out_seq=0 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.176+0000 7f000e7fc640 10 client.16014 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f000e7fc640 1 -- 172.21.15.86:0/3230705293 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55befb3eb380 con 0x55befb500480 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f000e7fc640 10 client.16014 handle_command_reply: tid=0 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 client.16014 shutdown 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 -- 172.21.15.86:0/3230705293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 msgr2=0x55befb4ed110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ed110 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f001802c170 tx=0x7f0018004270 comp rx=0 tx=0).stop 2024-04-16T05:36:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 -- 172.21.15.86:0/3230705293 shutdown_connections 2024-04-16T05:36:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55befb500480 0x55befb502870 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4edb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f000e7fc640 0 client.16014 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4ed650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 --2- 172.21.15.86:0/3230705293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ed110 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 -- 172.21.15.86:0/3230705293 >> 172.21.15.86:0/3230705293 conn(0x55befb4e1340 msgr2=0x55befb4e3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 -- 172.21.15.86:0/3230705293 shutdown_connections 2024-04-16T05:36:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 1 -- 172.21.15.86:0/3230705293 wait complete. 2024-04-16T05:36:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.177+0000 7f002fd64740 20 client.16014 trim_cache size 0 max 16384 2024-04-16T05:36:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f002fd64740 1 Processor -- start 2024-04-16T05:36:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f002fd64740 1 -- start start 2024-04-16T05:36:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f002fd64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f002fd64740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f002fd64740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f002fd64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00281af370 con 0x55befb4e6c10 2024-04-16T05:36:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f002fd64740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f00281b0eb0 con 0x55befb4e7530 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f002fd64740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00281afa40 con 0x55befb4f0110 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f0026d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f0026d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38336/0 (socket says 172.21.15.86:38336) 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f0026d9d640 1 -- 172.21.15.86:0/3222050958 learned_addr learned my addr 172.21.15.86:0/3222050958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f001e59c640 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.192+0000 7f002759e640 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f0026d9d640 1 -- 172.21.15.86:0/3222050958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 msgr2=0x55befb4e2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f0026d9d640 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4e2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f0026d9d640 1 -- 172.21.15.86:0/3222050958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 msgr2=0x55befb4fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f0026d9d640 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4fbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f0026d9d640 1 -- 172.21.15.86:0/3222050958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55befb4e5f90 con 0x55befb4e6c10 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f001e59c640 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f0026d9d640 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ef840 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0018000c00 tx=0x7f001802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da29a76860c0c46 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f001cd99640 1 -- 172.21.15.86:0/3222050958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001804d070 con 0x55befb4e6c10 2024-04-16T05:36:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f001cd99640 1 -- 172.21.15.86:0/3222050958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0018005620 con 0x55befb4e6c10 2024-04-16T05:36:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.193+0000 7f001cd99640 1 -- 172.21.15.86:0/3222050958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018034a40 con 0x55befb4e6c10 2024-04-16T05:36:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 -- 172.21.15.86:0/3222050958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 msgr2=0x55befb4ef840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ef840 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0018000c00 tx=0x7f001802c650 comp rx=0 tx=0).stop 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 -- 172.21.15.86:0/3222050958 shutdown_connections 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4f0110 0x55befb4e2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 --2- 172.21.15.86:0/3222050958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4e6c10 0x55befb4ef840 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 -- 172.21.15.86:0/3222050958 >> 172.21.15.86:0/3222050958 conn(0x55befb4e1340 msgr2=0x55befb4e2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 -- 172.21.15.86:0/3222050958 shutdown_connections 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 -- 172.21.15.86:0/3222050958 wait complete. 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 -- start start 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.194+0000 7f002fd64740 1 Processor -- start 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002fd64740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4e6c10 0x55befb4f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002fd64740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002fd64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4f0110 0x55befb4f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002fd64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00281b07e0 con 0x55befb4f0110 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002fd64740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f00281b0110 con 0x55befb4e7530 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002fd64740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00281ad330 con 0x55befb4e6c10 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002fd64740 10 client.0 _ll_register_callbacks cb 0x7f0018005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4f0110 0x55befb4f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4f0110 0x55befb4f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38352/0 (socket says 172.21.15.86:38352) 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002759e640 1 -- 172.21.15.86:0/4159132365 learned_addr learned my addr 172.21.15.86:0/4159132365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f0026d9d640 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4e6c10 0x55befb4f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f001e59c640 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002759e640 1 -- 172.21.15.86:0/4159132365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4e6c10 msgr2=0x55befb4f86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002759e640 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4e6c10 0x55befb4f86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002759e640 1 -- 172.21.15.86:0/4159132365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 msgr2=0x55befb4f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002759e640 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002759e640 1 -- 172.21.15.86:0/4159132365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55befb4f4810 con 0x55befb4f0110 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002759e640 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4f0110 0x55befb4f3f90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f002000fa30 tx=0x7f0020004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0ca0247d99546c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f000e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f000e7fc640 1 -- 172.21.15.86:0/4159132365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002001cb40 con 0x55befb4f0110 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.195+0000 7f002fd64740 1 -- 172.21.15.86:0/4159132365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0018005620 con 0x55befb4f0110 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f000e7fc640 1 -- 172.21.15.86:0/4159132365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f002001cce0 con 0x55befb4f0110 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f000e7fc640 1 -- 172.21.15.86:0/4159132365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0020026430 con 0x55befb4f0110 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f002fd64740 10 client.16020 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f002fd64740 1 -- 172.21.15.86:0/4159132365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55befb56acb0 con 0x55befb4f0110 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f000e7fc640 1 -- 172.21.15.86:0/4159132365 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0018005620 con 0x55befb4f0110 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f000e7fc640 1 -- 172.21.15.86:0/4159132365 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f002001d760 con 0x55befb4f0110 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f002fd64740 10 client.16020 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f002fd64740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f002fd64740 20 client.16020 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f002fd64740 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55befb56b3d0 0x55befb56d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f002fd64740 4 client.16020 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:36:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f002fd64740 1 -- 172.21.15.86:0/4159132365 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55befb4591f0 con 0x55befb56b3d0 2024-04-16T05:36:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.196+0000 7f0026d9d640 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55befb56b3d0 0x55befb56d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.197+0000 7f0026d9d640 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55befb56b3d0 0x55befb56d7f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a807f90ea6c6fc0 server_cookie=c8c6579d082e08d1 in_seq=0 out_seq=0 2024-04-16T05:36:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.197+0000 7f000e7fc640 10 client.16020 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.200+0000 7f000e7fc640 1 -- 172.21.15.86:0/4159132365 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55befb4591f0 con 0x55befb56b3d0 2024-04-16T05:36:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.200+0000 7f000e7fc640 10 client.16020 handle_command_reply: tid=0 2024-04-16T05:36:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.200+0000 7f002fd64740 1 client.16020 shutdown 2024-04-16T05:36:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 -- 172.21.15.86:0/4159132365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4f0110 msgr2=0x55befb4f3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4f0110 0x55befb4f3f90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f002000fa30 tx=0x7f0020004a40 comp rx=0 tx=0).stop 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 -- 172.21.15.86:0/4159132365 shutdown_connections 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55befb56b3d0 0x55befb56d7f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f000e7fc640 0 client.16020 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55befb4f0110 0x55befb4f3f90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55befb4e7530 0x55befb4f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 --2- 172.21.15.86:0/4159132365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55befb4e6c10 0x55befb4f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 -- 172.21.15.86:0/4159132365 >> 172.21.15.86:0/4159132365 conn(0x55befb4e1340 msgr2=0x55befb4e0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 -- 172.21.15.86:0/4159132365 shutdown_connections 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 1 -- 172.21.15.86:0/4159132365 wait complete. 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002fd64740 20 client.16020 trim_cache size 0 max 16384 2024-04-16T05:36:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-16T05:36:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-16T05:36:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-16T05:36:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-16T05:36:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 -- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f00140a3ff0 msgr2=0x7f00140a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f00140a3ff0 0x7f00140a64b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0020008ca0 tx=0x7f0020005f70 comp rx=0 tx=0).stop 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 -- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028107330 msgr2=0x7f002819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028107330 0x7f002819eb20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f001000d770 tx=0x7f001000dc30 comp rx=0 tx=0).stop 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 -- 172.21.15.86:0/2478538791 shutdown_connections 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f00140a3ff0 0x7f00140a64b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00281986f0 0x7f0028198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0028107330 0x7f002819eb20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 --2- 172.21.15.86:0/2478538791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028102260 0x7f002819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.201+0000 7f002d17c640 1 -- 172.21.15.86:0/2478538791 >> 172.21.15.86:0/2478538791 conn(0x7f00280fdac0 msgr2=0x7f00280ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.202+0000 7f002d17c640 1 -- 172.21.15.86:0/2478538791 shutdown_connections 2024-04-16T05:36:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:19.202+0000 7f002d17c640 1 -- 172.21.15.86:0/2478538791 wait complete. 2024-04-16T05:36:20.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:20 smithi086 ceph-mon[32028]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-04-16T05:36:20.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:20 smithi196 ceph-mon[36373]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-04-16T05:36:21.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:20 smithi194 ceph-mon[35865]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-04-16T05:36:22.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:21 smithi194 ceph-mon[35865]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:36:22.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:21 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:36:22.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:21 smithi086 ceph-mon[32028]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:36:22.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:21 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:36:22.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:21 smithi196 ceph-mon[36373]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:36:22.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:21 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:36:23.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:23 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:23 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:23 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:23 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:23 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:36:23.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:23 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:36:23.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:23 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:23 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:23 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:23 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:23 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:23 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:36:23.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:23 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:36:23.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:23 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:23 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:23 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:23 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:23 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:23.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:23 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:36:23.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:23 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:36:23.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:23 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:36:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:24 smithi194 ceph-mon[35865]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:36:24.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:24 smithi086 ceph-mon[32028]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:36:24.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:24 smithi196 ceph-mon[36373]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:36:26.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:26 smithi194 ceph-mon[35865]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:36:26.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:26 smithi086 ceph-mon[32028]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:36:26.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:26 smithi196 ceph-mon[36373]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:36:27.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:36:27.904 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:36:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.037+0000 7f30d268f640 1 -- 172.21.15.86:0/3687855299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107390 msgr2=0x7f30cc10ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.037+0000 7f30d268f640 1 --2- 172.21.15.86:0/3687855299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107390 0x7f30cc10ef10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009ca0 tx=0x7f30c402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.037+0000 7f30d268f640 1 -- 172.21.15.86:0/3687855299 shutdown_connections 2024-04-16T05:36:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.037+0000 7f30d268f640 1 --2- 172.21.15.86:0/3687855299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107390 0x7f30cc10ef10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.037+0000 7f30d268f640 1 --2- 172.21.15.86:0/3687855299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30cc101130 0x7f30cc106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.037+0000 7f30d268f640 1 --2- 172.21.15.86:0/3687855299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc100780 0x7f30cc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.037+0000 7f30d268f640 1 -- 172.21.15.86:0/3687855299 >> 172.21.15.86:0/3687855299 conn(0x7f30cc0774a0 msgr2=0x7f30cc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.037+0000 7f30d268f640 1 -- 172.21.15.86:0/3687855299 shutdown_connections 2024-04-16T05:36:28.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.037+0000 7f30d268f640 1 -- 172.21.15.86:0/3687855299 wait complete. 2024-04-16T05:36:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d268f640 1 Processor -- start 2024-04-16T05:36:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d268f640 1 -- start start 2024-04-16T05:36:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d268f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc100780 0x7f30cc19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d268f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30cc107390 0x7f30cc19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d268f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc198720 0x7f30cc198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d268f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc112400 con 0x7f30cc198720 2024-04-16T05:36:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d268f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc112280 con 0x7f30cc100780 2024-04-16T05:36:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d268f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc112580 con 0x7f30cc107390 2024-04-16T05:36:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d0c2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc198720 0x7f30cc198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30cbfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc100780 0x7f30cc19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d0c2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc198720 0x7f30cc198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38956/0 (socket says 172.21.15.86:38956) 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.038+0000 7f30d0c2c640 1 -- 172.21.15.86:0/2882549552 learned_addr learned my addr 172.21.15.86:0/2882549552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30cc107390 0x7f30cc19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30cbfff640 1 -- 172.21.15.86:0/2882549552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30cc107390 msgr2=0x7f30cc19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30cbfff640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30cc107390 0x7f30cc19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30cbfff640 1 -- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc198720 msgr2=0x7f30cc198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30cbfff640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc198720 0x7f30cc198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30cbfff640 1 -- 172.21.15.86:0/2882549552 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc199320 con 0x7f30cc100780 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30d0c2c640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc198720 0x7f30cc198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30cbfff640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc100780 0x7f30cc19e610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00e9f0 tx=0x7f30bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30c97fa640 1 -- 172.21.15.86:0/2882549552 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00ccc0 con 0x7f30cc100780 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30c97fa640 1 -- 172.21.15.86:0/2882549552 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30bc010910 con 0x7f30cc100780 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30d268f640 1 -- 172.21.15.86:0/2882549552 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc1a53b0 con 0x7f30cc100780 2024-04-16T05:36:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30c97fa640 1 -- 172.21.15.86:0/2882549552 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc018960 con 0x7f30cc100780 2024-04-16T05:36:28.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.039+0000 7f30d268f640 1 -- 172.21.15.86:0/2882549552 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc1a5760 con 0x7f30cc100780 2024-04-16T05:36:28.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.043+0000 7f30c97fa640 1 -- 172.21.15.86:0/2882549552 <== mon.2 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f30bc010ac0 con 0x7f30cc100780 2024-04-16T05:36:28.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.044+0000 7f30c97fa640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30b40a3f50 0x7f30b40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.044+0000 7f30c97fa640 1 -- 172.21.15.86:0/2882549552 <== mon.2 v2:172.21.15.194:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f30bc0d1180 con 0x7f30cc100780 2024-04-16T05:36:28.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.044+0000 7f30d527e740 1 Processor -- start 2024-04-16T05:36:28.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.044+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30b40a3f50 0x7f30b40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.044+0000 7f30d527e740 1 -- start start 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.044+0000 7f30d527e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 0x55a7c115f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.044+0000 7f30d527e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 0x55a7c11534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d527e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c11539f0 0x55a7c115a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d527e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a7c10be180 con 0x55a7c11539f0 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d527e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a7c105b360 con 0x55a7c115eeb0 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d527e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a7c10ba140 con 0x55a7c1153030 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30b40a3f50 0x7f30b40a6410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30cc199f20 tx=0x7f30c0005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d0c2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c11539f0 0x55a7c115a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d0c2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c11539f0 0x55a7c115a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38964/0 (socket says 172.21.15.86:38964) 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d0c2c640 1 -- 172.21.15.86:0/3928035457 learned_addr learned my addr 172.21.15.86:0/3928035457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30cbfff640 1 --2- 172.21.15.86:0/3928035457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 0x55a7c115f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/3928035457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 0x55a7c11534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d0c2c640 1 -- 172.21.15.86:0/3928035457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 msgr2=0x55a7c11534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d0c2c640 1 --2- 172.21.15.86:0/3928035457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 0x55a7c11534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d0c2c640 1 -- 172.21.15.86:0/3928035457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 msgr2=0x55a7c115f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d0c2c640 1 --2- 172.21.15.86:0/3928035457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 0x55a7c115f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.045+0000 7f30d0c2c640 1 -- 172.21.15.86:0/3928035457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c0e9a7d0 con 0x55a7c11539f0 2024-04-16T05:36:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.046+0000 7f30d0c2c640 1 --2- 172.21.15.86:0/3928035457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c11539f0 0x55a7c115a8f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006010 tx=0x7f30c40098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8afe887e6f3807 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.046+0000 7f30b27fc640 1 -- 172.21.15.86:0/3928035457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4043070 con 0x55a7c11539f0 2024-04-16T05:36:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.046+0000 7f30b27fc640 1 -- 172.21.15.86:0/3928035457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c4032c90 con 0x55a7c11539f0 2024-04-16T05:36:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.046+0000 7f30b27fc640 1 -- 172.21.15.86:0/3928035457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4039040 con 0x55a7c11539f0 2024-04-16T05:36:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 -- 172.21.15.86:0/3928035457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c11539f0 msgr2=0x55a7c115a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 --2- 172.21.15.86:0/3928035457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c11539f0 0x55a7c115a8f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006010 tx=0x7f30c40098c0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 -- 172.21.15.86:0/3928035457 shutdown_connections 2024-04-16T05:36:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 --2- 172.21.15.86:0/3928035457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c11539f0 0x55a7c115a8f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 --2- 172.21.15.86:0/3928035457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 0x55a7c11534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 --2- 172.21.15.86:0/3928035457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 0x55a7c115f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 -- 172.21.15.86:0/3928035457 >> 172.21.15.86:0/3928035457 conn(0x55a7c1155af0 msgr2=0x55a7c1155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 -- 172.21.15.86:0/3928035457 shutdown_connections 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 -- 172.21.15.86:0/3928035457 wait complete. 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.047+0000 7f30d527e740 1 -- start start 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d527e740 1 Processor -- start 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d527e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 0x55a7c11791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d527e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 0x55a7c1179700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d527e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c115eeb0 0x55a7c1173590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d0c2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c115eeb0 0x55a7c1173590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d0c2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c115eeb0 0x55a7c1173590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38966/0 (socket says 172.21.15.86:38966) 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30cbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 0x55a7c11791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d0c2c640 1 -- 172.21.15.86:0/75499113 learned_addr learned my addr 172.21.15.86:0/75499113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 0x55a7c1179700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a7c115b070 con 0x55a7c115eeb0 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a7c10772c0 con 0x55a7c11539f0 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.048+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a7c115b1f0 con 0x55a7c1153030 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30d0c2c640 1 -- 172.21.15.86:0/75499113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 msgr2=0x55a7c11791c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30d0c2c640 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 0x55a7c11791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30d0c2c640 1 -- 172.21.15.86:0/75499113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 msgr2=0x55a7c1179700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30d0c2c640 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 0x55a7c1179700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30d0c2c640 1 -- 172.21.15.86:0/75499113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c0dd03f0 con 0x55a7c115eeb0 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 0x55a7c1179700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30d0c2c640 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c115eeb0 0x55a7c1173590 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009bf0 tx=0x7f30c4032b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88876ca4c3bfecb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30d527e740 10 client.0 _ll_register_callbacks cb 0x55a7c105e090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.049+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c4032c90 con 0x55a7c115eeb0 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30ab7fe640 1 -- 172.21.15.86:0/75499113 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4057050 con 0x55a7c115eeb0 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30ab7fe640 1 -- 172.21.15.86:0/75499113 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f30c40347f0 con 0x55a7c115eeb0 2024-04-16T05:36:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30ab7fe640 1 -- 172.21.15.86:0/75499113 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30c404c030 con 0x55a7c115eeb0 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30d527e740 10 client.16032 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55a7c0d900f0 con 0x55a7c115eeb0 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30ab7fe640 1 -- 172.21.15.86:0/75499113 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f30c405a020 con 0x55a7c115eeb0 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30d527e740 10 client.16032 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30d527e740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30d527e740 20 client.16032 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30d527e740 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a7c11830e0 0x55a7c11854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30d527e740 4 client.16032 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.050+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7c10cbba0 con 0x55a7c11830e0 2024-04-16T05:36:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.051+0000 7f30cbfff640 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a7c11830e0 0x55a7c11854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.051+0000 7f30ab7fe640 10 client.16032 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.051+0000 7f30cbfff640 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a7c11830e0 0x55a7c11854d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cddb1c9ca39ee6 server_cookie=ef83fcb5467bb4c9 in_seq=0 out_seq=0 2024-04-16T05:36:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30ab7fe640 1 -- 172.21.15.86:0/75499113 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7c10cbba0 con 0x55a7c11830e0 2024-04-16T05:36:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30ab7fe640 10 client.16032 handle_command_reply: tid=0 2024-04-16T05:36:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 client.16032 shutdown 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c115eeb0 msgr2=0x55a7c1173590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c115eeb0 0x55a7c1173590 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009bf0 tx=0x7f30c4032b50 comp rx=0 tx=0).stop 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 shutdown_connections 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a7c11830e0 0x55a7c11854d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30ab7fe640 0 client.16032 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c115eeb0 0x55a7c1173590 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 0x55a7c1179700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 --2- 172.21.15.86:0/75499113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c1153030 0x55a7c11791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 >> 172.21.15.86:0/75499113 conn(0x55a7c1155af0 msgr2=0x55a7c1152760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 shutdown_connections 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 1 -- 172.21.15.86:0/75499113 wait complete. 2024-04-16T05:36:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.052+0000 7f30d527e740 20 client.16032 trim_cache size 0 max 16384 2024-04-16T05:36:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30d527e740 1 Processor -- start 2024-04-16T05:36:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30d527e740 1 -- start start 2024-04-16T05:36:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30d527e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c115a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30d527e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 0x55a7c11793e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30d527e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c115eeb0 0x55a7c115c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30d527e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1ad220 con 0x55a7c1153030 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30d527e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1aed60 con 0x55a7c11539f0 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30d527e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1ad8f0 con 0x55a7c115eeb0 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c115a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.087+0000 7f30cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c115a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38972/0 (socket says 172.21.15.86:38972) 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30cbfff640 1 -- 172.21.15.86:0/3824528864 learned_addr learned my addr 172.21.15.86:0/3824528864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/3824528864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 0x55a7c11793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30d0c2c640 1 --2- 172.21.15.86:0/3824528864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c115eeb0 0x55a7c115c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30cbfff640 1 -- 172.21.15.86:0/3824528864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c115eeb0 msgr2=0x55a7c115c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30cbfff640 1 --2- 172.21.15.86:0/3824528864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c115eeb0 0x55a7c115c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30cbfff640 1 -- 172.21.15.86:0/3824528864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 msgr2=0x55a7c11793e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30cbfff640 1 --2- 172.21.15.86:0/3824528864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 0x55a7c11793e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30cbfff640 1 -- 172.21.15.86:0/3824528864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c1185b00 con 0x55a7c1153030 2024-04-16T05:36:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30cbfff640 1 --2- 172.21.15.86:0/3824528864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c115a890 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0a52b0 tx=0x7f30bc009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23dde4e46a457dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30b1ffb640 1 -- 172.21.15.86:0/3824528864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0a0f20 con 0x55a7c1153030 2024-04-16T05:36:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30b1ffb640 1 -- 172.21.15.86:0/3824528864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30bc0a57e0 con 0x55a7c1153030 2024-04-16T05:36:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.088+0000 7f30b1ffb640 1 -- 172.21.15.86:0/3824528864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc09c070 con 0x55a7c1153030 2024-04-16T05:36:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.089+0000 7f30d527e740 1 -- 172.21.15.86:0/3824528864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 msgr2=0x55a7c115a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.089+0000 7f30d527e740 1 --2- 172.21.15.86:0/3824528864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c115a890 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0a52b0 tx=0x7f30bc009700 comp rx=0 tx=0).stop 2024-04-16T05:36:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.089+0000 7f30d527e740 1 -- 172.21.15.86:0/3824528864 shutdown_connections 2024-04-16T05:36:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.089+0000 7f30d527e740 1 --2- 172.21.15.86:0/3824528864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c115eeb0 0x55a7c115c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.089+0000 7f30d527e740 1 --2- 172.21.15.86:0/3824528864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c11539f0 0x55a7c11793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.089+0000 7f30d527e740 1 --2- 172.21.15.86:0/3824528864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c115a890 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.089+0000 7f30d527e740 1 -- 172.21.15.86:0/3824528864 >> 172.21.15.86:0/3824528864 conn(0x55a7c1155af0 msgr2=0x55a7c1178cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.090+0000 7f30d527e740 1 -- 172.21.15.86:0/3824528864 shutdown_connections 2024-04-16T05:36:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.090+0000 7f30d527e740 1 -- 172.21.15.86:0/3824528864 wait complete. 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.090+0000 7f30d527e740 1 -- start start 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.090+0000 7f30d527e740 1 Processor -- start 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30d527e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c117fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30d527e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c11539f0 0x55a7c117d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c117fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30d527e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 0x55a7c117d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c117fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38976/0 (socket says 172.21.15.86:38976) 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cbfff640 1 -- 172.21.15.86:0/3230895867 learned_addr learned my addr 172.21.15.86:0/3230895867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1ae690 con 0x55a7c1153030 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1adfc0 con 0x55a7c115eeb0 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1ab1e0 con 0x55a7c11539f0 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c11539f0 0x55a7c117d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30d527e740 10 client.0 _ll_register_callbacks cb 0x7f30c0009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30d0c2c640 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 0x55a7c117d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cbfff640 1 -- 172.21.15.86:0/3230895867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c11539f0 msgr2=0x55a7c117d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cbfff640 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c11539f0 0x55a7c117d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cbfff640 1 -- 172.21.15.86:0/3230895867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 msgr2=0x55a7c117d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cbfff640 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 0x55a7c117d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cbfff640 1 -- 172.21.15.86:0/3230895867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c11e85b0 con 0x55a7c1153030 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30cbfff640 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c117fdc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0ce460 tx=0x7f30bc098810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea676bc12e0cab5d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.091+0000 7f30ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30ab7fe640 1 -- 172.21.15.86:0/3230895867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0a0f20 con 0x55a7c1153030 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30ab7fe640 1 -- 172.21.15.86:0/3230895867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f30bc09af20 con 0x55a7c1153030 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30bc0a57e0 con 0x55a7c1153030 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30ab7fe640 1 -- 172.21.15.86:0/3230895867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30bc0bd040 con 0x55a7c1153030 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30d527e740 10 client.16038 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7c11e96f0 con 0x55a7c1153030 2024-04-16T05:36:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30ab7fe640 1 -- 172.21.15.86:0/3230895867 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30bc0a57e0 con 0x55a7c1153030 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30ab7fe640 1 -- 172.21.15.86:0/3230895867 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f30bc0b7070 con 0x55a7c1153030 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30d527e740 10 client.16038 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30d527e740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30d527e740 20 client.16038 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30d527e740 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a7c11e9e10 0x55a7c11ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30d527e740 4 client.16038 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.092+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a7c10c5a10 con 0x55a7c11e9e10 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.093+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a7c11e9e10 0x55a7c11ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.093+0000 7f30ab7fe640 10 client.16038 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.093+0000 7f30cb7fe640 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a7c11e9e10 0x55a7c11ec200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451fcc2fc0aea1bb server_cookie=6a788e0ff78cf615 in_seq=0 out_seq=0 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.093+0000 7f30ab7fe640 1 -- 172.21.15.86:0/3230895867 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a7c10c5a10 con 0x55a7c11e9e10 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.093+0000 7f30ab7fe640 10 client.16038 handle_command_reply: tid=0 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.093+0000 7f30d527e740 1 client.16038 shutdown 2024-04-16T05:36:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 msgr2=0x55a7c117fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c117fdc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0ce460 tx=0x7f30bc098810 comp rx=0 tx=0).stop 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 shutdown_connections 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a7c11e9e10 0x55a7c11ec200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a7c115eeb0 0x55a7c117d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30ab7fe640 0 client.16038 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7c11539f0 0x55a7c117d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 --2- 172.21.15.86:0/3230895867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7c1153030 0x55a7c117fdc0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 >> 172.21.15.86:0/3230895867 conn(0x55a7c1155af0 msgr2=0x55a7c1173530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 shutdown_connections 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 1 -- 172.21.15.86:0/3230895867 wait complete. 2024-04-16T05:36:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.094+0000 7f30d527e740 20 client.16038 trim_cache size 0 max 16384 2024-04-16T05:36:28.099 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:36:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:36:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:36:28.099 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 -- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30b40a3f50 msgr2=0x7f30b40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30b40a3f50 0x7f30b40a6410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30cc199f20 tx=0x7f30c0005f90 comp rx=0 tx=0).stop 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 -- 172.21.15.86:0/2882549552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc100780 msgr2=0x7f30cc19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc100780 0x7f30cc19e610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00e9f0 tx=0x7f30bc00eeb0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 -- 172.21.15.86:0/2882549552 shutdown_connections 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc198720 0x7f30cc198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30cc107390 0x7f30cc19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f30b40a3f50 0x7f30b40a6410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 --2- 172.21.15.86:0/2882549552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc100780 0x7f30cc19e610 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00e9f0 tx=0x7f30bc00eeb0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 -- 172.21.15.86:0/2882549552 >> 172.21.15.86:0/2882549552 conn(0x7f30cc0774a0 msgr2=0x7f30cc105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 -- 172.21.15.86:0/2882549552 shutdown_connections 2024-04-16T05:36:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.095+0000 7f30d268f640 1 -- 172.21.15.86:0/2882549552 wait complete. 2024-04-16T05:36:28.108 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:36:28.108 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:36:28.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.219+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1335027153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4101b00 msgr2=0x7fa4c4105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.219+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1335027153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4101b00 0x7fa4c4105b40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009900 tx=0x7fa4b002bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.219+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1335027153 shutdown_connections 2024-04-16T05:36:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.219+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1335027153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c4106220 0x7fa4c410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.219+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1335027153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4101b00 0x7fa4c4105b40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.219+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1335027153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4c4101150 0x7fa4c4101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.219+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1335027153 >> 172.21.15.86:0/1335027153 conn(0x7fa4c4078000 msgr2=0x7fa4c4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.220+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1335027153 shutdown_connections 2024-04-16T05:36:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.220+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1335027153 wait complete. 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.220+0000 7fa4cae2c640 1 Processor -- start 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.220+0000 7fa4cae2c640 1 -- start start 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.220+0000 7fa4cae2c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4c4101150 0x7fa4c41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.220+0000 7fa4cae2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4106220 0x7fa4c41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4cae2c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c419a910 0x7fa4c419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4cae2c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c41123b0 con 0x7fa4c4106220 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4cae2c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4112230 con 0x7fa4c4101150 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4cae2c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4112530 con 0x7fa4c419a910 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4106220 0x7fa4c41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4106220 0x7fa4c41a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38996/0 (socket says 172.21.15.86:38996) 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c3fff640 1 -- 172.21.15.86:0/1080631836 learned_addr learned my addr 172.21.15.86:0/1080631836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c93c9640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c419a910 0x7fa4c419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c8bc8640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4c4101150 0x7fa4c41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c3fff640 1 -- 172.21.15.86:0/1080631836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c419a910 msgr2=0x7fa4c419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c3fff640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c419a910 0x7fa4c419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c3fff640 1 -- 172.21.15.86:0/1080631836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4c4101150 msgr2=0x7fa4c41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c3fff640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4c4101150 0x7fa4c41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c3fff640 1 -- 172.21.15.86:0/1080631836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c419b430 con 0x7fa4c4106220 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c8bc8640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4c4101150 0x7fa4c41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c3fff640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4106220 0x7fa4c41a0ca0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0036040 tx=0x7fa4b002cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4c1ffb640 1 -- 172.21.15.86:0/1080631836 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0041070 con 0x7fa4c4106220 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.221+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c419b6c0 con 0x7fa4c4106220 2024-04-16T05:36:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.222+0000 7fa4c1ffb640 1 -- 172.21.15.86:0/1080631836 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b00054e0 con 0x7fa4c4106220 2024-04-16T05:36:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.222+0000 7fa4c1ffb640 1 -- 172.21.15.86:0/1080631836 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b004b060 con 0x7fa4c4106220 2024-04-16T05:36:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.222+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c41a7910 con 0x7fa4c4106220 2024-04-16T05:36:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.223+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c4068e20 con 0x7fa4c4106220 2024-04-16T05:36:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.224+0000 7fa4c1ffb640 1 -- 172.21.15.86:0/1080631836 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa4b0044070 con 0x7fa4c4106220 2024-04-16T05:36:28.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.225+0000 7fa4c1ffb640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa4b40a3ed0 0x7fa4b40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.225+0000 7fa4c1ffb640 1 -- 172.21.15.86:0/1080631836 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa4b00f1ff0 con 0x7fa4c4106220 2024-04-16T05:36:28.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.234+0000 7fa4c8bc8640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa4b40a3ed0 0x7fa4b40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.234+0000 7fa4c1ffb640 1 -- 172.21.15.86:0/1080631836 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4b00be1f0 con 0x7fa4c4106220 2024-04-16T05:36:28.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.234+0000 7fa4c8bc8640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa4b40a3ed0 0x7fa4b40a6390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4c419ba50 tx=0x7fa4b8005e70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.462+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4c4101b30 con 0x7fa4c4106220 2024-04-16T05:36:28.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.463+0000 7fa4c1ffb640 1 -- 172.21.15.86:0/1080631836 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fa4b00bdaf0 con 0x7fa4c4106220 2024-04-16T05:36:28.462 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:36:28.462 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:36:28.462 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:36:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa4b40a3ed0 msgr2=0x7fa4b40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa4b40a3ed0 0x7fa4b40a6390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4c419ba50 tx=0x7fa4b8005e70 comp rx=0 tx=0).stop 2024-04-16T05:36:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4106220 msgr2=0x7fa4c41a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4106220 0x7fa4c41a0ca0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0036040 tx=0x7fa4b002cbf0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 shutdown_connections 2024-04-16T05:36:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c419a910 0x7fa4c419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4c4106220 0x7fa4c41a0ca0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa4b40a3ed0 0x7fa4b40a6390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 --2- 172.21.15.86:0/1080631836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4c4101150 0x7fa4c41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 >> 172.21.15.86:0/1080631836 conn(0x7fa4c4078000 msgr2=0x7fa4c40ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 shutdown_connections 2024-04-16T05:36:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.465+0000 7fa4cae2c640 1 -- 172.21.15.86:0/1080631836 wait complete. 2024-04-16T05:36:28.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:36:28.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.587+0000 7fdb608ba640 1 -- 172.21.15.86:0/1056516499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c106200 msgr2=0x7fdb5c10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.587+0000 7fdb608ba640 1 --2- 172.21.15.86:0/1056516499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c106200 0x7fdb5c10ddd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009ca0 tx=0x7fdb5402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.587+0000 7fdb608ba640 1 -- 172.21.15.86:0/1056516499 shutdown_connections 2024-04-16T05:36:28.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.587+0000 7fdb608ba640 1 --2- 172.21.15.86:0/1056516499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c106200 0x7fdb5c10ddd0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.587+0000 7fdb608ba640 1 --2- 172.21.15.86:0/1056516499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c101ae0 0x7fdb5c105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.587+0000 7fdb608ba640 1 --2- 172.21.15.86:0/1056516499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb5c101130 0x7fdb5c101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.587+0000 7fdb608ba640 1 -- 172.21.15.86:0/1056516499 >> 172.21.15.86:0/1056516499 conn(0x7fdb5c0780b0 msgr2=0x7fdb5c0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.587+0000 7fdb608ba640 1 -- 172.21.15.86:0/1056516499 shutdown_connections 2024-04-16T05:36:28.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.587+0000 7fdb608ba640 1 -- 172.21.15.86:0/1056516499 wait complete. 2024-04-16T05:36:28.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.588+0000 7fdb608ba640 1 Processor -- start 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.588+0000 7fdb608ba640 1 -- start start 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb608ba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c101130 0x7fdb5c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb608ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb5c106200 0x7fdb5c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb608ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c198620 0x7fdb5c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb608ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1123c0 con 0x7fdb5c198620 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb608ba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c112240 con 0x7fdb5c101130 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb608ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c112540 con 0x7fdb5c106200 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c198620 0x7fdb5c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c198620 0x7fdb5c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39004/0 (socket says 172.21.15.86:39004) 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5ad9d640 1 -- 172.21.15.86:0/3559887326 learned_addr learned my addr 172.21.15.86:0/3559887326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb5c106200 0x7fdb5c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c101130 0x7fdb5c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5ad9d640 1 -- 172.21.15.86:0/3559887326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb5c106200 msgr2=0x7fdb5c19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5ad9d640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb5c106200 0x7fdb5c19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5ad9d640 1 -- 172.21.15.86:0/3559887326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c101130 msgr2=0x7fdb5c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5ad9d640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c101130 0x7fdb5c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5ad9d640 1 -- 172.21.15.86:0/3559887326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c199140 con 0x7fdb5c198620 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.589+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c101130 0x7fdb5c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.590+0000 7fdb5ad9d640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c198620 0x7fdb5c198ad0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdb54037040 tx=0x7fdb54032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.590+0000 7fdb4b7fe640 1 -- 172.21.15.86:0/3559887326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54041020 con 0x7fdb5c198620 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.590+0000 7fdb608ba640 1 -- 172.21.15.86:0/3559887326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c1993d0 con 0x7fdb5c198620 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.590+0000 7fdb4b7fe640 1 -- 172.21.15.86:0/3559887326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb54034cd0 con 0x7fdb5c198620 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.590+0000 7fdb4b7fe640 1 -- 172.21.15.86:0/3559887326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54039040 con 0x7fdb5c198620 2024-04-16T05:36:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.590+0000 7fdb608ba640 1 -- 172.21.15.86:0/3559887326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c1a55d0 con 0x7fdb5c198620 2024-04-16T05:36:28.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.593+0000 7fdb63577740 1 Processor -- start 2024-04-16T05:36:28.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.593+0000 7fdb4b7fe640 1 -- 172.21.15.86:0/3559887326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fdb54044020 con 0x7fdb5c198620 2024-04-16T05:36:28.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.593+0000 7fdb63577740 1 -- start start 2024-04-16T05:36:28.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb63577740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160eb500 0x561a160eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb63577740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160d8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb63577740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb63577740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a160430e0 con 0x561a160eb500 2024-04-16T05:36:28.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb63577740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561a15fe0520 con 0x561a160ebeb0 2024-04-16T05:36:28.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb63577740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a1603f0a0 con 0x561a160d8660 2024-04-16T05:36:28.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160eb500 0x561a160eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160eb500 0x561a160eb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39020/0 (socket says 172.21.15.86:39020) 2024-04-16T05:36:28.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb59d9b640 1 -- 172.21.15.86:0/1340972464 learned_addr learned my addr 172.21.15.86:0/1340972464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb4b7fe640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdb440a3ed0 0x7fdb440a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb59d9b640 1 -- 172.21.15.86:0/1340972464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 msgr2=0x561a160d8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/1340972464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb59d9b640 1 -- 172.21.15.86:0/1340972464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 msgr2=0x561a160d8040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/1340972464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160d8040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb59d9b640 1 -- 172.21.15.86:0/1340972464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a160088d0 con 0x561a160eb500 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.594+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/1340972464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160eb500 0x561a160eb8e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000bc70 tx=0x7fdb500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ec742af3ce604e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.595+0000 7fdb4b7fe640 1 -- 172.21.15.86:0/3559887326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fdb540f2b10 con 0x7fdb5c198620 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.595+0000 7fdb5ad9d640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdb440a3ed0 0x7fdb440a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.595+0000 7fdb48ff9640 1 -- 172.21.15.86:0/1340972464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50004030 con 0x561a160eb500 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.595+0000 7fdb48ff9640 1 -- 172.21.15.86:0/1340972464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb50005500 con 0x561a160eb500 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.595+0000 7fdb48ff9640 1 -- 172.21.15.86:0/1340972464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50004d60 con 0x561a160eb500 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.595+0000 7fdb63577740 1 -- 172.21.15.86:0/1340972464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160eb500 msgr2=0x561a160eb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.595+0000 7fdb5ad9d640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdb440a3ed0 0x7fdb440a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb540efe20 tx=0x7fdb54032b00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 --2- 172.21.15.86:0/1340972464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160eb500 0x561a160eb8e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000bc70 tx=0x7fdb500107a0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 -- 172.21.15.86:0/1340972464 shutdown_connections 2024-04-16T05:36:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 --2- 172.21.15.86:0/1340972464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 --2- 172.21.15.86:0/1340972464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160d8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 --2- 172.21.15.86:0/1340972464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160eb500 0x561a160eb8e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 -- 172.21.15.86:0/1340972464 >> 172.21.15.86:0/1340972464 conn(0x561a160dab00 msgr2=0x561a160daee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 -- 172.21.15.86:0/1340972464 shutdown_connections 2024-04-16T05:36:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 -- 172.21.15.86:0/1340972464 wait complete. 2024-04-16T05:36:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 -- start start 2024-04-16T05:36:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 Processor -- start 2024-04-16T05:36:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160ef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160efad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f0010 0x561a160fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a160f2940 con 0x561a160f0010 2024-04-16T05:36:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.596+0000 7fdb63577740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561a15ffc4d0 con 0x561a160ebeb0 2024-04-16T05:36:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb63577740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a160f2ac0 con 0x561a160d8660 2024-04-16T05:36:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb63577740 10 client.0 _ll_register_callbacks cb 0x561a15fe12c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f0010 0x561a160fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f0010 0x561a160fbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39036/0 (socket says 172.21.15.86:39036) 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 -- 172.21.15.86:0/2061116750 learned_addr learned my addr 172.21.15.86:0/2061116750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 -- 172.21.15.86:0/2061116750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 msgr2=0x561a160ef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160ef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 -- 172.21.15.86:0/2061116750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 msgr2=0x561a160efad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160efad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 -- 172.21.15.86:0/2061116750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a15d553f0 con 0x561a160f0010 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f0010 0x561a160fbe20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdb50004a40 tx=0x7fdb50004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f723c888bfffc83 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.597+0000 7fdb41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb41ffb640 1 -- 172.21.15.86:0/2061116750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50024040 con 0x561a160f0010 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb63577740 1 -- 172.21.15.86:0/2061116750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb50005500 con 0x561a160f0010 2024-04-16T05:36:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb41ffb640 1 -- 172.21.15.86:0/2061116750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fdb50004ce0 con 0x561a160f0010 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb41ffb640 1 -- 172.21.15.86:0/2061116750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb50033050 con 0x561a160f0010 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb63577740 10 client.16056 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb63577740 1 -- 172.21.15.86:0/2061116750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a15e1f7d0 con 0x561a160f0010 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb41ffb640 1 -- 172.21.15.86:0/2061116750 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb50005500 con 0x561a160f0010 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb41ffb640 1 -- 172.21.15.86:0/2061116750 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb5002d4f0 con 0x561a160f0010 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb63577740 10 client.16056 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb63577740 10 client.16056 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb63577740 20 client.16056 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb63577740 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x561a16100030 0x561a16102420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb63577740 4 client.16056 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.598+0000 7fdb63577740 1 -- 172.21.15.86:0/2061116750 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a16050b00 con 0x561a16100030 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.599+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x561a16100030 0x561a16102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.599+0000 7fdb41ffb640 10 client.16056 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:36:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.599+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x561a16100030 0x561a16102420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac36130773611481 server_cookie=1f0f5c9e56d642d4 in_seq=0 out_seq=0 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb41ffb640 1 -- 172.21.15.86:0/2061116750 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a16050b00 con 0x561a16100030 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb41ffb640 10 client.16056 handle_command_reply: tid=0 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 client.16056 shutdown 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 -- 172.21.15.86:0/2061116750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f0010 msgr2=0x561a160fbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f0010 0x561a160fbe20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdb50004a40 tx=0x7fdb50004940 comp rx=0 tx=0).stop 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 -- 172.21.15.86:0/2061116750 shutdown_connections 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x561a16100030 0x561a16102420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f0010 0x561a160fbe20 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160efad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb41ffb640 0 client.16056 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:36:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 --2- 172.21.15.86:0/2061116750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 -- 172.21.15.86:0/2061116750 >> 172.21.15.86:0/2061116750 conn(0x561a160dab00 msgr2=0x561a160d7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 -- 172.21.15.86:0/2061116750 shutdown_connections 2024-04-16T05:36:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 1 -- 172.21.15.86:0/2061116750 wait complete. 2024-04-16T05:36:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.600+0000 7fdb63577740 20 client.16056 trim_cache size 0 max 16384 2024-04-16T05:36:28.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb63577740 1 Processor -- start 2024-04-16T05:36:28.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb63577740 1 -- start start 2024-04-16T05:36:28.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb63577740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160ee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb63577740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a16107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb63577740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f1dd0 0x561a16107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb63577740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ad1d0 con 0x561a160f1dd0 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb63577740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1aed10 con 0x561a160ebeb0 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb63577740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ad8a0 con 0x561a160d8660 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f1dd0 0x561a16107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb5a59c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a16107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f1dd0 0x561a16107ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39052/0 (socket says 172.21.15.86:39052) 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb59d9b640 1 -- 172.21.15.86:0/3055240691 learned_addr learned my addr 172.21.15.86:0/3055240691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.614+0000 7fdb5a59c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a16107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58296/0 (socket says 172.21.15.86:58296) 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb5a59c640 1 -- 172.21.15.86:0/3055240691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 msgr2=0x561a160ee570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/3055240691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160ee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb5a59c640 1 -- 172.21.15.86:0/3055240691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f1dd0 msgr2=0x561a16107ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/3055240691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f1dd0 0x561a16107ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb5a59c640 1 -- 172.21.15.86:0/3055240691 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a160e9160 con 0x561a160ebeb0 2024-04-16T05:36:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/3055240691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f1dd0 0x561a16107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:28.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/3055240691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a16107ab0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x561a16102a50 tx=0x7fdb4c012cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f070b901af4507b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb43fff640 1 -- 172.21.15.86:0/3055240691 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0120d0 con 0x561a160ebeb0 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb43fff640 1 -- 172.21.15.86:0/3055240691 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4c01d910 con 0x561a160ebeb0 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.615+0000 7fdb43fff640 1 -- 172.21.15.86:0/3055240691 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0106e0 con 0x561a160ebeb0 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.616+0000 7fdb63577740 1 -- 172.21.15.86:0/3055240691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 msgr2=0x561a16107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.616+0000 7fdb63577740 1 --2- 172.21.15.86:0/3055240691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a16107ab0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x561a16102a50 tx=0x7fdb4c012cb0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.616+0000 7fdb63577740 1 -- 172.21.15.86:0/3055240691 shutdown_connections 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.616+0000 7fdb63577740 1 --2- 172.21.15.86:0/3055240691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160f1dd0 0x561a16107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.616+0000 7fdb63577740 1 --2- 172.21.15.86:0/3055240691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a16107ab0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.616+0000 7fdb63577740 1 --2- 172.21.15.86:0/3055240691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160d8660 0x561a160ee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.616+0000 7fdb63577740 1 -- 172.21.15.86:0/3055240691 >> 172.21.15.86:0/3055240691 conn(0x561a160dab00 msgr2=0x561a160ede60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.616+0000 7fdb63577740 1 -- 172.21.15.86:0/3055240691 shutdown_connections 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.616+0000 7fdb63577740 1 -- 172.21.15.86:0/3055240691 wait complete. 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb63577740 1 -- start start 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb63577740 1 Processor -- start 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb63577740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160d8660 0x561a160f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb63577740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb63577740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160f1dd0 0x561a160f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb63577740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ae640 con 0x561a160d8660 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb63577740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1adf70 con 0x561a160ebeb0 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb63577740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ab190 con 0x561a160f1dd0 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160d8660 0x561a160f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160d8660 0x561a160f7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39062/0 (socket says 172.21.15.86:39062) 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb59d9b640 1 -- 172.21.15.86:0/3053000376 learned_addr learned my addr 172.21.15.86:0/3053000376 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.617+0000 7fdb63577740 10 client.0 _ll_register_callbacks cb 0x7fdb5000bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160f1dd0 0x561a160f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb59d9b640 1 -- 172.21.15.86:0/3053000376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160f1dd0 msgr2=0x561a160f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160f1dd0 0x561a160f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb59d9b640 1 -- 172.21.15.86:0/3053000376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 msgr2=0x561a160f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb59d9b640 1 -- 172.21.15.86:0/3053000376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a160f5390 con 0x561a160d8660 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb59d9b640 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160d8660 0x561a160f7d80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x561a160fedd0 tx=0x7fdb500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5125dded558408fd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb41ffb640 1 -- 172.21.15.86:0/3053000376 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5002dc60 con 0x561a160d8660 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb41ffb640 1 -- 172.21.15.86:0/3053000376 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fdb5002de00 con 0x561a160d8660 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb63577740 1 -- 172.21.15.86:0/3053000376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4c01d910 con 0x561a160d8660 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb41ffb640 1 -- 172.21.15.86:0/3053000376 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb50043050 con 0x561a160d8660 2024-04-16T05:36:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb63577740 10 client.16062 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.618+0000 7fdb63577740 1 -- 172.21.15.86:0/3053000376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a16166c30 con 0x561a160d8660 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb41ffb640 1 -- 172.21.15.86:0/3053000376 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4c01d910 con 0x561a160d8660 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb41ffb640 1 -- 172.21.15.86:0/3053000376 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb50004e00 con 0x561a160d8660 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb63577740 10 client.16062 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb63577740 10 client.16062 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb63577740 20 client.16062 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb63577740 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x561a16167270 0x561a16169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb63577740 4 client.16062 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb63577740 1 -- 172.21.15.86:0/3053000376 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a1604a970 con 0x561a16167270 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x561a16167270 0x561a16169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb41ffb640 10 client.16062 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.619+0000 7fdb5a59c640 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x561a16167270 0x561a16169690 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b129141f3b3728f server_cookie=53616e2375875ce3 in_seq=0 out_seq=0 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb41ffb640 1 -- 172.21.15.86:0/3053000376 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a1604a970 con 0x561a16167270 2024-04-16T05:36:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb41ffb640 10 client.16062 handle_command_reply: tid=0 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 client.16062 shutdown 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 -- 172.21.15.86:0/3053000376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160d8660 msgr2=0x561a160f7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160d8660 0x561a160f7d80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x561a160fedd0 tx=0x7fdb500107a0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 -- 172.21.15.86:0/3053000376 shutdown_connections 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x561a16167270 0x561a16169690 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a160f1dd0 0x561a160f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb41ffb640 0 client.16062 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a160ebeb0 0x561a160f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 --2- 172.21.15.86:0/3053000376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a160d8660 0x561a160f7d80 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 -- 172.21.15.86:0/3053000376 >> 172.21.15.86:0/3053000376 conn(0x561a160dab00 msgr2=0x561a160ee6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 -- 172.21.15.86:0/3053000376 shutdown_connections 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 1 -- 172.21.15.86:0/3053000376 wait complete. 2024-04-16T05:36:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.620+0000 7fdb63577740 20 client.16062 trim_cache size 0 max 16384 2024-04-16T05:36:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:36:28.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 -- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdb440a3ed0 msgr2=0x7fdb440a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdb440a3ed0 0x7fdb440a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb540efe20 tx=0x7fdb54032b00 comp rx=0 tx=0).stop 2024-04-16T05:36:28.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 -- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c198620 msgr2=0x7fdb5c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c198620 0x7fdb5c198ad0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdb54037040 tx=0x7fdb54032ec0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 -- 172.21.15.86:0/3559887326 shutdown_connections 2024-04-16T05:36:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c198620 0x7fdb5c198ad0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb5c106200 0x7fdb5c19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdb440a3ed0 0x7fdb440a6390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 --2- 172.21.15.86:0/3559887326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c101130 0x7fdb5c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 -- 172.21.15.86:0/3559887326 >> 172.21.15.86:0/3559887326 conn(0x7fdb5c0780b0 msgr2=0x7fdb5c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 -- 172.21.15.86:0/3559887326 shutdown_connections 2024-04-16T05:36:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.621+0000 7fdb608ba640 1 -- 172.21.15.86:0/3559887326 wait complete. 2024-04-16T05:36:28.635 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:36:28.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:28 smithi194 ceph-mon[35865]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:36:28.659 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:28 smithi086 ceph-mon[32028]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:36:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:28 smithi196 ceph-mon[36373]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:36:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.745+0000 7f624671c640 1 -- 172.21.15.86:0/848624255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f624010c5b0 msgr2=0x7f624010ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.745+0000 7f624671c640 1 --2- 172.21.15.86:0/848624255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f624010c5b0 0x7f624010ea50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6238009ca0 tx=0x7f6238030640 comp rx=0 tx=0).stop 2024-04-16T05:36:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.745+0000 7f624671c640 1 -- 172.21.15.86:0/848624255 shutdown_connections 2024-04-16T05:36:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.745+0000 7f624671c640 1 --2- 172.21.15.86:0/848624255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f624010c5b0 0x7f624010ea50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.745+0000 7f624671c640 1 --2- 172.21.15.86:0/848624255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6240103b90 0x7f624010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.745+0000 7f624671c640 1 --2- 172.21.15.86:0/848624255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240103270 0x7f6240103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.745+0000 7f624671c640 1 -- 172.21.15.86:0/848624255 >> 172.21.15.86:0/848624255 conn(0x7f62400780b0 msgr2=0x7f62400784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.745+0000 7f624671c640 1 -- 172.21.15.86:0/848624255 shutdown_connections 2024-04-16T05:36:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.746+0000 7f624671c640 1 -- 172.21.15.86:0/848624255 wait complete. 2024-04-16T05:36:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.746+0000 7f624671c640 1 Processor -- start 2024-04-16T05:36:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.746+0000 7f624671c640 1 -- start start 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.746+0000 7f624671c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240103270 0x7f6240196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.746+0000 7f624671c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6240103b90 0x7f6240196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f624671c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f624010c5b0 0x7f624019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f624671c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6240110210 con 0x7f624010c5b0 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f624671c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6240110090 con 0x7f6240103b90 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f624671c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6240110390 con 0x7f6240103270 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f6244cb9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f624010c5b0 0x7f624019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240103270 0x7f6240196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240103270 0x7f6240196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:60074/0 (socket says 172.21.15.86:60074) 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623ffff640 1 -- 172.21.15.86:0/2875498911 learned_addr learned my addr 172.21.15.86:0/2875498911 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f6244cb9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f624010c5b0 0x7f624019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39092/0 (socket says 172.21.15.86:39092) 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623f7fe640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6240103b90 0x7f6240196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623ffff640 1 -- 172.21.15.86:0/2875498911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6240103b90 msgr2=0x7f6240196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623ffff640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6240103b90 0x7f6240196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623ffff640 1 -- 172.21.15.86:0/2875498911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f624010c5b0 msgr2=0x7f624019aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623ffff640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f624010c5b0 0x7f624019aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623ffff640 1 -- 172.21.15.86:0/2875498911 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624019b620 con 0x7f6240103270 2024-04-16T05:36:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.747+0000 7f623ffff640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240103270 0x7f6240196630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9f0 tx=0x7f623000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.748+0000 7f623d7fa640 1 -- 172.21.15.86:0/2875498911 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623000cd80 con 0x7f6240103270 2024-04-16T05:36:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.748+0000 7f624671c640 1 -- 172.21.15.86:0/2875498911 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624019b910 con 0x7f6240103270 2024-04-16T05:36:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.748+0000 7f623d7fa640 1 -- 172.21.15.86:0/2875498911 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6230005500 con 0x7f6240103270 2024-04-16T05:36:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.748+0000 7f624671c640 1 -- 172.21.15.86:0/2875498911 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62401a31a0 con 0x7f6240103270 2024-04-16T05:36:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.748+0000 7f623d7fa640 1 -- 172.21.15.86:0/2875498911 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230010690 con 0x7f6240103270 2024-04-16T05:36:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.750+0000 7f623d7fa640 1 -- 172.21.15.86:0/2875498911 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f62300046c0 con 0x7f6240103270 2024-04-16T05:36:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.751+0000 7f623d7fa640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62280a3ed0 0x7f62280a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.751+0000 7f62493e4740 1 Processor -- start 2024-04-16T05:36:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.751+0000 7f623d7fa640 1 -- 172.21.15.86:0/2875498911 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f62300d1240 con 0x7f6240103270 2024-04-16T05:36:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.751+0000 7f62493e4740 1 -- start start 2024-04-16T05:36:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.751+0000 7f623f7fe640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62280a3ed0 0x7f62280a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.751+0000 7f62493e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe9201c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.751+0000 7f62493e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9213630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f62493e4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9215f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f62493e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557fe91670e0 con 0x557fe92021f0 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f62493e4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557fe9104520 con 0x557fe9213b70 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f62493e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557fe91630a0 con 0x557fe9201840 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9213630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9213630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39098/0 (socket says 172.21.15.86:39098) 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 -- 172.21.15.86:0/1674018497 learned_addr learned my addr 172.21.15.86:0/1674018497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62280a3ed0 0x7f62280a6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6240100820 tx=0x7f623400b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623ffff640 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe9201c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f6244cb9640 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9215f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 -- 172.21.15.86:0/1674018497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 msgr2=0x557fe9201c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe9201c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 -- 172.21.15.86:0/1674018497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 msgr2=0x557fe9215f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9215f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 -- 172.21.15.86:0/1674018497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fe912c8d0 con 0x557fe92021f0 2024-04-16T05:36:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f6244cb9640 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9215f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.752+0000 7f623f7fe640 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9213630 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6234010f40 tx=0x7f6234013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98e50f4d6933f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.753+0000 7f62267fc640 1 -- 172.21.15.86:0/1674018497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623401e620 con 0x557fe92021f0 2024-04-16T05:36:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.753+0000 7f62267fc640 1 -- 172.21.15.86:0/1674018497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f623401eca0 con 0x557fe92021f0 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62267fc640 1 -- 172.21.15.86:0/1674018497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62340253a0 con 0x557fe92021f0 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 -- 172.21.15.86:0/1674018497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 msgr2=0x557fe9213630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9213630 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6234010f40 tx=0x7f6234013000 comp rx=0 tx=0).stop 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 -- 172.21.15.86:0/1674018497 shutdown_connections 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9215f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9213630 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 --2- 172.21.15.86:0/1674018497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe9201c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 -- 172.21.15.86:0/1674018497 >> 172.21.15.86:0/1674018497 conn(0x557fe91fc590 msgr2=0x557fe91fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 -- 172.21.15.86:0/1674018497 shutdown_connections 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 -- 172.21.15.86:0/1674018497 wait complete. 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.754+0000 7f62493e4740 1 -- start start 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.755+0000 7f62493e4740 1 Processor -- start 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.755+0000 7f62493e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe9210a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.755+0000 7f62493e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9210fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.755+0000 7f62493e4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9211510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.755+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9210fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.755+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9210fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39100/0 (socket says 172.21.15.86:39100) 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.755+0000 7f623f7fe640 1 -- 172.21.15.86:0/2593420987 learned_addr learned my addr 172.21.15.86:0/2593420987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557fe92169d0 con 0x557fe92021f0 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f623ffff640 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe9210a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557fe91204d0 con 0x557fe9213b70 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557fe9216b50 con 0x557fe9201840 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f6244cb9640 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9211510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f62493e4740 10 client.0 _ll_register_callbacks cb 0x557fe91052c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f623f7fe640 1 -- 172.21.15.86:0/2593420987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 msgr2=0x557fe9210a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f623f7fe640 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe9210a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f623f7fe640 1 -- 172.21.15.86:0/2593420987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 msgr2=0x557fe9211510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f623f7fe640 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9211510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f623f7fe640 1 -- 172.21.15.86:0/2593420987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fe8e793f0 con 0x557fe92021f0 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f623ffff640 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe9210a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f6244cb9640 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9211510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f623f7fe640 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9210fd0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f623401d770 tx=0x7f623401d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e116154edd4d0d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f621f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f623401eca0 con 0x557fe92021f0 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f621f7fe640 1 -- 172.21.15.86:0/2593420987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234036060 con 0x557fe92021f0 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.756+0000 7f621f7fe640 1 -- 172.21.15.86:0/2593420987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f623401de40 con 0x557fe92021f0 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f621f7fe640 1 -- 172.21.15.86:0/2593420987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f623401e420 con 0x557fe92021f0 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f62493e4740 10 client.16074 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=29}) -- 0x557fe8f437d0 con 0x557fe92021f0 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f621f7fe640 1 -- 172.21.15.86:0/2593420987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f623401e5c0 con 0x557fe92021f0 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f62493e4740 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f62493e4740 10 client.16074 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:36:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f62493e4740 20 client.16074 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f62493e4740 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x557fe9223860 0x557fe9225c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f62493e4740 4 client.16074 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.757+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f623401eca0 con 0x557fe9223860 2024-04-16T05:36:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.758+0000 7f623ffff640 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x557fe9223860 0x557fe9225c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.758+0000 7f621f7fe640 10 client.16074 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:36:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.758+0000 7f623ffff640 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x557fe9223860 0x557fe9225c50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53f8e478a2dd1110 server_cookie=a066f8020eaf684d in_seq=0 out_seq=0 2024-04-16T05:36:28.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.759+0000 7f621f7fe640 1 -- 172.21.15.86:0/2593420987 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f623401eca0 con 0x557fe9223860 2024-04-16T05:36:28.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.759+0000 7f621f7fe640 10 client.16074 handle_command_reply: tid=0 2024-04-16T05:36:28.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.759+0000 7f62493e4740 1 client.16074 shutdown 2024-04-16T05:36:28.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 msgr2=0x557fe9210fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9210fd0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f623401d770 tx=0x7f623401d5e0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 shutdown_connections 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9213b70 0x557fe9211510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92021f0 0x557fe9210fd0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x557fe9223860 0x557fe9225c50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 --2- 172.21.15.86:0/2593420987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe9210a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 >> 172.21.15.86:0/2593420987 conn(0x557fe91fc590 msgr2=0x557fe91fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f621f7fe640 0 client.16074 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 shutdown_connections 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 1 -- 172.21.15.86:0/2593420987 wait complete. 2024-04-16T05:36:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.762+0000 7f62493e4740 20 client.16074 trim_cache size 0 max 16384 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f62493e4740 1 Processor -- start 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f62493e4740 1 -- start start 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f62493e4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9201840 0x557fe921f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f62493e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe9213b70 0x557fe9211160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f62493e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe92116a0 0x557fe9210050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f62493e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62401ad270 con 0x557fe9213b70 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f62493e4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62401aedb0 con 0x557fe9201840 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f62493e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62401ad940 con 0x557fe92116a0 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe9213b70 0x557fe9211160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe9213b70 0x557fe9211160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39106/0 (socket says 172.21.15.86:39106) 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f623f7fe640 1 -- 172.21.15.86:0/1061131383 learned_addr learned my addr 172.21.15.86:0/1061131383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.788+0000 7f6244cb9640 1 --2- 172.21.15.86:0/1061131383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe92116a0 0x557fe9210050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f623ffff640 1 --2- 172.21.15.86:0/1061131383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9201840 0x557fe921f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f623f7fe640 1 -- 172.21.15.86:0/1061131383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe92116a0 msgr2=0x557fe9210050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f623f7fe640 1 --2- 172.21.15.86:0/1061131383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe92116a0 0x557fe9210050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f623f7fe640 1 -- 172.21.15.86:0/1061131383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9201840 msgr2=0x557fe921f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f623f7fe640 1 --2- 172.21.15.86:0/1061131383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9201840 0x557fe921f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f623f7fe640 1 -- 172.21.15.86:0/1061131383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fe9201f90 con 0x557fe9213b70 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f623f7fe640 1 --2- 172.21.15.86:0/1061131383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe9213b70 0x557fe9211160 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6234034040 tx=0x7f623402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9ab6be0e3e19b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f6225ffb640 1 -- 172.21.15.86:0/1061131383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623401eac0 con 0x557fe9213b70 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f6225ffb640 1 -- 172.21.15.86:0/1061131383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f623401ee50 con 0x557fe9213b70 2024-04-16T05:36:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.789+0000 7f6225ffb640 1 -- 172.21.15.86:0/1061131383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234004620 con 0x557fe9213b70 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 -- 172.21.15.86:0/1061131383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe9213b70 msgr2=0x557fe9211160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 --2- 172.21.15.86:0/1061131383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe9213b70 0x557fe9211160 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6234034040 tx=0x7f623402c000 comp rx=0 tx=0).stop 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 -- 172.21.15.86:0/1061131383 shutdown_connections 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 --2- 172.21.15.86:0/1061131383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe92116a0 0x557fe9210050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 --2- 172.21.15.86:0/1061131383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe9213b70 0x557fe9211160 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 --2- 172.21.15.86:0/1061131383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe9201840 0x557fe921f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 -- 172.21.15.86:0/1061131383 >> 172.21.15.86:0/1061131383 conn(0x557fe91fc590 msgr2=0x557fe9210a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 -- 172.21.15.86:0/1061131383 shutdown_connections 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 -- 172.21.15.86:0/1061131383 wait complete. 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.790+0000 7f62493e4740 1 -- start start 2024-04-16T05:36:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f62493e4740 1 Processor -- start 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f62493e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe921c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f62493e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92116a0 0x557fe92173b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f62493e4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe92178f0 0x557fe9217d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f62493e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62401ae6e0 con 0x557fe92116a0 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f62493e4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62401ae010 con 0x557fe92178f0 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f62493e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62401ab230 con 0x557fe9201840 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f62493e4740 10 client.0 _ll_register_callbacks cb 0x7f62300056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92116a0 0x557fe92173b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92116a0 0x557fe92173b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39114/0 (socket says 172.21.15.86:39114) 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f623f7fe640 1 -- 172.21.15.86:0/2568252428 learned_addr learned my addr 172.21.15.86:0/2568252428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f6244cb9640 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe92178f0 0x557fe9217d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f623ffff640 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe921c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f623f7fe640 1 -- 172.21.15.86:0/2568252428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 msgr2=0x557fe921c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f623f7fe640 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe921c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f623f7fe640 1 -- 172.21.15.86:0/2568252428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe92178f0 msgr2=0x557fe9217d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f623f7fe640 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe92178f0 0x557fe9217d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.791+0000 7f623f7fe640 1 -- 172.21.15.86:0/2568252428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fe9288b20 con 0x557fe92116a0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.792+0000 7f623f7fe640 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92116a0 0x557fe92173b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f62340057c0 tx=0x7f623401df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f5ce166f86df4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.792+0000 7f621f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.792+0000 7f621f7fe640 1 -- 172.21.15.86:0/2568252428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234048060 con 0x557fe92116a0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.792+0000 7f621f7fe640 1 -- 172.21.15.86:0/2568252428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f623401d840 con 0x557fe92116a0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.792+0000 7f62493e4740 1 -- 172.21.15.86:0/2568252428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f623401ee50 con 0x557fe92116a0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.792+0000 7f621f7fe640 1 -- 172.21.15.86:0/2568252428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f623404f050 con 0x557fe92116a0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.792+0000 7f62493e4740 10 client.16080 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.792+0000 7f62493e4740 1 -- 172.21.15.86:0/2568252428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557fe9289de0 con 0x557fe92116a0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.792+0000 7f621f7fe640 1 -- 172.21.15.86:0/2568252428 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f623401ee50 con 0x557fe92116a0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f621f7fe640 1 -- 172.21.15.86:0/2568252428 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6234004a10 con 0x557fe92116a0 2024-04-16T05:36:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f62493e4740 10 client.16080 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f62493e4740 10 client.16080 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f62493e4740 20 client.16080 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f62493e4740 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x557fe928a440 0x557fe928c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f62493e4740 4 client.16080 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f62493e4740 1 -- 172.21.15.86:0/2568252428 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fe9174b00 con 0x557fe928a440 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f623ffff640 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x557fe928a440 0x557fe928c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f621f7fe640 10 client.16080 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.793+0000 7f623ffff640 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x557fe928a440 0x557fe928c830 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c688c91d6290fa6 server_cookie=ab69aae71f0258ca in_seq=0 out_seq=0 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f621f7fe640 1 -- 172.21.15.86:0/2568252428 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fe9174b00 con 0x557fe928a440 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f621f7fe640 10 client.16080 handle_command_reply: tid=0 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 client.16080 shutdown 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 -- 172.21.15.86:0/2568252428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92116a0 msgr2=0x557fe92173b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92116a0 0x557fe92173b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f62340057c0 tx=0x7f623401df80 comp rx=0 tx=0).stop 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 -- 172.21.15.86:0/2568252428 shutdown_connections 2024-04-16T05:36:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557fe92178f0 0x557fe9217d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fe92116a0 0x557fe92173b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x557fe928a440 0x557fe928c830 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 --2- 172.21.15.86:0/2568252428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fe9201840 0x557fe921c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 -- 172.21.15.86:0/2568252428 >> 172.21.15.86:0/2568252428 conn(0x557fe91fc590 msgr2=0x557fe91fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f621f7fe640 0 client.16080 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:36:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 -- 172.21.15.86:0/2568252428 shutdown_connections 2024-04-16T05:36:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 1 -- 172.21.15.86:0/2568252428 wait complete. 2024-04-16T05:36:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.794+0000 7f62493e4740 20 client.16080 trim_cache size 0 max 16384 2024-04-16T05:36:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:36:28.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 -- 172.21.15.86:0/2875498911 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62280a3ed0 msgr2=0x7f62280a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62280a3ed0 0x7f62280a6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6240100820 tx=0x7f623400b390 comp rx=0 tx=0).stop 2024-04-16T05:36:28.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 -- 172.21.15.86:0/2875498911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240103270 msgr2=0x7f6240196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240103270 0x7f6240196630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9f0 tx=0x7f623000edf0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 -- 172.21.15.86:0/2875498911 shutdown_connections 2024-04-16T05:36:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f624010c5b0 0x7f624019aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62280a3ed0 0x7f62280a6390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6240103b90 0x7f6240196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 --2- 172.21.15.86:0/2875498911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240103270 0x7f6240196630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 -- 172.21.15.86:0/2875498911 >> 172.21.15.86:0/2875498911 conn(0x7f62400780b0 msgr2=0x7f62400fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 -- 172.21.15.86:0/2875498911 shutdown_connections 2024-04-16T05:36:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.795+0000 7f624671c640 1 -- 172.21.15.86:0/2875498911 wait complete. 2024-04-16T05:36:28.805 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:36:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.970+0000 7f37b7077640 1 -- 172.21.15.86:0/3581999711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0101130 msgr2=0x7f37b0101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.970+0000 7f37b7077640 1 --2- 172.21.15.86:0/3581999711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0101130 0x7f37b0101510 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009900 tx=0x7f37a402bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.970+0000 7f37b7077640 1 -- 172.21.15.86:0/3581999711 shutdown_connections 2024-04-16T05:36:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.970+0000 7f37b7077640 1 --2- 172.21.15.86:0/3581999711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b0106200 0x7f37b010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.970+0000 7f37b7077640 1 --2- 172.21.15.86:0/3581999711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37b0101ae0 0x7f37b0105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.970+0000 7f37b7077640 1 --2- 172.21.15.86:0/3581999711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0101130 0x7f37b0101510 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.970+0000 7f37b7077640 1 -- 172.21.15.86:0/3581999711 >> 172.21.15.86:0/3581999711 conn(0x7f37b00780b0 msgr2=0x7f37b00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.970+0000 7f37b7077640 1 -- 172.21.15.86:0/3581999711 shutdown_connections 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.970+0000 7f37b7077640 1 -- 172.21.15.86:0/3581999711 wait complete. 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b7077640 1 Processor -- start 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b7077640 1 -- start start 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b7077640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0106200 0x7f37b0102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b7077640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37b0103100 0x7f37b0103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b7077640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b01a4c70 0x7f37b01a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b7077640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01112c0 con 0x7f37b0106200 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b7077640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0111140 con 0x7f37b0103100 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b7077640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0111440 con 0x7f37b01a4c70 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b4e13640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0106200 0x7f37b0102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b4e13640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0106200 0x7f37b0102bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39126/0 (socket says 172.21.15.86:39126) 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37b4e13640 1 -- 172.21.15.86:0/3003451464 learned_addr learned my addr 172.21.15.86:0/3003451464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.971+0000 7f37affff640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37b0103100 0x7f37b0103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37b5614640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b01a4c70 0x7f37b01a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37b4e13640 1 -- 172.21.15.86:0/3003451464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b01a4c70 msgr2=0x7f37b01a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b01a4c70 0x7f37b01a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37b4e13640 1 -- 172.21.15.86:0/3003451464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37b0103100 msgr2=0x7f37b0103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37b0103100 0x7f37b0103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37b4e13640 1 -- 172.21.15.86:0/3003451464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b01a7730 con 0x7f37b0106200 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37affff640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37b0103100 0x7f37b0103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0106200 0x7f37b0102bc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f37a4035000 tx=0x7f37a402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37adffb640 1 -- 172.21.15.86:0/3003451464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4041070 con 0x7f37b0106200 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37b7077640 1 -- 172.21.15.86:0/3003451464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b01a79c0 con 0x7f37b0106200 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37adffb640 1 -- 172.21.15.86:0/3003451464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a4004450 con 0x7f37b0106200 2024-04-16T05:36:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.972+0000 7f37b7077640 1 -- 172.21.15.86:0/3003451464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b01a7eb0 con 0x7f37b0106200 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.973+0000 7f37adffb640 1 -- 172.21.15.86:0/3003451464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4005200 con 0x7f37b0106200 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37b9c5c740 1 Processor -- start 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37b9c5c740 1 -- start start 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160769e60 0x55816076a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x558160772e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160773360 0x558160775750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581606cf0e0 con 0x558160773360 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55816066c520 con 0x55816076a780 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581606cb0a0 con 0x558160769e60 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37affff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160773360 0x558160775750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37affff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160773360 0x558160775750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39132/0 (socket says 172.21.15.86:39132) 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.975+0000 7f37affff640 1 -- 172.21.15.86:0/3378159711 learned_addr learned my addr 172.21.15.86:0/3378159711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f37b5614640 1 --2- 172.21.15.86:0/3378159711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160769e60 0x55816076a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f37affff640 1 -- 172.21.15.86:0/3378159711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160769e60 msgr2=0x55816076a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f37affff640 1 --2- 172.21.15.86:0/3378159711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160769e60 0x55816076a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f37affff640 1 -- 172.21.15.86:0/3378159711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 msgr2=0x558160772e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f37affff640 1 --2- 172.21.15.86:0/3378159711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x558160772e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f37affff640 1 -- 172.21.15.86:0/3378159711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581606948d0 con 0x558160773360 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f37affff640 1 --2- 172.21.15.86:0/3378159711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160773360 0x558160775750 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f379c00bcb0 tx=0x7f379c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7582756fba0a396 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f37adffb640 1 -- 172.21.15.86:0/3003451464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f37a4044070 con 0x7f37b0106200 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f379affd640 1 -- 172.21.15.86:0/3378159711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c011440 con 0x558160773360 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f379affd640 1 -- 172.21.15.86:0/3378159711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379c011ac0 con 0x558160773360 2024-04-16T05:36:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.976+0000 7f379affd640 1 -- 172.21.15.86:0/3378159711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c010ad0 con 0x558160773360 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.977+0000 7f37adffb640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37a00a40d0 0x7f37a00a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.977+0000 7f37b5614640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37a00a40d0 0x7f37a00a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.977+0000 7f37adffb640 1 -- 172.21.15.86:0/3003451464 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37a4005000 con 0x7f37b0106200 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.977+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3378159711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160773360 msgr2=0x558160775750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.977+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3378159711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160773360 0x558160775750 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f379c00bcb0 tx=0x7f379c00f730 comp rx=0 tx=0).stop 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3378159711 shutdown_connections 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3378159711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160773360 0x558160775750 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3378159711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x558160772e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3378159711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160769e60 0x55816076a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3378159711 >> 172.21.15.86:0/3378159711 conn(0x558160764590 msgr2=0x558160764970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b5614640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37a00a40d0 0x7f37a00a6590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005fd0 tx=0x7f37a8009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3378159711 shutdown_connections 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3378159711 wait complete. 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b9c5c740 1 -- start start 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.978+0000 7f37b9c5c740 1 Processor -- start 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x5581607702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x558160770830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x558160770d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581607760c0 con 0x558160769e60 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b4e13640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x5581607702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5581606884d0 con 0x55816076a780 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558160776240 con 0x558160773360 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b4e13640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x5581607702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39146/0 (socket says 172.21.15.86:39146) 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b4e13640 1 -- 172.21.15.86:0/3269233555 learned_addr learned my addr 172.21.15.86:0/3269233555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b9c5c740 10 client.0 _ll_register_callbacks cb 0x55816066d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37affff640 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x558160770830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b4e13640 1 -- 172.21.15.86:0/3269233555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 msgr2=0x558160770d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x558160770d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b4e13640 1 -- 172.21.15.86:0/3269233555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 msgr2=0x558160770830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x558160770830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.979+0000 7f37b4e13640 1 -- 172.21.15.86:0/3269233555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581603e13f0 con 0x558160769e60 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.980+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x5581607702f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f37a40bd560 tx=0x7f37a40bd1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe70c32e407d0e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.980+0000 7f378ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.980+0000 7f378ffff640 1 -- 172.21.15.86:0/3269233555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a40bd5d0 con 0x558160769e60 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.980+0000 7f378ffff640 1 -- 172.21.15.86:0/3269233555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37a4033c30 con 0x558160769e60 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.980+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3269233555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f379c011ac0 con 0x558160769e60 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.980+0000 7f378ffff640 1 -- 172.21.15.86:0/3269233555 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37a40c1980 con 0x558160769e60 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.980+0000 7f37b9c5c740 10 client.16092 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.980+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3269233555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581604ab7d0 con 0x558160769e60 2024-04-16T05:36:28.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.980+0000 7f378ffff640 1 -- 172.21.15.86:0/3269233555 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379c011ac0 con 0x558160769e60 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.981+0000 7f378ffff640 1 -- 172.21.15.86:0/3269233555 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37a40c2b10 con 0x558160769e60 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.981+0000 7f37b9c5c740 10 client.16092 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.981+0000 7f37b9c5c740 10 client.16092 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.981+0000 7f37b9c5c740 20 client.16092 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.981+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5581607831f0 0x5581607855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.981+0000 7f37b9c5c740 4 client.16092 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.981+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3269233555 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581606dcb00 con 0x5581607831f0 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.981+0000 7f37affff640 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5581607831f0 0x5581607855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.982+0000 7f37affff640 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5581607831f0 0x5581607855e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5619195b3b3c85dd server_cookie=32aefc1a3196e4cd in_seq=0 out_seq=0 2024-04-16T05:36:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.982+0000 7f378ffff640 10 client.16092 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f378ffff640 1 -- 172.21.15.86:0/3269233555 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581606dcb00 con 0x5581607831f0 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f378ffff640 10 client.16092 handle_command_reply: tid=0 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 client.16092 shutdown 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3269233555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 msgr2=0x5581607702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x5581607702f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f37a40bd560 tx=0x7f37a40bd1d0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3269233555 shutdown_connections 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x558160770d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x558160770830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5581607831f0 0x5581607855e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f378ffff640 0 client.16092 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3269233555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x5581607702f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3269233555 >> 172.21.15.86:0/3269233555 conn(0x558160764590 msgr2=0x5581607662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3269233555 shutdown_connections 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.983+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3269233555 wait complete. 2024-04-16T05:36:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:28.984+0000 7f37b9c5c740 20 client.16092 trim_cache size 0 max 16384 2024-04-16T05:36:29.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.009+0000 7f37b9c5c740 1 Processor -- start 2024-04-16T05:36:29.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.009+0000 7f37b9c5c740 1 -- start start 2024-04-16T05:36:29.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.009+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x55816077f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.009+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x5581607635b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.009+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x55816078ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.009+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b1ad0 con 0x558160769e60 2024-04-16T05:36:29.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.009+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b3610 con 0x55816076a780 2024-04-16T05:36:29.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.009+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b21a0 con 0x558160773360 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b4e13640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x55816077f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b4e13640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x55816077f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39154/0 (socket says 172.21.15.86:39154) 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b4e13640 1 -- 172.21.15.86:0/3352235768 learned_addr learned my addr 172.21.15.86:0/3352235768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b5614640 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x55816078ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b4e13640 1 -- 172.21.15.86:0/3352235768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 msgr2=0x55816078ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x55816078ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b4e13640 1 -- 172.21.15.86:0/3352235768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 msgr2=0x5581607635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37affff640 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x5581607635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x5581607635b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b4e13640 1 -- 172.21.15.86:0/3352235768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816076a5b0 con 0x558160769e60 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b5614640 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x55816078ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37affff640 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x5581607635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x55816077f0c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f37a4006010 tx=0x7f37a40bd240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9411381f9cb74b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f379a7fc640 1 -- 172.21.15.86:0/3352235768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a40bd5d0 con 0x558160769e60 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f379a7fc640 1 -- 172.21.15.86:0/3352235768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a4004450 con 0x558160769e60 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.010+0000 7f379a7fc640 1 -- 172.21.15.86:0/3352235768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4033c30 con 0x558160769e60 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3352235768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 msgr2=0x55816077f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x55816077f0c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f37a4006010 tx=0x7f37a40bd240 comp rx=0 tx=0).stop 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3352235768 shutdown_connections 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x55816078ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x5581607635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3352235768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558160769e60 0x55816077f0c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3352235768 >> 172.21.15.86:0/3352235768 conn(0x558160764590 msgr2=0x558160762ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3352235768 shutdown_connections 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3352235768 wait complete. 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.011+0000 7f37b9c5c740 1 -- start start 2024-04-16T05:36:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b9c5c740 1 Processor -- start 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x55816077ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x558160776de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b9c5c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581607831f0 0x5581607773f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b2f40 con 0x5581607831f0 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b2870 con 0x55816076a780 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b9c5c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01afa90 con 0x558160773360 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b9c5c740 10 client.0 _ll_register_callbacks cb 0x7f37a40bd3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b5614640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581607831f0 0x5581607773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b5614640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581607831f0 0x5581607773f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39170/0 (socket says 172.21.15.86:39170) 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b5614640 1 -- 172.21.15.86:0/3239505030 learned_addr learned my addr 172.21.15.86:0/3239505030 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x55816077ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.012+0000 7f37affff640 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x558160776de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b5614640 1 -- 172.21.15.86:0/3239505030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 msgr2=0x558160776de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b5614640 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x558160776de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b5614640 1 -- 172.21.15.86:0/3239505030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 msgr2=0x55816077ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b5614640 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x55816077ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b5614640 1 -- 172.21.15.86:0/3239505030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558160777bc0 con 0x5581607831f0 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x55816077ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b5614640 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581607831f0 0x5581607773f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f37a801fa10 tx=0x7f37a801fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8b67ca4a6a59ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f378ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f378ffff640 1 -- 172.21.15.86:0/3239505030 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a801d910 con 0x5581607831f0 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f378ffff640 1 -- 172.21.15.86:0/3239505030 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37a8017030 con 0x5581607831f0 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3239505030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a4004450 con 0x5581607831f0 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f378ffff640 1 -- 172.21.15.86:0/3239505030 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37a8035050 con 0x5581607831f0 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b9c5c740 10 client.16098 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.013+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3239505030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581607e9720 con 0x5581607831f0 2024-04-16T05:36:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.014+0000 7f378ffff640 1 -- 172.21.15.86:0/3239505030 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a4004450 con 0x5581607831f0 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.014+0000 7f378ffff640 1 -- 172.21.15.86:0/3239505030 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37a8030030 con 0x5581607831f0 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.014+0000 7f37b9c5c740 10 client.16098 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.014+0000 7f37b9c5c740 10 client.16098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.014+0000 7f37b9c5c740 20 client.16098 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.014+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5581607e9ef0 0x5581607ec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.014+0000 7f37b9c5c740 4 client.16098 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.014+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3239505030 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581606d6970 con 0x5581607e9ef0 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.014+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5581607e9ef0 0x5581607ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b4e13640 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5581607e9ef0 0x5581607ec310 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a28a1a11a7453179 server_cookie=454ece92dc7dfb89 in_seq=0 out_seq=0 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f378ffff640 10 client.16098 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f378ffff640 1 -- 172.21.15.86:0/3239505030 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581606d6970 con 0x5581607e9ef0 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f378ffff640 10 client.16098 handle_command_reply: tid=0 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 client.16098 shutdown 2024-04-16T05:36:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3239505030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581607831f0 msgr2=0x5581607773f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581607831f0 0x5581607773f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f37a801fa10 tx=0x7f37a801fe10 comp rx=0 tx=0).stop 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3239505030 shutdown_connections 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5581607e9ef0 0x5581607ec310 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581607831f0 0x5581607773f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558160773360 0x558160776de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 --2- 172.21.15.86:0/3239505030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55816076a780 0x55816077ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3239505030 >> 172.21.15.86:0/3239505030 conn(0x558160764590 msgr2=0x558160770b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f378ffff640 0 client.16098 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3239505030 shutdown_connections 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 1 -- 172.21.15.86:0/3239505030 wait complete. 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.015+0000 7f37b9c5c740 20 client.16098 trim_cache size 0 max 16384 2024-04-16T05:36:29.019 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 -- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37a00a40d0 msgr2=0x7f37a00a6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37a00a40d0 0x7f37a00a6590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005fd0 tx=0x7f37a8009500 comp rx=0 tx=0).stop 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 -- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0106200 msgr2=0x7f37b0102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0106200 0x7f37b0102bc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f37a4035000 tx=0x7f37a402cbf0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 -- 172.21.15.86:0/3003451464 shutdown_connections 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b01a4c70 0x7f37b01a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37a00a40d0 0x7f37a00a6590 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37b0103100 0x7f37b0103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 --2- 172.21.15.86:0/3003451464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b0106200 0x7f37b0102bc0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 -- 172.21.15.86:0/3003451464 >> 172.21.15.86:0/3003451464 conn(0x7f37b00780b0 msgr2=0x7f37b01000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.016+0000 7f37b7077640 1 -- 172.21.15.86:0/3003451464 shutdown_connections 2024-04-16T05:36:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.017+0000 7f37b7077640 1 -- 172.21.15.86:0/3003451464 wait complete. 2024-04-16T05:36:29.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:36:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.136+0000 7f5f7b150640 1 -- 172.21.15.86:0/3683480231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7410c5b0 msgr2=0x7f5f7410ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.136+0000 7f5f7b150640 1 --2- 172.21.15.86:0/3683480231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7410c5b0 0x7f5f7410ea50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009ca0 tx=0x7f5f6c02c230 comp rx=0 tx=0).stop 2024-04-16T05:36:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.136+0000 7f5f7b150640 1 -- 172.21.15.86:0/3683480231 shutdown_connections 2024-04-16T05:36:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.136+0000 7f5f7b150640 1 --2- 172.21.15.86:0/3683480231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7410c5b0 0x7f5f7410ea50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.136+0000 7f5f7b150640 1 --2- 172.21.15.86:0/3683480231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f74103b90 0x7f5f7410c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.136+0000 7f5f7b150640 1 --2- 172.21.15.86:0/3683480231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f74103270 0x7f5f74103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.136+0000 7f5f7b150640 1 -- 172.21.15.86:0/3683480231 >> 172.21.15.86:0/3683480231 conn(0x7f5f740780b0 msgr2=0x7f5f740784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.136+0000 7f5f7b150640 1 -- 172.21.15.86:0/3683480231 shutdown_connections 2024-04-16T05:36:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.136+0000 7f5f7b150640 1 -- 172.21.15.86:0/3683480231 wait complete. 2024-04-16T05:36:29.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f7b150640 1 Processor -- start 2024-04-16T05:36:29.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f7b150640 1 -- start start 2024-04-16T05:36:29.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f7b150640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f74103270 0x7f5f74100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f7b150640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74103b90 0x7f5f74101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f7b150640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f7410c5b0 0x7f5f74101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f7b150640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74112390 con 0x7f5f74103b90 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f7b150640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74112210 con 0x7f5f74103270 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f7b150640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74112510 con 0x7f5f7410c5b0 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74103b90 0x7f5f74101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74103b90 0x7f5f74101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39190/0 (socket says 172.21.15.86:39190) 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f73fff640 1 -- 172.21.15.86:0/305838351 learned_addr learned my addr 172.21.15.86:0/305838351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f78eec640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f74103270 0x7f5f74100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f796ed640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f7410c5b0 0x7f5f74101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f73fff640 1 -- 172.21.15.86:0/305838351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f7410c5b0 msgr2=0x7f5f74101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f73fff640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f7410c5b0 0x7f5f74101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f73fff640 1 -- 172.21.15.86:0/305838351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f74103270 msgr2=0x7f5f74100ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f73fff640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f74103270 0x7f5f74100ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.137+0000 7f5f73fff640 1 -- 172.21.15.86:0/305838351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f741a6d70 con 0x7f5f74103b90 2024-04-16T05:36:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.138+0000 7f5f73fff640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74103b90 0x7f5f74101400 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9f0 tx=0x7f5f6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.138+0000 7f5f71ffb640 1 -- 172.21.15.86:0/305838351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000cd80 con 0x7f5f74103b90 2024-04-16T05:36:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.138+0000 7f5f7b150640 1 -- 172.21.15.86:0/305838351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f741a7060 con 0x7f5f74103b90 2024-04-16T05:36:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.138+0000 7f5f71ffb640 1 -- 172.21.15.86:0/305838351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f60010910 con 0x7f5f74103b90 2024-04-16T05:36:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.138+0000 7f5f7b150640 1 -- 172.21.15.86:0/305838351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f741a7600 con 0x7f5f74103b90 2024-04-16T05:36:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.139+0000 7f5f71ffb640 1 -- 172.21.15.86:0/305838351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60010630 con 0x7f5f74103b90 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f7de13740 1 Processor -- start 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f7de13740 1 -- start start 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d8e60 0x5581eb6d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 0x5581eb6e1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6e2360 0x5581eb6e4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f7de13740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581eb63e0e0 con 0x5581eb6d8e60 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f7de13740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5581eb5db520 con 0x5581eb6e2360 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f7de13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581eb63a0a0 con 0x5581eb6d9780 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f796ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d8e60 0x5581eb6d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f796ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d8e60 0x5581eb6d9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39192/0 (socket says 172.21.15.86:39192) 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f796ed640 1 -- 172.21.15.86:0/3902795919 learned_addr learned my addr 172.21.15.86:0/3902795919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f78eec640 1 --2- 172.21.15.86:0/3902795919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 0x5581eb6e1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f796ed640 1 -- 172.21.15.86:0/3902795919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 msgr2=0x5581eb6e1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f796ed640 1 --2- 172.21.15.86:0/3902795919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 0x5581eb6e1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f796ed640 1 -- 172.21.15.86:0/3902795919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6e2360 msgr2=0x5581eb6e4750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T05:36:29.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f796ed640 1 --2- 172.21.15.86:0/3902795919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6e2360 0x5581eb6e4750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.141+0000 7f5f796ed640 1 -- 172.21.15.86:0/3902795919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581eb6038d0 con 0x5581eb6d8e60 2024-04-16T05:36:29.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.142+0000 7f5f796ed640 1 --2- 172.21.15.86:0/3902795919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d8e60 0x5581eb6d9240 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009ca0 tx=0x7f5f6c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cbeaeba42b24e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.142+0000 7f5f5effd640 1 -- 172.21.15.86:0/3902795919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c043020 con 0x5581eb6d8e60 2024-04-16T05:36:29.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.142+0000 7f5f5effd640 1 -- 172.21.15.86:0/3902795919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6c032540 con 0x5581eb6d8e60 2024-04-16T05:36:29.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.142+0000 7f5f5effd640 1 -- 172.21.15.86:0/3902795919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c039040 con 0x5581eb6d8e60 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.143+0000 7f5f71ffb640 1 -- 172.21.15.86:0/305838351 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5f6001f020 con 0x7f5f74103b90 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.143+0000 7f5f7de13740 1 -- 172.21.15.86:0/3902795919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d8e60 msgr2=0x5581eb6d9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.143+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3902795919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d8e60 0x5581eb6d9240 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009ca0 tx=0x7f5f6c0093f0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.143+0000 7f5f7de13740 1 -- 172.21.15.86:0/3902795919 shutdown_connections 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.143+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3902795919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6e2360 0x5581eb6e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.143+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3902795919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 0x5581eb6e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.143+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3902795919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d8e60 0x5581eb6d9240 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.143+0000 7f5f7de13740 1 -- 172.21.15.86:0/3902795919 >> 172.21.15.86:0/3902795919 conn(0x5581eb6d3590 msgr2=0x5581eb6d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f71ffb640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5f640a40d0 0x7f5f640a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f7de13740 1 -- 172.21.15.86:0/3902795919 shutdown_connections 2024-04-16T05:36:29.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f7de13740 1 -- 172.21.15.86:0/3902795919 wait complete. 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f78eec640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5f640a40d0 0x7f5f640a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f71ffb640 1 -- 172.21.15.86:0/305838351 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f60010430 con 0x7f5f74103b90 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f7de13740 1 -- start start 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f7de13740 1 Processor -- start 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f78eec640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5f640a40d0 0x7f5f640a6590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5581eb6d8930 tx=0x7f5f68005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 0x5581eb6df9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.144+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6dff30 0x5581eb6ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581eb6e5000 con 0x5581eb6dff30 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5581eb5f74d0 con 0x5581eb6d8e60 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581eb6e5180 con 0x5581eb6d9780 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 10 client.0 _ll_register_callbacks cb 0x5581eb5dc2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f796ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6dff30 0x5581eb6ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f78eec640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f796ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6dff30 0x5581eb6ee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39206/0 (socket says 172.21.15.86:39206) 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f796ed640 1 -- 172.21.15.86:0/3372923375 learned_addr learned my addr 172.21.15.86:0/3372923375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f73fff640 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 0x5581eb6df9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f78eec640 1 -- 172.21.15.86:0/3372923375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 msgr2=0x5581eb6df9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f78eec640 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 0x5581eb6df9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f78eec640 1 -- 172.21.15.86:0/3372923375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6dff30 msgr2=0x5581eb6ee5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f78eec640 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6dff30 0x5581eb6ee5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f78eec640 1 -- 172.21.15.86:0/3372923375 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581eb3503f0 con 0x5581eb6d8e60 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f796ed640 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6dff30 0x5581eb6ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f78eec640 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6df4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f68011890 tx=0x7f5f680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cabe719a0f3204f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f53fff640 1 -- 172.21.15.86:0/3372923375 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68004020 con 0x5581eb6d8e60 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 1 -- 172.21.15.86:0/3372923375 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6c032540 con 0x5581eb6d8e60 2024-04-16T05:36:29.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f53fff640 1 -- 172.21.15.86:0/3372923375 <== mon.2 v2:172.21.15.194:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f680041c0 con 0x5581eb6d8e60 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f53fff640 1 -- 172.21.15.86:0/3372923375 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f68031050 con 0x5581eb6d8e60 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 10 client.25472 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 1 -- 172.21.15.86:0/3372923375 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581eb41a7d0 con 0x5581eb6d8e60 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f53fff640 1 -- 172.21.15.86:0/3372923375 <== mon.2 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6c032540 con 0x5581eb6d8e60 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f53fff640 1 -- 172.21.15.86:0/3372923375 <== mon.2 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5f6801a040 con 0x5581eb6d8e60 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 10 client.25472 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 10 client.25472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 20 client.25472 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5581eb6f2830 0x5581eb6f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 4 client.25472 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.145+0000 7f5f7de13740 1 -- 172.21.15.86:0/3372923375 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581eb64bb00 con 0x5581eb6f2830 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.146+0000 7f5f73fff640 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5581eb6f2830 0x5581eb6f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.146+0000 7f5f53fff640 10 client.25472 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:36:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.146+0000 7f5f73fff640 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5581eb6f2830 0x5581eb6f4c90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c68db708e9a3dfc2 server_cookie=45e3f00d57bb590b in_seq=0 out_seq=0 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f53fff640 1 -- 172.21.15.86:0/3372923375 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581eb64bb00 con 0x5581eb6f2830 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f53fff640 10 client.25472 handle_command_reply: tid=0 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 client.25472 shutdown 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 -- 172.21.15.86:0/3372923375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 msgr2=0x5581eb6df4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6df4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f68011890 tx=0x7f5f680046c0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 -- 172.21.15.86:0/3372923375 shutdown_connections 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5581eb6f2830 0x5581eb6f4c90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6dff30 0x5581eb6ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f53fff640 0 client.25472 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d9780 0x5581eb6df9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3372923375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6df4b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 -- 172.21.15.86:0/3372923375 >> 172.21.15.86:0/3372923375 conn(0x5581eb6d3590 msgr2=0x5581eb6d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 -- 172.21.15.86:0/3372923375 shutdown_connections 2024-04-16T05:36:29.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 1 -- 172.21.15.86:0/3372923375 wait complete. 2024-04-16T05:36:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.147+0000 7f5f7de13740 20 client.25472 trim_cache size 0 max 16384 2024-04-16T05:36:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.160+0000 7f5f7de13740 1 Processor -- start 2024-04-16T05:36:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.160+0000 7f5f7de13740 1 -- start start 2024-04-16T05:36:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6de630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6dff30 0x5581eb6fa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f7de13740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b16a0 con 0x5581eb6d9780 2024-04-16T05:36:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f7de13740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b31e0 con 0x5581eb6d8e60 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f7de13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b1d70 con 0x5581eb6dff30 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6d25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39210/0 (socket says 172.21.15.86:39210) 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f73fff640 1 -- 172.21.15.86:0/2516752168 learned_addr learned my addr 172.21.15.86:0/2516752168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f78eec640 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f796ed640 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6dff30 0x5581eb6fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f73fff640 1 -- 172.21.15.86:0/2516752168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6dff30 msgr2=0x5581eb6fa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f73fff640 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6dff30 0x5581eb6fa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f73fff640 1 -- 172.21.15.86:0/2516752168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 msgr2=0x5581eb6de630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f73fff640 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6de630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f73fff640 1 -- 172.21.15.86:0/2516752168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581eb6cee60 con 0x5581eb6d9780 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f78eec640 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.161+0000 7f5f73fff640 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6d25b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5f600cdbe0 tx=0x7f5f60098790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6690fcfdd8b30c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f5e7fc640 1 -- 172.21.15.86:0/2516752168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600a5040 con 0x5581eb6d9780 2024-04-16T05:36:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f5e7fc640 1 -- 172.21.15.86:0/2516752168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f60010910 con 0x5581eb6d9780 2024-04-16T05:36:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f5e7fc640 1 -- 172.21.15.86:0/2516752168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600a4bb0 con 0x5581eb6d9780 2024-04-16T05:36:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f7de13740 1 -- 172.21.15.86:0/2516752168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 msgr2=0x5581eb6d25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f7de13740 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6d25b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5f600cdbe0 tx=0x7f5f60098790 comp rx=0 tx=0).stop 2024-04-16T05:36:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f7de13740 1 -- 172.21.15.86:0/2516752168 shutdown_connections 2024-04-16T05:36:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f7de13740 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6dff30 0x5581eb6fa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f7de13740 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6d25b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f7de13740 1 --2- 172.21.15.86:0/2516752168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6d8e60 0x5581eb6de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.162+0000 7f5f7de13740 1 -- 172.21.15.86:0/2516752168 >> 172.21.15.86:0/2516752168 conn(0x5581eb6d3590 msgr2=0x5581eb6d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.163+0000 7f5f7de13740 1 -- 172.21.15.86:0/2516752168 shutdown_connections 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.163+0000 7f5f7de13740 1 -- 172.21.15.86:0/2516752168 wait complete. 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.163+0000 7f5f7de13740 1 -- start start 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.163+0000 7f5f7de13740 1 Processor -- start 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d8e60 0x5581eb6ea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f7de13740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6dff30 0x5581eb6dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f7de13740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b2b10 con 0x5581eb6d9780 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f7de13740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b2440 con 0x5581eb6dff30 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f7de13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741af660 con 0x5581eb6d8e60 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f7de13740 10 client.0 _ll_register_callbacks cb 0x7f5f68011540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6eaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39222/0 (socket says 172.21.15.86:39222) 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f73fff640 1 -- 172.21.15.86:0/3119453140 learned_addr learned my addr 172.21.15.86:0/3119453140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f78eec640 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d8e60 0x5581eb6ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f796ed640 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6dff30 0x5581eb6dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f73fff640 1 -- 172.21.15.86:0/3119453140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d8e60 msgr2=0x5581eb6ea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f73fff640 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d8e60 0x5581eb6ea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f73fff640 1 -- 172.21.15.86:0/3119453140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6dff30 msgr2=0x5581eb6dbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f73fff640 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6dff30 0x5581eb6dbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f73fff640 1 -- 172.21.15.86:0/3119453140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581eb6dc370 con 0x5581eb6d9780 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f78eec640 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d8e60 0x5581eb6ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.164+0000 7f5f73fff640 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6eaa50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5f6009c3f0 tx=0x7f5f600106a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d04496934cea4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f53fff640 1 -- 172.21.15.86:0/3119453140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600af070 con 0x5581eb6d9780 2024-04-16T05:36:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f53fff640 1 -- 172.21.15.86:0/3119453140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f60098ad0 con 0x5581eb6d9780 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f7de13740 1 -- 172.21.15.86:0/3119453140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f60010910 con 0x5581eb6d9780 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f53fff640 1 -- 172.21.15.86:0/3119453140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f600bb2b0 con 0x5581eb6d9780 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f7de13740 10 client.16116 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f7de13740 1 -- 172.21.15.86:0/3119453140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581eb758870 con 0x5581eb6d9780 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f53fff640 1 -- 172.21.15.86:0/3119453140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f60010910 con 0x5581eb6d9780 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f53fff640 1 -- 172.21.15.86:0/3119453140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5f600a7070 con 0x5581eb6d9780 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f7de13740 10 client.16116 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f7de13740 10 client.16116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f7de13740 20 client.16116 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.165+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5581eb6f2830 0x5581eb75b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.166+0000 7f5f7de13740 4 client.16116 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.166+0000 7f5f7de13740 1 -- 172.21.15.86:0/3119453140 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581eb645970 con 0x5581eb6f2830 2024-04-16T05:36:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.166+0000 7f5f78eec640 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5581eb6f2830 0x5581eb75b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.166+0000 7f5f53fff640 10 client.16116 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.166+0000 7f5f78eec640 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5581eb6f2830 0x5581eb75b000 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1d10f9d616fe635 server_cookie=5308cf02027b3859 in_seq=0 out_seq=0 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f53fff640 1 -- 172.21.15.86:0/3119453140 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581eb645970 con 0x5581eb6f2830 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f53fff640 10 client.16116 handle_command_reply: tid=0 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 client.16116 shutdown 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 -- 172.21.15.86:0/3119453140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 msgr2=0x5581eb6eaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6eaa50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5f6009c3f0 tx=0x7f5f600106a0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 -- 172.21.15.86:0/3119453140 shutdown_connections 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5581eb6f2830 0x5581eb75b000 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f53fff640 0 client.16116 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581eb6dff30 0x5581eb6dbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581eb6d9780 0x5581eb6eaa50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 --2- 172.21.15.86:0/3119453140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581eb6d8e60 0x5581eb6ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 -- 172.21.15.86:0/3119453140 >> 172.21.15.86:0/3119453140 conn(0x5581eb6d3590 msgr2=0x5581eb6d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 -- 172.21.15.86:0/3119453140 shutdown_connections 2024-04-16T05:36:29.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 1 -- 172.21.15.86:0/3119453140 wait complete. 2024-04-16T05:36:29.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.167+0000 7f5f7de13740 20 client.16116 trim_cache size 0 max 16384 2024-04-16T05:36:29.172 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:36:29.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 -- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5f640a40d0 msgr2=0x7f5f640a6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5f640a40d0 0x7f5f640a6590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5581eb6d8930 tx=0x7f5f68005f90 comp rx=0 tx=0).stop 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 -- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74103b90 msgr2=0x7f5f74101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74103b90 0x7f5f74101400 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9f0 tx=0x7f5f6000eeb0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 -- 172.21.15.86:0/305838351 shutdown_connections 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f7410c5b0 0x7f5f74101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74103b90 0x7f5f74101400 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5f640a40d0 0x7f5f640a6590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 --2- 172.21.15.86:0/305838351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f74103270 0x7f5f74100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 -- 172.21.15.86:0/305838351 >> 172.21.15.86:0/305838351 conn(0x7f5f740780b0 msgr2=0x7f5f740ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 -- 172.21.15.86:0/305838351 shutdown_connections 2024-04-16T05:36:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.168+0000 7f5f7b150640 1 -- 172.21.15.86:0/305838351 wait complete. 2024-04-16T05:36:29.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:36:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.288+0000 7ff80edf4640 1 -- 172.21.15.86:0/3179983121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8081072e0 msgr2=0x7ff80810eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.288+0000 7ff80edf4640 1 --2- 172.21.15.86:0/3179983121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8081072e0 0x7ff80810eeb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff800009ca0 tx=0x7ff80002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.288+0000 7ff80edf4640 1 -- 172.21.15.86:0/3179983121 shutdown_connections 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.288+0000 7ff80edf4640 1 --2- 172.21.15.86:0/3179983121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8081072e0 0x7ff80810eeb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.288+0000 7ff80edf4640 1 --2- 172.21.15.86:0/3179983121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff808102bc0 0x7ff808106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.288+0000 7ff80edf4640 1 --2- 172.21.15.86:0/3179983121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff808102210 0x7ff8081025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.288+0000 7ff80edf4640 1 -- 172.21.15.86:0/3179983121 >> 172.21.15.86:0/3179983121 conn(0x7ff8080fdad0 msgr2=0x7ff8080ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.288+0000 7ff80edf4640 1 -- 172.21.15.86:0/3179983121 shutdown_connections 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.288+0000 7ff80edf4640 1 -- 172.21.15.86:0/3179983121 wait complete. 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80edf4640 1 Processor -- start 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80edf4640 1 -- start start 2024-04-16T05:36:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80edf4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff808102210 0x7ff8081a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80edf4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8081072e0 0x7ff8081a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80edf4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff80819a940 0x7ff80819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80edf4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8081123a0 con 0x7ff808102210 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80edf4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff808112220 con 0x7ff8081072e0 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80edf4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff808112520 con 0x7ff80819a940 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80cb90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff808102210 0x7ff8081a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80cb90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff808102210 0x7ff8081a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39240/0 (socket says 172.21.15.86:39240) 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80cb90640 1 -- 172.21.15.86:0/1397385800 learned_addr learned my addr 172.21.15.86:0/1397385800 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff807fff640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8081072e0 0x7ff8081a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80cb90640 1 -- 172.21.15.86:0/1397385800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff80819a940 msgr2=0x7ff80819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80cb90640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff80819a940 0x7ff80819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.289+0000 7ff80cb90640 1 -- 172.21.15.86:0/1397385800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8081072e0 msgr2=0x7ff8081a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.290+0000 7ff80cb90640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8081072e0 0x7ff8081a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.290+0000 7ff80cb90640 1 -- 172.21.15.86:0/1397385800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80819b460 con 0x7ff808102210 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.290+0000 7ff80cb90640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff808102210 0x7ff8081a0790 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e9f0 tx=0x7ff7fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.290+0000 7ff805ffb640 1 -- 172.21.15.86:0/1397385800 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00cd80 con 0x7ff808102210 2024-04-16T05:36:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.290+0000 7ff80edf4640 1 -- 172.21.15.86:0/1397385800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80819b750 con 0x7ff808102210 2024-04-16T05:36:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.290+0000 7ff805ffb640 1 -- 172.21.15.86:0/1397385800 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7fc005500 con 0x7ff808102210 2024-04-16T05:36:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.291+0000 7ff80edf4640 1 -- 172.21.15.86:0/1397385800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8081a79a0 con 0x7ff808102210 2024-04-16T05:36:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.291+0000 7ff805ffb640 1 -- 172.21.15.86:0/1397385800 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc010690 con 0x7ff808102210 2024-04-16T05:36:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.293+0000 7ff805ffb640 1 -- 172.21.15.86:0/1397385800 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ff7fc004680 con 0x7ff808102210 2024-04-16T05:36:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.294+0000 7ff805ffb640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7f80a3e30 0x7ff7f80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.294+0000 7ff805ffb640 1 -- 172.21.15.86:0/1397385800 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff7fc0d0f30 con 0x7ff808102210 2024-04-16T05:36:29.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.294+0000 7ff807fff640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7f80a3e30 0x7ff7f80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff807fff640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7f80a3e30 0x7ff7f80a62f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff8081013e0 tx=0x7ff7f4005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff8119e3740 1 Processor -- start 2024-04-16T05:36:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff8119e3740 1 -- start start 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 0x561fc14b9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff8119e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fc14130e0 con 0x561fc14ade60 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff8119e3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561fc13b0520 con 0x561fc14b7360 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff8119e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561fc140f0a0 con 0x561fc14ae780 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80cb90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80cb90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39252/0 (socket says 172.21.15.86:39252) 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80cb90640 1 -- 172.21.15.86:0/3094750475 learned_addr learned my addr 172.21.15.86:0/3094750475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80d391640 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 0x561fc14b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff807fff640 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80cb90640 1 -- 172.21.15.86:0/3094750475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 msgr2=0x561fc14b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80cb90640 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14b6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80cb90640 1 -- 172.21.15.86:0/3094750475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 msgr2=0x561fc14b9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80cb90640 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 0x561fc14b9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80cb90640 1 -- 172.21.15.86:0/3094750475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc13d88d0 con 0x561fc14ade60 2024-04-16T05:36:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.295+0000 7ff80d391640 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 0x561fc14b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.296+0000 7ff80cb90640 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14ae240 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc098d80 tx=0x7ff7fc09c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b786d91f0e1af9f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.296+0000 7ff7f2ffd640 1 -- 172.21.15.86:0/3094750475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc085e60 con 0x561fc14ade60 2024-04-16T05:36:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.296+0000 7ff7f2ffd640 1 -- 172.21.15.86:0/3094750475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7fc09c920 con 0x561fc14ade60 2024-04-16T05:36:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.296+0000 7ff7f2ffd640 1 -- 172.21.15.86:0/3094750475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc016bc0 con 0x561fc14ade60 2024-04-16T05:36:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 -- 172.21.15.86:0/3094750475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 msgr2=0x561fc14ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14ae240 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc098d80 tx=0x7ff7fc09c590 comp rx=0 tx=0).stop 2024-04-16T05:36:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 -- 172.21.15.86:0/3094750475 shutdown_connections 2024-04-16T05:36:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 0x561fc14b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 --2- 172.21.15.86:0/3094750475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14ae240 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 -- 172.21.15.86:0/3094750475 >> 172.21.15.86:0/3094750475 conn(0x561fc14a8590 msgr2=0x561fc14a8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 -- 172.21.15.86:0/3094750475 shutdown_connections 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 -- 172.21.15.86:0/3094750475 wait complete. 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 -- start start 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.297+0000 7ff8119e3740 1 Processor -- start 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14b7360 0x561fc14b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff8119e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fc14b9fa0 con 0x561fc14b7360 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff8119e3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561fc13cc4d0 con 0x561fc14ade60 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff8119e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561fc14ba120 con 0x561fc14ae780 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff80d391640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14b7360 0x561fc14b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff8119e3740 10 client.0 _ll_register_callbacks cb 0x561fc13b12c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff80d391640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14b7360 0x561fc14b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39256/0 (socket says 172.21.15.86:39256) 2024-04-16T05:36:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff80d391640 1 -- 172.21.15.86:0/397947737 learned_addr learned my addr 172.21.15.86:0/397947737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff80cb90640 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.298+0000 7ff807fff640 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff80d391640 1 -- 172.21.15.86:0/397947737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 msgr2=0x561fc14b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff80d391640 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff80d391640 1 -- 172.21.15.86:0/397947737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 msgr2=0x561fc14b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff80d391640 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff80d391640 1 -- 172.21.15.86:0/397947737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc11253f0 con 0x561fc14b7360 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff80cb90640 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff807fff640 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff80d391640 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14b7360 0x561fc14b4f40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff80002c700 tx=0x7ff800032460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8cf74d174d9ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff7e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff7e7fff640 1 -- 172.21.15.86:0/397947737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff800043020 con 0x561fc14b7360 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff7e7fff640 1 -- 172.21.15.86:0/397947737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff800032810 con 0x561fc14b7360 2024-04-16T05:36:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff8119e3740 1 -- 172.21.15.86:0/397947737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7fc09c920 con 0x561fc14b7360 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff7e7fff640 1 -- 172.21.15.86:0/397947737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff80004e050 con 0x561fc14b7360 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff8119e3740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.299+0000 7ff8119e3740 1 -- 172.21.15.86:0/397947737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561fc11ef7d0 con 0x561fc14b7360 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.300+0000 7ff7e7fff640 1 -- 172.21.15.86:0/397947737 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7fc09c920 con 0x561fc14b7360 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.300+0000 7ff7e7fff640 1 -- 172.21.15.86:0/397947737 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff80003f470 con 0x561fc14b7360 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.300+0000 7ff8119e3740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.300+0000 7ff8119e3740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.300+0000 7ff8119e3740 20 client.16128 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.300+0000 7ff8119e3740 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561fc14c7330 0x561fc14c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.300+0000 7ff8119e3740 4 client.16128 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.300+0000 7ff8119e3740 1 -- 172.21.15.86:0/397947737 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fc1420b00 con 0x561fc14c7330 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.300+0000 7ff80cb90640 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561fc14c7330 0x561fc14c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.301+0000 7ff7e7fff640 10 client.16128 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.301+0000 7ff80cb90640 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561fc14c7330 0x561fc14c9770 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d5650a9263b73d6 server_cookie=f518e5e296d9289 in_seq=0 out_seq=0 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.301+0000 7ff7e7fff640 1 -- 172.21.15.86:0/397947737 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fc1420b00 con 0x561fc14c7330 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.301+0000 7ff7e7fff640 10 client.16128 handle_command_reply: tid=0 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.301+0000 7ff8119e3740 1 client.16128 shutdown 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 -- 172.21.15.86:0/397947737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14b7360 msgr2=0x561fc14b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14b7360 0x561fc14b4f40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff80002c700 tx=0x7ff800032460 comp rx=0 tx=0).stop 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 -- 172.21.15.86:0/397947737 shutdown_connections 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561fc14c7330 0x561fc14c9770 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14b7360 0x561fc14b4f40 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff7e7fff640 0 client.16128 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 --2- 172.21.15.86:0/397947737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 -- 172.21.15.86:0/397947737 >> 172.21.15.86:0/397947737 conn(0x561fc14a8590 msgr2=0x561fc14aa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 -- 172.21.15.86:0/397947737 shutdown_connections 2024-04-16T05:36:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 1 -- 172.21.15.86:0/397947737 wait complete. 2024-04-16T05:36:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.302+0000 7ff8119e3740 20 client.16128 trim_cache size 0 max 16384 2024-04-16T05:36:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff8119e3740 1 Processor -- start 2024-04-16T05:36:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff8119e3740 1 -- start start 2024-04-16T05:36:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 0x561fc14cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff8119e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8081b13a0 con 0x561fc14ade60 2024-04-16T05:36:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff8119e3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8081b2ee0 con 0x561fc14b7360 2024-04-16T05:36:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff8119e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8081b1a70 con 0x561fc14ae780 2024-04-16T05:36:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff80cb90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff80cb90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39264/0 (socket says 172.21.15.86:39264) 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.328+0000 7ff80cb90640 1 -- 172.21.15.86:0/3028768002 learned_addr learned my addr 172.21.15.86:0/3028768002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff807fff640 1 --2- 172.21.15.86:0/3028768002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff80cb90640 1 -- 172.21.15.86:0/3028768002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 msgr2=0x561fc14cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff80cb90640 1 --2- 172.21.15.86:0/3028768002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff80cb90640 1 -- 172.21.15.86:0/3028768002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 msgr2=0x561fc14cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff80cb90640 1 --2- 172.21.15.86:0/3028768002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 0x561fc14cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff80cb90640 1 -- 172.21.15.86:0/3028768002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc14b2a30 con 0x561fc14ade60 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff807fff640 1 --2- 172.21.15.86:0/3028768002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff80cb90640 1 --2- 172.21.15.86:0/3028768002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14b4d60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc09c8b0 tx=0x7ff7fc0bd0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2695f2356249cd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff7f27fc640 1 -- 172.21.15.86:0/3028768002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc085e60 con 0x561fc14ade60 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff7f27fc640 1 -- 172.21.15.86:0/3028768002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7fc098f90 con 0x561fc14ade60 2024-04-16T05:36:29.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.329+0000 7ff7f27fc640 1 -- 172.21.15.86:0/3028768002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc09bfd0 con 0x561fc14ade60 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 -- 172.21.15.86:0/3028768002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 msgr2=0x561fc14b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 --2- 172.21.15.86:0/3028768002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14b4d60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc09c8b0 tx=0x7ff7fc0bd0d0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 -- 172.21.15.86:0/3028768002 shutdown_connections 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 --2- 172.21.15.86:0/3028768002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14b7360 0x561fc14cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 --2- 172.21.15.86:0/3028768002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14ae780 0x561fc14cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 --2- 172.21.15.86:0/3028768002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ade60 0x561fc14b4d60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 -- 172.21.15.86:0/3028768002 >> 172.21.15.86:0/3028768002 conn(0x561fc14a8590 msgr2=0x561fc14a8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 -- 172.21.15.86:0/3028768002 shutdown_connections 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 -- 172.21.15.86:0/3028768002 wait complete. 2024-04-16T05:36:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 -- start start 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.330+0000 7ff8119e3740 1 Processor -- start 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ae780 0x561fc14b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff8119e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14b7360 0x561fc14b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff8119e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8081b2810 con 0x561fc14ae780 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff8119e3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8081b2140 con 0x561fc14ade60 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff8119e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8081af360 con 0x561fc14b7360 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff807fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ae780 0x561fc14b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff8119e3740 10 client.0 _ll_register_callbacks cb 0x7ff800037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff807fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ae780 0x561fc14b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39266/0 (socket says 172.21.15.86:39266) 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff807fff640 1 -- 172.21.15.86:0/964571960 learned_addr learned my addr 172.21.15.86:0/964571960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff80cb90640 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff80d391640 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14b7360 0x561fc14b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff807fff640 1 -- 172.21.15.86:0/964571960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14b7360 msgr2=0x561fc14b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff807fff640 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14b7360 0x561fc14b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff807fff640 1 -- 172.21.15.86:0/964571960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 msgr2=0x561fc14bf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff807fff640 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14bf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.331+0000 7ff807fff640 1 -- 172.21.15.86:0/964571960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc14b1cc0 con 0x561fc14ae780 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff80cb90640 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14bf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff807fff640 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ae780 0x561fc14b0f30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x561fc14ad930 tx=0x7ff7f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c32caede49e89e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff7e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff7e7fff640 1 -- 172.21.15.86:0/964571960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f401e860 con 0x561fc14ae780 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff7e7fff640 1 -- 172.21.15.86:0/964571960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff7f401ea00 con 0x561fc14ae780 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff8119e3740 1 -- 172.21.15.86:0/964571960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7fc098f90 con 0x561fc14ae780 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff7e7fff640 1 -- 172.21.15.86:0/964571960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7f401ec00 con 0x561fc14ae780 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff8119e3740 10 client.16134 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff8119e3740 1 -- 172.21.15.86:0/964571960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561fc152d9a0 con 0x561fc14ae780 2024-04-16T05:36:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff7e7fff640 1 -- 172.21.15.86:0/964571960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f401eda0 con 0x561fc14ae780 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.332+0000 7ff7e7fff640 1 -- 172.21.15.86:0/964571960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff7f4028020 con 0x561fc14ae780 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.333+0000 7ff8119e3740 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.333+0000 7ff8119e3740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.333+0000 7ff8119e3740 20 client.16134 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.333+0000 7ff8119e3740 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561fc14c7330 0x561fc1530160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.333+0000 7ff8119e3740 4 client.16134 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.333+0000 7ff8119e3740 1 -- 172.21.15.86:0/964571960 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff7fc098f90 con 0x561fc14c7330 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.333+0000 7ff80cb90640 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561fc14c7330 0x561fc1530160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.333+0000 7ff7e7fff640 10 client.16134 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.333+0000 7ff80cb90640 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561fc14c7330 0x561fc1530160 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=731e1423ab01bb16 server_cookie=48e9d2283a2288f3 in_seq=0 out_seq=0 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff7e7fff640 1 -- 172.21.15.86:0/964571960 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff7fc098f90 con 0x561fc14c7330 2024-04-16T05:36:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff7e7fff640 10 client.16134 handle_command_reply: tid=0 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 client.16134 shutdown 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 -- 172.21.15.86:0/964571960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ae780 msgr2=0x561fc14b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ae780 0x561fc14b0f30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x561fc14ad930 tx=0x7ff7f4004730 comp rx=0 tx=0).stop 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 -- 172.21.15.86:0/964571960 shutdown_connections 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561fc14c7330 0x561fc1530160 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff7e7fff640 0 client.16134 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fc14b7360 0x561fc14b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc14ae780 0x561fc14b0f30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 --2- 172.21.15.86:0/964571960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561fc14ade60 0x561fc14bf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 -- 172.21.15.86:0/964571960 >> 172.21.15.86:0/964571960 conn(0x561fc14a8590 msgr2=0x561fc14a6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 -- 172.21.15.86:0/964571960 shutdown_connections 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 1 -- 172.21.15.86:0/964571960 wait complete. 2024-04-16T05:36:29.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.334+0000 7ff8119e3740 20 client.16134 trim_cache size 0 max 16384 2024-04-16T05:36:29.340 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:36:29.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 -- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7f80a3e30 msgr2=0x7ff7f80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7f80a3e30 0x7ff7f80a62f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff8081013e0 tx=0x7ff7f4005f90 comp rx=0 tx=0).stop 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 -- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff808102210 msgr2=0x7ff8081a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff808102210 0x7ff8081a0790 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e9f0 tx=0x7ff7fc00edf0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 -- 172.21.15.86:0/1397385800 shutdown_connections 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff80819a940 0x7ff80819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7f80a3e30 0x7ff7f80a62f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8081072e0 0x7ff8081a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 --2- 172.21.15.86:0/1397385800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff808102210 0x7ff8081a0790 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 -- 172.21.15.86:0/1397385800 >> 172.21.15.86:0/1397385800 conn(0x7ff8080fdad0 msgr2=0x7ff8080ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 -- 172.21.15.86:0/1397385800 shutdown_connections 2024-04-16T05:36:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:29.335+0000 7ff80edf4640 1 -- 172.21.15.86:0/1397385800 wait complete. 2024-04-16T05:36:29.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:29 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1080631836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:36:29.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:29 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1080631836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:36:29.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:29 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1080631836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:36:30.349 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:36:30.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:36:30.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:36:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.523+0000 7fe492c2a640 1 -- 172.21.15.86:0/3602812162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 msgr2=0x7fe48c102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.523+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3602812162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 0x7fe48c102620 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe480009900 tx=0x7fe48002bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.523+0000 7fe492c2a640 1 -- 172.21.15.86:0/3602812162 shutdown_connections 2024-04-16T05:36:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.523+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3602812162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe48c107310 0x7fe48c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.523+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3602812162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c102bf0 0x7fe48c106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.523+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3602812162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 0x7fe48c102620 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.523+0000 7fe492c2a640 1 -- 172.21.15.86:0/3602812162 >> 172.21.15.86:0/3602812162 conn(0x7fe48c079450 msgr2=0x7fe48c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.524+0000 7fe492c2a640 1 -- 172.21.15.86:0/3602812162 shutdown_connections 2024-04-16T05:36:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.524+0000 7fe492c2a640 1 -- 172.21.15.86:0/3602812162 wait complete. 2024-04-16T05:36:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.524+0000 7fe492c2a640 1 Processor -- start 2024-04-16T05:36:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.524+0000 7fe492c2a640 1 -- start start 2024-04-16T05:36:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe492c2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 0x7fe48c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe4909c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 0x7fe48c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe4909c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 0x7fe48c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39284/0 (socket says 172.21.15.86:39284) 2024-04-16T05:36:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe4909c6640 1 -- 172.21.15.86:0/3073804318 learned_addr learned my addr 172.21.15.86:0/3073804318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c107310 0x7fe48c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe48c19cfa0 0x7fe48c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c1123a0 con 0x7fe48c102240 2024-04-16T05:36:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c112220 con 0x7fe48c19cfa0 2024-04-16T05:36:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c112520 con 0x7fe48c107310 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.525+0000 7fe4911c7640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe48c19cfa0 0x7fe48c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe48bfff640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c107310 0x7fe48c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe4909c6640 1 -- 172.21.15.86:0/3073804318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c107310 msgr2=0x7fe48c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe4909c6640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c107310 0x7fe48c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe4909c6640 1 -- 172.21.15.86:0/3073804318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe48c19cfa0 msgr2=0x7fe48c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe4909c6640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe48c19cfa0 0x7fe48c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe4909c6640 1 -- 172.21.15.86:0/3073804318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48c19da90 con 0x7fe48c102240 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe4911c7640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe48c19cfa0 0x7fe48c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe4909c6640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 0x7fe48c1986e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe480036040 tx=0x7fe48002ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe489ffb640 1 -- 172.21.15.86:0/3073804318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480041070 con 0x7fe48c102240 2024-04-16T05:36:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48c19dd20 con 0x7fe48c102240 2024-04-16T05:36:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe489ffb640 1 -- 172.21.15.86:0/3073804318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4800054e0 con 0x7fe48c102240 2024-04-16T05:36:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.526+0000 7fe489ffb640 1 -- 172.21.15.86:0/3073804318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480034440 con 0x7fe48c102240 2024-04-16T05:36:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.527+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48c074520 con 0x7fe48c102240 2024-04-16T05:36:30.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.530+0000 7fe495815740 1 Processor -- start 2024-04-16T05:36:30.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.530+0000 7fe495815740 1 -- start start 2024-04-16T05:36:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.530+0000 7fe489ffb640 1 -- 172.21.15.86:0/3073804318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe480044070 con 0x7fe48c102240 2024-04-16T05:36:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.530+0000 7fe495815740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c62c0 0x55a4123c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.530+0000 7fe495815740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.530+0000 7fe495815740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.530+0000 7fe495815740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a412322e90 con 0x55a4123c6be0 2024-04-16T05:36:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe495815740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a4122c04c0 con 0x55a4123c62c0 2024-04-16T05:36:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe495815740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a41231ee50 con 0x55a4123cf7c0 2024-04-16T05:36:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe4909c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe48bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe4909c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39296/0 (socket says 172.21.15.86:39296) 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe48bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123d1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:60310/0 (socket says 172.21.15.86:60310) 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe4909c6640 1 -- 172.21.15.86:0/3165922361 learned_addr learned my addr 172.21.15.86:0/3165922361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe4911c7640 1 --2- 172.21.15.86:0/3165922361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c62c0 0x55a4123c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe48bfff640 1 -- 172.21.15.86:0/3165922361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c62c0 msgr2=0x55a4123c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe48bfff640 1 --2- 172.21.15.86:0/3165922361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c62c0 0x55a4123c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe48bfff640 1 -- 172.21.15.86:0/3165922361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 msgr2=0x55a4123cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe48bfff640 1 --2- 172.21.15.86:0/3165922361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe48bfff640 1 -- 172.21.15.86:0/3165922361 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4120fd7e0 con 0x55a4123cf7c0 2024-04-16T05:36:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.531+0000 7fe4909c6640 1 --2- 172.21.15.86:0/3165922361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.532+0000 7fe48bfff640 1 --2- 172.21.15.86:0/3165922361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123d1bb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe48c068f00 tx=0x7fe4780113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66ca463b386a2fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.532+0000 7fe476ffd640 1 -- 172.21.15.86:0/3165922361 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478011de0 con 0x55a4123cf7c0 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.532+0000 7fe476ffd640 1 -- 172.21.15.86:0/3165922361 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe478012930 con 0x55a4123cf7c0 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.532+0000 7fe476ffd640 1 -- 172.21.15.86:0/3165922361 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47800fb20 con 0x55a4123cf7c0 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.532+0000 7fe489ffb640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe47c0a3fd0 0x7fe47c0a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.532+0000 7fe4911c7640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe47c0a3fd0 0x7fe47c0a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe489ffb640 1 -- 172.21.15.86:0/3073804318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe4800f2e00 con 0x7fe48c102240 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe495815740 1 -- 172.21.15.86:0/3165922361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 msgr2=0x55a4123d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe495815740 1 --2- 172.21.15.86:0/3165922361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123d1bb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe48c068f00 tx=0x7fe4780113e0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe4911c7640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe47c0a3fd0 0x7fe47c0a6490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe484005fd0 tx=0x7fe484012040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe495815740 1 -- 172.21.15.86:0/3165922361 shutdown_connections 2024-04-16T05:36:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe495815740 1 --2- 172.21.15.86:0/3165922361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123d1bb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe495815740 1 --2- 172.21.15.86:0/3165922361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe495815740 1 --2- 172.21.15.86:0/3165922361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c62c0 0x55a4123c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe495815740 1 -- 172.21.15.86:0/3165922361 >> 172.21.15.86:0/3165922361 conn(0x55a4123c09f0 msgr2=0x55a4123c0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe495815740 1 -- 172.21.15.86:0/3165922361 shutdown_connections 2024-04-16T05:36:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.533+0000 7fe495815740 1 -- 172.21.15.86:0/3165922361 wait complete. 2024-04-16T05:36:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.534+0000 7fe495815740 1 -- start start 2024-04-16T05:36:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.534+0000 7fe495815740 1 Processor -- start 2024-04-16T05:36:30.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe495815740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c62c0 0x55a4123cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe495815740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 0x55a4123ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe495815740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe495815740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a4123d25b0 con 0x55a4123c62c0 2024-04-16T05:36:30.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe495815740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a4122ab410 con 0x55a4123c6be0 2024-04-16T05:36:30.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe495815740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a4123d2730 con 0x55a4123cf7c0 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4909c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c62c0 0x55a4123cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4909c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c62c0 0x55a4123cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39306/0 (socket says 172.21.15.86:39306) 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe48bfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 0x55a4123ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4909c6640 1 -- 172.21.15.86:0/1585219739 learned_addr learned my addr 172.21.15.86:0/1585219739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe495815740 10 client.0 _ll_register_callbacks cb 0x55a4122a6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4911c7640 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4909c6640 1 -- 172.21.15.86:0/1585219739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 msgr2=0x55a4123cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4909c6640 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4909c6640 1 -- 172.21.15.86:0/1585219739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 msgr2=0x55a4123ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4909c6640 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 0x55a4123ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4909c6640 1 -- 172.21.15.86:0/1585219739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a41215df70 con 0x55a4123c62c0 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe4911c7640 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.535+0000 7fe48bfff640 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 0x55a4123ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.536+0000 7fe4909c6640 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c62c0 0x55a4123cc7d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe4800be890 tx=0x7fe480005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f380a00b353d627 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.536+0000 7fe46bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.536+0000 7fe46bfff640 1 -- 172.21.15.86:0/1585219739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48004b030 con 0x55a4123c62c0 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.536+0000 7fe46bfff640 1 -- 172.21.15.86:0/1585219739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe4800a90b0 con 0x55a4123c62c0 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.536+0000 7fe495815740 1 -- 172.21.15.86:0/1585219739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe478012930 con 0x55a4123c62c0 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.536+0000 7fe46bfff640 1 -- 172.21.15.86:0/1585219739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4800be6d0 con 0x55a4123c62c0 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.536+0000 7fe495815740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.536+0000 7fe495815740 1 -- 172.21.15.86:0/1585219739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a411ff5170 con 0x55a4123c62c0 2024-04-16T05:36:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.537+0000 7fe46bfff640 1 -- 172.21.15.86:0/1585219739 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe478012930 con 0x55a4123c62c0 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.537+0000 7fe46bfff640 1 -- 172.21.15.86:0/1585219739 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe4800beb60 con 0x55a4123c62c0 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.537+0000 7fe495815740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.537+0000 7fe495815740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.537+0000 7fe495815740 20 client.16146 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.537+0000 7fe495815740 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a4123dfad0 0x55a4123e1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.537+0000 7fe495815740 4 client.16146 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.537+0000 7fe495815740 1 -- 172.21.15.86:0/1585219739 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4123308b0 con 0x55a4123dfad0 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.538+0000 7fe48bfff640 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a4123dfad0 0x55a4123e1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.538+0000 7fe48bfff640 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a4123dfad0 0x55a4123e1f60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf52f9dd63dfd07 server_cookie=f4db48fdfdc0e0b8 in_seq=0 out_seq=0 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.538+0000 7fe46bfff640 10 client.16146 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:30.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.539+0000 7fe46bfff640 1 -- 172.21.15.86:0/1585219739 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4123308b0 con 0x55a4123dfad0 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.539+0000 7fe46bfff640 10 client.16146 handle_command_reply: tid=0 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.539+0000 7fe495815740 1 client.16146 shutdown 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.539+0000 7fe495815740 1 -- 172.21.15.86:0/1585219739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c62c0 msgr2=0x55a4123cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.539+0000 7fe495815740 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c62c0 0x55a4123cc7d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe4800be890 tx=0x7fe480005890 comp rx=0 tx=0).stop 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.539+0000 7fe495815740 1 -- 172.21.15.86:0/1585219739 shutdown_connections 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.539+0000 7fe495815740 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a4123dfad0 0x55a4123e1f60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.540+0000 7fe46bfff640 0 client.16146 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.540+0000 7fe495815740 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123cf7c0 0x55a4123cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.540+0000 7fe495815740 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 0x55a4123ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.540+0000 7fe495815740 1 --2- 172.21.15.86:0/1585219739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c62c0 0x55a4123cc7d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.540+0000 7fe495815740 1 -- 172.21.15.86:0/1585219739 >> 172.21.15.86:0/1585219739 conn(0x55a4123c09f0 msgr2=0x55a4123c2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.540+0000 7fe495815740 1 -- 172.21.15.86:0/1585219739 shutdown_connections 2024-04-16T05:36:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.540+0000 7fe495815740 1 -- 172.21.15.86:0/1585219739 wait complete. 2024-04-16T05:36:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.540+0000 7fe495815740 20 client.16146 trim_cache size 0 max 16384 2024-04-16T05:36:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.558+0000 7fe495815740 1 Processor -- start 2024-04-16T05:36:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.558+0000 7fe495815740 1 -- start start 2024-04-16T05:36:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.558+0000 7fe495815740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.558+0000 7fe495815740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 0x55a4123db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.558+0000 7fe495815740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123cf7c0 0x55a4123b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.558+0000 7fe495815740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c1ad040 con 0x55a4123cf7c0 2024-04-16T05:36:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.558+0000 7fe495815740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c1aeb80 con 0x55a4123c6be0 2024-04-16T05:36:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.558+0000 7fe495815740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c1ad710 con 0x55a4123c62c0 2024-04-16T05:36:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4911c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123cf7c0 0x55a4123b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4909c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4909c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:60322/0 (socket says 172.21.15.86:60322) 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4909c6640 1 -- 172.21.15.86:0/173878022 learned_addr learned my addr 172.21.15.86:0/173878022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe48bfff640 1 --2- 172.21.15.86:0/173878022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 0x55a4123db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4909c6640 1 -- 172.21.15.86:0/173878022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 msgr2=0x55a4123db4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4909c6640 1 --2- 172.21.15.86:0/173878022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 0x55a4123db4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4909c6640 1 -- 172.21.15.86:0/173878022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123cf7c0 msgr2=0x55a4123b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4909c6640 1 --2- 172.21.15.86:0/173878022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123cf7c0 0x55a4123b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4909c6640 1 -- 172.21.15.86:0/173878022 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4123b6660 con 0x55a4123c62c0 2024-04-16T05:36:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4911c7640 1 --2- 172.21.15.86:0/173878022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123cf7c0 0x55a4123b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4909c6640 1 --2- 172.21.15.86:0/173878022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123cef00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe480005ab0 tx=0x7fe48002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b94771741db8346b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4767fc640 1 -- 172.21.15.86:0/173878022 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48004b030 con 0x55a4123c62c0 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4767fc640 1 -- 172.21.15.86:0/173878022 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4800bf000 con 0x55a4123c62c0 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.559+0000 7fe4767fc640 1 -- 172.21.15.86:0/173878022 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4800be260 con 0x55a4123c62c0 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.560+0000 7fe495815740 1 -- 172.21.15.86:0/173878022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 msgr2=0x55a4123cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.560+0000 7fe495815740 1 --2- 172.21.15.86:0/173878022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123cef00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe480005ab0 tx=0x7fe48002bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.561+0000 7fe495815740 1 -- 172.21.15.86:0/173878022 shutdown_connections 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.561+0000 7fe495815740 1 --2- 172.21.15.86:0/173878022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123cf7c0 0x55a4123b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.561+0000 7fe495815740 1 --2- 172.21.15.86:0/173878022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123c6be0 0x55a4123db4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.561+0000 7fe495815740 1 --2- 172.21.15.86:0/173878022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123cef00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.561+0000 7fe495815740 1 -- 172.21.15.86:0/173878022 >> 172.21.15.86:0/173878022 conn(0x55a4123c09f0 msgr2=0x55a4123b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.561+0000 7fe495815740 1 -- 172.21.15.86:0/173878022 shutdown_connections 2024-04-16T05:36:30.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.561+0000 7fe495815740 1 -- 172.21.15.86:0/173878022 wait complete. 2024-04-16T05:36:30.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.561+0000 7fe495815740 1 -- start start 2024-04-16T05:36:30.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.561+0000 7fe495815740 1 Processor -- start 2024-04-16T05:36:30.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe495815740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe495815740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe495815740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123cf7c0 0x55a4123c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe495815740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c1ae4b0 con 0x55a4123c6be0 2024-04-16T05:36:30.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe495815740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c1adde0 con 0x55a4123cf7c0 2024-04-16T05:36:30.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe495815740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c1ab000 con 0x55a4123c62c0 2024-04-16T05:36:30.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe495815740 10 client.0 _ll_register_callbacks cb 0x7fe480004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe48bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe48bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39332/0 (socket says 172.21.15.86:39332) 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe48bfff640 1 -- 172.21.15.86:0/3224786637 learned_addr learned my addr 172.21.15.86:0/3224786637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe4911c7640 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123cf7c0 0x55a4123c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe4909c6640 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe48bfff640 1 -- 172.21.15.86:0/3224786637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 msgr2=0x55a4123d7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe48bfff640 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123d7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe48bfff640 1 -- 172.21.15.86:0/3224786637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123cf7c0 msgr2=0x55a4123c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe48bfff640 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123cf7c0 0x55a4123c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe48bfff640 1 -- 172.21.15.86:0/3224786637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4124450a0 con 0x55a4123c6be0 2024-04-16T05:36:30.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe48bfff640 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123c9310 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55a4123c1b80 tx=0x7fe478005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e08d022b10e7394 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.562+0000 7fe46bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe46bfff640 1 -- 172.21.15.86:0/3224786637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478034060 con 0x55a4123c6be0 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe495815740 1 -- 172.21.15.86:0/3224786637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4800bf000 con 0x55a4123c6be0 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe46bfff640 1 -- 172.21.15.86:0/3224786637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe4780052b0 con 0x55a4123c6be0 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe46bfff640 1 -- 172.21.15.86:0/3224786637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe47803b050 con 0x55a4123c6be0 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe495815740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe495815740 1 -- 172.21.15.86:0/3224786637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a412446220 con 0x55a4123c6be0 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe46bfff640 1 -- 172.21.15.86:0/3224786637 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4800bf000 con 0x55a4123c6be0 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe46bfff640 1 -- 172.21.15.86:0/3224786637 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe478003710 con 0x55a4123c6be0 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe495815740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:30.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe495815740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe495815740 20 client.16152 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe495815740 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a4123dfad0 0x55a4124486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe495815740 4 client.16152 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.563+0000 7fe495815740 1 -- 172.21.15.86:0/3224786637 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a41232a720 con 0x55a4123dfad0 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.564+0000 7fe4909c6640 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a4123dfad0 0x55a4124486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.564+0000 7fe46bfff640 10 client.16152 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.564+0000 7fe4909c6640 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a4123dfad0 0x55a4124486d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d5cffe393db31e server_cookie=5d7390e0a68c6a7f in_seq=0 out_seq=0 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe46bfff640 1 -- 172.21.15.86:0/3224786637 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a41232a720 con 0x55a4123dfad0 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe46bfff640 10 client.16152 handle_command_reply: tid=0 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 client.16152 shutdown 2024-04-16T05:36:30.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 -- 172.21.15.86:0/3224786637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 msgr2=0x55a4123c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123c9310 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55a4123c1b80 tx=0x7fe478005f90 comp rx=0 tx=0).stop 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 -- 172.21.15.86:0/3224786637 shutdown_connections 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a4123dfad0 0x55a4124486d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a4123cf7c0 0x55a4123c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4123c6be0 0x55a4123c9310 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe46bfff640 0 client.16152 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 --2- 172.21.15.86:0/3224786637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4123c62c0 0x55a4123d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 -- 172.21.15.86:0/3224786637 >> 172.21.15.86:0/3224786637 conn(0x55a4123c09f0 msgr2=0x55a4123b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 -- 172.21.15.86:0/3224786637 shutdown_connections 2024-04-16T05:36:30.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 1 -- 172.21.15.86:0/3224786637 wait complete. 2024-04-16T05:36:30.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.566+0000 7fe495815740 20 client.16152 trim_cache size 0 max 16384 2024-04-16T05:36:30.573 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:36:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:36:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "0172cc0a-ad97-40b1-bb10-4837798f3a69", 2024-04-16T05:36:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:36:30.573 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:36:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe47c0a3fd0 msgr2=0x7fe47c0a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe47c0a3fd0 0x7fe47c0a6490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe484005fd0 tx=0x7fe484012040 comp rx=0 tx=0).stop 2024-04-16T05:36:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 msgr2=0x7fe48c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 0x7fe48c1986e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe480036040 tx=0x7fe48002ce20 comp rx=0 tx=0).stop 2024-04-16T05:36:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 shutdown_connections 2024-04-16T05:36:30.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe47c0a3fd0 0x7fe47c0a6490 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe48c19cfa0 0x7fe48c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c107310 0x7fe48c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 --2- 172.21.15.86:0/3073804318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c102240 0x7fe48c1986e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 >> 172.21.15.86:0/3073804318 conn(0x7fe48c079450 msgr2=0x7fe48c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 shutdown_connections 2024-04-16T05:36:30.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.567+0000 7fe492c2a640 1 -- 172.21.15.86:0/3073804318 wait complete. 2024-04-16T05:36:30.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:36:30.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:36:30.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:30 smithi194 ceph-mon[35865]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:36:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.688+0000 7f7471f24640 1 -- 172.21.15.86:0/39053876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c102270 msgr2=0x7f746c102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.688+0000 7f7471f24640 1 --2- 172.21.15.86:0/39053876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c102270 0x7f746c102650 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f745c009900 tx=0x7f745c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.688+0000 7f7471f24640 1 -- 172.21.15.86:0/39053876 shutdown_connections 2024-04-16T05:36:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.688+0000 7f7471f24640 1 --2- 172.21.15.86:0/39053876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f746c107340 0x7f746c10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.688+0000 7f7471f24640 1 --2- 172.21.15.86:0/39053876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f746c102c20 0x7f746c106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.688+0000 7f7471f24640 1 --2- 172.21.15.86:0/39053876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c102270 0x7f746c102650 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.688+0000 7f7471f24640 1 -- 172.21.15.86:0/39053876 >> 172.21.15.86:0/39053876 conn(0x7f746c079490 msgr2=0x7f746c0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.688+0000 7f7471f24640 1 -- 172.21.15.86:0/39053876 shutdown_connections 2024-04-16T05:36:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.688+0000 7f7471f24640 1 -- 172.21.15.86:0/39053876 wait complete. 2024-04-16T05:36:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.689+0000 7f7471f24640 1 Processor -- start 2024-04-16T05:36:30.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.689+0000 7f7471f24640 1 -- start start 2024-04-16T05:36:30.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.689+0000 7f7471f24640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f746c102270 0x7f746c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.689+0000 7f7471f24640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c107340 0x7f746c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.689+0000 7f7471f24640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f746c198780 0x7f746c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f7471f24640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f746c112420 con 0x7f746c107340 2024-04-16T05:36:30.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f7471f24640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1122a0 con 0x7f746c198780 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f7471f24640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1125a0 con 0x7f746c102270 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c107340 0x7f746c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c107340 0x7f746c19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54902/0 (socket says 172.21.15.86:54902) 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746affd640 1 -- 172.21.15.86:0/3774196023 learned_addr learned my addr 172.21.15.86:0/3774196023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746b7fe640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f746c102270 0x7f746c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746bfff640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f746c198780 0x7f746c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746b7fe640 1 -- 172.21.15.86:0/3774196023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f746c198780 msgr2=0x7f746c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746b7fe640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f746c198780 0x7f746c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746b7fe640 1 -- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c107340 msgr2=0x7f746c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746b7fe640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c107340 0x7f746c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746b7fe640 1 -- 172.21.15.86:0/3774196023 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c199310 con 0x7f746c102270 2024-04-16T05:36:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746affd640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c107340 0x7f746c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:30.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f746b7fe640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f746c102270 0x7f746c19e610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f745c036040 tx=0x7f745c02cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f7468ff9640 1 -- 172.21.15.86:0/3774196023 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c049070 con 0x7f746c102270 2024-04-16T05:36:30.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f7471f24640 1 -- 172.21.15.86:0/3774196023 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c1995a0 con 0x7f746c102270 2024-04-16T05:36:30.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f7468ff9640 1 -- 172.21.15.86:0/3774196023 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f745c004450 con 0x7f746c102270 2024-04-16T05:36:30.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.690+0000 7f7471f24640 1 -- 172.21.15.86:0/3774196023 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c074550 con 0x7f746c102270 2024-04-16T05:36:30.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.691+0000 7f7468ff9640 1 -- 172.21.15.86:0/3774196023 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c003dc0 con 0x7f746c102270 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.693+0000 7f7474bed740 1 Processor -- start 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.693+0000 7f7474bed740 1 -- start start 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f7474bed740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe6e50 0x55a510fe7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f7474bed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe7770 0x55a510fefe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f7474bed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510ff0350 0x55a510ff2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f7474bed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a510f4c180 con 0x55a510ff0350 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f7474bed740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a510ee9360 con 0x55a510fe6e50 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f7474bed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a510f48140 con 0x55a510fe7770 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510ff0350 0x55a510ff2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510ff0350 0x55a510ff2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54912/0 (socket says 172.21.15.86:54912) 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746affd640 1 -- 172.21.15.86:0/999617593 learned_addr learned my addr 172.21.15.86:0/999617593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746bfff640 1 --2- 172.21.15.86:0/999617593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe6e50 0x55a510fe7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f7468ff9640 1 -- 172.21.15.86:0/3774196023 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f745c04c070 con 0x7f746c102270 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746affd640 1 -- 172.21.15.86:0/999617593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe7770 msgr2=0x55a510fefe10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746affd640 1 --2- 172.21.15.86:0/999617593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe7770 0x55a510fefe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746affd640 1 -- 172.21.15.86:0/999617593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe6e50 msgr2=0x55a510fe7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746affd640 1 --2- 172.21.15.86:0/999617593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe6e50 0x55a510fe7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746affd640 1 -- 172.21.15.86:0/999617593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a510d287d0 con 0x55a510ff0350 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.694+0000 7f746affd640 1 --2- 172.21.15.86:0/999617593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510ff0350 0x55a510ff2740 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f74600104d0 tx=0x7f7460010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc608752592923c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.695+0000 7f7451ffb640 1 -- 172.21.15.86:0/999617593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460011820 con 0x55a510ff0350 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.695+0000 7f7451ffb640 1 -- 172.21.15.86:0/999617593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f746001c480 con 0x55a510ff0350 2024-04-16T05:36:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.695+0000 7f7451ffb640 1 -- 172.21.15.86:0/999617593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746000e690 con 0x55a510ff0350 2024-04-16T05:36:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.695+0000 7f7468ff9640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74540a3f50 0x7f74540a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.695+0000 7f7468ff9640 1 -- 172.21.15.86:0/3774196023 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f745c0f60b0 con 0x7f746c102270 2024-04-16T05:36:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.695+0000 7f7474bed740 1 -- 172.21.15.86:0/999617593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510ff0350 msgr2=0x55a510ff2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.695+0000 7f7474bed740 1 --2- 172.21.15.86:0/999617593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510ff0350 0x55a510ff2740 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f74600104d0 tx=0x7f7460010990 comp rx=0 tx=0).stop 2024-04-16T05:36:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f746bfff640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74540a3f50 0x7f74540a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f7474bed740 1 -- 172.21.15.86:0/999617593 shutdown_connections 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f7474bed740 1 --2- 172.21.15.86:0/999617593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510ff0350 0x55a510ff2740 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f7474bed740 1 --2- 172.21.15.86:0/999617593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe7770 0x55a510fefe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f7474bed740 1 --2- 172.21.15.86:0/999617593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe6e50 0x55a510fe7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f7474bed740 1 -- 172.21.15.86:0/999617593 >> 172.21.15.86:0/999617593 conn(0x55a510fe1580 msgr2=0x55a510fe1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f7474bed740 1 -- 172.21.15.86:0/999617593 shutdown_connections 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f7474bed740 1 -- 172.21.15.86:0/999617593 wait complete. 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f746bfff640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74540a3f50 0x7f74540a6410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7464005fd0 tx=0x7f7464009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f7474bed740 1 -- start start 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.696+0000 7f7474bed740 1 Processor -- start 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f7474bed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe6e50 0x55a510fed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f7474bed740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe7770 0x55a510fed980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f7474bed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510ff0350 0x55a510fedec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe6e50 0x55a510fed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe6e50 0x55a510fed440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54918/0 (socket says 172.21.15.86:54918) 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746b7fe640 1 -- 172.21.15.86:0/69730560 learned_addr learned my addr 172.21.15.86:0/69730560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a510ff3140 con 0x55a510fe6e50 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a510f052c0 con 0x55a510fe7770 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a510ff32c0 con 0x55a510ff0350 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746affd640 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe7770 0x55a510fed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746bfff640 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510ff0350 0x55a510fedec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746b7fe640 1 -- 172.21.15.86:0/69730560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510ff0350 msgr2=0x55a510fedec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746b7fe640 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510ff0350 0x55a510fedec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746b7fe640 1 -- 172.21.15.86:0/69730560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe7770 msgr2=0x55a510fed980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746b7fe640 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe7770 0x55a510fed980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746b7fe640 1 -- 172.21.15.86:0/69730560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a510c5e3f0 con 0x55a510fe6e50 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f746bfff640 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510ff0350 0x55a510fedec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.697+0000 7f7474bed740 10 client.0 _ll_register_callbacks cb 0x55a510eec090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f746001c480 con 0x55a510fe6e50 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f746b7fe640 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe6e50 0x55a510fed440 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f745c009900 tx=0x7f745c0c1980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e595ec77155afd08 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f744affd640 10 client.16164 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f744affd640 1 -- 172.21.15.86:0/69730560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c0c22b0 con 0x55a510fe6e50 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f744affd640 1 -- 172.21.15.86:0/69730560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f745c0f3be0 con 0x55a510fe6e50 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f744affd640 1 -- 172.21.15.86:0/69730560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f745c0f3de0 con 0x55a510fe6e50 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f7474bed740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55a510c1e0f0 con 0x55a510fe6e50 2024-04-16T05:36:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f744affd640 1 -- 172.21.15.86:0/69730560 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f745c0ca390 con 0x55a510fe6e50 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f7474bed740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f7474bed740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.698+0000 7f7474bed740 20 client.16164 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.699+0000 7f7474bed740 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a511008cb0 0x55a51100b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.699+0000 7f7474bed740 4 client.16164 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.699+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a510f59ba0 con 0x55a511008cb0 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.699+0000 7f746affd640 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a511008cb0 0x55a51100b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.699+0000 7f744affd640 10 client.16164 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.699+0000 7f746affd640 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a511008cb0 0x55a51100b100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ef98a93a645df2 server_cookie=15301a7fbd778063 in_seq=0 out_seq=0 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f744affd640 1 -- 172.21.15.86:0/69730560 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a510f59ba0 con 0x55a511008cb0 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f744affd640 10 client.16164 handle_command_reply: tid=0 2024-04-16T05:36:30.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 client.16164 shutdown 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe6e50 msgr2=0x55a510fed440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe6e50 0x55a510fed440 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f745c009900 tx=0x7f745c0c1980 comp rx=0 tx=0).stop 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 shutdown_connections 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a511008cb0 0x55a51100b100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510ff0350 0x55a510fedec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510fe7770 0x55a510fed980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 --2- 172.21.15.86:0/69730560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe6e50 0x55a510fed440 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 >> 172.21.15.86:0/69730560 conn(0x55a510fe1580 msgr2=0x55a510fe32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f744affd640 0 client.16164 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.700+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 shutdown_connections 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.701+0000 7f7474bed740 1 -- 172.21.15.86:0/69730560 wait complete. 2024-04-16T05:36:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.701+0000 7f7474bed740 20 client.16164 trim_cache size 0 max 16384 2024-04-16T05:36:30.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f7474bed740 1 Processor -- start 2024-04-16T05:36:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:30 smithi086 ceph-mon[32028]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:36:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f7474bed740 1 -- start start 2024-04-16T05:36:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f7474bed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 0x55a510fefb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f7474bed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a5110048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f7474bed740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 0x55a510fec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f7474bed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1ad220 con 0x55a510fe7770 2024-04-16T05:36:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f7474bed740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1aed60 con 0x55a510ff0350 2024-04-16T05:36:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f7474bed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1ad8f0 con 0x55a510fe6e50 2024-04-16T05:36:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f746affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a5110048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f746affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a5110048c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54920/0 (socket says 172.21.15.86:54920) 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.718+0000 7f746affd640 1 -- 172.21.15.86:0/1598650084 learned_addr learned my addr 172.21.15.86:0/1598650084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f746b7fe640 1 --2- 172.21.15.86:0/1598650084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 0x55a510fefb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f746bfff640 1 --2- 172.21.15.86:0/1598650084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 0x55a510fec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f746affd640 1 -- 172.21.15.86:0/1598650084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 msgr2=0x55a510fefb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f746affd640 1 --2- 172.21.15.86:0/1598650084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 0x55a510fefb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f746affd640 1 -- 172.21.15.86:0/1598650084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 msgr2=0x55a510fec5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f746affd640 1 --2- 172.21.15.86:0/1598650084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 0x55a510fec5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f746affd640 1 -- 172.21.15.86:0/1598650084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a510fe61d0 con 0x55a510fe7770 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f746affd640 1 --2- 172.21.15.86:0/1598650084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a5110048c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f74600104d0 tx=0x7f746001ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a883d1ca2a77bc6f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f74517fa640 1 -- 172.21.15.86:0/1598650084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460005430 con 0x55a510fe7770 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f74517fa640 1 -- 172.21.15.86:0/1598650084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74600055d0 con 0x55a510fe7770 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.719+0000 7f74517fa640 1 -- 172.21.15.86:0/1598650084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746002c8a0 con 0x55a510fe7770 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 -- 172.21.15.86:0/1598650084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 msgr2=0x55a5110048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 --2- 172.21.15.86:0/1598650084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a5110048c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f74600104d0 tx=0x7f746001ce60 comp rx=0 tx=0).stop 2024-04-16T05:36:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 -- 172.21.15.86:0/1598650084 shutdown_connections 2024-04-16T05:36:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 --2- 172.21.15.86:0/1598650084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 0x55a510fec5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 --2- 172.21.15.86:0/1598650084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a5110048c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 --2- 172.21.15.86:0/1598650084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 0x55a510fefb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 -- 172.21.15.86:0/1598650084 >> 172.21.15.86:0/1598650084 conn(0x55a510fe1580 msgr2=0x55a510fdfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 -- 172.21.15.86:0/1598650084 shutdown_connections 2024-04-16T05:36:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 -- 172.21.15.86:0/1598650084 wait complete. 2024-04-16T05:36:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.720+0000 7f7474bed740 1 -- start start 2024-04-16T05:36:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f7474bed740 1 Processor -- start 2024-04-16T05:36:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f7474bed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 0x55a511001340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f7474bed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a510ffe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f7474bed740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 0x55a510ffedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f7474bed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1ae690 con 0x55a510fe7770 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f7474bed740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1adfc0 con 0x55a510ff0350 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f7474bed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1ab1e0 con 0x55a510fe6e50 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f7474bed740 10 client.0 _ll_register_callbacks cb 0x7f745c02cdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a510ffe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a510ffe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54934/0 (socket says 172.21.15.86:54934) 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746affd640 1 -- 172.21.15.86:0/1103248029 learned_addr learned my addr 172.21.15.86:0/1103248029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746bfff640 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 0x55a510ffedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746b7fe640 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 0x55a511001340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746affd640 1 -- 172.21.15.86:0/1103248029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 msgr2=0x55a511001340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746affd640 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 0x55a511001340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746affd640 1 -- 172.21.15.86:0/1103248029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 msgr2=0x55a510ffedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746affd640 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 0x55a510ffedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746affd640 1 -- 172.21.15.86:0/1103248029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51106e290 con 0x55a510fe7770 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.721+0000 7f746b7fe640 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 0x55a511001340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.722+0000 7f746affd640 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a510ffe6d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7460000c00 tx=0x7f746001ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5f3109079ccd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.722+0000 7f744affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.722+0000 7f744affd640 1 -- 172.21.15.86:0/1103248029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460005430 con 0x55a510fe7770 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.722+0000 7f744affd640 1 -- 172.21.15.86:0/1103248029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f746002c620 con 0x55a510fe7770 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.722+0000 7f7474bed740 1 -- 172.21.15.86:0/1103248029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74600055d0 con 0x55a510fe7770 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.722+0000 7f744affd640 1 -- 172.21.15.86:0/1103248029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f746002c860 con 0x55a510fe7770 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.722+0000 7f7474bed740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.722+0000 7f7474bed740 1 -- 172.21.15.86:0/1103248029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a51106f440 con 0x55a510fe7770 2024-04-16T05:36:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.722+0000 7f744affd640 1 -- 172.21.15.86:0/1103248029 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74600055d0 con 0x55a510fe7770 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f744affd640 1 -- 172.21.15.86:0/1103248029 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f74600049d0 con 0x55a510fe7770 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f7474bed740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f7474bed740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f7474bed740 20 client.16170 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f7474bed740 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a51106f8b0 0x55a511071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f7474bed740 4 client.16170 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f7474bed740 1 -- 172.21.15.86:0/1103248029 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a510f53a10 con 0x55a51106f8b0 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f746b7fe640 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a51106f8b0 0x55a511071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f744affd640 10 client.16170 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.723+0000 7f746b7fe640 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a51106f8b0 0x55a511071ca0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b2dd6dd904b8a9 server_cookie=3972c82dd79e2586 in_seq=0 out_seq=0 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f744affd640 1 -- 172.21.15.86:0/1103248029 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a510f53a10 con 0x55a51106f8b0 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f744affd640 10 client.16170 handle_command_reply: tid=0 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 client.16170 shutdown 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 -- 172.21.15.86:0/1103248029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 msgr2=0x55a510ffe6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a510ffe6d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7460000c00 tx=0x7f746001ced0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 -- 172.21.15.86:0/1103248029 shutdown_connections 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a51106f8b0 0x55a511071ca0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f744affd640 0 client.16170 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a510ff0350 0x55a510ffedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a510fe7770 0x55a510ffe6d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 --2- 172.21.15.86:0/1103248029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a510fe6e50 0x55a511001340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 -- 172.21.15.86:0/1103248029 >> 172.21.15.86:0/1103248029 conn(0x55a510fe1580 msgr2=0x55a510fe0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 -- 172.21.15.86:0/1103248029 shutdown_connections 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 1 -- 172.21.15.86:0/1103248029 wait complete. 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.724+0000 7f7474bed740 20 client.16170 trim_cache size 0 max 16384 2024-04-16T05:36:30.729 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:36:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:36:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:36:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "0172cc0a-ad97-40b1-bb10-4837798f3a69": { 2024-04-16T05:36:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:36:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "0172cc0a-ad97-40b1-bb10-4837798f3a69", 2024-04-16T05:36:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:36:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:36:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:36:30.730 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:36:30.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 -- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74540a3f50 msgr2=0x7f74540a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74540a3f50 0x7f74540a6410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7464005fd0 tx=0x7f7464009500 comp rx=0 tx=0).stop 2024-04-16T05:36:30.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 -- 172.21.15.86:0/3774196023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f746c102270 msgr2=0x7f746c19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:30.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f746c102270 0x7f746c19e610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f745c036040 tx=0x7f745c02cb20 comp rx=0 tx=0).stop 2024-04-16T05:36:30.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 -- 172.21.15.86:0/3774196023 shutdown_connections 2024-04-16T05:36:30.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74540a3f50 0x7f74540a6410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f746c198780 0x7f746c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f746c107340 0x7f746c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 --2- 172.21.15.86:0/3774196023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f746c102270 0x7f746c19e610 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:30.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 -- 172.21.15.86:0/3774196023 >> 172.21.15.86:0/3774196023 conn(0x7f746c079490 msgr2=0x7f746c108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:30.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 -- 172.21.15.86:0/3774196023 shutdown_connections 2024-04-16T05:36:30.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:36:30.725+0000 7f7471f24640 1 -- 172.21.15.86:0/3774196023 wait complete. 2024-04-16T05:36:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:30 smithi196 ceph-mon[36373]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:36:30.736 INFO:tasks.cephfs.filesystem:scrub status for tag:0172cc0a-ad97-40b1-bb10-4837798f3a69 - {'path': '/', 'tag': '0172cc0a-ad97-40b1-bb10-4837798f3a69', 'options': 'recursive,force'} 2024-04-16T05:36:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.319+0000 7f38043a8640 1 -- 172.21.15.86:0/1747582679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 msgr2=0x7f37fc103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.319+0000 7f38043a8640 1 --2- 172.21.15.86:0/1747582679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 0x7f37fc103670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f802bd60 comp rx=0 tx=0).stop 2024-04-16T05:36:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.319+0000 7f38043a8640 1 -- 172.21.15.86:0/1747582679 shutdown_connections 2024-04-16T05:36:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.319+0000 7f38043a8640 1 --2- 172.21.15.86:0/1747582679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37fc10c5d0 0x7f37fc10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.319+0000 7f38043a8640 1 --2- 172.21.15.86:0/1747582679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37fc103bb0 0x7f37fc10c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.319+0000 7f38043a8640 1 --2- 172.21.15.86:0/1747582679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 0x7f37fc103670 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.319+0000 7f38043a8640 1 -- 172.21.15.86:0/1747582679 >> 172.21.15.86:0/1747582679 conn(0x7f37fc078080 msgr2=0x7f37fc078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.319+0000 7f38043a8640 1 -- 172.21.15.86:0/1747582679 shutdown_connections 2024-04-16T05:36:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.319+0000 7f38043a8640 1 -- 172.21.15.86:0/1747582679 wait complete. 2024-04-16T05:36:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f38043a8640 1 Processor -- start 2024-04-16T05:36:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f38043a8640 1 -- start start 2024-04-16T05:36:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f38043a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 0x7f37fc19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f38043a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37fc103bb0 0x7f37fc19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f38043a8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37fc10c5d0 0x7f37fc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3802144640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 0x7f37fc19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f38043a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc112430 con 0x7f37fc103290 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3802144640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 0x7f37fc19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54954/0 (socket says 172.21.15.86:54954) 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3802144640 1 -- 172.21.15.86:0/2016523817 learned_addr learned my addr 172.21.15.86:0/2016523817 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1122b0 con 0x7f37fc10c5d0 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1125b0 con 0x7f37fc103bb0 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3801943640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37fc103bb0 0x7f37fc19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3802945640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37fc10c5d0 0x7f37fc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3802144640 1 -- 172.21.15.86:0/2016523817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37fc103bb0 msgr2=0x7f37fc19eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3802144640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37fc103bb0 0x7f37fc19eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3802144640 1 -- 172.21.15.86:0/2016523817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37fc10c5d0 msgr2=0x7f37fc1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3802144640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37fc10c5d0 0x7f37fc1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.320+0000 7f3802144640 1 -- 172.21.15.86:0/2016523817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37fc198f70 con 0x7f37fc103290 2024-04-16T05:36:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.321+0000 7f3802144640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 0x7f37fc19e5f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009a20 tx=0x7f37f802df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.321+0000 7f37f37fe640 1 -- 172.21.15.86:0/2016523817 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8043020 con 0x7f37fc103290 2024-04-16T05:36:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.321+0000 7f37f37fe640 1 -- 172.21.15.86:0/2016523817 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f80054e0 con 0x7f37fc103290 2024-04-16T05:36:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.321+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37fc199200 con 0x7f37fc103290 2024-04-16T05:36:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.321+0000 7f37f37fe640 1 -- 172.21.15.86:0/2016523817 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8003ce0 con 0x7f37fc103290 2024-04-16T05:36:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.321+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37fc1a5570 con 0x7f37fc103290 2024-04-16T05:36:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.324+0000 7f3806f9a740 1 Processor -- start 2024-04-16T05:36:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3806f9a740 1 -- start start 2024-04-16T05:36:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x56468027fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680280530 0x564680288bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680289110 0x56468028b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3806f9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646801e47d0 con 0x564680280530 2024-04-16T05:36:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3806f9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564680181f70 con 0x564680289110 2024-04-16T05:36:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3806f9a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646801e0790 con 0x56468027fc10 2024-04-16T05:36:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3801943640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680289110 0x56468028b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3801943640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680289110 0x56468028b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41104/0 (socket says 172.21.15.86:41104) 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3801943640 1 -- 172.21.15.86:0/3559548067 learned_addr learned my addr 172.21.15.86:0/3559548067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f37f37fe640 1 -- 172.21.15.86:0/2016523817 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f37f8046020 con 0x7f37fc103290 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.325+0000 7f3802945640 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x56468027fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f3802144640 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680280530 0x564680288bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f3801943640 1 -- 172.21.15.86:0/3559548067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 msgr2=0x56468027fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f3801943640 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x56468027fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f3801943640 1 -- 172.21.15.86:0/3559548067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680280530 msgr2=0x564680288bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f3801943640 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680280530 0x564680288bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f3801943640 1 -- 172.21.15.86:0/3559548067 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56467fef7210 con 0x564680289110 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f3802945640 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x56468027fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f3801943640 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680289110 0x56468028b500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b740 tx=0x7f37ec00bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b159587bfc1dcb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f37f37fe640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37e80a3ff0 0x7f37e80a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f3802945640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37e80a3ff0 0x7f37e80a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.326+0000 7f37f37fe640 1 -- 172.21.15.86:0/2016523817 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37f80f2bc0 con 0x7f37fc103290 2024-04-16T05:36:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.327+0000 7f37f0bfe640 1 -- 172.21.15.86:0/3559548067 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec003dd0 con 0x564680289110 2024-04-16T05:36:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.327+0000 7f37f0bfe640 1 -- 172.21.15.86:0/3559548067 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec015070 con 0x564680289110 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f37f0bfe640 1 -- 172.21.15.86:0/3559548067 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011440 con 0x564680289110 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3802945640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37e80a3ff0 0x7f37e80a64b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005fd0 tx=0x7f37f4014040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 -- 172.21.15.86:0/3559548067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680289110 msgr2=0x56468028b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680289110 0x56468028b500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b740 tx=0x7f37ec00bc00 comp rx=0 tx=0).stop 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 -- 172.21.15.86:0/3559548067 shutdown_connections 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680289110 0x56468028b500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680280530 0x564680288bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 --2- 172.21.15.86:0/3559548067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x56468027fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 -- 172.21.15.86:0/3559548067 >> 172.21.15.86:0/3559548067 conn(0x56468027a340 msgr2=0x56468027a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 -- 172.21.15.86:0/3559548067 shutdown_connections 2024-04-16T05:36:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 -- 172.21.15.86:0/3559548067 wait complete. 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 -- start start 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.328+0000 7f3806f9a740 1 Processor -- start 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x564680286010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x564680286550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680289110 0x564680286a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3806f9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56468028bf50 con 0x564680289110 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3806f9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56468016ce90 con 0x564680280530 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3806f9a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56468028c0d0 con 0x56468027fc10 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802144640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x564680286010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802945640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680289110 0x564680286a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3801943640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x564680286550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802144640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x564680286010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:44764/0 (socket says 172.21.15.86:44764) 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802144640 1 -- 172.21.15.86:0/1026904143 learned_addr learned my addr 172.21.15.86:0/1026904143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3806f9a740 10 client.0 _ll_register_callbacks cb 0x5646801a0cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802144640 1 -- 172.21.15.86:0/1026904143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 msgr2=0x564680286550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802144640 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x564680286550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802144640 1 -- 172.21.15.86:0/1026904143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680289110 msgr2=0x564680286a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802144640 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680289110 0x564680286a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802144640 1 -- 172.21.15.86:0/1026904143 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56467ffc3620 con 0x56468027fc10 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802945640 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680289110 0x564680286a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3801943640 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x564680286550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f3802144640 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x564680286010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f37f80babb0 tx=0x7f37f8009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd81a02d1c72653b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.329+0000 7f37e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.330+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1026904143 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f80be290 con 0x56468027fc10 2024-04-16T05:36:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.330+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1026904143 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37f80bca50 con 0x56468027fc10 2024-04-16T05:36:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f3806f9a740 1 -- 172.21.15.86:0/1026904143 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ec015070 con 0x56468027fc10 2024-04-16T05:36:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1026904143 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37f80be0b0 con 0x56468027fc10 2024-04-16T05:36:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f3806f9a740 10 client.25522 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f3806f9a740 1 -- 172.21.15.86:0/1026904143 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56468028bf50 con 0x56468027fc10 2024-04-16T05:36:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1026904143 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec015070 con 0x56468027fc10 2024-04-16T05:36:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1026904143 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37f8050060 con 0x56468027fc10 2024-04-16T05:36:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f3806f9a740 10 client.25522 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f3806f9a740 10 client.25522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f3806f9a740 20 client.25522 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564680299530 0x56468029b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f3806f9a740 4 client.25522 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.331+0000 7f3806f9a740 1 -- 172.21.15.86:0/1026904143 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564680184380 con 0x564680299530 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.332+0000 7f3801943640 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564680299530 0x56468029b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.332+0000 7f37e5ffb640 10 client.25522 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.332+0000 7f3801943640 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564680299530 0x56468029b9d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dcff2e7a76a664d server_cookie=b5ccc9a2d86379ca in_seq=0 out_seq=0 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1026904143 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564680184380 con 0x564680299530 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f37e5ffb640 10 client.25522 handle_command_reply: tid=0 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 client.25522 shutdown 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 -- 172.21.15.86:0/1026904143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 msgr2=0x564680286010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x564680286010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f37f80babb0 tx=0x7f37f8009290 comp rx=0 tx=0).stop 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 -- 172.21.15.86:0/1026904143 shutdown_connections 2024-04-16T05:36:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x564680299530 0x56468029b9d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564680289110 0x564680286a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f37e5ffb640 0 client.25522 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x564680286550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1026904143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468027fc10 0x564680286010 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 -- 172.21.15.86:0/1026904143 >> 172.21.15.86:0/1026904143 conn(0x56468027a340 msgr2=0x56468027c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 -- 172.21.15.86:0/1026904143 shutdown_connections 2024-04-16T05:36:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 1 -- 172.21.15.86:0/1026904143 wait complete. 2024-04-16T05:36:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.333+0000 7f3806f9a740 20 client.25522 trim_cache size 0 max 16384 2024-04-16T05:36:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.362+0000 7f3806f9a740 1 Processor -- start 2024-04-16T05:36:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.362+0000 7f3806f9a740 1 -- start start 2024-04-16T05:36:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.362+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680294e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.362+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x5646802a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.362+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 0x5646802a1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3806f9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1ad1f0 con 0x56468027fc10 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3806f9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1aed30 con 0x564680280530 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3806f9a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1ad8c0 con 0x564680289110 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802144640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680294e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802144640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680294e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54988/0 (socket says 172.21.15.86:54988) 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802144640 1 -- 172.21.15.86:0/550732847 learned_addr learned my addr 172.21.15.86:0/550732847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3801943640 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x5646802a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802945640 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 0x5646802a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802144640 1 -- 172.21.15.86:0/550732847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 msgr2=0x5646802a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802144640 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 0x5646802a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802144640 1 -- 172.21.15.86:0/550732847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 msgr2=0x5646802a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802144640 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x5646802a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802144640 1 -- 172.21.15.86:0/550732847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468027a170 con 0x56468027fc10 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3801943640 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x5646802a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f3802144640 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680294e90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009320 tx=0x7f37f8003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8456ceaf2cdd9ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f37e7fff640 1 -- 172.21.15.86:0/550732847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f80be290 con 0x56468027fc10 2024-04-16T05:36:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f37e7fff640 1 -- 172.21.15.86:0/550732847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f80f7040 con 0x56468027fc10 2024-04-16T05:36:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.363+0000 7f37e7fff640 1 -- 172.21.15.86:0/550732847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f80ba9f0 con 0x56468027fc10 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 -- 172.21.15.86:0/550732847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 msgr2=0x564680294e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680294e90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009320 tx=0x7f37f8003f70 comp rx=0 tx=0).stop 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 -- 172.21.15.86:0/550732847 shutdown_connections 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 0x5646802a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x5646802a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 --2- 172.21.15.86:0/550732847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680294e90 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 -- 172.21.15.86:0/550732847 >> 172.21.15.86:0/550732847 conn(0x56468027a340 msgr2=0x564680281270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 -- 172.21.15.86:0/550732847 shutdown_connections 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 -- 172.21.15.86:0/550732847 wait complete. 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.364+0000 7f3806f9a740 1 -- start start 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3806f9a740 1 Processor -- start 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680291400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x56468028c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3806f9a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 0x56468028ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3806f9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1ae660 con 0x56468027fc10 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3806f9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1adf90 con 0x564680280530 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3806f9a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1ab1b0 con 0x564680289110 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3806f9a740 10 client.0 _ll_register_callbacks cb 0x7f37f8003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3802144640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680291400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3801943640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x56468028c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3802144640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680291400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54994/0 (socket says 172.21.15.86:54994) 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3802144640 1 -- 172.21.15.86:0/1122696567 learned_addr learned my addr 172.21.15.86:0/1122696567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3801943640 1 -- 172.21.15.86:0/1122696567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 msgr2=0x56468028ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3802945640 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 0x56468028ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3801943640 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 0x56468028ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3801943640 1 -- 172.21.15.86:0/1122696567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 msgr2=0x564680291400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3801943640 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680291400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3801943640 1 -- 172.21.15.86:0/1122696567 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468028d1f0 con 0x564680280530 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3802945640 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 0x56468028ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.365+0000 7f3802144640 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680291400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f3801943640 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x56468028c770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f37ec004650 tx=0x7f37ec005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e3c253b84947d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f37e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1122696567 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec003de0 con 0x564680280530 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f3806f9a740 1 -- 172.21.15.86:0/1122696567 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f80f7040 con 0x564680280530 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1122696567 <== mon.2 v2:172.21.15.194:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37ec003f80 con 0x564680280530 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1122696567 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37ec038050 con 0x564680280530 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f3806f9a740 10 client.25547 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f3806f9a740 1 -- 172.21.15.86:0/1122696567 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646802ff810 con 0x564680280530 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1122696567 <== mon.2 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f80f7040 con 0x564680280530 2024-04-16T05:36:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.366+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1122696567 <== mon.2 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37ec0116a0 con 0x564680280530 2024-04-16T05:36:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.367+0000 7f3806f9a740 10 client.25547 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.367+0000 7f3806f9a740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.367+0000 7f3806f9a740 20 client.25547 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.367+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5646802ffe80 0x5646803022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.367+0000 7f3806f9a740 4 client.25547 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:36:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.367+0000 7f3806f9a740 1 -- 172.21.15.86:0/1122696567 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646801f21f0 con 0x5646802ffe80 2024-04-16T05:36:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.367+0000 7f3802144640 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5646802ffe80 0x5646803022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.367+0000 7f37e5ffb640 10 client.25547 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.367+0000 7f3802144640 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5646802ffe80 0x5646803022a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe8feef13d3504 server_cookie=81d29a10cbb94afa in_seq=0 out_seq=0 2024-04-16T05:36:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f37e5ffb640 1 -- 172.21.15.86:0/1122696567 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646801f21f0 con 0x5646802ffe80 2024-04-16T05:36:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f37e5ffb640 10 client.25547 handle_command_reply: tid=0 2024-04-16T05:36:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 client.25547 shutdown 2024-04-16T05:36:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 -- 172.21.15.86:0/1122696567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 msgr2=0x56468028c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x56468028c770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f37ec004650 tx=0x7f37ec005e50 comp rx=0 tx=0).stop 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 -- 172.21.15.86:0/1122696567 shutdown_connections 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5646802ffe80 0x5646803022a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564680289110 0x56468028ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564680280530 0x56468028c770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 --2- 172.21.15.86:0/1122696567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56468027fc10 0x564680291400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f37e5ffb640 0 client.25547 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 -- 172.21.15.86:0/1122696567 >> 172.21.15.86:0/1122696567 conn(0x56468027a340 msgr2=0x564680285fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 -- 172.21.15.86:0/1122696567 shutdown_connections 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.371+0000 7f3806f9a740 1 -- 172.21.15.86:0/1122696567 wait complete. 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f3806f9a740 20 client.25547 trim_cache size 0 max 16384 2024-04-16T05:36:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-16T05:36:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-16T05:36:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-16T05:36:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37e80a3ff0 msgr2=0x7f37e80a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37e80a3ff0 0x7f37e80a64b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005fd0 tx=0x7f37f4014040 comp rx=0 tx=0).stop 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 msgr2=0x7f37fc19e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 0x7f37fc19e5f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009a20 tx=0x7f37f802df60 comp rx=0 tx=0).stop 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 shutdown_connections 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37e80a3ff0 0x7f37e80a64b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37fc10c5d0 0x7f37fc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37fc103bb0 0x7f37fc19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 --2- 172.21.15.86:0/2016523817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37fc103290 0x7f37fc19e5f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 >> 172.21.15.86:0/2016523817 conn(0x7f37fc078080 msgr2=0x7f37fc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 shutdown_connections 2024-04-16T05:36:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:32.372+0000 7f38043a8640 1 -- 172.21.15.86:0/2016523817 wait complete. 2024-04-16T05:36:32.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:32 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:36:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:32 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:32 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:36:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:32 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:32 smithi086 ceph-mon[32028]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:36:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:32 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:36:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:32 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:32 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:36:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:32 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:32 smithi196 ceph-mon[36373]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:36:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:32 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:36:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:32 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:32 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:36:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:32 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:32 smithi194 ceph-mon[35865]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T05:36:34.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:34 smithi086 ceph-mon[32028]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-16T05:36:34.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:34 smithi196 ceph-mon[36373]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-16T05:36:34.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:34 smithi194 ceph-mon[35865]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-16T05:36:35.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:35 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:36:35.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:35 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:36:35.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:35 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:36:36.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:36 smithi086 ceph-mon[32028]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:36 smithi196 ceph-mon[36373]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:36.807 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:36 smithi194 ceph-mon[35865]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:38.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:38 smithi086 ceph-mon[32028]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:38.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:38 smithi196 ceph-mon[36373]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:38 smithi194 ceph-mon[35865]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:40.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:40 smithi086 ceph-mon[32028]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-16T05:36:40.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:40 smithi196 ceph-mon[36373]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-16T05:36:40.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:40 smithi194 ceph-mon[35865]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-16T05:36:42.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:42 smithi086 ceph-mon[32028]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:42.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:42 smithi196 ceph-mon[36373]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:42.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:42 smithi194 ceph-mon[35865]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:36:44.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:44 smithi086 ceph-mon[32028]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-16T05:36:44.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:44 smithi196 ceph-mon[36373]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-16T05:36:44.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:44 smithi194 ceph-mon[35865]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-16T05:36:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.512+0000 7f980d759640 1 -- 172.21.15.86:0/539295150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 msgr2=0x7f9808106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.512+0000 7f980d759640 1 --2- 172.21.15.86:0/539295150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 0x7f9808106a40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f97f800aa40 tx=0x7f97f802cf50 comp rx=0 tx=0).stop 2024-04-16T05:36:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.512+0000 7f980d759640 1 -- 172.21.15.86:0/539295150 shutdown_connections 2024-04-16T05:36:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.512+0000 7f980d759640 1 --2- 172.21.15.86:0/539295150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9808101010 0x7f980810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.512+0000 7f980d759640 1 --2- 172.21.15.86:0/539295150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9808100650 0x7f9808100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.512+0000 7f980d759640 1 --2- 172.21.15.86:0/539295150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 0x7f9808106a40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.512+0000 7f980d759640 1 -- 172.21.15.86:0/539295150 >> 172.21.15.86:0/539295150 conn(0x7f9808077490 msgr2=0x7f98080fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.513+0000 7f980d759640 1 -- 172.21.15.86:0/539295150 shutdown_connections 2024-04-16T05:36:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.513+0000 7f980d759640 1 -- 172.21.15.86:0/539295150 wait complete. 2024-04-16T05:36:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.513+0000 7f980d759640 1 Processor -- start 2024-04-16T05:36:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f980d759640 1 -- start start 2024-04-16T05:36:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f980d759640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9808100650 0x7f980819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f980d759640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9808101010 0x7f980819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f980d759640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 0x7f9808198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f980d759640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9808110150 con 0x7f9808106660 2024-04-16T05:36:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f980d759640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f980810ffd0 con 0x7f9808101010 2024-04-16T05:36:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f980d759640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98081102d0 con 0x7f9808100650 2024-04-16T05:36:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f98077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 0x7f9808198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f98077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 0x7f9808198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38134/0 (socket says 172.21.15.86:38134) 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f98077fe640 1 -- 172.21.15.86:0/134772061 learned_addr learned my addr 172.21.15.86:0/134772061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.514+0000 7f9806ffd640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9808100650 0x7f980819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f98077fe640 1 -- 172.21.15.86:0/134772061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9808100650 msgr2=0x7f980819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f98077fe640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9808100650 0x7f980819e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f98077fe640 1 -- 172.21.15.86:0/134772061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9808101010 msgr2=0x7f980819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f98077fe640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9808101010 0x7f980819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f98077fe640 1 -- 172.21.15.86:0/134772061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9808198f40 con 0x7f9808106660 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f98077fe640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 0x7f9808198680 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f980000eaa0 tx=0x7f980000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f9804ff9640 1 -- 172.21.15.86:0/134772061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980000cdb0 con 0x7f9808106660 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f980d759640 1 -- 172.21.15.86:0/134772061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98081991d0 con 0x7f9808106660 2024-04-16T05:36:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f9804ff9640 1 -- 172.21.15.86:0/134772061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9800005020 con 0x7f9808106660 2024-04-16T05:36:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f980d759640 1 -- 172.21.15.86:0/134772061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9808196560 con 0x7f9808106660 2024-04-16T05:36:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.515+0000 7f9804ff9640 1 -- 172.21.15.86:0/134772061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98000116b0 con 0x7f9808106660 2024-04-16T05:36:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.518+0000 7f9804ff9640 1 -- 172.21.15.86:0/134772061 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9800005560 con 0x7f9808106660 2024-04-16T05:36:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9804ff9640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97f40a3ff0 0x7f97f40a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9810426740 1 Processor -- start 2024-04-16T05:36:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9810426740 1 -- start start 2024-04-16T05:36:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9806ffd640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97f40a3ff0 0x7f97f40a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9804ff9640 1 -- 172.21.15.86:0/134772061 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f98000d0fa0 con 0x7f9808106660 2024-04-16T05:36:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9810426740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b41763ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9810426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 0x555b4176cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9810426740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9810426740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555b416c87d0 con 0x555b4176d110 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9810426740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555b41665f70 con 0x555b41763c10 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f9810426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b416c4790 con 0x555b41764530 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f98077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f98077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38144/0 (socket says 172.21.15.86:38144) 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.519+0000 7f98077fe640 1 -- 172.21.15.86:0/368131756 learned_addr learned my addr 172.21.15.86:0/368131756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f9806ffd640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97f40a3ff0 0x7f97f40a64b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005ab0 tx=0x7f97f8005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f97fe7fc640 1 --2- 172.21.15.86:0/368131756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 0x555b4176cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f9806ffd640 1 --2- 172.21.15.86:0/368131756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b41763ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f98077fe640 1 -- 172.21.15.86:0/368131756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 msgr2=0x555b4176cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f98077fe640 1 --2- 172.21.15.86:0/368131756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 0x555b4176cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f98077fe640 1 -- 172.21.15.86:0/368131756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 msgr2=0x555b41763ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f98077fe640 1 --2- 172.21.15.86:0/368131756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b41763ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f98077fe640 1 -- 172.21.15.86:0/368131756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b413db210 con 0x555b4176d110 2024-04-16T05:36:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f98077fe640 1 --2- 172.21.15.86:0/368131756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176f500 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f980009c920 tx=0x7f9800098d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aee1f1162bbaf4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f97fd7fa640 1 -- 172.21.15.86:0/368131756 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980001f070 con 0x555b4176d110 2024-04-16T05:36:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f97fd7fa640 1 -- 172.21.15.86:0/368131756 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98000a5600 con 0x555b4176d110 2024-04-16T05:36:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.520+0000 7f97fd7fa640 1 -- 172.21.15.86:0/368131756 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98000870b0 con 0x555b4176d110 2024-04-16T05:36:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.521+0000 7f9810426740 1 -- 172.21.15.86:0/368131756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 msgr2=0x555b4176f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.521+0000 7f9810426740 1 --2- 172.21.15.86:0/368131756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176f500 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f980009c920 tx=0x7f9800098d60 comp rx=0 tx=0).stop 2024-04-16T05:36:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.521+0000 7f9810426740 1 -- 172.21.15.86:0/368131756 shutdown_connections 2024-04-16T05:36:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.521+0000 7f9810426740 1 --2- 172.21.15.86:0/368131756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176f500 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.521+0000 7f9810426740 1 --2- 172.21.15.86:0/368131756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 0x555b4176cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.521+0000 7f9810426740 1 --2- 172.21.15.86:0/368131756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b41763ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.521+0000 7f9810426740 1 -- 172.21.15.86:0/368131756 >> 172.21.15.86:0/368131756 conn(0x555b4175e340 msgr2=0x555b4175e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 -- 172.21.15.86:0/368131756 shutdown_connections 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 -- 172.21.15.86:0/368131756 wait complete. 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 -- start start 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 Processor -- start 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b4176a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 0x555b4176a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555b4176fe40 con 0x555b4176d110 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555b41650e90 con 0x555b41763c10 2024-04-16T05:36:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b4176ffc0 con 0x555b41764530 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.522+0000 7f9810426740 10 client.0 _ll_register_callbacks cb 0x555b41684cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f9806ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b4176a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f9806ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b4176a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51010/0 (socket says 172.21.15.86:51010) 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f9806ffd640 1 -- 172.21.15.86:0/2023277006 learned_addr learned my addr 172.21.15.86:0/2023277006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f98077fe640 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f97fe7fc640 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 0x555b4176a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f9806ffd640 1 -- 172.21.15.86:0/2023277006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 msgr2=0x555b4176a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f9806ffd640 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 0x555b4176a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f9806ffd640 1 -- 172.21.15.86:0/2023277006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 msgr2=0x555b4176ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f9806ffd640 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f9806ffd640 1 -- 172.21.15.86:0/2023277006 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b414a7620 con 0x555b41763c10 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f98077fe640 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f9806ffd640 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b4176a190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97f8036000 tx=0x7f97f802cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c13fd37c05aeec8f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.523+0000 7f97ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.524+0000 7f97ee7fc640 1 -- 172.21.15.86:0/2023277006 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8004310 con 0x555b41763c10 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.524+0000 7f97ee7fc640 1 -- 172.21.15.86:0/2023277006 <== mon.2 v2:172.21.15.194:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f97f8043030 con 0x555b41763c10 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.524+0000 7f9810426740 1 -- 172.21.15.86:0/2023277006 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98000a5600 con 0x555b41763c10 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.524+0000 7f97ee7fc640 1 -- 172.21.15.86:0/2023277006 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97f80044b0 con 0x555b41763c10 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.524+0000 7f9810426740 10 client.25556 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.524+0000 7f9810426740 1 -- 172.21.15.86:0/2023277006 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b4177bf50 con 0x555b41763c10 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.524+0000 7f97ee7fc640 1 -- 172.21.15.86:0/2023277006 <== mon.2 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98000a5600 con 0x555b41763c10 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.525+0000 7f97ee7fc640 1 -- 172.21.15.86:0/2023277006 <== mon.2 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97f800b040 con 0x555b41763c10 2024-04-16T05:36:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.525+0000 7f9810426740 10 client.25556 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.525+0000 7f9810426740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.525+0000 7f9810426740 20 client.25556 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.525+0000 7f9810426740 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555b4177d410 0x555b4177f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.525+0000 7f9810426740 4 client.25556 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.525+0000 7f9810426740 1 -- 172.21.15.86:0/2023277006 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b41668380 con 0x555b4177d410 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.525+0000 7f97fe7fc640 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555b4177d410 0x555b4177f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.526+0000 7f97fe7fc640 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555b4177d410 0x555b4177f800 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40cf90eaf3801c server_cookie=116e48ea793166d1 in_seq=0 out_seq=0 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.526+0000 7f97ee7fc640 10 client.25556 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f97ee7fc640 1 -- 172.21.15.86:0/2023277006 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b41668380 con 0x555b4177d410 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f97ee7fc640 10 client.25556 handle_command_reply: tid=0 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 client.25556 shutdown 2024-04-16T05:36:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 -- 172.21.15.86:0/2023277006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 msgr2=0x555b4176a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b4176a190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97f8036000 tx=0x7f97f802cf50 comp rx=0 tx=0).stop 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 -- 172.21.15.86:0/2023277006 shutdown_connections 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555b4177d410 0x555b4177f800 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f97ee7fc640 0 client.25556 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b4176ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41764530 0x555b4176a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 --2- 172.21.15.86:0/2023277006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41763c10 0x555b4176a190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 -- 172.21.15.86:0/2023277006 >> 172.21.15.86:0/2023277006 conn(0x555b4175e340 msgr2=0x555b41760090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 -- 172.21.15.86:0/2023277006 shutdown_connections 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 1 -- 172.21.15.86:0/2023277006 wait complete. 2024-04-16T05:36:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.527+0000 7f9810426740 20 client.25556 trim_cache size 0 max 16384 2024-04-16T05:36:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.556+0000 7f9810426740 1 Processor -- start 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.556+0000 7f9810426740 1 -- start start 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f9810426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 0x555b4176c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f9810426740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b41764530 0x555b41778d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f9810426740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b4176d110 0x555b4175fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f9810426740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ad200 con 0x555b41764530 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f9810426740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f98081aed40 con 0x555b4176d110 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f9810426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ad8d0 con 0x555b41763c10 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f97fe7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b41764530 0x555b41778d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f97fe7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b41764530 0x555b41778d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38174/0 (socket says 172.21.15.86:38174) 2024-04-16T05:36:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f97fe7fc640 1 -- 172.21.15.86:0/3070810710 learned_addr learned my addr 172.21.15.86:0/3070810710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f9806ffd640 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 0x555b4176c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f98077fe640 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b4176d110 0x555b4175fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f98077fe640 1 -- 172.21.15.86:0/3070810710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 msgr2=0x555b4176c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f98077fe640 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 0x555b4176c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f98077fe640 1 -- 172.21.15.86:0/3070810710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b41764530 msgr2=0x555b41778d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f98077fe640 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b41764530 0x555b41778d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f98077fe640 1 -- 172.21.15.86:0/3070810710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b4177fb70 con 0x555b4176d110 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.557+0000 7f97fe7fc640 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b41764530 0x555b41778d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.558+0000 7f98077fe640 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b4176d110 0x555b4175fea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x555b4175f4d0 tx=0x7f980009c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19dfa094e9d24e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.558+0000 7f97fcff9640 1 -- 172.21.15.86:0/3070810710 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980001f070 con 0x555b4176d110 2024-04-16T05:36:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.558+0000 7f97fcff9640 1 -- 172.21.15.86:0/3070810710 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98000051d0 con 0x555b4176d110 2024-04-16T05:36:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.558+0000 7f97fcff9640 1 -- 172.21.15.86:0/3070810710 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98000bbd50 con 0x555b4176d110 2024-04-16T05:36:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 -- 172.21.15.86:0/3070810710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b4176d110 msgr2=0x555b4175fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b4176d110 0x555b4175fea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x555b4175f4d0 tx=0x7f980009c760 comp rx=0 tx=0).stop 2024-04-16T05:36:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 -- 172.21.15.86:0/3070810710 shutdown_connections 2024-04-16T05:36:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b4176d110 0x555b4175fea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b41764530 0x555b41778d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 --2- 172.21.15.86:0/3070810710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 0x555b4176c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 -- 172.21.15.86:0/3070810710 >> 172.21.15.86:0/3070810710 conn(0x555b4175e340 msgr2=0x555b4175f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 -- 172.21.15.86:0/3070810710 shutdown_connections 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 -- 172.21.15.86:0/3070810710 wait complete. 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 -- start start 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 Processor -- start 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 0x555b41775240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41764530 0x555b41775780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b41772210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.559+0000 7f9810426740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ae670 con 0x555b4176d110 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f9810426740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f98081adfa0 con 0x555b41764530 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f9810426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ab1c0 con 0x555b41763c10 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f9810426740 10 client.0 _ll_register_callbacks cb 0x7f97f8035570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97fe7fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41764530 0x555b41775780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97fe7fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41764530 0x555b41775780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51028/0 (socket says 172.21.15.86:51028) 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97fe7fc640 1 -- 172.21.15.86:0/1899806126 learned_addr learned my addr 172.21.15.86:0/1899806126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f9806ffd640 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 0x555b41775240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f98077fe640 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b41772210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97fe7fc640 1 -- 172.21.15.86:0/1899806126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 msgr2=0x555b41775240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97fe7fc640 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 0x555b41775240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97fe7fc640 1 -- 172.21.15.86:0/1899806126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 msgr2=0x555b41772210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97fe7fc640 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b41772210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97fe7fc640 1 -- 172.21.15.86:0/1899806126 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b417727d0 con 0x555b41764530 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f98077fe640 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b41772210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97fe7fc640 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41764530 0x555b41775780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97f0011670 tx=0x7f97f0011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3763b78a0e961 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97ee7fc640 1 -- 172.21.15.86:0/1899806126 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0004220 con 0x555b41764530 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f9810426740 1 -- 172.21.15.86:0/1899806126 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98000051d0 con 0x555b41764530 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.560+0000 7f97ee7fc640 1 -- 172.21.15.86:0/1899806126 <== mon.2 v2:172.21.15.194:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f97f00043c0 con 0x555b41764530 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f97ee7fc640 1 -- 172.21.15.86:0/1899806126 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97f002e050 con 0x555b41764530 2024-04-16T05:36:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f9810426740 10 client.25562 fetch_fsmap learned FSMap version 29 2024-04-16T05:36:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f9810426740 1 -- 172.21.15.86:0/1899806126 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b417e42c0 con 0x555b41764530 2024-04-16T05:36:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f97ee7fc640 1 -- 172.21.15.86:0/1899806126 <== mon.2 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98000051d0 con 0x555b41764530 2024-04-16T05:36:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f97ee7fc640 1 -- 172.21.15.86:0/1899806126 <== mon.2 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97f00050b0 con 0x555b41764530 2024-04-16T05:36:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f9810426740 10 client.25562 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:36:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f9810426740 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f9810426740 20 client.25562 populate_metadata read hostname 'smithi086' 2024-04-16T05:36:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f9810426740 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555b4177d410 0x555b417e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f9810426740 4 client.25562 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:36:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f9810426740 1 -- 172.21.15.86:0/1899806126 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b416d61f0 con 0x555b4177d410 2024-04-16T05:36:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.561+0000 7f9806ffd640 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555b4177d410 0x555b417e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.562+0000 7f9806ffd640 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555b4177d410 0x555b417e6a80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4f66528e53307c server_cookie=2ca26d1804c78205 in_seq=0 out_seq=0 2024-04-16T05:36:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.562+0000 7f97ee7fc640 10 client.25562 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f97ee7fc640 1 -- 172.21.15.86:0/1899806126 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b416d61f0 con 0x555b4177d410 2024-04-16T05:36:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f97ee7fc640 10 client.25562 handle_command_reply: tid=0 2024-04-16T05:36:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 client.25562 shutdown 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 -- 172.21.15.86:0/1899806126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41764530 msgr2=0x555b41775780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41764530 0x555b41775780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97f0011670 tx=0x7f97f0011b30 comp rx=0 tx=0).stop 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 -- 172.21.15.86:0/1899806126 shutdown_connections 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555b4177d410 0x555b417e6a80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4176d110 0x555b41772210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b41764530 0x555b41775780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 --2- 172.21.15.86:0/1899806126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b41763c10 0x555b41775240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 -- 172.21.15.86:0/1899806126 >> 172.21.15.86:0/1899806126 conn(0x555b4175e340 msgr2=0x555b4176a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f97ee7fc640 0 client.25562 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 -- 172.21.15.86:0/1899806126 shutdown_connections 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 1 -- 172.21.15.86:0/1899806126 wait complete. 2024-04-16T05:36:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.566+0000 7f9810426740 20 client.25562 trim_cache size 0 max 16384 2024-04-16T05:36:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-16T05:36:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-16T05:36:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-16T05:36:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-16T05:36:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 -- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97f40a3ff0 msgr2=0x7f97f40a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97f40a3ff0 0x7f97f40a64b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005ab0 tx=0x7f97f8005fd0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 -- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 msgr2=0x7f9808198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 0x7f9808198680 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f980000eaa0 tx=0x7f980000ef60 comp rx=0 tx=0).stop 2024-04-16T05:36:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 -- 172.21.15.86:0/134772061 shutdown_connections 2024-04-16T05:36:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808106660 0x7f9808198680 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97f40a3ff0 0x7f97f40a64b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9808101010 0x7f980819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 --2- 172.21.15.86:0/134772061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9808100650 0x7f980819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 -- 172.21.15.86:0/134772061 >> 172.21.15.86:0/134772061 conn(0x7f9808077490 msgr2=0x7f9808103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 -- 172.21.15.86:0/134772061 shutdown_connections 2024-04-16T05:36:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-16T05:36:45.567+0000 7f980d759640 1 -- 172.21.15.86:0/134772061 wait complete. 2024-04-16T05:36:46.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:46 smithi086 ceph-mon[32028]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:36:46.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:46 smithi196 ceph-mon[36373]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:36:46.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:46 smithi194 ceph-mon[35865]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:36:48.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:48 smithi086 ceph-mon[32028]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:36:48.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:48 smithi196 ceph-mon[36373]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:36:48.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:48 smithi194 ceph-mon[35865]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:36:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:50 smithi086 ceph-mon[32028]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-16T05:36:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:50 smithi086 ceph-mon[32028]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T05:36:50.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:50 smithi196 ceph-mon[36373]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-16T05:36:50.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:50 smithi196 ceph-mon[36373]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T05:36:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:50 smithi194 ceph-mon[35865]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-16T05:36:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:50 smithi194 ceph-mon[35865]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr:Note: switching to 'a87fcff6340baeedd43af126e793f3241b6400ea'. 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr:state without impacting any branches by switching back to a branch. 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr: git switch -c 2024-04-16T05:36:51.748 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-16T05:36:51.749 INFO:tasks.workunit.client.0.smithi086.stderr:Or undo this operation with: 2024-04-16T05:36:51.749 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-16T05:36:51.749 INFO:tasks.workunit.client.0.smithi086.stderr: git switch - 2024-04-16T05:36:51.749 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-16T05:36:51.749 INFO:tasks.workunit.client.0.smithi086.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-16T05:36:51.749 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-16T05:36:51.749 INFO:tasks.workunit.client.0.smithi086.stderr:HEAD is now at a87fcff6340 qa: enable dynamic debug for dbench workunit 2024-04-16T05:36:51.754 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-16T05:36:51.818 INFO:tasks.workunit.client.0.smithi086.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-16T05:36:51.821 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T05:36:51.821 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-16T05:36:52.527 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-16T05:36:52.594 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-16T05:36:52.634 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T05:36:52.636 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T05:36:52.636 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-16T05:36:52.676 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T05:36:52.679 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:36:52.679 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-16T05:36:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:52 smithi086 ceph-mon[32028]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T05:36:52.733 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-16T05:36:52.734 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-16T05:36:52.735 DEBUG:teuthology.orchestra.run.smithi086:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a87fcff6340baeedd43af126e793f3241b6400ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-16T05:36:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:52 smithi196 ceph-mon[36373]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T05:36:52.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:36:52.796 INFO:tasks.workunit.client.0.smithi086.stderr:+ echo -n 'module ceph +p' 2024-04-16T05:36:52.796 INFO:tasks.workunit.client.0.smithi086.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh: line 4: /sys/kernel/debug/dynamic_debug/control: Permission denied 2024-04-16T05:36:52.796 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-16T05:36:52.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-16T05:36:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:52 smithi194 ceph-mon[35865]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T05:36:53.082 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi086 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a87fcff6340baeedd43af126e793f3241b6400ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-16T05:36:53.315 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c8dc2f6780524070b550d5c4b0c989e9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi086 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a87fcff6340baeedd43af126e793f3241b6400ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-16T05:36:53.317 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-16T05:36:53.328 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:36:53.341 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-16T05:36:53.370 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-16T05:36:53.370 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T05:36:53.370 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:36:53.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-16T05:36:53.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-16T05:36:54.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:54 smithi086 ceph-mon[32028]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-16T05:36:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:54 smithi196 ceph-mon[36373]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-16T05:36:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:54 smithi194 ceph-mon[35865]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-16T05:36:56.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:56 smithi086 ceph-mon[32028]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T05:36:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:56 smithi196 ceph-mon[36373]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T05:36:56.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:56 smithi194 ceph-mon[35865]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T05:36:58.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:36:58 smithi086 ceph-mon[32028]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T05:36:58.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:36:58 smithi196 ceph-mon[36373]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T05:36:58.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:36:58 smithi194 ceph-mon[35865]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T05:37:00.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:00 smithi086 ceph-mon[32028]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-04-16T05:37:00.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:00 smithi196 ceph-mon[36373]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-04-16T05:37:00.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:37:00.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:37:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.856+0000 7f79770b1640 1 -- 172.21.15.86:0/1360563166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 msgr2=0x7f7970101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.856+0000 7f79770b1640 1 --2- 172.21.15.86:0/1360563166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 0x7f7970101520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7964009900 tx=0x7f796402bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.856+0000 7f79770b1640 1 -- 172.21.15.86:0/1360563166 shutdown_connections 2024-04-16T05:37:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.856+0000 7f79770b1640 1 --2- 172.21.15.86:0/1360563166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970106210 0x7f797010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.856+0000 7f79770b1640 1 --2- 172.21.15.86:0/1360563166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7970101af0 0x7f7970105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.856+0000 7f79770b1640 1 --2- 172.21.15.86:0/1360563166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 0x7f7970101520 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.856+0000 7f79770b1640 1 -- 172.21.15.86:0/1360563166 >> 172.21.15.86:0/1360563166 conn(0x7f7970078000 msgr2=0x7f7970078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:00.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.856+0000 7f79770b1640 1 -- 172.21.15.86:0/1360563166 shutdown_connections 2024-04-16T05:37:00.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.856+0000 7f79770b1640 1 -- 172.21.15.86:0/1360563166 wait complete. 2024-04-16T05:37:00.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f79770b1640 1 Processor -- start 2024-04-16T05:37:00.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f79770b1640 1 -- start start 2024-04-16T05:37:00.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f79770b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 0x7f797019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f79770b1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7970106210 0x7f797019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f79770b1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970198770 0x7f7970198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f79770b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79701112c0 con 0x7f7970101140 2024-04-16T05:37:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f79770b1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7970111140 con 0x7f7970198770 2024-04-16T05:37:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f79770b1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7970111440 con 0x7f7970106210 2024-04-16T05:37:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f7974e4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 0x7f797019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f7974e4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 0x7f797019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50836/0 (socket says 172.21.15.86:50836) 2024-04-16T05:37:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f7974e4d640 1 -- 172.21.15.86:0/1838399023 learned_addr learned my addr 172.21.15.86:0/1838399023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:00.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f797564e640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970198770 0x7f7970198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f7974e4d640 1 -- 172.21.15.86:0/1838399023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7970106210 msgr2=0x7f797019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:00.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f7974e4d640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7970106210 0x7f797019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f7974e4d640 1 -- 172.21.15.86:0/1838399023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970198770 msgr2=0x7f7970198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:00.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f7974e4d640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970198770 0x7f7970198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f7974e4d640 1 -- 172.21.15.86:0/1838399023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7970199370 con 0x7f7970101140 2024-04-16T05:37:00.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.857+0000 7f7974e4d640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 0x7f797019e660 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7964006010 tx=0x7f796402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:00.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.858+0000 7f796dffb640 1 -- 172.21.15.86:0/1838399023 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964041070 con 0x7f7970101140 2024-04-16T05:37:00.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.858+0000 7f796dffb640 1 -- 172.21.15.86:0/1838399023 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7964004450 con 0x7f7970101140 2024-04-16T05:37:00.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.858+0000 7f79770b1640 1 -- 172.21.15.86:0/1838399023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7970199600 con 0x7f7970101140 2024-04-16T05:37:00.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.858+0000 7f796dffb640 1 -- 172.21.15.86:0/1838399023 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964034440 con 0x7f7970101140 2024-04-16T05:37:00.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.858+0000 7f79770b1640 1 -- 172.21.15.86:0/1838399023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79701a56f0 con 0x7f7970101140 2024-04-16T05:37:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.860+0000 7f796dffb640 1 -- 172.21.15.86:0/1838399023 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7964044070 con 0x7f7970101140 2024-04-16T05:37:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.860+0000 7f7979d72740 1 Processor -- start 2024-04-16T05:37:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.860+0000 7f7979d72740 1 -- start start 2024-04-16T05:37:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f7979d72740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6bbe50 0x55df4b6bc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f7979d72740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f7979d72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6c5350 0x55df4b6c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f7979d72740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df4b621180 con 0x55df4b6bbe50 2024-04-16T05:37:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f7979d72740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55df4b5be360 con 0x55df4b6bc770 2024-04-16T05:37:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f7979d72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df4b61d140 con 0x55df4b6c5350 2024-04-16T05:37:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6bbe50 0x55df4b6bc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6bbe50 0x55df4b6bc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50850/0 (socket says 172.21.15.86:50850) 2024-04-16T05:37:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 -- 172.21.15.86:0/3424379485 learned_addr learned my addr 172.21.15.86:0/3424379485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f796dffb640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79600a3d20 0x7f79600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f7974e4d640 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79600a3d20 0x7f79600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f796ffff640 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6c5350 0x55df4b6c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f796dffb640 1 -- 172.21.15.86:0/1838399023 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f79640f26b0 con 0x7f7970101140 2024-04-16T05:37:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 -- 172.21.15.86:0/3424379485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6c5350 msgr2=0x55df4b6c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6c5350 0x55df4b6c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 -- 172.21.15.86:0/3424379485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 msgr2=0x55df4b6c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 -- 172.21.15.86:0/3424379485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4b3fd7d0 con 0x55df4b6bbe50 2024-04-16T05:37:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f796ffff640 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6c5350 0x55df4b6c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f7974e4d640 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.861+0000 7f797564e640 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6bbe50 0x55df4b6bc230 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7968014a60 tx=0x7f7968014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e68539e4cf2fa38 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f795affd640 1 -- 172.21.15.86:0/3424379485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796801ec50 con 0x55df4b6bbe50 2024-04-16T05:37:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f795affd640 1 -- 172.21.15.86:0/3424379485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7968023070 con 0x55df4b6bbe50 2024-04-16T05:37:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f795affd640 1 -- 172.21.15.86:0/3424379485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796801de00 con 0x55df4b6bbe50 2024-04-16T05:37:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f7979d72740 1 -- 172.21.15.86:0/3424379485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6bbe50 msgr2=0x55df4b6bc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f7979d72740 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6bbe50 0x55df4b6bc230 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7968014a60 tx=0x7f7968014f20 comp rx=0 tx=0).stop 2024-04-16T05:37:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f797564e640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79600a3d20 0x7f79600a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7968022000 tx=0x7f796801e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f7979d72740 1 -- 172.21.15.86:0/3424379485 shutdown_connections 2024-04-16T05:37:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f7979d72740 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6c5350 0x55df4b6c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f7979d72740 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f7979d72740 1 --2- 172.21.15.86:0/3424379485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6bbe50 0x55df4b6bc230 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f7979d72740 1 -- 172.21.15.86:0/3424379485 >> 172.21.15.86:0/3424379485 conn(0x55df4b6b6580 msgr2=0x55df4b6b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f7979d72740 1 -- 172.21.15.86:0/3424379485 shutdown_connections 2024-04-16T05:37:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.862+0000 7f7979d72740 1 -- 172.21.15.86:0/3424379485 wait complete. 2024-04-16T05:37:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f7979d72740 1 -- start start 2024-04-16T05:37:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f7979d72740 1 Processor -- start 2024-04-16T05:37:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f7979d72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bbe50 0x55df4b6c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f7979d72740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f7979d72740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f7979d72740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df4b6c8140 con 0x55df4b6c5350 2024-04-16T05:37:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f7979d72740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55df4b5da2c0 con 0x55df4b6bc770 2024-04-16T05:37:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f7979d72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df4b6c82c0 con 0x55df4b6bbe50 2024-04-16T05:37:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f7979d72740 10 client.0 _ll_register_callbacks cb 0x55df4b5c1090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f797564e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f797564e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6c2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50854/0 (socket says 172.21.15.86:50854) 2024-04-16T05:37:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.863+0000 7f797564e640 1 -- 172.21.15.86:0/3848826923 learned_addr learned my addr 172.21.15.86:0/3848826923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f797564e640 1 -- 172.21.15.86:0/3848826923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bbe50 msgr2=0x55df4b6c2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f797564e640 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bbe50 0x55df4b6c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f797564e640 1 -- 172.21.15.86:0/3848826923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 msgr2=0x55df4b6c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f797564e640 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f797564e640 1 -- 172.21.15.86:0/3848826923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4b3333f0 con 0x55df4b6c5350 2024-04-16T05:37:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f797564e640 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6c2e90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7968015010 tx=0x7f79680144f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a2362de0a10478 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f794ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f794ffff640 1 -- 172.21.15.86:0/3848826923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968037060 con 0x55df4b6c5350 2024-04-16T05:37:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f7979d72740 1 -- 172.21.15.86:0/3848826923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7968023070 con 0x55df4b6c5350 2024-04-16T05:37:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f794ffff640 1 -- 172.21.15.86:0/3848826923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7968014590 con 0x55df4b6c5350 2024-04-16T05:37:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f794ffff640 1 -- 172.21.15.86:0/3848826923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f796803e050 con 0x55df4b6c5350 2024-04-16T05:37:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f7979d72740 10 client.16218 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f7979d72740 1 -- 172.21.15.86:0/3848826923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df4b2f30f0 con 0x55df4b6c5350 2024-04-16T05:37:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f794ffff640 1 -- 172.21.15.86:0/3848826923 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7968023070 con 0x55df4b6c5350 2024-04-16T05:37:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f794ffff640 1 -- 172.21.15.86:0/3848826923 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f796800f040 con 0x55df4b6c5350 2024-04-16T05:37:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f7979d72740 10 client.16218 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f7979d72740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f7979d72740 20 client.16218 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.864+0000 7f7979d72740 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55df4b6d5620 0x55df4b6d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.865+0000 7f7979d72740 4 client.16218 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.865+0000 7f7979d72740 1 -- 172.21.15.86:0/3848826923 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df4b62eba0 con 0x55df4b6d5620 2024-04-16T05:37:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.865+0000 7f7974e4d640 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55df4b6d5620 0x55df4b6d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.865+0000 7f7974e4d640 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55df4b6d5620 0x55df4b6d7a10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0e28b41eec27e3e server_cookie=b594247c4b8108ef in_seq=0 out_seq=0 2024-04-16T05:37:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.865+0000 7f794ffff640 10 client.16218 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f794ffff640 1 -- 172.21.15.86:0/3848826923 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df4b62eba0 con 0x55df4b6d5620 2024-04-16T05:37:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f794ffff640 10 client.16218 handle_command_reply: tid=0 2024-04-16T05:37:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f7979d72740 1 client.16218 shutdown 2024-04-16T05:37:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f7979d72740 1 -- 172.21.15.86:0/3848826923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 msgr2=0x55df4b6c2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f7979d72740 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6c2e90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7968015010 tx=0x7f79680144f0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f7979d72740 1 -- 172.21.15.86:0/3848826923 shutdown_connections 2024-04-16T05:37:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f7979d72740 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55df4b6d5620 0x55df4b6d7a10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f7979d72740 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6c2e90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f7979d72740 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f794ffff640 0 client.16218 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f7979d72740 1 --2- 172.21.15.86:0/3848826923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bbe50 0x55df4b6c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.866+0000 7f7979d72740 1 -- 172.21.15.86:0/3848826923 >> 172.21.15.86:0/3848826923 conn(0x55df4b6b6580 msgr2=0x55df4b6b82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.867+0000 7f7979d72740 1 -- 172.21.15.86:0/3848826923 shutdown_connections 2024-04-16T05:37:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.867+0000 7f7979d72740 1 -- 172.21.15.86:0/3848826923 wait complete. 2024-04-16T05:37:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.867+0000 7f7979d72740 20 client.16218 trim_cache size 0 max 16384 2024-04-16T05:37:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f7979d72740 1 Processor -- start 2024-04-16T05:37:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f7979d72740 1 -- start start 2024-04-16T05:37:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f7979d72740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bbe50 0x55df4b6c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f7979d72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bc770 0x55df4b6d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f7979d72740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6dd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f7979d72740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79701af4a0 con 0x55df4b6c5350 2024-04-16T05:37:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f7979d72740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f79701b0fe0 con 0x55df4b6bbe50 2024-04-16T05:37:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f7979d72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79701afb70 con 0x55df4b6bc770 2024-04-16T05:37:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f797564e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6dd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f797564e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6dd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50858/0 (socket says 172.21.15.86:50858) 2024-04-16T05:37:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f797564e640 1 -- 172.21.15.86:0/389748463 learned_addr learned my addr 172.21.15.86:0/389748463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f797564e640 1 -- 172.21.15.86:0/389748463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bc770 msgr2=0x55df4b6d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f797564e640 1 --2- 172.21.15.86:0/389748463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bc770 0x55df4b6d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f797564e640 1 -- 172.21.15.86:0/389748463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bbe50 msgr2=0x55df4b6c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f797564e640 1 --2- 172.21.15.86:0/389748463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bbe50 0x55df4b6c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.884+0000 7f797564e640 1 -- 172.21.15.86:0/389748463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4b6d7d80 con 0x55df4b6c5350 2024-04-16T05:37:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f797564e640 1 --2- 172.21.15.86:0/389748463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6dd2b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f796801dc60 tx=0x7f7968009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b4f89fab8b1dca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f795a7fc640 1 -- 172.21.15.86:0/389748463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968037040 con 0x55df4b6c5350 2024-04-16T05:37:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f795a7fc640 1 -- 172.21.15.86:0/389748463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7968023030 con 0x55df4b6c5350 2024-04-16T05:37:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f795a7fc640 1 -- 172.21.15.86:0/389748463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968012c10 con 0x55df4b6c5350 2024-04-16T05:37:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f7979d72740 1 -- 172.21.15.86:0/389748463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 msgr2=0x55df4b6dd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f7979d72740 1 --2- 172.21.15.86:0/389748463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6dd2b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f796801dc60 tx=0x7f7968009b70 comp rx=0 tx=0).stop 2024-04-16T05:37:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f7979d72740 1 -- 172.21.15.86:0/389748463 shutdown_connections 2024-04-16T05:37:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f7979d72740 1 --2- 172.21.15.86:0/389748463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b6dd2b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f7979d72740 1 --2- 172.21.15.86:0/389748463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bc770 0x55df4b6d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f7979d72740 1 --2- 172.21.15.86:0/389748463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bbe50 0x55df4b6c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f7979d72740 1 -- 172.21.15.86:0/389748463 >> 172.21.15.86:0/389748463 conn(0x55df4b6b6580 msgr2=0x55df4b6bcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f7979d72740 1 -- 172.21.15.86:0/389748463 shutdown_connections 2024-04-16T05:37:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.885+0000 7f7979d72740 1 -- 172.21.15.86:0/389748463 wait complete. 2024-04-16T05:37:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7979d72740 1 -- start start 2024-04-16T05:37:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7979d72740 1 Processor -- start 2024-04-16T05:37:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7979d72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bbe50 0x55df4b6ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7979d72740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6cad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7979d72740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b733bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7979d72740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79701b0910 con 0x55df4b6c5350 2024-04-16T05:37:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7979d72740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f79701b0240 con 0x55df4b6bc770 2024-04-16T05:37:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7979d72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79701ad460 con 0x55df4b6bbe50 2024-04-16T05:37:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7979d72740 10 client.0 _ll_register_callbacks cb 0x7f7968014860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f797564e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b733bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f797564e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b733bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50866/0 (socket says 172.21.15.86:50866) 2024-04-16T05:37:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f797564e640 1 -- 172.21.15.86:0/3491924711 learned_addr learned my addr 172.21.15.86:0/3491924711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.886+0000 7f7974e4d640 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bbe50 0x55df4b6ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f796ffff640 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f797564e640 1 -- 172.21.15.86:0/3491924711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bbe50 msgr2=0x55df4b6ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f797564e640 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bbe50 0x55df4b6ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f797564e640 1 -- 172.21.15.86:0/3491924711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 msgr2=0x55df4b6cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f797564e640 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f797564e640 1 -- 172.21.15.86:0/3491924711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4b734210 con 0x55df4b6c5350 2024-04-16T05:37:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f796ffff640 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f797564e640 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b733bd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7968005f40 tx=0x7f7968012ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e46d9b8c2211dc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f794ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f794ffff640 1 -- 172.21.15.86:0/3491924711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968046070 con 0x55df4b6c5350 2024-04-16T05:37:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f794ffff640 1 -- 172.21.15.86:0/3491924711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7968004bb0 con 0x55df4b6c5350 2024-04-16T05:37:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f7979d72740 1 -- 172.21.15.86:0/3491924711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7968023030 con 0x55df4b6c5350 2024-04-16T05:37:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f794ffff640 1 -- 172.21.15.86:0/3491924711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7968052050 con 0x55df4b6c5350 2024-04-16T05:37:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f7979d72740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f7979d72740 1 -- 172.21.15.86:0/3491924711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df4b73b590 con 0x55df4b6c5350 2024-04-16T05:37:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f794ffff640 1 -- 172.21.15.86:0/3491924711 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7968023030 con 0x55df4b6c5350 2024-04-16T05:37:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f794ffff640 1 -- 172.21.15.86:0/3491924711 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f796800e780 con 0x55df4b6c5350 2024-04-16T05:37:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f7979d72740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f7979d72740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f7979d72740 20 client.16224 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f7979d72740 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55df4b73bc40 0x55df4b73e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f7979d72740 4 client.16224 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:37:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.887+0000 7f7979d72740 1 -- 172.21.15.86:0/3491924711 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df4b628a10 con 0x55df4b73bc40 2024-04-16T05:37:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.888+0000 7f7974e4d640 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55df4b73bc40 0x55df4b73e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.888+0000 7f794ffff640 10 client.16224 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.888+0000 7f7974e4d640 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55df4b73bc40 0x55df4b73e030 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=947edeead8c114cb server_cookie=84287148e5ab2b8f in_seq=0 out_seq=0 2024-04-16T05:37:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.888+0000 7f794ffff640 1 -- 172.21.15.86:0/3491924711 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df4b628a10 con 0x55df4b73bc40 2024-04-16T05:37:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.888+0000 7f794ffff640 10 client.16224 handle_command_reply: tid=0 2024-04-16T05:37:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.888+0000 7f7979d72740 1 client.16224 shutdown 2024-04-16T05:37:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.888+0000 7f7979d72740 1 -- 172.21.15.86:0/3491924711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 msgr2=0x55df4b733bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.888+0000 7f7979d72740 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b733bd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7968005f40 tx=0x7f7968012ec0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f7979d72740 1 -- 172.21.15.86:0/3491924711 shutdown_connections 2024-04-16T05:37:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f7979d72740 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55df4b73bc40 0x55df4b73e030 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f7979d72740 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df4b6c5350 0x55df4b733bd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f7979d72740 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df4b6bc770 0x55df4b6cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f794ffff640 0 client.16224 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f7979d72740 1 --2- 172.21.15.86:0/3491924711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df4b6bbe50 0x55df4b6ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f7979d72740 1 -- 172.21.15.86:0/3491924711 >> 172.21.15.86:0/3491924711 conn(0x55df4b6b6580 msgr2=0x55df4b6b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f7979d72740 1 -- 172.21.15.86:0/3491924711 shutdown_connections 2024-04-16T05:37:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f7979d72740 1 -- 172.21.15.86:0/3491924711 wait complete. 2024-04-16T05:37:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f7979d72740 20 client.16224 trim_cache size 0 max 16384 2024-04-16T05:37:00.898 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:37:00.899 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:37:00.899 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:37:00.899 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:37:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 -- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79600a3d20 msgr2=0x7f79600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79600a3d20 0x7f79600a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7968022000 tx=0x7f796801e000 comp rx=0 tx=0).stop 2024-04-16T05:37:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 -- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 msgr2=0x7f797019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 0x7f797019e660 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7964006010 tx=0x7f796402cc70 comp rx=0 tx=0).stop 2024-04-16T05:37:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 -- 172.21.15.86:0/1838399023 shutdown_connections 2024-04-16T05:37:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79600a3d20 0x7f79600a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970198770 0x7f7970198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7970106210 0x7f797019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 --2- 172.21.15.86:0/1838399023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7970101140 0x7f797019e660 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 -- 172.21.15.86:0/1838399023 >> 172.21.15.86:0/1838399023 conn(0x7f7970078000 msgr2=0x7f7970104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 -- 172.21.15.86:0/1838399023 shutdown_connections 2024-04-16T05:37:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:00.889+0000 7f79770b1640 1 -- 172.21.15.86:0/1838399023 wait complete. 2024-04-16T05:37:00.902 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:37:00.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:37:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:00 smithi194 ceph-mon[35865]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-04-16T05:37:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.017+0000 7feda1c1b640 1 -- 172.21.15.86:0/1412714455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 msgr2=0x7fed9c101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.017+0000 7feda1c1b640 1 --2- 172.21.15.86:0/1412714455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 0x7fed9c101530 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009900 tx=0x7fed8c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.017+0000 7feda1c1b640 1 -- 172.21.15.86:0/1412714455 shutdown_connections 2024-04-16T05:37:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.017+0000 7feda1c1b640 1 --2- 172.21.15.86:0/1412714455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed9c106220 0x7fed9c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.017+0000 7feda1c1b640 1 --2- 172.21.15.86:0/1412714455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed9c101b00 0x7fed9c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.017+0000 7feda1c1b640 1 --2- 172.21.15.86:0/1412714455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 0x7fed9c101530 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.017+0000 7feda1c1b640 1 -- 172.21.15.86:0/1412714455 >> 172.21.15.86:0/1412714455 conn(0x7fed9c078000 msgr2=0x7fed9c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.017+0000 7feda1c1b640 1 -- 172.21.15.86:0/1412714455 shutdown_connections 2024-04-16T05:37:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.017+0000 7feda1c1b640 1 -- 172.21.15.86:0/1412714455 wait complete. 2024-04-16T05:37:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.018+0000 7feda1c1b640 1 Processor -- start 2024-04-16T05:37:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.018+0000 7feda1c1b640 1 -- start start 2024-04-16T05:37:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.018+0000 7feda1c1b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 0x7fed9c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.018+0000 7feda1c1b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed9c106220 0x7fed9c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.018+0000 7feda1c1b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed9c19a200 0x7fed9c1a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.018+0000 7feda1c1b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1112b0 con 0x7fed9c101150 2024-04-16T05:37:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.018+0000 7feda1c1b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c111130 con 0x7fed9c106220 2024-04-16T05:37:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.018+0000 7feda1c1b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c111430 con 0x7fed9c19a200 2024-04-16T05:37:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 0x7fed9c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 0x7fed9c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50882/0 (socket says 172.21.15.86:50882) 2024-04-16T05:37:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9b7fe640 1 -- 172.21.15.86:0/259649782 learned_addr learned my addr 172.21.15.86:0/259649782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9bfff640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed9c19a200 0x7fed9c1a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9affd640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed9c106220 0x7fed9c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9b7fe640 1 -- 172.21.15.86:0/259649782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed9c19a200 msgr2=0x7fed9c1a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9b7fe640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed9c19a200 0x7fed9c1a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9b7fe640 1 -- 172.21.15.86:0/259649782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed9c106220 msgr2=0x7fed9c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9b7fe640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed9c106220 0x7fed9c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9b7fe640 1 -- 172.21.15.86:0/259649782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c1a4f80 con 0x7fed9c101150 2024-04-16T05:37:01.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed9b7fe640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 0x7fed9c199730 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed8c006010 tx=0x7fed8c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed98ff9640 1 -- 172.21.15.86:0/259649782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c041070 con 0x7fed9c101150 2024-04-16T05:37:01.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c1a5210 con 0x7fed9c101150 2024-04-16T05:37:01.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed98ff9640 1 -- 172.21.15.86:0/259649782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed8c004450 con 0x7fed9c101150 2024-04-16T05:37:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7fed98ff9640 1 -- 172.21.15.86:0/259649782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c003e00 con 0x7fed9c101150 2024-04-16T05:37:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.019+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c1a5750 con 0x7fed9c101150 2024-04-16T05:37:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.021+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9c068e20 con 0x7fed9c101150 2024-04-16T05:37:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.022+0000 7fed98ff9640 1 -- 172.21.15.86:0/259649782 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed8c044070 con 0x7fed9c101150 2024-04-16T05:37:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.023+0000 7fed98ff9640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fed840a3d20 0x7fed840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.023+0000 7fed9affd640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fed840a3d20 0x7fed840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.023+0000 7fed98ff9640 1 -- 172.21.15.86:0/259649782 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fed8c0f2470 con 0x7fed9c101150 2024-04-16T05:37:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.024+0000 7fed9affd640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fed840a3d20 0x7fed840a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed9c069ce0 tx=0x7fed90005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.028+0000 7fed98ff9640 1 -- 172.21.15.86:0/259649782 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed8c0be300 con 0x7fed9c101150 2024-04-16T05:37:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.246+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed9c109450 con 0x7fed9c101150 2024-04-16T05:37:01.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.247+0000 7fed98ff9640 1 -- 172.21.15.86:0/259649782 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fed8c0c2600 con 0x7fed9c101150 2024-04-16T05:37:01.246 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:37:01.247 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:37:01.247 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:37:01.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fed840a3d20 msgr2=0x7fed840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fed840a3d20 0x7fed840a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed9c069ce0 tx=0x7fed90005f90 comp rx=0 tx=0).stop 2024-04-16T05:37:01.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 msgr2=0x7fed9c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 0x7fed9c199730 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed8c006010 tx=0x7fed8c02cb30 comp rx=0 tx=0).stop 2024-04-16T05:37:01.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 shutdown_connections 2024-04-16T05:37:01.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed9c19a200 0x7fed9c1a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fed840a3d20 0x7fed840a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed9c106220 0x7fed9c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 --2- 172.21.15.86:0/259649782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c101150 0x7fed9c199730 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.249+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 >> 172.21.15.86:0/259649782 conn(0x7fed9c078000 msgr2=0x7fed9c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.250+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 shutdown_connections 2024-04-16T05:37:01.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.250+0000 7feda1c1b640 1 -- 172.21.15.86:0/259649782 wait complete. 2024-04-16T05:37:01.262 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:37:01.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.368+0000 7fc6bc596640 1 -- 172.21.15.86:0/2156823047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b4106200 msgr2=0x7fc6b410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.368+0000 7fc6bc596640 1 --2- 172.21.15.86:0/2156823047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b4106200 0x7fc6b410ddd0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009ca0 tx=0x7fc6ac02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.368+0000 7fc6bc596640 1 -- 172.21.15.86:0/2156823047 shutdown_connections 2024-04-16T05:37:01.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.368+0000 7fc6bc596640 1 --2- 172.21.15.86:0/2156823047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b4106200 0x7fc6b410ddd0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.368+0000 7fc6bc596640 1 --2- 172.21.15.86:0/2156823047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b4101ae0 0x7fc6b4105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.368+0000 7fc6bc596640 1 --2- 172.21.15.86:0/2156823047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6b4101130 0x7fc6b4101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.368+0000 7fc6bc596640 1 -- 172.21.15.86:0/2156823047 >> 172.21.15.86:0/2156823047 conn(0x7fc6b40780b0 msgr2=0x7fc6b40784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.368+0000 7fc6bc596640 1 -- 172.21.15.86:0/2156823047 shutdown_connections 2024-04-16T05:37:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.368+0000 7fc6bc596640 1 -- 172.21.15.86:0/2156823047 wait complete. 2024-04-16T05:37:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bc596640 1 Processor -- start 2024-04-16T05:37:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bc596640 1 -- start start 2024-04-16T05:37:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bc596640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b4101130 0x7fc6b419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bc596640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6b4106200 0x7fc6b419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bc596640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b41985e0 0x7fc6b4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bc596640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41123c0 con 0x7fc6b41985e0 2024-04-16T05:37:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bc596640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4112240 con 0x7fc6b4106200 2024-04-16T05:37:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bc596640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4112540 con 0x7fc6b4101130 2024-04-16T05:37:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bab33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b41985e0 0x7fc6b4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bab33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b41985e0 0x7fc6b4198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50904/0 (socket says 172.21.15.86:50904) 2024-04-16T05:37:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bab33640 1 -- 172.21.15.86:0/1792929054 learned_addr learned my addr 172.21.15.86:0/1792929054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6ba332640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b4101130 0x7fc6b419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bab33640 1 -- 172.21.15.86:0/1792929054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b4101130 msgr2=0x7fc6b419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bab33640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b4101130 0x7fc6b419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bab33640 1 -- 172.21.15.86:0/1792929054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6b4106200 msgr2=0x7fc6b419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bab33640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6b4106200 0x7fc6b419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.369+0000 7fc6bab33640 1 -- 172.21.15.86:0/1792929054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b4199100 con 0x7fc6b41985e0 2024-04-16T05:37:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.370+0000 7fc6bab33640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b41985e0 0x7fc6b4198a90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac02c700 tx=0x7fc6ac035b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.370+0000 7fc6ab7fe640 1 -- 172.21.15.86:0/1792929054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac039040 con 0x7fc6b41985e0 2024-04-16T05:37:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.370+0000 7fc6ab7fe640 1 -- 172.21.15.86:0/1792929054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac044070 con 0x7fc6b41985e0 2024-04-16T05:37:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.370+0000 7fc6bc596640 1 -- 172.21.15.86:0/1792929054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b4199390 con 0x7fc6b41985e0 2024-04-16T05:37:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.370+0000 7fc6ab7fe640 1 -- 172.21.15.86:0/1792929054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0043a0 con 0x7fc6b41985e0 2024-04-16T05:37:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.370+0000 7fc6bc596640 1 -- 172.21.15.86:0/1792929054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b41a5580 con 0x7fc6b41985e0 2024-04-16T05:37:01.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.372+0000 7fc6ab7fe640 1 -- 172.21.15.86:0/1792929054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6ac004540 con 0x7fc6b41985e0 2024-04-16T05:37:01.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bf177740 1 Processor -- start 2024-04-16T05:37:01.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bf177740 1 -- start start 2024-04-16T05:37:01.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6ab7fe640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6a00a3d20 0x7fc6a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a0780 0x55617f5a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a9360 0x55617f5ab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bf177740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55617f5050e0 con 0x55617f5a9360 2024-04-16T05:37:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bf177740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55617f4a2520 con 0x55617f5a0780 2024-04-16T05:37:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bf177740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55617f5010a0 con 0x55617f59fe60 2024-04-16T05:37:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6a00a3d20 0x7fc6a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bab33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a9360 0x55617f5ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bab33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a9360 0x55617f5ab750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50908/0 (socket says 172.21.15.86:50908) 2024-04-16T05:37:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bab33640 1 -- 172.21.15.86:0/3601407228 learned_addr learned my addr 172.21.15.86:0/3601407228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6ab7fe640 1 -- 172.21.15.86:0/1792929054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc6ac0f3080 con 0x7fc6b41985e0 2024-04-16T05:37:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6ba332640 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a0780 0x55617f5a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bab33640 1 -- 172.21.15.86:0/3601407228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 msgr2=0x55617f5a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bab33640 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bab33640 1 -- 172.21.15.86:0/3601407228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a0780 msgr2=0x55617f5a8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bab33640 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a0780 0x55617f5a8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.373+0000 7fc6bab33640 1 -- 172.21.15.86:0/3601407228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617f4ca8d0 con 0x55617f5a9360 2024-04-16T05:37:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.374+0000 7fc6ba332640 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.374+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6a00a3d20 0x7fc6a00a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40fe540 tx=0x7fc6a40137d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.374+0000 7fc6bab33640 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a9360 0x55617f5ab750 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0f05d0 tx=0x7fc6ac02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3894ab06723fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.374+0000 7fc6a8bfe640 1 -- 172.21.15.86:0/3601407228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0be720 con 0x55617f5a9360 2024-04-16T05:37:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.374+0000 7fc6a8bfe640 1 -- 172.21.15.86:0/3601407228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac004830 con 0x55617f5a9360 2024-04-16T05:37:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.374+0000 7fc6a8bfe640 1 -- 172.21.15.86:0/3601407228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac086b50 con 0x55617f5a9360 2024-04-16T05:37:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 -- 172.21.15.86:0/3601407228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a9360 msgr2=0x55617f5ab750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a9360 0x55617f5ab750 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0f05d0 tx=0x7fc6ac02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 -- 172.21.15.86:0/3601407228 shutdown_connections 2024-04-16T05:37:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a9360 0x55617f5ab750 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a0780 0x55617f5a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3601407228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 -- 172.21.15.86:0/3601407228 >> 172.21.15.86:0/3601407228 conn(0x55617f59a590 msgr2=0x55617f59a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 -- 172.21.15.86:0/3601407228 shutdown_connections 2024-04-16T05:37:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 -- 172.21.15.86:0/3601407228 wait complete. 2024-04-16T05:37:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 -- start start 2024-04-16T05:37:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 Processor -- start 2024-04-16T05:37:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f59fe60 0x55617f5a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f5a0780 0x55617f5a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 0x55617f5a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55617f5abf10 con 0x55617f59fe60 2024-04-16T05:37:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55617f4be4d0 con 0x55617f5a9360 2024-04-16T05:37:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55617f5ac090 con 0x55617f5a0780 2024-04-16T05:37:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6bf177740 10 client.0 _ll_register_callbacks cb 0x55617f4a32c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6ba332640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f59fe60 0x55617f5a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6ba332640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f59fe60 0x55617f5a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50920/0 (socket says 172.21.15.86:50920) 2024-04-16T05:37:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.375+0000 7fc6ba332640 1 -- 172.21.15.86:0/3295059529 learned_addr learned my addr 172.21.15.86:0/3295059529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6ba332640 1 -- 172.21.15.86:0/3295059529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f5a0780 msgr2=0x55617f5a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6ba332640 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f5a0780 0x55617f5a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6ba332640 1 -- 172.21.15.86:0/3295059529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 msgr2=0x55617f5a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6ba332640 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 0x55617f5a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6ba332640 1 -- 172.21.15.86:0/3295059529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617f2173f0 con 0x55617f59fe60 2024-04-16T05:37:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6ba332640 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f59fe60 0x55617f5a62e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0010860 tx=0x7fc6b0010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28560037aaf92337 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc69dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc69dffb640 1 -- 172.21.15.86:0/3295059529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000ed40 con 0x55617f59fe60 2024-04-16T05:37:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc69dffb640 1 -- 172.21.15.86:0/3295059529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc6b001c480 con 0x55617f59fe60 2024-04-16T05:37:01.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6bf177740 1 -- 172.21.15.86:0/3295059529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6ac004830 con 0x55617f59fe60 2024-04-16T05:37:01.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc69dffb640 1 -- 172.21.15.86:0/3295059529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6b0025050 con 0x55617f59fe60 2024-04-16T05:37:01.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6bf177740 10 client.16242 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6bf177740 1 -- 172.21.15.86:0/3295059529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55617f2e17d0 con 0x55617f59fe60 2024-04-16T05:37:01.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc69dffb640 1 -- 172.21.15.86:0/3295059529 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac004830 con 0x55617f59fe60 2024-04-16T05:37:01.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc69dffb640 1 -- 172.21.15.86:0/3295059529 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc6b00117a0 con 0x55617f59fe60 2024-04-16T05:37:01.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6bf177740 10 client.16242 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6bf177740 10 client.16242 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:37:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6bf177740 20 client.16242 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55617f5b9080 0x55617f5bb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6bf177740 4 client.16242 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.376+0000 7fc6bf177740 1 -- 172.21.15.86:0/3295059529 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55617f512b00 con 0x55617f5b9080 2024-04-16T05:37:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.377+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55617f5b9080 0x55617f5bb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.377+0000 7fc69dffb640 10 client.16242 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:37:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.377+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55617f5b9080 0x55617f5bb520 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9379cb6ff4518131 server_cookie=2f66c8484c33b261 in_seq=0 out_seq=0 2024-04-16T05:37:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc69dffb640 1 -- 172.21.15.86:0/3295059529 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55617f512b00 con 0x55617f5b9080 2024-04-16T05:37:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc69dffb640 10 client.16242 handle_command_reply: tid=0 2024-04-16T05:37:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 client.16242 shutdown 2024-04-16T05:37:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 -- 172.21.15.86:0/3295059529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f59fe60 msgr2=0x55617f5a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f59fe60 0x55617f5a62e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0010860 tx=0x7fc6b0010d20 comp rx=0 tx=0).stop 2024-04-16T05:37:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 -- 172.21.15.86:0/3295059529 shutdown_connections 2024-04-16T05:37:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55617f5b9080 0x55617f5bb520 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 0x55617f5a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc69dffb640 0 client.16242 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:37:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f5a0780 0x55617f5a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3295059529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f59fe60 0x55617f5a62e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 -- 172.21.15.86:0/3295059529 >> 172.21.15.86:0/3295059529 conn(0x55617f59a590 msgr2=0x55617f59c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 -- 172.21.15.86:0/3295059529 shutdown_connections 2024-04-16T05:37:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 1 -- 172.21.15.86:0/3295059529 wait complete. 2024-04-16T05:37:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.378+0000 7fc6bf177740 20 client.16242 trim_cache size 0 max 16384 2024-04-16T05:37:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.391+0000 7fc6bf177740 1 Processor -- start 2024-04-16T05:37:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.391+0000 7fc6bf177740 1 -- start start 2024-04-16T05:37:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.391+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.391+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.391+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 0x55617f5955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.391+0000 7fc6bf177740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41af400 con 0x55617f5a0780 2024-04-16T05:37:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.391+0000 7fc6bf177740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41b0f40 con 0x55617f5a9360 2024-04-16T05:37:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.391+0000 7fc6bf177740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41afad0 con 0x55617f59fe60 2024-04-16T05:37:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc6b9b31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc6b9b31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50934/0 (socket says 172.21.15.86:50934) 2024-04-16T05:37:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc6b9b31640 1 -- 172.21.15.86:0/2135877718 learned_addr learned my addr 172.21.15.86:0/2135877718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc6b9b31640 1 -- 172.21.15.86:0/2135877718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 msgr2=0x55617f5a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/2135877718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc6b9b31640 1 -- 172.21.15.86:0/2135877718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 msgr2=0x55617f5955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/2135877718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 0x55617f5955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc6b9b31640 1 -- 172.21.15.86:0/2135877718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617f5a5290 con 0x55617f5a0780 2024-04-16T05:37:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/2135877718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5b4c10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005fd0 tx=0x7fc6a4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b566d77f9003509e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc69ffff640 1 -- 172.21.15.86:0/2135877718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a402b030 con 0x55617f5a0780 2024-04-16T05:37:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc69ffff640 1 -- 172.21.15.86:0/2135877718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a401fd30 con 0x55617f5a0780 2024-04-16T05:37:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.392+0000 7fc69ffff640 1 -- 172.21.15.86:0/2135877718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a402f3c0 con 0x55617f5a0780 2024-04-16T05:37:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 -- 172.21.15.86:0/2135877718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 msgr2=0x55617f5b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 --2- 172.21.15.86:0/2135877718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5b4c10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005fd0 tx=0x7fc6a4015040 comp rx=0 tx=0).stop 2024-04-16T05:37:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 -- 172.21.15.86:0/2135877718 shutdown_connections 2024-04-16T05:37:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 --2- 172.21.15.86:0/2135877718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 0x55617f5955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 --2- 172.21.15.86:0/2135877718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5b4c10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 --2- 172.21.15.86:0/2135877718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 -- 172.21.15.86:0/2135877718 >> 172.21.15.86:0/2135877718 conn(0x55617f59a590 msgr2=0x55617f599730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 -- 172.21.15.86:0/2135877718 shutdown_connections 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 -- 172.21.15.86:0/2135877718 wait complete. 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 -- start start 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 Processor -- start 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 0x55617f5a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41b0870 con 0x55617f5a0780 2024-04-16T05:37:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41b01a0 con 0x55617f5a9360 2024-04-16T05:37:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b41ad3c0 con 0x55617f59fe60 2024-04-16T05:37:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6bf177740 10 client.0 _ll_register_callbacks cb 0x7fc6b00103f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6b9b31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6b9b31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50950/0 (socket says 172.21.15.86:50950) 2024-04-16T05:37:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.393+0000 7fc6b9b31640 1 -- 172.21.15.86:0/3747252429 learned_addr learned my addr 172.21.15.86:0/3747252429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6b9b31640 1 -- 172.21.15.86:0/3747252429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 msgr2=0x55617f5b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6b9b31640 1 -- 172.21.15.86:0/3747252429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 msgr2=0x55617f5a30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 0x55617f5a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6b9b31640 1 -- 172.21.15.86:0/3747252429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617f5a35f0 con 0x55617f5a0780 2024-04-16T05:37:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6b9b31640 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5a2b70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4000c00 tx=0x7fc6a400aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ab8fbad027b60d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc69dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc69dffb640 1 -- 172.21.15.86:0/3747252429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4034070 con 0x55617f5a0780 2024-04-16T05:37:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc69dffb640 1 -- 172.21.15.86:0/3747252429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc6a4020880 con 0x55617f5a0780 2024-04-16T05:37:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6bf177740 1 -- 172.21.15.86:0/3747252429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a401fd30 con 0x55617f5a0780 2024-04-16T05:37:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc69dffb640 1 -- 172.21.15.86:0/3747252429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6a4020ac0 con 0x55617f5a0780 2024-04-16T05:37:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6bf177740 10 client.16248 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6bf177740 1 -- 172.21.15.86:0/3747252429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55617f61f3b0 con 0x55617f5a0780 2024-04-16T05:37:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc69dffb640 1 -- 172.21.15.86:0/3747252429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a401fd30 con 0x55617f5a0780 2024-04-16T05:37:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc69dffb640 1 -- 172.21.15.86:0/3747252429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc6a400f040 con 0x55617f5a0780 2024-04-16T05:37:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6bf177740 10 client.16248 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6bf177740 10 client.16248 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:37:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6bf177740 20 client.16248 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55617f5b9080 0x55617f621ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6bf177740 4 client.16248 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.394+0000 7fc6bf177740 1 -- 172.21.15.86:0/3747252429 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55617f50c970 con 0x55617f5b9080 2024-04-16T05:37:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6ba332640 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55617f5b9080 0x55617f621ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc69dffb640 10 client.16248 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:37:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6ba332640 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55617f5b9080 0x55617f621ae0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=efea07741b7a4bc2 server_cookie=c0b0c381a7271e13 in_seq=0 out_seq=0 2024-04-16T05:37:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc69dffb640 1 -- 172.21.15.86:0/3747252429 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55617f50c970 con 0x55617f5b9080 2024-04-16T05:37:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc69dffb640 10 client.16248 handle_command_reply: tid=0 2024-04-16T05:37:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 client.16248 shutdown 2024-04-16T05:37:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 -- 172.21.15.86:0/3747252429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 msgr2=0x55617f5a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5a2b70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4000c00 tx=0x7fc6a400aba0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 -- 172.21.15.86:0/3747252429 shutdown_connections 2024-04-16T05:37:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55617f5b9080 0x55617f621ae0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55617f5a9360 0x55617f5a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617f5a0780 0x55617f5a2b70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc69dffb640 0 client.16248 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:37:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 --2- 172.21.15.86:0/3747252429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55617f59fe60 0x55617f5b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 -- 172.21.15.86:0/3747252429 >> 172.21.15.86:0/3747252429 conn(0x55617f59a590 msgr2=0x55617f598ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 -- 172.21.15.86:0/3747252429 shutdown_connections 2024-04-16T05:37:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.395+0000 7fc6bf177740 1 -- 172.21.15.86:0/3747252429 wait complete. 2024-04-16T05:37:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bf177740 20 client.16248 trim_cache size 0 max 16384 2024-04-16T05:37:01.406 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 -- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6a00a3d20 msgr2=0x7fc6a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6a00a3d20 0x7fc6a00a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40fe540 tx=0x7fc6a40137d0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 -- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b41985e0 msgr2=0x7fc6b4198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b41985e0 0x7fc6b4198a90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac02c700 tx=0x7fc6ac035b60 comp rx=0 tx=0).stop 2024-04-16T05:37:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 -- 172.21.15.86:0/1792929054 shutdown_connections 2024-04-16T05:37:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6b41985e0 0x7fc6b4198a90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6a00a3d20 0x7fc6a00a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6b4106200 0x7fc6b419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 --2- 172.21.15.86:0/1792929054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b4101130 0x7fc6b419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 -- 172.21.15.86:0/1792929054 >> 172.21.15.86:0/1792929054 conn(0x7fc6b40780b0 msgr2=0x7fc6b40fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 -- 172.21.15.86:0/1792929054 shutdown_connections 2024-04-16T05:37:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.396+0000 7fc6bc596640 1 -- 172.21.15.86:0/1792929054 wait complete. 2024-04-16T05:37:01.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:37:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 -- 172.21.15.86:0/3153437723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758107320 msgr2=0x7fb75810eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 --2- 172.21.15.86:0/3153437723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758107320 0x7fb75810eef0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb750009ca0 tx=0x7fb75002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 -- 172.21.15.86:0/3153437723 shutdown_connections 2024-04-16T05:37:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 --2- 172.21.15.86:0/3153437723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758107320 0x7fb75810eef0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 --2- 172.21.15.86:0/3153437723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb758102c00 0x7fb758106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 --2- 172.21.15.86:0/3153437723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758102250 0x7fb758102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 -- 172.21.15.86:0/3153437723 >> 172.21.15.86:0/3153437723 conn(0x7fb758079490 msgr2=0x7fb7580798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 -- 172.21.15.86:0/3153437723 shutdown_connections 2024-04-16T05:37:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 -- 172.21.15.86:0/3153437723 wait complete. 2024-04-16T05:37:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 Processor -- start 2024-04-16T05:37:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.518+0000 7fb760324640 1 -- start start 2024-04-16T05:37:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb760324640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758102250 0x7fb75819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb760324640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758107320 0x7fb75819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb760324640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7581985c0 0x7fb758198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb760324640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581123c0 con 0x7fb758102250 2024-04-16T05:37:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb760324640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb758112240 con 0x7fb758107320 2024-04-16T05:37:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb760324640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb758112540 con 0x7fb7581985c0 2024-04-16T05:37:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e0c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758102250 0x7fb75819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e0c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758102250 0x7fb75819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50970/0 (socket says 172.21.15.86:50970) 2024-04-16T05:37:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e0c0640 1 -- 172.21.15.86:0/950413943 learned_addr learned my addr 172.21.15.86:0/950413943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75d8bf640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758107320 0x7fb75819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7581985c0 0x7fb758198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e0c0640 1 -- 172.21.15.86:0/950413943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7581985c0 msgr2=0x7fb758198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7581985c0 0x7fb758198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e0c0640 1 -- 172.21.15.86:0/950413943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758107320 msgr2=0x7fb75819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758107320 0x7fb75819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e0c0640 1 -- 172.21.15.86:0/950413943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7581990e0 con 0x7fb758102250 2024-04-16T05:37:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758102250 0x7fb75819e410 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb74800c920 tx=0x7fb74800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb74f7fe640 1 -- 172.21.15.86:0/950413943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748007c70 con 0x7fb758102250 2024-04-16T05:37:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb760324640 1 -- 172.21.15.86:0/950413943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7581993d0 con 0x7fb758102250 2024-04-16T05:37:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb74f7fe640 1 -- 172.21.15.86:0/950413943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb748007e10 con 0x7fb758102250 2024-04-16T05:37:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.519+0000 7fb74f7fe640 1 -- 172.21.15.86:0/950413943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748018890 con 0x7fb758102250 2024-04-16T05:37:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.520+0000 7fb760324640 1 -- 172.21.15.86:0/950413943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7581a5670 con 0x7fb758102250 2024-04-16T05:37:01.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.522+0000 7fb74f7fe640 1 -- 172.21.15.86:0/950413943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb748018a70 con 0x7fb758102250 2024-04-16T05:37:01.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.522+0000 7fb762fee740 1 Processor -- start 2024-04-16T05:37:01.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.522+0000 7fb762fee740 1 -- start start 2024-04-16T05:37:01.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb762fee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f24e60 0x558aa0f25240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb762fee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f25780 0x558aa0f2de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb762fee740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f2e360 0x558aa0f30750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb762fee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558aa0e8a0e0 con 0x558aa0f25780 2024-04-16T05:37:01.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb762fee740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558aa0e27520 con 0x558aa0f2e360 2024-04-16T05:37:01.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb762fee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558aa0e860a0 con 0x558aa0f24e60 2024-04-16T05:37:01.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb74f7fe640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7440a3d20 0x7fb7440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e0c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f25780 0x558aa0f2de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7440a3d20 0x7fb7440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e0c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f25780 0x558aa0f2de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50972/0 (socket says 172.21.15.86:50972) 2024-04-16T05:37:01.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e0c0640 1 -- 172.21.15.86:0/3669072069 learned_addr learned my addr 172.21.15.86:0/3669072069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75d8bf640 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f2e360 0x558aa0f30750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f24e60 0x558aa0f25240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb74f7fe640 1 -- 172.21.15.86:0/950413943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb7480d0cc0 con 0x7fb758102250 2024-04-16T05:37:01.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e0c0640 1 -- 172.21.15.86:0/3669072069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f24e60 msgr2=0x558aa0f25240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f24e60 0x558aa0f25240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e0c0640 1 -- 172.21.15.86:0/3669072069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f2e360 msgr2=0x558aa0f30750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f2e360 0x558aa0f30750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e0c0640 1 -- 172.21.15.86:0/3669072069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa0e4f8d0 con 0x558aa0f25780 2024-04-16T05:37:01.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75d8bf640 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f2e360 0x558aa0f30750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:37:01.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f25780 0x558aa0f2de20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb7480992c0 tx=0x7fb74800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef51d0480b3ed46e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7440a3d20 0x7fb7440a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb750009430 tx=0x7fb750009880 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.523+0000 7fb74cbfe640 1 -- 172.21.15.86:0/3669072069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480a0ca0 con 0x558aa0f25780 2024-04-16T05:37:01.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb74cbfe640 1 -- 172.21.15.86:0/3669072069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7480a5560 con 0x558aa0f25780 2024-04-16T05:37:01.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb74cbfe640 1 -- 172.21.15.86:0/3669072069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480ce320 con 0x558aa0f25780 2024-04-16T05:37:01.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb762fee740 1 -- 172.21.15.86:0/3669072069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f25780 msgr2=0x558aa0f2de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb762fee740 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f25780 0x558aa0f2de20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb7480992c0 tx=0x7fb74800cf90 comp rx=0 tx=0).stop 2024-04-16T05:37:01.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb762fee740 1 -- 172.21.15.86:0/3669072069 shutdown_connections 2024-04-16T05:37:01.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb762fee740 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f2e360 0x558aa0f30750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb762fee740 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f25780 0x558aa0f2de20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb762fee740 1 --2- 172.21.15.86:0/3669072069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f24e60 0x558aa0f25240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb762fee740 1 -- 172.21.15.86:0/3669072069 >> 172.21.15.86:0/3669072069 conn(0x558aa0f1f590 msgr2=0x558aa0f1f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb762fee740 1 -- 172.21.15.86:0/3669072069 shutdown_connections 2024-04-16T05:37:01.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.524+0000 7fb762fee740 1 -- 172.21.15.86:0/3669072069 wait complete. 2024-04-16T05:37:01.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb762fee740 1 -- start start 2024-04-16T05:37:01.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb762fee740 1 Processor -- start 2024-04-16T05:37:01.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb762fee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f24e60 0x558aa0f2b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb762fee740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f25780 0x558aa0f2baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb762fee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f2e360 0x558aa0f2bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb762fee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558aa0f30fa0 con 0x558aa0f24e60 2024-04-16T05:37:01.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb762fee740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558aa0e434d0 con 0x558aa0f25780 2024-04-16T05:37:01.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb762fee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558aa0f31120 con 0x558aa0f2e360 2024-04-16T05:37:01.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb762fee740 10 client.0 _ll_register_callbacks cb 0x558aa0e282c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75e0c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f24e60 0x558aa0f2b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75e0c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f24e60 0x558aa0f2b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50986/0 (socket says 172.21.15.86:50986) 2024-04-16T05:37:01.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75e0c0640 1 -- 172.21.15.86:0/3787854129 learned_addr learned my addr 172.21.15.86:0/3787854129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75d8bf640 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f25780 0x558aa0f2baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75e0c0640 1 -- 172.21.15.86:0/3787854129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f2e360 msgr2=0x558aa0f2bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f2e360 0x558aa0f2bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75e0c0640 1 -- 172.21.15.86:0/3787854129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f25780 msgr2=0x558aa0f2baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f25780 0x558aa0f2baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75e0c0640 1 -- 172.21.15.86:0/3787854129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aa0b9c3f0 con 0x558aa0f24e60 2024-04-16T05:37:01.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.525+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f24e60 0x558aa0f2b560 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb748098d00 tx=0x7fb748005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab395d59d962ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb741ffb640 1 -- 172.21.15.86:0/3787854129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748087110 con 0x558aa0f24e60 2024-04-16T05:37:01.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb741ffb640 1 -- 172.21.15.86:0/3787854129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb7480ce320 con 0x558aa0f24e60 2024-04-16T05:37:01.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb762fee740 1 -- 172.21.15.86:0/3787854129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7480a5560 con 0x558aa0f24e60 2024-04-16T05:37:01.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb741ffb640 1 -- 172.21.15.86:0/3787854129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7480ce520 con 0x558aa0f24e60 2024-04-16T05:37:01.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb762fee740 10 client.16260 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb762fee740 1 -- 172.21.15.86:0/3787854129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558aa0c667d0 con 0x558aa0f24e60 2024-04-16T05:37:01.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb741ffb640 1 -- 172.21.15.86:0/3787854129 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7480a5560 con 0x558aa0f24e60 2024-04-16T05:37:01.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb741ffb640 1 -- 172.21.15.86:0/3787854129 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb74807b200 con 0x558aa0f24e60 2024-04-16T05:37:01.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb762fee740 10 client.16260 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb762fee740 10 client.16260 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:37:01.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb762fee740 20 client.16260 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb762fee740 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x558aa0f3e230 0x558aa0f40620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb762fee740 4 client.16260 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:01.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb762fee740 1 -- 172.21.15.86:0/3787854129 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558aa0e97b00 con 0x558aa0f3e230 2024-04-16T05:37:01.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.526+0000 7fb75d8bf640 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x558aa0f3e230 0x558aa0f40620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.527+0000 7fb741ffb640 10 client.16260 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:37:01.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.527+0000 7fb75d8bf640 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x558aa0f3e230 0x558aa0f40620 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e4dab92813fcba72 server_cookie=5eec4bd05b1b03bc in_seq=0 out_seq=0 2024-04-16T05:37:01.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb741ffb640 1 -- 172.21.15.86:0/3787854129 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558aa0e97b00 con 0x558aa0f3e230 2024-04-16T05:37:01.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb741ffb640 10 client.16260 handle_command_reply: tid=0 2024-04-16T05:37:01.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 client.16260 shutdown 2024-04-16T05:37:01.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 -- 172.21.15.86:0/3787854129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f24e60 msgr2=0x558aa0f2b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f24e60 0x558aa0f2b560 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb748098d00 tx=0x7fb748005740 comp rx=0 tx=0).stop 2024-04-16T05:37:01.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 -- 172.21.15.86:0/3787854129 shutdown_connections 2024-04-16T05:37:01.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x558aa0f3e230 0x558aa0f40620 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f2e360 0x558aa0f2bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f25780 0x558aa0f2baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb741ffb640 0 client.16260 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:37:01.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 --2- 172.21.15.86:0/3787854129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f24e60 0x558aa0f2b560 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 -- 172.21.15.86:0/3787854129 >> 172.21.15.86:0/3787854129 conn(0x558aa0f1f590 msgr2=0x558aa0f212e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 -- 172.21.15.86:0/3787854129 shutdown_connections 2024-04-16T05:37:01.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 1 -- 172.21.15.86:0/3787854129 wait complete. 2024-04-16T05:37:01.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.528+0000 7fb762fee740 20 client.16260 trim_cache size 0 max 16384 2024-04-16T05:37:01.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb762fee740 1 Processor -- start 2024-04-16T05:37:01.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb762fee740 1 -- start start 2024-04-16T05:37:01.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb762fee740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f24e60 0x558aa0f2dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb762fee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 0x558aa0f2bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb762fee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f43640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb762fee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581af460 con 0x558aa0f2e360 2024-04-16T05:37:01.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb762fee740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581b0fa0 con 0x558aa0f24e60 2024-04-16T05:37:01.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb762fee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581afb30 con 0x558aa0f25780 2024-04-16T05:37:01.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb75e8c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f43640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb75e8c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f43640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50992/0 (socket says 172.21.15.86:50992) 2024-04-16T05:37:01.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb75e8c1640 1 -- 172.21.15.86:0/2800353012 learned_addr learned my addr 172.21.15.86:0/2800353012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb75e8c1640 1 -- 172.21.15.86:0/2800353012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 msgr2=0x558aa0f2bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.541+0000 7fb75d8bf640 1 --2- 172.21.15.86:0/2800353012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 0x558aa0f2bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/2800353012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 0x558aa0f2bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb75e8c1640 1 -- 172.21.15.86:0/2800353012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f24e60 msgr2=0x558aa0f2dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/2800353012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f24e60 0x558aa0f2dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb75e8c1640 1 -- 172.21.15.86:0/2800353012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa0f25480 con 0x558aa0f2e360 2024-04-16T05:37:01.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb75d8bf640 1 --2- 172.21.15.86:0/2800353012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 0x558aa0f2bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:01.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/2800353012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f43640 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb750009ca0 tx=0x7fb750046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a83fbcd9419595 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb743fff640 1 -- 172.21.15.86:0/2800353012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750004b40 con 0x558aa0f2e360 2024-04-16T05:37:01.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb743fff640 1 -- 172.21.15.86:0/2800353012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb750052070 con 0x558aa0f2e360 2024-04-16T05:37:01.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb743fff640 1 -- 172.21.15.86:0/2800353012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7500042a0 con 0x558aa0f2e360 2024-04-16T05:37:01.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb762fee740 1 -- 172.21.15.86:0/2800353012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 msgr2=0x558aa0f43640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb762fee740 1 --2- 172.21.15.86:0/2800353012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f43640 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb750009ca0 tx=0x7fb750046000 comp rx=0 tx=0).stop 2024-04-16T05:37:01.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb762fee740 1 -- 172.21.15.86:0/2800353012 shutdown_connections 2024-04-16T05:37:01.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb762fee740 1 --2- 172.21.15.86:0/2800353012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f43640 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb762fee740 1 --2- 172.21.15.86:0/2800353012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 0x558aa0f2bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb762fee740 1 --2- 172.21.15.86:0/2800353012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f24e60 0x558aa0f2dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.542+0000 7fb762fee740 1 -- 172.21.15.86:0/2800353012 >> 172.21.15.86:0/2800353012 conn(0x558aa0f1f590 msgr2=0x558aa0f2b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 -- 172.21.15.86:0/2800353012 shutdown_connections 2024-04-16T05:37:01.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 -- 172.21.15.86:0/2800353012 wait complete. 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 -- start start 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 Processor -- start 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f24e60 0x558aa0f43790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 0x558aa0f40b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f41060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581b08d0 con 0x558aa0f2e360 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581b0200 con 0x558aa0f24e60 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581ad420 con 0x558aa0f25780 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb762fee740 10 client.0 _ll_register_callbacks cb 0x7fb75400b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb75e8c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f41060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb75e8c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f41060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50996/0 (socket says 172.21.15.86:50996) 2024-04-16T05:37:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb75e8c1640 1 -- 172.21.15.86:0/2917270251 learned_addr learned my addr 172.21.15.86:0/2917270251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb75d8bf640 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 0x558aa0f40b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb75e8c1640 1 -- 172.21.15.86:0/2917270251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 msgr2=0x558aa0f40b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 0x558aa0f40b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb75e8c1640 1 -- 172.21.15.86:0/2917270251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f24e60 msgr2=0x558aa0f43790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f24e60 0x558aa0f43790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.543+0000 7fb75e8c1640 1 -- 172.21.15.86:0/2917270251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aa0f415d0 con 0x558aa0f2e360 2024-04-16T05:37:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb75e8c1640 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f41060 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb75002c130 tx=0x7fb750004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431edbecb176e34f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb741ffb640 1 -- 172.21.15.86:0/2917270251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7500327a0 con 0x558aa0f2e360 2024-04-16T05:37:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb741ffb640 1 -- 172.21.15.86:0/2917270251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb750032940 con 0x558aa0f2e360 2024-04-16T05:37:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb762fee740 1 -- 172.21.15.86:0/2917270251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb750052070 con 0x558aa0f2e360 2024-04-16T05:37:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb741ffb640 1 -- 172.21.15.86:0/2917270251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb750065050 con 0x558aa0f2e360 2024-04-16T05:37:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb762fee740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb762fee740 1 -- 172.21.15.86:0/2917270251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558aa0fa4100 con 0x558aa0f2e360 2024-04-16T05:37:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb741ffb640 1 -- 172.21.15.86:0/2917270251 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb750052070 con 0x558aa0f2e360 2024-04-16T05:37:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb741ffb640 1 -- 172.21.15.86:0/2917270251 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb750034440 con 0x558aa0f2e360 2024-04-16T05:37:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb762fee740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb762fee740 10 client.16266 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:37:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb762fee740 20 client.16266 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb762fee740 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x558aa0fa4840 0x558aa0fa6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb762fee740 4 client.16266 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.544+0000 7fb762fee740 1 -- 172.21.15.86:0/2917270251 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558aa0e91970 con 0x558aa0fa4840 2024-04-16T05:37:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x558aa0fa4840 0x558aa0fa6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb741ffb640 10 client.16266 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:37:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb75e0c0640 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x558aa0fa4840 0x558aa0fa6c60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a596b308fb3012a5 server_cookie=be0ee1328d8d01e3 in_seq=0 out_seq=0 2024-04-16T05:37:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb741ffb640 1 -- 172.21.15.86:0/2917270251 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558aa0e91970 con 0x558aa0fa4840 2024-04-16T05:37:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb741ffb640 10 client.16266 handle_command_reply: tid=0 2024-04-16T05:37:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 client.16266 shutdown 2024-04-16T05:37:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 -- 172.21.15.86:0/2917270251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 msgr2=0x558aa0f41060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f41060 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb75002c130 tx=0x7fb750004d70 comp rx=0 tx=0).stop 2024-04-16T05:37:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 -- 172.21.15.86:0/2917270251 shutdown_connections 2024-04-16T05:37:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aa0f2e360 0x558aa0f41060 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x558aa0fa4840 0x558aa0fa6c60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aa0f25780 0x558aa0f40b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb741ffb640 0 client.16266 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:37:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 --2- 172.21.15.86:0/2917270251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aa0f24e60 0x558aa0f43790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 -- 172.21.15.86:0/2917270251 >> 172.21.15.86:0/2917270251 conn(0x558aa0f1f590 msgr2=0x558aa0f39c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 -- 172.21.15.86:0/2917270251 shutdown_connections 2024-04-16T05:37:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 1 -- 172.21.15.86:0/2917270251 wait complete. 2024-04-16T05:37:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.545+0000 7fb762fee740 20 client.16266 trim_cache size 0 max 16384 2024-04-16T05:37:01.555 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:01.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 -- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7440a3d20 msgr2=0x7fb7440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7440a3d20 0x7fb7440a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb750009430 tx=0x7fb750009880 comp rx=0 tx=0).stop 2024-04-16T05:37:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 -- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758102250 msgr2=0x7fb75819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758102250 0x7fb75819e410 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb74800c920 tx=0x7fb74800cde0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 -- 172.21.15.86:0/950413943 shutdown_connections 2024-04-16T05:37:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7581985c0 0x7fb758198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7440a3d20 0x7fb7440a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758107320 0x7fb75819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 --2- 172.21.15.86:0/950413943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb758102250 0x7fb75819e410 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 -- 172.21.15.86:0/950413943 >> 172.21.15.86:0/950413943 conn(0x7fb758079490 msgr2=0x7fb7580ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 -- 172.21.15.86:0/950413943 shutdown_connections 2024-04-16T05:37:01.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.546+0000 7fb760324640 1 -- 172.21.15.86:0/950413943 wait complete. 2024-04-16T05:37:01.557 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:01 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/259649782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:01.558 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:37:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.665+0000 7f0faacba640 1 -- 172.21.15.86:0/3037715068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4102250 msgr2=0x7f0fa4102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.665+0000 7f0faacba640 1 --2- 172.21.15.86:0/3037715068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4102250 0x7f0fa4102630 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009900 tx=0x7f0f9802bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.665+0000 7f0faacba640 1 -- 172.21.15.86:0/3037715068 shutdown_connections 2024-04-16T05:37:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.665+0000 7f0faacba640 1 --2- 172.21.15.86:0/3037715068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fa4107320 0x7f0fa410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.665+0000 7f0faacba640 1 --2- 172.21.15.86:0/3037715068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4102c00 0x7f0fa4106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.665+0000 7f0faacba640 1 --2- 172.21.15.86:0/3037715068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4102250 0x7f0fa4102630 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.665+0000 7f0faacba640 1 -- 172.21.15.86:0/3037715068 >> 172.21.15.86:0/3037715068 conn(0x7f0fa4079490 msgr2=0x7f0fa40798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.665+0000 7f0faacba640 1 -- 172.21.15.86:0/3037715068 shutdown_connections 2024-04-16T05:37:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.665+0000 7f0faacba640 1 -- 172.21.15.86:0/3037715068 wait complete. 2024-04-16T05:37:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0faacba640 1 Processor -- start 2024-04-16T05:37:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0faacba640 1 -- start start 2024-04-16T05:37:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0faacba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fa4102250 0x7f0fa4074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0faacba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4107320 0x7f0fa4074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0faacba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4074d80 0x7f0fa41a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0faacba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41123b0 con 0x7f0fa4107320 2024-04-16T05:37:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0faacba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4112230 con 0x7f0fa4102250 2024-04-16T05:37:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0faacba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4112530 con 0x7f0fa4074d80 2024-04-16T05:37:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4107320 0x7f0fa4074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4107320 0x7f0fa4074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51008/0 (socket says 172.21.15.86:51008) 2024-04-16T05:37:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa3fff640 1 -- 172.21.15.86:0/2303338276 learned_addr learned my addr 172.21.15.86:0/2303338276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa8a56640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fa4102250 0x7f0fa4074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa9257640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4074d80 0x7f0fa41a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa3fff640 1 -- 172.21.15.86:0/2303338276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4074d80 msgr2=0x7f0fa41a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa3fff640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4074d80 0x7f0fa41a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa3fff640 1 -- 172.21.15.86:0/2303338276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fa4102250 msgr2=0x7f0fa4074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa3fff640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fa4102250 0x7f0fa4074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa3fff640 1 -- 172.21.15.86:0/2303338276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa41a7180 con 0x7f0fa4107320 2024-04-16T05:37:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.666+0000 7f0fa9257640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4074d80 0x7f0fa41a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.667+0000 7f0fa3fff640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4107320 0x7f0fa4074840 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000c970 tx=0x7f0f9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.667+0000 7f0fa1ffb640 1 -- 172.21.15.86:0/2303338276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90007bb0 con 0x7f0fa4107320 2024-04-16T05:37:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.667+0000 7f0fa1ffb640 1 -- 172.21.15.86:0/2303338276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f90007d50 con 0x7f0fa4107320 2024-04-16T05:37:01.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.667+0000 7f0faacba640 1 -- 172.21.15.86:0/2303338276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa41a7470 con 0x7f0fa4107320 2024-04-16T05:37:01.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.667+0000 7f0fa1ffb640 1 -- 172.21.15.86:0/2303338276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000f450 con 0x7f0fa4107320 2024-04-16T05:37:01.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.667+0000 7f0faacba640 1 -- 172.21.15.86:0/2303338276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa41a79c0 con 0x7f0fa4107320 2024-04-16T05:37:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.669+0000 7f0fa1ffb640 1 -- 172.21.15.86:0/2303338276 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0f900046c0 con 0x7f0fa4107320 2024-04-16T05:37:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.669+0000 7f0fad977740 1 Processor -- start 2024-04-16T05:37:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.669+0000 7f0fad977740 1 -- start start 2024-04-16T05:37:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fad977740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99ee60 0x55da9a99f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fad977740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99f780 0x55da9a9a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa1ffb640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0f940a3d20 0x7f0f940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fad977740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9aa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fad977740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55da9a9040e0 con 0x55da9a9a8360 2024-04-16T05:37:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fad977740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55da9a8a1520 con 0x55da9a99ee60 2024-04-16T05:37:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fad977740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da9a9000a0 con 0x55da9a99f780 2024-04-16T05:37:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa3fff640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0f940a3d20 0x7f0f940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9aa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51012/0 (socket says 172.21.15.86:51012) 2024-04-16T05:37:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 -- 172.21.15.86:0/3491644971 learned_addr learned my addr 172.21.15.86:0/3491644971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa1ffb640 1 -- 172.21.15.86:0/2303338276 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0f900d0b90 con 0x7f0fa4107320 2024-04-16T05:37:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa8a56640 1 --2- 172.21.15.86:0/3491644971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99f780 0x55da9a9a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 --2- 172.21.15.86:0/3491644971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99ee60 0x55da9a99f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 -- 172.21.15.86:0/3491644971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99f780 msgr2=0x55da9a9a7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 --2- 172.21.15.86:0/3491644971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99f780 0x55da9a9a7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 -- 172.21.15.86:0/3491644971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99ee60 msgr2=0x55da9a99f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 --2- 172.21.15.86:0/3491644971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99ee60 0x55da9a99f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 -- 172.21.15.86:0/3491644971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da9a8c98d0 con 0x55da9a9a8360 2024-04-16T05:37:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa3fff640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0f940a3d20 0x7f0f940a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0f90007590 tx=0x7f0f90087040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0fa9257640 1 --2- 172.21.15.86:0/3491644971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9aa750 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c013820 tx=0x7f0f9c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2029574c739fc441 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.670+0000 7f0f8effd640 1 -- 172.21.15.86:0/3491644971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c019030 con 0x55da9a9a8360 2024-04-16T05:37:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0f8effd640 1 -- 172.21.15.86:0/3491644971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c011dc0 con 0x55da9a9a8360 2024-04-16T05:37:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0f8effd640 1 -- 172.21.15.86:0/3491644971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c00fdd0 con 0x55da9a9a8360 2024-04-16T05:37:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 -- 172.21.15.86:0/3491644971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 msgr2=0x55da9a9aa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 --2- 172.21.15.86:0/3491644971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9aa750 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c013820 tx=0x7f0f9c009a50 comp rx=0 tx=0).stop 2024-04-16T05:37:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 -- 172.21.15.86:0/3491644971 shutdown_connections 2024-04-16T05:37:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 --2- 172.21.15.86:0/3491644971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9aa750 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 --2- 172.21.15.86:0/3491644971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99f780 0x55da9a9a7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 --2- 172.21.15.86:0/3491644971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99ee60 0x55da9a99f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 -- 172.21.15.86:0/3491644971 >> 172.21.15.86:0/3491644971 conn(0x55da9a999590 msgr2=0x55da9a999970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 -- 172.21.15.86:0/3491644971 shutdown_connections 2024-04-16T05:37:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 -- 172.21.15.86:0/3491644971 wait complete. 2024-04-16T05:37:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.671+0000 7f0fad977740 1 -- start start 2024-04-16T05:37:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fad977740 1 Processor -- start 2024-04-16T05:37:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fad977740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99ee60 0x55da9a9a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fad977740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99f780 0x55da9a9a5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fad977740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a9a8360 0x55da9a9a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fad977740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55da9a9ab060 con 0x55da9a99ee60 2024-04-16T05:37:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fad977740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55da9a8bd4d0 con 0x55da9a99f780 2024-04-16T05:37:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fad977740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da9a9ab1e0 con 0x55da9a9a8360 2024-04-16T05:37:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fad977740 10 client.0 _ll_register_callbacks cb 0x55da9a8a22c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fa9257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99ee60 0x55da9a9a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fa9257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99ee60 0x55da9a9a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51028/0 (socket says 172.21.15.86:51028) 2024-04-16T05:37:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fa9257640 1 -- 172.21.15.86:0/2702135462 learned_addr learned my addr 172.21.15.86:0/2702135462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fa9257640 1 -- 172.21.15.86:0/2702135462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a9a8360 msgr2=0x55da9a9a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fa9257640 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a9a8360 0x55da9a9a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fa9257640 1 -- 172.21.15.86:0/2702135462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99f780 msgr2=0x55da9a9a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fa9257640 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99f780 0x55da9a9a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fa9257640 1 -- 172.21.15.86:0/2702135462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da9a6163f0 con 0x55da9a99ee60 2024-04-16T05:37:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.672+0000 7f0fa9257640 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99ee60 0x55da9a9a53e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c005c40 tx=0x7f0f9c00fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cca683845f2041 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0f83fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0f83fff640 1 -- 172.21.15.86:0/2702135462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c019040 con 0x55da9a99ee60 2024-04-16T05:37:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0f83fff640 1 -- 172.21.15.86:0/2702135462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0f9c013070 con 0x55da9a99ee60 2024-04-16T05:37:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0fad977740 1 -- 172.21.15.86:0/2702135462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f9c011dc0 con 0x55da9a99ee60 2024-04-16T05:37:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0f83fff640 1 -- 172.21.15.86:0/2702135462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f9c013270 con 0x55da9a99ee60 2024-04-16T05:37:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0fad977740 10 client.16278 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0fad977740 1 -- 172.21.15.86:0/2702135462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55da9a6e07d0 con 0x55da9a99ee60 2024-04-16T05:37:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0f83fff640 1 -- 172.21.15.86:0/2702135462 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c011dc0 con 0x55da9a99ee60 2024-04-16T05:37:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0f83fff640 1 -- 172.21.15.86:0/2702135462 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0f9c025070 con 0x55da9a99ee60 2024-04-16T05:37:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0fad977740 10 client.16278 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0fad977740 10 client.16278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:37:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0fad977740 20 client.16278 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0fad977740 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55da9a9b82a0 0x55da9a9ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0fad977740 4 client.16278 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.673+0000 7f0fad977740 1 -- 172.21.15.86:0/2702135462 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da9a911b00 con 0x55da9a9b82a0 2024-04-16T05:37:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.674+0000 7f0fa8a56640 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55da9a9b82a0 0x55da9a9ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.674+0000 7f0f83fff640 10 client.16278 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:37:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.674+0000 7f0fa8a56640 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55da9a9b82a0 0x55da9a9ba690 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=745fb4fb9d48c2d0 server_cookie=831ef10ebc4e6bbb in_seq=0 out_seq=0 2024-04-16T05:37:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0f83fff640 1 -- 172.21.15.86:0/2702135462 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da9a911b00 con 0x55da9a9b82a0 2024-04-16T05:37:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0f83fff640 10 client.16278 handle_command_reply: tid=0 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 client.16278 shutdown 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 -- 172.21.15.86:0/2702135462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99ee60 msgr2=0x55da9a9a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99ee60 0x55da9a9a53e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c005c40 tx=0x7f0f9c00fda0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 -- 172.21.15.86:0/2702135462 shutdown_connections 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a9a8360 0x55da9a9a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99f780 0x55da9a9a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55da9a9b82a0 0x55da9a9ba690 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 --2- 172.21.15.86:0/2702135462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99ee60 0x55da9a9a53e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0f83fff640 0 client.16278 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:37:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 -- 172.21.15.86:0/2702135462 >> 172.21.15.86:0/2702135462 conn(0x55da9a999590 msgr2=0x55da9a99b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 -- 172.21.15.86:0/2702135462 shutdown_connections 2024-04-16T05:37:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 1 -- 172.21.15.86:0/2702135462 wait complete. 2024-04-16T05:37:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.675+0000 7f0fad977740 20 client.16278 trim_cache size 0 max 16384 2024-04-16T05:37:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.688+0000 7f0fad977740 1 Processor -- start 2024-04-16T05:37:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.688+0000 7f0fad977740 1 -- start start 2024-04-16T05:37:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.688+0000 7f0fad977740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 0x55da9a9b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.688+0000 7f0fad977740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99f780 0x55da9a998970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.688+0000 7f0fad977740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a9a8360 0x55da9a9a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.688+0000 7f0fad977740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41b1770 con 0x55da9a99f780 2024-04-16T05:37:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.688+0000 7f0fad977740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41b32b0 con 0x55da9a9a8360 2024-04-16T05:37:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.688+0000 7f0fad977740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41b1e40 con 0x55da9a99ee60 2024-04-16T05:37:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.688+0000 7f0fa8a56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99f780 0x55da9a998970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fa8a56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99f780 0x55da9a998970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51040/0 (socket says 172.21.15.86:51040) 2024-04-16T05:37:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fa8a56640 1 -- 172.21.15.86:0/96755157 learned_addr learned my addr 172.21.15.86:0/96755157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fa9257640 1 --2- 172.21.15.86:0/96755157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 0x55da9a9b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fa8a56640 1 -- 172.21.15.86:0/96755157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 msgr2=0x55da9a9b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fa8a56640 1 --2- 172.21.15.86:0/96755157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 0x55da9a9b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fa8a56640 1 -- 172.21.15.86:0/96755157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a9a8360 msgr2=0x55da9a9a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fa8a56640 1 --2- 172.21.15.86:0/96755157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a9a8360 0x55da9a9a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fa8a56640 1 -- 172.21.15.86:0/96755157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da9a9a6340 con 0x55da9a99f780 2024-04-16T05:37:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fa8a56640 1 --2- 172.21.15.86:0/96755157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99f780 0x55da9a998970 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0f98006010 tx=0x7f0f9802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e890bedf4b89dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0f8e7fc640 1 -- 172.21.15.86:0/96755157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9804a070 con 0x55da9a99f780 2024-04-16T05:37:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0f8e7fc640 1 -- 172.21.15.86:0/96755157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f98046c10 con 0x55da9a99f780 2024-04-16T05:37:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0f8e7fc640 1 -- 172.21.15.86:0/96755157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98004920 con 0x55da9a99f780 2024-04-16T05:37:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fad977740 1 -- 172.21.15.86:0/96755157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99f780 msgr2=0x55da9a998970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.689+0000 7f0fad977740 1 --2- 172.21.15.86:0/96755157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99f780 0x55da9a998970 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0f98006010 tx=0x7f0f9802e040 comp rx=0 tx=0).stop 2024-04-16T05:37:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 -- 172.21.15.86:0/96755157 shutdown_connections 2024-04-16T05:37:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 --2- 172.21.15.86:0/96755157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a9a8360 0x55da9a9a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 --2- 172.21.15.86:0/96755157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a99f780 0x55da9a998970 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 --2- 172.21.15.86:0/96755157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 0x55da9a9b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 -- 172.21.15.86:0/96755157 >> 172.21.15.86:0/96755157 conn(0x55da9a999590 msgr2=0x55da9a9a5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 -- 172.21.15.86:0/96755157 shutdown_connections 2024-04-16T05:37:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 -- 172.21.15.86:0/96755157 wait complete. 2024-04-16T05:37:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 -- start start 2024-04-16T05:37:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 Processor -- start 2024-04-16T05:37:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 0x55da9a9b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99f780 0x55da9a9b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41b2be0 con 0x55da9a9a8360 2024-04-16T05:37:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41b2510 con 0x55da9a99f780 2024-04-16T05:37:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41af730 con 0x55da9a99ee60 2024-04-16T05:37:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fad977740 10 client.0 _ll_register_callbacks cb 0x7f0f98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fa9257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fa9257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51048/0 (socket says 172.21.15.86:51048) 2024-04-16T05:37:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.690+0000 7f0fa9257640 1 -- 172.21.15.86:0/1768261021 learned_addr learned my addr 172.21.15.86:0/1768261021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fa9257640 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 0x55da9a9b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fa9257640 1 -- 172.21.15.86:0/1768261021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 msgr2=0x55da9a9b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fa9257640 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 0x55da9a9b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fa9257640 1 -- 172.21.15.86:0/1768261021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99f780 msgr2=0x55da9a9b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fa9257640 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99f780 0x55da9a9b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fa9257640 1 -- 172.21.15.86:0/1768261021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da9a9a2270 con 0x55da9a9a8360 2024-04-16T05:37:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fa9257640 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9a1a40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009350 tx=0x7f0f9c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc84ca23b1d64d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0f83fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0f83fff640 1 -- 172.21.15.86:0/1768261021 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c013e60 con 0x55da9a9a8360 2024-04-16T05:37:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0f83fff640 1 -- 172.21.15.86:0/1768261021 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0f9c013410 con 0x55da9a9a8360 2024-04-16T05:37:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fad977740 1 -- 172.21.15.86:0/1768261021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f98046c10 con 0x55da9a9a8360 2024-04-16T05:37:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0f83fff640 1 -- 172.21.15.86:0/1768261021 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f9c040050 con 0x55da9a9a8360 2024-04-16T05:37:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fad977740 10 client.16284 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fad977740 1 -- 172.21.15.86:0/1768261021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55da9aa1e130 con 0x55da9a9a8360 2024-04-16T05:37:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0f83fff640 1 -- 172.21.15.86:0/1768261021 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f98046c10 con 0x55da9a9a8360 2024-04-16T05:37:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0f83fff640 1 -- 172.21.15.86:0/1768261021 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0f9c035070 con 0x55da9a9a8360 2024-04-16T05:37:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fad977740 10 client.16284 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fad977740 10 client.16284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:37:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fad977740 20 client.16284 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fad977740 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55da9aa1e710 0x55da9aa20b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fad977740 4 client.16284 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.691+0000 7f0fad977740 1 -- 172.21.15.86:0/1768261021 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da9a90b970 con 0x55da9aa1e710 2024-04-16T05:37:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fa8a56640 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55da9aa1e710 0x55da9aa20b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0f83fff640 10 client.16284 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:37:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fa8a56640 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55da9aa1e710 0x55da9aa20b30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4bf7542585c0561 server_cookie=e623cc55d7ec8f6a in_seq=0 out_seq=0 2024-04-16T05:37:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0f83fff640 1 -- 172.21.15.86:0/1768261021 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da9a90b970 con 0x55da9aa1e710 2024-04-16T05:37:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0f83fff640 10 client.16284 handle_command_reply: tid=0 2024-04-16T05:37:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fad977740 1 client.16284 shutdown 2024-04-16T05:37:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fad977740 1 -- 172.21.15.86:0/1768261021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 msgr2=0x55da9a9a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fad977740 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9a1a40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009350 tx=0x7f0f9c005bd0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fad977740 1 -- 172.21.15.86:0/1768261021 shutdown_connections 2024-04-16T05:37:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fad977740 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55da9aa1e710 0x55da9aa20b30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fad977740 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da9a9a8360 0x55da9a9a1a40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0f83fff640 0 client.16284 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:37:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fad977740 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da9a99f780 0x55da9a9b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fad977740 1 --2- 172.21.15.86:0/1768261021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9a99ee60 0x55da9a9b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.692+0000 7f0fad977740 1 -- 172.21.15.86:0/1768261021 >> 172.21.15.86:0/1768261021 conn(0x55da9a999590 msgr2=0x55da9a99a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0fad977740 1 -- 172.21.15.86:0/1768261021 shutdown_connections 2024-04-16T05:37:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0fad977740 1 -- 172.21.15.86:0/1768261021 wait complete. 2024-04-16T05:37:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0fad977740 20 client.16284 trim_cache size 0 max 16384 2024-04-16T05:37:01.701 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 -- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0f940a3d20 msgr2=0x7f0f940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0f940a3d20 0x7f0f940a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0f90007590 tx=0x7f0f90087040 comp rx=0 tx=0).stop 2024-04-16T05:37:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 -- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4107320 msgr2=0x7f0fa4074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4107320 0x7f0fa4074840 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000c970 tx=0x7f0f9000ce30 comp rx=0 tx=0).stop 2024-04-16T05:37:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 -- 172.21.15.86:0/2303338276 shutdown_connections 2024-04-16T05:37:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4074d80 0x7f0fa41a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa4107320 0x7f0fa4074840 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0f940a3d20 0x7f0f940a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 --2- 172.21.15.86:0/2303338276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fa4102250 0x7f0fa4074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 -- 172.21.15.86:0/2303338276 >> 172.21.15.86:0/2303338276 conn(0x7f0fa4079490 msgr2=0x7f0fa40fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 -- 172.21.15.86:0/2303338276 shutdown_connections 2024-04-16T05:37:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.693+0000 7f0faacba640 1 -- 172.21.15.86:0/2303338276 wait complete. 2024-04-16T05:37:01.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:37:01.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:01 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/259649782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:01.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.812+0000 7f9725874640 1 -- 172.21.15.86:0/1764236532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103b80 msgr2=0x7f972010c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.812+0000 7f9725874640 1 --2- 172.21.15.86:0/1764236532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103b80 0x7f972010c060 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9714009900 tx=0x7f971402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.812+0000 7f9725874640 1 -- 172.21.15.86:0/1764236532 shutdown_connections 2024-04-16T05:37:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.812+0000 7f9725874640 1 --2- 172.21.15.86:0/1764236532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f972010c630 0x7f972010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.812+0000 7f9725874640 1 --2- 172.21.15.86:0/1764236532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103b80 0x7f972010c060 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.812+0000 7f9725874640 1 --2- 172.21.15.86:0/1764236532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9720103260 0x7f9720103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.812+0000 7f9725874640 1 -- 172.21.15.86:0/1764236532 >> 172.21.15.86:0/1764236532 conn(0x7f9720077280 msgr2=0x7f9720077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.812+0000 7f9725874640 1 -- 172.21.15.86:0/1764236532 shutdown_connections 2024-04-16T05:37:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.812+0000 7f9725874640 1 -- 172.21.15.86:0/1764236532 wait complete. 2024-04-16T05:37:01.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f9725874640 1 Processor -- start 2024-04-16T05:37:01.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f9725874640 1 -- start start 2024-04-16T05:37:01.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f9725874640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103260 0x7f972019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f9725874640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720103b80 0x7f972019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f9725874640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f972010c630 0x7f97201986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f9725874640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97201123c0 con 0x7f9720103260 2024-04-16T05:37:01.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f9725874640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9720112240 con 0x7f972010c630 2024-04-16T05:37:01.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f9725874640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9720112540 con 0x7f9720103b80 2024-04-16T05:37:01.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f971effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103260 0x7f972019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f971effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103260 0x7f972019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51056/0 (socket says 172.21.15.86:51056) 2024-04-16T05:37:01.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f971effd640 1 -- 172.21.15.86:0/1409771724 learned_addr learned my addr 172.21.15.86:0/1409771724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f971e7fc640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720103b80 0x7f972019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.813+0000 7f971f7fe640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f972010c630 0x7f97201986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f971effd640 1 -- 172.21.15.86:0/1409771724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720103b80 msgr2=0x7f972019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f971effd640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720103b80 0x7f972019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f971effd640 1 -- 172.21.15.86:0/1409771724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f972010c630 msgr2=0x7f97201986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f971effd640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f972010c630 0x7f97201986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f971effd640 1 -- 172.21.15.86:0/1409771724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9720198fa0 con 0x7f9720103260 2024-04-16T05:37:01.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f971f7fe640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f972010c630 0x7f97201986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:01.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f971effd640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103260 0x7f972019e620 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f971000eab0 tx=0x7f971000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f9724872640 1 -- 172.21.15.86:0/1409771724 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971000ce40 con 0x7f9720103260 2024-04-16T05:37:01.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f9725874640 1 -- 172.21.15.86:0/1409771724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9720199290 con 0x7f9720103260 2024-04-16T05:37:01.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f9724872640 1 -- 172.21.15.86:0/1409771724 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9710011950 con 0x7f9720103260 2024-04-16T05:37:01.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f9724872640 1 -- 172.21.15.86:0/1409771724 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710010ae0 con 0x7f9720103260 2024-04-16T05:37:01.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.814+0000 7f9725874640 1 -- 172.21.15.86:0/1409771724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97201a55a0 con 0x7f9720103260 2024-04-16T05:37:01.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.816+0000 7f9724872640 1 -- 172.21.15.86:0/1409771724 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9710011b00 con 0x7f9720103260 2024-04-16T05:37:01.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9728467740 1 Processor -- start 2024-04-16T05:37:01.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9728467740 1 -- start start 2024-04-16T05:37:01.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9724872640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97080a3d20 0x7f97080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9728467740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedbe60 0x55e81cedc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9728467740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedc780 0x55e81cee4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9728467740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cee7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9728467740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e81ce410e0 con 0x55e81cedbe60 2024-04-16T05:37:01.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9728467740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e81cdde520 con 0x55e81cedc780 2024-04-16T05:37:01.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9728467740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e81ce3d0a0 con 0x55e81cee5360 2024-04-16T05:37:01.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971f7fe640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97080a3d20 0x7f97080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f9724872640 1 -- 172.21.15.86:0/1409771724 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f97100d1220 con 0x7f9720103260 2024-04-16T05:37:01.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedbe60 0x55e81cedc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedbe60 0x55e81cedc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51072/0 (socket says 172.21.15.86:51072) 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971effd640 1 -- 172.21.15.86:0/493595216 learned_addr learned my addr 172.21.15.86:0/493595216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971e7fc640 1 --2- 172.21.15.86:0/493595216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedc780 0x55e81cee4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971effd640 1 -- 172.21.15.86:0/493595216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 msgr2=0x55e81cee7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971effd640 1 --2- 172.21.15.86:0/493595216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cee7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971effd640 1 -- 172.21.15.86:0/493595216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedc780 msgr2=0x55e81cee4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971effd640 1 --2- 172.21.15.86:0/493595216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedc780 0x55e81cee4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971effd640 1 -- 172.21.15.86:0/493595216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81ce068d0 con 0x55e81cedbe60 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971e7fc640 1 --2- 172.21.15.86:0/493595216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedc780 0x55e81cee4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971f7fe640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97080a3d20 0x7f97080a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9718005940 tx=0x7f971800ce90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.817+0000 7f971effd640 1 --2- 172.21.15.86:0/493595216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedbe60 0x55e81cedc240 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f971001d000 tx=0x7f971009c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ad7053e49e80b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9705ffb640 1 -- 172.21.15.86:0/493595216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971009ce50 con 0x55e81cedbe60 2024-04-16T05:37:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9705ffb640 1 -- 172.21.15.86:0/493595216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f971009f070 con 0x55e81cedbe60 2024-04-16T05:37:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9705ffb640 1 -- 172.21.15.86:0/493595216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710003970 con 0x55e81cedbe60 2024-04-16T05:37:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9728467740 1 -- 172.21.15.86:0/493595216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedbe60 msgr2=0x55e81cedc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9728467740 1 --2- 172.21.15.86:0/493595216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedbe60 0x55e81cedc240 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f971001d000 tx=0x7f971009c870 comp rx=0 tx=0).stop 2024-04-16T05:37:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9728467740 1 -- 172.21.15.86:0/493595216 shutdown_connections 2024-04-16T05:37:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9728467740 1 --2- 172.21.15.86:0/493595216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cee7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9728467740 1 --2- 172.21.15.86:0/493595216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedc780 0x55e81cee4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9728467740 1 --2- 172.21.15.86:0/493595216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedbe60 0x55e81cedc240 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9728467740 1 -- 172.21.15.86:0/493595216 >> 172.21.15.86:0/493595216 conn(0x55e81ced6590 msgr2=0x55e81ced6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9728467740 1 -- 172.21.15.86:0/493595216 shutdown_connections 2024-04-16T05:37:01.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.818+0000 7f9728467740 1 -- 172.21.15.86:0/493595216 wait complete. 2024-04-16T05:37:01.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f9728467740 1 -- start start 2024-04-16T05:37:01.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f9728467740 1 Processor -- start 2024-04-16T05:37:01.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f9728467740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f9728467740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f9728467740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cee2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f9728467740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e81cee8000 con 0x55e81cedc780 2024-04-16T05:37:01.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f9728467740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e81cdfa4d0 con 0x55e81cedbe60 2024-04-16T05:37:01.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f9728467740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e81cee8180 con 0x55e81cee5360 2024-04-16T05:37:01.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f9728467740 10 client.0 _ll_register_callbacks cb 0x55e81cddf2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51086/0 (socket says 172.21.15.86:51086) 2024-04-16T05:37:01.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971e7fc640 1 -- 172.21.15.86:0/2316329210 learned_addr learned my addr 172.21.15.86:0/2316329210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971effd640 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971e7fc640 1 -- 172.21.15.86:0/2316329210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 msgr2=0x55e81cee2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971e7fc640 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cee2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971e7fc640 1 -- 172.21.15.86:0/2316329210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 msgr2=0x55e81cee2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971e7fc640 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971e7fc640 1 -- 172.21.15.86:0/2316329210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e81cb533f0 con 0x55e81cedc780 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.819+0000 7f971effd640 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f971e7fc640 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee2870 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f971402c210 tx=0x7f971402ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818e752659229f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f96feffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f96feffd640 1 -- 172.21.15.86:0/2316329210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714004380 con 0x55e81cedc780 2024-04-16T05:37:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f96feffd640 1 -- 172.21.15.86:0/2316329210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f97140046c0 con 0x55e81cedc780 2024-04-16T05:37:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f9728467740 1 -- 172.21.15.86:0/2316329210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971009f070 con 0x55e81cedc780 2024-04-16T05:37:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f96feffd640 1 -- 172.21.15.86:0/2316329210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97140048c0 con 0x55e81cedc780 2024-04-16T05:37:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f9728467740 10 client.16296 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f9728467740 1 -- 172.21.15.86:0/2316329210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e81cc1d7d0 con 0x55e81cedc780 2024-04-16T05:37:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f96feffd640 1 -- 172.21.15.86:0/2316329210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f971009f070 con 0x55e81cedc780 2024-04-16T05:37:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f96feffd640 1 -- 172.21.15.86:0/2316329210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97140035f0 con 0x55e81cedc780 2024-04-16T05:37:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f9728467740 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f9728467740 10 client.16296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:37:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f9728467740 20 client.16296 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f9728467740 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e81cef52b0 0x55e81cef7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f9728467740 4 client.16296 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:01.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.820+0000 7f9728467740 1 -- 172.21.15.86:0/2316329210 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e81ce4eb00 con 0x55e81cef52b0 2024-04-16T05:37:01.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.821+0000 7f971effd640 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e81cef52b0 0x55e81cef7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.821+0000 7f96feffd640 10 client.16296 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:37:01.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.821+0000 7f971effd640 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e81cef52b0 0x55e81cef7770 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9423ac0d1393e8b server_cookie=372bd3605fd63749 in_seq=0 out_seq=0 2024-04-16T05:37:01.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f96feffd640 1 -- 172.21.15.86:0/2316329210 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e81ce4eb00 con 0x55e81cef52b0 2024-04-16T05:37:01.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f96feffd640 10 client.16296 handle_command_reply: tid=0 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 client.16296 shutdown 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 -- 172.21.15.86:0/2316329210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 msgr2=0x55e81cee2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee2870 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f971402c210 tx=0x7f971402ced0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 -- 172.21.15.86:0/2316329210 shutdown_connections 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e81cef52b0 0x55e81cef7770 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cee2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee2870 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f96feffd640 0 client.16296 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 --2- 172.21.15.86:0/2316329210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 -- 172.21.15.86:0/2316329210 >> 172.21.15.86:0/2316329210 conn(0x55e81ced6590 msgr2=0x55e81ced82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 -- 172.21.15.86:0/2316329210 shutdown_connections 2024-04-16T05:37:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 1 -- 172.21.15.86:0/2316329210 wait complete. 2024-04-16T05:37:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.822+0000 7f9728467740 20 client.16296 trim_cache size 0 max 16384 2024-04-16T05:37:01.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f9728467740 1 Processor -- start 2024-04-16T05:37:01.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f9728467740 1 -- start start 2024-04-16T05:37:01.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f9728467740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f9728467740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81ced15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f9728467740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cefd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f9728467740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97201af4a0 con 0x55e81cedc780 2024-04-16T05:37:01.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f9728467740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97201b0fe0 con 0x55e81cedbe60 2024-04-16T05:37:01.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f9728467740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97201afb70 con 0x55e81cee5360 2024-04-16T05:37:01.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f971e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81ced15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f971e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81ced15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51088/0 (socket says 172.21.15.86:51088) 2024-04-16T05:37:01.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f971e7fc640 1 -- 172.21.15.86:0/1481916308 learned_addr learned my addr 172.21.15.86:0/1481916308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.835+0000 7f971effd640 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f971f7fe640 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cefd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f971e7fc640 1 -- 172.21.15.86:0/1481916308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 msgr2=0x55e81cefd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f971e7fc640 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cefd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f971e7fc640 1 -- 172.21.15.86:0/1481916308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 msgr2=0x55e81cee4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f971e7fc640 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f971e7fc640 1 -- 172.21.15.86:0/1481916308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81cee12e0 con 0x55e81cedc780 2024-04-16T05:37:01.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f971effd640 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:01.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f971f7fe640 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cefd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:01.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f971e7fc640 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81ced15c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f971402c8c0 tx=0x7f971402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c67f9288a8f76a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f97057fa640 1 -- 172.21.15.86:0/1481916308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140037f0 con 0x55e81cedc780 2024-04-16T05:37:01.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f97057fa640 1 -- 172.21.15.86:0/1481916308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9714003c80 con 0x55e81cedc780 2024-04-16T05:37:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f97057fa640 1 -- 172.21.15.86:0/1481916308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714004180 con 0x55e81cedc780 2024-04-16T05:37:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f9728467740 1 -- 172.21.15.86:0/1481916308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 msgr2=0x55e81ced15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f9728467740 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81ced15c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f971402c8c0 tx=0x7f971402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f9728467740 1 -- 172.21.15.86:0/1481916308 shutdown_connections 2024-04-16T05:37:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f9728467740 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cee5360 0x55e81cefd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.836+0000 7f9728467740 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81ced15c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 --2- 172.21.15.86:0/1481916308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cedbe60 0x55e81cee4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 -- 172.21.15.86:0/1481916308 >> 172.21.15.86:0/1481916308 conn(0x55e81ced6590 msgr2=0x55e81ced5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 -- 172.21.15.86:0/1481916308 shutdown_connections 2024-04-16T05:37:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 -- 172.21.15.86:0/1481916308 wait complete. 2024-04-16T05:37:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 -- start start 2024-04-16T05:37:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 Processor -- start 2024-04-16T05:37:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cedbe60 0x55e81cee8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cee5360 0x55e81cee9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97201b0910 con 0x55e81cedc780 2024-04-16T05:37:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97201b0240 con 0x55e81cee5360 2024-04-16T05:37:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97201ad460 con 0x55e81cedbe60 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f9728467740 10 client.0 _ll_register_callbacks cb 0x7f971402cd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f971e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f971e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51094/0 (socket says 172.21.15.86:51094) 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.837+0000 7f971e7fc640 1 -- 172.21.15.86:0/1753121629 learned_addr learned my addr 172.21.15.86:0/1753121629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f971effd640 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cedbe60 0x55e81cee8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f971f7fe640 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cee5360 0x55e81cee9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f971e7fc640 1 -- 172.21.15.86:0/1753121629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cedbe60 msgr2=0x55e81cee8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f971e7fc640 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cedbe60 0x55e81cee8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f971e7fc640 1 -- 172.21.15.86:0/1753121629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cee5360 msgr2=0x55e81cee9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f971e7fc640 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cee5360 0x55e81cee9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f971e7fc640 1 -- 172.21.15.86:0/1753121629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e81cf5ed00 con 0x55e81cedc780 2024-04-16T05:37:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f971f7fe640 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cee5360 0x55e81cee9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f971e7fc640 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee93c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9714009850 tx=0x7f971402cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fa918d12e991f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f96feffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f96feffd640 1 -- 172.21.15.86:0/1753121629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971405a040 con 0x55e81cedc780 2024-04-16T05:37:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f9728467740 1 -- 172.21.15.86:0/1753121629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9714003c80 con 0x55e81cedc780 2024-04-16T05:37:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f96feffd640 1 -- 172.21.15.86:0/1753121629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9714004a60 con 0x55e81cedc780 2024-04-16T05:37:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f96feffd640 1 -- 172.21.15.86:0/1753121629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9714004ca0 con 0x55e81cedc780 2024-04-16T05:37:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f9728467740 10 client.16302 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f9728467740 1 -- 172.21.15.86:0/1753121629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e81cf5ff10 con 0x55e81cedc780 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.838+0000 7f96feffd640 1 -- 172.21.15.86:0/1753121629 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9714003c80 con 0x55e81cedc780 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f96feffd640 1 -- 172.21.15.86:0/1753121629 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f971405d030 con 0x55e81cedc780 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f9728467740 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f9728467740 10 client.16302 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f9728467740 20 client.16302 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f9728467740 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e81cf603f0 0x55e81cf62810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f9728467740 4 client.16302 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f9728467740 1 -- 172.21.15.86:0/1753121629 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e81ce48970 con 0x55e81cf603f0 2024-04-16T05:37:01.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f971effd640 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e81cf603f0 0x55e81cf62810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f96feffd640 10 client.16302 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:37:01.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f971effd640 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e81cf603f0 0x55e81cf62810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcb53cdfca8d0782 server_cookie=ca8e3951547464ac in_seq=0 out_seq=0 2024-04-16T05:37:01.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f96feffd640 1 -- 172.21.15.86:0/1753121629 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e81ce48970 con 0x55e81cf603f0 2024-04-16T05:37:01.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f96feffd640 10 client.16302 handle_command_reply: tid=0 2024-04-16T05:37:01.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.839+0000 7f9728467740 1 client.16302 shutdown 2024-04-16T05:37:01.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 -- 172.21.15.86:0/1753121629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 msgr2=0x55e81cee93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee93c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9714009850 tx=0x7f971402cf90 comp rx=0 tx=0).stop 2024-04-16T05:37:01.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 -- 172.21.15.86:0/1753121629 shutdown_connections 2024-04-16T05:37:01.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e81cf603f0 0x55e81cf62810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e81cee5360 0x55e81cee9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81cedc780 0x55e81cee93c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f96feffd640 0 client.16302 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:37:01.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 --2- 172.21.15.86:0/1753121629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e81cedbe60 0x55e81cee8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 -- 172.21.15.86:0/1753121629 >> 172.21.15.86:0/1753121629 conn(0x55e81ced6590 msgr2=0x55e81cee22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 -- 172.21.15.86:0/1753121629 shutdown_connections 2024-04-16T05:37:01.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 1 -- 172.21.15.86:0/1753121629 wait complete. 2024-04-16T05:37:01.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9728467740 20 client.16302 trim_cache size 0 max 16384 2024-04-16T05:37:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:01.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 -- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97080a3d20 msgr2=0x7f97080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97080a3d20 0x7f97080a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9718005940 tx=0x7f971800ce90 comp rx=0 tx=0).stop 2024-04-16T05:37:01.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 -- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103260 msgr2=0x7f972019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103260 0x7f972019e620 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f971000eab0 tx=0x7f971000ef70 comp rx=0 tx=0).stop 2024-04-16T05:37:01.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 -- 172.21.15.86:0/1409771724 shutdown_connections 2024-04-16T05:37:01.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97080a3d20 0x7f97080a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f972010c630 0x7f97201986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720103b80 0x7f972019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 --2- 172.21.15.86:0/1409771724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720103260 0x7f972019e620 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 -- 172.21.15.86:0/1409771724 >> 172.21.15.86:0/1409771724 conn(0x7f9720077280 msgr2=0x7f97200fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 -- 172.21.15.86:0/1409771724 shutdown_connections 2024-04-16T05:37:01.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.840+0000 7f9725874640 1 -- 172.21.15.86:0/1409771724 wait complete. 2024-04-16T05:37:01.851 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:37:01.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:01 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/259649782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:01.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 -- 172.21.15.86:0/3591805302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 msgr2=0x7f72001011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 --2- 172.21.15.86:0/3591805302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 0x7f72001011d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009ca0 tx=0x7f71f002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 -- 172.21.15.86:0/3591805302 shutdown_connections 2024-04-16T05:37:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 --2- 172.21.15.86:0/3591805302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720010e9a0 0x7f7200110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 --2- 172.21.15.86:0/3591805302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7200101710 0x7f720010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 --2- 172.21.15.86:0/3591805302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 0x7f72001011d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 -- 172.21.15.86:0/3591805302 >> 172.21.15.86:0/3591805302 conn(0x7f72000780b0 msgr2=0x7f72000784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 -- 172.21.15.86:0/3591805302 shutdown_connections 2024-04-16T05:37:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 -- 172.21.15.86:0/3591805302 wait complete. 2024-04-16T05:37:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 Processor -- start 2024-04-16T05:37:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.959+0000 7f7207888640 1 -- start start 2024-04-16T05:37:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7207888640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 0x7f72001a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7207888640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7200101710 0x7f72001a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7207888640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720010e9a0 0x7f720019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7207888640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72001145c0 con 0x7f7200100df0 2024-04-16T05:37:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7207888640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7200114440 con 0x7f7200101710 2024-04-16T05:37:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7207888640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7200114740 con 0x7f720010e9a0 2024-04-16T05:37:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7205624640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 0x7f72001a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7205624640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 0x7f72001a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51106/0 (socket says 172.21.15.86:51106) 2024-04-16T05:37:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7205624640 1 -- 172.21.15.86:0/2604302077 learned_addr learned my addr 172.21.15.86:0/2604302077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7204e23640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7200101710 0x7f72001a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7205624640 1 -- 172.21.15.86:0/2604302077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720010e9a0 msgr2=0x7f720019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7205624640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720010e9a0 0x7f720019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7205624640 1 -- 172.21.15.86:0/2604302077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7200101710 msgr2=0x7f72001a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7205624640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7200101710 0x7f72001a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7205624640 1 -- 172.21.15.86:0/2604302077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720019b000 con 0x7f7200100df0 2024-04-16T05:37:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7204e23640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7200101710 0x7f72001a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7205624640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 0x7f72001a06d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f71f0005e30 tx=0x7f71f0034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f71fe7fc640 1 -- 172.21.15.86:0/2604302077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0039040 con 0x7f7200100df0 2024-04-16T05:37:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f71fe7fc640 1 -- 172.21.15.86:0/2604302077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0044070 con 0x7f7200100df0 2024-04-16T05:37:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7207888640 1 -- 172.21.15.86:0/2604302077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720019b290 con 0x7f7200100df0 2024-04-16T05:37:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f71fe7fc640 1 -- 172.21.15.86:0/2604302077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00042a0 con 0x7f7200100df0 2024-04-16T05:37:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.960+0000 7f7207888640 1 -- 172.21.15.86:0/2604302077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72001a7590 con 0x7f7200100df0 2024-04-16T05:37:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.962+0000 7f71fe7fc640 1 -- 172.21.15.86:0/2604302077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71f0003970 con 0x7f7200100df0 2024-04-16T05:37:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f720a475740 1 Processor -- start 2024-04-16T05:37:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f720a475740 1 -- start start 2024-04-16T05:37:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f720a475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 0x557b5bac4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f720a475740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bacce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f720a475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bacd360 0x557b5bacf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f720a475740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b5ba290e0 con 0x557b5bac4780 2024-04-16T05:37:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f720a475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557b5b9c6520 con 0x557b5bac3e60 2024-04-16T05:37:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f720a475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b5ba250a0 con 0x557b5bacd360 2024-04-16T05:37:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f71fe7fc640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f71e80a3d20 0x7f71e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f7205624640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bacce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f7205624640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bacce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51112/0 (socket says 172.21.15.86:51112) 2024-04-16T05:37:01.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f7205624640 1 -- 172.21.15.86:0/4287372635 learned_addr learned my addr 172.21.15.86:0/4287372635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f7205e25640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f71e80a3d20 0x7f71e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.963+0000 7f7205e25640 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 0x557b5bac4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f7204e23640 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bacd360 0x557b5bacf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f71fe7fc640 1 -- 172.21.15.86:0/2604302077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f71f00f2e00 con 0x7f7200100df0 2024-04-16T05:37:01.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f7205624640 1 -- 172.21.15.86:0/4287372635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bacd360 msgr2=0x557b5bacf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f7205624640 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bacd360 0x557b5bacf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f7205624640 1 -- 172.21.15.86:0/4287372635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 msgr2=0x557b5bac4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f7205624640 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 0x557b5bac4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f7205624640 1 -- 172.21.15.86:0/4287372635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b5b9ee8d0 con 0x557b5bac4780 2024-04-16T05:37:01.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f7204e23640 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bacd360 0x557b5bacf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:01.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f7205e25640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f71e80a3d20 0x7f71e80a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71f800dfd0 tx=0x7f71f800d8c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f7205624640 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bacce20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f71f0000f80 tx=0x7f71f0005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71713255b7ad466 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f71e77fe640 1 -- 172.21.15.86:0/4287372635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00d80b0 con 0x557b5bac4780 2024-04-16T05:37:01.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f71e77fe640 1 -- 172.21.15.86:0/4287372635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f00bf240 con 0x557b5bac4780 2024-04-16T05:37:01.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.964+0000 7f71e77fe640 1 -- 172.21.15.86:0/4287372635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00bcf10 con 0x557b5bac4780 2024-04-16T05:37:01.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 -- 172.21.15.86:0/4287372635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 msgr2=0x557b5bacce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bacce20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f71f0000f80 tx=0x7f71f0005c30 comp rx=0 tx=0).stop 2024-04-16T05:37:01.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 -- 172.21.15.86:0/4287372635 shutdown_connections 2024-04-16T05:37:01.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bacd360 0x557b5bacf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bacce20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 --2- 172.21.15.86:0/4287372635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 0x557b5bac4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 -- 172.21.15.86:0/4287372635 >> 172.21.15.86:0/4287372635 conn(0x557b5babe590 msgr2=0x557b5babe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 -- 172.21.15.86:0/4287372635 shutdown_connections 2024-04-16T05:37:01.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 -- 172.21.15.86:0/4287372635 wait complete. 2024-04-16T05:37:01.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 -- start start 2024-04-16T05:37:01.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.965+0000 7f720a475740 1 Processor -- start 2024-04-16T05:37:01.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f720a475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 0x557b5baca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f720a475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 0x557b5baca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f720a475740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bacd360 0x557b5bacae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f720a475740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b5bacff10 con 0x557b5bacd360 2024-04-16T05:37:01.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f720a475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557b5b9e24d0 con 0x557b5bac3e60 2024-04-16T05:37:01.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f720a475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b5bad0090 con 0x557b5bac4780 2024-04-16T05:37:01.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f720a475740 10 client.0 _ll_register_callbacks cb 0x557b5b9c72c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205e25640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bacd360 0x557b5bacae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205e25640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bacd360 0x557b5bacae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51118/0 (socket says 172.21.15.86:51118) 2024-04-16T05:37:01.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205e25640 1 -- 172.21.15.86:0/2517022053 learned_addr learned my addr 172.21.15.86:0/2517022053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205624640 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 0x557b5baca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7204e23640 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 0x557b5baca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205e25640 1 -- 172.21.15.86:0/2517022053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 msgr2=0x557b5baca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205e25640 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 0x557b5baca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205e25640 1 -- 172.21.15.86:0/2517022053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 msgr2=0x557b5baca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205e25640 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 0x557b5baca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205e25640 1 -- 172.21.15.86:0/2517022053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b5b73b3f0 con 0x557b5bacd360 2024-04-16T05:37:01.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205624640 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 0x557b5baca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:01.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7204e23640 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 0x557b5baca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:01.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f7205e25640 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bacd360 0x557b5bacae80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f71f8004360 tx=0x7f71f8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396c302a75ec69fc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f71e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f71e4ff9640 1 -- 172.21.15.86:0/2517022053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8005020 con 0x557b5bacd360 2024-04-16T05:37:01.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f71e4ff9640 1 -- 172.21.15.86:0/2517022053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f71f80051c0 con 0x557b5bacd360 2024-04-16T05:37:01.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.966+0000 7f720a475740 1 -- 172.21.15.86:0/2517022053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f00bf240 con 0x557b5bacd360 2024-04-16T05:37:01.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f71e4ff9640 1 -- 172.21.15.86:0/2517022053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71f802f050 con 0x557b5bacd360 2024-04-16T05:37:01.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f720a475740 10 client.16314 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f720a475740 1 -- 172.21.15.86:0/2517022053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557b5b8057d0 con 0x557b5bacd360 2024-04-16T05:37:01.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f71e4ff9640 1 -- 172.21.15.86:0/2517022053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f00bf240 con 0x557b5bacd360 2024-04-16T05:37:01.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f71e4ff9640 1 -- 172.21.15.86:0/2517022053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f71f800f580 con 0x557b5bacd360 2024-04-16T05:37:01.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f720a475740 10 client.16314 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f720a475740 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:01.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f720a475740 20 client.16314 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f720a475740 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557b5badcfe0 0x557b5badf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f720a475740 4 client.16314 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:01.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f720a475740 1 -- 172.21.15.86:0/2517022053 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b5ba36b00 con 0x557b5badcfe0 2024-04-16T05:37:01.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.967+0000 7f7205624640 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557b5badcfe0 0x557b5badf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.968+0000 7f71e4ff9640 10 client.16314 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:01.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.968+0000 7f7205624640 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557b5badcfe0 0x557b5badf440 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce474cc21d08308 server_cookie=939712a9783a846e in_seq=0 out_seq=0 2024-04-16T05:37:01.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.968+0000 7f71e4ff9640 1 -- 172.21.15.86:0/2517022053 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b5ba36b00 con 0x557b5badcfe0 2024-04-16T05:37:01.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.968+0000 7f71e4ff9640 10 client.16314 handle_command_reply: tid=0 2024-04-16T05:37:01.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.968+0000 7f720a475740 1 client.16314 shutdown 2024-04-16T05:37:01.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 -- 172.21.15.86:0/2517022053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bacd360 msgr2=0x557b5bacae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bacd360 0x557b5bacae80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f71f8004360 tx=0x7f71f8005850 comp rx=0 tx=0).stop 2024-04-16T05:37:01.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 -- 172.21.15.86:0/2517022053 shutdown_connections 2024-04-16T05:37:01.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557b5badcfe0 0x557b5badf440 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bacd360 0x557b5bacae80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 0x557b5baca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f71e4ff9640 0 client.16314 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:01.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 --2- 172.21.15.86:0/2517022053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bac3e60 0x557b5baca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 -- 172.21.15.86:0/2517022053 >> 172.21.15.86:0/2517022053 conn(0x557b5babe590 msgr2=0x557b5bac02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 -- 172.21.15.86:0/2517022053 shutdown_connections 2024-04-16T05:37:01.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 1 -- 172.21.15.86:0/2517022053 wait complete. 2024-04-16T05:37:01.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.969+0000 7f720a475740 20 client.16314 trim_cache size 0 max 16384 2024-04-16T05:37:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f720a475740 1 Processor -- start 2024-04-16T05:37:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f720a475740 1 -- start start 2024-04-16T05:37:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f720a475740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac3e60 0x557b5bad8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f720a475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 0x557b5bac9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f720a475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bacd360 0x557b5bae4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f720a475740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b16b0 con 0x557b5bac3e60 2024-04-16T05:37:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f720a475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b31f0 con 0x557b5bacd360 2024-04-16T05:37:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f720a475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b1d80 con 0x557b5bac4780 2024-04-16T05:37:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f7205624640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac3e60 0x557b5bad8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f7205624640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac3e60 0x557b5bad8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51130/0 (socket says 172.21.15.86:51130) 2024-04-16T05:37:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f7205624640 1 -- 172.21.15.86:0/37388678 learned_addr learned my addr 172.21.15.86:0/37388678 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f7205624640 1 -- 172.21.15.86:0/37388678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 msgr2=0x557b5bac9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f7205624640 1 --2- 172.21.15.86:0/37388678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 0x557b5bac9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f7205624640 1 -- 172.21.15.86:0/37388678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bacd360 msgr2=0x557b5bae4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.982+0000 7f7205624640 1 --2- 172.21.15.86:0/37388678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bacd360 0x557b5bae4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f7205624640 1 -- 172.21.15.86:0/37388678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b5bac8950 con 0x557b5bac3e60 2024-04-16T05:37:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f7205624640 1 --2- 172.21.15.86:0/37388678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac3e60 0x557b5bad8d00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f71f0005cc0 tx=0x7f71f0003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaa7a2adbce0eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f71e6ffd640 1 -- 172.21.15.86:0/37388678 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f003e3a0 con 0x557b5bac3e60 2024-04-16T05:37:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f71e6ffd640 1 -- 172.21.15.86:0/37388678 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f00bf240 con 0x557b5bac3e60 2024-04-16T05:37:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f71e6ffd640 1 -- 172.21.15.86:0/37388678 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00beda0 con 0x557b5bac3e60 2024-04-16T05:37:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f720a475740 1 -- 172.21.15.86:0/37388678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac3e60 msgr2=0x557b5bad8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f720a475740 1 --2- 172.21.15.86:0/37388678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac3e60 0x557b5bad8d00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f71f0005cc0 tx=0x7f71f0003ca0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f720a475740 1 -- 172.21.15.86:0/37388678 shutdown_connections 2024-04-16T05:37:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f720a475740 1 --2- 172.21.15.86:0/37388678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bacd360 0x557b5bae4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f720a475740 1 --2- 172.21.15.86:0/37388678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac4780 0x557b5bac9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f720a475740 1 --2- 172.21.15.86:0/37388678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac3e60 0x557b5bad8d00 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f720a475740 1 -- 172.21.15.86:0/37388678 >> 172.21.15.86:0/37388678 conn(0x557b5babe590 msgr2=0x557b5bab95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f720a475740 1 -- 172.21.15.86:0/37388678 shutdown_connections 2024-04-16T05:37:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.983+0000 7f720a475740 1 -- 172.21.15.86:0/37388678 wait complete. 2024-04-16T05:37:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f720a475740 1 -- start start 2024-04-16T05:37:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f720a475740 1 Processor -- start 2024-04-16T05:37:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f720a475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac3e60 0x557b5bad0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f720a475740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bad10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f720a475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bacd360 0x557b5bad1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f720a475740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b2b20 con 0x557b5bac4780 2024-04-16T05:37:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f720a475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b2450 con 0x557b5bacd360 2024-04-16T05:37:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f720a475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72001af670 con 0x557b5bac3e60 2024-04-16T05:37:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f720a475740 10 client.0 _ll_register_callbacks cb 0x7f71f800d370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f7204e23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bad10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f7204e23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bad10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51146/0 (socket says 172.21.15.86:51146) 2024-04-16T05:37:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f7204e23640 1 -- 172.21.15.86:0/614585031 learned_addr learned my addr 172.21.15.86:0/614585031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f7205624640 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac3e60 0x557b5bad0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.984+0000 7f7205e25640 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bacd360 0x557b5bad1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f7204e23640 1 -- 172.21.15.86:0/614585031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac3e60 msgr2=0x557b5bad0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f7204e23640 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac3e60 0x557b5bad0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f7204e23640 1 -- 172.21.15.86:0/614585031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bacd360 msgr2=0x557b5bad1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f7204e23640 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bacd360 0x557b5bad1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f7204e23640 1 -- 172.21.15.86:0/614585031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b5bb42580 con 0x557b5bac4780 2024-04-16T05:37:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f7204e23640 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bad10d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f720019c070 tx=0x7f71f400f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f226b213cc694c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f71e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f71e4ff9640 1 -- 172.21.15.86:0/614585031 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f401c060 con 0x557b5bac4780 2024-04-16T05:37:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f71e4ff9640 1 -- 172.21.15.86:0/614585031 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f71f400b790 con 0x557b5bac4780 2024-04-16T05:37:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f720a475740 1 -- 172.21.15.86:0/614585031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f00bf240 con 0x557b5bac4780 2024-04-16T05:37:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f71e4ff9640 1 -- 172.21.15.86:0/614585031 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71f4023050 con 0x557b5bac4780 2024-04-16T05:37:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f720a475740 10 client.16320 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f720a475740 1 -- 172.21.15.86:0/614585031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557b5bb43850 con 0x557b5bac4780 2024-04-16T05:37:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f71e4ff9640 1 -- 172.21.15.86:0/614585031 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f00bf240 con 0x557b5bac4780 2024-04-16T05:37:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f71e4ff9640 1 -- 172.21.15.86:0/614585031 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f71f4017b30 con 0x557b5bac4780 2024-04-16T05:37:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f720a475740 10 client.16320 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f720a475740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.985+0000 7f720a475740 20 client.16320 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557b5bb44030 0x557b5bb46450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 4 client.16320 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 -- 172.21.15.86:0/614585031 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b5ba30970 con 0x557b5bb44030 2024-04-16T05:37:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f7205624640 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557b5bb44030 0x557b5bb46450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f7205624640 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557b5bb44030 0x557b5bb46450 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2febf6d999875bd3 server_cookie=c4cb27eb582a2547 in_seq=0 out_seq=0 2024-04-16T05:37:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f71e4ff9640 10 client.16320 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f71e4ff9640 1 -- 172.21.15.86:0/614585031 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b5ba30970 con 0x557b5bb44030 2024-04-16T05:37:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f71e4ff9640 10 client.16320 handle_command_reply: tid=0 2024-04-16T05:37:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 client.16320 shutdown 2024-04-16T05:37:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 -- 172.21.15.86:0/614585031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 msgr2=0x557b5bad10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bad10d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f720019c070 tx=0x7f71f400f7e0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 -- 172.21.15.86:0/614585031 shutdown_connections 2024-04-16T05:37:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557b5bb44030 0x557b5bb46450 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b5bacd360 0x557b5bad1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b5bac4780 0x557b5bad10d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f71e4ff9640 0 client.16320 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 --2- 172.21.15.86:0/614585031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5bac3e60 0x557b5bad0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.986+0000 7f720a475740 1 -- 172.21.15.86:0/614585031 >> 172.21.15.86:0/614585031 conn(0x557b5babe590 msgr2=0x557b5babcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f720a475740 1 -- 172.21.15.86:0/614585031 shutdown_connections 2024-04-16T05:37:01.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f720a475740 1 -- 172.21.15.86:0/614585031 wait complete. 2024-04-16T05:37:01.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f720a475740 20 client.16320 trim_cache size 0 max 16384 2024-04-16T05:37:01.997 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:01.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 -- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f71e80a3d20 msgr2=0x7f71e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f71e80a3d20 0x7f71e80a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71f800dfd0 tx=0x7f71f800d8c0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 -- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 msgr2=0x7f72001a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:01.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 0x7f72001a06d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f71f0005e30 tx=0x7f71f0034760 comp rx=0 tx=0).stop 2024-04-16T05:37:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 -- 172.21.15.86:0/2604302077 shutdown_connections 2024-04-16T05:37:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720010e9a0 0x7f720019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f71e80a3d20 0x7f71e80a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7200101710 0x7f72001a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 --2- 172.21.15.86:0/2604302077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200100df0 0x7f72001a06d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 -- 172.21.15.86:0/2604302077 >> 172.21.15.86:0/2604302077 conn(0x7f72000780b0 msgr2=0x7f720010fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 -- 172.21.15.86:0/2604302077 shutdown_connections 2024-04-16T05:37:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:01.987+0000 7f7207888640 1 -- 172.21.15.86:0/2604302077 wait complete. 2024-04-16T05:37:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:02 smithi086 ceph-mon[32028]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-16T05:37:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:02 smithi196 ceph-mon[36373]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-16T05:37:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:02 smithi194 ceph-mon[35865]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-16T05:37:03.000 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:37:03.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:37:03.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:37:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.123+0000 7f490947e640 1 -- 172.21.15.86:0/2237140395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4904102240 msgr2=0x7f4904102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.123+0000 7f490947e640 1 --2- 172.21.15.86:0/2237140395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4904102240 0x7f4904102620 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f402bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.123+0000 7f490947e640 1 -- 172.21.15.86:0/2237140395 shutdown_connections 2024-04-16T05:37:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.123+0000 7f490947e640 1 --2- 172.21.15.86:0/2237140395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4904107310 0x7f490410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.123+0000 7f490947e640 1 --2- 172.21.15.86:0/2237140395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4904102bf0 0x7f4904106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.123+0000 7f490947e640 1 --2- 172.21.15.86:0/2237140395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4904102240 0x7f4904102620 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.123+0000 7f490947e640 1 -- 172.21.15.86:0/2237140395 >> 172.21.15.86:0/2237140395 conn(0x7f4904079450 msgr2=0x7f4904079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.123+0000 7f490947e640 1 -- 172.21.15.86:0/2237140395 shutdown_connections 2024-04-16T05:37:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.123+0000 7f490947e640 1 -- 172.21.15.86:0/2237140395 wait complete. 2024-04-16T05:37:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f490947e640 1 Processor -- start 2024-04-16T05:37:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f490947e640 1 -- start start 2024-04-16T05:37:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f490947e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4904102240 0x7f490419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f490947e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4904107310 0x7f490419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f490947e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49041987d0 0x7f4904198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f490947e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4904112470 con 0x7f49041987d0 2024-04-16T05:37:03.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f490947e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f49041122f0 con 0x7f4904102240 2024-04-16T05:37:03.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f490947e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49041125f0 con 0x7f4904107310 2024-04-16T05:37:03.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f49037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49041987d0 0x7f4904198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f49037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49041987d0 0x7f4904198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51164/0 (socket says 172.21.15.86:51164) 2024-04-16T05:37:03.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f49037fe640 1 -- 172.21.15.86:0/1378844004 learned_addr learned my addr 172.21.15.86:0/1378844004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f49037fe640 1 -- 172.21.15.86:0/1378844004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4904107310 msgr2=0x7f490419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f49037fe640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4904107310 0x7f490419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f49037fe640 1 -- 172.21.15.86:0/1378844004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4904102240 msgr2=0x7f490419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f49037fe640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4904102240 0x7f490419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.124+0000 7f49037fe640 1 -- 172.21.15.86:0/1378844004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4904199360 con 0x7f49041987d0 2024-04-16T05:37:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.125+0000 7f49037fe640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49041987d0 0x7f4904198c80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00b970 tx=0x7f48fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.125+0000 7f4900ff9640 1 -- 172.21.15.86:0/1378844004 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc019070 con 0x7f49041987d0 2024-04-16T05:37:03.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.125+0000 7f4900ff9640 1 -- 172.21.15.86:0/1378844004 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc00cb40 con 0x7f49041987d0 2024-04-16T05:37:03.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.125+0000 7f490947e640 1 -- 172.21.15.86:0/1378844004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4904199650 con 0x7f49041987d0 2024-04-16T05:37:03.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.125+0000 7f4900ff9640 1 -- 172.21.15.86:0/1378844004 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc007650 con 0x7f49041987d0 2024-04-16T05:37:03.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.125+0000 7f490947e640 1 -- 172.21.15.86:0/1378844004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49041a57e0 con 0x7f49041987d0 2024-04-16T05:37:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.128+0000 7f490c05f740 1 Processor -- start 2024-04-16T05:37:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.128+0000 7f4900ff9640 1 -- 172.21.15.86:0/1378844004 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f48fc01c070 con 0x7f49041987d0 2024-04-16T05:37:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.128+0000 7f490c05f740 1 -- start start 2024-04-16T05:37:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.128+0000 7f490c05f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 0x5616ffd0ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.128+0000 7f490c05f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd15a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.128+0000 7f490c05f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f490c05f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616ffc71e90 con 0x5616ffd0d3d0 2024-04-16T05:37:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f490c05f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5616ffc0f4c0 con 0x5616ffd0cab0 2024-04-16T05:37:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f490c05f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5616ffc6de50 con 0x5616ffd15fb0 2024-04-16T05:37:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4902ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd15a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4902ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd15a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51180/0 (socket says 172.21.15.86:51180) 2024-04-16T05:37:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4902ffd640 1 -- 172.21.15.86:0/3047191031 learned_addr learned my addr 172.21.15.86:0/3047191031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f48fbfff640 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f48fbfff640 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 0x5616ffd0ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4902ffd640 1 -- 172.21.15.86:0/3047191031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 msgr2=0x5616ffd183a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4902ffd640 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd183a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4902ffd640 1 -- 172.21.15.86:0/3047191031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 msgr2=0x5616ffd0ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4902ffd640 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 0x5616ffd0ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4902ffd640 1 -- 172.21.15.86:0/3047191031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ffa4c7e0 con 0x5616ffd0d3d0 2024-04-16T05:37:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4900ff9640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48f00a3da0 0x7f48f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f48fbfff640 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4902ffd640 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd15a70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f48f40057c0 tx=0x7f48f4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417f0ba0f1adc7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f48fbfff640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48f00a3da0 0x7f48f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f4900ff9640 1 -- 172.21.15.86:0/1378844004 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f48fc0d0060 con 0x7f49041987d0 2024-04-16T05:37:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f48f97fa640 1 -- 172.21.15.86:0/3047191031 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4042020 con 0x5616ffd0d3d0 2024-04-16T05:37:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f48f97fa640 1 -- 172.21.15.86:0/3047191031 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f402c880 con 0x5616ffd0d3d0 2024-04-16T05:37:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.129+0000 7f48f97fa640 1 -- 172.21.15.86:0/3047191031 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4003af0 con 0x5616ffd0d3d0 2024-04-16T05:37:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f490c05f740 1 -- 172.21.15.86:0/3047191031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 msgr2=0x5616ffd15a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f48fbfff640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48f00a3da0 0x7f48f00a6260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5616ffd08370 tx=0x7f48ec0046c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f490c05f740 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd15a70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f48f40057c0 tx=0x7f48f4005d50 comp rx=0 tx=0).stop 2024-04-16T05:37:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f490c05f740 1 -- 172.21.15.86:0/3047191031 shutdown_connections 2024-04-16T05:37:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f490c05f740 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f490c05f740 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd15a70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f490c05f740 1 --2- 172.21.15.86:0/3047191031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 0x5616ffd0ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f490c05f740 1 -- 172.21.15.86:0/3047191031 >> 172.21.15.86:0/3047191031 conn(0x5616ffd071e0 msgr2=0x5616ffd075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f490c05f740 1 -- 172.21.15.86:0/3047191031 shutdown_connections 2024-04-16T05:37:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.130+0000 7f490c05f740 1 -- 172.21.15.86:0/3047191031 wait complete. 2024-04-16T05:37:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f490c05f740 1 -- start start 2024-04-16T05:37:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f490c05f740 1 Processor -- start 2024-04-16T05:37:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f490c05f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0cab0 0x5616ffd12ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f490c05f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0d3d0 0x5616ffd13530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f490c05f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd13a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f490c05f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616ffd18d40 con 0x5616ffd0cab0 2024-04-16T05:37:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f490c05f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5616ffbfa410 con 0x5616ffd0d3d0 2024-04-16T05:37:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f490c05f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5616ffd18ec0 con 0x5616ffd15fb0 2024-04-16T05:37:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f4902ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0cab0 0x5616ffd12ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f490c05f740 10 client.0 _ll_register_callbacks cb 0x5616ffbf5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f4902ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0cab0 0x5616ffd12ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51182/0 (socket says 172.21.15.86:51182) 2024-04-16T05:37:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f4902ffd640 1 -- 172.21.15.86:0/2555711648 learned_addr learned my addr 172.21.15.86:0/2555711648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f48fbfff640 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0d3d0 0x5616ffd13530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f49037fe640 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd13a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f4902ffd640 1 -- 172.21.15.86:0/2555711648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 msgr2=0x5616ffd13a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f4902ffd640 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd13a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f4902ffd640 1 -- 172.21.15.86:0/2555711648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0d3d0 msgr2=0x5616ffd13530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f4902ffd640 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0d3d0 0x5616ffd13530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f4902ffd640 1 -- 172.21.15.86:0/2555711648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ffaacf70 con 0x5616ffd0cab0 2024-04-16T05:37:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f49037fe640 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd13a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f4902ffd640 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0cab0 0x5616ffd12ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f48f40042a0 tx=0x7f48f40042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54fc4613a8f6fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.131+0000 7f48ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f48ea7fc640 1 -- 172.21.15.86:0/2555711648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4003a40 con 0x5616ffd0cab0 2024-04-16T05:37:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f490c05f740 1 -- 172.21.15.86:0/2555711648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f402c880 con 0x5616ffd0cab0 2024-04-16T05:37:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f48ea7fc640 1 -- 172.21.15.86:0/2555711648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f48f4003be0 con 0x5616ffd0cab0 2024-04-16T05:37:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f48ea7fc640 1 -- 172.21.15.86:0/2555711648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f48f4003de0 con 0x5616ffd0cab0 2024-04-16T05:37:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f490c05f740 10 client.16332 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f490c05f740 1 -- 172.21.15.86:0/2555711648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5616ff944170 con 0x5616ffd0cab0 2024-04-16T05:37:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f48ea7fc640 1 -- 172.21.15.86:0/2555711648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f402c880 con 0x5616ffd0cab0 2024-04-16T05:37:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f48ea7fc640 1 -- 172.21.15.86:0/2555711648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f48f4047440 con 0x5616ffd0cab0 2024-04-16T05:37:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f490c05f740 10 client.16332 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f490c05f740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f490c05f740 20 client.16332 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f490c05f740 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5616ffd26160 0x5616ffd28550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f490c05f740 4 client.16332 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.132+0000 7f490c05f740 1 -- 172.21.15.86:0/2555711648 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616ffc7f8b0 con 0x5616ffd26160 2024-04-16T05:37:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.133+0000 7f48fbfff640 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5616ffd26160 0x5616ffd28550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.133+0000 7f48ea7fc640 10 client.16332 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.133+0000 7f48fbfff640 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5616ffd26160 0x5616ffd28550 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ac14c2a85f373b server_cookie=304cdc954b18f88f in_seq=0 out_seq=0 2024-04-16T05:37:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f48ea7fc640 1 -- 172.21.15.86:0/2555711648 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616ffc7f8b0 con 0x5616ffd26160 2024-04-16T05:37:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f48ea7fc640 10 client.16332 handle_command_reply: tid=0 2024-04-16T05:37:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 client.16332 shutdown 2024-04-16T05:37:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 -- 172.21.15.86:0/2555711648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0cab0 msgr2=0x5616ffd12ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0cab0 0x5616ffd12ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f48f40042a0 tx=0x7f48f40042d0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 -- 172.21.15.86:0/2555711648 shutdown_connections 2024-04-16T05:37:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5616ffd26160 0x5616ffd28550 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd15fb0 0x5616ffd13a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0d3d0 0x5616ffd13530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f48ea7fc640 0 client.16332 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 --2- 172.21.15.86:0/2555711648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0cab0 0x5616ffd12ff0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 -- 172.21.15.86:0/2555711648 >> 172.21.15.86:0/2555711648 conn(0x5616ffd071e0 msgr2=0x5616ffd08f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 -- 172.21.15.86:0/2555711648 shutdown_connections 2024-04-16T05:37:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 1 -- 172.21.15.86:0/2555711648 wait complete. 2024-04-16T05:37:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.134+0000 7f490c05f740 20 client.16332 trim_cache size 0 max 16384 2024-04-16T05:37:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.151+0000 7f490c05f740 1 Processor -- start 2024-04-16T05:37:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.151+0000 7f490c05f740 1 -- start start 2024-04-16T05:37:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f490c05f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 0x5616ffd21c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f490c05f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0d3d0 0x5616ffd2dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f490c05f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd15fb0 0x5616ffd2e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f490c05f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49041af510 con 0x5616ffd15fb0 2024-04-16T05:37:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f490c05f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f49041b1050 con 0x5616ffd0cab0 2024-04-16T05:37:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f490c05f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49041afbe0 con 0x5616ffd0d3d0 2024-04-16T05:37:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f49037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd15fb0 0x5616ffd2e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f49037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd15fb0 0x5616ffd2e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51192/0 (socket says 172.21.15.86:51192) 2024-04-16T05:37:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f49037fe640 1 -- 172.21.15.86:0/3583261424 learned_addr learned my addr 172.21.15.86:0/3583261424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f48fbfff640 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0d3d0 0x5616ffd2dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f4902ffd640 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 0x5616ffd21c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f49037fe640 1 -- 172.21.15.86:0/3583261424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0d3d0 msgr2=0x5616ffd2dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f49037fe640 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0d3d0 0x5616ffd2dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f49037fe640 1 -- 172.21.15.86:0/3583261424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 msgr2=0x5616ffd21c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f49037fe640 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 0x5616ffd21c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f49037fe640 1 -- 172.21.15.86:0/3583261424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ffd06a20 con 0x5616ffd15fb0 2024-04-16T05:37:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f48fbfff640 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0d3d0 0x5616ffd2dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f4902ffd640 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 0x5616ffd21c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f49037fe640 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd15fb0 0x5616ffd2e0f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f48fc014d40 tx=0x7f48fc09ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1262e633ec637b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.152+0000 7f48f8ff9640 1 -- 172.21.15.86:0/3583261424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc09f070 con 0x5616ffd15fb0 2024-04-16T05:37:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f48f8ff9640 1 -- 172.21.15.86:0/3583261424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc0a4a10 con 0x5616ffd15fb0 2024-04-16T05:37:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f48f8ff9640 1 -- 172.21.15.86:0/3583261424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0a47e0 con 0x5616ffd15fb0 2024-04-16T05:37:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f490c05f740 1 -- 172.21.15.86:0/3583261424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd15fb0 msgr2=0x5616ffd2e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f490c05f740 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd15fb0 0x5616ffd2e0f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f48fc014d40 tx=0x7f48fc09ba10 comp rx=0 tx=0).stop 2024-04-16T05:37:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f490c05f740 1 -- 172.21.15.86:0/3583261424 shutdown_connections 2024-04-16T05:37:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f490c05f740 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd15fb0 0x5616ffd2e0f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f490c05f740 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0d3d0 0x5616ffd2dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f490c05f740 1 --2- 172.21.15.86:0/3583261424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd0cab0 0x5616ffd21c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f490c05f740 1 -- 172.21.15.86:0/3583261424 >> 172.21.15.86:0/3583261424 conn(0x5616ffd071e0 msgr2=0x5616ffd13550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f490c05f740 1 -- 172.21.15.86:0/3583261424 shutdown_connections 2024-04-16T05:37:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.153+0000 7f490c05f740 1 -- 172.21.15.86:0/3583261424 wait complete. 2024-04-16T05:37:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f490c05f740 1 -- start start 2024-04-16T05:37:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f490c05f740 1 Processor -- start 2024-04-16T05:37:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f490c05f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0cab0 0x5616ffd1c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f490c05f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd1c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f490c05f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd15fb0 0x5616ffd8b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f490c05f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49041b0980 con 0x5616ffd0d3d0 2024-04-16T05:37:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f490c05f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f49041b02b0 con 0x5616ffd15fb0 2024-04-16T05:37:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f490c05f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49041ad4d0 con 0x5616ffd0cab0 2024-04-16T05:37:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f490c05f740 10 client.0 _ll_register_callbacks cb 0x7f48f4009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd1c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd1c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51202/0 (socket says 172.21.15.86:51202) 2024-04-16T05:37:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48fbfff640 1 -- 172.21.15.86:0/3748885806 learned_addr learned my addr 172.21.15.86:0/3748885806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f49037fe640 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd15fb0 0x5616ffd8b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48fbfff640 1 -- 172.21.15.86:0/3748885806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0cab0 msgr2=0x5616ffd1c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48fbfff640 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0cab0 0x5616ffd1c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48fbfff640 1 -- 172.21.15.86:0/3748885806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd15fb0 msgr2=0x5616ffd8b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48fbfff640 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd15fb0 0x5616ffd8b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48fbfff640 1 -- 172.21.15.86:0/3748885806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ffd8ba20 con 0x5616ffd0d3d0 2024-04-16T05:37:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48fbfff640 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd1c860 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f49041018f0 tx=0x7f48ec005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6084a1c5c55d03e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.154+0000 7f48ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f48ea7fc640 1 -- 172.21.15.86:0/3748885806 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec018070 con 0x5616ffd0d3d0 2024-04-16T05:37:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f48ea7fc640 1 -- 172.21.15.86:0/3748885806 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f48ec003a00 con 0x5616ffd0d3d0 2024-04-16T05:37:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f490c05f740 1 -- 172.21.15.86:0/3748885806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48fc0a4a10 con 0x5616ffd0d3d0 2024-04-16T05:37:03.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f48ea7fc640 1 -- 172.21.15.86:0/3748885806 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f48ec03b050 con 0x5616ffd0d3d0 2024-04-16T05:37:03.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f490c05f740 10 client.16338 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:03.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f490c05f740 1 -- 172.21.15.86:0/3748885806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5616ffd8cd40 con 0x5616ffd0d3d0 2024-04-16T05:37:03.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f48ea7fc640 1 -- 172.21.15.86:0/3748885806 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc0a4a10 con 0x5616ffd0d3d0 2024-04-16T05:37:03.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f48ea7fc640 1 -- 172.21.15.86:0/3748885806 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f48ec032c50 con 0x5616ffd0d3d0 2024-04-16T05:37:03.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f490c05f740 10 client.16338 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:03.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f490c05f740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:03.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f490c05f740 20 client.16338 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f490c05f740 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5616ffd8d4a0 0x5616ffd8f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f490c05f740 4 client.16338 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:37:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f490c05f740 1 -- 172.21.15.86:0/3748885806 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5616ffc79720 con 0x5616ffd8d4a0 2024-04-16T05:37:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.155+0000 7f4902ffd640 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5616ffd8d4a0 0x5616ffd8f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.156+0000 7f4902ffd640 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5616ffd8d4a0 0x5616ffd8f8c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86202c33bbed7682 server_cookie=c541b1c222f702e7 in_seq=0 out_seq=0 2024-04-16T05:37:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.156+0000 7f48ea7fc640 10 client.16338 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f48ea7fc640 1 -- 172.21.15.86:0/3748885806 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616ffc79720 con 0x5616ffd8d4a0 2024-04-16T05:37:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f48ea7fc640 10 client.16338 handle_command_reply: tid=0 2024-04-16T05:37:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 client.16338 shutdown 2024-04-16T05:37:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 -- 172.21.15.86:0/3748885806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 msgr2=0x5616ffd1c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd1c860 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f49041018f0 tx=0x7f48ec005c70 comp rx=0 tx=0).stop 2024-04-16T05:37:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 -- 172.21.15.86:0/3748885806 shutdown_connections 2024-04-16T05:37:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5616ffd8d4a0 0x5616ffd8f8c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616ffd15fb0 0x5616ffd8b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616ffd0d3d0 0x5616ffd1c860 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 --2- 172.21.15.86:0/3748885806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616ffd0cab0 0x5616ffd1c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f48ea7fc640 0 client.16338 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 -- 172.21.15.86:0/3748885806 >> 172.21.15.86:0/3748885806 conn(0x5616ffd071e0 msgr2=0x5616ffd059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 -- 172.21.15.86:0/3748885806 shutdown_connections 2024-04-16T05:37:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 1 -- 172.21.15.86:0/3748885806 wait complete. 2024-04-16T05:37:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.157+0000 7f490c05f740 20 client.16338 trim_cache size 0 max 16384 2024-04-16T05:37:03.166 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:37:03.166 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:37:03.166 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "8e3edae6-cede-497a-a1c1-941a0f9c60c9", 2024-04-16T05:37:03.167 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:37:03.167 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:37:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 -- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48f00a3da0 msgr2=0x7f48f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48f00a3da0 0x7f48f00a6260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5616ffd08370 tx=0x7f48ec0046c0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 -- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49041987d0 msgr2=0x7f4904198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49041987d0 0x7f4904198c80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00b970 tx=0x7f48fc00be30 comp rx=0 tx=0).stop 2024-04-16T05:37:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 -- 172.21.15.86:0/1378844004 shutdown_connections 2024-04-16T05:37:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49041987d0 0x7f4904198c80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4904107310 0x7f490419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48f00a3da0 0x7f48f00a6260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 --2- 172.21.15.86:0/1378844004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4904102240 0x7f490419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 -- 172.21.15.86:0/1378844004 >> 172.21.15.86:0/1378844004 conn(0x7f4904079450 msgr2=0x7f49040ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 -- 172.21.15.86:0/1378844004 shutdown_connections 2024-04-16T05:37:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.158+0000 7f490947e640 1 -- 172.21.15.86:0/1378844004 wait complete. 2024-04-16T05:37:03.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:37:03.170 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:37:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.278+0000 7f8968cc8640 1 -- 172.21.15.86:0/1307577384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964100650 msgr2=0x7f8964100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.278+0000 7f8968cc8640 1 --2- 172.21.15.86:0/1307577384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964100650 0x7f8964100ad0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f894c009900 tx=0x7f894c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.278+0000 7f8968cc8640 1 -- 172.21.15.86:0/1307577384 shutdown_connections 2024-04-16T05:37:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.278+0000 7f8968cc8640 1 --2- 172.21.15.86:0/1307577384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8964101010 0x7f896410d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.278+0000 7f8968cc8640 1 --2- 172.21.15.86:0/1307577384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964100650 0x7f8964100ad0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.278+0000 7f8968cc8640 1 --2- 172.21.15.86:0/1307577384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8964106660 0x7f8964106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.278+0000 7f8968cc8640 1 -- 172.21.15.86:0/1307577384 >> 172.21.15.86:0/1307577384 conn(0x7f89640774c0 msgr2=0x7f89640fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.278+0000 7f8968cc8640 1 -- 172.21.15.86:0/1307577384 shutdown_connections 2024-04-16T05:37:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.278+0000 7f8968cc8640 1 -- 172.21.15.86:0/1307577384 wait complete. 2024-04-16T05:37:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8968cc8640 1 Processor -- start 2024-04-16T05:37:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8968cc8640 1 -- start start 2024-04-16T05:37:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8968cc8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8964100650 0x7f896410edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8968cc8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964101010 0x7f896410f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8968cc8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8964106660 0x7f896410f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8968cc8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8964112480 con 0x7f8964101010 2024-04-16T05:37:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8968cc8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8964112300 con 0x7f8964106660 2024-04-16T05:37:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8968cc8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8964112600 con 0x7f8964100650 2024-04-16T05:37:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8961d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964101010 0x7f896410f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8961d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964101010 0x7f896410f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51222/0 (socket says 172.21.15.86:51222) 2024-04-16T05:37:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8961d9b640 1 -- 172.21.15.86:0/2048235931 learned_addr learned my addr 172.21.15.86:0/2048235931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f896259c640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8964100650 0x7f896410edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8962d9d640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8964106660 0x7f896410f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8961d9b640 1 -- 172.21.15.86:0/2048235931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8964100650 msgr2=0x7f896410edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8961d9b640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8964100650 0x7f896410edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8961d9b640 1 -- 172.21.15.86:0/2048235931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8964106660 msgr2=0x7f896410f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8961d9b640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8964106660 0x7f896410f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8961d9b640 1 -- 172.21.15.86:0/2048235931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89641a6e40 con 0x7f8964101010 2024-04-16T05:37:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f896259c640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8964100650 0x7f896410edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.279+0000 7f8962d9d640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8964106660 0x7f896410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.280+0000 7f8961d9b640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964101010 0x7f896410f300 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f894c005ab0 tx=0x7f894c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.280+0000 7f895b7fe640 1 -- 172.21.15.86:0/2048235931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c043020 con 0x7f8964101010 2024-04-16T05:37:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.280+0000 7f8968cc8640 1 -- 172.21.15.86:0/2048235931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89641a70d0 con 0x7f8964101010 2024-04-16T05:37:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.280+0000 7f895b7fe640 1 -- 172.21.15.86:0/2048235931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f894c0054e0 con 0x7f8964101010 2024-04-16T05:37:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.280+0000 7f895b7fe640 1 -- 172.21.15.86:0/2048235931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c003d40 con 0x7f8964101010 2024-04-16T05:37:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.280+0000 7f8968cc8640 1 -- 172.21.15.86:0/2048235931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89641a7670 con 0x7f8964101010 2024-04-16T05:37:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.282+0000 7f895b7fe640 1 -- 172.21.15.86:0/2048235931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f894c046020 con 0x7f8964101010 2024-04-16T05:37:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.282+0000 7f896b993740 1 Processor -- start 2024-04-16T05:37:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.282+0000 7f896b993740 1 -- start start 2024-04-16T05:37:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896b993740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdaee50 0x55611bdaf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896b993740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896b993740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdb8350 0x55611bdba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896b993740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55611bd14180 con 0x55611bdaf770 2024-04-16T05:37:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896b993740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55611bcb1360 con 0x55611bdaee50 2024-04-16T05:37:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896b993740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55611bd10140 con 0x55611bdb8350 2024-04-16T05:37:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f895b7fe640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89500a3d50 0x7f89500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51230/0 (socket says 172.21.15.86:51230) 2024-04-16T05:37:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896259c640 1 -- 172.21.15.86:0/1549972368 learned_addr learned my addr 172.21.15.86:0/1549972368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f8962d9d640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89500a3d50 0x7f89500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f8962d9d640 1 --2- 172.21.15.86:0/1549972368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdaee50 0x55611bdaf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f895b7fe640 1 -- 172.21.15.86:0/2048235931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f894c0f2cd0 con 0x7f8964101010 2024-04-16T05:37:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f8961d9b640 1 --2- 172.21.15.86:0/1549972368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdb8350 0x55611bdba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896259c640 1 -- 172.21.15.86:0/1549972368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdb8350 msgr2=0x55611bdba740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896259c640 1 --2- 172.21.15.86:0/1549972368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdb8350 0x55611bdba740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896259c640 1 -- 172.21.15.86:0/1549972368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdaee50 msgr2=0x55611bdaf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896259c640 1 --2- 172.21.15.86:0/1549972368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdaee50 0x55611bdaf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896259c640 1 -- 172.21.15.86:0/1549972368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611baf07d0 con 0x55611bdaf770 2024-04-16T05:37:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f896259c640 1 --2- 172.21.15.86:0/1549972368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb7e10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f895400aa10 tx=0x7f895400aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb5dc1632329716 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f8962d9d640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89500a3d50 0x7f89500a6210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f895c014730 tx=0x7f895c00a360 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.283+0000 7f8958ff9640 1 -- 172.21.15.86:0/1549972368 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954015070 con 0x55611bdaf770 2024-04-16T05:37:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f8958ff9640 1 -- 172.21.15.86:0/1549972368 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895400cc60 con 0x55611bdaf770 2024-04-16T05:37:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f8958ff9640 1 -- 172.21.15.86:0/1549972368 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89540106c0 con 0x55611bdaf770 2024-04-16T05:37:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f896b993740 1 -- 172.21.15.86:0/1549972368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 msgr2=0x55611bdb7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f896b993740 1 --2- 172.21.15.86:0/1549972368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb7e10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f895400aa10 tx=0x7f895400aed0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f896b993740 1 -- 172.21.15.86:0/1549972368 shutdown_connections 2024-04-16T05:37:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f896b993740 1 --2- 172.21.15.86:0/1549972368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdb8350 0x55611bdba740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f896b993740 1 --2- 172.21.15.86:0/1549972368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb7e10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f896b993740 1 --2- 172.21.15.86:0/1549972368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdaee50 0x55611bdaf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f896b993740 1 -- 172.21.15.86:0/1549972368 >> 172.21.15.86:0/1549972368 conn(0x55611bda9580 msgr2=0x55611bda9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f896b993740 1 -- 172.21.15.86:0/1549972368 shutdown_connections 2024-04-16T05:37:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.284+0000 7f896b993740 1 -- 172.21.15.86:0/1549972368 wait complete. 2024-04-16T05:37:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f896b993740 1 -- start start 2024-04-16T05:37:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f896b993740 1 Processor -- start 2024-04-16T05:37:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f896b993740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdb52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f896b993740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f896b993740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdb8350 0x55611bdb5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f896b993740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55611bdbb140 con 0x55611bdaf770 2024-04-16T05:37:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f896b993740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55611bccd2c0 con 0x55611bdb8350 2024-04-16T05:37:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f896b993740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55611bdbb2c0 con 0x55611bdaee50 2024-04-16T05:37:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f896b993740 10 client.0 _ll_register_callbacks cb 0x55611bcb4090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8961d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8961d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51234/0 (socket says 172.21.15.86:51234) 2024-04-16T05:37:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8961d9b640 1 -- 172.21.15.86:0/535201562 learned_addr learned my addr 172.21.15.86:0/535201562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8961d9b640 1 -- 172.21.15.86:0/535201562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 msgr2=0x55611bdb52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8962d9d640 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdb8350 0x55611bdb5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8961d9b640 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdb52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8961d9b640 1 -- 172.21.15.86:0/535201562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdb8350 msgr2=0x55611bdb5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8961d9b640 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdb8350 0x55611bdb5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8961d9b640 1 -- 172.21.15.86:0/535201562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611ba263f0 con 0x55611bdaf770 2024-04-16T05:37:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8962d9d640 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdb8350 0x55611bdb5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8961d9b640 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb5830 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f894c0be9a0 tx=0x7f894c003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c065537d79cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.285+0000 7f8949ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f8949ffb640 1 -- 172.21.15.86:0/535201562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c005690 con 0x55611bdaf770 2024-04-16T05:37:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f8949ffb640 1 -- 172.21.15.86:0/535201562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f894c0f0330 con 0x55611bdaf770 2024-04-16T05:37:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896b993740 1 -- 172.21.15.86:0/535201562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f895400cc60 con 0x55611bdaf770 2024-04-16T05:37:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f8949ffb640 1 -- 172.21.15.86:0/535201562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f894c0f0530 con 0x55611bdaf770 2024-04-16T05:37:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896b993740 10 client.16350 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896b993740 1 -- 172.21.15.86:0/535201562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55611b9e60f0 con 0x55611bdaf770 2024-04-16T05:37:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f8949ffb640 1 -- 172.21.15.86:0/535201562 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895400cc60 con 0x55611bdaf770 2024-04-16T05:37:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f8949ffb640 1 -- 172.21.15.86:0/535201562 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f894c0bec70 con 0x55611bdaf770 2024-04-16T05:37:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896b993740 10 client.16350 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896b993740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896b993740 20 client.16350 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896b993740 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55611bdc8620 0x55611bdcaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896b993740 4 client.16350 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896b993740 1 -- 172.21.15.86:0/535201562 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55611bd21ba0 con 0x55611bdc8620 2024-04-16T05:37:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.286+0000 7f896259c640 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55611bdc8620 0x55611bdcaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.287+0000 7f8949ffb640 10 client.16350 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.287+0000 7f896259c640 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55611bdc8620 0x55611bdcaae0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ee7b444c07192e server_cookie=6da00c6f96a25f6c in_seq=0 out_seq=0 2024-04-16T05:37:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.287+0000 7f8949ffb640 1 -- 172.21.15.86:0/535201562 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55611bd21ba0 con 0x55611bdc8620 2024-04-16T05:37:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.287+0000 7f8949ffb640 10 client.16350 handle_command_reply: tid=0 2024-04-16T05:37:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 client.16350 shutdown 2024-04-16T05:37:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 -- 172.21.15.86:0/535201562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 msgr2=0x55611bdb5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb5830 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f894c0be9a0 tx=0x7f894c003840 comp rx=0 tx=0).stop 2024-04-16T05:37:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 -- 172.21.15.86:0/535201562 shutdown_connections 2024-04-16T05:37:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55611bdc8620 0x55611bdcaae0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bdb8350 0x55611bdb5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f8949ffb640 0 client.16350 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdb5830 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 --2- 172.21.15.86:0/535201562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdb52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 -- 172.21.15.86:0/535201562 >> 172.21.15.86:0/535201562 conn(0x55611bda9580 msgr2=0x55611bdab2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 -- 172.21.15.86:0/535201562 shutdown_connections 2024-04-16T05:37:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 1 -- 172.21.15.86:0/535201562 wait complete. 2024-04-16T05:37:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.288+0000 7f896b993740 20 client.16350 trim_cache size 0 max 16384 2024-04-16T05:37:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.305+0000 7f896b993740 1 Processor -- start 2024-04-16T05:37:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.305+0000 7f896b993740 1 -- start start 2024-04-16T05:37:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.305+0000 7f896b993740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bda9580 0x55611bdb7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.305+0000 7f896b993740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdc4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.305+0000 7f896b993740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdd03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.305+0000 7f896b993740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b1660 con 0x55611bdaf770 2024-04-16T05:37:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.305+0000 7f896b993740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b31a0 con 0x55611bda9580 2024-04-16T05:37:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.305+0000 7f896b993740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b1d30 con 0x55611bdaee50 2024-04-16T05:37:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.305+0000 7f8962d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdd03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8962d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdd03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51242/0 (socket says 172.21.15.86:51242) 2024-04-16T05:37:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8962d9d640 1 -- 172.21.15.86:0/1871372801 learned_addr learned my addr 172.21.15.86:0/1871372801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f896259c640 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bda9580 0x55611bdb7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8961d9b640 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdc4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8962d9d640 1 -- 172.21.15.86:0/1871372801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 msgr2=0x55611bdc4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8962d9d640 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdc4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8962d9d640 1 -- 172.21.15.86:0/1871372801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bda9580 msgr2=0x55611bdb7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8962d9d640 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bda9580 0x55611bdb7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8962d9d640 1 -- 172.21.15.86:0/1871372801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611bda93b0 con 0x55611bdaf770 2024-04-16T05:37:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8961d9b640 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdc4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f8962d9d640 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdd03b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f895c014410 tx=0x7f895c005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252fa0ff1f27b027 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f894bfff640 1 -- 172.21.15.86:0/1871372801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c023030 con 0x55611bdaf770 2024-04-16T05:37:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f894bfff640 1 -- 172.21.15.86:0/1871372801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c018040 con 0x55611bdaf770 2024-04-16T05:37:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.306+0000 7f894bfff640 1 -- 172.21.15.86:0/1871372801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c012a50 con 0x55611bdaf770 2024-04-16T05:37:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 -- 172.21.15.86:0/1871372801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 msgr2=0x55611bdd03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdd03b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f895c014410 tx=0x7f895c005eb0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 -- 172.21.15.86:0/1871372801 shutdown_connections 2024-04-16T05:37:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611bdd03b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdc4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 --2- 172.21.15.86:0/1871372801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bda9580 0x55611bdb7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 -- 172.21.15.86:0/1871372801 >> 172.21.15.86:0/1871372801 conn(0x55611bdb3f30 msgr2=0x55611bda8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 -- 172.21.15.86:0/1871372801 shutdown_connections 2024-04-16T05:37:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 -- 172.21.15.86:0/1871372801 wait complete. 2024-04-16T05:37:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 -- start start 2024-04-16T05:37:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 Processor -- start 2024-04-16T05:37:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bda9580 0x55611bdbe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdbec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611be2dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b2ad0 con 0x55611bdaf770 2024-04-16T05:37:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b2400 con 0x55611bda9580 2024-04-16T05:37:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89641af620 con 0x55611bdaee50 2024-04-16T05:37:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f896b993740 10 client.0 _ll_register_callbacks cb 0x7f894c0a7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:03.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f8962d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611be2dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.307+0000 7f8962d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611be2dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51256/0 (socket says 172.21.15.86:51256) 2024-04-16T05:37:03.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8962d9d640 1 -- 172.21.15.86:0/1482995468 learned_addr learned my addr 172.21.15.86:0/1482995468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:03.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8961d9b640 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdbec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8962d9d640 1 -- 172.21.15.86:0/1482995468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 msgr2=0x55611bdbec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8962d9d640 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdbec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8962d9d640 1 -- 172.21.15.86:0/1482995468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bda9580 msgr2=0x55611bdbe720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:03.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8962d9d640 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bda9580 0x55611bdbe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8962d9d640 1 -- 172.21.15.86:0/1482995468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611be2e220 con 0x55611bdaf770 2024-04-16T05:37:03.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8962d9d640 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611be2dc30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f895c00a730 tx=0x7f895c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7430ffbaaa9dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:03.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8949ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:03.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8949ffb640 1 -- 172.21.15.86:0/1482995468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c023040 con 0x55611bdaf770 2024-04-16T05:37:03.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8949ffb640 1 -- 172.21.15.86:0/1482995468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f895c012a30 con 0x55611bdaf770 2024-04-16T05:37:03.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f896b993740 1 -- 172.21.15.86:0/1482995468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f895c018040 con 0x55611bdaf770 2024-04-16T05:37:03.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8949ffb640 1 -- 172.21.15.86:0/1482995468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f895c012c30 con 0x55611bdaf770 2024-04-16T05:37:03.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f896b993740 10 client.16356 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:03.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f896b993740 1 -- 172.21.15.86:0/1482995468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55611be2f480 con 0x55611bdaf770 2024-04-16T05:37:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.308+0000 7f8949ffb640 1 -- 172.21.15.86:0/1482995468 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c018040 con 0x55611bdaf770 2024-04-16T05:37:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f8949ffb640 1 -- 172.21.15.86:0/1482995468 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f895c01d040 con 0x55611bdaf770 2024-04-16T05:37:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896b993740 10 client.16356 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896b993740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896b993740 20 client.16356 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896b993740 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55611bdb8350 0x55611be31ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896b993740 4 client.16356 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:37:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896b993740 1 -- 172.21.15.86:0/1482995468 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55611bd1ba10 con 0x55611bdb8350 2024-04-16T05:37:03.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896259c640 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55611bdb8350 0x55611be31ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:03.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f8949ffb640 10 client.16356 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:03.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896259c640 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55611bdb8350 0x55611be31ba0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a95ce685808af6 server_cookie=2d5a7cd16a07fae1 in_seq=0 out_seq=0 2024-04-16T05:37:03.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f8949ffb640 1 -- 172.21.15.86:0/1482995468 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55611bd1ba10 con 0x55611bdb8350 2024-04-16T05:37:03.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f8949ffb640 10 client.16356 handle_command_reply: tid=0 2024-04-16T05:37:03.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896b993740 1 client.16356 shutdown 2024-04-16T05:37:03.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896b993740 1 -- 172.21.15.86:0/1482995468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 msgr2=0x55611be2dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.309+0000 7f896b993740 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611be2dc30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f895c00a730 tx=0x7f895c012450 comp rx=0 tx=0).stop 2024-04-16T05:37:03.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f896b993740 1 -- 172.21.15.86:0/1482995468 shutdown_connections 2024-04-16T05:37:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f896b993740 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55611bdb8350 0x55611be31ba0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f896b993740 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55611bdaf770 0x55611be2dc30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f896b993740 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55611bdaee50 0x55611bdbec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8949ffb640 0 client.16356 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f896b993740 1 --2- 172.21.15.86:0/1482995468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611bda9580 0x55611bdbe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f896b993740 1 -- 172.21.15.86:0/1482995468 >> 172.21.15.86:0/1482995468 conn(0x55611bdb3f30 msgr2=0x55611bdb5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f896b993740 1 -- 172.21.15.86:0/1482995468 shutdown_connections 2024-04-16T05:37:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f896b993740 1 -- 172.21.15.86:0/1482995468 wait complete. 2024-04-16T05:37:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f896b993740 20 client.16356 trim_cache size 0 max 16384 2024-04-16T05:37:03.320 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:37:03.320 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:37:03.320 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:37:03.320 INFO:teuthology.orchestra.run.smithi086.stdout: "8e3edae6-cede-497a-a1c1-941a0f9c60c9": { 2024-04-16T05:37:03.320 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:37:03.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "8e3edae6-cede-497a-a1c1-941a0f9c60c9", 2024-04-16T05:37:03.321 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:37:03.321 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:37:03.321 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:37:03.321 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:37:03.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 -- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89500a3d50 msgr2=0x7f89500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89500a3d50 0x7f89500a6210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f895c014730 tx=0x7f895c00a360 comp rx=0 tx=0).stop 2024-04-16T05:37:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 -- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964101010 msgr2=0x7f896410f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964101010 0x7f896410f300 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f894c005ab0 tx=0x7f894c0045e0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 -- 172.21.15.86:0/2048235931 shutdown_connections 2024-04-16T05:37:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89500a3d50 0x7f89500a6210 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8964106660 0x7f896410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8964101010 0x7f896410f300 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 --2- 172.21.15.86:0/2048235931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8964100650 0x7f896410edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 -- 172.21.15.86:0/2048235931 >> 172.21.15.86:0/2048235931 conn(0x7f89640774c0 msgr2=0x7f8964103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 -- 172.21.15.86:0/2048235931 shutdown_connections 2024-04-16T05:37:03.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:03.310+0000 7f8968cc8640 1 -- 172.21.15.86:0/2048235931 wait complete. 2024-04-16T05:37:03.324 INFO:tasks.cephfs.filesystem:scrub status for tag:8e3edae6-cede-497a-a1c1-941a0f9c60c9 - {'path': '/', 'tag': '8e3edae6-cede-497a-a1c1-941a0f9c60c9', 'options': 'recursive,force'} 2024-04-16T05:37:04.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:04 smithi086 ceph-mon[32028]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-16T05:37:04.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:04 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:37:04.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:04 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:04.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:04 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:37:04.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:04 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:04.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:04 smithi196 ceph-mon[36373]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-16T05:37:04.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:04 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:37:04.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:04 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:04.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:04 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:37:04.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:04 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:04.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:04 smithi194 ceph-mon[35865]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-16T05:37:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:04 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:37:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:04 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:04 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:37:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:04 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:05.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:05 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:37:05.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:05 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:37:05.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:05 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:37:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:06 smithi196 ceph-mon[36373]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-16T05:37:06.807 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:06 smithi194 ceph-mon[35865]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-16T05:37:06.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:06 smithi086 ceph-mon[32028]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-16T05:37:08.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:08 smithi196 ceph-mon[36373]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-16T05:37:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:08 smithi194 ceph-mon[35865]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-16T05:37:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:08 smithi086 ceph-mon[32028]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-16T05:37:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:10 smithi196 ceph-mon[36373]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-16T05:37:10.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:10 smithi194 ceph-mon[35865]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-16T05:37:10.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:10 smithi086 ceph-mon[32028]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-16T05:37:12.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:12 smithi196 ceph-mon[36373]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-16T05:37:12.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:12 smithi086 ceph-mon[32028]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-16T05:37:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:12 smithi194 ceph-mon[35865]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-16T05:37:14.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:14 smithi194 ceph-mon[35865]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-16T05:37:14.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:14 smithi086 ceph-mon[32028]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-16T05:37:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:14 smithi196 ceph-mon[36373]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-16T05:37:16.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:16 smithi194 ceph-mon[35865]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:37:16.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:16 smithi086 ceph-mon[32028]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:37:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:16 smithi196 ceph-mon[36373]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:37:18.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:18 smithi194 ceph-mon[35865]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:37:18.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:18 smithi086 ceph-mon[32028]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:37:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:18 smithi196 ceph-mon[36373]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:37:20.787 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:20 smithi196 ceph-mon[36373]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-16T05:37:20.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:20 smithi194 ceph-mon[35865]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-16T05:37:20.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:20 smithi086 ceph-mon[32028]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-16T05:37:22.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:22 smithi194 ceph-mon[35865]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:37:22.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:22 smithi086 ceph-mon[32028]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:37:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:22 smithi196 ceph-mon[36373]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:37:23.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:23 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:37:23.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:23 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:37:23.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:23 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:37:24.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:24 smithi194 ceph-mon[35865]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:37:24.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:24 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:37:24.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:24 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:37:24.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:24 smithi086 ceph-mon[32028]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:37:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:24 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:37:24.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:24 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:37:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:24 smithi196 ceph-mon[36373]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:37:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:24 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:37:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:24 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:37:25.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:25 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:37:25.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:25 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:37:25.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:25 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:37:25.970 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:25 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:37:25.971 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:25 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:37:25.971 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:25 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:37:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:25 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:37:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:25 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:37:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:25 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:37:26.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:26 smithi194 ceph-mon[35865]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:26.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:26 smithi086 ceph-mon[32028]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:26 smithi196 ceph-mon[36373]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:28.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:28 smithi194 ceph-mon[35865]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:28.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:28 smithi086 ceph-mon[32028]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:28.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:28 smithi196 ceph-mon[36373]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:30.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:30 smithi194 ceph-mon[35865]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-16T05:37:30.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:30 smithi086 ceph-mon[32028]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-16T05:37:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:30 smithi196 ceph-mon[36373]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-16T05:37:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:32 smithi194 ceph-mon[35865]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-16T05:37:32.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:32 smithi086 ceph-mon[32028]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-16T05:37:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:32 smithi196 ceph-mon[36373]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-16T05:37:33.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:37:33.325 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:37:33.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.445+0000 7fbf8f729640 1 -- 172.21.15.86:0/1839926345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102bf0 msgr2=0x7fbf88106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.445+0000 7fbf8f729640 1 --2- 172.21.15.86:0/1839926345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102bf0 0x7fbf88106c30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009900 tx=0x7fbf7c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:33.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 -- 172.21.15.86:0/1839926345 shutdown_connections 2024-04-16T05:37:33.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 --2- 172.21.15.86:0/1839926345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf88107310 0x7fbf8810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 --2- 172.21.15.86:0/1839926345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102bf0 0x7fbf88106c30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 --2- 172.21.15.86:0/1839926345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf88102240 0x7fbf88102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 -- 172.21.15.86:0/1839926345 >> 172.21.15.86:0/1839926345 conn(0x7fbf88079450 msgr2=0x7fbf88079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 -- 172.21.15.86:0/1839926345 shutdown_connections 2024-04-16T05:37:33.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 -- 172.21.15.86:0/1839926345 wait complete. 2024-04-16T05:37:33.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 Processor -- start 2024-04-16T05:37:33.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 -- start start 2024-04-16T05:37:33.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102240 0x7fbf8819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.446+0000 7fbf8f729640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf88107310 0x7fbf8819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8f729640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf881987e0 0x7fbf88198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8f729640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88112400 con 0x7fbf88102240 2024-04-16T05:37:33.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8f729640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88112280 con 0x7fbf881987e0 2024-04-16T05:37:33.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8f729640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88112580 con 0x7fbf88107310 2024-04-16T05:37:33.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8d4c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102240 0x7fbf8819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8d4c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102240 0x7fbf8819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56808/0 (socket says 172.21.15.86:56808) 2024-04-16T05:37:33.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8d4c5640 1 -- 172.21.15.86:0/895922274 learned_addr learned my addr 172.21.15.86:0/895922274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf88107310 0x7fbf8819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf881987e0 0x7fbf88198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8d4c5640 1 -- 172.21.15.86:0/895922274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf88107310 msgr2=0x7fbf8819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8d4c5640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf88107310 0x7fbf8819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8d4c5640 1 -- 172.21.15.86:0/895922274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf881987e0 msgr2=0x7fbf88198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8d4c5640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf881987e0 0x7fbf88198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8d4c5640 1 -- 172.21.15.86:0/895922274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf88199370 con 0x7fbf88102240 2024-04-16T05:37:33.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf88107310 0x7fbf8819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:33.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf881987e0 0x7fbf88198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:33.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8d4c5640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102240 0x7fbf8819e670 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800ba50 tx=0x7fbf7800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf867fc640 1 -- 172.21.15.86:0/895922274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78019070 con 0x7fbf88102240 2024-04-16T05:37:33.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf867fc640 1 -- 172.21.15.86:0/895922274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf7800cbe0 con 0x7fbf88102240 2024-04-16T05:37:33.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8f729640 1 -- 172.21.15.86:0/895922274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf88199660 con 0x7fbf88102240 2024-04-16T05:37:33.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf867fc640 1 -- 172.21.15.86:0/895922274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf780076e0 con 0x7fbf88102240 2024-04-16T05:37:33.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.447+0000 7fbf8f729640 1 -- 172.21.15.86:0/895922274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf881a5840 con 0x7fbf88102240 2024-04-16T05:37:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.449+0000 7fbf867fc640 1 -- 172.21.15.86:0/895922274 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbf7801c070 con 0x7fbf88102240 2024-04-16T05:37:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf923ea740 1 Processor -- start 2024-04-16T05:37:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf923ea740 1 -- start start 2024-04-16T05:37:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf867fc640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbf700a3d20 0x7fbf700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d25e50 0x55f424d26230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d2ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d2f350 0x55f424d31740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf923ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f424c8b180 con 0x55f424d26770 2024-04-16T05:37:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf923ea740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f424c28360 con 0x55f424d25e50 2024-04-16T05:37:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf923ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f424c87140 con 0x55f424d2f350 2024-04-16T05:37:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf8ccc4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d2ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.450+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbf700a3d20 0x7fbf700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8ccc4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d2ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56816/0 (socket says 172.21.15.86:56816) 2024-04-16T05:37:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/1137638720 learned_addr learned my addr 172.21.15.86:0/1137638720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf867fc640 1 -- 172.21.15.86:0/895922274 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbf780d00e0 con 0x7fbf88102240 2024-04-16T05:37:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8d4c5640 1 --2- 172.21.15.86:0/1137638720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d25e50 0x55f424d26230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/1137638720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d2f350 0x55f424d31740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/1137638720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d2f350 msgr2=0x55f424d31740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/1137638720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d2f350 0x55f424d31740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/1137638720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d25e50 msgr2=0x55f424d26230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/1137638720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d25e50 0x55f424d26230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/1137638720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f424a677d0 con 0x55f424d26770 2024-04-16T05:37:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbf700a3d20 0x7fbf700a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005d00 tx=0x7fbf800157f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/1137638720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d2ee10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c006010 tx=0x7fbf7c02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a6ac3dfa528b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf6f7fe640 1 -- 172.21.15.86:0/1137638720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c02ca80 con 0x55f424d26770 2024-04-16T05:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf6f7fe640 1 -- 172.21.15.86:0/1137638720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf7c02cc20 con 0x55f424d26770 2024-04-16T05:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.451+0000 7fbf6f7fe640 1 -- 172.21.15.86:0/1137638720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c004920 con 0x55f424d26770 2024-04-16T05:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 -- 172.21.15.86:0/1137638720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 msgr2=0x55f424d2ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 --2- 172.21.15.86:0/1137638720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d2ee10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c006010 tx=0x7fbf7c02c5d0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 -- 172.21.15.86:0/1137638720 shutdown_connections 2024-04-16T05:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 --2- 172.21.15.86:0/1137638720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d2f350 0x55f424d31740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 --2- 172.21.15.86:0/1137638720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d2ee10 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 --2- 172.21.15.86:0/1137638720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d25e50 0x55f424d26230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 -- 172.21.15.86:0/1137638720 >> 172.21.15.86:0/1137638720 conn(0x55f424d20580 msgr2=0x55f424d20960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 -- 172.21.15.86:0/1137638720 shutdown_connections 2024-04-16T05:37:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 -- 172.21.15.86:0/1137638720 wait complete. 2024-04-16T05:37:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 -- start start 2024-04-16T05:37:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.452+0000 7fbf923ea740 1 Processor -- start 2024-04-16T05:37:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d25e50 0x55f424d2c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d26770 0x55f424d2c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d2f350 0x55f424d2cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf923ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f424d320e0 con 0x55f424d2f350 2024-04-16T05:37:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf923ea740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f424c442c0 con 0x55f424d26770 2024-04-16T05:37:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf923ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f424d32260 con 0x55f424d25e50 2024-04-16T05:37:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf923ea740 10 client.0 _ll_register_callbacks cb 0x55f424c2b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8dcc6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d2f350 0x55f424d2cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8dcc6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d2f350 0x55f424d2cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56820/0 (socket says 172.21.15.86:56820) 2024-04-16T05:37:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8dcc6640 1 -- 172.21.15.86:0/3890252259 learned_addr learned my addr 172.21.15.86:0/3890252259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d26770 0x55f424d2c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8dcc6640 1 -- 172.21.15.86:0/3890252259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d25e50 msgr2=0x55f424d2c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d25e50 0x55f424d2c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8dcc6640 1 -- 172.21.15.86:0/3890252259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d26770 msgr2=0x55f424d2c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d26770 0x55f424d2c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8dcc6640 1 -- 172.21.15.86:0/3890252259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f42499d3f0 con 0x55f424d2f350 2024-04-16T05:37:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d26770 0x55f424d2c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d2f350 0x55f424d2cd70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbf80012700 tx=0x7fbf80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723ec5bc255619c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.453+0000 7fbf6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf6cff9640 1 -- 172.21.15.86:0/3890252259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80005020 con 0x55f424d2f350 2024-04-16T05:37:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf6cff9640 1 -- 172.21.15.86:0/3890252259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbf800051c0 con 0x55f424d2f350 2024-04-16T05:37:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf923ea740 1 -- 172.21.15.86:0/3890252259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf7c02cc20 con 0x55f424d2f350 2024-04-16T05:37:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf6cff9640 1 -- 172.21.15.86:0/3890252259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf8002e050 con 0x55f424d2f350 2024-04-16T05:37:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf923ea740 10 client.16368 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf923ea740 1 -- 172.21.15.86:0/3890252259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f42495d0f0 con 0x55f424d2f350 2024-04-16T05:37:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf6cff9640 1 -- 172.21.15.86:0/3890252259 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf7c02cc20 con 0x55f424d2f350 2024-04-16T05:37:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf6cff9640 1 -- 172.21.15.86:0/3890252259 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbf8001f8a0 con 0x55f424d2f350 2024-04-16T05:37:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf923ea740 10 client.16368 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf923ea740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf923ea740 20 client.16368 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf923ea740 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f424d3f670 0x55f424d41b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf923ea740 4 client.16368 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf923ea740 1 -- 172.21.15.86:0/3890252259 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f424c98ba0 con 0x55f424d3f670 2024-04-16T05:37:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.454+0000 7fbf8d4c5640 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f424d3f670 0x55f424d41b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.455+0000 7fbf6cff9640 10 client.16368 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.455+0000 7fbf8d4c5640 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f424d3f670 0x55f424d41b30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade406bdf35fec7b server_cookie=cd37c6f65cf309bf in_seq=0 out_seq=0 2024-04-16T05:37:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf6cff9640 1 -- 172.21.15.86:0/3890252259 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f424c98ba0 con 0x55f424d3f670 2024-04-16T05:37:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf6cff9640 10 client.16368 handle_command_reply: tid=0 2024-04-16T05:37:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 client.16368 shutdown 2024-04-16T05:37:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 -- 172.21.15.86:0/3890252259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d2f350 msgr2=0x55f424d2cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d2f350 0x55f424d2cd70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbf80012700 tx=0x7fbf80004730 comp rx=0 tx=0).stop 2024-04-16T05:37:33.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 -- 172.21.15.86:0/3890252259 shutdown_connections 2024-04-16T05:37:33.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f424d3f670 0x55f424d41b30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d2f350 0x55f424d2cd70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d26770 0x55f424d2c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 --2- 172.21.15.86:0/3890252259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d25e50 0x55f424d2c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 -- 172.21.15.86:0/3890252259 >> 172.21.15.86:0/3890252259 conn(0x55f424d20580 msgr2=0x55f424d222d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf6cff9640 0 client.16368 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:33.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 -- 172.21.15.86:0/3890252259 shutdown_connections 2024-04-16T05:37:33.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 1 -- 172.21.15.86:0/3890252259 wait complete. 2024-04-16T05:37:33.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.456+0000 7fbf923ea740 20 client.16368 trim_cache size 0 max 16384 2024-04-16T05:37:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf923ea740 1 Processor -- start 2024-04-16T05:37:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf923ea740 1 -- start start 2024-04-16T05:37:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d25e50 0x55f424d2ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d3afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d2f350 0x55f424d47400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf923ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881af4c0 con 0x55f424d26770 2024-04-16T05:37:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf923ea740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881b1000 con 0x55f424d25e50 2024-04-16T05:37:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf923ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881afb90 con 0x55f424d2f350 2024-04-16T05:37:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf8ccc4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d3afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.473+0000 7fbf8ccc4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d3afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56834/0 (socket says 172.21.15.86:56834) 2024-04-16T05:37:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/1873404754 learned_addr learned my addr 172.21.15.86:0/1873404754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/1873404754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d2f350 msgr2=0x55f424d47400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/1873404754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d2f350 0x55f424d47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/1873404754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d25e50 msgr2=0x55f424d2ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/1873404754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d25e50 0x55f424d2ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/1873404754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f424d203b0 con 0x55f424d26770 2024-04-16T05:37:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/1873404754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d3afa0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c004000 tx=0x7fbf7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa89909350c31843 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf6effd640 1 -- 172.21.15.86:0/1873404754 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c049070 con 0x55f424d26770 2024-04-16T05:37:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf6effd640 1 -- 172.21.15.86:0/1873404754 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf7c02cb40 con 0x55f424d26770 2024-04-16T05:37:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.474+0000 7fbf6effd640 1 -- 172.21.15.86:0/1873404754 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c003e60 con 0x55f424d26770 2024-04-16T05:37:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 -- 172.21.15.86:0/1873404754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 msgr2=0x55f424d3afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 --2- 172.21.15.86:0/1873404754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d3afa0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c004000 tx=0x7fbf7c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 -- 172.21.15.86:0/1873404754 shutdown_connections 2024-04-16T05:37:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 --2- 172.21.15.86:0/1873404754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d2f350 0x55f424d47400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 --2- 172.21.15.86:0/1873404754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d3afa0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 --2- 172.21.15.86:0/1873404754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d25e50 0x55f424d2ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 -- 172.21.15.86:0/1873404754 >> 172.21.15.86:0/1873404754 conn(0x55f424d20580 msgr2=0x55f424d1f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 -- 172.21.15.86:0/1873404754 shutdown_connections 2024-04-16T05:37:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 -- 172.21.15.86:0/1873404754 wait complete. 2024-04-16T05:37:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 -- start start 2024-04-16T05:37:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 Processor -- start 2024-04-16T05:37:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d25e50 0x55f424d35120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d35660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d2f350 0x55f424d35ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881b0930 con 0x55f424d26770 2024-04-16T05:37:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881b0260 con 0x55f424d2f350 2024-04-16T05:37:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881ad480 con 0x55f424d25e50 2024-04-16T05:37:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.475+0000 7fbf923ea740 10 client.0 _ll_register_callbacks cb 0x7fbf800151e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:33.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8ccc4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d35660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8ccc4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d35660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56850/0 (socket says 172.21.15.86:56850) 2024-04-16T05:37:33.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/363788999 learned_addr learned my addr 172.21.15.86:0/363788999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8d4c5640 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d25e50 0x55f424d35120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/363788999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d25e50 msgr2=0x55f424d35120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d25e50 0x55f424d35120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/363788999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d2f350 msgr2=0x55f424d35ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d2f350 0x55f424d35ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d2f350 0x55f424d35ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8ccc4640 1 -- 172.21.15.86:0/363788999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f424da4ed0 con 0x55f424d26770 2024-04-16T05:37:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8dcc6640 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d2f350 0x55f424d35ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf8ccc4640 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d35660 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c006010 tx=0x7fbf7c02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70cc3cef1e75662 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf6cff9640 1 -- 172.21.15.86:0/363788999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c054070 con 0x55f424d26770 2024-04-16T05:37:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf6cff9640 1 -- 172.21.15.86:0/363788999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbf7c004240 con 0x55f424d26770 2024-04-16T05:37:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf923ea740 1 -- 172.21.15.86:0/363788999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf7c02cb40 con 0x55f424d26770 2024-04-16T05:37:33.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf6cff9640 1 -- 172.21.15.86:0/363788999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf7c049040 con 0x55f424d26770 2024-04-16T05:37:33.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf923ea740 10 client.16374 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:33.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf923ea740 1 -- 172.21.15.86:0/363788999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f424da61c0 con 0x55f424d26770 2024-04-16T05:37:33.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.476+0000 7fbf6cff9640 1 -- 172.21.15.86:0/363788999 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf7c02cb40 con 0x55f424d26770 2024-04-16T05:37:33.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf6cff9640 1 -- 172.21.15.86:0/363788999 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbf7c034440 con 0x55f424d26770 2024-04-16T05:37:33.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf923ea740 10 client.16374 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:33.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf923ea740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:33.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf923ea740 20 client.16374 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:33.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf923ea740 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f424da6570 0x55f424da8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf923ea740 4 client.16374 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:37:33.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf923ea740 1 -- 172.21.15.86:0/363788999 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f424c92a10 con 0x55f424da6570 2024-04-16T05:37:33.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf8d4c5640 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f424da6570 0x55f424da8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf6cff9640 10 client.16374 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:33.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf8d4c5640 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f424da6570 0x55f424da8960 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e66cab123702f1e server_cookie=6709ab7abb8cf2d8 in_seq=0 out_seq=0 2024-04-16T05:37:33.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf6cff9640 1 -- 172.21.15.86:0/363788999 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f424c92a10 con 0x55f424da6570 2024-04-16T05:37:33.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf6cff9640 10 client.16374 handle_command_reply: tid=0 2024-04-16T05:37:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.477+0000 7fbf923ea740 1 client.16374 shutdown 2024-04-16T05:37:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 -- 172.21.15.86:0/363788999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 msgr2=0x55f424d35660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d35660 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c006010 tx=0x7fbf7c02c5d0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 -- 172.21.15.86:0/363788999 shutdown_connections 2024-04-16T05:37:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f424da6570 0x55f424da8960 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f424d2f350 0x55f424d35ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f424d26770 0x55f424d35660 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf6cff9640 0 client.16374 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 --2- 172.21.15.86:0/363788999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f424d25e50 0x55f424d35120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 -- 172.21.15.86:0/363788999 >> 172.21.15.86:0/363788999 conn(0x55f424d20580 msgr2=0x55f424d2c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 -- 172.21.15.86:0/363788999 shutdown_connections 2024-04-16T05:37:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 1 -- 172.21.15.86:0/363788999 wait complete. 2024-04-16T05:37:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf923ea740 20 client.16374 trim_cache size 0 max 16384 2024-04-16T05:37:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:37:33.489 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:37:33.490 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:37:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:37:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 -- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbf700a3d20 msgr2=0x7fbf700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbf700a3d20 0x7fbf700a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005d00 tx=0x7fbf800157f0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 -- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102240 msgr2=0x7fbf8819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102240 0x7fbf8819e670 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800ba50 tx=0x7fbf7800bf10 comp rx=0 tx=0).stop 2024-04-16T05:37:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 -- 172.21.15.86:0/895922274 shutdown_connections 2024-04-16T05:37:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbf700a3d20 0x7fbf700a61e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf881987e0 0x7fbf88198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf88107310 0x7fbf8819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 --2- 172.21.15.86:0/895922274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf88102240 0x7fbf8819e670 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 -- 172.21.15.86:0/895922274 >> 172.21.15.86:0/895922274 conn(0x7fbf88079450 msgr2=0x7fbf88100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 -- 172.21.15.86:0/895922274 shutdown_connections 2024-04-16T05:37:33.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.478+0000 7fbf8f729640 1 -- 172.21.15.86:0/895922274 wait complete. 2024-04-16T05:37:33.493 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:37:33.493 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:37:33.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.599+0000 7f77dee74640 1 -- 172.21.15.86:0/1747109297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d810c630 msgr2=0x7f77d810ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.599+0000 7f77dee74640 1 --2- 172.21.15.86:0/1747109297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d810c630 0x7f77d810ea60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009ca0 tx=0x7f77d002c180 comp rx=0 tx=0).stop 2024-04-16T05:37:33.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.599+0000 7f77dee74640 1 -- 172.21.15.86:0/1747109297 shutdown_connections 2024-04-16T05:37:33.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.599+0000 7f77dee74640 1 --2- 172.21.15.86:0/1747109297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d810c630 0x7f77d810ea60 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.599+0000 7f77dee74640 1 --2- 172.21.15.86:0/1747109297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d8103b80 0x7f77d810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.599+0000 7f77dee74640 1 --2- 172.21.15.86:0/1747109297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d8103260 0x7f77d8103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.599+0000 7f77dee74640 1 -- 172.21.15.86:0/1747109297 >> 172.21.15.86:0/1747109297 conn(0x7f77d8077280 msgr2=0x7f77d8077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.599+0000 7f77dee74640 1 -- 172.21.15.86:0/1747109297 shutdown_connections 2024-04-16T05:37:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.599+0000 7f77dee74640 1 -- 172.21.15.86:0/1747109297 wait complete. 2024-04-16T05:37:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77dee74640 1 Processor -- start 2024-04-16T05:37:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77dee74640 1 -- start start 2024-04-16T05:37:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77dee74640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d8103260 0x7f77d819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77dee74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d8103b80 0x7f77d819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77dee74640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d810c630 0x7f77d8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77dee74640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8112400 con 0x7f77d8103b80 2024-04-16T05:37:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77dee74640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8112280 con 0x7f77d8103260 2024-04-16T05:37:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77dee74640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8112580 con 0x7f77d810c630 2024-04-16T05:37:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d8103b80 0x7f77d819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d8103b80 0x7f77d819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56872/0 (socket says 172.21.15.86:56872) 2024-04-16T05:37:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77d7fff640 1 -- 172.21.15.86:0/1912197632 learned_addr learned my addr 172.21.15.86:0/1912197632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77d7fff640 1 -- 172.21.15.86:0/1912197632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d810c630 msgr2=0x7f77d8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77d7fff640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d810c630 0x7f77d8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77d7fff640 1 -- 172.21.15.86:0/1912197632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d8103260 msgr2=0x7f77d819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77d7fff640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d8103260 0x7f77d819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.600+0000 7f77d7fff640 1 -- 172.21.15.86:0/1912197632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d8198fe0 con 0x7f77d8103b80 2024-04-16T05:37:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.601+0000 7f77d7fff640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d8103b80 0x7f77d819eb50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f77c400e9f0 tx=0x7f77c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.601+0000 7f77d5ffb640 1 -- 172.21.15.86:0/1912197632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c400cd80 con 0x7f77d8103b80 2024-04-16T05:37:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.601+0000 7f77d5ffb640 1 -- 172.21.15.86:0/1912197632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c4010910 con 0x7f77d8103b80 2024-04-16T05:37:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.601+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d81992d0 con 0x7f77d8103b80 2024-04-16T05:37:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.601+0000 7f77d5ffb640 1 -- 172.21.15.86:0/1912197632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4018990 con 0x7f77d8103b80 2024-04-16T05:37:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.601+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d81a5590 con 0x7f77d8103b80 2024-04-16T05:37:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.602+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d8069eb0 con 0x7f77d8103b80 2024-04-16T05:37:33.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.603+0000 7f77d5ffb640 1 -- 172.21.15.86:0/1912197632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77c4010ac0 con 0x7f77d8103b80 2024-04-16T05:37:33.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.604+0000 7f77d5ffb640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77c80a3d20 0x7f77c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.604+0000 7f77dcc10640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77c80a3d20 0x7f77c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.604+0000 7f77d5ffb640 1 -- 172.21.15.86:0/1912197632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f77c40d0ff0 con 0x7f77d8103b80 2024-04-16T05:37:33.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.604+0000 7f77dcc10640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77c80a3d20 0x7f77c80a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009710 tx=0x7f77cc006c90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.609+0000 7f77d5ffb640 1 -- 172.21.15.86:0/1912197632 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77c409d320 con 0x7f77d8103b80 2024-04-16T05:37:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.826+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77d8107720 con 0x7f77d8103b80 2024-04-16T05:37:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.827+0000 7f77d5ffb640 1 -- 172.21.15.86:0/1912197632 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f77c409cc20 con 0x7f77d8103b80 2024-04-16T05:37:33.826 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:37:33.827 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:37:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:37:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77c80a3d20 msgr2=0x7f77c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77c80a3d20 0x7f77c80a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009710 tx=0x7f77cc006c90 comp rx=0 tx=0).stop 2024-04-16T05:37:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d8103b80 msgr2=0x7f77d819eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d8103b80 0x7f77d819eb50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f77c400e9f0 tx=0x7f77c400eeb0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 shutdown_connections 2024-04-16T05:37:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d810c630 0x7f77d8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d8103b80 0x7f77d819eb50 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f77c80a3d20 0x7f77c80a61e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 --2- 172.21.15.86:0/1912197632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d8103260 0x7f77d819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 >> 172.21.15.86:0/1912197632 conn(0x7f77d8077280 msgr2=0x7f77d80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 shutdown_connections 2024-04-16T05:37:33.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.829+0000 7f77dee74640 1 -- 172.21.15.86:0/1912197632 wait complete. 2024-04-16T05:37:33.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:37:33.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 -- 172.21.15.86:0/1559077591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664106200 msgr2=0x7f566410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 --2- 172.21.15.86:0/1559077591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664106200 0x7f566410ddd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f565c009ca0 tx=0x7f565c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 -- 172.21.15.86:0/1559077591 shutdown_connections 2024-04-16T05:37:33.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 --2- 172.21.15.86:0/1559077591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664106200 0x7f566410ddd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 --2- 172.21.15.86:0/1559077591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5664101ae0 0x7f5664105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 --2- 172.21.15.86:0/1559077591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5664101130 0x7f5664101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 -- 172.21.15.86:0/1559077591 >> 172.21.15.86:0/1559077591 conn(0x7f56640780b0 msgr2=0x7f56640784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 -- 172.21.15.86:0/1559077591 shutdown_connections 2024-04-16T05:37:33.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 -- 172.21.15.86:0/1559077591 wait complete. 2024-04-16T05:37:33.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 Processor -- start 2024-04-16T05:37:33.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.948+0000 7f566a48f640 1 -- start start 2024-04-16T05:37:33.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f566a48f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664101130 0x7f566410e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f566a48f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5664106200 0x7f566410eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f566a48f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566410f1e0 0x7f56641a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f566a48f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56641123d0 con 0x7f5664101130 2024-04-16T05:37:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f566a48f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5664112250 con 0x7f5664106200 2024-04-16T05:37:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f566a48f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5664112550 con 0x7f566410f1e0 2024-04-16T05:37:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5663fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664101130 0x7f566410e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5663fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664101130 0x7f566410e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56892/0 (socket says 172.21.15.86:56892) 2024-04-16T05:37:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5663fff640 1 -- 172.21.15.86:0/3643387754 learned_addr learned my addr 172.21.15.86:0/3643387754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f56637fe640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5664106200 0x7f566410eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5668a2c640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566410f1e0 0x7f56641a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5663fff640 1 -- 172.21.15.86:0/3643387754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566410f1e0 msgr2=0x7f56641a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5663fff640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566410f1e0 0x7f56641a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5663fff640 1 -- 172.21.15.86:0/3643387754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5664106200 msgr2=0x7f566410eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5663fff640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5664106200 0x7f566410eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5663fff640 1 -- 172.21.15.86:0/3643387754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56641a7140 con 0x7f5664101130 2024-04-16T05:37:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5668a2c640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566410f1e0 0x7f56641a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f5663fff640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664101130 0x7f566410e760 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f565400d770 tx=0x7f565400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f56617fa640 1 -- 172.21.15.86:0/3643387754 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654017070 con 0x7f5664101130 2024-04-16T05:37:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f56617fa640 1 -- 172.21.15.86:0/3643387754 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5654005500 con 0x7f5664101130 2024-04-16T05:37:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f566a48f640 1 -- 172.21.15.86:0/3643387754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56641a7430 con 0x7f5664101130 2024-04-16T05:37:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f56617fa640 1 -- 172.21.15.86:0/3643387754 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654004e00 con 0x7f5664101130 2024-04-16T05:37:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.949+0000 7f566a48f640 1 -- 172.21.15.86:0/3643387754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56641a7930 con 0x7f5664101130 2024-04-16T05:37:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.951+0000 7f56617fa640 1 -- 172.21.15.86:0/3643387754 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5654004fa0 con 0x7f5664101130 2024-04-16T05:37:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f566d07e740 1 Processor -- start 2024-04-16T05:37:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f566d07e740 1 -- start start 2024-04-16T05:37:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f56617fa640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f56500a3d50 0x7f56500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f566d07e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3045e60 0x55fae3046240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f566d07e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3046780 0x55fae304ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f566d07e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae3051750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f566d07e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fae2fab0e0 con 0x55fae304f360 2024-04-16T05:37:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f566d07e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fae2f48520 con 0x55fae3046780 2024-04-16T05:37:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f566d07e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fae2fa70a0 con 0x55fae3045e60 2024-04-16T05:37:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f5668a2c640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f56500a3d50 0x7f56500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f56617fa640 1 -- 172.21.15.86:0/3643387754 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f56540d0590 con 0x7f5664101130 2024-04-16T05:37:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f5668a2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae3051750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f5668a2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae3051750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56908/0 (socket says 172.21.15.86:56908) 2024-04-16T05:37:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f5668a2c640 1 -- 172.21.15.86:0/486473137 learned_addr learned my addr 172.21.15.86:0/486473137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.952+0000 7f56637fe640 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3046780 0x55fae304ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f5663fff640 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3045e60 0x55fae3046240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f56637fe640 1 -- 172.21.15.86:0/486473137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3045e60 msgr2=0x55fae3046240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f56637fe640 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3045e60 0x55fae3046240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f56637fe640 1 -- 172.21.15.86:0/486473137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 msgr2=0x55fae3051750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f56637fe640 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae3051750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f56637fe640 1 -- 172.21.15.86:0/486473137 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae2f708d0 con 0x55fae3046780 2024-04-16T05:37:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f5668a2c640 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae3051750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:37:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f5663fff640 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3045e60 0x55fae3046240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f5668a2c640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f56500a3d50 0x7f56500a6210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f565c036000 tx=0x7f565c0324f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f56637fe640 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3046780 0x55fae304ee20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f566406b7a0 tx=0x7f564c0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=746caba0c33ebf35 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f565a7fc640 1 -- 172.21.15.86:0/486473137 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c010e90 con 0x55fae3046780 2024-04-16T05:37:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f565a7fc640 1 -- 172.21.15.86:0/486473137 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c0035f0 con 0x55fae3046780 2024-04-16T05:37:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.953+0000 7f565a7fc640 1 -- 172.21.15.86:0/486473137 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c0116a0 con 0x55fae3046780 2024-04-16T05:37:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 -- 172.21.15.86:0/486473137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3046780 msgr2=0x55fae304ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3046780 0x55fae304ee20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f566406b7a0 tx=0x7f564c0103d0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 -- 172.21.15.86:0/486473137 shutdown_connections 2024-04-16T05:37:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae3051750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3046780 0x55fae304ee20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 --2- 172.21.15.86:0/486473137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3045e60 0x55fae3046240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 -- 172.21.15.86:0/486473137 >> 172.21.15.86:0/486473137 conn(0x55fae3040590 msgr2=0x55fae3040970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 -- 172.21.15.86:0/486473137 shutdown_connections 2024-04-16T05:37:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 -- 172.21.15.86:0/486473137 wait complete. 2024-04-16T05:37:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.954+0000 7f566d07e740 1 -- start start 2024-04-16T05:37:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f566d07e740 1 Processor -- start 2024-04-16T05:37:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f566d07e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae304c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f566d07e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae304c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f566d07e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 0x55fae304cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f566d07e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fae3052150 con 0x55fae3046780 2024-04-16T05:37:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f566d07e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fae2f644d0 con 0x55fae3045e60 2024-04-16T05:37:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f566d07e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fae30522d0 con 0x55fae304f360 2024-04-16T05:37:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f566d07e740 10 client.0 _ll_register_callbacks cb 0x55fae2f492c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae304c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae304c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56910/0 (socket says 172.21.15.86:56910) 2024-04-16T05:37:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56637fe640 1 -- 172.21.15.86:0/1811701025 learned_addr learned my addr 172.21.15.86:0/1811701025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f5663fff640 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae304c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f5668a2c640 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 0x55fae304cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56637fe640 1 -- 172.21.15.86:0/1811701025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 msgr2=0x55fae304cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56637fe640 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 0x55fae304cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56637fe640 1 -- 172.21.15.86:0/1811701025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 msgr2=0x55fae304c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56637fe640 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae304c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56637fe640 1 -- 172.21.15.86:0/1811701025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae2cbd3f0 con 0x55fae3046780 2024-04-16T05:37:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56637fe640 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae304c9a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f564c004650 tx=0x7f564c010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596e64e2d78ab4f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.955+0000 7f56437fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f56437fe640 1 -- 172.21.15.86:0/1811701025 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c004210 con 0x55fae3046780 2024-04-16T05:37:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f56437fe640 1 -- 172.21.15.86:0/1811701025 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f564c0037a0 con 0x55fae3046780 2024-04-16T05:37:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f566d07e740 1 -- 172.21.15.86:0/1811701025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564c0035f0 con 0x55fae3046780 2024-04-16T05:37:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f56437fe640 1 -- 172.21.15.86:0/1811701025 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f564c0039a0 con 0x55fae3046780 2024-04-16T05:37:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f566d07e740 10 client.16392 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f566d07e740 1 -- 172.21.15.86:0/1811701025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fae2d877d0 con 0x55fae3046780 2024-04-16T05:37:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f56437fe640 1 -- 172.21.15.86:0/1811701025 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c0035f0 con 0x55fae3046780 2024-04-16T05:37:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f56437fe640 1 -- 172.21.15.86:0/1811701025 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f564c004a20 con 0x55fae3046780 2024-04-16T05:37:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f566d07e740 10 client.16392 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f566d07e740 10 client.16392 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:37:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f566d07e740 20 client.16392 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f566d07e740 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55fae305f2f0 0x55fae30616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f566d07e740 4 client.16392 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f566d07e740 1 -- 172.21.15.86:0/1811701025 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fae2fb8b00 con 0x55fae305f2f0 2024-04-16T05:37:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.956+0000 7f5663fff640 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55fae305f2f0 0x55fae30616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.957+0000 7f56437fe640 10 client.16392 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:37:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.957+0000 7f5663fff640 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55fae305f2f0 0x55fae30616e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a419a7320ffb9ff8 server_cookie=dff2ae8d6b7c539b in_seq=0 out_seq=0 2024-04-16T05:37:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f56437fe640 1 -- 172.21.15.86:0/1811701025 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fae2fb8b00 con 0x55fae305f2f0 2024-04-16T05:37:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f56437fe640 10 client.16392 handle_command_reply: tid=0 2024-04-16T05:37:33.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 client.16392 shutdown 2024-04-16T05:37:33.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 -- 172.21.15.86:0/1811701025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 msgr2=0x55fae304c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae304c9a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f564c004650 tx=0x7f564c010ec0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 -- 172.21.15.86:0/1811701025 shutdown_connections 2024-04-16T05:37:33.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55fae305f2f0 0x55fae30616e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 0x55fae304cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f56437fe640 0 client.16392 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:37:33.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae304c9a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 --2- 172.21.15.86:0/1811701025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae304c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 -- 172.21.15.86:0/1811701025 >> 172.21.15.86:0/1811701025 conn(0x55fae3040590 msgr2=0x55fae30422e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 -- 172.21.15.86:0/1811701025 shutdown_connections 2024-04-16T05:37:33.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 1 -- 172.21.15.86:0/1811701025 wait complete. 2024-04-16T05:37:33.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.958+0000 7f566d07e740 20 client.16392 trim_cache size 0 max 16384 2024-04-16T05:37:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f566d07e740 1 Processor -- start 2024-04-16T05:37:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f566d07e740 1 -- start start 2024-04-16T05:37:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f566d07e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae304eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f566d07e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3046780 0x55fae3066e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f566d07e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae30673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f566d07e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b15f0 con 0x55fae304f360 2024-04-16T05:37:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f566d07e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b3130 con 0x55fae3045e60 2024-04-16T05:37:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f566d07e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b1cc0 con 0x55fae3046780 2024-04-16T05:37:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f5668a2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae30673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f5668a2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae30673b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56922/0 (socket says 172.21.15.86:56922) 2024-04-16T05:37:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f5668a2c640 1 -- 172.21.15.86:0/392055396 learned_addr learned my addr 172.21.15.86:0/392055396 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f56637fe640 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3046780 0x55fae3066e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f5663fff640 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae304eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f5668a2c640 1 -- 172.21.15.86:0/392055396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3046780 msgr2=0x55fae3066e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f5668a2c640 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3046780 0x55fae3066e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f5668a2c640 1 -- 172.21.15.86:0/392055396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 msgr2=0x55fae304eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.971+0000 7f5668a2c640 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae304eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f5668a2c640 1 -- 172.21.15.86:0/392055396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae304a9b0 con 0x55fae304f360 2024-04-16T05:37:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f5663fff640 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae304eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f5668a2c640 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae30673b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f565c006010 tx=0x7f565c039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f87f5d290191e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f5659ffb640 1 -- 172.21.15.86:0/392055396 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c039070 con 0x55fae304f360 2024-04-16T05:37:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f5659ffb640 1 -- 172.21.15.86:0/392055396 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f565c051070 con 0x55fae304f360 2024-04-16T05:37:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f5659ffb640 1 -- 172.21.15.86:0/392055396 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c004450 con 0x55fae304f360 2024-04-16T05:37:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f566d07e740 1 -- 172.21.15.86:0/392055396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 msgr2=0x55fae30673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f566d07e740 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae30673b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f565c006010 tx=0x7f565c039000 comp rx=0 tx=0).stop 2024-04-16T05:37:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f566d07e740 1 -- 172.21.15.86:0/392055396 shutdown_connections 2024-04-16T05:37:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f566d07e740 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae304f360 0x55fae30673b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f566d07e740 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae3046780 0x55fae3066e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f566d07e740 1 --2- 172.21.15.86:0/392055396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae304eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.972+0000 7f566d07e740 1 -- 172.21.15.86:0/392055396 >> 172.21.15.86:0/392055396 conn(0x55fae3040590 msgr2=0x55fae303b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 -- 172.21.15.86:0/392055396 shutdown_connections 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 -- 172.21.15.86:0/392055396 wait complete. 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 -- start start 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 Processor -- start 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae30524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae3052a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 0x55fae3052f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b2a60 con 0x55fae3046780 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b2390 con 0x55fae3045e60 2024-04-16T05:37:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56641af5b0 con 0x55fae304f360 2024-04-16T05:37:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.973+0000 7f566d07e740 10 client.0 _ll_register_callbacks cb 0x7f565c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae3052a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae3052a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56926/0 (socket says 172.21.15.86:56926) 2024-04-16T05:37:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56637fe640 1 -- 172.21.15.86:0/2126034915 learned_addr learned my addr 172.21.15.86:0/2126034915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f5663fff640 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae30524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f5668a2c640 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 0x55fae3052f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56637fe640 1 -- 172.21.15.86:0/2126034915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 msgr2=0x55fae3052f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56637fe640 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 0x55fae3052f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56637fe640 1 -- 172.21.15.86:0/2126034915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 msgr2=0x55fae30524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56637fe640 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae30524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56637fe640 1 -- 172.21.15.86:0/2126034915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae30cc130 con 0x55fae3046780 2024-04-16T05:37:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56637fe640 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae3052a20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f564c010410 tx=0x7f564c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1cc4fba3215824 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56437fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56437fe640 1 -- 172.21.15.86:0/2126034915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c0108d0 con 0x55fae3046780 2024-04-16T05:37:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56437fe640 1 -- 172.21.15.86:0/2126034915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f564c003b40 con 0x55fae3046780 2024-04-16T05:37:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f566d07e740 1 -- 172.21.15.86:0/2126034915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565c051070 con 0x55fae3046780 2024-04-16T05:37:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f56437fe640 1 -- 172.21.15.86:0/2126034915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f564c03f050 con 0x55fae3046780 2024-04-16T05:37:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f566d07e740 10 client.16398 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.974+0000 7f566d07e740 1 -- 172.21.15.86:0/2126034915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fae30cd3a0 con 0x55fae3046780 2024-04-16T05:37:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f56437fe640 1 -- 172.21.15.86:0/2126034915 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f565c051070 con 0x55fae3046780 2024-04-16T05:37:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f56437fe640 1 -- 172.21.15.86:0/2126034915 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f564c004680 con 0x55fae3046780 2024-04-16T05:37:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f566d07e740 10 client.16398 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f566d07e740 10 client.16398 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:37:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f566d07e740 20 client.16398 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f566d07e740 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55fae30cdab0 0x55fae30cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f566d07e740 4 client.16398 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f566d07e740 1 -- 172.21.15.86:0/2126034915 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fae2fb2970 con 0x55fae30cdab0 2024-04-16T05:37:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f5663fff640 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55fae30cdab0 0x55fae30cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f56437fe640 10 client.16398 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:37:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.975+0000 7f5663fff640 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55fae30cdab0 0x55fae30cfed0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83a1046f71d2a94b server_cookie=5f1467caed05d3c in_seq=0 out_seq=0 2024-04-16T05:37:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f56437fe640 1 -- 172.21.15.86:0/2126034915 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fae2fb2970 con 0x55fae30cdab0 2024-04-16T05:37:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f56437fe640 10 client.16398 handle_command_reply: tid=0 2024-04-16T05:37:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 client.16398 shutdown 2024-04-16T05:37:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 -- 172.21.15.86:0/2126034915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 msgr2=0x55fae3052a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae3052a20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f564c010410 tx=0x7f564c00b8a0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 -- 172.21.15.86:0/2126034915 shutdown_connections 2024-04-16T05:37:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55fae30cdab0 0x55fae30cfed0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fae304f360 0x55fae3052f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae3046780 0x55fae3052a20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 --2- 172.21.15.86:0/2126034915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fae3045e60 0x55fae30524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f56437fe640 0 client.16398 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:37:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 -- 172.21.15.86:0/2126034915 >> 172.21.15.86:0/2126034915 conn(0x55fae3040590 msgr2=0x55fae304acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 -- 172.21.15.86:0/2126034915 shutdown_connections 2024-04-16T05:37:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 1 -- 172.21.15.86:0/2126034915 wait complete. 2024-04-16T05:37:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566d07e740 20 client.16398 trim_cache size 0 max 16384 2024-04-16T05:37:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566a48f640 1 -- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f56500a3d50 msgr2=0x7f56500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566a48f640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f56500a3d50 0x7f56500a6210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f565c036000 tx=0x7f565c0324f0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566a48f640 1 -- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664101130 msgr2=0x7f566410e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.976+0000 7f566a48f640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664101130 0x7f566410e760 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f565400d770 tx=0x7f565400dc30 comp rx=0 tx=0).stop 2024-04-16T05:37:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.977+0000 7f566a48f640 1 -- 172.21.15.86:0/3643387754 shutdown_connections 2024-04-16T05:37:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.977+0000 7f566a48f640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566410f1e0 0x7f56641a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.977+0000 7f566a48f640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f56500a3d50 0x7f56500a6210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.977+0000 7f566a48f640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5664106200 0x7f566410eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.977+0000 7f566a48f640 1 --2- 172.21.15.86:0/3643387754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5664101130 0x7f566410e760 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.977+0000 7f566a48f640 1 -- 172.21.15.86:0/3643387754 >> 172.21.15.86:0/3643387754 conn(0x7f56640780b0 msgr2=0x7f56640fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.977+0000 7f566a48f640 1 -- 172.21.15.86:0/3643387754 shutdown_connections 2024-04-16T05:37:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:33.977+0000 7f566a48f640 1 -- 172.21.15.86:0/3643387754 wait complete. 2024-04-16T05:37:33.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:37:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 -- 172.21.15.86:0/3834128714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b0103bc0 msgr2=0x7f95b010c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 --2- 172.21.15.86:0/3834128714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b0103bc0 0x7f95b010c0a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009900 tx=0x7f95a402bd60 comp rx=0 tx=0).stop 2024-04-16T05:37:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 -- 172.21.15.86:0/3834128714 shutdown_connections 2024-04-16T05:37:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 --2- 172.21.15.86:0/3834128714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b010c5e0 0x7f95b010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 --2- 172.21.15.86:0/3834128714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b0103bc0 0x7f95b010c0a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 --2- 172.21.15.86:0/3834128714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b01032a0 0x7f95b0103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 -- 172.21.15.86:0/3834128714 >> 172.21.15.86:0/3834128714 conn(0x7f95b0077280 msgr2=0x7f95b0077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 -- 172.21.15.86:0/3834128714 shutdown_connections 2024-04-16T05:37:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 -- 172.21.15.86:0/3834128714 wait complete. 2024-04-16T05:37:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.098+0000 7f95b5959640 1 Processor -- start 2024-04-16T05:37:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b5959640 1 -- start start 2024-04-16T05:37:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b5959640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01032a0 0x7f95b019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b5959640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b0103bc0 0x7f95b019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b5959640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b010c5e0 0x7f95b01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b5959640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0112410 con 0x7f95b01032a0 2024-04-16T05:37:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b5959640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0112290 con 0x7f95b0103bc0 2024-04-16T05:37:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b5959640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0112590 con 0x7f95b010c5e0 2024-04-16T05:37:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01032a0 0x7f95b019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01032a0 0x7f95b019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56938/0 (socket says 172.21.15.86:56938) 2024-04-16T05:37:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95aeffd640 1 -- 172.21.15.86:0/4225259690 learned_addr learned my addr 172.21.15.86:0/4225259690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95ae7fc640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b0103bc0 0x7f95b019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95af7fe640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b010c5e0 0x7f95b01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95aeffd640 1 -- 172.21.15.86:0/4225259690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b010c5e0 msgr2=0x7f95b01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95aeffd640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b010c5e0 0x7f95b01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95aeffd640 1 -- 172.21.15.86:0/4225259690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b0103bc0 msgr2=0x7f95b019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95aeffd640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b0103bc0 0x7f95b019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95aeffd640 1 -- 172.21.15.86:0/4225259690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b0198f90 con 0x7f95b01032a0 2024-04-16T05:37:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95aeffd640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01032a0 0x7f95b019e610 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f95a000c970 tx=0x7f95a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b4957640 1 -- 172.21.15.86:0/4225259690 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0007bb0 con 0x7f95b01032a0 2024-04-16T05:37:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b4957640 1 -- 172.21.15.86:0/4225259690 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a0007d50 con 0x7f95b01032a0 2024-04-16T05:37:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.099+0000 7f95b5959640 1 -- 172.21.15.86:0/4225259690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b0199280 con 0x7f95b01032a0 2024-04-16T05:37:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.100+0000 7f95b4957640 1 -- 172.21.15.86:0/4225259690 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a000f450 con 0x7f95b01032a0 2024-04-16T05:37:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.100+0000 7f95b5959640 1 -- 172.21.15.86:0/4225259690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b01a5590 con 0x7f95b01032a0 2024-04-16T05:37:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95b4957640 1 -- 172.21.15.86:0/4225259690 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f95a0005020 con 0x7f95b01032a0 2024-04-16T05:37:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95b8629740 1 Processor -- start 2024-04-16T05:37:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95b8629740 1 -- start start 2024-04-16T05:37:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95b8629740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d691e60 0x56119d692240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95b8629740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 0x56119d69ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95b8629740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d69b360 0x56119d69d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95b8629740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56119d5f70e0 con 0x56119d691e60 2024-04-16T05:37:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95b8629740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56119d594520 con 0x56119d692780 2024-04-16T05:37:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95b8629740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56119d5f30a0 con 0x56119d69b360 2024-04-16T05:37:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d691e60 0x56119d692240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d691e60 0x56119d692240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56954/0 (socket says 172.21.15.86:56954) 2024-04-16T05:37:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.102+0000 7f95af7fe640 1 -- 172.21.15.86:0/1858771536 learned_addr learned my addr 172.21.15.86:0/1858771536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95aeffd640 1 --2- 172.21.15.86:0/1858771536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 0x56119d69ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95ae7fc640 1 --2- 172.21.15.86:0/1858771536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d69b360 0x56119d69d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95af7fe640 1 -- 172.21.15.86:0/1858771536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d69b360 msgr2=0x56119d69d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95af7fe640 1 --2- 172.21.15.86:0/1858771536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d69b360 0x56119d69d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95af7fe640 1 -- 172.21.15.86:0/1858771536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 msgr2=0x56119d69ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95af7fe640 1 --2- 172.21.15.86:0/1858771536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 0x56119d69ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95af7fe640 1 -- 172.21.15.86:0/1858771536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119d5bc8d0 con 0x56119d691e60 2024-04-16T05:37:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95b4957640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f95980a3d20 0x7f95980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95aeffd640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f95980a3d20 0x7f95980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95af7fe640 1 --2- 172.21.15.86:0/1858771536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d691e60 0x56119d692240 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f95a8008d40 tx=0x7f95a800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983b57265ec24c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f95b4957640 1 -- 172.21.15.86:0/4225259690 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f95a00d0980 con 0x7f95b01032a0 2024-04-16T05:37:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f9595ffb640 1 -- 172.21.15.86:0/1858771536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8014070 con 0x56119d691e60 2024-04-16T05:37:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f9595ffb640 1 -- 172.21.15.86:0/1858771536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a8003b70 con 0x56119d691e60 2024-04-16T05:37:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.103+0000 7f9595ffb640 1 -- 172.21.15.86:0/1858771536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8010750 con 0x56119d691e60 2024-04-16T05:37:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 -- 172.21.15.86:0/1858771536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d691e60 msgr2=0x56119d692240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95aeffd640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f95980a3d20 0x7f95980a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x56119d691930 tx=0x7f95a0087040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 --2- 172.21.15.86:0/1858771536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d691e60 0x56119d692240 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f95a8008d40 tx=0x7f95a800e720 comp rx=0 tx=0).stop 2024-04-16T05:37:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 -- 172.21.15.86:0/1858771536 shutdown_connections 2024-04-16T05:37:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 --2- 172.21.15.86:0/1858771536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d69b360 0x56119d69d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 --2- 172.21.15.86:0/1858771536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 0x56119d69ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 --2- 172.21.15.86:0/1858771536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d691e60 0x56119d692240 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 -- 172.21.15.86:0/1858771536 >> 172.21.15.86:0/1858771536 conn(0x56119d68c590 msgr2=0x56119d68c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 -- 172.21.15.86:0/1858771536 shutdown_connections 2024-04-16T05:37:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 -- 172.21.15.86:0/1858771536 wait complete. 2024-04-16T05:37:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 -- start start 2024-04-16T05:37:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.104+0000 7f95b8629740 1 Processor -- start 2024-04-16T05:37:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95b8629740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d691e60 0x56119d6983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95b8629740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d692780 0x56119d698900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95b8629740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d698e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95b8629740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56119d69dfa0 con 0x56119d69b360 2024-04-16T05:37:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95b8629740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56119d5b04d0 con 0x56119d691e60 2024-04-16T05:37:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95b8629740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56119d69e120 con 0x56119d692780 2024-04-16T05:37:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95b8629740 10 client.0 _ll_register_callbacks cb 0x56119d5952c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d698e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d698e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56968/0 (socket says 172.21.15.86:56968) 2024-04-16T05:37:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 -- 172.21.15.86:0/122405602 learned_addr learned my addr 172.21.15.86:0/122405602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d691e60 0x56119d6983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95ae7fc640 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d692780 0x56119d698900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 -- 172.21.15.86:0/122405602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d692780 msgr2=0x56119d698900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d692780 0x56119d698900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 -- 172.21.15.86:0/122405602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d691e60 msgr2=0x56119d6983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d691e60 0x56119d6983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 -- 172.21.15.86:0/122405602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119d3093f0 con 0x56119d69b360 2024-04-16T05:37:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95af7fe640 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d698e40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005cc0 tx=0x7f95a80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccf9d665191c1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f958effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f958effd640 1 -- 172.21.15.86:0/122405602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8014040 con 0x56119d69b360 2024-04-16T05:37:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f958effd640 1 -- 172.21.15.86:0/122405602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f95a8010930 con 0x56119d69b360 2024-04-16T05:37:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.105+0000 7f95b8629740 1 -- 172.21.15.86:0/122405602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a8003b70 con 0x56119d69b360 2024-04-16T05:37:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f958effd640 1 -- 172.21.15.86:0/122405602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95a8004b60 con 0x56119d69b360 2024-04-16T05:37:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f95b8629740 10 client.16410 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f95b8629740 1 -- 172.21.15.86:0/122405602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56119d3d37d0 con 0x56119d69b360 2024-04-16T05:37:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f958effd640 1 -- 172.21.15.86:0/122405602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a8003b70 con 0x56119d69b360 2024-04-16T05:37:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f958effd640 1 -- 172.21.15.86:0/122405602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f95a8026040 con 0x56119d69b360 2024-04-16T05:37:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f95b8629740 10 client.16410 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f95b8629740 10 client.16410 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:37:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f95b8629740 20 client.16410 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f95b8629740 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56119d6ab230 0x56119d6ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f95b8629740 4 client.16410 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f95b8629740 1 -- 172.21.15.86:0/122405602 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56119d604b00 con 0x56119d6ab230 2024-04-16T05:37:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.106+0000 7f95ae7fc640 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56119d6ab230 0x56119d6ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.107+0000 7f958effd640 10 client.16410 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:37:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.107+0000 7f95ae7fc640 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56119d6ab230 0x56119d6ad620 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66dc6c9d074d6e23 server_cookie=47dabe7334b1368b in_seq=0 out_seq=0 2024-04-16T05:37:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.107+0000 7f958effd640 1 -- 172.21.15.86:0/122405602 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56119d604b00 con 0x56119d6ab230 2024-04-16T05:37:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.107+0000 7f958effd640 10 client.16410 handle_command_reply: tid=0 2024-04-16T05:37:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.107+0000 7f95b8629740 1 client.16410 shutdown 2024-04-16T05:37:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.107+0000 7f95b8629740 1 -- 172.21.15.86:0/122405602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 msgr2=0x56119d698e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.107+0000 7f95b8629740 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d698e40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005cc0 tx=0x7f95a80048a0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f95b8629740 1 -- 172.21.15.86:0/122405602 shutdown_connections 2024-04-16T05:37:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f95b8629740 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d698e40 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f95b8629740 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56119d6ab230 0x56119d6ad620 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f95b8629740 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d692780 0x56119d698900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f958effd640 0 client.16410 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:37:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f95b8629740 1 --2- 172.21.15.86:0/122405602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d691e60 0x56119d6983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f95b8629740 1 -- 172.21.15.86:0/122405602 >> 172.21.15.86:0/122405602 conn(0x56119d68c590 msgr2=0x56119d68e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f95b8629740 1 -- 172.21.15.86:0/122405602 shutdown_connections 2024-04-16T05:37:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f95b8629740 1 -- 172.21.15.86:0/122405602 wait complete. 2024-04-16T05:37:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.108+0000 7f95b8629740 20 client.16410 trim_cache size 0 max 16384 2024-04-16T05:37:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95b8629740 1 Processor -- start 2024-04-16T05:37:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95b8629740 1 -- start start 2024-04-16T05:37:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95b8629740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d691e60 0x56119d69aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95b8629740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 0x56119d6a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95b8629740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d6b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95b8629740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01af460 con 0x56119d69b360 2024-04-16T05:37:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95b8629740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01b0fa0 con 0x56119d692780 2024-04-16T05:37:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95b8629740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01afb30 con 0x56119d691e60 2024-04-16T05:37:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d6b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d6b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56978/0 (socket says 172.21.15.86:56978) 2024-04-16T05:37:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95af7fe640 1 -- 172.21.15.86:0/3349345202 learned_addr learned my addr 172.21.15.86:0/3349345202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95ae7fc640 1 --2- 172.21.15.86:0/3349345202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 0x56119d6a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95af7fe640 1 -- 172.21.15.86:0/3349345202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d691e60 msgr2=0x56119d69aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95af7fe640 1 --2- 172.21.15.86:0/3349345202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d691e60 0x56119d69aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95af7fe640 1 -- 172.21.15.86:0/3349345202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 msgr2=0x56119d6a6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95af7fe640 1 --2- 172.21.15.86:0/3349345202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 0x56119d6a6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95af7fe640 1 -- 172.21.15.86:0/3349345202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119d687c30 con 0x56119d69b360 2024-04-16T05:37:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.121+0000 7f95af7fe640 1 --2- 172.21.15.86:0/3349345202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d6b2d60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f95a80089e0 tx=0x7f95a8010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e163312ee8fd9c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95957fa640 1 -- 172.21.15.86:0/3349345202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8014040 con 0x56119d69b360 2024-04-16T05:37:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95957fa640 1 -- 172.21.15.86:0/3349345202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a800ec20 con 0x56119d69b360 2024-04-16T05:37:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95957fa640 1 -- 172.21.15.86:0/3349345202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8011900 con 0x56119d69b360 2024-04-16T05:37:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 -- 172.21.15.86:0/3349345202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 msgr2=0x56119d6b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 --2- 172.21.15.86:0/3349345202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d6b2d60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f95a80089e0 tx=0x7f95a8010dd0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 -- 172.21.15.86:0/3349345202 shutdown_connections 2024-04-16T05:37:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 --2- 172.21.15.86:0/3349345202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d69b360 0x56119d6b2d60 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 --2- 172.21.15.86:0/3349345202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d692780 0x56119d6a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 --2- 172.21.15.86:0/3349345202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d691e60 0x56119d69aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 -- 172.21.15.86:0/3349345202 >> 172.21.15.86:0/3349345202 conn(0x56119d68c590 msgr2=0x56119d68b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 -- 172.21.15.86:0/3349345202 shutdown_connections 2024-04-16T05:37:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 -- 172.21.15.86:0/3349345202 wait complete. 2024-04-16T05:37:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.122+0000 7f95b8629740 1 -- start start 2024-04-16T05:37:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95b8629740 1 Processor -- start 2024-04-16T05:37:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95b8629740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d691e60 0x56119d6a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95b8629740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d692780 0x56119d69eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95b8629740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d69b360 0x56119d69f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95b8629740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01b08d0 con 0x56119d692780 2024-04-16T05:37:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95b8629740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01b0200 con 0x56119d69b360 2024-04-16T05:37:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95b8629740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01ad420 con 0x56119d691e60 2024-04-16T05:37:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95b8629740 10 client.0 _ll_register_callbacks cb 0x7f95a8008b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d692780 0x56119d69eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d692780 0x56119d69eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56990/0 (socket says 172.21.15.86:56990) 2024-04-16T05:37:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95ae7fc640 1 -- 172.21.15.86:0/4125582680 learned_addr learned my addr 172.21.15.86:0/4125582680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95af7fe640 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d691e60 0x56119d6a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95af7fe640 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d69b360 0x56119d69f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95ae7fc640 1 -- 172.21.15.86:0/4125582680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d691e60 msgr2=0x56119d6a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95ae7fc640 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d691e60 0x56119d6a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95ae7fc640 1 -- 172.21.15.86:0/4125582680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d69b360 msgr2=0x56119d69f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95ae7fc640 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d69b360 0x56119d69f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95ae7fc640 1 -- 172.21.15.86:0/4125582680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119d69f790 con 0x56119d692780 2024-04-16T05:37:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95af7fe640 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d69b360 0x56119d69f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f95ae7fc640 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d692780 0x56119d69eb10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f95a4005ab0 tx=0x7f95a4004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f7697d4a833221 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.123+0000 7f958effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f958effd640 1 -- 172.21.15.86:0/4125582680 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a404a070 con 0x56119d692780 2024-04-16T05:37:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f958effd640 1 -- 172.21.15.86:0/4125582680 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f95a4046b50 con 0x56119d692780 2024-04-16T05:37:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f95b8629740 1 -- 172.21.15.86:0/4125582680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a800ec20 con 0x56119d692780 2024-04-16T05:37:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f958effd640 1 -- 172.21.15.86:0/4125582680 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95a4056050 con 0x56119d692780 2024-04-16T05:37:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f95b8629740 10 client.16416 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f95b8629740 1 -- 172.21.15.86:0/4125582680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56119d711550 con 0x56119d692780 2024-04-16T05:37:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f958effd640 1 -- 172.21.15.86:0/4125582680 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a800ec20 con 0x56119d692780 2024-04-16T05:37:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f958effd640 1 -- 172.21.15.86:0/4125582680 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f95a4004b30 con 0x56119d692780 2024-04-16T05:37:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f95b8629740 10 client.16416 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f95b8629740 10 client.16416 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:37:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f95b8629740 20 client.16416 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f95b8629740 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56119d711d00 0x56119d714120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f95b8629740 4 client.16416 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.124+0000 7f95b8629740 1 -- 172.21.15.86:0/4125582680 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56119d5fe970 con 0x56119d711d00 2024-04-16T05:37:34.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95af7fe640 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56119d711d00 0x56119d714120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95af7fe640 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56119d711d00 0x56119d714120 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52cc824295c70a40 server_cookie=1c5037c83bbc1827 in_seq=0 out_seq=0 2024-04-16T05:37:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f958effd640 10 client.16416 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:37:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f958effd640 1 -- 172.21.15.86:0/4125582680 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56119d5fe970 con 0x56119d711d00 2024-04-16T05:37:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f958effd640 10 client.16416 handle_command_reply: tid=0 2024-04-16T05:37:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 client.16416 shutdown 2024-04-16T05:37:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 -- 172.21.15.86:0/4125582680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d692780 msgr2=0x56119d69eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d692780 0x56119d69eb10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f95a4005ab0 tx=0x7f95a4004f10 comp rx=0 tx=0).stop 2024-04-16T05:37:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 -- 172.21.15.86:0/4125582680 shutdown_connections 2024-04-16T05:37:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56119d69b360 0x56119d69f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56119d692780 0x56119d69eb10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56119d711d00 0x56119d714120 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 --2- 172.21.15.86:0/4125582680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56119d691e60 0x56119d6a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f958effd640 0 client.16416 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:37:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 -- 172.21.15.86:0/4125582680 >> 172.21.15.86:0/4125582680 conn(0x56119d68c590 msgr2=0x56119d698380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 -- 172.21.15.86:0/4125582680 shutdown_connections 2024-04-16T05:37:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 1 -- 172.21.15.86:0/4125582680 wait complete. 2024-04-16T05:37:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.125+0000 7f95b8629740 20 client.16416 trim_cache size 0 max 16384 2024-04-16T05:37:34.139 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 -- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f95980a3d20 msgr2=0x7f95980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f95980a3d20 0x7f95980a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x56119d691930 tx=0x7f95a0087040 comp rx=0 tx=0).stop 2024-04-16T05:37:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 -- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01032a0 msgr2=0x7f95b019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01032a0 0x7f95b019e610 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f95a000c970 tx=0x7f95a000ce30 comp rx=0 tx=0).stop 2024-04-16T05:37:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 -- 172.21.15.86:0/4225259690 shutdown_connections 2024-04-16T05:37:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b010c5e0 0x7f95b01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f95980a3d20 0x7f95980a61e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b0103bc0 0x7f95b019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 --2- 172.21.15.86:0/4225259690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01032a0 0x7f95b019e610 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 -- 172.21.15.86:0/4225259690 >> 172.21.15.86:0/4225259690 conn(0x7f95b0077280 msgr2=0x7f95b00fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 -- 172.21.15.86:0/4225259690 shutdown_connections 2024-04-16T05:37:34.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.126+0000 7f95b5959640 1 -- 172.21.15.86:0/4225259690 wait complete. 2024-04-16T05:37:34.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:37:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 -- 172.21.15.86:0/3342437856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014101af0 msgr2=0x7fe014105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3342437856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014101af0 0x7fe014105b30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe010009900 tx=0x7fe01002bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 -- 172.21.15.86:0/3342437856 shutdown_connections 2024-04-16T05:37:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3342437856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe014106210 0x7fe01410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3342437856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014101af0 0x7fe014105b30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3342437856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe014101140 0x7fe014101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 -- 172.21.15.86:0/3342437856 >> 172.21.15.86:0/3342437856 conn(0x7fe014077280 msgr2=0x7fe014077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 -- 172.21.15.86:0/3342437856 shutdown_connections 2024-04-16T05:37:34.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 -- 172.21.15.86:0/3342437856 wait complete. 2024-04-16T05:37:34.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.251+0000 7fe01c20d640 1 Processor -- start 2024-04-16T05:37:34.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01c20d640 1 -- start start 2024-04-16T05:37:34.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01c20d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe014101140 0x7fe0141a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01c20d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014106210 0x7fe0141a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01c20d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe01419a8e0 0x7fe01419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01c20d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141123e0 con 0x7fe014106210 2024-04-16T05:37:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01c20d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe014112260 con 0x7fe01419a8e0 2024-04-16T05:37:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01c20d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe014112560 con 0x7fe014101140 2024-04-16T05:37:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe0197a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014106210 0x7fe0141a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe0197a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014106210 0x7fe0141a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57018/0 (socket says 172.21.15.86:57018) 2024-04-16T05:37:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe0197a8640 1 -- 172.21.15.86:0/3888059566 learned_addr learned my addr 172.21.15.86:0/3888059566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe01419a8e0 0x7fe01419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe019fa9640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe014101140 0x7fe0141a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe0197a8640 1 -- 172.21.15.86:0/3888059566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe014101140 msgr2=0x7fe0141a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe0197a8640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe014101140 0x7fe0141a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe0197a8640 1 -- 172.21.15.86:0/3888059566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe01419a8e0 msgr2=0x7fe01419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe0197a8640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe01419a8e0 0x7fe01419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe0197a8640 1 -- 172.21.15.86:0/3888059566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01419b400 con 0x7fe014106210 2024-04-16T05:37:34.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe01419a8e0 0x7fe01419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:34.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe0197a8640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014106210 0x7fe0141a0c70 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe010036040 tx=0x7fe01002ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe00affd640 1 -- 172.21.15.86:0/3888059566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010041070 con 0x7fe014106210 2024-04-16T05:37:34.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe00affd640 1 -- 172.21.15.86:0/3888059566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0100054e0 con 0x7fe014106210 2024-04-16T05:37:34.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01c20d640 1 -- 172.21.15.86:0/3888059566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01419b690 con 0x7fe014106210 2024-04-16T05:37:34.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe00affd640 1 -- 172.21.15.86:0/3888059566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010034440 con 0x7fe014106210 2024-04-16T05:37:34.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.252+0000 7fe01c20d640 1 -- 172.21.15.86:0/3888059566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0141a7880 con 0x7fe014106210 2024-04-16T05:37:34.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.254+0000 7fe00affd640 1 -- 172.21.15.86:0/3888059566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe010044070 con 0x7fe014106210 2024-04-16T05:37:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe01edf3740 1 Processor -- start 2024-04-16T05:37:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe01edf3740 1 -- start start 2024-04-16T05:37:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 0x55ebd593f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5947e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd594a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe01edf3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ebd58a40e0 con 0x55ebd5948360 2024-04-16T05:37:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe01edf3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ebd5841520 con 0x55ebd593ee60 2024-04-16T05:37:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe00affd640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe0000a3d20 0x7fe0000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe01edf3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ebd58a00a0 con 0x55ebd593f780 2024-04-16T05:37:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe0197a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd594a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe0197a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd594a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57022/0 (socket says 172.21.15.86:57022) 2024-04-16T05:37:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.255+0000 7fe0197a8640 1 -- 172.21.15.86:0/3489454199 learned_addr learned my addr 172.21.15.86:0/3489454199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe0000a3d20 0x7fe0000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe00affd640 1 -- 172.21.15.86:0/3888059566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe0100f2760 con 0x7fe014106210 2024-04-16T05:37:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe019fa9640 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5947e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 0x55ebd593f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe0197a8640 1 -- 172.21.15.86:0/3489454199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 msgr2=0x55ebd5947e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe0197a8640 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5947e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe0197a8640 1 -- 172.21.15.86:0/3489454199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 msgr2=0x55ebd593f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe0197a8640 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 0x55ebd593f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe0197a8640 1 -- 172.21.15.86:0/3489454199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd58698d0 con 0x55ebd5948360 2024-04-16T05:37:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe019fa9640 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5947e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe0000a3d20 0x7fe0000a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe00c005c40 tx=0x7fe00c009d50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fe0197a8640 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd594a750 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe0100efcb0 tx=0x7fe01008cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02d4f481cad84b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fdffffff640 1 -- 172.21.15.86:0/3489454199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01004b070 con 0x55ebd5948360 2024-04-16T05:37:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fdffffff640 1 -- 172.21.15.86:0/3489454199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe010005690 con 0x55ebd5948360 2024-04-16T05:37:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.256+0000 7fdffffff640 1 -- 172.21.15.86:0/3489454199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010004c50 con 0x55ebd5948360 2024-04-16T05:37:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 -- 172.21.15.86:0/3489454199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 msgr2=0x55ebd594a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd594a750 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe0100efcb0 tx=0x7fe01008cee0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 -- 172.21.15.86:0/3489454199 shutdown_connections 2024-04-16T05:37:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd594a750 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5947e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3489454199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 0x55ebd593f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 -- 172.21.15.86:0/3489454199 >> 172.21.15.86:0/3489454199 conn(0x55ebd5939590 msgr2=0x55ebd5939970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 -- 172.21.15.86:0/3489454199 shutdown_connections 2024-04-16T05:37:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 -- 172.21.15.86:0/3489454199 wait complete. 2024-04-16T05:37:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 -- start start 2024-04-16T05:37:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.257+0000 7fe01edf3740 1 Processor -- start 2024-04-16T05:37:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593ee60 0x55ebd5945320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5945860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 0x55ebd5945da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe01edf3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ebd594af10 con 0x55ebd593ee60 2024-04-16T05:37:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe01edf3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ebd585d4d0 con 0x55ebd5948360 2024-04-16T05:37:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe01edf3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ebd594b090 con 0x55ebd593f780 2024-04-16T05:37:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe01edf3740 10 client.0 _ll_register_callbacks cb 0x55ebd58422c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe019fa9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593ee60 0x55ebd5945320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe019fa9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593ee60 0x55ebd5945320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57038/0 (socket says 172.21.15.86:57038) 2024-04-16T05:37:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe019fa9640 1 -- 172.21.15.86:0/2707267755 learned_addr learned my addr 172.21.15.86:0/2707267755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 0x55ebd5945da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe0197a8640 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5945860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe019fa9640 1 -- 172.21.15.86:0/2707267755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 msgr2=0x55ebd5945860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe019fa9640 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5945860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe019fa9640 1 -- 172.21.15.86:0/2707267755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 msgr2=0x55ebd5945da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe019fa9640 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 0x55ebd5945da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe019fa9640 1 -- 172.21.15.86:0/2707267755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd55b63f0 con 0x55ebd593ee60 2024-04-16T05:37:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 0x55ebd5945da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe0197a8640 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5945860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fe019fa9640 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593ee60 0x55ebd5945320 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe004010450 tx=0x7fe004010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc28360b7770c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fdffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.258+0000 7fdffd7fa640 1 -- 172.21.15.86:0/2707267755 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004005220 con 0x55ebd593ee60 2024-04-16T05:37:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fdffd7fa640 1 -- 172.21.15.86:0/2707267755 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe0040053c0 con 0x55ebd593ee60 2024-04-16T05:37:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fe01edf3740 1 -- 172.21.15.86:0/2707267755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ebd58b1b00 con 0x55ebd593ee60 2024-04-16T05:37:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fdffd7fa640 1 -- 172.21.15.86:0/2707267755 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe004024050 con 0x55ebd593ee60 2024-04-16T05:37:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fe01edf3740 10 client.16428 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fe01edf3740 1 -- 172.21.15.86:0/2707267755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ebd56807d0 con 0x55ebd593ee60 2024-04-16T05:37:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fdffd7fa640 1 -- 172.21.15.86:0/2707267755 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ebd58b1b00 con 0x55ebd593ee60 2024-04-16T05:37:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fdffd7fa640 1 -- 172.21.15.86:0/2707267755 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe00400f040 con 0x55ebd593ee60 2024-04-16T05:37:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fe01edf3740 10 client.16428 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fe01edf3740 10 client.16428 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:37:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fe01edf3740 20 client.16428 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ebd59580a0 0x55ebd595a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fe01edf3740 4 client.16428 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.259+0000 7fe01edf3740 1 -- 172.21.15.86:0/2707267755 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebd58ab970 con 0x55ebd59580a0 2024-04-16T05:37:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.260+0000 7fe0197a8640 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ebd59580a0 0x55ebd595a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.260+0000 7fdffd7fa640 10 client.16428 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:37:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.260+0000 7fe0197a8640 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ebd59580a0 0x55ebd595a540 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=961a18a03e27a568 server_cookie=84977458d4533ea1 in_seq=0 out_seq=0 2024-04-16T05:37:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fdffd7fa640 1 -- 172.21.15.86:0/2707267755 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebd58ab970 con 0x55ebd59580a0 2024-04-16T05:37:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fdffd7fa640 10 client.16428 handle_command_reply: tid=0 2024-04-16T05:37:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 client.16428 shutdown 2024-04-16T05:37:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 -- 172.21.15.86:0/2707267755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593ee60 msgr2=0x55ebd5945320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593ee60 0x55ebd5945320 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe004010450 tx=0x7fe004010910 comp rx=0 tx=0).stop 2024-04-16T05:37:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 -- 172.21.15.86:0/2707267755 shutdown_connections 2024-04-16T05:37:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 0x55ebd5945da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5945860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ebd59580a0 0x55ebd595a540 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2707267755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593ee60 0x55ebd5945320 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fdffd7fa640 0 client.16428 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:37:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 -- 172.21.15.86:0/2707267755 >> 172.21.15.86:0/2707267755 conn(0x55ebd5939590 msgr2=0x55ebd593b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 -- 172.21.15.86:0/2707267755 shutdown_connections 2024-04-16T05:37:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 1 -- 172.21.15.86:0/2707267755 wait complete. 2024-04-16T05:37:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.261+0000 7fe01edf3740 20 client.16428 trim_cache size 0 max 16384 2024-04-16T05:37:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.274+0000 7fe01edf3740 1 Processor -- start 2024-04-16T05:37:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.274+0000 7fe01edf3740 1 -- start start 2024-04-16T05:37:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.274+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 0x55ebd5953c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.274+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5944c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.274+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd593b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.274+0000 7fe01edf3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b16e0 con 0x55ebd5948360 2024-04-16T05:37:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.274+0000 7fe01edf3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b3220 con 0x55ebd593ee60 2024-04-16T05:37:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.274+0000 7fe01edf3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b1db0 con 0x55ebd593f780 2024-04-16T05:37:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.274+0000 7fe01a7aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd593b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe01a7aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd593b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57040/0 (socket says 172.21.15.86:57040) 2024-04-16T05:37:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe01a7aa640 1 -- 172.21.15.86:0/2090233321 learned_addr learned my addr 172.21.15.86:0/2090233321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe019fa9640 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 0x55ebd5953c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe0197a8640 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5944c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe01a7aa640 1 -- 172.21.15.86:0/2090233321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 msgr2=0x55ebd5944c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5944c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe01a7aa640 1 -- 172.21.15.86:0/2090233321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 msgr2=0x55ebd5953c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 0x55ebd5953c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe01a7aa640 1 -- 172.21.15.86:0/2090233321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd593f5b0 con 0x55ebd5948360 2024-04-16T05:37:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe019fa9640 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 0x55ebd5953c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd593b200 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009830 tx=0x7fe00c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470b781367c3ff9c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fdfff7fe640 1 -- 172.21.15.86:0/2090233321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c01e5f0 con 0x55ebd5948360 2024-04-16T05:37:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fdfff7fe640 1 -- 172.21.15.86:0/2090233321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe00c01ec70 con 0x55ebd5948360 2024-04-16T05:37:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.275+0000 7fdfff7fe640 1 -- 172.21.15.86:0/2090233321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c02cd80 con 0x55ebd5948360 2024-04-16T05:37:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 -- 172.21.15.86:0/2090233321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 msgr2=0x55ebd593b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd593b200 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009830 tx=0x7fe00c004ab0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 -- 172.21.15.86:0/2090233321 shutdown_connections 2024-04-16T05:37:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd5948360 0x55ebd593b200 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593f780 0x55ebd5944c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 --2- 172.21.15.86:0/2090233321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd593ee60 0x55ebd5953c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 -- 172.21.15.86:0/2090233321 >> 172.21.15.86:0/2090233321 conn(0x55ebd5939590 msgr2=0x55ebd5937ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 -- 172.21.15.86:0/2090233321 shutdown_connections 2024-04-16T05:37:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 -- 172.21.15.86:0/2090233321 wait complete. 2024-04-16T05:37:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 -- start start 2024-04-16T05:37:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 Processor -- start 2024-04-16T05:37:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593ee60 0x55ebd594b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593f780 0x55ebd594b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 0x55ebd59b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b2b50 con 0x55ebd593f780 2024-04-16T05:37:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b2480 con 0x55ebd5948360 2024-04-16T05:37:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.276+0000 7fe01edf3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141af6a0 con 0x55ebd593ee60 2024-04-16T05:37:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe01edf3740 10 client.0 _ll_register_callbacks cb 0x7fe00400ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe0197a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593f780 0x55ebd594b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe0197a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593f780 0x55ebd594b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57056/0 (socket says 172.21.15.86:57056) 2024-04-16T05:37:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe0197a8640 1 -- 172.21.15.86:0/3003309148 learned_addr learned my addr 172.21.15.86:0/3003309148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe019fa9640 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593ee60 0x55ebd594b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe01a7aa640 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 0x55ebd59b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe0197a8640 1 -- 172.21.15.86:0/3003309148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593ee60 msgr2=0x55ebd594b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe0197a8640 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593ee60 0x55ebd594b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe0197a8640 1 -- 172.21.15.86:0/3003309148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 msgr2=0x55ebd59b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe0197a8640 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 0x55ebd59b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe0197a8640 1 -- 172.21.15.86:0/3003309148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd59b6c00 con 0x55ebd593f780 2024-04-16T05:37:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe019fa9640 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593ee60 0x55ebd594b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe0197a8640 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593f780 0x55ebd594b900 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe0100baf00 tx=0x7fe010000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d7c32b214078e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fdffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fdffd7fa640 1 -- 172.21.15.86:0/3003309148 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0100a3e60 con 0x55ebd593f780 2024-04-16T05:37:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fdffd7fa640 1 -- 172.21.15.86:0/3003309148 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe0100035f0 con 0x55ebd593f780 2024-04-16T05:37:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe01edf3740 1 -- 172.21.15.86:0/3003309148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe00c01ec70 con 0x55ebd593f780 2024-04-16T05:37:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fdffd7fa640 1 -- 172.21.15.86:0/3003309148 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe01004b040 con 0x55ebd593f780 2024-04-16T05:37:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe01edf3740 10 client.16434 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fe01edf3740 1 -- 172.21.15.86:0/3003309148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ebd59bdf70 con 0x55ebd593f780 2024-04-16T05:37:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.277+0000 7fdffd7fa640 1 -- 172.21.15.86:0/3003309148 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe00c01ec70 con 0x55ebd593f780 2024-04-16T05:37:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fdffd7fa640 1 -- 172.21.15.86:0/3003309148 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe0100f23a0 con 0x55ebd593f780 2024-04-16T05:37:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fe01edf3740 10 client.16434 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fe01edf3740 10 client.16434 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:37:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fe01edf3740 20 client.16434 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ebd59be500 0x55ebd59c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fe01edf3740 4 client.16434 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fe01edf3740 1 -- 172.21.15.86:0/3003309148 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebd58276b0 con 0x55ebd59be500 2024-04-16T05:37:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fe019fa9640 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ebd59be500 0x55ebd59c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fe019fa9640 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ebd59be500 0x55ebd59c0920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1b7888949ef9e53 server_cookie=35c5194366e79277 in_seq=0 out_seq=0 2024-04-16T05:37:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.278+0000 7fdffd7fa640 10 client.16434 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:37:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fdffd7fa640 1 -- 172.21.15.86:0/3003309148 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebd58276b0 con 0x55ebd59be500 2024-04-16T05:37:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fdffd7fa640 10 client.16434 handle_command_reply: tid=0 2024-04-16T05:37:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 client.16434 shutdown 2024-04-16T05:37:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 -- 172.21.15.86:0/3003309148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593f780 msgr2=0x55ebd594b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593f780 0x55ebd594b900 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe0100baf00 tx=0x7fe010000bc0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 -- 172.21.15.86:0/3003309148 shutdown_connections 2024-04-16T05:37:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ebd5948360 0x55ebd59b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55ebd59be500 0x55ebd59c0920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebd593f780 0x55ebd594b900 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 --2- 172.21.15.86:0/3003309148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebd593ee60 0x55ebd594b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fdffd7fa640 0 client.16434 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:37:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 -- 172.21.15.86:0/3003309148 >> 172.21.15.86:0/3003309148 conn(0x55ebd5939590 msgr2=0x55ebd5945c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 -- 172.21.15.86:0/3003309148 shutdown_connections 2024-04-16T05:37:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 1 -- 172.21.15.86:0/3003309148 wait complete. 2024-04-16T05:37:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01edf3740 20 client.16434 trim_cache size 0 max 16384 2024-04-16T05:37:34.289 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 -- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe0000a3d20 msgr2=0x7fe0000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe0000a3d20 0x7fe0000a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe00c005c40 tx=0x7fe00c009d50 comp rx=0 tx=0).stop 2024-04-16T05:37:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 -- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014106210 msgr2=0x7fe0141a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014106210 0x7fe0141a0c70 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe010036040 tx=0x7fe01002ce20 comp rx=0 tx=0).stop 2024-04-16T05:37:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 -- 172.21.15.86:0/3888059566 shutdown_connections 2024-04-16T05:37:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe0000a3d20 0x7fe0000a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe01419a8e0 0x7fe01419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe014106210 0x7fe0141a0c70 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 --2- 172.21.15.86:0/3888059566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe014101140 0x7fe0141a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.279+0000 7fe01c20d640 1 -- 172.21.15.86:0/3888059566 >> 172.21.15.86:0/3888059566 conn(0x7fe014077280 msgr2=0x7fe0140feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.280+0000 7fe01c20d640 1 -- 172.21.15.86:0/3888059566 shutdown_connections 2024-04-16T05:37:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.280+0000 7fe01c20d640 1 -- 172.21.15.86:0/3888059566 wait complete. 2024-04-16T05:37:34.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:37:34.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.401+0000 7fd772167640 1 -- 172.21.15.86:0/1245734154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101250 msgr2=0x7fd76c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.401+0000 7fd772167640 1 --2- 172.21.15.86:0/1245734154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101250 0x7fd76c1016b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd760009900 tx=0x7fd76002bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:34.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.401+0000 7fd772167640 1 -- 172.21.15.86:0/1245734154 shutdown_connections 2024-04-16T05:37:34.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.401+0000 7fd772167640 1 --2- 172.21.15.86:0/1245734154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c101bf0 0x7fd76c10dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.401+0000 7fd772167640 1 --2- 172.21.15.86:0/1245734154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101250 0x7fd76c1016b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.401+0000 7fd772167640 1 --2- 172.21.15.86:0/1245734154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76c107040 0x7fd76c107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.401+0000 7fd772167640 1 -- 172.21.15.86:0/1245734154 >> 172.21.15.86:0/1245734154 conn(0x7fd76c076d00 msgr2=0x7fd76c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.401+0000 7fd772167640 1 -- 172.21.15.86:0/1245734154 shutdown_connections 2024-04-16T05:37:34.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.401+0000 7fd772167640 1 -- 172.21.15.86:0/1245734154 wait complete. 2024-04-16T05:37:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd772167640 1 Processor -- start 2024-04-16T05:37:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd772167640 1 -- start start 2024-04-16T05:37:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd772167640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c101250 0x7fd76c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd772167640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101bf0 0x7fd76c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd772167640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76c107040 0x7fd76c1a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd772167640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1112f0 con 0x7fd76c101bf0 2024-04-16T05:37:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd772167640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c111170 con 0x7fd76c107040 2024-04-16T05:37:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd772167640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c111470 con 0x7fd76c101250 2024-04-16T05:37:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101bf0 0x7fd76c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101bf0 0x7fd76c199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57076/0 (socket says 172.21.15.86:57076) 2024-04-16T05:37:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76affd640 1 -- 172.21.15.86:0/4251283089 learned_addr learned my addr 172.21.15.86:0/4251283089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76bfff640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76c107040 0x7fd76c1a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c101250 0x7fd76c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76affd640 1 -- 172.21.15.86:0/4251283089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c101250 msgr2=0x7fd76c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76affd640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c101250 0x7fd76c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76affd640 1 -- 172.21.15.86:0/4251283089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76c107040 msgr2=0x7fd76c1a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76affd640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76c107040 0x7fd76c1a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.402+0000 7fd76affd640 1 -- 172.21.15.86:0/4251283089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c1a4d10 con 0x7fd76c101bf0 2024-04-16T05:37:34.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.403+0000 7fd76affd640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101bf0 0x7fd76c199c90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd760006010 tx=0x7fd76002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.403+0000 7fd768ff9640 1 -- 172.21.15.86:0/4251283089 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760041070 con 0x7fd76c101bf0 2024-04-16T05:37:34.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.403+0000 7fd768ff9640 1 -- 172.21.15.86:0/4251283089 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd760004450 con 0x7fd76c101bf0 2024-04-16T05:37:34.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.403+0000 7fd772167640 1 -- 172.21.15.86:0/4251283089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c1a4ee0 con 0x7fd76c101bf0 2024-04-16T05:37:34.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.403+0000 7fd768ff9640 1 -- 172.21.15.86:0/4251283089 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760034440 con 0x7fd76c101bf0 2024-04-16T05:37:34.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.403+0000 7fd772167640 1 -- 172.21.15.86:0/4251283089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1a52e0 con 0x7fd76c101bf0 2024-04-16T05:37:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.405+0000 7fd768ff9640 1 -- 172.21.15.86:0/4251283089 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd760044070 con 0x7fd76c101bf0 2024-04-16T05:37:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.405+0000 7fd774e2d740 1 Processor -- start 2024-04-16T05:37:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.405+0000 7fd774e2d740 1 -- start start 2024-04-16T05:37:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.405+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x56379266a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.405+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56379266a780 0x563792672e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.405+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x563792675750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd768ff9640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7540a3d20 0x7fd7540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd774e2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5637925cf0e0 con 0x563792669e60 2024-04-16T05:37:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd774e2d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56379256c520 con 0x56379266a780 2024-04-16T05:37:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd774e2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5637925cb0a0 con 0x563792673360 2024-04-16T05:37:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x56379266a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x56379266a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57084/0 (socket says 172.21.15.86:57084) 2024-04-16T05:37:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 -- 172.21.15.86:0/978031623 learned_addr learned my addr 172.21.15.86:0/978031623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7540a3d20 0x7fd7540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd768ff9640 1 -- 172.21.15.86:0/4251283089 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd7600f26c0 con 0x7fd76c101bf0 2024-04-16T05:37:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76affd640 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x563792675750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56379266a780 0x563792672e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 -- 172.21.15.86:0/978031623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 msgr2=0x563792675750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x563792675750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 -- 172.21.15.86:0/978031623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56379266a780 msgr2=0x563792672e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56379266a780 0x563792672e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 -- 172.21.15.86:0/978031623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637925948d0 con 0x563792669e60 2024-04-16T05:37:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56379266a780 0x563792672e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x56379266a240 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd764005c20 tx=0x7fd764010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3e73e7eb5e8bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd76bfff640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7540a3d20 0x7fd7540a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd764017000 tx=0x7fd76401e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd751ffb640 1 -- 172.21.15.86:0/978031623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764009db0 con 0x563792669e60 2024-04-16T05:37:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd751ffb640 1 -- 172.21.15.86:0/978031623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd764012890 con 0x563792669e60 2024-04-16T05:37:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.406+0000 7fd751ffb640 1 -- 172.21.15.86:0/978031623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76401ec20 con 0x563792669e60 2024-04-16T05:37:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 -- 172.21.15.86:0/978031623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 msgr2=0x56379266a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x56379266a240 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd764005c20 tx=0x7fd764010740 comp rx=0 tx=0).stop 2024-04-16T05:37:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 -- 172.21.15.86:0/978031623 shutdown_connections 2024-04-16T05:37:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x563792675750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56379266a780 0x563792672e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 --2- 172.21.15.86:0/978031623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x56379266a240 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 -- 172.21.15.86:0/978031623 >> 172.21.15.86:0/978031623 conn(0x563792664590 msgr2=0x563792664970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 -- 172.21.15.86:0/978031623 shutdown_connections 2024-04-16T05:37:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 -- 172.21.15.86:0/978031623 wait complete. 2024-04-16T05:37:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.407+0000 7fd774e2d740 1 -- start start 2024-04-16T05:37:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd774e2d740 1 Processor -- start 2024-04-16T05:37:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 0x5637926703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x563792670910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x563792670e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd774e2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563792675fa0 con 0x56379266a780 2024-04-16T05:37:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd774e2d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5637925884d0 con 0x563792669e60 2024-04-16T05:37:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd774e2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563792676120 con 0x563792673360 2024-04-16T05:37:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd774e2d740 10 client.0 _ll_register_callbacks cb 0x56379256d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x563792670910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x563792670910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57090/0 (socket says 172.21.15.86:57090) 2024-04-16T05:37:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76affd640 1 -- 172.21.15.86:0/198738974 learned_addr learned my addr 172.21.15.86:0/198738974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76bfff640 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x563792670e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 0x5637926703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76affd640 1 -- 172.21.15.86:0/198738974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 msgr2=0x563792670e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76affd640 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x563792670e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76affd640 1 -- 172.21.15.86:0/198738974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 msgr2=0x5637926703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76affd640 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 0x5637926703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76affd640 1 -- 172.21.15.86:0/198738974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637922e13f0 con 0x56379266a780 2024-04-16T05:37:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd76affd640 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x563792670910 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd7600ba860 tx=0x7fd7600be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a0db391eafe95f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.408+0000 7fd74affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd74affd640 1 -- 172.21.15.86:0/198738974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76004b040 con 0x56379266a780 2024-04-16T05:37:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd74affd640 1 -- 172.21.15.86:0/198738974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd7600ce7f0 con 0x56379266a780 2024-04-16T05:37:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd774e2d740 1 -- 172.21.15.86:0/198738974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd764012890 con 0x56379266a780 2024-04-16T05:37:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd74affd640 1 -- 172.21.15.86:0/198738974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd7600eff20 con 0x56379266a780 2024-04-16T05:37:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd774e2d740 10 client.16446 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd774e2d740 1 -- 172.21.15.86:0/198738974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637923ab7d0 con 0x56379266a780 2024-04-16T05:37:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd74affd640 1 -- 172.21.15.86:0/198738974 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd764012890 con 0x56379266a780 2024-04-16T05:37:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd74affd640 1 -- 172.21.15.86:0/198738974 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd7600f2300 con 0x56379266a780 2024-04-16T05:37:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd774e2d740 10 client.16446 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd774e2d740 10 client.16446 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:37:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd774e2d740 20 client.16446 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd774e2d740 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x563792683180 0x5637926855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd774e2d740 4 client.16446 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd774e2d740 1 -- 172.21.15.86:0/198738974 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637925dcb00 con 0x563792683180 2024-04-16T05:37:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.409+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x563792683180 0x5637926855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.410+0000 7fd74affd640 10 client.16446 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:37:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.410+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x563792683180 0x5637926855b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57bd61b445befc46 server_cookie=569bb5c13705eddf in_seq=0 out_seq=0 2024-04-16T05:37:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.410+0000 7fd74affd640 1 -- 172.21.15.86:0/198738974 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637925dcb00 con 0x563792683180 2024-04-16T05:37:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.410+0000 7fd74affd640 10 client.16446 handle_command_reply: tid=0 2024-04-16T05:37:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.410+0000 7fd774e2d740 1 client.16446 shutdown 2024-04-16T05:37:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 -- 172.21.15.86:0/198738974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 msgr2=0x563792670910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x563792670910 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd7600ba860 tx=0x7fd7600be000 comp rx=0 tx=0).stop 2024-04-16T05:37:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 -- 172.21.15.86:0/198738974 shutdown_connections 2024-04-16T05:37:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x563792683180 0x5637926855b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x563792670e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x563792670910 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd74affd640 0 client.16446 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:37:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 --2- 172.21.15.86:0/198738974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 0x5637926703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 -- 172.21.15.86:0/198738974 >> 172.21.15.86:0/198738974 conn(0x563792664590 msgr2=0x5637926662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 -- 172.21.15.86:0/198738974 shutdown_connections 2024-04-16T05:37:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 1 -- 172.21.15.86:0/198738974 wait complete. 2024-04-16T05:37:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.411+0000 7fd774e2d740 20 client.16446 trim_cache size 0 max 16384 2024-04-16T05:37:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd774e2d740 1 Processor -- start 2024-04-16T05:37:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd774e2d740 1 -- start start 2024-04-16T05:37:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x563792672b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379266a780 0x56379267f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792673360 0x56379268aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd774e2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1af3f0 con 0x563792669e60 2024-04-16T05:37:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd774e2d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1b0f30 con 0x563792673360 2024-04-16T05:37:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd774e2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1afac0 con 0x56379266a780 2024-04-16T05:37:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd76b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x563792672b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd76b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x563792672b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57096/0 (socket says 172.21.15.86:57096) 2024-04-16T05:37:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd76b7fe640 1 -- 172.21.15.86:0/3707362116 learned_addr learned my addr 172.21.15.86:0/3707362116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd76affd640 1 --2- 172.21.15.86:0/3707362116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379266a780 0x56379267f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd76b7fe640 1 -- 172.21.15.86:0/3707362116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379266a780 msgr2=0x56379267f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/3707362116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379266a780 0x56379267f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd76b7fe640 1 -- 172.21.15.86:0/3707362116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792673360 msgr2=0x56379268aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/3707362116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792673360 0x56379268aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.424+0000 7fd76b7fe640 1 -- 172.21.15.86:0/3707362116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563792675ac0 con 0x563792669e60 2024-04-16T05:37:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/3707362116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x563792672b00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00bba0 tx=0x7fd75c005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a6dfcafcc23a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd7517fa640 1 -- 172.21.15.86:0/3707362116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c01adc0 con 0x563792669e60 2024-04-16T05:37:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd7517fa640 1 -- 172.21.15.86:0/3707362116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd75c01f070 con 0x563792669e60 2024-04-16T05:37:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd7517fa640 1 -- 172.21.15.86:0/3707362116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c012800 con 0x563792669e60 2024-04-16T05:37:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd774e2d740 1 -- 172.21.15.86:0/3707362116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 msgr2=0x563792672b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3707362116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x563792672b00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00bba0 tx=0x7fd75c005cd0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd774e2d740 1 -- 172.21.15.86:0/3707362116 shutdown_connections 2024-04-16T05:37:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3707362116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792673360 0x56379268aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3707362116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379266a780 0x56379267f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3707362116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563792669e60 0x563792672b00 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd774e2d740 1 -- 172.21.15.86:0/3707362116 >> 172.21.15.86:0/3707362116 conn(0x563792664590 msgr2=0x563792663730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd774e2d740 1 -- 172.21.15.86:0/3707362116 shutdown_connections 2024-04-16T05:37:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.425+0000 7fd774e2d740 1 -- 172.21.15.86:0/3707362116 wait complete. 2024-04-16T05:37:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd774e2d740 1 -- start start 2024-04-16T05:37:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd774e2d740 1 Processor -- start 2024-04-16T05:37:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 0x56379266d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x56379266dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd774e2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x5637926e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd774e2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1b0860 con 0x56379266a780 2024-04-16T05:37:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd774e2d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1b0190 con 0x563792669e60 2024-04-16T05:37:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd774e2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1ad3b0 con 0x563792673360 2024-04-16T05:37:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd774e2d740 10 client.0 _ll_register_callbacks cb 0x7fd7600f2940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x56379266dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x56379266dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57100/0 (socket says 172.21.15.86:57100) 2024-04-16T05:37:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76affd640 1 -- 172.21.15.86:0/3907905534 learned_addr learned my addr 172.21.15.86:0/3907905534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76bfff640 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x5637926e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 0x56379266d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76affd640 1 -- 172.21.15.86:0/3907905534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 msgr2=0x5637926e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76affd640 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x5637926e8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76affd640 1 -- 172.21.15.86:0/3907905534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 msgr2=0x56379266d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76affd640 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 0x56379266d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.426+0000 7fd76affd640 1 -- 172.21.15.86:0/3907905534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637926e8c70 con 0x56379266a780 2024-04-16T05:37:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd76affd640 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x56379266dcf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd760005940 tx=0x7fd7600be070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3fc1f394b15ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd74affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd74affd640 1 -- 172.21.15.86:0/3907905534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7600dc060 con 0x56379266a780 2024-04-16T05:37:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd74affd640 1 -- 172.21.15.86:0/3907905534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd7600efea0 con 0x56379266a780 2024-04-16T05:37:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd774e2d740 1 -- 172.21.15.86:0/3907905534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd75c01f070 con 0x56379266a780 2024-04-16T05:37:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd74affd640 1 -- 172.21.15.86:0/3907905534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd7600ba6a0 con 0x56379266a780 2024-04-16T05:37:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd774e2d740 10 client.16452 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd774e2d740 1 -- 172.21.15.86:0/3907905534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637926e9e30 con 0x56379266a780 2024-04-16T05:37:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd74affd640 1 -- 172.21.15.86:0/3907905534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd75c01f070 con 0x56379266a780 2024-04-16T05:37:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd74affd640 1 -- 172.21.15.86:0/3907905534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd7600d7b80 con 0x56379266a780 2024-04-16T05:37:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd774e2d740 10 client.16452 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd774e2d740 10 client.16452 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:37:34.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd774e2d740 20 client.16452 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:34.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5637926ea310 0x5637926ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd774e2d740 4 client.16452 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:34.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.427+0000 7fd774e2d740 1 -- 172.21.15.86:0/3907905534 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637925d6970 con 0x5637926ea310 2024-04-16T05:37:34.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5637926ea310 0x5637926ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd74affd640 10 client.16452 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:37:34.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd76b7fe640 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5637926ea310 0x5637926ec700 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c65683d11a796959 server_cookie=84dfc9c4ecf2343 in_seq=0 out_seq=0 2024-04-16T05:37:34.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd74affd640 1 -- 172.21.15.86:0/3907905534 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637925d6970 con 0x5637926ea310 2024-04-16T05:37:34.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd74affd640 10 client.16452 handle_command_reply: tid=0 2024-04-16T05:37:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 client.16452 shutdown 2024-04-16T05:37:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 -- 172.21.15.86:0/3907905534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 msgr2=0x56379266dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x56379266dcf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd760005940 tx=0x7fd7600be070 comp rx=0 tx=0).stop 2024-04-16T05:37:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 -- 172.21.15.86:0/3907905534 shutdown_connections 2024-04-16T05:37:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5637926ea310 0x5637926ec700 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563792673360 0x5637926e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56379266a780 0x56379266dcf0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 --2- 172.21.15.86:0/3907905534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563792669e60 0x56379266d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd74affd640 0 client.16452 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:37:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 -- 172.21.15.86:0/3907905534 >> 172.21.15.86:0/3907905534 conn(0x563792664590 msgr2=0x563792670390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 -- 172.21.15.86:0/3907905534 shutdown_connections 2024-04-16T05:37:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 1 -- 172.21.15.86:0/3907905534 wait complete. 2024-04-16T05:37:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.428+0000 7fd774e2d740 20 client.16452 trim_cache size 0 max 16384 2024-04-16T05:37:34.441 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 -- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7540a3d20 msgr2=0x7fd7540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7540a3d20 0x7fd7540a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd764017000 tx=0x7fd76401e000 comp rx=0 tx=0).stop 2024-04-16T05:37:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 -- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101bf0 msgr2=0x7fd76c199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101bf0 0x7fd76c199c90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd760006010 tx=0x7fd76002cc70 comp rx=0 tx=0).stop 2024-04-16T05:37:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 -- 172.21.15.86:0/4251283089 shutdown_connections 2024-04-16T05:37:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd7540a3d20 0x7fd7540a61e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76c107040 0x7fd76c1a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd76c101bf0 0x7fd76c199c90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 --2- 172.21.15.86:0/4251283089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c101250 0x7fd76c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 -- 172.21.15.86:0/4251283089 >> 172.21.15.86:0/4251283089 conn(0x7fd76c076d00 msgr2=0x7fd76c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 -- 172.21.15.86:0/4251283089 shutdown_connections 2024-04-16T05:37:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.429+0000 7fd772167640 1 -- 172.21.15.86:0/4251283089 wait complete. 2024-04-16T05:37:34.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:37:34.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 -- 172.21.15.86:0/2832161218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c107310 msgr2=0x7f664c10eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 --2- 172.21.15.86:0/2832161218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c107310 0x7f664c10eee0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6644009900 tx=0x7f664402bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:34.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 -- 172.21.15.86:0/2832161218 shutdown_connections 2024-04-16T05:37:34.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 --2- 172.21.15.86:0/2832161218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c107310 0x7f664c10eee0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 --2- 172.21.15.86:0/2832161218 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f664c102bf0 0x7f664c106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 --2- 172.21.15.86:0/2832161218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f664c102240 0x7f664c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 -- 172.21.15.86:0/2832161218 >> 172.21.15.86:0/2832161218 conn(0x7f664c0fdae0 msgr2=0x7f664c0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 -- 172.21.15.86:0/2832161218 shutdown_connections 2024-04-16T05:37:34.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 -- 172.21.15.86:0/2832161218 wait complete. 2024-04-16T05:37:34.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.554+0000 7f6651fa8640 1 Processor -- start 2024-04-16T05:37:34.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f6651fa8640 1 -- start start 2024-04-16T05:37:34.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f6651fa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c102240 0x7f664c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f6651fa8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f664c107310 0x7f664c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f6651fa8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f664c1986e0 0x7f664c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f6651fa8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f664c112420 con 0x7f664c102240 2024-04-16T05:37:34.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f6651fa8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f664c1122a0 con 0x7f664c1986e0 2024-04-16T05:37:34.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f6651fa8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f664c1125a0 con 0x7f664c107310 2024-04-16T05:37:34.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c102240 0x7f664c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c102240 0x7f664c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57110/0 (socket says 172.21.15.86:57110) 2024-04-16T05:37:34.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664b7fe640 1 -- 172.21.15.86:0/646581354 learned_addr learned my addr 172.21.15.86:0/646581354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664bfff640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f664c1986e0 0x7f664c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664b7fe640 1 -- 172.21.15.86:0/646581354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f664c107310 msgr2=0x7f664c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664affd640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f664c107310 0x7f664c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664b7fe640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f664c107310 0x7f664c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664b7fe640 1 -- 172.21.15.86:0/646581354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f664c1986e0 msgr2=0x7f664c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664b7fe640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f664c1986e0 0x7f664c198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664b7fe640 1 -- 172.21.15.86:0/646581354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664c1992e0 con 0x7f664c102240 2024-04-16T05:37:34.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664bfff640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f664c1986e0 0x7f664c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:34.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f664b7fe640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c102240 0x7f664c19e620 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f663c00c920 tx=0x7f663c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.555+0000 7f6648ff9640 1 -- 172.21.15.86:0/646581354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c007c70 con 0x7f664c102240 2024-04-16T05:37:34.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.556+0000 7f6648ff9640 1 -- 172.21.15.86:0/646581354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c007e10 con 0x7f664c102240 2024-04-16T05:37:34.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.556+0000 7f6651fa8640 1 -- 172.21.15.86:0/646581354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664c1a53c0 con 0x7f664c102240 2024-04-16T05:37:34.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.556+0000 7f6648ff9640 1 -- 172.21.15.86:0/646581354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c018890 con 0x7f664c102240 2024-04-16T05:37:34.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.556+0000 7f6651fa8640 1 -- 172.21.15.86:0/646581354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664c1a57c0 con 0x7f664c102240 2024-04-16T05:37:34.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.558+0000 7f6648ff9640 1 -- 172.21.15.86:0/646581354 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f663c018a70 con 0x7f664c102240 2024-04-16T05:37:34.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.558+0000 7f6654b92740 1 Processor -- start 2024-04-16T05:37:34.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.558+0000 7f6654b92740 1 -- start start 2024-04-16T05:37:34.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.558+0000 7f6654b92740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 0x5581d68e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.558+0000 7f6654b92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68e3780 0x5581d68ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.558+0000 7f6654b92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68ee750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.558+0000 7f6654b92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581d68480e0 con 0x5581d68ec360 2024-04-16T05:37:34.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.558+0000 7f6654b92740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5581d67e5520 con 0x5581d68e2e60 2024-04-16T05:37:34.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.558+0000 7f6654b92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581d68440a0 con 0x5581d68e3780 2024-04-16T05:37:34.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f6648ff9640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66340a3d20 0x7f66340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68ee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68ee750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57116/0 (socket says 172.21.15.86:57116) 2024-04-16T05:37:34.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664affd640 1 -- 172.21.15.86:0/1001103723 learned_addr learned my addr 172.21.15.86:0/1001103723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664b7fe640 1 --2- 172.21.15.86:0/1001103723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68e3780 0x5581d68ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664bfff640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66340a3d20 0x7f66340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664bfff640 1 --2- 172.21.15.86:0/1001103723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 0x5581d68e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664affd640 1 -- 172.21.15.86:0/1001103723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68e3780 msgr2=0x5581d68ebe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664affd640 1 --2- 172.21.15.86:0/1001103723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68e3780 0x5581d68ebe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f6648ff9640 1 -- 172.21.15.86:0/646581354 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f663c0d0da0 con 0x7f664c102240 2024-04-16T05:37:34.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664affd640 1 -- 172.21.15.86:0/1001103723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 msgr2=0x5581d68e3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664affd640 1 --2- 172.21.15.86:0/1001103723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 0x5581d68e3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664affd640 1 -- 172.21.15.86:0/1001103723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d680d8d0 con 0x5581d68ec360 2024-04-16T05:37:34.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664affd640 1 --2- 172.21.15.86:0/1001103723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68ee750 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6640007db0 tx=0x7f664000c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1600c6aec1e553cf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f664bfff640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66340a3d20 0x7f66340a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6644005ab0 tx=0x7f6644005a20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f6631ffb640 1 -- 172.21.15.86:0/1001103723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640015020 con 0x5581d68ec360 2024-04-16T05:37:34.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f6631ffb640 1 -- 172.21.15.86:0/1001103723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6640011b10 con 0x5581d68ec360 2024-04-16T05:37:34.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.559+0000 7f6631ffb640 1 -- 172.21.15.86:0/1001103723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640010ad0 con 0x5581d68ec360 2024-04-16T05:37:34.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 -- 172.21.15.86:0/1001103723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 msgr2=0x5581d68ee750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 --2- 172.21.15.86:0/1001103723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68ee750 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6640007db0 tx=0x7f664000c560 comp rx=0 tx=0).stop 2024-04-16T05:37:34.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 -- 172.21.15.86:0/1001103723 shutdown_connections 2024-04-16T05:37:34.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 --2- 172.21.15.86:0/1001103723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68ee750 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 --2- 172.21.15.86:0/1001103723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68e3780 0x5581d68ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 --2- 172.21.15.86:0/1001103723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 0x5581d68e3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 -- 172.21.15.86:0/1001103723 >> 172.21.15.86:0/1001103723 conn(0x5581d68dd590 msgr2=0x5581d68dd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 -- 172.21.15.86:0/1001103723 shutdown_connections 2024-04-16T05:37:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 -- 172.21.15.86:0/1001103723 wait complete. 2024-04-16T05:37:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.560+0000 7f6654b92740 1 -- start start 2024-04-16T05:37:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f6654b92740 1 Processor -- start 2024-04-16T05:37:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f6654b92740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 0x5581d68e9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f6654b92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68e3780 0x5581d68e98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f6654b92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f6654b92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581d68ef000 con 0x5581d68ec360 2024-04-16T05:37:34.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f6654b92740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5581d68014d0 con 0x5581d68e2e60 2024-04-16T05:37:34.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f6654b92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581d68ef180 con 0x5581d68e3780 2024-04-16T05:37:34.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f6654b92740 10 client.0 _ll_register_callbacks cb 0x5581d67e62c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:34.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68e9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57130/0 (socket says 172.21.15.86:57130) 2024-04-16T05:37:34.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664bfff640 1 -- 172.21.15.86:0/3033228263 learned_addr learned my addr 172.21.15.86:0/3033228263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664b7fe640 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 0x5581d68e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664bfff640 1 -- 172.21.15.86:0/3033228263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68e3780 msgr2=0x5581d68e98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664bfff640 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68e3780 0x5581d68e98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664bfff640 1 -- 172.21.15.86:0/3033228263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 msgr2=0x5581d68e9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664bfff640 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 0x5581d68e9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664bfff640 1 -- 172.21.15.86:0/3033228263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d655a3f0 con 0x5581d68ec360 2024-04-16T05:37:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f664bfff640 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68e9e10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6644035000 tx=0x7f6644009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e779f964d3e4441f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.561+0000 7f662affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f662affd640 1 -- 172.21.15.86:0/3033228263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644003d00 con 0x5581d68ec360 2024-04-16T05:37:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f662affd640 1 -- 172.21.15.86:0/3033228263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f664404d460 con 0x5581d68ec360 2024-04-16T05:37:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f6654b92740 1 -- 172.21.15.86:0/3033228263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6640011b10 con 0x5581d68ec360 2024-04-16T05:37:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f662affd640 1 -- 172.21.15.86:0/3033228263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f664405b050 con 0x5581d68ec360 2024-04-16T05:37:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f6654b92740 10 client.16464 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f6654b92740 1 -- 172.21.15.86:0/3033228263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581d66247d0 con 0x5581d68ec360 2024-04-16T05:37:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f662affd640 1 -- 172.21.15.86:0/3033228263 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6640011b10 con 0x5581d68ec360 2024-04-16T05:37:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f662affd640 1 -- 172.21.15.86:0/3033228263 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f66440345b0 con 0x5581d68ec360 2024-04-16T05:37:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f6654b92740 10 client.16464 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f6654b92740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f6654b92740 20 client.16464 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f6654b92740 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5581d68fc3b0 0x5581d68fe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f6654b92740 4 client.16464 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f6654b92740 1 -- 172.21.15.86:0/3033228263 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d6855b00 con 0x5581d68fc3b0 2024-04-16T05:37:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.562+0000 7f664b7fe640 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5581d68fc3b0 0x5581d68fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.563+0000 7f664b7fe640 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5581d68fc3b0 0x5581d68fe870 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520c3902f8b65ea0 server_cookie=daa074834e046942 in_seq=0 out_seq=0 2024-04-16T05:37:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.563+0000 7f662affd640 10 client.16464 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.563+0000 7f662affd640 1 -- 172.21.15.86:0/3033228263 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581d6855b00 con 0x5581d68fc3b0 2024-04-16T05:37:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.563+0000 7f662affd640 10 client.16464 handle_command_reply: tid=0 2024-04-16T05:37:34.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.563+0000 7f6654b92740 1 client.16464 shutdown 2024-04-16T05:37:34.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 -- 172.21.15.86:0/3033228263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 msgr2=0x5581d68e9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68e9e10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6644035000 tx=0x7f6644009260 comp rx=0 tx=0).stop 2024-04-16T05:37:34.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 -- 172.21.15.86:0/3033228263 shutdown_connections 2024-04-16T05:37:34.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5581d68fc3b0 0x5581d68fe870 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68ec360 0x5581d68e9e10 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68e3780 0x5581d68e98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f662affd640 0 client.16464 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:34.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 --2- 172.21.15.86:0/3033228263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e2e60 0x5581d68e9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 -- 172.21.15.86:0/3033228263 >> 172.21.15.86:0/3033228263 conn(0x5581d68dd590 msgr2=0x5581d68df2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 -- 172.21.15.86:0/3033228263 shutdown_connections 2024-04-16T05:37:34.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 1 -- 172.21.15.86:0/3033228263 wait complete. 2024-04-16T05:37:34.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.564+0000 7f6654b92740 20 client.16464 trim_cache size 0 max 16384 2024-04-16T05:37:34.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f6654b92740 1 Processor -- start 2024-04-16T05:37:34.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f6654b92740 1 -- start start 2024-04-16T05:37:34.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f6654b92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f6654b92740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e3780 0x5581d68f8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f6654b92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68ec360 0x5581d6904140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f6654b92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f664c1af370 con 0x5581d68e2e60 2024-04-16T05:37:34.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f6654b92740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f664c1b0eb0 con 0x5581d68e3780 2024-04-16T05:37:34.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f6654b92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f664c1afa40 con 0x5581d68ec360 2024-04-16T05:37:34.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f664b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f664b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68ebac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57138/0 (socket says 172.21.15.86:57138) 2024-04-16T05:37:34.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f664b7fe640 1 -- 172.21.15.86:0/857152561 learned_addr learned my addr 172.21.15.86:0/857152561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f664b7fe640 1 -- 172.21.15.86:0/857152561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68ec360 msgr2=0x5581d6904140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f664b7fe640 1 --2- 172.21.15.86:0/857152561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68ec360 0x5581d6904140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f664b7fe640 1 -- 172.21.15.86:0/857152561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e3780 msgr2=0x5581d68f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f664affd640 1 --2- 172.21.15.86:0/857152561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e3780 0x5581d68f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.577+0000 7f664b7fe640 1 --2- 172.21.15.86:0/857152561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e3780 0x5581d68f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f664b7fe640 1 -- 172.21.15.86:0/857152561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d68ee750 con 0x5581d68e2e60 2024-04-16T05:37:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f664b7fe640 1 --2- 172.21.15.86:0/857152561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68ebac0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f663c0ce1e0 tx=0x7f663c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762fdf2d290c8d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f66317fa640 1 -- 172.21.15.86:0/857152561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0b5070 con 0x5581d68e2e60 2024-04-16T05:37:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f66317fa640 1 -- 172.21.15.86:0/857152561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c0a5640 con 0x5581d68e2e60 2024-04-16T05:37:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f66317fa640 1 -- 172.21.15.86:0/857152561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c099040 con 0x5581d68e2e60 2024-04-16T05:37:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f6654b92740 1 -- 172.21.15.86:0/857152561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 msgr2=0x5581d68ebac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f6654b92740 1 --2- 172.21.15.86:0/857152561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68ebac0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f663c0ce1e0 tx=0x7f663c087040 comp rx=0 tx=0).stop 2024-04-16T05:37:34.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f6654b92740 1 -- 172.21.15.86:0/857152561 shutdown_connections 2024-04-16T05:37:34.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f6654b92740 1 --2- 172.21.15.86:0/857152561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68ec360 0x5581d6904140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f6654b92740 1 --2- 172.21.15.86:0/857152561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e3780 0x5581d68f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f6654b92740 1 --2- 172.21.15.86:0/857152561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68ebac0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f6654b92740 1 -- 172.21.15.86:0/857152561 >> 172.21.15.86:0/857152561 conn(0x5581d68dd590 msgr2=0x5581d68dc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f6654b92740 1 -- 172.21.15.86:0/857152561 shutdown_connections 2024-04-16T05:37:34.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.578+0000 7f6654b92740 1 -- 172.21.15.86:0/857152561 wait complete. 2024-04-16T05:37:34.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f6654b92740 1 -- start start 2024-04-16T05:37:34.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f6654b92740 1 Processor -- start 2024-04-16T05:37:34.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f6654b92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f6654b92740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e3780 0x5581d68e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f6654b92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68ec360 0x5581d69619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f6654b92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f664c1b07e0 con 0x5581d68e2e60 2024-04-16T05:37:34.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f6654b92740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f664c1b0110 con 0x5581d68e3780 2024-04-16T05:37:34.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f6654b92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f664c1ad330 con 0x5581d68ec360 2024-04-16T05:37:34.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f6654b92740 10 client.0 _ll_register_callbacks cb 0x7f6644009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:34.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f664b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f664b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68e63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57142/0 (socket says 172.21.15.86:57142) 2024-04-16T05:37:34.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f664b7fe640 1 -- 172.21.15.86:0/364708702 learned_addr learned my addr 172.21.15.86:0/364708702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:34.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.579+0000 7f664b7fe640 1 -- 172.21.15.86:0/364708702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68ec360 msgr2=0x5581d69619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f664b7fe640 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68ec360 0x5581d69619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f664b7fe640 1 -- 172.21.15.86:0/364708702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e3780 msgr2=0x5581d68e68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:34.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f664b7fe640 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e3780 0x5581d68e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f664b7fe640 1 -- 172.21.15.86:0/364708702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d6961ef0 con 0x5581d68e2e60 2024-04-16T05:37:34.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f664b7fe640 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68e63b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f663c09d0f0 tx=0x7f663c098c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883c67364a4c246 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:34.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f662affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:34.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f662affd640 1 -- 172.21.15.86:0/364708702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0b5050 con 0x5581d68e2e60 2024-04-16T05:37:34.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f662affd640 1 -- 172.21.15.86:0/364708702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f663c00fb30 con 0x5581d68e2e60 2024-04-16T05:37:34.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f6654b92740 1 -- 172.21.15.86:0/364708702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c0a5640 con 0x5581d68e2e60 2024-04-16T05:37:34.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f662affd640 1 -- 172.21.15.86:0/364708702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f663c018d60 con 0x5581d68e2e60 2024-04-16T05:37:34.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f6654b92740 10 client.16470 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:34.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f6654b92740 1 -- 172.21.15.86:0/364708702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581d6962ff0 con 0x5581d68e2e60 2024-04-16T05:37:34.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f662affd640 1 -- 172.21.15.86:0/364708702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c0a5640 con 0x5581d68e2e60 2024-04-16T05:37:34.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f662affd640 1 -- 172.21.15.86:0/364708702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f663c0a51b0 con 0x5581d68e2e60 2024-04-16T05:37:34.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f6654b92740 10 client.16470 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:34.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f6654b92740 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:34.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f6654b92740 20 client.16470 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:34.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f6654b92740 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5581d6963790 0x5581d6965bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:34.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f6654b92740 4 client.16470 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:34.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.580+0000 7f6654b92740 1 -- 172.21.15.86:0/364708702 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581d684f970 con 0x5581d6963790 2024-04-16T05:37:34.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.581+0000 7f664affd640 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5581d6963790 0x5581d6965bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:34.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.581+0000 7f662affd640 10 client.16470 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:34.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.581+0000 7f664affd640 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5581d6963790 0x5581d6965bb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebbccefcc282ce6e server_cookie=7d6369929dc4865b in_seq=0 out_seq=0 2024-04-16T05:37:34.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.581+0000 7f662affd640 1 -- 172.21.15.86:0/364708702 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581d684f970 con 0x5581d6963790 2024-04-16T05:37:34.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.581+0000 7f662affd640 10 client.16470 handle_command_reply: tid=0 2024-04-16T05:37:34.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.581+0000 7f6654b92740 1 client.16470 shutdown 2024-04-16T05:37:34.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 -- 172.21.15.86:0/364708702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 msgr2=0x5581d68e63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68e63b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f663c09d0f0 tx=0x7f663c098c70 comp rx=0 tx=0).stop 2024-04-16T05:37:34.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 -- 172.21.15.86:0/364708702 shutdown_connections 2024-04-16T05:37:34.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5581d6963790 0x5581d6965bb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581d68ec360 0x5581d69619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f662affd640 0 client.16470 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:34.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d68e3780 0x5581d68e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 --2- 172.21.15.86:0/364708702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d68e2e60 0x5581d68e63b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 -- 172.21.15.86:0/364708702 >> 172.21.15.86:0/364708702 conn(0x5581d68dd590 msgr2=0x5581d68de9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 -- 172.21.15.86:0/364708702 shutdown_connections 2024-04-16T05:37:34.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 1 -- 172.21.15.86:0/364708702 wait complete. 2024-04-16T05:37:34.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6654b92740 20 client.16470 trim_cache size 0 max 16384 2024-04-16T05:37:34.591 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:37:34.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 -- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66340a3d20 msgr2=0x7f66340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66340a3d20 0x7f66340a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6644005ab0 tx=0x7f6644005a20 comp rx=0 tx=0).stop 2024-04-16T05:37:34.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 -- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c102240 msgr2=0x7f664c19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:34.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c102240 0x7f664c19e620 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f663c00c920 tx=0x7f663c00cde0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 -- 172.21.15.86:0/646581354 shutdown_connections 2024-04-16T05:37:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f66340a3d20 0x7f66340a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f664c1986e0 0x7f664c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f664c107310 0x7f664c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 --2- 172.21.15.86:0/646581354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f664c102240 0x7f664c19e620 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 -- 172.21.15.86:0/646581354 >> 172.21.15.86:0/646581354 conn(0x7f664c0fdae0 msgr2=0x7f664c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 -- 172.21.15.86:0/646581354 shutdown_connections 2024-04-16T05:37:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:34.582+0000 7f6651fa8640 1 -- 172.21.15.86:0/646581354 wait complete. 2024-04-16T05:37:34.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:34 smithi086 ceph-mon[32028]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:37:34.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:34 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1912197632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:34.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:34 smithi194 ceph-mon[35865]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:37:34.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:34 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1912197632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:34.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:34 smithi196 ceph-mon[36373]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-16T05:37:34.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:34 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1912197632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:35.594 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:37:35.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:37:35.595 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:37:35.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/686786569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d38106ff0 msgr2=0x7f0d3806a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/686786569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d38106ff0 0x7f0d3806a600 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ca0 tx=0x7f0d3402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/686786569 shutdown_connections 2024-04-16T05:37:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/686786569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d3806b220 0x7f0d38104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/686786569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d381073d0 0x7f0d3806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/686786569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d38106ff0 0x7f0d3806a600 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/686786569 >> 172.21.15.86:0/686786569 conn(0x7f0d38076d00 msgr2=0x7f0d38077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/686786569 shutdown_connections 2024-04-16T05:37:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/686786569 wait complete. 2024-04-16T05:37:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 Processor -- start 2024-04-16T05:37:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.717+0000 7f0d3ffbd640 1 -- start start 2024-04-16T05:37:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3ffbd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d38106ff0 0x7f0d3819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3ffbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d381073d0 0x7f0d3819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3ffbd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d381a3860 0x7f0d381a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3ffbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381189e0 con 0x7f0d381073d0 2024-04-16T05:37:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3ffbd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38118860 con 0x7f0d381a3860 2024-04-16T05:37:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3ffbd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38118b60 con 0x7f0d38106ff0 2024-04-16T05:37:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3d558640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d381073d0 0x7f0d3819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3d558640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d381073d0 0x7f0d3819f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57164/0 (socket says 172.21.15.86:57164) 2024-04-16T05:37:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3d558640 1 -- 172.21.15.86:0/2560687853 learned_addr learned my addr 172.21.15.86:0/2560687853 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d381a3860 0x7f0d381a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d38106ff0 0x7f0d3819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3d558640 1 -- 172.21.15.86:0/2560687853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d38106ff0 msgr2=0x7f0d3819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3d558640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d38106ff0 0x7f0d3819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3d558640 1 -- 172.21.15.86:0/2560687853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d381a3860 msgr2=0x7f0d381a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3d558640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d381a3860 0x7f0d381a3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3d558640 1 -- 172.21.15.86:0/2560687853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d381a4350 con 0x7f0d381073d0 2024-04-16T05:37:35.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3d558640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d381073d0 0x7f0d3819f4e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800d770 tx=0x7f0d2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d2effd640 1 -- 172.21.15.86:0/2560687853 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28017070 con 0x7f0d381073d0 2024-04-16T05:37:35.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d2effd640 1 -- 172.21.15.86:0/2560687853 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d28005500 con 0x7f0d381073d0 2024-04-16T05:37:35.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/2560687853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d381a4640 con 0x7f0d381073d0 2024-04-16T05:37:35.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d2effd640 1 -- 172.21.15.86:0/2560687853 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28004e00 con 0x7f0d381073d0 2024-04-16T05:37:35.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.718+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/2560687853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d381abde0 con 0x7f0d381073d0 2024-04-16T05:37:35.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.720+0000 7f0d2effd640 1 -- 172.21.15.86:0/2560687853 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d28004fa0 con 0x7f0d381073d0 2024-04-16T05:37:35.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d42ba9740 1 Processor -- start 2024-04-16T05:37:35.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d42ba9740 1 -- start start 2024-04-16T05:37:35.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 0x55fd27ebaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27ebd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fd27e16e90 con 0x55fd27eb1ab0 2024-04-16T05:37:35.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fd27db44c0 con 0x55fd27ebafb0 2024-04-16T05:37:35.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fd27e12e50 con 0x55fd27eb23d0 2024-04-16T05:37:35.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d3e55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d3e55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57180/0 (socket says 172.21.15.86:57180) 2024-04-16T05:37:35.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d3e55a640 1 -- 172.21.15.86:0/2120201750 learned_addr learned my addr 172.21.15.86:0/2120201750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.721+0000 7f0d2effd640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d240a3d20 0x7f0d240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 0x55fd27ebaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d240a3d20 0x7f0d240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3e55a640 1 -- 172.21.15.86:0/2120201750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 msgr2=0x55fd27ebaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 0x55fd27ebaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d2effd640 1 -- 172.21.15.86:0/2560687853 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0d280d0580 con 0x7f0d381073d0 2024-04-16T05:37:35.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3d558640 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27ebd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3e55a640 1 -- 172.21.15.86:0/2120201750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 msgr2=0x55fd27ebd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27ebd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3e55a640 1 -- 172.21.15.86:0/2120201750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd27bf17e0 con 0x55fd27eb1ab0 2024-04-16T05:37:35.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 0x55fd27ebaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3d558640 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27ebd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb1e90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000bd20 tx=0x7f0d300039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51efa4bcd093b047 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d23fff640 1 -- 172.21.15.86:0/2120201750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3000b700 con 0x55fd27eb1ab0 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d23fff640 1 -- 172.21.15.86:0/2120201750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d30005570 con 0x55fd27eb1ab0 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.722+0000 7f0d23fff640 1 -- 172.21.15.86:0/2120201750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3001e470 con 0x55fd27eb1ab0 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2120201750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 msgr2=0x55fd27eb1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d240a3d20 0x7f0d240a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0d30021000 tx=0x7f0d30020000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb1e90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000bd20 tx=0x7f0d300039c0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2120201750 shutdown_connections 2024-04-16T05:37:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27ebd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 0x55fd27ebaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2120201750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb1e90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2120201750 >> 172.21.15.86:0/2120201750 conn(0x55fd27eac1e0 msgr2=0x55fd27eac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2120201750 shutdown_connections 2024-04-16T05:37:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2120201750 wait complete. 2024-04-16T05:37:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 -- start start 2024-04-16T05:37:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.723+0000 7f0d42ba9740 1 Processor -- start 2024-04-16T05:37:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27eb23d0 0x55fd27eb85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27ebafb0 0x55fd27eb8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fd27ebdda0 con 0x55fd27ebafb0 2024-04-16T05:37:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fd27d9f410 con 0x55fd27eb23d0 2024-04-16T05:37:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fd27ebdf20 con 0x55fd27eb1ab0 2024-04-16T05:37:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d42ba9740 10 client.0 _ll_register_callbacks cb 0x55fd27d9ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3e55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27ebafb0 0x55fd27eb8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3e55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27ebafb0 0x55fd27eb8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57196/0 (socket says 172.21.15.86:57196) 2024-04-16T05:37:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3e55a640 1 -- 172.21.15.86:0/2481601390 learned_addr learned my addr 172.21.15.86:0/2481601390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3e55a640 1 -- 172.21.15.86:0/2481601390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 msgr2=0x55fd27eb8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3e55a640 1 -- 172.21.15.86:0/2481601390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27eb23d0 msgr2=0x55fd27eb85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27eb23d0 0x55fd27eb85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3e55a640 1 -- 172.21.15.86:0/2481601390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd27c51f70 con 0x55fd27ebafb0 2024-04-16T05:37:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27ebafb0 0x55fd27eb8af0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000b320 tx=0x7f0d30004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ee2a846a9cd13f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.724+0000 7f0d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d217fa640 1 -- 172.21.15.86:0/2481601390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3001e470 con 0x55fd27ebafb0 2024-04-16T05:37:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d217fa640 1 -- 172.21.15.86:0/2481601390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0d30023040 con 0x55fd27ebafb0 2024-04-16T05:37:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2481601390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d30005570 con 0x55fd27ebafb0 2024-04-16T05:37:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d217fa640 1 -- 172.21.15.86:0/2481601390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d3003a050 con 0x55fd27ebafb0 2024-04-16T05:37:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d42ba9740 10 client.16482 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2481601390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fd27ae9170 con 0x55fd27ebafb0 2024-04-16T05:37:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d217fa640 1 -- 172.21.15.86:0/2481601390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d30005570 con 0x55fd27ebafb0 2024-04-16T05:37:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d217fa640 1 -- 172.21.15.86:0/2481601390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0d3001eb10 con 0x55fd27ebafb0 2024-04-16T05:37:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d42ba9740 10 client.16482 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d42ba9740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d42ba9740 20 client.16482 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fd27ecb380 0x55fd27ecd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d42ba9740 4 client.16482 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.725+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2481601390 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd27e248b0 con 0x55fd27ecb380 2024-04-16T05:37:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.726+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fd27ecb380 0x55fd27ecd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.726+0000 7f0d217fa640 10 client.16482 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.726+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fd27ecb380 0x55fd27ecd770 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e081ef3562fafcbe server_cookie=7439de6b358e13b in_seq=0 out_seq=0 2024-04-16T05:37:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d217fa640 1 -- 172.21.15.86:0/2481601390 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd27e248b0 con 0x55fd27ecb380 2024-04-16T05:37:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d217fa640 10 client.16482 handle_command_reply: tid=0 2024-04-16T05:37:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 client.16482 shutdown 2024-04-16T05:37:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2481601390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27ebafb0 msgr2=0x55fd27eb8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27ebafb0 0x55fd27eb8af0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000b320 tx=0x7f0d30004d80 comp rx=0 tx=0).stop 2024-04-16T05:37:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2481601390 shutdown_connections 2024-04-16T05:37:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fd27ecb380 0x55fd27ecd770 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27ebafb0 0x55fd27eb8af0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27eb23d0 0x55fd27eb85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/2481601390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d217fa640 0 client.16482 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:35.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2481601390 >> 172.21.15.86:0/2481601390 conn(0x55fd27eac1e0 msgr2=0x55fd27eadf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2481601390 shutdown_connections 2024-04-16T05:37:35.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 1 -- 172.21.15.86:0/2481601390 wait complete. 2024-04-16T05:37:35.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.727+0000 7f0d42ba9740 20 client.16482 trim_cache size 0 max 16384 2024-04-16T05:37:35.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.744+0000 7f0d42ba9740 1 Processor -- start 2024-04-16T05:37:35.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.744+0000 7f0d42ba9740 1 -- start start 2024-04-16T05:37:35.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 0x55fd27ec6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27eb71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381b5a60 con 0x55fd27eb1ab0 2024-04-16T05:37:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381b75a0 con 0x55fd27ebafb0 2024-04-16T05:37:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381b6130 con 0x55fd27eb23d0 2024-04-16T05:37:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d3dd59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d3dd59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eba7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57198/0 (socket says 172.21.15.86:57198) 2024-04-16T05:37:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d3dd59640 1 -- 172.21.15.86:0/728816582 learned_addr learned my addr 172.21.15.86:0/728816582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d3dd59640 1 -- 172.21.15.86:0/728816582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 msgr2=0x55fd27ec6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/728816582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 0x55fd27ec6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d3dd59640 1 -- 172.21.15.86:0/728816582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 msgr2=0x55fd27eb71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/728816582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27eb71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d3dd59640 1 -- 172.21.15.86:0/728816582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd27ebd710 con 0x55fd27eb1ab0 2024-04-16T05:37:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/728816582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eba7a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0d34000f80 tx=0x7f0d34009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc228d4b508a300 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d237fe640 1 -- 172.21.15.86:0/728816582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34048070 con 0x55fd27eb1ab0 2024-04-16T05:37:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d237fe640 1 -- 172.21.15.86:0/728816582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d34032c90 con 0x55fd27eb1ab0 2024-04-16T05:37:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.745+0000 7f0d237fe640 1 -- 172.21.15.86:0/728816582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34044e60 con 0x55fd27eb1ab0 2024-04-16T05:37:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 -- 172.21.15.86:0/728816582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 msgr2=0x55fd27eba7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/728816582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eba7a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0d34000f80 tx=0x7f0d34009820 comp rx=0 tx=0).stop 2024-04-16T05:37:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 -- 172.21.15.86:0/728816582 shutdown_connections 2024-04-16T05:37:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/728816582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27eb71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/728816582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb23d0 0x55fd27ec6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/728816582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb1ab0 0x55fd27eba7a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 -- 172.21.15.86:0/728816582 >> 172.21.15.86:0/728816582 conn(0x55fd27eac1e0 msgr2=0x55fd27eb8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 -- 172.21.15.86:0/728816582 shutdown_connections 2024-04-16T05:37:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 -- 172.21.15.86:0/728816582 wait complete. 2024-04-16T05:37:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 -- start start 2024-04-16T05:37:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.746+0000 7f0d42ba9740 1 Processor -- start 2024-04-16T05:37:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb23d0 0x55fd27eb5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d42ba9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27eb5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381b6ed0 con 0x55fd27eb23d0 2024-04-16T05:37:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381b6800 con 0x55fd27ebafb0 2024-04-16T05:37:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d42ba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381b3a20 con 0x55fd27eb1ab0 2024-04-16T05:37:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d42ba9740 10 client.0 _ll_register_callbacks cb 0x7f0d3000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3d558640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb23d0 0x55fd27eb5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3d558640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb23d0 0x55fd27eb5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57204/0 (socket says 172.21.15.86:57204) 2024-04-16T05:37:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3d558640 1 -- 172.21.15.86:0/1518570798 learned_addr learned my addr 172.21.15.86:0/1518570798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27eb5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3d558640 1 -- 172.21.15.86:0/1518570798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 msgr2=0x55fd27eb4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3d558640 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3d558640 1 -- 172.21.15.86:0/1518570798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 msgr2=0x55fd27eb5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3d558640 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27eb5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3d558640 1 -- 172.21.15.86:0/1518570798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd27f309a0 con 0x55fd27eb23d0 2024-04-16T05:37:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3e55a640 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27eb5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d3d558640 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb23d0 0x55fd27eb5440 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0d2809beb0 tx=0x7f0d2809bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8197866dff997645 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d217fa640 1 -- 172.21.15.86:0/1518570798 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28020040 con 0x55fd27eb23d0 2024-04-16T05:37:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d217fa640 1 -- 172.21.15.86:0/1518570798 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0d28098820 con 0x55fd27eb23d0 2024-04-16T05:37:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d42ba9740 1 -- 172.21.15.86:0/1518570798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d34032c90 con 0x55fd27eb23d0 2024-04-16T05:37:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.747+0000 7f0d217fa640 1 -- 172.21.15.86:0/1518570798 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d2809c1b0 con 0x55fd27eb23d0 2024-04-16T05:37:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d42ba9740 10 client.16488 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d42ba9740 1 -- 172.21.15.86:0/1518570798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fd27f31d10 con 0x55fd27eb23d0 2024-04-16T05:37:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d217fa640 1 -- 172.21.15.86:0/1518570798 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d34032c90 con 0x55fd27eb23d0 2024-04-16T05:37:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d217fa640 1 -- 172.21.15.86:0/1518570798 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0d280cdbe0 con 0x55fd27eb23d0 2024-04-16T05:37:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d42ba9740 10 client.16488 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d42ba9740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d42ba9740 20 client.16488 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fd27f32470 0x55fd27f34890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d42ba9740 4 client.16488 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:37:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d42ba9740 1 -- 172.21.15.86:0/1518570798 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fd27e1e720 con 0x55fd27f32470 2024-04-16T05:37:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.748+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fd27f32470 0x55fd27f34890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.749+0000 7f0d217fa640 10 client.16488 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.749+0000 7f0d3dd59640 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fd27f32470 0x55fd27f34890 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f6b019e36df1a2 server_cookie=5c062fb444348cab in_seq=0 out_seq=0 2024-04-16T05:37:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d217fa640 1 -- 172.21.15.86:0/1518570798 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fd27e1e720 con 0x55fd27f32470 2024-04-16T05:37:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d217fa640 10 client.16488 handle_command_reply: tid=0 2024-04-16T05:37:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 client.16488 shutdown 2024-04-16T05:37:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 -- 172.21.15.86:0/1518570798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb23d0 msgr2=0x55fd27eb5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb23d0 0x55fd27eb5440 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0d2809beb0 tx=0x7f0d2809bdb0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 -- 172.21.15.86:0/1518570798 shutdown_connections 2024-04-16T05:37:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fd27f32470 0x55fd27f34890 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd27ebafb0 0x55fd27eb5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd27eb23d0 0x55fd27eb5440 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d217fa640 0 client.16488 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 --2- 172.21.15.86:0/1518570798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd27eb1ab0 0x55fd27eb4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 -- 172.21.15.86:0/1518570798 >> 172.21.15.86:0/1518570798 conn(0x55fd27eac1e0 msgr2=0x55fd27eaddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 -- 172.21.15.86:0/1518570798 shutdown_connections 2024-04-16T05:37:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 1 -- 172.21.15.86:0/1518570798 wait complete. 2024-04-16T05:37:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.750+0000 7f0d42ba9740 20 client.16488 trim_cache size 0 max 16384 2024-04-16T05:37:35.759 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:37:35.759 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:37:35.759 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "7ea803fd-7e6d-46db-b2f9-54f6ffc6aaa4", 2024-04-16T05:37:35.759 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:37:35.759 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:37:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d240a3d20 msgr2=0x7f0d240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d240a3d20 0x7f0d240a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0d30021000 tx=0x7f0d30020000 comp rx=0 tx=0).stop 2024-04-16T05:37:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d381073d0 msgr2=0x7f0d3819f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d381073d0 0x7f0d3819f4e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800d770 tx=0x7f0d2800dc30 comp rx=0 tx=0).stop 2024-04-16T05:37:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/2560687853 shutdown_connections 2024-04-16T05:37:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d240a3d20 0x7f0d240a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d381a3860 0x7f0d381a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d381073d0 0x7f0d3819f4e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 --2- 172.21.15.86:0/2560687853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d38106ff0 0x7f0d3819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/2560687853 >> 172.21.15.86:0/2560687853 conn(0x7f0d38076d00 msgr2=0x7f0d38103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/2560687853 shutdown_connections 2024-04-16T05:37:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.751+0000 7f0d3ffbd640 1 -- 172.21.15.86:0/2560687853 wait complete. 2024-04-16T05:37:35.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:37:35.762 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:37:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.870+0000 7f47f9a63640 1 -- 172.21.15.86:0/2698031911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4106220 msgr2=0x7f47f410ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.870+0000 7f47f9a63640 1 --2- 172.21.15.86:0/2698031911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4106220 0x7f47f410ddf0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009900 tx=0x7f47ec02bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.870+0000 7f47f9a63640 1 -- 172.21.15.86:0/2698031911 shutdown_connections 2024-04-16T05:37:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.870+0000 7f47f9a63640 1 --2- 172.21.15.86:0/2698031911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4106220 0x7f47f410ddf0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.870+0000 7f47f9a63640 1 --2- 172.21.15.86:0/2698031911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f4101b00 0x7f47f4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.870+0000 7f47f9a63640 1 --2- 172.21.15.86:0/2698031911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f4101150 0x7f47f4101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.870+0000 7f47f9a63640 1 -- 172.21.15.86:0/2698031911 >> 172.21.15.86:0/2698031911 conn(0x7f47f4077f60 msgr2=0x7f47f4078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.870+0000 7f47f9a63640 1 -- 172.21.15.86:0/2698031911 shutdown_connections 2024-04-16T05:37:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.870+0000 7f47f9a63640 1 -- 172.21.15.86:0/2698031911 wait complete. 2024-04-16T05:37:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.871+0000 7f47f9a63640 1 Processor -- start 2024-04-16T05:37:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.871+0000 7f47f9a63640 1 -- start start 2024-04-16T05:37:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.871+0000 7f47f9a63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4101150 0x7f47f4102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.871+0000 7f47f9a63640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f4106220 0x7f47f4103120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.871+0000 7f47f9a63640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f4103660 0x7f47f41a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.871+0000 7f47f9a63640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41112e0 con 0x7f47f4101150 2024-04-16T05:37:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.871+0000 7f47f9a63640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4111160 con 0x7f47f4103660 2024-04-16T05:37:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.871+0000 7f47f9a63640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4111460 con 0x7f47f4106220 2024-04-16T05:37:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4101150 0x7f47f4102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4101150 0x7f47f4102be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57224/0 (socket says 172.21.15.86:57224) 2024-04-16T05:37:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f37fe640 1 -- 172.21.15.86:0/4015803625 learned_addr learned my addr 172.21.15.86:0/4015803625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f4106220 0x7f47f4103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f3fff640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f4103660 0x7f47f41a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f37fe640 1 -- 172.21.15.86:0/4015803625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f4106220 msgr2=0x7f47f4103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f37fe640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f4106220 0x7f47f4103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f37fe640 1 -- 172.21.15.86:0/4015803625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f4103660 msgr2=0x7f47f41a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f37fe640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f4103660 0x7f47f41a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f37fe640 1 -- 172.21.15.86:0/4015803625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f41a7130 con 0x7f47f4101150 2024-04-16T05:37:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f37fe640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4101150 0x7f47f4102be0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f47e400d770 tx=0x7f47e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f0ff9640 1 -- 172.21.15.86:0/4015803625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4017070 con 0x7f47f4101150 2024-04-16T05:37:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f0ff9640 1 -- 172.21.15.86:0/4015803625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4005500 con 0x7f47f4101150 2024-04-16T05:37:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f9a63640 1 -- 172.21.15.86:0/4015803625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f41a7420 con 0x7f47f4101150 2024-04-16T05:37:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f0ff9640 1 -- 172.21.15.86:0/4015803625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4004e00 con 0x7f47f4101150 2024-04-16T05:37:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.872+0000 7f47f9a63640 1 -- 172.21.15.86:0/4015803625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f41a7920 con 0x7f47f4101150 2024-04-16T05:37:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.874+0000 7f47f0ff9640 1 -- 172.21.15.86:0/4015803625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f47e4004fa0 con 0x7f47f4101150 2024-04-16T05:37:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47fc652740 1 Processor -- start 2024-04-16T05:37:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47fc652740 1 -- start start 2024-04-16T05:37:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47fc652740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47fc652740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1c3770 0x56050e1cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47fc652740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1cc350 0x56050e1ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47fc652740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56050e128180 con 0x56050e1cc350 2024-04-16T05:37:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47fc652740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56050e0c5360 con 0x56050e1c3770 2024-04-16T05:37:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47fc652740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56050e124140 con 0x56050e1c2e50 2024-04-16T05:37:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f0ff9640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47dc0a3d50 0x7f47dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1cc350 0x56050e1ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1cc350 0x56050e1ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57240/0 (socket says 172.21.15.86:57240) 2024-04-16T05:37:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f2ffd640 1 -- 172.21.15.86:0/743203338 learned_addr learned my addr 172.21.15.86:0/743203338 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f37fe640 1 --2- 172.21.15.86:0/743203338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1c3770 0x56050e1cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f3fff640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47dc0a3d50 0x7f47dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f3fff640 1 --2- 172.21.15.86:0/743203338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f0ff9640 1 -- 172.21.15.86:0/4015803625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f47e40d0460 con 0x7f47f4101150 2024-04-16T05:37:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f2ffd640 1 -- 172.21.15.86:0/743203338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 msgr2=0x56050e1c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/743203338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f2ffd640 1 -- 172.21.15.86:0/743203338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1c3770 msgr2=0x56050e1cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/743203338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1c3770 0x56050e1cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.875+0000 7f47f2ffd640 1 -- 172.21.15.86:0/743203338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050df047d0 con 0x56050e1cc350 2024-04-16T05:37:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/743203338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1cc350 0x56050e1ce740 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009f90 tx=0x7f47e8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2492419569fede1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47f3fff640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47dc0a3d50 0x7f47dc0a6210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47ec005ab0 tx=0x7f47ec005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47d9ffb640 1 -- 172.21.15.86:0/743203338 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8016070 con 0x56050e1cc350 2024-04-16T05:37:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47d9ffb640 1 -- 172.21.15.86:0/743203338 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8004d20 con 0x56050e1cc350 2024-04-16T05:37:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47d9ffb640 1 -- 172.21.15.86:0/743203338 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8005460 con 0x56050e1cc350 2024-04-16T05:37:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47fc652740 1 -- 172.21.15.86:0/743203338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1cc350 msgr2=0x56050e1ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47fc652740 1 --2- 172.21.15.86:0/743203338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1cc350 0x56050e1ce740 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009f90 tx=0x7f47e8010800 comp rx=0 tx=0).stop 2024-04-16T05:37:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47fc652740 1 -- 172.21.15.86:0/743203338 shutdown_connections 2024-04-16T05:37:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47fc652740 1 --2- 172.21.15.86:0/743203338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1cc350 0x56050e1ce740 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47fc652740 1 --2- 172.21.15.86:0/743203338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1c3770 0x56050e1cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47fc652740 1 --2- 172.21.15.86:0/743203338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.876+0000 7f47fc652740 1 -- 172.21.15.86:0/743203338 >> 172.21.15.86:0/743203338 conn(0x56050e1bd580 msgr2=0x56050e1bd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 -- 172.21.15.86:0/743203338 shutdown_connections 2024-04-16T05:37:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 -- 172.21.15.86:0/743203338 wait complete. 2024-04-16T05:37:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 -- start start 2024-04-16T05:37:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 Processor -- start 2024-04-16T05:37:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 0x56050e1c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56050e1cf140 con 0x56050e1c3770 2024-04-16T05:37:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56050e0e12c0 con 0x56050e1cc350 2024-04-16T05:37:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56050e1cf2c0 con 0x56050e1c2e50 2024-04-16T05:37:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47fc652740 10 client.0 _ll_register_callbacks cb 0x56050e0c8090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57250/0 (socket says 172.21.15.86:57250) 2024-04-16T05:37:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.877+0000 7f47f2ffd640 1 -- 172.21.15.86:0/712608843 learned_addr learned my addr 172.21.15.86:0/712608843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47f3fff640 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 0x56050e1c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47f37fe640 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47f2ffd640 1 -- 172.21.15.86:0/712608843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 msgr2=0x56050e1c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47f2ffd640 1 -- 172.21.15.86:0/712608843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 msgr2=0x56050e1c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 0x56050e1c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47f2ffd640 1 -- 172.21.15.86:0/712608843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050de3a3f0 con 0x56050e1c3770 2024-04-16T05:37:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1c98f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f47e8005060 tx=0x7f47e8004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1b9958eb75aad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47d2ffd640 1 -- 172.21.15.86:0/712608843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8029060 con 0x56050e1c3770 2024-04-16T05:37:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47d2ffd640 1 -- 172.21.15.86:0/712608843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f47e8012510 con 0x56050e1c3770 2024-04-16T05:37:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47fc652740 1 -- 172.21.15.86:0/712608843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e8004d20 con 0x56050e1c3770 2024-04-16T05:37:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47d2ffd640 1 -- 172.21.15.86:0/712608843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47e8030050 con 0x56050e1c3770 2024-04-16T05:37:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47fc652740 10 client.16500 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47fc652740 1 -- 172.21.15.86:0/712608843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56050ddfa0f0 con 0x56050e1c3770 2024-04-16T05:37:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.878+0000 7f47d2ffd640 1 -- 172.21.15.86:0/712608843 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8004d20 con 0x56050e1c3770 2024-04-16T05:37:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47d2ffd640 1 -- 172.21.15.86:0/712608843 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f47e800f040 con 0x56050e1c3770 2024-04-16T05:37:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47fc652740 10 client.16500 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47fc652740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47fc652740 20 client.16500 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47fc652740 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56050e1dc640 0x56050e1dea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47fc652740 4 client.16500 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47fc652740 1 -- 172.21.15.86:0/712608843 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050e135ba0 con 0x56050e1dc640 2024-04-16T05:37:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47f37fe640 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56050e1dc640 0x56050e1dea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47d2ffd640 10 client.16500 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.879+0000 7f47f37fe640 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56050e1dc640 0x56050e1dea30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c474547a50e3d79 server_cookie=ba4b4092dbceafc6 in_seq=0 out_seq=0 2024-04-16T05:37:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47d2ffd640 1 -- 172.21.15.86:0/712608843 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56050e135ba0 con 0x56050e1dc640 2024-04-16T05:37:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47d2ffd640 10 client.16500 handle_command_reply: tid=0 2024-04-16T05:37:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 client.16500 shutdown 2024-04-16T05:37:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 -- 172.21.15.86:0/712608843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 msgr2=0x56050e1c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1c98f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f47e8005060 tx=0x7f47e8004c70 comp rx=0 tx=0).stop 2024-04-16T05:37:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 -- 172.21.15.86:0/712608843 shutdown_connections 2024-04-16T05:37:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56050e1dc640 0x56050e1dea30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 0x56050e1c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1c98f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 --2- 172.21.15.86:0/712608843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47d2ffd640 0 client.16500 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 -- 172.21.15.86:0/712608843 >> 172.21.15.86:0/712608843 conn(0x56050e1bd580 msgr2=0x56050e1bf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 -- 172.21.15.86:0/712608843 shutdown_connections 2024-04-16T05:37:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 1 -- 172.21.15.86:0/712608843 wait complete. 2024-04-16T05:37:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.880+0000 7f47fc652740 20 client.16500 trim_cache size 0 max 16384 2024-04-16T05:37:35.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47fc652740 1 Processor -- start 2024-04-16T05:37:35.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47fc652740 1 -- start start 2024-04-16T05:37:35.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47fc652740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1c2e50 0x56050e1cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47fc652740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47fc652740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1cc350 0x56050e1b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47fc652740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b1610 con 0x56050e1c3770 2024-04-16T05:37:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47fc652740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b3150 con 0x56050e1c2e50 2024-04-16T05:37:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47fc652740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b1ce0 con 0x56050e1cc350 2024-04-16T05:37:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57258/0 (socket says 172.21.15.86:57258) 2024-04-16T05:37:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47f2ffd640 1 -- 172.21.15.86:0/3100942337 learned_addr learned my addr 172.21.15.86:0/3100942337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47f2ffd640 1 -- 172.21.15.86:0/3100942337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1cc350 msgr2=0x56050e1b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/3100942337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1cc350 0x56050e1b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47f2ffd640 1 -- 172.21.15.86:0/3100942337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1c2e50 msgr2=0x56050e1cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/3100942337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1c2e50 0x56050e1cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47f2ffd640 1 -- 172.21.15.86:0/3100942337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050e1c8360 con 0x56050e1c3770 2024-04-16T05:37:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/3100942337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d8000 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f47e8002230 tx=0x7f47e8009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3170392cb7eaa82 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47d97fa640 1 -- 172.21.15.86:0/3100942337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8029050 con 0x56050e1c3770 2024-04-16T05:37:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47d97fa640 1 -- 172.21.15.86:0/3100942337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8005a20 con 0x56050e1c3770 2024-04-16T05:37:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.898+0000 7f47d97fa640 1 -- 172.21.15.86:0/3100942337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8016070 con 0x56050e1c3770 2024-04-16T05:37:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 -- 172.21.15.86:0/3100942337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 msgr2=0x56050e1d8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 --2- 172.21.15.86:0/3100942337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d8000 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f47e8002230 tx=0x7f47e8009ec0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 -- 172.21.15.86:0/3100942337 shutdown_connections 2024-04-16T05:37:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 --2- 172.21.15.86:0/3100942337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1cc350 0x56050e1b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 --2- 172.21.15.86:0/3100942337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d8000 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 --2- 172.21.15.86:0/3100942337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1c2e50 0x56050e1cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 -- 172.21.15.86:0/3100942337 >> 172.21.15.86:0/3100942337 conn(0x56050e1bd580 msgr2=0x56050e1bc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 -- 172.21.15.86:0/3100942337 shutdown_connections 2024-04-16T05:37:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 -- 172.21.15.86:0/3100942337 wait complete. 2024-04-16T05:37:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.899+0000 7f47fc652740 1 -- start start 2024-04-16T05:37:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47fc652740 1 Processor -- start 2024-04-16T05:37:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47fc652740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47fc652740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47fc652740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 0x56050e1d2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47fc652740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b2a80 con 0x56050e1c3770 2024-04-16T05:37:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47fc652740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b23b0 con 0x56050e1cc350 2024-04-16T05:37:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47fc652740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41af5d0 con 0x56050e1c2e50 2024-04-16T05:37:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47fc652740 10 client.0 _ll_register_callbacks cb 0x7f47e8009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57266/0 (socket says 172.21.15.86:57266) 2024-04-16T05:37:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f2ffd640 1 -- 172.21.15.86:0/2937472124 learned_addr learned my addr 172.21.15.86:0/2937472124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:37:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f37fe640 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f2ffd640 1 -- 172.21.15.86:0/2937472124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 msgr2=0x56050e1d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f3fff640 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 0x56050e1d2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1d1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f2ffd640 1 -- 172.21.15.86:0/2937472124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 msgr2=0x56050e1d2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 0x56050e1d2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f2ffd640 1 -- 172.21.15.86:0/2937472124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050e24a060 con 0x56050e1c3770 2024-04-16T05:37:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f3fff640 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 0x56050e1d2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47f2ffd640 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d23e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f47e8003e80 tx=0x7f47e8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d295657b54d826 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.900+0000 7f47d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:37:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47d2ffd640 1 -- 172.21.15.86:0/2937472124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8043060 con 0x56050e1c3770 2024-04-16T05:37:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47d2ffd640 1 -- 172.21.15.86:0/2937472124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f47e8005810 con 0x56050e1c3770 2024-04-16T05:37:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47fc652740 1 -- 172.21.15.86:0/2937472124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e8005a20 con 0x56050e1c3770 2024-04-16T05:37:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47d2ffd640 1 -- 172.21.15.86:0/2937472124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47e8016030 con 0x56050e1c3770 2024-04-16T05:37:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47fc652740 10 client.16506 fetch_fsmap learned FSMap version 29 2024-04-16T05:37:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47fc652740 1 -- 172.21.15.86:0/2937472124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56050e24b2b0 con 0x56050e1c3770 2024-04-16T05:37:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47d2ffd640 1 -- 172.21.15.86:0/2937472124 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8005a20 con 0x56050e1c3770 2024-04-16T05:37:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47d2ffd640 1 -- 172.21.15.86:0/2937472124 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f47e8024460 con 0x56050e1c3770 2024-04-16T05:37:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47fc652740 10 client.16506 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:37:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47fc652740 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47fc652740 20 client.16506 populate_metadata read hostname 'smithi086' 2024-04-16T05:37:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47fc652740 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56050e24b8e0 0x56050e24dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47fc652740 4 client.16506 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:37:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47fc652740 1 -- 172.21.15.86:0/2937472124 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56050e12fa10 con 0x56050e24b8e0 2024-04-16T05:37:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.901+0000 7f47f37fe640 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56050e24b8e0 0x56050e24dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47d2ffd640 10 client.16506 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47f37fe640 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56050e24b8e0 0x56050e24dd00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a95aebae7e8535 server_cookie=418a13e01a16a893 in_seq=0 out_seq=0 2024-04-16T05:37:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47d2ffd640 1 -- 172.21.15.86:0/2937472124 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56050e12fa10 con 0x56050e24b8e0 2024-04-16T05:37:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47d2ffd640 10 client.16506 handle_command_reply: tid=0 2024-04-16T05:37:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 client.16506 shutdown 2024-04-16T05:37:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 -- 172.21.15.86:0/2937472124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 msgr2=0x56050e1d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d23e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f47e8003e80 tx=0x7f47e8009cd0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 -- 172.21.15.86:0/2937472124 shutdown_connections 2024-04-16T05:37:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56050e24b8e0 0x56050e24dd00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56050e1cc350 0x56050e1d2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56050e1c3770 0x56050e1d23e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 --2- 172.21.15.86:0/2937472124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56050e1c2e50 0x56050e1d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47d2ffd640 0 client.16506 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:37:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 -- 172.21.15.86:0/2937472124 >> 172.21.15.86:0/2937472124 conn(0x56050e1bd580 msgr2=0x56050e1c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 -- 172.21.15.86:0/2937472124 shutdown_connections 2024-04-16T05:37:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 1 -- 172.21.15.86:0/2937472124 wait complete. 2024-04-16T05:37:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.902+0000 7f47fc652740 20 client.16506 trim_cache size 0 max 16384 2024-04-16T05:37:35.912 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:37:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:37:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:37:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "7ea803fd-7e6d-46db-b2f9-54f6ffc6aaa4": { 2024-04-16T05:37:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:37:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "7ea803fd-7e6d-46db-b2f9-54f6ffc6aaa4", 2024-04-16T05:37:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:37:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:37:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:37:35.913 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:37:35.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 -- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47dc0a3d50 msgr2=0x7f47dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47dc0a3d50 0x7f47dc0a6210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47ec005ab0 tx=0x7f47ec005fd0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 -- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4101150 msgr2=0x7f47f4102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4101150 0x7f47f4102be0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f47e400d770 tx=0x7f47e400dc30 comp rx=0 tx=0).stop 2024-04-16T05:37:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 -- 172.21.15.86:0/4015803625 shutdown_connections 2024-04-16T05:37:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47dc0a3d50 0x7f47dc0a6210 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f4103660 0x7f47f41a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f4106220 0x7f47f4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 --2- 172.21.15.86:0/4015803625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f4101150 0x7f47f4102be0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 -- 172.21.15.86:0/4015803625 >> 172.21.15.86:0/4015803625 conn(0x7f47f4077f60 msgr2=0x7f47f4104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 -- 172.21.15.86:0/4015803625 shutdown_connections 2024-04-16T05:37:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:37:35.903+0000 7f47f9a63640 1 -- 172.21.15.86:0/4015803625 wait complete. 2024-04-16T05:37:35.916 INFO:tasks.cephfs.filesystem:scrub status for tag:7ea803fd-7e6d-46db-b2f9-54f6ffc6aaa4 - {'path': '/', 'tag': '7ea803fd-7e6d-46db-b2f9-54f6ffc6aaa4', 'options': 'recursive,force'} 2024-04-16T05:37:36.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:36 smithi194 ceph-mon[35865]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:36.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:36 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:37:36.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:36 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:36.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:36 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:37:36.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:36 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:36.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:36 smithi086 ceph-mon[32028]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:36.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:36 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:37:36.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:36 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:36.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:36 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:37:36.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:36 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:36 smithi196 ceph-mon[36373]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:36 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:37:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:36 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:36 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:37:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:36 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:38.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:38 smithi194 ceph-mon[35865]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:38.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:38 smithi086 ceph-mon[32028]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:38 smithi196 ceph-mon[36373]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:39.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:39 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:37:39.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:39 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:37:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:39 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:37:40.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:40 smithi194 ceph-mon[35865]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-04-16T05:37:40.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:40 smithi086 ceph-mon[32028]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-04-16T05:37:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:40 smithi196 ceph-mon[36373]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-04-16T05:37:42.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:42 smithi194 ceph-mon[35865]: pgmap v75: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-04-16T05:37:42.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:42 smithi086 ceph-mon[32028]: pgmap v75: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-04-16T05:37:42.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:42 smithi196 ceph-mon[36373]: pgmap v75: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-04-16T05:37:44.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:44 smithi194 ceph-mon[35865]: pgmap v76: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 12 op/s 2024-04-16T05:37:44.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:44 smithi086 ceph-mon[32028]: pgmap v76: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 12 op/s 2024-04-16T05:37:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:44 smithi196 ceph-mon[36373]: pgmap v76: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 12 op/s 2024-04-16T05:37:46.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:46 smithi194 ceph-mon[35865]: pgmap v77: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:37:46.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:46 smithi086 ceph-mon[32028]: pgmap v77: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:37:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:46 smithi196 ceph-mon[36373]: pgmap v77: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:37:48.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:48 smithi194 ceph-mon[35865]: pgmap v78: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:37:48.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:48 smithi086 ceph-mon[32028]: pgmap v78: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:37:48.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:48 smithi196 ceph-mon[36373]: pgmap v78: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:37:50.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:50 smithi194 ceph-mon[35865]: pgmap v79: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:37:50.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:50 smithi086 ceph-mon[32028]: pgmap v79: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:37:50.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:50 smithi196 ceph-mon[36373]: pgmap v79: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:37:52.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:52 smithi194 ceph-mon[35865]: pgmap v80: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:52.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:52 smithi086 ceph-mon[32028]: pgmap v80: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:52.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:52 smithi196 ceph-mon[36373]: pgmap v80: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:54.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:54 smithi194 ceph-mon[35865]: pgmap v81: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:37:54.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:54 smithi086 ceph-mon[32028]: pgmap v81: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:37:54.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:54 smithi196 ceph-mon[36373]: pgmap v81: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:37:56.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:56 smithi194 ceph-mon[35865]: pgmap v82: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:56.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:56 smithi086 ceph-mon[32028]: pgmap v82: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:56.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:56 smithi196 ceph-mon[36373]: pgmap v82: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:58.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:37:58 smithi194 ceph-mon[35865]: pgmap v83: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:58.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:37:58 smithi086 ceph-mon[32028]: pgmap v83: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:37:58.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:37:58 smithi196 ceph-mon[36373]: pgmap v83: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:00 smithi194 ceph-mon[35865]: pgmap v84: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:38:00.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:00 smithi086 ceph-mon[32028]: pgmap v84: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:38:00.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:00 smithi196 ceph-mon[36373]: pgmap v84: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:38:02.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:02 smithi194 ceph-mon[35865]: pgmap v85: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:38:02.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:02 smithi086 ceph-mon[32028]: pgmap v85: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:38:02.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:02 smithi196 ceph-mon[36373]: pgmap v85: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:38:04.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:04 smithi194 ceph-mon[35865]: pgmap v86: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:38:04.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:04 smithi086 ceph-mon[32028]: pgmap v86: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:38:04.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:04 smithi196 ceph-mon[36373]: pgmap v86: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:38:05.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:38:05.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:38:06.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 -- 172.21.15.86:0/401107270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8107310 msgr2=0x7faae810eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 --2- 172.21.15.86:0/401107270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8107310 0x7faae810eee0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7faae0009ca0 tx=0x7faae002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 -- 172.21.15.86:0/401107270 shutdown_connections 2024-04-16T05:38:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 --2- 172.21.15.86:0/401107270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8107310 0x7faae810eee0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 --2- 172.21.15.86:0/401107270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8102bf0 0x7faae8106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 --2- 172.21.15.86:0/401107270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faae8102240 0x7faae8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 -- 172.21.15.86:0/401107270 >> 172.21.15.86:0/401107270 conn(0x7faae8079450 msgr2=0x7faae8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 -- 172.21.15.86:0/401107270 shutdown_connections 2024-04-16T05:38:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 -- 172.21.15.86:0/401107270 wait complete. 2024-04-16T05:38:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.038+0000 7faaeeeba640 1 Processor -- start 2024-04-16T05:38:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaeeeba640 1 -- start start 2024-04-16T05:38:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaeeeba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faae8102240 0x7faae8074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaeeeba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8107310 0x7faae8074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaeeeba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8074d80 0x7faae81a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaeeeba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faae81123b0 con 0x7faae8074d80 2024-04-16T05:38:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaeeeba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faae8112230 con 0x7faae8102240 2024-04-16T05:38:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaeeeba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faae8112530 con 0x7faae8107310 2024-04-16T05:38:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaed457640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8074d80 0x7faae81a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaed457640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8074d80 0x7faae81a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50984/0 (socket says 172.21.15.86:50984) 2024-04-16T05:38:06.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaed457640 1 -- 172.21.15.86:0/2896809263 learned_addr learned my addr 172.21.15.86:0/2896809263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaecc56640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faae8102240 0x7faae8074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaed457640 1 -- 172.21.15.86:0/2896809263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8107310 msgr2=0x7faae8074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faae7fff640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8107310 0x7faae8074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaed457640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8107310 0x7faae8074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaed457640 1 -- 172.21.15.86:0/2896809263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faae8102240 msgr2=0x7faae8074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaed457640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faae8102240 0x7faae8074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaed457640 1 -- 172.21.15.86:0/2896809263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae81a7120 con 0x7faae8074d80 2024-04-16T05:38:06.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faae7fff640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8107310 0x7faae8074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:06.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faaed457640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8074d80 0x7faae81a6a80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faae002c700 tx=0x7faae0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.039+0000 7faae5ffb640 1 -- 172.21.15.86:0/2896809263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0039040 con 0x7faae8074d80 2024-04-16T05:38:06.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.040+0000 7faae5ffb640 1 -- 172.21.15.86:0/2896809263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faae0044070 con 0x7faae8074d80 2024-04-16T05:38:06.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.040+0000 7faaeeeba640 1 -- 172.21.15.86:0/2896809263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae81a73b0 con 0x7faae8074d80 2024-04-16T05:38:06.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.040+0000 7faae5ffb640 1 -- 172.21.15.86:0/2896809263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae00042a0 con 0x7faae8074d80 2024-04-16T05:38:06.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.040+0000 7faaeeeba640 1 -- 172.21.15.86:0/2896809263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae81a78a0 con 0x7faae8074d80 2024-04-16T05:38:06.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.042+0000 7faae5ffb640 1 -- 172.21.15.86:0/2896809263 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faae0004440 con 0x7faae8074d80 2024-04-16T05:38:06.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.042+0000 7faaf1b77740 1 Processor -- start 2024-04-16T05:38:06.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.042+0000 7faaf1b77740 1 -- start start 2024-04-16T05:38:06.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.042+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da7a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da82e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da85740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faaf1b77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563c4d9df180 con 0x563c4da83350 2024-04-16T05:38:06.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faaf1b77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c4d97c360 con 0x563c4da79e50 2024-04-16T05:38:06.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faaf1b77740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c4d9db140 con 0x563c4da7a770 2024-04-16T05:38:06.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da85740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da85740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50992/0 (socket says 172.21.15.86:50992) 2024-04-16T05:38:06.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 -- 172.21.15.86:0/1628460607 learned_addr learned my addr 172.21.15.86:0/1628460607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 --2- 172.21.15.86:0/1628460607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da7a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faaecc56640 1 --2- 172.21.15.86:0/1628460607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da82e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae5ffb640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7faad80a3d20 0x7faad80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 -- 172.21.15.86:0/1628460607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 msgr2=0x563c4da82e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 --2- 172.21.15.86:0/1628460607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da82e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 -- 172.21.15.86:0/1628460607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 msgr2=0x563c4da7a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 --2- 172.21.15.86:0/1628460607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da7a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faaed457640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7faad80a3d20 0x7faad80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 -- 172.21.15.86:0/1628460607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c4d7bb7d0 con 0x563c4da83350 2024-04-16T05:38:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae5ffb640 1 -- 172.21.15.86:0/2896809263 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faae00f2e30 con 0x7faae8074d80 2024-04-16T05:38:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faae7fff640 1 --2- 172.21.15.86:0/1628460607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da85740 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faad4005fd0 tx=0x7faad400e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba5892af30c6e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.043+0000 7faad2ffd640 1 -- 172.21.15.86:0/1628460607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad400b6a0 con 0x563c4da83350 2024-04-16T05:38:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaed457640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7faad80a3d20 0x7faad80a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faae00be870 tx=0x7faae00905b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faad2ffd640 1 -- 172.21.15.86:0/1628460607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faad400bd20 con 0x563c4da83350 2024-04-16T05:38:06.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faad2ffd640 1 -- 172.21.15.86:0/1628460607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad40103d0 con 0x563c4da83350 2024-04-16T05:38:06.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaf1b77740 1 -- 172.21.15.86:0/1628460607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 msgr2=0x563c4da85740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1628460607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da85740 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faad4005fd0 tx=0x7faad400e3d0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaf1b77740 1 -- 172.21.15.86:0/1628460607 shutdown_connections 2024-04-16T05:38:06.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1628460607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da85740 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1628460607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da82e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1628460607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da7a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaf1b77740 1 -- 172.21.15.86:0/1628460607 >> 172.21.15.86:0/1628460607 conn(0x563c4da74580 msgr2=0x563c4da74960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaf1b77740 1 -- 172.21.15.86:0/1628460607 shutdown_connections 2024-04-16T05:38:06.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.044+0000 7faaf1b77740 1 -- 172.21.15.86:0/1628460607 wait complete. 2024-04-16T05:38:06.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaf1b77740 1 -- start start 2024-04-16T05:38:06.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaf1b77740 1 Processor -- start 2024-04-16T05:38:06.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da80930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da80e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaf1b77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563c4da86140 con 0x563c4da83350 2024-04-16T05:38:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaf1b77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c4d9982c0 con 0x563c4da79e50 2024-04-16T05:38:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaf1b77740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c4da862c0 con 0x563c4da7a770 2024-04-16T05:38:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaf1b77740 10 client.0 _ll_register_callbacks cb 0x563c4d97f090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faae7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da80e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faae7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da80e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51004/0 (socket says 172.21.15.86:51004) 2024-04-16T05:38:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faae7fff640 1 -- 172.21.15.86:0/692554551 learned_addr learned my addr 172.21.15.86:0/692554551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faae7fff640 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faaecc56640 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da80930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faae7fff640 1 -- 172.21.15.86:0/692554551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 msgr2=0x563c4da80930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faae7fff640 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da80930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faae7fff640 1 -- 172.21.15.86:0/692554551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 msgr2=0x563c4da803f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faae7fff640 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da803f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.045+0000 7faae7fff640 1 -- 172.21.15.86:0/692554551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c4d6f13f0 con 0x563c4da83350 2024-04-16T05:38:06.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faae7fff640 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da80e70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faad4001eb0 tx=0x7faad4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b4ac97dc8f148f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faac7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:06.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faac7fff640 1 -- 172.21.15.86:0/692554551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad40113e0 con 0x563c4da83350 2024-04-16T05:38:06.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faac7fff640 1 -- 172.21.15.86:0/692554551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faad4011580 con 0x563c4da83350 2024-04-16T05:38:06.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faaf1b77740 1 -- 172.21.15.86:0/692554551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faad400bd20 con 0x563c4da83350 2024-04-16T05:38:06.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faac7fff640 1 -- 172.21.15.86:0/692554551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faad4034050 con 0x563c4da83350 2024-04-16T05:38:06.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faaf1b77740 10 client.16518 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:06.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faaf1b77740 1 -- 172.21.15.86:0/692554551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c4d6b10f0 con 0x563c4da83350 2024-04-16T05:38:06.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faac7fff640 1 -- 172.21.15.86:0/692554551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faad400bd20 con 0x563c4da83350 2024-04-16T05:38:06.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faac7fff640 1 -- 172.21.15.86:0/692554551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faad402c020 con 0x563c4da83350 2024-04-16T05:38:06.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faaf1b77740 10 client.16518 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:06.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faaf1b77740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:06.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faaf1b77740 20 client.16518 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:06.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faaf1b77740 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563c4da93620 0x563c4da95a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faaf1b77740 4 client.16518 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.046+0000 7faaf1b77740 1 -- 172.21.15.86:0/692554551 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c4d9ecba0 con 0x563c4da93620 2024-04-16T05:38:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.047+0000 7faaecc56640 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563c4da93620 0x563c4da95a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.047+0000 7faaecc56640 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563c4da93620 0x563c4da95a10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e866bb1dbf75ef8c server_cookie=3b7a4d6346f50992 in_seq=0 out_seq=0 2024-04-16T05:38:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.047+0000 7faac7fff640 10 client.16518 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faac7fff640 1 -- 172.21.15.86:0/692554551 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c4d9ecba0 con 0x563c4da93620 2024-04-16T05:38:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faac7fff640 10 client.16518 handle_command_reply: tid=0 2024-04-16T05:38:06.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 client.16518 shutdown 2024-04-16T05:38:06.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 -- 172.21.15.86:0/692554551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 msgr2=0x563c4da80e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da80e70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faad4001eb0 tx=0x7faad4011af0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 -- 172.21.15.86:0/692554551 shutdown_connections 2024-04-16T05:38:06.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563c4da93620 0x563c4da95a10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da80e70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da80930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faac7fff640 0 client.16518 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 --2- 172.21.15.86:0/692554551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da803f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 -- 172.21.15.86:0/692554551 >> 172.21.15.86:0/692554551 conn(0x563c4da74580 msgr2=0x563c4da762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 -- 172.21.15.86:0/692554551 shutdown_connections 2024-04-16T05:38:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 1 -- 172.21.15.86:0/692554551 wait complete. 2024-04-16T05:38:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.048+0000 7faaf1b77740 20 client.16518 trim_cache size 0 max 16384 2024-04-16T05:38:06.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faaf1b77740 1 Processor -- start 2024-04-16T05:38:06.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faaf1b77740 1 -- start start 2024-04-16T05:38:06.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da82b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da8f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da9b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faaf1b77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faae81b1680 con 0x563c4da83350 2024-04-16T05:38:06.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faaf1b77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faae81b31c0 con 0x563c4da79e50 2024-04-16T05:38:06.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faaf1b77740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faae81b1d50 con 0x563c4da7a770 2024-04-16T05:38:06.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faae7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da9b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faae7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da9b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51018/0 (socket says 172.21.15.86:51018) 2024-04-16T05:38:06.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faae7fff640 1 -- 172.21.15.86:0/2292475444 learned_addr learned my addr 172.21.15.86:0/2292475444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faae7fff640 1 --2- 172.21.15.86:0/2292475444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da82b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faaecc56640 1 --2- 172.21.15.86:0/2292475444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da8f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faae7fff640 1 -- 172.21.15.86:0/2292475444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 msgr2=0x563c4da8f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faae7fff640 1 --2- 172.21.15.86:0/2292475444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da8f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faae7fff640 1 -- 172.21.15.86:0/2292475444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 msgr2=0x563c4da82b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faae7fff640 1 --2- 172.21.15.86:0/2292475444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da82b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.066+0000 7faae7fff640 1 -- 172.21.15.86:0/2292475444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c4da7b1c0 con 0x563c4da83350 2024-04-16T05:38:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faae7fff640 1 --2- 172.21.15.86:0/2292475444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da9b070 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faad4011980 tx=0x7faad40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b722b4d83295d920 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faad27fc640 1 -- 172.21.15.86:0/2292475444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad400e870 con 0x563c4da83350 2024-04-16T05:38:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faad27fc640 1 -- 172.21.15.86:0/2292475444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faad4029040 con 0x563c4da83350 2024-04-16T05:38:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faad27fc640 1 -- 172.21.15.86:0/2292475444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad400bc30 con 0x563c4da83350 2024-04-16T05:38:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faaf1b77740 1 -- 172.21.15.86:0/2292475444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 msgr2=0x563c4da9b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faaf1b77740 1 --2- 172.21.15.86:0/2292475444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da9b070 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faad4011980 tx=0x7faad40056d0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faaf1b77740 1 -- 172.21.15.86:0/2292475444 shutdown_connections 2024-04-16T05:38:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faaf1b77740 1 --2- 172.21.15.86:0/2292475444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da83350 0x563c4da9b070 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faaf1b77740 1 --2- 172.21.15.86:0/2292475444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da7a770 0x563c4da8f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faaf1b77740 1 --2- 172.21.15.86:0/2292475444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da79e50 0x563c4da82b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faaf1b77740 1 -- 172.21.15.86:0/2292475444 >> 172.21.15.86:0/2292475444 conn(0x563c4da74580 msgr2=0x563c4da76260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faaf1b77740 1 -- 172.21.15.86:0/2292475444 shutdown_connections 2024-04-16T05:38:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.067+0000 7faaf1b77740 1 -- 172.21.15.86:0/2292475444 wait complete. 2024-04-16T05:38:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaf1b77740 1 -- start start 2024-04-16T05:38:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaf1b77740 1 Processor -- start 2024-04-16T05:38:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da79e50 0x563c4da8b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da7a770 0x563c4da88ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaf1b77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da83350 0x563c4da89020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaf1b77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faae81b2af0 con 0x563c4da7a770 2024-04-16T05:38:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaf1b77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faae81b2420 con 0x563c4da83350 2024-04-16T05:38:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaf1b77740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faae81af640 con 0x563c4da79e50 2024-04-16T05:38:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaf1b77740 10 client.0 _ll_register_callbacks cb 0x7faad400ee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaecc56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da7a770 0x563c4da88ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaecc56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da7a770 0x563c4da88ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51022/0 (socket says 172.21.15.86:51022) 2024-04-16T05:38:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faaecc56640 1 -- 172.21.15.86:0/1952808555 learned_addr learned my addr 172.21.15.86:0/1952808555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.068+0000 7faae7fff640 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da79e50 0x563c4da8b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaecc56640 1 -- 172.21.15.86:0/1952808555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da79e50 msgr2=0x563c4da8b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaecc56640 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da79e50 0x563c4da8b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaecc56640 1 -- 172.21.15.86:0/1952808555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da83350 msgr2=0x563c4da89020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaecc56640 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da83350 0x563c4da89020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaecc56640 1 -- 172.21.15.86:0/1952808555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c4da89560 con 0x563c4da7a770 2024-04-16T05:38:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaecc56640 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da7a770 0x563c4da88ae0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x563c4da79920 tx=0x7faadc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7552ec4f58e6be12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faac7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faac7fff640 1 -- 172.21.15.86:0/1952808555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc01d910 con 0x563c4da7a770 2024-04-16T05:38:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faac7fff640 1 -- 172.21.15.86:0/1952808555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faadc00f420 con 0x563c4da7a770 2024-04-16T05:38:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaf1b77740 1 -- 172.21.15.86:0/1952808555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faad4029040 con 0x563c4da7a770 2024-04-16T05:38:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faac7fff640 1 -- 172.21.15.86:0/1952808555 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faadc02d050 con 0x563c4da7a770 2024-04-16T05:38:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaf1b77740 10 client.16524 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaf1b77740 1 -- 172.21.15.86:0/1952808555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c4daf96c0 con 0x563c4da7a770 2024-04-16T05:38:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faac7fff640 1 -- 172.21.15.86:0/1952808555 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faad4029040 con 0x563c4da7a770 2024-04-16T05:38:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faac7fff640 1 -- 172.21.15.86:0/1952808555 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faadc025920 con 0x563c4da7a770 2024-04-16T05:38:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaf1b77740 10 client.16524 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaf1b77740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaf1b77740 20 client.16524 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563c4daf9ec0 0x563c4dafc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaf1b77740 4 client.16524 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:38:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.069+0000 7faaf1b77740 1 -- 172.21.15.86:0/1952808555 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c4d9e6a10 con 0x563c4daf9ec0 2024-04-16T05:38:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.070+0000 7faae7fff640 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563c4daf9ec0 0x563c4dafc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.070+0000 7faac7fff640 10 client.16524 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.070+0000 7faae7fff640 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563c4daf9ec0 0x563c4dafc2e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51efc777e2636b55 server_cookie=435ab7bb643f18bd in_seq=0 out_seq=0 2024-04-16T05:38:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.070+0000 7faac7fff640 1 -- 172.21.15.86:0/1952808555 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c4d9e6a10 con 0x563c4daf9ec0 2024-04-16T05:38:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.070+0000 7faac7fff640 10 client.16524 handle_command_reply: tid=0 2024-04-16T05:38:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.070+0000 7faaf1b77740 1 client.16524 shutdown 2024-04-16T05:38:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 -- 172.21.15.86:0/1952808555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da7a770 msgr2=0x563c4da88ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da7a770 0x563c4da88ae0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x563c4da79920 tx=0x7faadc011b00 comp rx=0 tx=0).stop 2024-04-16T05:38:06.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 -- 172.21.15.86:0/1952808555 shutdown_connections 2024-04-16T05:38:06.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563c4daf9ec0 0x563c4dafc2e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c4da83350 0x563c4da89020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c4da7a770 0x563c4da88ae0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faac7fff640 0 client.16524 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:06.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 --2- 172.21.15.86:0/1952808555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c4da79e50 0x563c4da8b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 -- 172.21.15.86:0/1952808555 >> 172.21.15.86:0/1952808555 conn(0x563c4da74580 msgr2=0x563c4da75920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 -- 172.21.15.86:0/1952808555 shutdown_connections 2024-04-16T05:38:06.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 1 -- 172.21.15.86:0/1952808555 wait complete. 2024-04-16T05:38:06.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaf1b77740 20 client.16524 trim_cache size 0 max 16384 2024-04-16T05:38:06.083 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:38:06.083 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:38:06.084 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:38:06.084 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:38:06.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 -- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7faad80a3d20 msgr2=0x7faad80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7faad80a3d20 0x7faad80a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faae00be870 tx=0x7faae00905b0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 -- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8074d80 msgr2=0x7faae81a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8074d80 0x7faae81a6a80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faae002c700 tx=0x7faae0032f20 comp rx=0 tx=0).stop 2024-04-16T05:38:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 -- 172.21.15.86:0/2896809263 shutdown_connections 2024-04-16T05:38:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8074d80 0x7faae81a6a80 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8107310 0x7faae8074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7faad80a3d20 0x7faad80a61e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 --2- 172.21.15.86:0/2896809263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faae8102240 0x7faae8074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.071+0000 7faaeeeba640 1 -- 172.21.15.86:0/2896809263 >> 172.21.15.86:0/2896809263 conn(0x7faae8079450 msgr2=0x7faae80ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.072+0000 7faaeeeba640 1 -- 172.21.15.86:0/2896809263 shutdown_connections 2024-04-16T05:38:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.072+0000 7faaeeeba640 1 -- 172.21.15.86:0/2896809263 wait complete. 2024-04-16T05:38:06.087 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:38:06.087 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:38:06.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.192+0000 7f2d35bd5640 1 -- 172.21.15.86:0/2772195947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30101140 msgr2=0x7f2d30101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.192+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/2772195947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30101140 0x7f2d30101520 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009900 tx=0x7f2d2002bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.192+0000 7f2d35bd5640 1 -- 172.21.15.86:0/2772195947 shutdown_connections 2024-04-16T05:38:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.192+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/2772195947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d30106210 0x7f2d3010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.192+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/2772195947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d30101af0 0x7f2d30105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.192+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/2772195947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30101140 0x7f2d30101520 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.192+0000 7f2d35bd5640 1 -- 172.21.15.86:0/2772195947 >> 172.21.15.86:0/2772195947 conn(0x7f2d30077280 msgr2=0x7f2d30077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.192+0000 7f2d35bd5640 1 -- 172.21.15.86:0/2772195947 shutdown_connections 2024-04-16T05:38:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.192+0000 7f2d35bd5640 1 -- 172.21.15.86:0/2772195947 wait complete. 2024-04-16T05:38:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d35bd5640 1 Processor -- start 2024-04-16T05:38:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d35bd5640 1 -- start start 2024-04-16T05:38:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d35bd5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d30101140 0x7f2d3019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d35bd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30106210 0x7f2d3019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d35bd5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d30198720 0x7f2d30198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d35bd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301123e0 con 0x7f2d30106210 2024-04-16T05:38:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d35bd5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2d30112260 con 0x7f2d30198720 2024-04-16T05:38:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d35bd5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2d30112560 con 0x7f2d30101140 2024-04-16T05:38:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30106210 0x7f2d3019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30106210 0x7f2d3019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51048/0 (socket says 172.21.15.86:51048) 2024-04-16T05:38:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d27fff640 1 -- 172.21.15.86:0/551675067 learned_addr learned my addr 172.21.15.86:0/551675067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.193+0000 7f2d2f7fe640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d30101140 0x7f2d3019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d27fff640 1 -- 172.21.15.86:0/551675067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d30101140 msgr2=0x7f2d3019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d27fff640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d30101140 0x7f2d3019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d27fff640 1 -- 172.21.15.86:0/551675067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d30198720 msgr2=0x7f2d30198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d2ffff640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d30198720 0x7f2d30198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d27fff640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d30198720 0x7f2d30198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d27fff640 1 -- 172.21.15.86:0/551675067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d30199320 con 0x7f2d30106210 2024-04-16T05:38:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d2f7fe640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d30101140 0x7f2d3019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d2ffff640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d30198720 0x7f2d30198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d27fff640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30106210 0x7f2d3019eba0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800c970 tx=0x7f2d1800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d2d7fa640 1 -- 172.21.15.86:0/551675067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18007bb0 con 0x7f2d30106210 2024-04-16T05:38:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d2d7fa640 1 -- 172.21.15.86:0/551675067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d18007d50 con 0x7f2d30106210 2024-04-16T05:38:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d301a5400 con 0x7f2d30106210 2024-04-16T05:38:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d2d7fa640 1 -- 172.21.15.86:0/551675067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800f450 con 0x7f2d30106210 2024-04-16T05:38:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.194+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d301a5800 con 0x7f2d30106210 2024-04-16T05:38:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.195+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d30102610 con 0x7f2d30106210 2024-04-16T05:38:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.196+0000 7f2d2d7fa640 1 -- 172.21.15.86:0/551675067 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d180046c0 con 0x7f2d30106210 2024-04-16T05:38:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.197+0000 7f2d2d7fa640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2d1c0a3d20 0x7f2d1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.197+0000 7f2d2f7fe640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2d1c0a3d20 0x7f2d1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.197+0000 7f2d2d7fa640 1 -- 172.21.15.86:0/551675067 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2d180d0c30 con 0x7f2d30106210 2024-04-16T05:38:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.197+0000 7f2d2f7fe640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2d1c0a3d20 0x7f2d1c0a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d20036040 tx=0x7f2d20000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.202+0000 7f2d2d7fa640 1 -- 172.21.15.86:0/551675067 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d18014030 con 0x7f2d30106210 2024-04-16T05:38:06.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.419+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d30069270 con 0x7f2d30106210 2024-04-16T05:38:06.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.420+0000 7f2d2d7fa640 1 -- 172.21.15.86:0/551675067 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f2d1809cf60 con 0x7f2d30106210 2024-04-16T05:38:06.420 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:38:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:38:06.420 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:38:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2d1c0a3d20 msgr2=0x7f2d1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2d1c0a3d20 0x7f2d1c0a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d20036040 tx=0x7f2d20000bc0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30106210 msgr2=0x7f2d3019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30106210 0x7f2d3019eba0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800c970 tx=0x7f2d1800ce30 comp rx=0 tx=0).stop 2024-04-16T05:38:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 shutdown_connections 2024-04-16T05:38:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2d1c0a3d20 0x7f2d1c0a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d30198720 0x7f2d30198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d30106210 0x7f2d3019eba0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 --2- 172.21.15.86:0/551675067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d30101140 0x7f2d3019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.422+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 >> 172.21.15.86:0/551675067 conn(0x7f2d30077280 msgr2=0x7f2d300fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.423+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 shutdown_connections 2024-04-16T05:38:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.423+0000 7f2d35bd5640 1 -- 172.21.15.86:0/551675067 wait complete. 2024-04-16T05:38:06.434 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:38:06.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 -- 172.21.15.86:0/3274937392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102c00 msgr2=0x7f3988106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 --2- 172.21.15.86:0/3274937392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102c00 0x7f3988106c40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3978009900 tx=0x7f397802bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 -- 172.21.15.86:0/3274937392 shutdown_connections 2024-04-16T05:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 --2- 172.21.15.86:0/3274937392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3988107320 0x7f398810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 --2- 172.21.15.86:0/3274937392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102c00 0x7f3988106c40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 --2- 172.21.15.86:0/3274937392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3988102250 0x7f3988102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 -- 172.21.15.86:0/3274937392 >> 172.21.15.86:0/3274937392 conn(0x7f3988079490 msgr2=0x7f39880798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 -- 172.21.15.86:0/3274937392 shutdown_connections 2024-04-16T05:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 -- 172.21.15.86:0/3274937392 wait complete. 2024-04-16T05:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 Processor -- start 2024-04-16T05:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.542+0000 7f39906f2640 1 -- start start 2024-04-16T05:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f39906f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102250 0x7f398819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f39906f2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3988107320 0x7f398819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f39906f2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39881987f0 0x7f3988198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f39906f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3988112490 con 0x7f3988102250 2024-04-16T05:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f39906f2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3988112310 con 0x7f39881987f0 2024-04-16T05:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f39906f2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3988112610 con 0x7f3988107320 2024-04-16T05:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398e48e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102250 0x7f398819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398e48e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102250 0x7f398819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51076/0 (socket says 172.21.15.86:51076) 2024-04-16T05:38:06.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398e48e640 1 -- 172.21.15.86:0/2589605983 learned_addr learned my addr 172.21.15.86:0/2589605983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398dc8d640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3988107320 0x7f398819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398ec8f640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39881987f0 0x7f3988198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398e48e640 1 -- 172.21.15.86:0/2589605983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3988107320 msgr2=0x7f398819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398e48e640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3988107320 0x7f398819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398e48e640 1 -- 172.21.15.86:0/2589605983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39881987f0 msgr2=0x7f3988198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398e48e640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39881987f0 0x7f3988198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398e48e640 1 -- 172.21.15.86:0/2589605983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3988199380 con 0x7f3988102250 2024-04-16T05:38:06.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f398e48e640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102250 0x7f398819e630 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f398400b4a0 tx=0x7f398400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f397f7fe640 1 -- 172.21.15.86:0/2589605983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984014070 con 0x7f3988102250 2024-04-16T05:38:06.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f39906f2640 1 -- 172.21.15.86:0/2589605983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3988199670 con 0x7f3988102250 2024-04-16T05:38:06.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f397f7fe640 1 -- 172.21.15.86:0/2589605983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3984010af0 con 0x7f3988102250 2024-04-16T05:38:06.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f397f7fe640 1 -- 172.21.15.86:0/2589605983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984017d10 con 0x7f3988102250 2024-04-16T05:38:06.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.543+0000 7f39906f2640 1 -- 172.21.15.86:0/2589605983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39881a57d0 con 0x7f3988102250 2024-04-16T05:38:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.545+0000 7f397f7fe640 1 -- 172.21.15.86:0/2589605983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3984017070 con 0x7f3988102250 2024-04-16T05:38:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.546+0000 7f39933bc740 1 Processor -- start 2024-04-16T05:38:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.546+0000 7f39933bc740 1 -- start start 2024-04-16T05:38:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.546+0000 7f397f7fe640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39740a3d20 0x7f39740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.546+0000 7f39933bc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5254e60 0x558df5255240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.546+0000 7f39933bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df525de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.546+0000 7f39933bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df5260750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.546+0000 7f39933bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558df51ba0e0 con 0x558df525e360 2024-04-16T05:38:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.546+0000 7f39933bc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558df5157520 con 0x558df5254e60 2024-04-16T05:38:06.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.546+0000 7f39933bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558df51b60a0 con 0x558df5255780 2024-04-16T05:38:06.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f397f7fe640 1 -- 172.21.15.86:0/2589605983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f39840cfda0 con 0x7f3988102250 2024-04-16T05:38:06.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39740a3d20 0x7f39740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df5260750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df5260750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51084/0 (socket says 172.21.15.86:51084) 2024-04-16T05:38:06.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 -- 172.21.15.86:0/3458633445 learned_addr learned my addr 172.21.15.86:0/3458633445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398dc8d640 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df525de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398e48e640 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5254e60 0x558df5255240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 -- 172.21.15.86:0/3458633445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 msgr2=0x558df525de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df525de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 -- 172.21.15.86:0/3458633445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5254e60 msgr2=0x558df5255240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5254e60 0x558df5255240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 -- 172.21.15.86:0/3458633445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df517f8d0 con 0x558df525e360 2024-04-16T05:38:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398dc8d640 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df525de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:38:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39740a3d20 0x7f39740a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f398000bdd0 tx=0x7f398000b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f398ec8f640 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df5260750 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3980008b20 tx=0x7f398001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e66c464a08e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f397cff9640 1 -- 172.21.15.86:0/3458633445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980014040 con 0x558df525e360 2024-04-16T05:38:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f397cff9640 1 -- 172.21.15.86:0/3458633445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980009e20 con 0x558df525e360 2024-04-16T05:38:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.547+0000 7f397cff9640 1 -- 172.21.15.86:0/3458633445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398001a610 con 0x558df525e360 2024-04-16T05:38:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 -- 172.21.15.86:0/3458633445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 msgr2=0x558df5260750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df5260750 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3980008b20 tx=0x7f398001a000 comp rx=0 tx=0).stop 2024-04-16T05:38:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 -- 172.21.15.86:0/3458633445 shutdown_connections 2024-04-16T05:38:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df5260750 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df525de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 --2- 172.21.15.86:0/3458633445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5254e60 0x558df5255240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 -- 172.21.15.86:0/3458633445 >> 172.21.15.86:0/3458633445 conn(0x558df524f590 msgr2=0x558df524f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 -- 172.21.15.86:0/3458633445 shutdown_connections 2024-04-16T05:38:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 -- 172.21.15.86:0/3458633445 wait complete. 2024-04-16T05:38:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.548+0000 7f39933bc740 1 -- start start 2024-04-16T05:38:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f39933bc740 1 Processor -- start 2024-04-16T05:38:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f39933bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5254e60 0x558df525b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f39933bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df5255780 0x558df525b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f39933bc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df525e360 0x558df525bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f39933bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558df5260fa0 con 0x558df5255780 2024-04-16T05:38:06.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f39933bc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558df51734d0 con 0x558df525e360 2024-04-16T05:38:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f39933bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558df5261120 con 0x558df5254e60 2024-04-16T05:38:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f39933bc740 10 client.0 _ll_register_callbacks cb 0x558df51582c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398dc8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df5255780 0x558df525b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398dc8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df5255780 0x558df525b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51100/0 (socket says 172.21.15.86:51100) 2024-04-16T05:38:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398dc8d640 1 -- 172.21.15.86:0/3283760696 learned_addr learned my addr 172.21.15.86:0/3283760696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398ec8f640 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df525e360 0x558df525bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398e48e640 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5254e60 0x558df525b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398dc8d640 1 -- 172.21.15.86:0/3283760696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5254e60 msgr2=0x558df525b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398dc8d640 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5254e60 0x558df525b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398dc8d640 1 -- 172.21.15.86:0/3283760696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df525e360 msgr2=0x558df525bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398dc8d640 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df525e360 0x558df525bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398dc8d640 1 -- 172.21.15.86:0/3283760696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df4ecc3f0 con 0x558df5255780 2024-04-16T05:38:06.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398ec8f640 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df525e360 0x558df525bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:06.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f398dc8d640 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df5255780 0x558df525b9d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3978009230 tx=0x7f3978004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a931fce2fcddfd5c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f3971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:06.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f3971ffb640 1 -- 172.21.15.86:0/3283760696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39780046c0 con 0x558df5255780 2024-04-16T05:38:06.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.549+0000 7f3971ffb640 1 -- 172.21.15.86:0/3283760696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3978004860 con 0x558df5255780 2024-04-16T05:38:06.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f39933bc740 1 -- 172.21.15.86:0/3283760696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558df51c7b00 con 0x558df5255780 2024-04-16T05:38:06.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f3971ffb640 1 -- 172.21.15.86:0/3283760696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3978004a60 con 0x558df5255780 2024-04-16T05:38:06.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f39933bc740 10 client.16542 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:06.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f39933bc740 1 -- 172.21.15.86:0/3283760696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558df4f967d0 con 0x558df5255780 2024-04-16T05:38:06.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f3971ffb640 1 -- 172.21.15.86:0/3283760696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558df51c7b00 con 0x558df5255780 2024-04-16T05:38:06.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f3971ffb640 1 -- 172.21.15.86:0/3283760696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3978003820 con 0x558df5255780 2024-04-16T05:38:06.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f39933bc740 10 client.16542 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:06.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f39933bc740 10 client.16542 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:38:06.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f39933bc740 20 client.16542 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:06.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f39933bc740 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x558df526e360 0x558df52707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f39933bc740 4 client.16542 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:06.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f39933bc740 1 -- 172.21.15.86:0/3283760696 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df51c1970 con 0x558df526e360 2024-04-16T05:38:06.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.550+0000 7f398e48e640 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x558df526e360 0x558df52707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.551+0000 7f398e48e640 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x558df526e360 0x558df52707e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=99871c8d04acd0c8 server_cookie=e76560cd43daa1bf in_seq=0 out_seq=0 2024-04-16T05:38:06.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.551+0000 7f3971ffb640 10 client.16542 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:38:06.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.551+0000 7f3971ffb640 1 -- 172.21.15.86:0/3283760696 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558df51c1970 con 0x558df526e360 2024-04-16T05:38:06.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.551+0000 7f3971ffb640 10 client.16542 handle_command_reply: tid=0 2024-04-16T05:38:06.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 client.16542 shutdown 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 -- 172.21.15.86:0/3283760696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df5255780 msgr2=0x558df525b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df5255780 0x558df525b9d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3978009230 tx=0x7f3978004040 comp rx=0 tx=0).stop 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 -- 172.21.15.86:0/3283760696 shutdown_connections 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x558df526e360 0x558df52707e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df525e360 0x558df525bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df5255780 0x558df525b9d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f3971ffb640 0 client.16542 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 --2- 172.21.15.86:0/3283760696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5254e60 0x558df525b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 -- 172.21.15.86:0/3283760696 >> 172.21.15.86:0/3283760696 conn(0x558df524f590 msgr2=0x558df52512e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 -- 172.21.15.86:0/3283760696 shutdown_connections 2024-04-16T05:38:06.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 1 -- 172.21.15.86:0/3283760696 wait complete. 2024-04-16T05:38:06.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.553+0000 7f39933bc740 20 client.16542 trim_cache size 0 max 16384 2024-04-16T05:38:06.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f39933bc740 1 Processor -- start 2024-04-16T05:38:06.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f39933bc740 1 -- start start 2024-04-16T05:38:06.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f39933bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5254e60 0x558df525d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f39933bc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5255780 0x558df52760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f39933bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f39933bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39881af490 con 0x558df525e360 2024-04-16T05:38:06.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f39933bc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b0fd0 con 0x558df5255780 2024-04-16T05:38:06.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f39933bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39881afb60 con 0x558df5254e60 2024-04-16T05:38:06.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398ec8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398ec8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52765f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51112/0 (socket says 172.21.15.86:51112) 2024-04-16T05:38:06.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398ec8f640 1 -- 172.21.15.86:0/3438575263 learned_addr learned my addr 172.21.15.86:0/3438575263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398ec8f640 1 -- 172.21.15.86:0/3438575263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5254e60 msgr2=0x558df525d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398ec8f640 1 --2- 172.21.15.86:0/3438575263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5254e60 0x558df525d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398ec8f640 1 -- 172.21.15.86:0/3438575263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5255780 msgr2=0x558df52760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398dc8d640 1 --2- 172.21.15.86:0/3438575263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5255780 0x558df52760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398ec8f640 1 --2- 172.21.15.86:0/3438575263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5255780 0x558df52760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398ec8f640 1 -- 172.21.15.86:0/3438575263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df5270e60 con 0x558df525e360 2024-04-16T05:38:06.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.565+0000 7f398dc8d640 1 --2- 172.21.15.86:0/3438575263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5255780 0x558df52760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:06.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f398ec8f640 1 --2- 172.21.15.86:0/3438575263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52765f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3980004b90 tx=0x7f39800049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b203ecf5777ba8d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f3973fff640 1 -- 172.21.15.86:0/3438575263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980034070 con 0x558df525e360 2024-04-16T05:38:06.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f3973fff640 1 -- 172.21.15.86:0/3438575263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980019df0 con 0x558df525e360 2024-04-16T05:38:06.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f3973fff640 1 -- 172.21.15.86:0/3438575263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980024400 con 0x558df525e360 2024-04-16T05:38:06.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f39933bc740 1 -- 172.21.15.86:0/3438575263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 msgr2=0x558df52765f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f39933bc740 1 --2- 172.21.15.86:0/3438575263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52765f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3980004b90 tx=0x7f39800049c0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f39933bc740 1 -- 172.21.15.86:0/3438575263 shutdown_connections 2024-04-16T05:38:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f39933bc740 1 --2- 172.21.15.86:0/3438575263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52765f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f39933bc740 1 --2- 172.21.15.86:0/3438575263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5255780 0x558df52760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f39933bc740 1 --2- 172.21.15.86:0/3438575263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5254e60 0x558df525d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f39933bc740 1 -- 172.21.15.86:0/3438575263 >> 172.21.15.86:0/3438575263 conn(0x558df524f590 msgr2=0x558df525cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f39933bc740 1 -- 172.21.15.86:0/3438575263 shutdown_connections 2024-04-16T05:38:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.566+0000 7f39933bc740 1 -- 172.21.15.86:0/3438575263 wait complete. 2024-04-16T05:38:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f39933bc740 1 -- start start 2024-04-16T05:38:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f39933bc740 1 Processor -- start 2024-04-16T05:38:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f39933bc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5254e60 0x558df5266460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f39933bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df52669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f39933bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f39933bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b0900 con 0x558df525e360 2024-04-16T05:38:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f39933bc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b0230 con 0x558df5254e60 2024-04-16T05:38:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f39933bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39881ad450 con 0x558df5255780 2024-04-16T05:38:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f39933bc740 10 client.0 _ll_register_callbacks cb 0x7f3978009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398ec8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398ec8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51116/0 (socket says 172.21.15.86:51116) 2024-04-16T05:38:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398ec8f640 1 -- 172.21.15.86:0/4247845970 learned_addr learned my addr 172.21.15.86:0/4247845970 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398dc8d640 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df52669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398ec8f640 1 -- 172.21.15.86:0/4247845970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 msgr2=0x558df52669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398ec8f640 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df52669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398ec8f640 1 -- 172.21.15.86:0/4247845970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5254e60 msgr2=0x558df5266460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398ec8f640 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5254e60 0x558df5266460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398ec8f640 1 -- 172.21.15.86:0/4247845970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df52631e0 con 0x558df525e360 2024-04-16T05:38:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.567+0000 7f398dc8d640 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df52669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f398ec8f640 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52629e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f398002fb30 tx=0x7f3980008e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc77ea589798bb5d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f3971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f3971ffb640 1 -- 172.21.15.86:0/4247845970 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980034070 con 0x558df525e360 2024-04-16T05:38:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f3971ffb640 1 -- 172.21.15.86:0/4247845970 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f398002f420 con 0x558df525e360 2024-04-16T05:38:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f39933bc740 1 -- 172.21.15.86:0/4247845970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3980019df0 con 0x558df525e360 2024-04-16T05:38:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f3971ffb640 1 -- 172.21.15.86:0/4247845970 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f398002f620 con 0x558df525e360 2024-04-16T05:38:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f39933bc740 10 client.16548 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f39933bc740 1 -- 172.21.15.86:0/4247845970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558df52d54f0 con 0x558df525e360 2024-04-16T05:38:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f3971ffb640 1 -- 172.21.15.86:0/4247845970 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980019df0 con 0x558df525e360 2024-04-16T05:38:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f3971ffb640 1 -- 172.21.15.86:0/4247845970 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f398001a3f0 con 0x558df525e360 2024-04-16T05:38:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f39933bc740 10 client.16548 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f39933bc740 10 client.16548 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:38:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f39933bc740 20 client.16548 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f39933bc740 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x558df52d59c0 0x558df52d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f39933bc740 4 client.16548 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f39933bc740 1 -- 172.21.15.86:0/4247845970 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558df513d6b0 con 0x558df52d59c0 2024-04-16T05:38:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.568+0000 7f398e48e640 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x558df52d59c0 0x558df52d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f398e48e640 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x558df52d59c0 0x558df52d7db0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba0570610e16eb0a server_cookie=174e1475b10aa5ab in_seq=0 out_seq=0 2024-04-16T05:38:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f3971ffb640 10 client.16548 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:38:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f3971ffb640 1 -- 172.21.15.86:0/4247845970 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558df513d6b0 con 0x558df52d59c0 2024-04-16T05:38:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f3971ffb640 10 client.16548 handle_command_reply: tid=0 2024-04-16T05:38:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 client.16548 shutdown 2024-04-16T05:38:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 -- 172.21.15.86:0/4247845970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 msgr2=0x558df52629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52629e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f398002fb30 tx=0x7f3980008e60 comp rx=0 tx=0).stop 2024-04-16T05:38:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 -- 172.21.15.86:0/4247845970 shutdown_connections 2024-04-16T05:38:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x558df52d59c0 0x558df52d7db0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558df525e360 0x558df52629e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f3971ffb640 0 client.16548 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:38:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558df5255780 0x558df52669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 --2- 172.21.15.86:0/4247845970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558df5254e60 0x558df5266460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 -- 172.21.15.86:0/4247845970 >> 172.21.15.86:0/4247845970 conn(0x558df524f590 msgr2=0x558df524dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 -- 172.21.15.86:0/4247845970 shutdown_connections 2024-04-16T05:38:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 1 -- 172.21.15.86:0/4247845970 wait complete. 2024-04-16T05:38:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.569+0000 7f39933bc740 20 client.16548 trim_cache size 0 max 16384 2024-04-16T05:38:06.580 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:06.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 -- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39740a3d20 msgr2=0x7f39740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39740a3d20 0x7f39740a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f398000bdd0 tx=0x7f398000b410 comp rx=0 tx=0).stop 2024-04-16T05:38:06.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 -- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102250 msgr2=0x7f398819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102250 0x7f398819e630 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f398400b4a0 tx=0x7f398400b960 comp rx=0 tx=0).stop 2024-04-16T05:38:06.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 -- 172.21.15.86:0/2589605983 shutdown_connections 2024-04-16T05:38:06.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f39740a3d20 0x7f39740a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39881987f0 0x7f3988198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3988107320 0x7f398819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 --2- 172.21.15.86:0/2589605983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3988102250 0x7f398819e630 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 -- 172.21.15.86:0/2589605983 >> 172.21.15.86:0/2589605983 conn(0x7f3988079490 msgr2=0x7f39880ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 -- 172.21.15.86:0/2589605983 shutdown_connections 2024-04-16T05:38:06.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.570+0000 7f39906f2640 1 -- 172.21.15.86:0/2589605983 wait complete. 2024-04-16T05:38:06.582 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:38:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.690+0000 7fe6cb150640 1 -- 172.21.15.86:0/3531280863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 msgr2=0x7fe6c410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.690+0000 7fe6cb150640 1 --2- 172.21.15.86:0/3531280863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 0x7fe6c410dd50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009ca0 tx=0x7fe6bc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.690+0000 7fe6cb150640 1 -- 172.21.15.86:0/3531280863 shutdown_connections 2024-04-16T05:38:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.690+0000 7fe6cb150640 1 --2- 172.21.15.86:0/3531280863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 0x7fe6c410dd50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.690+0000 7fe6cb150640 1 --2- 172.21.15.86:0/3531280863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c4101150 0x7fe6c41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.690+0000 7fe6cb150640 1 --2- 172.21.15.86:0/3531280863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c4105e80 0x7fe6c4106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.690+0000 7fe6cb150640 1 -- 172.21.15.86:0/3531280863 >> 172.21.15.86:0/3531280863 conn(0x7fe6c4077cf0 msgr2=0x7fe6c4078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.690+0000 7fe6cb150640 1 -- 172.21.15.86:0/3531280863 shutdown_connections 2024-04-16T05:38:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.690+0000 7fe6cb150640 1 -- 172.21.15.86:0/3531280863 wait complete. 2024-04-16T05:38:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6cb150640 1 Processor -- start 2024-04-16T05:38:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6cb150640 1 -- start start 2024-04-16T05:38:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6cb150640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c4101150 0x7fe6c4197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6cb150640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 0x7fe6c4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6cb150640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c4105e80 0x7fe6c4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6cb150640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4111380 con 0x7fe6c4101af0 2024-04-16T05:38:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6cb150640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4111200 con 0x7fe6c4105e80 2024-04-16T05:38:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6cb150640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4111500 con 0x7fe6c4101150 2024-04-16T05:38:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 0x7fe6c4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 0x7fe6c4197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51124/0 (socket says 172.21.15.86:51124) 2024-04-16T05:38:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6bbfff640 1 -- 172.21.15.86:0/2969314938 learned_addr learned my addr 172.21.15.86:0/2969314938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6c96ed640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c4105e80 0x7fe6c4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.691+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c4101150 0x7fe6c4197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6bbfff640 1 -- 172.21.15.86:0/2969314938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c4101150 msgr2=0x7fe6c4197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c4101150 0x7fe6c4197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6bbfff640 1 -- 172.21.15.86:0/2969314938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c4105e80 msgr2=0x7fe6c4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c4105e80 0x7fe6c4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6bbfff640 1 -- 172.21.15.86:0/2969314938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c41a6da0 con 0x7fe6c4101af0 2024-04-16T05:38:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c4101150 0x7fe6c4197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 0x7fe6c4197a70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400c970 tx=0x7fe6b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6b9ffb640 1 -- 172.21.15.86:0/2969314938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4007bb0 con 0x7fe6c4101af0 2024-04-16T05:38:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6b9ffb640 1 -- 172.21.15.86:0/2969314938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b4007d50 con 0x7fe6c4101af0 2024-04-16T05:38:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6cb150640 1 -- 172.21.15.86:0/2969314938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c41a7090 con 0x7fe6c4101af0 2024-04-16T05:38:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6b9ffb640 1 -- 172.21.15.86:0/2969314938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b400f450 con 0x7fe6c4101af0 2024-04-16T05:38:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.692+0000 7fe6cb150640 1 -- 172.21.15.86:0/2969314938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c41a7590 con 0x7fe6c4101af0 2024-04-16T05:38:06.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.694+0000 7fe6b9ffb640 1 -- 172.21.15.86:0/2969314938 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe6b4005020 con 0x7fe6c4101af0 2024-04-16T05:38:06.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6cde12740 1 Processor -- start 2024-04-16T05:38:06.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6cde12740 1 -- start start 2024-04-16T05:38:06.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6b9ffb640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6ac0a3d20 0x7fe6ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f3d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3d780 0x561e56f45e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 0x561e56f48750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6cde12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e56ea20e0 con 0x561e56f3ce60 2024-04-16T05:38:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6cde12740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561e56e3f520 con 0x561e56f46360 2024-04-16T05:38:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6cde12740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e56e9e0a0 con 0x561e56f3d780 2024-04-16T05:38:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c96ed640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6ac0a3d20 0x7fe6ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c8eec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f3d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c8eec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f3d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51138/0 (socket says 172.21.15.86:51138) 2024-04-16T05:38:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6b9ffb640 1 -- 172.21.15.86:0/2969314938 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe6b40d0a30 con 0x7fe6c4101af0 2024-04-16T05:38:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c8eec640 1 -- 172.21.15.86:0/361345417 learned_addr learned my addr 172.21.15.86:0/361345417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c96ed640 1 --2- 172.21.15.86:0/361345417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 0x561e56f48750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c8eec640 1 -- 172.21.15.86:0/361345417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3d780 msgr2=0x561e56f45e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/361345417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3d780 0x561e56f45e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c8eec640 1 -- 172.21.15.86:0/361345417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 msgr2=0x561e56f48750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/361345417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 0x561e56f48750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c8eec640 1 -- 172.21.15.86:0/361345417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e56e678d0 con 0x561e56f3ce60 2024-04-16T05:38:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/361345417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f3d240 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000bb70 tx=0x7fe6c000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b120b68af12d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.695+0000 7fe6c96ed640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6ac0a3d20 0x7fe6ac0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc006010 tx=0x7fe6bc02cd70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6aaffd640 1 -- 172.21.15.86:0/361345417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0017620 con 0x561e56f3ce60 2024-04-16T05:38:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6aaffd640 1 -- 172.21.15.86:0/361345417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c0017ca0 con 0x561e56f3ce60 2024-04-16T05:38:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6aaffd640 1 -- 172.21.15.86:0/361345417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000fc00 con 0x561e56f3ce60 2024-04-16T05:38:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6cde12740 1 -- 172.21.15.86:0/361345417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 msgr2=0x561e56f3d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6cde12740 1 --2- 172.21.15.86:0/361345417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f3d240 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000bb70 tx=0x7fe6c000d710 comp rx=0 tx=0).stop 2024-04-16T05:38:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6cde12740 1 -- 172.21.15.86:0/361345417 shutdown_connections 2024-04-16T05:38:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6cde12740 1 --2- 172.21.15.86:0/361345417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 0x561e56f48750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6cde12740 1 --2- 172.21.15.86:0/361345417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3d780 0x561e56f45e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6cde12740 1 --2- 172.21.15.86:0/361345417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f3d240 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6cde12740 1 -- 172.21.15.86:0/361345417 >> 172.21.15.86:0/361345417 conn(0x561e56f37590 msgr2=0x561e56f37970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6cde12740 1 -- 172.21.15.86:0/361345417 shutdown_connections 2024-04-16T05:38:06.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.696+0000 7fe6cde12740 1 -- 172.21.15.86:0/361345417 wait complete. 2024-04-16T05:38:06.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6cde12740 1 -- start start 2024-04-16T05:38:06.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6cde12740 1 Processor -- start 2024-04-16T05:38:06.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f43420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 0x561e56f43960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f46360 0x561e56f43ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6cde12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e56f48fa0 con 0x561e56f3ce60 2024-04-16T05:38:06.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6cde12740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561e56e5b4d0 con 0x561e56f3d780 2024-04-16T05:38:06.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6cde12740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e56f49120 con 0x561e56f46360 2024-04-16T05:38:06.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6cde12740 10 client.0 _ll_register_callbacks cb 0x561e56e402c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:06.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6c8eec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f43420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6c8eec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f43420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51150/0 (socket says 172.21.15.86:51150) 2024-04-16T05:38:06.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6c8eec640 1 -- 172.21.15.86:0/935305883 learned_addr learned my addr 172.21.15.86:0/935305883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6c96ed640 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f46360 0x561e56f43ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.697+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 0x561e56f43960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6c8eec640 1 -- 172.21.15.86:0/935305883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f46360 msgr2=0x561e56f43ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f46360 0x561e56f43ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6c8eec640 1 -- 172.21.15.86:0/935305883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 msgr2=0x561e56f43960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 0x561e56f43960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6c8eec640 1 -- 172.21.15.86:0/935305883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e56bb43f0 con 0x561e56f3ce60 2024-04-16T05:38:06.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f43420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000f8c0 tx=0x7fe6c00176d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9f245ef59de1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:06.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6a3fff640 1 -- 172.21.15.86:0/935305883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0026070 con 0x561e56f3ce60 2024-04-16T05:38:06.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6a3fff640 1 -- 172.21.15.86:0/935305883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe6c0022640 con 0x561e56f3ce60 2024-04-16T05:38:06.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6cde12740 1 -- 172.21.15.86:0/935305883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c0017ca0 con 0x561e56f3ce60 2024-04-16T05:38:06.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6a3fff640 1 -- 172.21.15.86:0/935305883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6c0032050 con 0x561e56f3ce60 2024-04-16T05:38:06.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6cde12740 10 client.16560 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:06.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6cde12740 1 -- 172.21.15.86:0/935305883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e56c7e7d0 con 0x561e56f3ce60 2024-04-16T05:38:06.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6a3fff640 1 -- 172.21.15.86:0/935305883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c0017ca0 con 0x561e56f3ce60 2024-04-16T05:38:06.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.698+0000 7fe6a3fff640 1 -- 172.21.15.86:0/935305883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe6c0020410 con 0x561e56f3ce60 2024-04-16T05:38:06.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.699+0000 7fe6cde12740 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:06.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.699+0000 7fe6cde12740 10 client.16560 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:38:06.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.699+0000 7fe6cde12740 20 client.16560 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:06.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.699+0000 7fe6cde12740 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561e56f562e0 0x561e56f586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.699+0000 7fe6cde12740 4 client.16560 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:06.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.699+0000 7fe6cde12740 1 -- 172.21.15.86:0/935305883 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e56eafb00 con 0x561e56f562e0 2024-04-16T05:38:06.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.699+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561e56f562e0 0x561e56f586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.699+0000 7fe6a3fff640 10 client.16560 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:38:06.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.699+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561e56f562e0 0x561e56f586d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=82b0f302d382d7d2 server_cookie=3b8e2021f1877037 in_seq=0 out_seq=0 2024-04-16T05:38:06.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6a3fff640 1 -- 172.21.15.86:0/935305883 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e56eafb00 con 0x561e56f562e0 2024-04-16T05:38:06.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6a3fff640 10 client.16560 handle_command_reply: tid=0 2024-04-16T05:38:06.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 client.16560 shutdown 2024-04-16T05:38:06.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 -- 172.21.15.86:0/935305883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 msgr2=0x561e56f43420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f43420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000f8c0 tx=0x7fe6c00176d0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 -- 172.21.15.86:0/935305883 shutdown_connections 2024-04-16T05:38:06.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561e56f562e0 0x561e56f586d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f46360 0x561e56f43ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 0x561e56f43960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6a3fff640 0 client.16560 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:38:06.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 --2- 172.21.15.86:0/935305883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3ce60 0x561e56f43420 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 -- 172.21.15.86:0/935305883 >> 172.21.15.86:0/935305883 conn(0x561e56f37590 msgr2=0x561e56f392e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 -- 172.21.15.86:0/935305883 shutdown_connections 2024-04-16T05:38:06.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.700+0000 7fe6cde12740 1 -- 172.21.15.86:0/935305883 wait complete. 2024-04-16T05:38:06.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.701+0000 7fe6cde12740 20 client.16560 trim_cache size 0 max 16384 2024-04-16T05:38:06.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6cde12740 1 Processor -- start 2024-04-16T05:38:06.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6cde12740 1 -- start start 2024-04-16T05:38:06.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3ce60 0x561e56f45b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3d780 0x561e56f52000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 0x561e56f365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6cde12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b1660 con 0x561e56f3d780 2024-04-16T05:38:06.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6cde12740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b31a0 con 0x561e56f46360 2024-04-16T05:38:06.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6cde12740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b1d30 con 0x561e56f3ce60 2024-04-16T05:38:06.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3d780 0x561e56f52000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3d780 0x561e56f52000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51162/0 (socket says 172.21.15.86:51162) 2024-04-16T05:38:06.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6bbfff640 1 -- 172.21.15.86:0/3262181922 learned_addr learned my addr 172.21.15.86:0/3262181922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6c96ed640 1 --2- 172.21.15.86:0/3262181922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 0x561e56f365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6bbfff640 1 -- 172.21.15.86:0/3262181922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3ce60 msgr2=0x561e56f45b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/3262181922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3ce60 0x561e56f45b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6bbfff640 1 -- 172.21.15.86:0/3262181922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 msgr2=0x561e56f365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/3262181922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 0x561e56f365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6bbfff640 1 -- 172.21.15.86:0/3262181922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e56f3d5b0 con 0x561e56f3d780 2024-04-16T05:38:06.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/3262181922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3d780 0x561e56f52000 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40058c0 tx=0x7fe6b4004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653314006401780c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/3262181922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b401d040 con 0x561e56f3d780 2024-04-16T05:38:06.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/3262181922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b40a0a10 con 0x561e56f3d780 2024-04-16T05:38:06.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.714+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/3262181922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b409a790 con 0x561e56f3d780 2024-04-16T05:38:06.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 -- 172.21.15.86:0/3262181922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3d780 msgr2=0x561e56f52000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 --2- 172.21.15.86:0/3262181922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3d780 0x561e56f52000 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40058c0 tx=0x7fe6b4004920 comp rx=0 tx=0).stop 2024-04-16T05:38:06.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 -- 172.21.15.86:0/3262181922 shutdown_connections 2024-04-16T05:38:06.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 --2- 172.21.15.86:0/3262181922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f46360 0x561e56f365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 --2- 172.21.15.86:0/3262181922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f3d780 0x561e56f52000 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 --2- 172.21.15.86:0/3262181922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3ce60 0x561e56f45b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 -- 172.21.15.86:0/3262181922 >> 172.21.15.86:0/3262181922 conn(0x561e56f37590 msgr2=0x561e56f35ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 -- 172.21.15.86:0/3262181922 shutdown_connections 2024-04-16T05:38:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 -- 172.21.15.86:0/3262181922 wait complete. 2024-04-16T05:38:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 -- start start 2024-04-16T05:38:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.715+0000 7fe6cde12740 1 Processor -- start 2024-04-16T05:38:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3ce60 0x561e56f4e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 0x561e56f3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6cde12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f46360 0x561e56f40400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6cde12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b2ad0 con 0x561e56f46360 2024-04-16T05:38:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6cde12740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b2400 con 0x561e56f3d780 2024-04-16T05:38:06.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6cde12740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41af620 con 0x561e56f3ce60 2024-04-16T05:38:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6cde12740 10 client.0 _ll_register_callbacks cb 0x7fe6c000b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c96ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f46360 0x561e56f40400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c96ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f46360 0x561e56f40400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51168/0 (socket says 172.21.15.86:51168) 2024-04-16T05:38:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c96ed640 1 -- 172.21.15.86:0/4123091754 learned_addr learned my addr 172.21.15.86:0/4123091754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3ce60 0x561e56f4e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 0x561e56f3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c96ed640 1 -- 172.21.15.86:0/4123091754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3ce60 msgr2=0x561e56f4e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c96ed640 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3ce60 0x561e56f4e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c96ed640 1 -- 172.21.15.86:0/4123091754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 msgr2=0x561e56f3fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c96ed640 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 0x561e56f3fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c96ed640 1 -- 172.21.15.86:0/4123091754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e56f40c30 con 0x561e56f46360 2024-04-16T05:38:06.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6bbfff640 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 0x561e56f3fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:06.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6c96ed640 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f46360 0x561e56f40400 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009ca0 tx=0x7fe6bc02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ef9ae7f389e467 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:06.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6a3fff640 1 -- 172.21.15.86:0/4123091754 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc053070 con 0x561e56f46360 2024-04-16T05:38:06.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6cde12740 1 -- 172.21.15.86:0/4123091754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b40a0a10 con 0x561e56f46360 2024-04-16T05:38:06.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.716+0000 7fe6a3fff640 1 -- 172.21.15.86:0/4123091754 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe6bc004040 con 0x561e56f46360 2024-04-16T05:38:06.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6a3fff640 1 -- 172.21.15.86:0/4123091754 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6bc05c050 con 0x561e56f46360 2024-04-16T05:38:06.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6cde12740 10 client.16566 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:06.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6cde12740 1 -- 172.21.15.86:0/4123091754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e56fbc660 con 0x561e56f46360 2024-04-16T05:38:06.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6a3fff640 1 -- 172.21.15.86:0/4123091754 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b40a0a10 con 0x561e56f46360 2024-04-16T05:38:06.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6a3fff640 1 -- 172.21.15.86:0/4123091754 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe6bc056020 con 0x561e56f46360 2024-04-16T05:38:06.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6cde12740 10 client.16566 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:06.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6cde12740 10 client.16566 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:38:06.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6cde12740 20 client.16566 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:06.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6cde12740 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561e56fbcdf0 0x561e56fbf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6cde12740 4 client.16566 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:06.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6cde12740 1 -- 172.21.15.86:0/4123091754 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e56ea9970 con 0x561e56fbcdf0 2024-04-16T05:38:06.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.717+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561e56fbcdf0 0x561e56fbf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6a3fff640 10 client.16566 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:38:06.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6c8eec640 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561e56fbcdf0 0x561e56fbf210 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7dc4ae077fe9edeb server_cookie=7f15cf5fe56ddc5a in_seq=0 out_seq=0 2024-04-16T05:38:06.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6a3fff640 1 -- 172.21.15.86:0/4123091754 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e56ea9970 con 0x561e56fbcdf0 2024-04-16T05:38:06.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6a3fff640 10 client.16566 handle_command_reply: tid=0 2024-04-16T05:38:06.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 client.16566 shutdown 2024-04-16T05:38:06.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 -- 172.21.15.86:0/4123091754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f46360 msgr2=0x561e56f40400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f46360 0x561e56f40400 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009ca0 tx=0x7fe6bc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 -- 172.21.15.86:0/4123091754 shutdown_connections 2024-04-16T05:38:06.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e56f46360 0x561e56f40400 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e56f3d780 0x561e56f3fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x561e56fbcdf0 0x561e56fbf210 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 --2- 172.21.15.86:0/4123091754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e56f3ce60 0x561e56f4e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6a3fff640 0 client.16566 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:38:06.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 -- 172.21.15.86:0/4123091754 >> 172.21.15.86:0/4123091754 conn(0x561e56f37590 msgr2=0x561e56f43c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 -- 172.21.15.86:0/4123091754 shutdown_connections 2024-04-16T05:38:06.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 1 -- 172.21.15.86:0/4123091754 wait complete. 2024-04-16T05:38:06.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.718+0000 7fe6cde12740 20 client.16566 trim_cache size 0 max 16384 2024-04-16T05:38:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:06.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 -- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6ac0a3d20 msgr2=0x7fe6ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6ac0a3d20 0x7fe6ac0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc006010 tx=0x7fe6bc02cd70 comp rx=0 tx=0).stop 2024-04-16T05:38:06.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 -- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 msgr2=0x7fe6c4197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 0x7fe6c4197a70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400c970 tx=0x7fe6b400ce30 comp rx=0 tx=0).stop 2024-04-16T05:38:06.729 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:06 smithi086 ceph-mon[32028]: pgmap v87: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:38:06.729 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:06 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/551675067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:06.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 -- 172.21.15.86:0/2969314938 shutdown_connections 2024-04-16T05:38:06.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6ac0a3d20 0x7fe6ac0a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c4105e80 0x7fe6c4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c4101af0 0x7fe6c4197a70 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 --2- 172.21.15.86:0/2969314938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c4101150 0x7fe6c4197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 -- 172.21.15.86:0/2969314938 >> 172.21.15.86:0/2969314938 conn(0x7fe6c4077cf0 msgr2=0x7fe6c40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 -- 172.21.15.86:0/2969314938 shutdown_connections 2024-04-16T05:38:06.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.719+0000 7fe6cb150640 1 -- 172.21.15.86:0/2969314938 wait complete. 2024-04-16T05:38:06.732 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:38:06.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 -- 172.21.15.86:0/2190965222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0102250 msgr2=0x7fada0102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 --2- 172.21.15.86:0/2190965222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0102250 0x7fada0102630 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fad94009900 tx=0x7fad9402bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 -- 172.21.15.86:0/2190965222 shutdown_connections 2024-04-16T05:38:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 --2- 172.21.15.86:0/2190965222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0107320 0x7fada010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 --2- 172.21.15.86:0/2190965222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fada0102c00 0x7fada0106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 --2- 172.21.15.86:0/2190965222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0102250 0x7fada0102630 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 -- 172.21.15.86:0/2190965222 >> 172.21.15.86:0/2190965222 conn(0x7fada0079490 msgr2=0x7fada00798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 -- 172.21.15.86:0/2190965222 shutdown_connections 2024-04-16T05:38:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 -- 172.21.15.86:0/2190965222 wait complete. 2024-04-16T05:38:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.840+0000 7fada6e6a640 1 Processor -- start 2024-04-16T05:38:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada6e6a640 1 -- start start 2024-04-16T05:38:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada6e6a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0102250 0x7fada019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada6e6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0107320 0x7fada019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada6e6a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fada0198670 0x7fada0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada6e6a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fada01123b0 con 0x7fada0107320 2024-04-16T05:38:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada6e6a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fada0112230 con 0x7fada0198670 2024-04-16T05:38:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada6e6a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fada0112530 con 0x7fada0102250 2024-04-16T05:38:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0107320 0x7fada019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0107320 0x7fada019eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51174/0 (socket says 172.21.15.86:51174) 2024-04-16T05:38:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9ffff640 1 -- 172.21.15.86:0/329300883 learned_addr learned my addr 172.21.15.86:0/329300883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada5407640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fada0198670 0x7fada0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada4c06640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0102250 0x7fada019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9ffff640 1 -- 172.21.15.86:0/329300883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0102250 msgr2=0x7fada019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9ffff640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0102250 0x7fada019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9ffff640 1 -- 172.21.15.86:0/329300883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fada0198670 msgr2=0x7fada0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9ffff640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fada0198670 0x7fada0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9ffff640 1 -- 172.21.15.86:0/329300883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada0199270 con 0x7fada0107320 2024-04-16T05:38:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada5407640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fada0198670 0x7fada0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9ffff640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0107320 0x7fada019eaf0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00c970 tx=0x7fad8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9dffb640 1 -- 172.21.15.86:0/329300883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c014070 con 0x7fada0107320 2024-04-16T05:38:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9dffb640 1 -- 172.21.15.86:0/329300883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad8c004440 con 0x7fada0107320 2024-04-16T05:38:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fada6e6a640 1 -- 172.21.15.86:0/329300883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada01a5350 con 0x7fada0107320 2024-04-16T05:38:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.841+0000 7fad9dffb640 1 -- 172.21.15.86:0/329300883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00f450 con 0x7fada0107320 2024-04-16T05:38:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.842+0000 7fada6e6a640 1 -- 172.21.15.86:0/329300883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada01a5750 con 0x7fada0107320 2024-04-16T05:38:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fad9dffb640 1 -- 172.21.15.86:0/329300883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad8c017070 con 0x7fada0107320 2024-04-16T05:38:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fada9a58740 1 Processor -- start 2024-04-16T05:38:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fada9a58740 1 -- start start 2024-04-16T05:38:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fad9dffb640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fad900a3d20 0x7fad900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fada9a58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e589277240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fada9a58740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e58927fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fada9a58740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589280360 0x55e589282750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fada9a58740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5891dc0e0 con 0x55e589280360 2024-04-16T05:38:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fada9a58740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e589179520 con 0x55e589277780 2024-04-16T05:38:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fada9a58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5891d80a0 con 0x55e589276e60 2024-04-16T05:38:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fada5407640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fad900a3d20 0x7fad900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.844+0000 7fad9dffb640 1 -- 172.21.15.86:0/329300883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fad8c0d0a80 con 0x7fada0107320 2024-04-16T05:38:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada4c06640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e589277240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada5407640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589280360 0x55e589282750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada4c06640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e589277240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:55556/0 (socket says 172.21.15.86:55556) 2024-04-16T05:38:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada4c06640 1 -- 172.21.15.86:0/4200146856 learned_addr learned my addr 172.21.15.86:0/4200146856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fad9ffff640 1 --2- 172.21.15.86:0/4200146856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e58927fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada4c06640 1 -- 172.21.15.86:0/4200146856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 msgr2=0x55e58927fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada4c06640 1 --2- 172.21.15.86:0/4200146856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e58927fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada5407640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fad900a3d20 0x7fad900a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad98014580 tx=0x7fad98014980 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada4c06640 1 -- 172.21.15.86:0/4200146856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589280360 msgr2=0x55e589282750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada4c06640 1 --2- 172.21.15.86:0/4200146856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589280360 0x55e589282750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada4c06640 1 -- 172.21.15.86:0/4200146856 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5891a18d0 con 0x55e589276e60 2024-04-16T05:38:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fada4c06640 1 --2- 172.21.15.86:0/4200146856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e589277240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fad94036040 tx=0x7fad9402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42b634dd05b9a9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fad8affd640 1 -- 172.21.15.86:0/4200146856 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9402ca80 con 0x55e589276e60 2024-04-16T05:38:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fad8affd640 1 -- 172.21.15.86:0/4200146856 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad9402cc20 con 0x55e589276e60 2024-04-16T05:38:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.845+0000 7fad8affd640 1 -- 172.21.15.86:0/4200146856 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94004920 con 0x55e589276e60 2024-04-16T05:38:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 -- 172.21.15.86:0/4200146856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 msgr2=0x55e589277240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 --2- 172.21.15.86:0/4200146856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e589277240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fad94036040 tx=0x7fad9402c650 comp rx=0 tx=0).stop 2024-04-16T05:38:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 -- 172.21.15.86:0/4200146856 shutdown_connections 2024-04-16T05:38:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 --2- 172.21.15.86:0/4200146856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589280360 0x55e589282750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 --2- 172.21.15.86:0/4200146856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e58927fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 --2- 172.21.15.86:0/4200146856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e589277240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 -- 172.21.15.86:0/4200146856 >> 172.21.15.86:0/4200146856 conn(0x55e589271590 msgr2=0x55e589271970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 -- 172.21.15.86:0/4200146856 shutdown_connections 2024-04-16T05:38:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 -- 172.21.15.86:0/4200146856 wait complete. 2024-04-16T05:38:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 -- start start 2024-04-16T05:38:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.846+0000 7fada9a58740 1 Processor -- start 2024-04-16T05:38:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada9a58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e58927d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada9a58740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589277780 0x55e58927d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada9a58740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589280360 0x55e58927ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada9a58740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e589283000 con 0x55e589277780 2024-04-16T05:38:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada9a58740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e5891954d0 con 0x55e589280360 2024-04-16T05:38:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada9a58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e589283180 con 0x55e589276e60 2024-04-16T05:38:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada9a58740 10 client.0 _ll_register_callbacks cb 0x55e58917a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589277780 0x55e58927d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589277780 0x55e58927d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51184/0 (socket says 172.21.15.86:51184) 2024-04-16T05:38:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad9ffff640 1 -- 172.21.15.86:0/440276172 learned_addr learned my addr 172.21.15.86:0/440276172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada4c06640 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e58927d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada5407640 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589280360 0x55e58927ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad9ffff640 1 -- 172.21.15.86:0/440276172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 msgr2=0x55e58927d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad9ffff640 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e58927d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad9ffff640 1 -- 172.21.15.86:0/440276172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589280360 msgr2=0x55e58927ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad9ffff640 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589280360 0x55e58927ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad9ffff640 1 -- 172.21.15.86:0/440276172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e588eee3f0 con 0x55e589277780 2024-04-16T05:38:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fada5407640 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589280360 0x55e58927ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad9ffff640 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589277780 0x55e58927d8b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fad8c09c3b0 tx=0x7fad8c09cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64d2b3def72a63b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.847+0000 7fad7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fad7ffff640 1 -- 172.21.15.86:0/440276172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0a5320 con 0x55e589277780 2024-04-16T05:38:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fad7ffff640 1 -- 172.21.15.86:0/440276172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fad8c0ce0e0 con 0x55e589277780 2024-04-16T05:38:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada9a58740 1 -- 172.21.15.86:0/440276172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9402cc20 con 0x55e589277780 2024-04-16T05:38:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fad7ffff640 1 -- 172.21.15.86:0/440276172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad8c0ce2e0 con 0x55e589277780 2024-04-16T05:38:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada9a58740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada9a58740 1 -- 172.21.15.86:0/440276172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e588fb87d0 con 0x55e589277780 2024-04-16T05:38:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fad7ffff640 1 -- 172.21.15.86:0/440276172 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad9402cc20 con 0x55e589277780 2024-04-16T05:38:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fad7ffff640 1 -- 172.21.15.86:0/440276172 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fad8c0a4f20 con 0x55e589277780 2024-04-16T05:38:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada9a58740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada9a58740 10 client.16578 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:38:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada9a58740 20 client.16578 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada9a58740 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e589290200 0x55e589292620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada9a58740 4 client.16578 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada9a58740 1 -- 172.21.15.86:0/440276172 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5891e9b00 con 0x55e589290200 2024-04-16T05:38:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.848+0000 7fada4c06640 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e589290200 0x55e589292620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.849+0000 7fad7ffff640 10 client.16578 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:38:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.849+0000 7fada4c06640 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e589290200 0x55e589292620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b734a254644af30a server_cookie=17e8e43f161e9163 in_seq=0 out_seq=0 2024-04-16T05:38:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fad7ffff640 1 -- 172.21.15.86:0/440276172 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5891e9b00 con 0x55e589290200 2024-04-16T05:38:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fad7ffff640 10 client.16578 handle_command_reply: tid=0 2024-04-16T05:38:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 client.16578 shutdown 2024-04-16T05:38:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 -- 172.21.15.86:0/440276172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589277780 msgr2=0x55e58927d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589277780 0x55e58927d8b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fad8c09c3b0 tx=0x7fad8c09cef0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 -- 172.21.15.86:0/440276172 shutdown_connections 2024-04-16T05:38:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589280360 0x55e58927ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e589290200 0x55e589292620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589277780 0x55e58927d8b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fad7ffff640 0 client.16578 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:38:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 --2- 172.21.15.86:0/440276172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589276e60 0x55e58927d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 -- 172.21.15.86:0/440276172 >> 172.21.15.86:0/440276172 conn(0x55e589271590 msgr2=0x55e5892732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 -- 172.21.15.86:0/440276172 shutdown_connections 2024-04-16T05:38:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 1 -- 172.21.15.86:0/440276172 wait complete. 2024-04-16T05:38:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.850+0000 7fada9a58740 20 client.16578 trim_cache size 0 max 16384 2024-04-16T05:38:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.863+0000 7fada9a58740 1 Processor -- start 2024-04-16T05:38:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.863+0000 7fada9a58740 1 -- start start 2024-04-16T05:38:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.863+0000 7fada9a58740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e58928bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.863+0000 7fada9a58740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e589288890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.863+0000 7fada9a58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 0x55e58927da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.863+0000 7fada9a58740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fada01af4a0 con 0x55e589276e60 2024-04-16T05:38:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.863+0000 7fada9a58740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fada01b0fe0 con 0x55e589277780 2024-04-16T05:38:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.863+0000 7fada9a58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fada01afb70 con 0x55e589280360 2024-04-16T05:38:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada4c06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e58928bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada4c06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e58928bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51188/0 (socket says 172.21.15.86:51188) 2024-04-16T05:38:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada4c06640 1 -- 172.21.15.86:0/2192214135 learned_addr learned my addr 172.21.15.86:0/2192214135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada5407640 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 0x55e58927da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fad9ffff640 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e589288890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada4c06640 1 -- 172.21.15.86:0/2192214135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 msgr2=0x55e58927da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada4c06640 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 0x55e58927da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada4c06640 1 -- 172.21.15.86:0/2192214135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 msgr2=0x55e589288890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada4c06640 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e589288890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada4c06640 1 -- 172.21.15.86:0/2192214135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e58926ce60 con 0x55e589276e60 2024-04-16T05:38:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fad9ffff640 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e589288890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fada4c06640 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e58928bfa0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fad94005640 tx=0x7fad9402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5077dfd89a33d738 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fad8a7fc640 1 -- 172.21.15.86:0/2192214135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9402c650 con 0x55e589276e60 2024-04-16T05:38:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fad8a7fc640 1 -- 172.21.15.86:0/2192214135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad9404d330 con 0x55e589276e60 2024-04-16T05:38:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.864+0000 7fad8a7fc640 1 -- 172.21.15.86:0/2192214135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94003850 con 0x55e589276e60 2024-04-16T05:38:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 -- 172.21.15.86:0/2192214135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 msgr2=0x55e58928bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e58928bfa0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fad94005640 tx=0x7fad9402bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 -- 172.21.15.86:0/2192214135 shutdown_connections 2024-04-16T05:38:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 0x55e58927da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e589288890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 --2- 172.21.15.86:0/2192214135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e58928bfa0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 -- 172.21.15.86:0/2192214135 >> 172.21.15.86:0/2192214135 conn(0x55e589271590 msgr2=0x55e58927d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 -- 172.21.15.86:0/2192214135 shutdown_connections 2024-04-16T05:38:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 -- 172.21.15.86:0/2192214135 wait complete. 2024-04-16T05:38:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 -- start start 2024-04-16T05:38:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 Processor -- start 2024-04-16T05:38:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e5892797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e589279d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 0x55e58927a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fada01b0910 con 0x55e589276e60 2024-04-16T05:38:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fada01b0240 con 0x55e589277780 2024-04-16T05:38:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fada01ad460 con 0x55e589280360 2024-04-16T05:38:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada9a58740 10 client.0 _ll_register_callbacks cb 0x7fad8c09cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.865+0000 7fada4c06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e5892797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada4c06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e5892797d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51196/0 (socket says 172.21.15.86:51196) 2024-04-16T05:38:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada4c06640 1 -- 172.21.15.86:0/2761600461 learned_addr learned my addr 172.21.15.86:0/2761600461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fad9ffff640 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e589279d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada5407640 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 0x55e58927a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada4c06640 1 -- 172.21.15.86:0/2761600461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 msgr2=0x55e58927a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada4c06640 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 0x55e58927a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada4c06640 1 -- 172.21.15.86:0/2761600461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 msgr2=0x55e589279d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada4c06640 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e589279d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada4c06640 1 -- 172.21.15.86:0/2761600461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e58927ac80 con 0x55e589276e60 2024-04-16T05:38:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada5407640 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 0x55e58927a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada4c06640 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e5892797d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fad9402e040 tx=0x7fad94005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83e4d1a637a4776 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fad7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fad7ffff640 1 -- 172.21.15.86:0/2761600461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9402cd90 con 0x55e589276e60 2024-04-16T05:38:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fad7ffff640 1 -- 172.21.15.86:0/2761600461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fad94004230 con 0x55e589276e60 2024-04-16T05:38:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada9a58740 1 -- 172.21.15.86:0/2761600461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9404d330 con 0x55e589276e60 2024-04-16T05:38:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fad7ffff640 1 -- 172.21.15.86:0/2761600461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad94065050 con 0x55e589276e60 2024-04-16T05:38:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada9a58740 10 client.16584 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fada9a58740 1 -- 172.21.15.86:0/2761600461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5892fafd0 con 0x55e589276e60 2024-04-16T05:38:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.866+0000 7fad7ffff640 1 -- 172.21.15.86:0/2761600461 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad9404d330 con 0x55e589276e60 2024-04-16T05:38:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fad7ffff640 1 -- 172.21.15.86:0/2761600461 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fad94056070 con 0x55e589276e60 2024-04-16T05:38:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fada9a58740 10 client.16584 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fada9a58740 10 client.16584 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:38:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fada9a58740 20 client.16584 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fada9a58740 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e5892fb2c0 0x55e5892fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fada9a58740 4 client.16584 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fada9a58740 1 -- 172.21.15.86:0/2761600461 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5891e3970 con 0x55e5892fb2c0 2024-04-16T05:38:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fad9ffff640 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e5892fb2c0 0x55e5892fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fad9ffff640 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e5892fb2c0 0x55e5892fd6b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=299f35b60082569a server_cookie=49399e40a6a411da in_seq=0 out_seq=0 2024-04-16T05:38:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fad7ffff640 10 client.16584 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:38:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fad7ffff640 1 -- 172.21.15.86:0/2761600461 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5891e3970 con 0x55e5892fb2c0 2024-04-16T05:38:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fad7ffff640 10 client.16584 handle_command_reply: tid=0 2024-04-16T05:38:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.867+0000 7fada9a58740 1 client.16584 shutdown 2024-04-16T05:38:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 -- 172.21.15.86:0/2761600461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 msgr2=0x55e5892797d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e5892797d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fad9402e040 tx=0x7fad94005fd0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 -- 172.21.15.86:0/2761600461 shutdown_connections 2024-04-16T05:38:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e589280360 0x55e58927a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e589277780 0x55e589279d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e5892fb2c0 0x55e5892fd6b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 --2- 172.21.15.86:0/2761600461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e589276e60 0x55e5892797d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fad7ffff640 0 client.16584 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:38:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 -- 172.21.15.86:0/2761600461 >> 172.21.15.86:0/2761600461 conn(0x55e589271590 msgr2=0x55e58927dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 -- 172.21.15.86:0/2761600461 shutdown_connections 2024-04-16T05:38:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 1 -- 172.21.15.86:0/2761600461 wait complete. 2024-04-16T05:38:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada9a58740 20 client.16584 trim_cache size 0 max 16384 2024-04-16T05:38:06.878 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 -- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fad900a3d20 msgr2=0x7fad900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fad900a3d20 0x7fad900a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad98014580 tx=0x7fad98014980 comp rx=0 tx=0).stop 2024-04-16T05:38:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 -- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0107320 msgr2=0x7fada019eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0107320 0x7fada019eaf0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00c970 tx=0x7fad8c00ce30 comp rx=0 tx=0).stop 2024-04-16T05:38:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 -- 172.21.15.86:0/329300883 shutdown_connections 2024-04-16T05:38:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fad900a3d20 0x7fad900a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fada0198670 0x7fada0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada0107320 0x7fada019eaf0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 --2- 172.21.15.86:0/329300883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0102250 0x7fada019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 -- 172.21.15.86:0/329300883 >> 172.21.15.86:0/329300883 conn(0x7fada0079490 msgr2=0x7fada00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.868+0000 7fada6e6a640 1 -- 172.21.15.86:0/329300883 shutdown_connections 2024-04-16T05:38:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.869+0000 7fada6e6a640 1 -- 172.21.15.86:0/329300883 wait complete. 2024-04-16T05:38:06.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:38:06.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:06 smithi194 ceph-mon[35865]: pgmap v87: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:38:06.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:06 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/551675067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:06 smithi196 ceph-mon[36373]: pgmap v87: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:38:06.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:06 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/551675067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 -- 172.21.15.86:0/621234940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0598101ae0 msgr2=0x7f0598105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 --2- 172.21.15.86:0/621234940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0598101ae0 0x7f0598105b20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0594009900 tx=0x7f059402bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 -- 172.21.15.86:0/621234940 shutdown_connections 2024-04-16T05:38:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 --2- 172.21.15.86:0/621234940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0598106200 0x7f059810ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 --2- 172.21.15.86:0/621234940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0598101ae0 0x7f0598105b20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 --2- 172.21.15.86:0/621234940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0598101130 0x7f0598101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 -- 172.21.15.86:0/621234940 >> 172.21.15.86:0/621234940 conn(0x7f05980780b0 msgr2=0x7f05980784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 -- 172.21.15.86:0/621234940 shutdown_connections 2024-04-16T05:38:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 -- 172.21.15.86:0/621234940 wait complete. 2024-04-16T05:38:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 Processor -- start 2024-04-16T05:38:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.990+0000 7f059f98f640 1 -- start start 2024-04-16T05:38:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059f98f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0598101130 0x7f059819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059f98f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0598106200 0x7f059819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059f98f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05981986d0 0x7f0598198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059f98f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0598112410 con 0x7f05981986d0 2024-04-16T05:38:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059f98f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0598112290 con 0x7f0598106200 2024-04-16T05:38:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059f98f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0598112590 con 0x7f0598101130 2024-04-16T05:38:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059df2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05981986d0 0x7f0598198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059df2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05981986d0 0x7f0598198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51214/0 (socket says 172.21.15.86:51214) 2024-04-16T05:38:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059df2c640 1 -- 172.21.15.86:0/1472859381 learned_addr learned my addr 172.21.15.86:0/1472859381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059d72b640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0598101130 0x7f059819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059cf2a640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0598106200 0x7f059819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059df2c640 1 -- 172.21.15.86:0/1472859381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0598101130 msgr2=0x7f059819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059df2c640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0598101130 0x7f059819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059df2c640 1 -- 172.21.15.86:0/1472859381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0598106200 msgr2=0x7f059819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059df2c640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0598106200 0x7f059819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059df2c640 1 -- 172.21.15.86:0/1472859381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981992d0 con 0x7f05981986d0 2024-04-16T05:38:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059cf2a640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0598106200 0x7f059819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059df2c640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05981986d0 0x7f0598198b50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f059000b970 tx=0x7f059000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f058e7fc640 1 -- 172.21.15.86:0/1472859381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590019070 con 0x7f05981986d0 2024-04-16T05:38:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f058e7fc640 1 -- 172.21.15.86:0/1472859381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059000cb40 con 0x7f05981986d0 2024-04-16T05:38:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059f98f640 1 -- 172.21.15.86:0/1472859381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05981a53b0 con 0x7f05981986d0 2024-04-16T05:38:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f058e7fc640 1 -- 172.21.15.86:0/1472859381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590007650 con 0x7f05981986d0 2024-04-16T05:38:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.991+0000 7f059f98f640 1 -- 172.21.15.86:0/1472859381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a5760 con 0x7f05981986d0 2024-04-16T05:38:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.993+0000 7f058e7fc640 1 -- 172.21.15.86:0/1472859381 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f059001c070 con 0x7f05981986d0 2024-04-16T05:38:07.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f05a257a740 1 Processor -- start 2024-04-16T05:38:07.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f05a257a740 1 -- start start 2024-04-16T05:38:07.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f05a257a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9abf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f05a257a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 0x5565e9ac7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f05a257a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 0x5565e9aca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f05a257a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5565e9a240e0 con 0x5565e9abee60 2024-04-16T05:38:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f05a257a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5565e99c1520 con 0x5565e9ac8360 2024-04-16T05:38:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f05a257a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565e9a200a0 con 0x5565e9abf780 2024-04-16T05:38:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f058e7fc640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f05840a3d20 0x7f05840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f059cf2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9abf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f059cf2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9abf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51216/0 (socket says 172.21.15.86:51216) 2024-04-16T05:38:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f059cf2a640 1 -- 172.21.15.86:0/2233566569 learned_addr learned my addr 172.21.15.86:0/2233566569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f059df2c640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f05840a3d20 0x7f05840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.994+0000 7f058e7fc640 1 -- 172.21.15.86:0/1472859381 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f05900cffe0 con 0x7f05981986d0 2024-04-16T05:38:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f059cf2a640 1 --2- 172.21.15.86:0/2233566569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 0x5565e9aca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f059d72b640 1 --2- 172.21.15.86:0/2233566569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 0x5565e9ac7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f059cf2a640 1 -- 172.21.15.86:0/2233566569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 msgr2=0x5565e9ac7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f059cf2a640 1 --2- 172.21.15.86:0/2233566569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 0x5565e9ac7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f059cf2a640 1 -- 172.21.15.86:0/2233566569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 msgr2=0x5565e9aca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f059cf2a640 1 --2- 172.21.15.86:0/2233566569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 0x5565e9aca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f059cf2a640 1 -- 172.21.15.86:0/2233566569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e99e98d0 con 0x5565e9abee60 2024-04-16T05:38:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f059df2c640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f05840a3d20 0x7f05840a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0590016000 tx=0x7f0590004b50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f059cf2a640 1 --2- 172.21.15.86:0/2233566569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9abf240 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0594005ab0 tx=0x7f0594009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49e07c4f3abca96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f05837fe640 1 -- 172.21.15.86:0/2233566569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594004020 con 0x5565e9abee60 2024-04-16T05:38:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f05837fe640 1 -- 172.21.15.86:0/2233566569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0594005520 con 0x5565e9abee60 2024-04-16T05:38:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.995+0000 7f05837fe640 1 -- 172.21.15.86:0/2233566569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594004c10 con 0x5565e9abee60 2024-04-16T05:38:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 -- 172.21.15.86:0/2233566569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 msgr2=0x5565e9abf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 --2- 172.21.15.86:0/2233566569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9abf240 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0594005ab0 tx=0x7f0594009560 comp rx=0 tx=0).stop 2024-04-16T05:38:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 -- 172.21.15.86:0/2233566569 shutdown_connections 2024-04-16T05:38:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 --2- 172.21.15.86:0/2233566569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 0x5565e9aca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 --2- 172.21.15.86:0/2233566569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 0x5565e9ac7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 --2- 172.21.15.86:0/2233566569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9abf240 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 -- 172.21.15.86:0/2233566569 >> 172.21.15.86:0/2233566569 conn(0x5565e9ab9590 msgr2=0x5565e9ab9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 -- 172.21.15.86:0/2233566569 shutdown_connections 2024-04-16T05:38:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 -- 172.21.15.86:0/2233566569 wait complete. 2024-04-16T05:38:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 -- start start 2024-04-16T05:38:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.996+0000 7f05a257a740 1 Processor -- start 2024-04-16T05:38:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 0x5565e9ac5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 0x5565e9ac5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5565e9acafa0 con 0x5565e9abee60 2024-04-16T05:38:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5565e99dd4d0 con 0x5565e9ac8360 2024-04-16T05:38:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565e9acb120 con 0x5565e9abf780 2024-04-16T05:38:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 10 client.0 _ll_register_callbacks cb 0x5565e99c22c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51220/0 (socket says 172.21.15.86:51220) 2024-04-16T05:38:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 -- 172.21.15.86:0/1458614032 learned_addr learned my addr 172.21.15.86:0/1458614032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059d72b640 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 0x5565e9ac5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 0x5565e9ac5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 -- 172.21.15.86:0/1458614032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 msgr2=0x5565e9ac5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 0x5565e9ac5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 -- 172.21.15.86:0/1458614032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 msgr2=0x5565e9ac5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 0x5565e9ac5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 -- 172.21.15.86:0/1458614032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e97363f0 con 0x5565e9abee60 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f059cf2a640 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac5340 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0594036040 tx=0x7f0594004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e6571029241ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f0580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f0580ff9640 1 -- 172.21.15.86:0/1458614032 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05940035f0 con 0x5565e9abee60 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f0580ff9640 1 -- 172.21.15.86:0/1458614032 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0594003790 con 0x5565e9abee60 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 1 -- 172.21.15.86:0/1458614032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0594005520 con 0x5565e9abee60 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f0580ff9640 1 -- 172.21.15.86:0/1458614032 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0594003990 con 0x5565e9abee60 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 10 client.16596 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.997+0000 7f05a257a740 1 -- 172.21.15.86:0/1458614032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565e98007d0 con 0x5565e9abee60 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.998+0000 7f0580ff9640 1 -- 172.21.15.86:0/1458614032 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0594005520 con 0x5565e9abee60 2024-04-16T05:38:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.998+0000 7f0580ff9640 1 -- 172.21.15.86:0/1458614032 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0594005100 con 0x5565e9abee60 2024-04-16T05:38:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.998+0000 7f05a257a740 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.998+0000 7f05a257a740 10 client.16596 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:38:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.998+0000 7f05a257a740 20 client.16596 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.998+0000 7f05a257a740 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5565e9ad8070 0x5565e9ada460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.998+0000 7f05a257a740 4 client.16596 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.998+0000 7f05a257a740 1 -- 172.21.15.86:0/1458614032 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e9a31b00 con 0x5565e9ad8070 2024-04-16T05:38:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.998+0000 7f059d72b640 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5565e9ad8070 0x5565e9ada460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f059d72b640 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5565e9ad8070 0x5565e9ada460 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c21297b99d2dd8a server_cookie=f9fa2ecc4847ff2a in_seq=0 out_seq=0 2024-04-16T05:38:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f0580ff9640 10 client.16596 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:38:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f0580ff9640 1 -- 172.21.15.86:0/1458614032 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565e9a31b00 con 0x5565e9ad8070 2024-04-16T05:38:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f0580ff9640 10 client.16596 handle_command_reply: tid=0 2024-04-16T05:38:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f05a257a740 1 client.16596 shutdown 2024-04-16T05:38:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f05a257a740 1 -- 172.21.15.86:0/1458614032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 msgr2=0x5565e9ac5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f05a257a740 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac5340 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0594036040 tx=0x7f0594004960 comp rx=0 tx=0).stop 2024-04-16T05:38:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f05a257a740 1 -- 172.21.15.86:0/1458614032 shutdown_connections 2024-04-16T05:38:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f05a257a740 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5565e9ad8070 0x5565e9ada460 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f05a257a740 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9ac8360 0x5565e9ac5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f05a257a740 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9abf780 0x5565e9ac5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f05a257a740 1 --2- 172.21.15.86:0/1458614032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac5340 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f0580ff9640 0 client.16596 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:38:07.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:06.999+0000 7f05a257a740 1 -- 172.21.15.86:0/1458614032 >> 172.21.15.86:0/1458614032 conn(0x5565e9ab9590 msgr2=0x5565e9abb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.000+0000 7f05a257a740 1 -- 172.21.15.86:0/1458614032 shutdown_connections 2024-04-16T05:38:07.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.000+0000 7f05a257a740 1 -- 172.21.15.86:0/1458614032 wait complete. 2024-04-16T05:38:07.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.000+0000 7f05a257a740 20 client.16596 trim_cache size 0 max 16384 2024-04-16T05:38:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f05a257a740 1 Processor -- start 2024-04-16T05:38:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f05a257a740 1 -- start start 2024-04-16T05:38:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f05a257a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f05a257a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9abf780 0x5565e9add480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f05a257a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9ac8360 0x5565e9add9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f05a257a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05981af320 con 0x5565e9abee60 2024-04-16T05:38:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f05a257a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f05981b0e60 con 0x5565e9abf780 2024-04-16T05:38:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f05a257a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05981af9f0 con 0x5565e9ac8360 2024-04-16T05:38:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f059cf2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f059cf2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51228/0 (socket says 172.21.15.86:51228) 2024-04-16T05:38:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f059cf2a640 1 -- 172.21.15.86:0/2734770344 learned_addr learned my addr 172.21.15.86:0/2734770344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f059cf2a640 1 -- 172.21.15.86:0/2734770344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9ac8360 msgr2=0x5565e9add9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f059cf2a640 1 --2- 172.21.15.86:0/2734770344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9ac8360 0x5565e9add9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f059cf2a640 1 -- 172.21.15.86:0/2734770344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9abf780 msgr2=0x5565e9add480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f059cf2a640 1 --2- 172.21.15.86:0/2734770344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9abf780 0x5565e9add480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f059cf2a640 1 -- 172.21.15.86:0/2734770344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e9ae02a0 con 0x5565e9abee60 2024-04-16T05:38:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.013+0000 7f059cf2a640 1 --2- 172.21.15.86:0/2734770344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac41f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f059402bc70 tx=0x7f0594040830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f827a9020712a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f0582ffd640 1 -- 172.21.15.86:0/2734770344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594009520 con 0x5565e9abee60 2024-04-16T05:38:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f0582ffd640 1 -- 172.21.15.86:0/2734770344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059402c660 con 0x5565e9abee60 2024-04-16T05:38:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f0582ffd640 1 -- 172.21.15.86:0/2734770344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059402cd50 con 0x5565e9abee60 2024-04-16T05:38:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 -- 172.21.15.86:0/2734770344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 msgr2=0x5565e9ac41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 --2- 172.21.15.86:0/2734770344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac41f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f059402bc70 tx=0x7f0594040830 comp rx=0 tx=0).stop 2024-04-16T05:38:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 -- 172.21.15.86:0/2734770344 shutdown_connections 2024-04-16T05:38:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 --2- 172.21.15.86:0/2734770344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9ac8360 0x5565e9add9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 --2- 172.21.15.86:0/2734770344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9abf780 0x5565e9add480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 --2- 172.21.15.86:0/2734770344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9ac41f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 -- 172.21.15.86:0/2734770344 >> 172.21.15.86:0/2734770344 conn(0x5565e9ab9590 msgr2=0x5565e9ac3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 -- 172.21.15.86:0/2734770344 shutdown_connections 2024-04-16T05:38:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 -- 172.21.15.86:0/2734770344 wait complete. 2024-04-16T05:38:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.014+0000 7f05a257a740 1 -- start start 2024-04-16T05:38:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f05a257a740 1 Processor -- start 2024-04-16T05:38:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f05a257a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9add6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f05a257a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9abf780 0x5565e9adaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f05a257a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9ac8360 0x5565e9adaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f05a257a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05981b0790 con 0x5565e9abee60 2024-04-16T05:38:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f05a257a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f05981b00c0 con 0x5565e9abf780 2024-04-16T05:38:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f05a257a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05981ad2e0 con 0x5565e9ac8360 2024-04-16T05:38:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f05a257a740 10 client.0 _ll_register_callbacks cb 0x7f059402ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f059cf2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9add6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f059cf2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9add6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51234/0 (socket says 172.21.15.86:51234) 2024-04-16T05:38:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f059cf2a640 1 -- 172.21.15.86:0/706666747 learned_addr learned my addr 172.21.15.86:0/706666747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f059d72b640 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9abf780 0x5565e9adaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f059cf2a640 1 -- 172.21.15.86:0/706666747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9ac8360 msgr2=0x5565e9adaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f059cf2a640 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9ac8360 0x5565e9adaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f059cf2a640 1 -- 172.21.15.86:0/706666747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9abf780 msgr2=0x5565e9adaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f059cf2a640 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9abf780 0x5565e9adaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.015+0000 7f059cf2a640 1 -- 172.21.15.86:0/706666747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e9adb610 con 0x5565e9abee60 2024-04-16T05:38:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f059cf2a640 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9add6a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0594009900 tx=0x7f0594005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae4fb6ff2dfb7b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f0580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f0580ff9640 1 -- 172.21.15.86:0/706666747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05940035f0 con 0x5565e9abee60 2024-04-16T05:38:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f0580ff9640 1 -- 172.21.15.86:0/706666747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f05940404a0 con 0x5565e9abee60 2024-04-16T05:38:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f05a257a740 1 -- 172.21.15.86:0/706666747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059402c660 con 0x5565e9abee60 2024-04-16T05:38:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f0580ff9640 1 -- 172.21.15.86:0/706666747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f059406d050 con 0x5565e9abee60 2024-04-16T05:38:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f05a257a740 10 client.16602 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f05a257a740 1 -- 172.21.15.86:0/706666747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565e9b3e100 con 0x5565e9abee60 2024-04-16T05:38:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f0580ff9640 1 -- 172.21.15.86:0/706666747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059402c660 con 0x5565e9abee60 2024-04-16T05:38:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f0580ff9640 1 -- 172.21.15.86:0/706666747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0594034440 con 0x5565e9abee60 2024-04-16T05:38:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f05a257a740 10 client.16602 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f05a257a740 10 client.16602 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:38:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f05a257a740 20 client.16602 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f05a257a740 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5565e9b3e700 0x5565e9b40b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f05a257a740 4 client.16602 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.016+0000 7f05a257a740 1 -- 172.21.15.86:0/706666747 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565e9a2b970 con 0x5565e9b3e700 2024-04-16T05:38:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f059d72b640 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5565e9b3e700 0x5565e9b40b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f059d72b640 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5565e9b3e700 0x5565e9b40b20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e016f9cc087b50f7 server_cookie=f187205876a06cf7 in_seq=0 out_seq=0 2024-04-16T05:38:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f0580ff9640 10 client.16602 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:38:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f0580ff9640 1 -- 172.21.15.86:0/706666747 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565e9a2b970 con 0x5565e9b3e700 2024-04-16T05:38:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f0580ff9640 10 client.16602 handle_command_reply: tid=0 2024-04-16T05:38:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 client.16602 shutdown 2024-04-16T05:38:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 -- 172.21.15.86:0/706666747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 msgr2=0x5565e9add6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9add6a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0594009900 tx=0x7f0594005700 comp rx=0 tx=0).stop 2024-04-16T05:38:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 -- 172.21.15.86:0/706666747 shutdown_connections 2024-04-16T05:38:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5565e9b3e700 0x5565e9b40b20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565e9ac8360 0x5565e9adaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f0580ff9640 0 client.16602 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:38:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565e9abf780 0x5565e9adaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 --2- 172.21.15.86:0/706666747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e9abee60 0x5565e9add6a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 -- 172.21.15.86:0/706666747 >> 172.21.15.86:0/706666747 conn(0x5565e9ab9590 msgr2=0x5565e9ab8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 -- 172.21.15.86:0/706666747 shutdown_connections 2024-04-16T05:38:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 1 -- 172.21.15.86:0/706666747 wait complete. 2024-04-16T05:38:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.017+0000 7f05a257a740 20 client.16602 trim_cache size 0 max 16384 2024-04-16T05:38:07.026 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 -- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f05840a3d20 msgr2=0x7f05840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f05840a3d20 0x7f05840a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0590016000 tx=0x7f0590004b50 comp rx=0 tx=0).stop 2024-04-16T05:38:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 -- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05981986d0 msgr2=0x7f0598198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05981986d0 0x7f0598198b50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f059000b970 tx=0x7f059000be30 comp rx=0 tx=0).stop 2024-04-16T05:38:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 -- 172.21.15.86:0/1472859381 shutdown_connections 2024-04-16T05:38:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05981986d0 0x7f0598198b50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f05840a3d20 0x7f05840a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0598106200 0x7f059819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 --2- 172.21.15.86:0/1472859381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0598101130 0x7f059819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 -- 172.21.15.86:0/1472859381 >> 172.21.15.86:0/1472859381 conn(0x7f05980780b0 msgr2=0x7f05980feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 -- 172.21.15.86:0/1472859381 shutdown_connections 2024-04-16T05:38:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.018+0000 7f059f98f640 1 -- 172.21.15.86:0/1472859381 wait complete. 2024-04-16T05:38:07.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:38:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.136+0000 7f8af7888640 1 -- 172.21.15.86:0/3438479619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101bf0 msgr2=0x7f8af010dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.136+0000 7f8af7888640 1 --2- 172.21.15.86:0/3438479619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101bf0 0x7f8af010dcf0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ca0 tx=0x7f8ae802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.136+0000 7f8af7888640 1 -- 172.21.15.86:0/3438479619 shutdown_connections 2024-04-16T05:38:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.136+0000 7f8af7888640 1 --2- 172.21.15.86:0/3438479619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101bf0 0x7f8af010dcf0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.136+0000 7f8af7888640 1 --2- 172.21.15.86:0/3438479619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af0101250 0x7f8af01016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.136+0000 7f8af7888640 1 --2- 172.21.15.86:0/3438479619 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af0107040 0x7f8af0107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.136+0000 7f8af7888640 1 -- 172.21.15.86:0/3438479619 >> 172.21.15.86:0/3438479619 conn(0x7f8af0076d00 msgr2=0x7f8af0077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.136+0000 7f8af7888640 1 -- 172.21.15.86:0/3438479619 shutdown_connections 2024-04-16T05:38:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.136+0000 7f8af7888640 1 -- 172.21.15.86:0/3438479619 wait complete. 2024-04-16T05:38:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af7888640 1 Processor -- start 2024-04-16T05:38:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af7888640 1 -- start start 2024-04-16T05:38:07.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af7888640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101250 0x7f8af019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af7888640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af0101bf0 0x7f8af019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af7888640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af0107040 0x7f8af0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af7888640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01112f0 con 0x7f8af0101250 2024-04-16T05:38:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af7888640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0111170 con 0x7f8af0101bf0 2024-04-16T05:38:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af7888640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0111470 con 0x7f8af0107040 2024-04-16T05:38:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af5624640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101250 0x7f8af019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af5624640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101250 0x7f8af019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51252/0 (socket says 172.21.15.86:51252) 2024-04-16T05:38:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af5624640 1 -- 172.21.15.86:0/1963608488 learned_addr learned my addr 172.21.15.86:0/1963608488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af5624640 1 -- 172.21.15.86:0/1963608488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af0107040 msgr2=0x7f8af0198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af5624640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af0107040 0x7f8af0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af5624640 1 -- 172.21.15.86:0/1963608488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af0101bf0 msgr2=0x7f8af019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.137+0000 7f8af5624640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af0101bf0 0x7f8af019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.138+0000 7f8af5624640 1 -- 172.21.15.86:0/1963608488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af0198f20 con 0x7f8af0101250 2024-04-16T05:38:07.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.138+0000 7f8af5624640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101250 0x7f8af019e5a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c920 tx=0x7f8aec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.138+0000 7f8ae67fc640 1 -- 172.21.15.86:0/1963608488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec007c70 con 0x7f8af0101250 2024-04-16T05:38:07.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.138+0000 7f8ae67fc640 1 -- 172.21.15.86:0/1963608488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aec007e10 con 0x7f8af0101250 2024-04-16T05:38:07.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.138+0000 7f8af7888640 1 -- 172.21.15.86:0/1963608488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af0199210 con 0x7f8af0101250 2024-04-16T05:38:07.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.138+0000 7f8ae67fc640 1 -- 172.21.15.86:0/1963608488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec018890 con 0x7f8af0101250 2024-04-16T05:38:07.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.138+0000 7f8af7888640 1 -- 172.21.15.86:0/1963608488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af01a5520 con 0x7f8af0101250 2024-04-16T05:38:07.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.140+0000 7f8ae67fc640 1 -- 172.21.15.86:0/1963608488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8aec018a70 con 0x7f8af0101250 2024-04-16T05:38:07.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.140+0000 7f8afa47c740 1 Processor -- start 2024-04-16T05:38:07.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8afa47c740 1 -- start start 2024-04-16T05:38:07.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0be60 0x563a09f0c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0c780 0x563a09f14e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f17750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8afa47c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a09e710e0 con 0x563a09f15360 2024-04-16T05:38:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8afa47c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a09e0e520 con 0x563a09f0be60 2024-04-16T05:38:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8afa47c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563a09e6d0a0 con 0x563a09f0c780 2024-04-16T05:38:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af4e23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f17750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af4e23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f17750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51266/0 (socket says 172.21.15.86:51266) 2024-04-16T05:38:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af4e23640 1 -- 172.21.15.86:0/3208519067 learned_addr learned my addr 172.21.15.86:0/3208519067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af5e25640 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0be60 0x563a09f0c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8ae67fc640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8adc0a3d20 0x7f8adc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af5624640 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0c780 0x563a09f14e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af5e25640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8adc0a3d20 0x7f8adc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af4e23640 1 -- 172.21.15.86:0/3208519067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0c780 msgr2=0x563a09f14e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af4e23640 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0c780 0x563a09f14e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af4e23640 1 -- 172.21.15.86:0/3208519067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0be60 msgr2=0x563a09f0c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af4e23640 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0be60 0x563a09f0c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af4e23640 1 -- 172.21.15.86:0/3208519067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a09e368d0 con 0x563a09f15360 2024-04-16T05:38:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8ae67fc640 1 -- 172.21.15.86:0/1963608488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8aec0d0cd0 con 0x7f8af0101250 2024-04-16T05:38:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af5624640 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0c780 0x563a09f14e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.141+0000 7f8af4e23640 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f17750 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000bae0 tx=0x7f8ae000d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8944185d8628b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8af5e25640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8adc0a3d20 0x7f8adc0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009430 tx=0x7f8ae80095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8adb7fe640 1 -- 172.21.15.86:0/3208519067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0016070 con 0x563a09f15360 2024-04-16T05:38:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8adb7fe640 1 -- 172.21.15.86:0/3208519067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae0005500 con 0x563a09f15360 2024-04-16T05:38:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8adb7fe640 1 -- 172.21.15.86:0/3208519067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0010460 con 0x563a09f15360 2024-04-16T05:38:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8afa47c740 1 -- 172.21.15.86:0/3208519067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 msgr2=0x563a09f17750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f17750 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000bae0 tx=0x7f8ae000d2e0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8afa47c740 1 -- 172.21.15.86:0/3208519067 shutdown_connections 2024-04-16T05:38:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f17750 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0c780 0x563a09f14e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3208519067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0be60 0x563a09f0c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8afa47c740 1 -- 172.21.15.86:0/3208519067 >> 172.21.15.86:0/3208519067 conn(0x563a09f06590 msgr2=0x563a09f06970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8afa47c740 1 -- 172.21.15.86:0/3208519067 shutdown_connections 2024-04-16T05:38:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.142+0000 7f8afa47c740 1 -- 172.21.15.86:0/3208519067 wait complete. 2024-04-16T05:38:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8afa47c740 1 -- start start 2024-04-16T05:38:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8afa47c740 1 Processor -- start 2024-04-16T05:38:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f123a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 0x563a09f12e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8afa47c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a09f18060 con 0x563a09f0c780 2024-04-16T05:38:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8afa47c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a09e2a4d0 con 0x563a09f15360 2024-04-16T05:38:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8afa47c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563a09f181e0 con 0x563a09f0be60 2024-04-16T05:38:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8afa47c740 10 client.0 _ll_register_callbacks cb 0x563a09e0f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8af4e23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8af4e23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51276/0 (socket says 172.21.15.86:51276) 2024-04-16T05:38:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.143+0000 7f8af4e23640 1 -- 172.21.15.86:0/2257261643 learned_addr learned my addr 172.21.15.86:0/2257261643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8af5e25640 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 0x563a09f12e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8af5624640 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8af4e23640 1 -- 172.21.15.86:0/2257261643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 msgr2=0x563a09f123a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8af4e23640 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f123a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8af4e23640 1 -- 172.21.15.86:0/2257261643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 msgr2=0x563a09f12e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8af4e23640 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 0x563a09f12e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8af4e23640 1 -- 172.21.15.86:0/2257261643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a09b833f0 con 0x563a09f0c780 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8af5e25640 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 0x563a09f12e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8af4e23640 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f128e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0006010 tx=0x7f8ae0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a017aedde55695ee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8ad8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/2257261643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0016070 con 0x563a09f0c780 2024-04-16T05:38:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/2257261643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8ae000dab0 con 0x563a09f0c780 2024-04-16T05:38:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8afa47c740 1 -- 172.21.15.86:0/2257261643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae0005500 con 0x563a09f0c780 2024-04-16T05:38:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/2257261643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ae002d050 con 0x563a09f0c780 2024-04-16T05:38:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8afa47c740 10 client.16614 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8afa47c740 1 -- 172.21.15.86:0/2257261643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a09c4d7d0 con 0x563a09f0c780 2024-04-16T05:38:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/2257261643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae0005500 con 0x563a09f0c780 2024-04-16T05:38:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/2257261643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ae0022020 con 0x563a09f0c780 2024-04-16T05:38:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8afa47c740 10 client.16614 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8afa47c740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.144+0000 7f8afa47c740 20 client.16614 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.145+0000 7f8afa47c740 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563a09f252c0 0x563a09f276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.145+0000 7f8afa47c740 4 client.16614 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.145+0000 7f8afa47c740 1 -- 172.21.15.86:0/2257261643 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a09e7eb00 con 0x563a09f252c0 2024-04-16T05:38:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.145+0000 7f8af5624640 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563a09f252c0 0x563a09f276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.145+0000 7f8ad8ff9640 10 client.16614 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.145+0000 7f8af5624640 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563a09f252c0 0x563a09f276b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9422b38193e58b6d server_cookie=f6586ba274a7134c in_seq=0 out_seq=0 2024-04-16T05:38:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/2257261643 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a09e7eb00 con 0x563a09f252c0 2024-04-16T05:38:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8ad8ff9640 10 client.16614 handle_command_reply: tid=0 2024-04-16T05:38:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 client.16614 shutdown 2024-04-16T05:38:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 -- 172.21.15.86:0/2257261643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 msgr2=0x563a09f128e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f128e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0006010 tx=0x7f8ae0005780 comp rx=0 tx=0).stop 2024-04-16T05:38:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 -- 172.21.15.86:0/2257261643 shutdown_connections 2024-04-16T05:38:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563a09f252c0 0x563a09f276b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 0x563a09f12e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f128e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 --2- 172.21.15.86:0/2257261643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8ad8ff9640 0 client.16614 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 -- 172.21.15.86:0/2257261643 >> 172.21.15.86:0/2257261643 conn(0x563a09f06590 msgr2=0x563a09f082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 -- 172.21.15.86:0/2257261643 shutdown_connections 2024-04-16T05:38:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 1 -- 172.21.15.86:0/2257261643 wait complete. 2024-04-16T05:38:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.146+0000 7f8afa47c740 20 client.16614 trim_cache size 0 max 16384 2024-04-16T05:38:07.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.159+0000 7f8afa47c740 1 Processor -- start 2024-04-16T05:38:07.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.159+0000 7f8afa47c740 1 -- start start 2024-04-16T05:38:07.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f14ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f21060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 0x563a09f0cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8afa47c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01af350 con 0x563a09f0c780 2024-04-16T05:38:07.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8afa47c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01b0e90 con 0x563a09f15360 2024-04-16T05:38:07.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8afa47c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01afa20 con 0x563a09f0be60 2024-04-16T05:38:07.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af4e23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f21060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af4e23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f21060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51280/0 (socket says 172.21.15.86:51280) 2024-04-16T05:38:07.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af4e23640 1 -- 172.21.15.86:0/3181393621 learned_addr learned my addr 172.21.15.86:0/3181393621 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:07.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af4e23640 1 -- 172.21.15.86:0/3181393621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 msgr2=0x563a09f14ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af5624640 1 --2- 172.21.15.86:0/3181393621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f14ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af4e23640 1 --2- 172.21.15.86:0/3181393621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f14ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af4e23640 1 -- 172.21.15.86:0/3181393621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 msgr2=0x563a09f0cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af4e23640 1 --2- 172.21.15.86:0/3181393621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 0x563a09f0cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af4e23640 1 -- 172.21.15.86:0/3181393621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a09f0c5b0 con 0x563a09f0c780 2024-04-16T05:38:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8af4e23640 1 --2- 172.21.15.86:0/3181393621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f21060 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000d320 tx=0x7f8ae000d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c3682554cac575 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8adaffd640 1 -- 172.21.15.86:0/3181393621 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0016040 con 0x563a09f0c780 2024-04-16T05:38:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8adaffd640 1 -- 172.21.15.86:0/3181393621 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae0004860 con 0x563a09f0c780 2024-04-16T05:38:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.160+0000 7f8adaffd640 1 -- 172.21.15.86:0/3181393621 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000dd10 con 0x563a09f0c780 2024-04-16T05:38:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 -- 172.21.15.86:0/3181393621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 msgr2=0x563a09f21060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3181393621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f21060 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000d320 tx=0x7f8ae000d910 comp rx=0 tx=0).stop 2024-04-16T05:38:07.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 -- 172.21.15.86:0/3181393621 shutdown_connections 2024-04-16T05:38:07.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3181393621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f15360 0x563a09f0cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3181393621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f0c780 0x563a09f21060 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3181393621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f14ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 -- 172.21.15.86:0/3181393621 >> 172.21.15.86:0/3181393621 conn(0x563a09f06590 msgr2=0x563a09f015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 -- 172.21.15.86:0/3181393621 shutdown_connections 2024-04-16T05:38:07.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 -- 172.21.15.86:0/3181393621 wait complete. 2024-04-16T05:38:07.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 -- start start 2024-04-16T05:38:07.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.161+0000 7f8afa47c740 1 Processor -- start 2024-04-16T05:38:07.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f0fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0c780 0x563a09f0e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8afa47c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f0eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8afa47c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01b07c0 con 0x563a09f15360 2024-04-16T05:38:07.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8afa47c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01b00f0 con 0x563a09f0c780 2024-04-16T05:38:07.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8afa47c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01ad310 con 0x563a09f0be60 2024-04-16T05:38:07.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8afa47c740 10 client.0 _ll_register_callbacks cb 0x7f8af007a330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:07.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5e25640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f0eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5e25640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f0eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51284/0 (socket says 172.21.15.86:51284) 2024-04-16T05:38:07.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5e25640 1 -- 172.21.15.86:0/3532377896 learned_addr learned my addr 172.21.15.86:0/3532377896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:07.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af4e23640 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0c780 0x563a09f0e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5624640 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f0fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5e25640 1 -- 172.21.15.86:0/3532377896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 msgr2=0x563a09f0fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5e25640 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f0fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5e25640 1 -- 172.21.15.86:0/3532377896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0c780 msgr2=0x563a09f0e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:07.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5e25640 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0c780 0x563a09f0e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5e25640 1 -- 172.21.15.86:0/3532377896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a09f0f1e0 con 0x563a09f15360 2024-04-16T05:38:07.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8af5e25640 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f0eca0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8036000 tx=0x7f8ae80324b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5f64c5e225dba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:07.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.162+0000 7f8ad8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:07.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/3532377896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8051060 con 0x563a09f15360 2024-04-16T05:38:07.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/3532377896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8ae8004450 con 0x563a09f15360 2024-04-16T05:38:07.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8afa47c740 1 -- 172.21.15.86:0/3532377896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae0004860 con 0x563a09f15360 2024-04-16T05:38:07.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/3532377896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ae8039070 con 0x563a09f15360 2024-04-16T05:38:07.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8afa47c740 10 client.16620 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:07.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8afa47c740 1 -- 172.21.15.86:0/3532377896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a09f8f500 con 0x563a09f15360 2024-04-16T05:38:07.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/3532377896 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae0004860 con 0x563a09f15360 2024-04-16T05:38:07.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/3532377896 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ae804b070 con 0x563a09f15360 2024-04-16T05:38:07.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8afa47c740 10 client.16620 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:07.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8afa47c740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:07.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8afa47c740 20 client.16620 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:07.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563a09f8fc80 0x563a09f920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:07.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8afa47c740 4 client.16620 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:07.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8afa47c740 1 -- 172.21.15.86:0/3532377896 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a09e78970 con 0x563a09f8fc80 2024-04-16T05:38:07.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.163+0000 7f8af5624640 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563a09f8fc80 0x563a09f920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:07.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8af5624640 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563a09f8fc80 0x563a09f920a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=298907ff20b47a9d server_cookie=6883f136ba9450b4 in_seq=0 out_seq=0 2024-04-16T05:38:07.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8ad8ff9640 10 client.16620 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:07.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8ad8ff9640 1 -- 172.21.15.86:0/3532377896 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a09e78970 con 0x563a09f8fc80 2024-04-16T05:38:07.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8ad8ff9640 10 client.16620 handle_command_reply: tid=0 2024-04-16T05:38:07.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 client.16620 shutdown 2024-04-16T05:38:07.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 -- 172.21.15.86:0/3532377896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 msgr2=0x563a09f0eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f0eca0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8036000 tx=0x7f8ae80324b0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 -- 172.21.15.86:0/3532377896 shutdown_connections 2024-04-16T05:38:07.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563a09f8fc80 0x563a09f920a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a09f15360 0x563a09f0eca0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a09f0c780 0x563a09f0e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8ad8ff9640 0 client.16620 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:07.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 --2- 172.21.15.86:0/3532377896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a09f0be60 0x563a09f0fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 -- 172.21.15.86:0/3532377896 >> 172.21.15.86:0/3532377896 conn(0x563a09f06590 msgr2=0x563a09f04ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 -- 172.21.15.86:0/3532377896 shutdown_connections 2024-04-16T05:38:07.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 1 -- 172.21.15.86:0/3532377896 wait complete. 2024-04-16T05:38:07.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.164+0000 7f8afa47c740 20 client.16620 trim_cache size 0 max 16384 2024-04-16T05:38:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:07.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 -- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8adc0a3d20 msgr2=0x7f8adc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8adc0a3d20 0x7f8adc0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009430 tx=0x7f8ae80095c0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 -- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101250 msgr2=0x7f8af019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:07.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101250 0x7f8af019e5a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c920 tx=0x7f8aec00cde0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 -- 172.21.15.86:0/1963608488 shutdown_connections 2024-04-16T05:38:07.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af0107040 0x7f8af0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8adc0a3d20 0x7f8adc0a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af0101bf0 0x7f8af019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 --2- 172.21.15.86:0/1963608488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af0101250 0x7f8af019e5a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:07.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 -- 172.21.15.86:0/1963608488 >> 172.21.15.86:0/1963608488 conn(0x7f8af0076d00 msgr2=0x7f8af006a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:07.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 -- 172.21.15.86:0/1963608488 shutdown_connections 2024-04-16T05:38:07.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:07.165+0000 7f8af7888640 1 -- 172.21.15.86:0/1963608488 wait complete. 2024-04-16T05:38:08.176 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:38:08.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:38:08.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:38:08.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/921610034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24101150 msgr2=0x7f6c24101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/921610034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24101150 0x7f6c24101530 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009900 tx=0x7f6c1402bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/921610034 shutdown_connections 2024-04-16T05:38:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/921610034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c24106220 0x7f6c2410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/921610034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c24101b00 0x7f6c24105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/921610034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24101150 0x7f6c24101530 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/921610034 >> 172.21.15.86:0/921610034 conn(0x7f6c24077280 msgr2=0x7f6c24077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/921610034 shutdown_connections 2024-04-16T05:38:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/921610034 wait complete. 2024-04-16T05:38:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.299+0000 7f6c2a8c8640 1 Processor -- start 2024-04-16T05:38:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c2a8c8640 1 -- start start 2024-04-16T05:38:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c2a8c8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c24101150 0x7f6c2410e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c2a8c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24106220 0x7f6c2410ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c2a8c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2410f200 0x7f6c241a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c2a8c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24112430 con 0x7f6c24106220 2024-04-16T05:38:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c2a8c8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241122b0 con 0x7f6c24101150 2024-04-16T05:38:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c2a8c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241125b0 con 0x7f6c2410f200 2024-04-16T05:38:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c237fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24106220 0x7f6c2410ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c237fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24106220 0x7f6c2410ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51308/0 (socket says 172.21.15.86:51308) 2024-04-16T05:38:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c237fe640 1 -- 172.21.15.86:0/4081762802 learned_addr learned my addr 172.21.15.86:0/4081762802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c23fff640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c24101150 0x7f6c2410e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c28e65640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2410f200 0x7f6c241a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c237fe640 1 -- 172.21.15.86:0/4081762802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2410f200 msgr2=0x7f6c241a6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c237fe640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2410f200 0x7f6c241a6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c237fe640 1 -- 172.21.15.86:0/4081762802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c24101150 msgr2=0x7f6c2410e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c237fe640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c24101150 0x7f6c2410e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c237fe640 1 -- 172.21.15.86:0/4081762802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c241a71c0 con 0x7f6c24106220 2024-04-16T05:38:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.300+0000 7f6c23fff640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c24101150 0x7f6c2410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.301+0000 7f6c237fe640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24106220 0x7f6c2410ecc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800b450 tx=0x7f6c1800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.301+0000 7f6c217fa640 1 -- 172.21.15.86:0/4081762802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18015020 con 0x7f6c24106220 2024-04-16T05:38:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.301+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/4081762802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c241a74b0 con 0x7f6c24106220 2024-04-16T05:38:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.301+0000 7f6c217fa640 1 -- 172.21.15.86:0/4081762802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c18011d70 con 0x7f6c24106220 2024-04-16T05:38:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.301+0000 7f6c217fa640 1 -- 172.21.15.86:0/4081762802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800f340 con 0x7f6c24106220 2024-04-16T05:38:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.301+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/4081762802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c241a7a00 con 0x7f6c24106220 2024-04-16T05:38:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.303+0000 7f6c217fa640 1 -- 172.21.15.86:0/4081762802 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6c18018020 con 0x7f6c24106220 2024-04-16T05:38:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.303+0000 7f6c2d592740 1 Processor -- start 2024-04-16T05:38:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c2d592740 1 -- start start 2024-04-16T05:38:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec46e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec473d0 0x55ccaec4fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec4ffb0 0x55ccaec523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c2d592740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ccaebabe90 con 0x55ccaec46ab0 2024-04-16T05:38:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c2d592740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ccaeb494c0 con 0x55ccaec4ffb0 2024-04-16T05:38:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c2d592740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ccaeba7e50 con 0x55ccaec473d0 2024-04-16T05:38:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec46e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c217fa640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6c0c0a3d50 0x7f6c0c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec46e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51314/0 (socket says 172.21.15.86:51314) 2024-04-16T05:38:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 -- 172.21.15.86:0/253234327 learned_addr learned my addr 172.21.15.86:0/253234327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c237fe640 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec4ffb0 0x55ccaec523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c23fff640 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec473d0 0x55ccaec4fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6c0c0a3d50 0x7f6c0c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c217fa640 1 -- 172.21.15.86:0/4081762802 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6c180d05c0 con 0x7f6c24106220 2024-04-16T05:38:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 -- 172.21.15.86:0/253234327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec473d0 msgr2=0x55ccaec4fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec473d0 0x55ccaec4fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 -- 172.21.15.86:0/253234327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec4ffb0 msgr2=0x55ccaec523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec4ffb0 0x55ccaec523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 -- 172.21.15.86:0/253234327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccae9867e0 con 0x55ccaec46ab0 2024-04-16T05:38:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c237fe640 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec4ffb0 0x55ccaec523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.304+0000 7f6c28e65640 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec46e90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c014980 tx=0x7f6c1c014e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d532754b729ef580 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c0a7fc640 1 -- 172.21.15.86:0/253234327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c021070 con 0x55ccaec46ab0 2024-04-16T05:38:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c0a7fc640 1 -- 172.21.15.86:0/253234327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1c00e910 con 0x55ccaec46ab0 2024-04-16T05:38:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c0a7fc640 1 -- 172.21.15.86:0/253234327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00f630 con 0x55ccaec46ab0 2024-04-16T05:38:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c2d592740 1 -- 172.21.15.86:0/253234327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 msgr2=0x55ccaec46e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c28e65640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6c0c0a3d50 0x7f6c0c0a6210 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c01e000 tx=0x7f6c1c01a000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c2d592740 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec46e90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c014980 tx=0x7f6c1c014e40 comp rx=0 tx=0).stop 2024-04-16T05:38:08.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c2d592740 1 -- 172.21.15.86:0/253234327 shutdown_connections 2024-04-16T05:38:08.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c2d592740 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec4ffb0 0x55ccaec523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c2d592740 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec473d0 0x55ccaec4fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c2d592740 1 --2- 172.21.15.86:0/253234327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec46e90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c2d592740 1 -- 172.21.15.86:0/253234327 >> 172.21.15.86:0/253234327 conn(0x55ccaec411e0 msgr2=0x55ccaec415c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c2d592740 1 -- 172.21.15.86:0/253234327 shutdown_connections 2024-04-16T05:38:08.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.305+0000 7f6c2d592740 1 -- 172.21.15.86:0/253234327 wait complete. 2024-04-16T05:38:08.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c2d592740 1 -- start start 2024-04-16T05:38:08.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c2d592740 1 Processor -- start 2024-04-16T05:38:08.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec46ab0 0x55ccaec4d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 0x55ccaec4d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c2d592740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ccaec52da0 con 0x55ccaec4ffb0 2024-04-16T05:38:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c2d592740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ccaeb34410 con 0x55ccaec473d0 2024-04-16T05:38:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c2d592740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ccaec52f20 con 0x55ccaec46ab0 2024-04-16T05:38:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c2d592740 10 client.0 _ll_register_callbacks cb 0x55ccaeb2fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c28e65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c28e65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51328/0 (socket says 172.21.15.86:51328) 2024-04-16T05:38:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.306+0000 7f6c28e65640 1 -- 172.21.15.86:0/1906419618 learned_addr learned my addr 172.21.15.86:0/1906419618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec46ab0 0x55ccaec4d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c28e65640 1 -- 172.21.15.86:0/1906419618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec46ab0 msgr2=0x55ccaec4d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c28e65640 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec46ab0 0x55ccaec4d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c28e65640 1 -- 172.21.15.86:0/1906419618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 msgr2=0x55ccaec4d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c28e65640 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 0x55ccaec4d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c28e65640 1 -- 172.21.15.86:0/1906419618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccae9e6f70 con 0x55ccaec4ffb0 2024-04-16T05:38:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec46ab0 0x55ccaec4d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c28e65640 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4daf0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c014950 tx=0x7f6c1c008820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e3a84e1ef0e360 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c037fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:08.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c037fe640 1 -- 172.21.15.86:0/1906419618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c021040 con 0x55ccaec4ffb0 2024-04-16T05:38:08.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c037fe640 1 -- 172.21.15.86:0/1906419618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6c1c00e630 con 0x55ccaec4ffb0 2024-04-16T05:38:08.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c2d592740 1 -- 172.21.15.86:0/1906419618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c1c00e910 con 0x55ccaec4ffb0 2024-04-16T05:38:08.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c037fe640 1 -- 172.21.15.86:0/1906419618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c1c03b050 con 0x55ccaec4ffb0 2024-04-16T05:38:08.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c2d592740 10 client.16632 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:08.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c2d592740 1 -- 172.21.15.86:0/1906419618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ccae87e170 con 0x55ccaec4ffb0 2024-04-16T05:38:08.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c037fe640 1 -- 172.21.15.86:0/1906419618 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1c00e910 con 0x55ccaec4ffb0 2024-04-16T05:38:08.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.307+0000 7f6c037fe640 1 -- 172.21.15.86:0/1906419618 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c1c00f990 con 0x55ccaec4ffb0 2024-04-16T05:38:08.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.308+0000 7f6c2d592740 10 client.16632 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:08.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.308+0000 7f6c2d592740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:08.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.308+0000 7f6c2d592740 20 client.16632 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:08.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.308+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ccaec60250 0x55ccaec62640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.308+0000 7f6c2d592740 4 client.16632 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:08.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.308+0000 7f6c2d592740 1 -- 172.21.15.86:0/1906419618 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccaebb98b0 con 0x55ccaec60250 2024-04-16T05:38:08.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.308+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ccaec60250 0x55ccaec62640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.308+0000 7f6c037fe640 10 client.16632 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.308+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ccaec60250 0x55ccaec62640 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1303d7a3f6759690 server_cookie=1ca1b40105daea4f in_seq=0 out_seq=0 2024-04-16T05:38:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c037fe640 1 -- 172.21.15.86:0/1906419618 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccaebb98b0 con 0x55ccaec60250 2024-04-16T05:38:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c037fe640 10 client.16632 handle_command_reply: tid=0 2024-04-16T05:38:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 client.16632 shutdown 2024-04-16T05:38:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 -- 172.21.15.86:0/1906419618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec4ffb0 msgr2=0x55ccaec4daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4daf0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c014950 tx=0x7f6c1c008820 comp rx=0 tx=0).stop 2024-04-16T05:38:08.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 -- 172.21.15.86:0/1906419618 shutdown_connections 2024-04-16T05:38:08.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ccaec60250 0x55ccaec62640 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4daf0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 0x55ccaec4d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1906419618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec46ab0 0x55ccaec4d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c037fe640 0 client.16632 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:08.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 -- 172.21.15.86:0/1906419618 >> 172.21.15.86:0/1906419618 conn(0x55ccaec411e0 msgr2=0x55ccaec42f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 -- 172.21.15.86:0/1906419618 shutdown_connections 2024-04-16T05:38:08.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 1 -- 172.21.15.86:0/1906419618 wait complete. 2024-04-16T05:38:08.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.309+0000 7f6c2d592740 20 client.16632 trim_cache size 0 max 16384 2024-04-16T05:38:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c2d592740 1 Processor -- start 2024-04-16T05:38:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c2d592740 1 -- start start 2024-04-16T05:38:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec4f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 0x55ccaec5bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c2d592740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b15e0 con 0x55ccaec46ab0 2024-04-16T05:38:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c2d592740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b3120 con 0x55ccaec473d0 2024-04-16T05:38:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c2d592740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b1cb0 con 0x55ccaec4ffb0 2024-04-16T05:38:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c23fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec4f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c23fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec4f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51334/0 (socket says 172.21.15.86:51334) 2024-04-16T05:38:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c23fff640 1 -- 172.21.15.86:0/1878947219 learned_addr learned my addr 172.21.15.86:0/1878947219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c28e65640 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c23fff640 1 -- 172.21.15.86:0/1878947219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 msgr2=0x55ccaec4d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c237fe640 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 0x55ccaec5bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c23fff640 1 -- 172.21.15.86:0/1878947219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 msgr2=0x55ccaec5bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 0x55ccaec5bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c23fff640 1 -- 172.21.15.86:0/1878947219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccaec3f660 con 0x55ccaec46ab0 2024-04-16T05:38:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c237fe640 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 0x55ccaec5bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.327+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec4f7a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6c1402c170 tx=0x7f6c1402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a769573b4fc122 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c09ffb640 1 -- 172.21.15.86:0/1878947219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1404d070 con 0x55ccaec46ab0 2024-04-16T05:38:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c09ffb640 1 -- 172.21.15.86:0/1878947219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1402cb40 con 0x55ccaec46ab0 2024-04-16T05:38:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c09ffb640 1 -- 172.21.15.86:0/1878947219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14004a40 con 0x55ccaec46ab0 2024-04-16T05:38:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 -- 172.21.15.86:0/1878947219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 msgr2=0x55ccaec4f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec4f7a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6c1402c170 tx=0x7f6c1402e040 comp rx=0 tx=0).stop 2024-04-16T05:38:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 -- 172.21.15.86:0/1878947219 shutdown_connections 2024-04-16T05:38:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 0x55ccaec4d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec473d0 0x55ccaec5bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1878947219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec46ab0 0x55ccaec4f7a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 -- 172.21.15.86:0/1878947219 >> 172.21.15.86:0/1878947219 conn(0x55ccaec411e0 msgr2=0x55ccaec4d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 -- 172.21.15.86:0/1878947219 shutdown_connections 2024-04-16T05:38:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 -- 172.21.15.86:0/1878947219 wait complete. 2024-04-16T05:38:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.328+0000 7f6c2d592740 1 -- start start 2024-04-16T05:38:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c2d592740 1 Processor -- start 2024-04-16T05:38:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec46ab0 0x55ccaec58170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec473d0 0x55ccaec586b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c2d592740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 0x55ccaec55140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c2d592740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b2a50 con 0x55ccaec473d0 2024-04-16T05:38:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c2d592740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b2380 con 0x55ccaec46ab0 2024-04-16T05:38:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c2d592740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241af5a0 con 0x55ccaec4ffb0 2024-04-16T05:38:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c2d592740 10 client.0 _ll_register_callbacks cb 0x7f6c1c01f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c237fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec473d0 0x55ccaec586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c237fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec473d0 0x55ccaec586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51342/0 (socket says 172.21.15.86:51342) 2024-04-16T05:38:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c237fe640 1 -- 172.21.15.86:0/1096761574 learned_addr learned my addr 172.21.15.86:0/1096761574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec46ab0 0x55ccaec58170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c28e65640 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 0x55ccaec55140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c237fe640 1 -- 172.21.15.86:0/1096761574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 msgr2=0x55ccaec55140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c237fe640 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 0x55ccaec55140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c237fe640 1 -- 172.21.15.86:0/1096761574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec46ab0 msgr2=0x55ccaec58170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c237fe640 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec46ab0 0x55ccaec58170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c237fe640 1 -- 172.21.15.86:0/1096761574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccaec55880 con 0x55ccaec473d0 2024-04-16T05:38:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec46ab0 0x55ccaec58170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.329+0000 7f6c28e65640 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 0x55ccaec55140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c237fe640 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec473d0 0x55ccaec586b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6c18098cd0 tx=0x7f6c1801eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b300d21229beb401 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c037fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c037fe640 1 -- 172.21.15.86:0/1096761574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18066e10 con 0x55ccaec473d0 2024-04-16T05:38:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c037fe640 1 -- 172.21.15.86:0/1096761574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6c180cdc20 con 0x55ccaec473d0 2024-04-16T05:38:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c2d592740 1 -- 172.21.15.86:0/1096761574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c1402cb40 con 0x55ccaec473d0 2024-04-16T05:38:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c037fe640 1 -- 172.21.15.86:0/1096761574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c180cde20 con 0x55ccaec473d0 2024-04-16T05:38:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c2d592740 10 client.16638 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c2d592740 1 -- 172.21.15.86:0/1096761574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ccaecc71a0 con 0x55ccaec473d0 2024-04-16T05:38:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c037fe640 1 -- 172.21.15.86:0/1096761574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1402cb40 con 0x55ccaec473d0 2024-04-16T05:38:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c037fe640 1 -- 172.21.15.86:0/1096761574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c180d0210 con 0x55ccaec473d0 2024-04-16T05:38:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c2d592740 10 client.16638 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c2d592740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c2d592740 20 client.16638 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ccaecc78d0 0x55ccaecc9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c2d592740 4 client.16638 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:38:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.330+0000 7f6c2d592740 1 -- 172.21.15.86:0/1096761574 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ccaebb3720 con 0x55ccaecc78d0 2024-04-16T05:38:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.331+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ccaecc78d0 0x55ccaecc9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.331+0000 7f6c23fff640 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ccaecc78d0 0x55ccaecc9cf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed30111028cc26e server_cookie=fcb774186d73edf1 in_seq=0 out_seq=0 2024-04-16T05:38:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.331+0000 7f6c037fe640 10 client.16638 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:08.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c037fe640 1 -- 172.21.15.86:0/1096761574 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ccaebb3720 con 0x55ccaecc78d0 2024-04-16T05:38:08.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c037fe640 10 client.16638 handle_command_reply: tid=0 2024-04-16T05:38:08.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c2d592740 1 client.16638 shutdown 2024-04-16T05:38:08.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c2d592740 1 -- 172.21.15.86:0/1096761574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec473d0 msgr2=0x55ccaec586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec473d0 0x55ccaec586b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6c18098cd0 tx=0x7f6c1801eb80 comp rx=0 tx=0).stop 2024-04-16T05:38:08.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c2d592740 1 -- 172.21.15.86:0/1096761574 shutdown_connections 2024-04-16T05:38:08.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ccaecc78d0 0x55ccaecc9cf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccaec4ffb0 0x55ccaec55140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccaec473d0 0x55ccaec586b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c037fe640 0 client.16638 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:08.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c2d592740 1 --2- 172.21.15.86:0/1096761574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccaec46ab0 0x55ccaec58170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.332+0000 7f6c2d592740 1 -- 172.21.15.86:0/1096761574 >> 172.21.15.86:0/1096761574 conn(0x55ccaec411e0 msgr2=0x55ccaec42df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2d592740 1 -- 172.21.15.86:0/1096761574 shutdown_connections 2024-04-16T05:38:08.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2d592740 1 -- 172.21.15.86:0/1096761574 wait complete. 2024-04-16T05:38:08.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2d592740 20 client.16638 trim_cache size 0 max 16384 2024-04-16T05:38:08.344 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:38:08.344 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:38:08.344 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "513591df-2678-4caf-8b29-71b196c49e66", 2024-04-16T05:38:08.345 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:38:08.345 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:38:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6c0c0a3d50 msgr2=0x7f6c0c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6c0c0a3d50 0x7f6c0c0a6210 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c01e000 tx=0x7f6c1c01a000 comp rx=0 tx=0).stop 2024-04-16T05:38:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24106220 msgr2=0x7f6c2410ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24106220 0x7f6c2410ecc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800b450 tx=0x7f6c1800b910 comp rx=0 tx=0).stop 2024-04-16T05:38:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/4081762802 shutdown_connections 2024-04-16T05:38:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2410f200 0x7f6c241a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c24106220 0x7f6c2410ecc0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f6c0c0a3d50 0x7f6c0c0a6210 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 --2- 172.21.15.86:0/4081762802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c24101150 0x7f6c2410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/4081762802 >> 172.21.15.86:0/4081762802 conn(0x7f6c24077280 msgr2=0x7f6c240fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/4081762802 shutdown_connections 2024-04-16T05:38:08.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.333+0000 7f6c2a8c8640 1 -- 172.21.15.86:0/4081762802 wait complete. 2024-04-16T05:38:08.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:38:08.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:38:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 -- 172.21.15.86:0/407314363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100500 msgr2=0x7fe438100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 --2- 172.21.15.86:0/407314363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100500 0x7fe438100980 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe428009930 tx=0x7fe42802bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 -- 172.21.15.86:0/407314363 shutdown_connections 2024-04-16T05:38:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 --2- 172.21.15.86:0/407314363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe438100ec0 0x7fe43810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 --2- 172.21.15.86:0/407314363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100500 0x7fe438100980 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 --2- 172.21.15.86:0/407314363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe438106fe0 0x7fe4381073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 -- 172.21.15.86:0/407314363 >> 172.21.15.86:0/407314363 conn(0x7fe438077490 msgr2=0x7fe4380fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 -- 172.21.15.86:0/407314363 shutdown_connections 2024-04-16T05:38:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 -- 172.21.15.86:0/407314363 wait complete. 2024-04-16T05:38:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 Processor -- start 2024-04-16T05:38:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.457+0000 7fe4402e4640 1 -- start start 2024-04-16T05:38:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe4402e4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe438100500 0x7fe43819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe4402e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100ec0 0x7fe43819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe4402e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe438106fe0 0x7fe438198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe4402e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4381124f0 con 0x7fe438100ec0 2024-04-16T05:38:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe4402e4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe438112370 con 0x7fe438100500 2024-04-16T05:38:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe4402e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe438112670 con 0x7fe438106fe0 2024-04-16T05:38:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43d87f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100ec0 0x7fe43819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43d87f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100ec0 0x7fe43819ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51360/0 (socket says 172.21.15.86:51360) 2024-04-16T05:38:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43d87f640 1 -- 172.21.15.86:0/4199721647 learned_addr learned my addr 172.21.15.86:0/4199721647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43e080640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe438100500 0x7fe43819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43e881640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe438106fe0 0x7fe438198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43d87f640 1 -- 172.21.15.86:0/4199721647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe438106fe0 msgr2=0x7fe438198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43d87f640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe438106fe0 0x7fe438198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43d87f640 1 -- 172.21.15.86:0/4199721647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe438100500 msgr2=0x7fe43819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43d87f640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe438100500 0x7fe43819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43d87f640 1 -- 172.21.15.86:0/4199721647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe438199000 con 0x7fe438100ec0 2024-04-16T05:38:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe43d87f640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100ec0 0x7fe43819ec00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe428009230 tx=0x7fe42802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe42f7fe640 1 -- 172.21.15.86:0/4199721647 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428041070 con 0x7fe438100ec0 2024-04-16T05:38:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe42f7fe640 1 -- 172.21.15.86:0/4199721647 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe428004450 con 0x7fe438100ec0 2024-04-16T05:38:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.458+0000 7fe4402e4640 1 -- 172.21.15.86:0/4199721647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe438199290 con 0x7fe438100ec0 2024-04-16T05:38:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.459+0000 7fe42f7fe640 1 -- 172.21.15.86:0/4199721647 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428034440 con 0x7fe438100ec0 2024-04-16T05:38:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.459+0000 7fe4402e4640 1 -- 172.21.15.86:0/4199721647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4381a5510 con 0x7fe438100ec0 2024-04-16T05:38:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.461+0000 7fe42f7fe640 1 -- 172.21.15.86:0/4199721647 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe428044070 con 0x7fe438100ec0 2024-04-16T05:38:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.461+0000 7fe442faf740 1 Processor -- start 2024-04-16T05:38:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe442faf740 1 -- start start 2024-04-16T05:38:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe442faf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 0x556524f45230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe442faf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524f4de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe442faf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524f50740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe442faf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556524eaa180 con 0x556524f4e350 2024-04-16T05:38:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe442faf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556524e47360 con 0x556524f44e50 2024-04-16T05:38:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe442faf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556524ea6140 con 0x556524f45770 2024-04-16T05:38:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe42f7fe640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4240a3d20 0x7fe4240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe43d87f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524f50740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe43d87f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524f50740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51370/0 (socket says 172.21.15.86:51370) 2024-04-16T05:38:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe43d87f640 1 -- 172.21.15.86:0/1300926062 learned_addr learned my addr 172.21.15.86:0/1300926062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe43e881640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4240a3d20 0x7fe4240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe43e881640 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 0x556524f45230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe43e080640 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524f4de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.462+0000 7fe42f7fe640 1 -- 172.21.15.86:0/4199721647 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe4280f25d0 con 0x7fe438100ec0 2024-04-16T05:38:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe43d87f640 1 -- 172.21.15.86:0/1300926062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 msgr2=0x556524f4de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe43d87f640 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524f4de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe43d87f640 1 -- 172.21.15.86:0/1300926062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 msgr2=0x556524f45230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe43d87f640 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 0x556524f45230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe43d87f640 1 -- 172.21.15.86:0/1300926062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556524c867d0 con 0x556524f4e350 2024-04-16T05:38:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe43e881640 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 0x556524f45230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:38:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe43d87f640 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524f50740 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe42802cfd0 tx=0x7fe4280035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb09ec66a4dade6a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe43e881640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4240a3d20 0x7fe4240a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe430005fd0 tx=0x7fe4300094a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe42cbfe640 1 -- 172.21.15.86:0/1300926062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42804b070 con 0x556524f4e350 2024-04-16T05:38:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe42cbfe640 1 -- 172.21.15.86:0/1300926062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe428004c80 con 0x556524f4e350 2024-04-16T05:38:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.463+0000 7fe42cbfe640 1 -- 172.21.15.86:0/1300926062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280c6f80 con 0x556524f4e350 2024-04-16T05:38:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 -- 172.21.15.86:0/1300926062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 msgr2=0x556524f50740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524f50740 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe42802cfd0 tx=0x7fe4280035f0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 -- 172.21.15.86:0/1300926062 shutdown_connections 2024-04-16T05:38:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524f50740 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524f4de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 --2- 172.21.15.86:0/1300926062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 0x556524f45230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 -- 172.21.15.86:0/1300926062 >> 172.21.15.86:0/1300926062 conn(0x556524f3f580 msgr2=0x556524f3f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 -- 172.21.15.86:0/1300926062 shutdown_connections 2024-04-16T05:38:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 -- 172.21.15.86:0/1300926062 wait complete. 2024-04-16T05:38:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 -- start start 2024-04-16T05:38:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 Processor -- start 2024-04-16T05:38:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f44e50 0x556524f4b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f45770 0x556524f4b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 0x556524f4be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556524f50f00 con 0x556524f45770 2024-04-16T05:38:08.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556524e632c0 con 0x556524f4e350 2024-04-16T05:38:08.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556524f51080 con 0x556524f44e50 2024-04-16T05:38:08.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe442faf740 10 client.0 _ll_register_callbacks cb 0x556524e4a090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:08.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.464+0000 7fe43d87f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f45770 0x556524f4b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43d87f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f45770 0x556524f4b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51376/0 (socket says 172.21.15.86:51376) 2024-04-16T05:38:08.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43d87f640 1 -- 172.21.15.86:0/3116855967 learned_addr learned my addr 172.21.15.86:0/3116855967 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43e080640 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f44e50 0x556524f4b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43e881640 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 0x556524f4be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43d87f640 1 -- 172.21.15.86:0/3116855967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f44e50 msgr2=0x556524f4b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43d87f640 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f44e50 0x556524f4b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43d87f640 1 -- 172.21.15.86:0/3116855967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 msgr2=0x556524f4be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43d87f640 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 0x556524f4be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43d87f640 1 -- 172.21.15.86:0/3116855967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556524bbc3f0 con 0x556524f45770 2024-04-16T05:38:08.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43e080640 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f44e50 0x556524f4b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:08.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43e881640 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 0x556524f4be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:08.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe43d87f640 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f45770 0x556524f4b930 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe42802c820 tx=0x7fe428000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f297b700c2843444 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:08.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe421ffb640 1 -- 172.21.15.86:0/3116855967 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280d7060 con 0x556524f45770 2024-04-16T05:38:08.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe421ffb640 1 -- 172.21.15.86:0/3116855967 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe4280efc30 con 0x556524f45770 2024-04-16T05:38:08.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe442faf740 1 -- 172.21.15.86:0/3116855967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe428004c80 con 0x556524f45770 2024-04-16T05:38:08.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe421ffb640 1 -- 172.21.15.86:0/3116855967 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe42804b040 con 0x556524f45770 2024-04-16T05:38:08.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe442faf740 10 client.16650 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:08.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe442faf740 1 -- 172.21.15.86:0/3116855967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556524b7c0f0 con 0x556524f45770 2024-04-16T05:38:08.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe421ffb640 1 -- 172.21.15.86:0/3116855967 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe428004c80 con 0x556524f45770 2024-04-16T05:38:08.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.465+0000 7fe421ffb640 1 -- 172.21.15.86:0/3116855967 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe4280c6af0 con 0x556524f45770 2024-04-16T05:38:08.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.466+0000 7fe442faf740 10 client.16650 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:08.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.466+0000 7fe442faf740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:08.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.466+0000 7fe442faf740 20 client.16650 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:08.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.466+0000 7fe442faf740 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556524f5e2f0 0x556524f606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.466+0000 7fe442faf740 4 client.16650 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:08.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.466+0000 7fe442faf740 1 -- 172.21.15.86:0/3116855967 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556524eb7ba0 con 0x556524f5e2f0 2024-04-16T05:38:08.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.466+0000 7fe43e080640 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556524f5e2f0 0x556524f606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.466+0000 7fe421ffb640 10 client.16650 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:08.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.466+0000 7fe43e080640 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556524f5e2f0 0x556524f606e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f63de010d7a0752 server_cookie=39ebd00f69a2d1b1 in_seq=0 out_seq=0 2024-04-16T05:38:08.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe421ffb640 1 -- 172.21.15.86:0/3116855967 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556524eb7ba0 con 0x556524f5e2f0 2024-04-16T05:38:08.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe421ffb640 10 client.16650 handle_command_reply: tid=0 2024-04-16T05:38:08.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 client.16650 shutdown 2024-04-16T05:38:08.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 -- 172.21.15.86:0/3116855967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f45770 msgr2=0x556524f4b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f45770 0x556524f4b930 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe42802c820 tx=0x7fe428000bc0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 -- 172.21.15.86:0/3116855967 shutdown_connections 2024-04-16T05:38:08.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556524f5e2f0 0x556524f606e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 0x556524f4be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f45770 0x556524f4b930 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 --2- 172.21.15.86:0/3116855967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f44e50 0x556524f4b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe421ffb640 0 client.16650 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:08.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 -- 172.21.15.86:0/3116855967 >> 172.21.15.86:0/3116855967 conn(0x556524f3f580 msgr2=0x556524f412d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 -- 172.21.15.86:0/3116855967 shutdown_connections 2024-04-16T05:38:08.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 1 -- 172.21.15.86:0/3116855967 wait complete. 2024-04-16T05:38:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.467+0000 7fe442faf740 20 client.16650 trim_cache size 0 max 16384 2024-04-16T05:38:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.484+0000 7fe442faf740 1 Processor -- start 2024-04-16T05:38:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.484+0000 7fe442faf740 1 -- start start 2024-04-16T05:38:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.484+0000 7fe442faf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f44e50 0x556524f4db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.484+0000 7fe442faf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524f63710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.484+0000 7fe442faf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 0x556524f3ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.484+0000 7fe442faf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4381af480 con 0x556524f44e50 2024-04-16T05:38:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.484+0000 7fe442faf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4381b0fc0 con 0x556524f4e350 2024-04-16T05:38:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.484+0000 7fe442faf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4381afb50 con 0x556524f45770 2024-04-16T05:38:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43e080640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f44e50 0x556524f4db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43e080640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f44e50 0x556524f4db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51386/0 (socket says 172.21.15.86:51386) 2024-04-16T05:38:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43e080640 1 -- 172.21.15.86:0/4109705797 learned_addr learned my addr 172.21.15.86:0/4109705797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43d87f640 1 --2- 172.21.15.86:0/4109705797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524f63710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43e080640 1 -- 172.21.15.86:0/4109705797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 msgr2=0x556524f63710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43e080640 1 --2- 172.21.15.86:0/4109705797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524f63710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43e080640 1 -- 172.21.15.86:0/4109705797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 msgr2=0x556524f3ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43e080640 1 --2- 172.21.15.86:0/4109705797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 0x556524f3ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43e080640 1 -- 172.21.15.86:0/4109705797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556524f407b0 con 0x556524f44e50 2024-04-16T05:38:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe43e080640 1 --2- 172.21.15.86:0/4109705797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f44e50 0x556524f4db20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe4340095e0 tx=0x7fe434008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea881941f047d5c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe423fff640 1 -- 172.21.15.86:0/4109705797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434021070 con 0x556524f44e50 2024-04-16T05:38:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe423fff640 1 -- 172.21.15.86:0/4109705797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe434016d40 con 0x556524f44e50 2024-04-16T05:38:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.485+0000 7fe423fff640 1 -- 172.21.15.86:0/4109705797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434015620 con 0x556524f44e50 2024-04-16T05:38:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 -- 172.21.15.86:0/4109705797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f44e50 msgr2=0x556524f4db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 --2- 172.21.15.86:0/4109705797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f44e50 0x556524f4db20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe4340095e0 tx=0x7fe434008040 comp rx=0 tx=0).stop 2024-04-16T05:38:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 -- 172.21.15.86:0/4109705797 shutdown_connections 2024-04-16T05:38:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 --2- 172.21.15.86:0/4109705797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f4e350 0x556524f3ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 --2- 172.21.15.86:0/4109705797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524f63710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 --2- 172.21.15.86:0/4109705797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f44e50 0x556524f4db20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 -- 172.21.15.86:0/4109705797 >> 172.21.15.86:0/4109705797 conn(0x556524f3f580 msgr2=0x556524f49cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 -- 172.21.15.86:0/4109705797 shutdown_connections 2024-04-16T05:38:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 -- 172.21.15.86:0/4109705797 wait complete. 2024-04-16T05:38:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 -- start start 2024-04-16T05:38:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.486+0000 7fe442faf740 1 Processor -- start 2024-04-16T05:38:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe442faf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 0x556524fc0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe442faf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524fc0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe442faf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524fbac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe442faf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4381b08f0 con 0x556524f4e350 2024-04-16T05:38:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe442faf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4381b0220 con 0x556524f44e50 2024-04-16T05:38:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe442faf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4381ad440 con 0x556524f45770 2024-04-16T05:38:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe442faf740 10 client.0 _ll_register_callbacks cb 0x7fe430009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e881640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524fbac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e881640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524fbac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51392/0 (socket says 172.21.15.86:51392) 2024-04-16T05:38:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e881640 1 -- 172.21.15.86:0/1673524700 learned_addr learned my addr 172.21.15.86:0/1673524700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43d87f640 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524fc0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e080640 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 0x556524fc0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e881640 1 -- 172.21.15.86:0/1673524700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 msgr2=0x556524fc0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e881640 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524fc0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e881640 1 -- 172.21.15.86:0/1673524700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 msgr2=0x556524fc0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e881640 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 0x556524fc0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e881640 1 -- 172.21.15.86:0/1673524700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556524fbb300 con 0x556524f4e350 2024-04-16T05:38:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe43e881640 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524fbac10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe430012270 tx=0x7fe4300099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d533730af4c14d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe421ffb640 1 -- 172.21.15.86:0/1673524700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430025030 con 0x556524f4e350 2024-04-16T05:38:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe421ffb640 1 -- 172.21.15.86:0/1673524700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe43001b070 con 0x556524f4e350 2024-04-16T05:38:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.487+0000 7fe442faf740 1 -- 172.21.15.86:0/1673524700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe434016d40 con 0x556524f4e350 2024-04-16T05:38:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe421ffb640 1 -- 172.21.15.86:0/1673524700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe430004b20 con 0x556524f4e350 2024-04-16T05:38:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe442faf740 10 client.16656 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe442faf740 1 -- 172.21.15.86:0/1673524700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556524fc8010 con 0x556524f4e350 2024-04-16T05:38:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe421ffb640 1 -- 172.21.15.86:0/1673524700 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe434016d40 con 0x556524f4e350 2024-04-16T05:38:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe421ffb640 1 -- 172.21.15.86:0/1673524700 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe43001e050 con 0x556524f4e350 2024-04-16T05:38:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe442faf740 10 client.16656 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe442faf740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe442faf740 20 client.16656 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe442faf740 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556524fc87c0 0x556524fcabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe442faf740 4 client.16656 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:38:08.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe442faf740 1 -- 172.21.15.86:0/1673524700 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556524eb1a10 con 0x556524fc87c0 2024-04-16T05:38:08.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe43e080640 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556524fc87c0 0x556524fcabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:08.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe421ffb640 10 client.16656 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:08.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.488+0000 7fe43e080640 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556524fc87c0 0x556524fcabe0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ac95d86c927475 server_cookie=e03ded5c8260aa64 in_seq=0 out_seq=0 2024-04-16T05:38:08.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe421ffb640 1 -- 172.21.15.86:0/1673524700 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556524eb1a10 con 0x556524fc87c0 2024-04-16T05:38:08.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe421ffb640 10 client.16656 handle_command_reply: tid=0 2024-04-16T05:38:08.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 client.16656 shutdown 2024-04-16T05:38:08.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 -- 172.21.15.86:0/1673524700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 msgr2=0x556524fbac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524fbac10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe430012270 tx=0x7fe4300099c0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 -- 172.21.15.86:0/1673524700 shutdown_connections 2024-04-16T05:38:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556524fc87c0 0x556524fcabe0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556524f4e350 0x556524fbac10 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556524f45770 0x556524fc0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe421ffb640 0 client.16656 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 --2- 172.21.15.86:0/1673524700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556524f44e50 0x556524fc0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 -- 172.21.15.86:0/1673524700 >> 172.21.15.86:0/1673524700 conn(0x556524f3f580 msgr2=0x556524f4bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 -- 172.21.15.86:0/1673524700 shutdown_connections 2024-04-16T05:38:08.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 1 -- 172.21.15.86:0/1673524700 wait complete. 2024-04-16T05:38:08.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe442faf740 20 client.16656 trim_cache size 0 max 16384 2024-04-16T05:38:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:38:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:38:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:38:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "513591df-2678-4caf-8b29-71b196c49e66": { 2024-04-16T05:38:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:38:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "513591df-2678-4caf-8b29-71b196c49e66", 2024-04-16T05:38:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:38:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:38:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:38:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:38:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe4402e4640 1 -- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4240a3d20 msgr2=0x7fe4240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe4402e4640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4240a3d20 0x7fe4240a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe430005fd0 tx=0x7fe4300094a0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe4402e4640 1 -- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100ec0 msgr2=0x7fe43819ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.489+0000 7fe4402e4640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100ec0 0x7fe43819ec00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe428009230 tx=0x7fe42802cc70 comp rx=0 tx=0).stop 2024-04-16T05:38:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.490+0000 7fe4402e4640 1 -- 172.21.15.86:0/4199721647 shutdown_connections 2024-04-16T05:38:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.490+0000 7fe4402e4640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe438106fe0 0x7fe438198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.490+0000 7fe4402e4640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe438100ec0 0x7fe43819ec00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.490+0000 7fe4402e4640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4240a3d20 0x7fe4240a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.490+0000 7fe4402e4640 1 --2- 172.21.15.86:0/4199721647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe438100500 0x7fe43819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.490+0000 7fe4402e4640 1 -- 172.21.15.86:0/4199721647 >> 172.21.15.86:0/4199721647 conn(0x7fe438077490 msgr2=0x7fe438105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.490+0000 7fe4402e4640 1 -- 172.21.15.86:0/4199721647 shutdown_connections 2024-04-16T05:38:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:08.490+0000 7fe4402e4640 1 -- 172.21.15.86:0/4199721647 wait complete. 2024-04-16T05:38:08.506 INFO:tasks.cephfs.filesystem:scrub status for tag:513591df-2678-4caf-8b29-71b196c49e66 - {'path': '/', 'tag': '513591df-2678-4caf-8b29-71b196c49e66', 'options': 'recursive,force'} 2024-04-16T05:38:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:08 smithi194 ceph-mon[35865]: pgmap v88: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:38:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:08 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:38:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:08 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:08 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:38:08.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:08 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:08 smithi086 ceph-mon[32028]: pgmap v88: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:38:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:08 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:38:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:08 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:08.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:08 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:38:08.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:08 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:08 smithi196 ceph-mon[36373]: pgmap v88: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:38:08.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:08 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:38:08.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:08 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:08.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:08 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:38:08.986 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:08 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:09.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:09 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:38:09.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:09 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:38:09.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:09 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:38:10.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:10 smithi086 ceph-mon[32028]: pgmap v89: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:38:10.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:10 smithi196 ceph-mon[36373]: pgmap v89: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:38:11.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:10 smithi194 ceph-mon[35865]: pgmap v89: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:38:12.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:12 smithi086 ceph-mon[32028]: pgmap v90: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:38:12.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:12 smithi196 ceph-mon[36373]: pgmap v90: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:38:13.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:12 smithi194 ceph-mon[35865]: pgmap v90: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:38:14.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:14 smithi086 ceph-mon[32028]: pgmap v91: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-16T05:38:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:14 smithi196 ceph-mon[36373]: pgmap v91: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-16T05:38:15.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:14 smithi194 ceph-mon[35865]: pgmap v91: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-16T05:38:16.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:16 smithi086 ceph-mon[32028]: pgmap v92: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:38:16.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:16 smithi196 ceph-mon[36373]: pgmap v92: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:38:17.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:16 smithi194 ceph-mon[35865]: pgmap v92: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:38:18.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:18 smithi086 ceph-mon[32028]: pgmap v93: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:38:18.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:18 smithi196 ceph-mon[36373]: pgmap v93: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:38:19.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:18 smithi194 ceph-mon[35865]: pgmap v93: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-16T05:38:20.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:20 smithi086 ceph-mon[32028]: pgmap v94: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-16T05:38:20.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:20 smithi196 ceph-mon[36373]: pgmap v94: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-16T05:38:21.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:20 smithi194 ceph-mon[35865]: pgmap v94: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-16T05:38:22.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:22 smithi086 ceph-mon[32028]: pgmap v95: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-16T05:38:22.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:22 smithi196 ceph-mon[36373]: pgmap v95: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-16T05:38:23.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:22 smithi194 ceph-mon[35865]: pgmap v95: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-16T05:38:24.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:24 smithi086 ceph-mon[32028]: pgmap v96: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-16T05:38:24.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:24 smithi196 ceph-mon[36373]: pgmap v96: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-16T05:38:25.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:24 smithi194 ceph-mon[35865]: pgmap v96: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-16T05:38:25.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:25 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:38:25.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:25 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:38:25.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:25 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:38:25.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:25 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:38:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:25 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:38:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:25 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:38:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:25 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:38:25.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:25 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:38:26.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:25 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:38:26.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:25 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:38:26.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:25 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:38:26.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:25 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:38:26.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:26 smithi086 ceph-mon[32028]: pgmap v97: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:26.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:26 smithi196 ceph-mon[36373]: pgmap v97: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:27.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:26 smithi194 ceph-mon[35865]: pgmap v97: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:28.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:28 smithi086 ceph-mon[32028]: pgmap v98: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:28.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:28 smithi196 ceph-mon[36373]: pgmap v98: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:29.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:28 smithi194 ceph-mon[35865]: pgmap v98: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:30.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:30 smithi086 ceph-mon[32028]: pgmap v99: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:38:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:30 smithi196 ceph-mon[36373]: pgmap v99: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:38:31.157 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:30 smithi194 ceph-mon[35865]: pgmap v99: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:38:32.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:32 smithi086 ceph-mon[32028]: pgmap v100: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:32.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:32 smithi196 ceph-mon[36373]: pgmap v100: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:33.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:32 smithi194 ceph-mon[35865]: pgmap v100: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:34.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:34 smithi086 ceph-mon[32028]: pgmap v101: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:38:34.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:34 smithi196 ceph-mon[36373]: pgmap v101: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:38:35.158 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:34 smithi194 ceph-mon[35865]: pgmap v101: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:38:36.958 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:36 smithi194 ceph-mon[35865]: pgmap v102: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:36.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:36 smithi086 ceph-mon[32028]: pgmap v102: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:36.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:36 smithi196 ceph-mon[36373]: pgmap v102: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:38.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:38:38.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:38:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.629+0000 7f10f4fe2640 1 -- 172.21.15.86:0/778059074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101bf0 msgr2=0x7f10f010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.629+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/778059074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101bf0 0x7f10f010dd30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009cd0 tx=0x7f10e802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.629+0000 7f10f4fe2640 1 -- 172.21.15.86:0/778059074 shutdown_connections 2024-04-16T05:38:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.629+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/778059074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101bf0 0x7f10f010dd30 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.629+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/778059074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f0101250 0x7f10f01016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.629+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/778059074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10f01070e0 0x7f10f01074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.629+0000 7f10f4fe2640 1 -- 172.21.15.86:0/778059074 >> 172.21.15.86:0/778059074 conn(0x7f10f0076d00 msgr2=0x7f10f0077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.629+0000 7f10f4fe2640 1 -- 172.21.15.86:0/778059074 shutdown_connections 2024-04-16T05:38:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.629+0000 7f10f4fe2640 1 -- 172.21.15.86:0/778059074 wait complete. 2024-04-16T05:38:38.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.630+0000 7f10f4fe2640 1 Processor -- start 2024-04-16T05:38:38.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.630+0000 7f10f4fe2640 1 -- start start 2024-04-16T05:38:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.630+0000 7f10f4fe2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101250 0x7f10f0102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.630+0000 7f10f4fe2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f0101bf0 0x7f10f0103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.630+0000 7f10f4fe2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10f01070e0 0x7f10f0103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.630+0000 7f10f4fe2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10f01112e0 con 0x7f10f0101250 2024-04-16T05:38:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.630+0000 7f10f4fe2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f10f0111160 con 0x7f10f01070e0 2024-04-16T05:38:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.630+0000 7f10f4fe2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10f0111460 con 0x7f10f0101bf0 2024-04-16T05:38:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.630+0000 7f10ee59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101250 0x7f10f0102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10ee59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101250 0x7f10f0102c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39428/0 (socket says 172.21.15.86:39428) 2024-04-16T05:38:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10ee59c640 1 -- 172.21.15.86:0/1686480241 learned_addr learned my addr 172.21.15.86:0/1686480241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f0101bf0 0x7f10f0103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10eed9d640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10f01070e0 0x7f10f0103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10ee59c640 1 -- 172.21.15.86:0/1686480241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f0101bf0 msgr2=0x7f10f0103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10ee59c640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f0101bf0 0x7f10f0103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10ee59c640 1 -- 172.21.15.86:0/1686480241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10f01070e0 msgr2=0x7f10f0103680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10ee59c640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10f01070e0 0x7f10f0103680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10ee59c640 1 -- 172.21.15.86:0/1686480241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f01a6da0 con 0x7f10f0101250 2024-04-16T05:38:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10eed9d640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10f01070e0 0x7f10f0103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10ee59c640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101250 0x7f10f0102c00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f10e000d950 tx=0x7f10e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10e7fff640 1 -- 172.21.15.86:0/1686480241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0017070 con 0x7f10f0101250 2024-04-16T05:38:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10f4fe2640 1 -- 172.21.15.86:0/1686480241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f01a7090 con 0x7f10f0101250 2024-04-16T05:38:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10e7fff640 1 -- 172.21.15.86:0/1686480241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e0009c30 con 0x7f10f0101250 2024-04-16T05:38:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10e7fff640 1 -- 172.21.15.86:0/1686480241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e000f6b0 con 0x7f10f0101250 2024-04-16T05:38:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.631+0000 7f10f4fe2640 1 -- 172.21.15.86:0/1686480241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f01a75e0 con 0x7f10f0101250 2024-04-16T05:38:38.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.633+0000 7f10e7fff640 1 -- 172.21.15.86:0/1686480241 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10e00109d0 con 0x7f10f0101250 2024-04-16T05:38:38.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10f7bce740 1 Processor -- start 2024-04-16T05:38:38.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10f7bce740 1 -- start start 2024-04-16T05:38:38.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae89e50 0x5619fae8a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae8a770 0x5619fae92e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae95740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10f7bce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5619fadef180 con 0x5619fae93350 2024-04-16T05:38:38.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10f7bce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5619fad8c360 con 0x5619fae89e50 2024-04-16T05:38:38.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10f7bce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5619fadeb140 con 0x5619fae8a770 2024-04-16T05:38:38.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10e7fff640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f10dc0a3d50 0x7f10dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae95740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae95740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39432/0 (socket says 172.21.15.86:39432) 2024-04-16T05:38:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10e5d9b640 1 -- 172.21.15.86:0/3442716662 learned_addr learned my addr 172.21.15.86:0/3442716662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10eed9d640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f10dc0a3d50 0x7f10dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.634+0000 7f10e7fff640 1 -- 172.21.15.86:0/1686480241 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f10e00d06f0 con 0x7f10f0101250 2024-04-16T05:38:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10ee59c640 1 --2- 172.21.15.86:0/3442716662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae8a770 0x5619fae92e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10eed9d640 1 --2- 172.21.15.86:0/3442716662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae89e50 0x5619fae8a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10e5d9b640 1 -- 172.21.15.86:0/3442716662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae8a770 msgr2=0x5619fae92e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/3442716662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae8a770 0x5619fae92e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10e5d9b640 1 -- 172.21.15.86:0/3442716662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae89e50 msgr2=0x5619fae8a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/3442716662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae89e50 0x5619fae8a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10e5d9b640 1 -- 172.21.15.86:0/3442716662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fabcb7d0 con 0x5619fae93350 2024-04-16T05:38:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10eed9d640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f10dc0a3d50 0x7f10dc0a6210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f10e8032530 tx=0x7f10e8032d70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/3442716662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae95740 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f10d800b740 tx=0x7f10d800bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be5ce0508db70e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10e4d99640 1 -- 172.21.15.86:0/3442716662 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8003dd0 con 0x5619fae93350 2024-04-16T05:38:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10e4d99640 1 -- 172.21.15.86:0/3442716662 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d8015070 con 0x5619fae93350 2024-04-16T05:38:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.635+0000 7f10e4d99640 1 -- 172.21.15.86:0/3442716662 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8004e40 con 0x5619fae93350 2024-04-16T05:38:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 -- 172.21.15.86:0/3442716662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 msgr2=0x5619fae95740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 --2- 172.21.15.86:0/3442716662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae95740 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f10d800b740 tx=0x7f10d800bc00 comp rx=0 tx=0).stop 2024-04-16T05:38:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 -- 172.21.15.86:0/3442716662 shutdown_connections 2024-04-16T05:38:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 --2- 172.21.15.86:0/3442716662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae95740 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 --2- 172.21.15.86:0/3442716662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae8a770 0x5619fae92e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 --2- 172.21.15.86:0/3442716662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae89e50 0x5619fae8a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 -- 172.21.15.86:0/3442716662 >> 172.21.15.86:0/3442716662 conn(0x5619fae84580 msgr2=0x5619fae84960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 -- 172.21.15.86:0/3442716662 shutdown_connections 2024-04-16T05:38:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 -- 172.21.15.86:0/3442716662 wait complete. 2024-04-16T05:38:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 -- start start 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.636+0000 7f10f7bce740 1 Processor -- start 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae89e50 0x5619fae903c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 0x5619fae90900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae93350 0x5619fae90e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5619fae96140 con 0x5619fae89e50 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5619fada82c0 con 0x5619fae8a770 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5619fae962c0 con 0x5619fae93350 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 10 client.0 _ll_register_callbacks cb 0x5619fad8f090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10ee59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae89e50 0x5619fae903c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10ee59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae89e50 0x5619fae903c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39444/0 (socket says 172.21.15.86:39444) 2024-04-16T05:38:38.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10ee59c640 1 -- 172.21.15.86:0/2850746004 learned_addr learned my addr 172.21.15.86:0/2850746004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:38.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 0x5619fae90900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10eed9d640 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae93350 0x5619fae90e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10ee59c640 1 -- 172.21.15.86:0/2850746004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae93350 msgr2=0x5619fae90e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:38.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10ee59c640 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae93350 0x5619fae90e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10ee59c640 1 -- 172.21.15.86:0/2850746004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 msgr2=0x5619fae90900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:38.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10ee59c640 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 0x5619fae90900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10ee59c640 1 -- 172.21.15.86:0/2850746004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619fab013f0 con 0x5619fae89e50 2024-04-16T05:38:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10eed9d640 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae93350 0x5619fae90e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10ee59c640 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae89e50 0x5619fae903c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f10e00cdb30 tx=0x7f10e009ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bab6e64790c170e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10d5ffb640 1 -- 172.21.15.86:0/2850746004 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0020040 con 0x5619fae89e50 2024-04-16T05:38:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10d5ffb640 1 -- 172.21.15.86:0/2850746004 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f10e00cdd50 con 0x5619fae89e50 2024-04-16T05:38:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 1 -- 172.21.15.86:0/2850746004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d8015070 con 0x5619fae89e50 2024-04-16T05:38:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10d5ffb640 1 -- 172.21.15.86:0/2850746004 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10e00cdf50 con 0x5619fae89e50 2024-04-16T05:38:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 10 client.16668 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.637+0000 7f10f7bce740 1 -- 172.21.15.86:0/2850746004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619faac10f0 con 0x5619fae89e50 2024-04-16T05:38:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10d5ffb640 1 -- 172.21.15.86:0/2850746004 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d8015070 con 0x5619fae89e50 2024-04-16T05:38:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10d5ffb640 1 -- 172.21.15.86:0/2850746004 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f10e00a4b90 con 0x5619fae89e50 2024-04-16T05:38:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10f7bce740 10 client.16668 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10f7bce740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10f7bce740 20 client.16668 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619faea3780 0x5619faea5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10f7bce740 4 client.16668 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10f7bce740 1 -- 172.21.15.86:0/2850746004 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619fadfcba0 con 0x5619faea3780 2024-04-16T05:38:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619faea3780 0x5619faea5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10d5ffb640 10 client.16668 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.638+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619faea3780 0x5619faea5b70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc3fa9b97e7cb86 server_cookie=4a7a49731af0673c in_seq=0 out_seq=0 2024-04-16T05:38:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10d5ffb640 1 -- 172.21.15.86:0/2850746004 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619fadfcba0 con 0x5619faea3780 2024-04-16T05:38:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10d5ffb640 10 client.16668 handle_command_reply: tid=0 2024-04-16T05:38:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 client.16668 shutdown 2024-04-16T05:38:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 -- 172.21.15.86:0/2850746004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae89e50 msgr2=0x5619fae903c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae89e50 0x5619fae903c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f10e00cdb30 tx=0x7f10e009ca20 comp rx=0 tx=0).stop 2024-04-16T05:38:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 -- 172.21.15.86:0/2850746004 shutdown_connections 2024-04-16T05:38:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619faea3780 0x5619faea5b70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae93350 0x5619fae90e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10d5ffb640 0 client.16668 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 0x5619fae90900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2850746004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae89e50 0x5619fae903c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 -- 172.21.15.86:0/2850746004 >> 172.21.15.86:0/2850746004 conn(0x5619fae84580 msgr2=0x5619fae862d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 -- 172.21.15.86:0/2850746004 shutdown_connections 2024-04-16T05:38:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 1 -- 172.21.15.86:0/2850746004 wait complete. 2024-04-16T05:38:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.639+0000 7f10f7bce740 20 client.16668 trim_cache size 0 max 16384 2024-04-16T05:38:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10f7bce740 1 Processor -- start 2024-04-16T05:38:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10f7bce740 1 -- start start 2024-04-16T05:38:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 0x5619fae9f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 0x5619fae90c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae835e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10f7bce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10f01b1650 con 0x5619fae93350 2024-04-16T05:38:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10f7bce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f10f01b3190 con 0x5619fae8a770 2024-04-16T05:38:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10f7bce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10f01b1d20 con 0x5619fae89e50 2024-04-16T05:38:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10eed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae835e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10eed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae835e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39450/0 (socket says 172.21.15.86:39450) 2024-04-16T05:38:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10eed9d640 1 -- 172.21.15.86:0/2693848794 learned_addr learned my addr 172.21.15.86:0/2693848794 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10eed9d640 1 -- 172.21.15.86:0/2693848794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 msgr2=0x5619fae9f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10ee59c640 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 0x5619fae9f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 0x5619fae90c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10eed9d640 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 0x5619fae9f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10eed9d640 1 -- 172.21.15.86:0/2693848794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 msgr2=0x5619fae90c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10eed9d640 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 0x5619fae90c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10eed9d640 1 -- 172.21.15.86:0/2693848794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fae8a5a0 con 0x5619fae93350 2024-04-16T05:38:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10ee59c640 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 0x5619fae9f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 0x5619fae90c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.657+0000 7f10eed9d640 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae835e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f10e8036000 tx=0x7f10e802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f4cfcab31b5918 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10d7fff640 1 -- 172.21.15.86:0/2693848794 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8048040 con 0x5619fae93350 2024-04-16T05:38:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10d7fff640 1 -- 172.21.15.86:0/2693848794 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e8053070 con 0x5619fae93350 2024-04-16T05:38:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10d7fff640 1 -- 172.21.15.86:0/2693848794 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8039070 con 0x5619fae93350 2024-04-16T05:38:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 -- 172.21.15.86:0/2693848794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 msgr2=0x5619fae835e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae835e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f10e8036000 tx=0x7f10e802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 -- 172.21.15.86:0/2693848794 shutdown_connections 2024-04-16T05:38:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae93350 0x5619fae835e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae8a770 0x5619fae90c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 --2- 172.21.15.86:0/2693848794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 0x5619fae9f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 -- 172.21.15.86:0/2693848794 >> 172.21.15.86:0/2693848794 conn(0x5619fae84580 msgr2=0x5619fae82ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 -- 172.21.15.86:0/2693848794 shutdown_connections 2024-04-16T05:38:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 -- 172.21.15.86:0/2693848794 wait complete. 2024-04-16T05:38:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.658+0000 7f10f7bce740 1 -- start start 2024-04-16T05:38:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10f7bce740 1 Processor -- start 2024-04-16T05:38:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 0x5619fae98ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae8a770 0x5619fae99020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10f7bce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae93350 0x5619faf01c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10f7bce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10f01b2ac0 con 0x5619fae8a770 2024-04-16T05:38:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10f7bce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f10f01b23f0 con 0x5619fae93350 2024-04-16T05:38:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10f7bce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10f01af610 con 0x5619fae89e50 2024-04-16T05:38:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10f7bce740 10 client.0 _ll_register_callbacks cb 0x7f10e000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae8a770 0x5619fae99020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae8a770 0x5619fae99020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39466/0 (socket says 172.21.15.86:39466) 2024-04-16T05:38:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10e5d9b640 1 -- 172.21.15.86:0/862085331 learned_addr learned my addr 172.21.15.86:0/862085331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10e5d9b640 1 -- 172.21.15.86:0/862085331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 msgr2=0x5619fae98ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 0x5619fae98ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10e5d9b640 1 -- 172.21.15.86:0/862085331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae93350 msgr2=0x5619faf01c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae93350 0x5619faf01c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10e5d9b640 1 -- 172.21.15.86:0/862085331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619faf022f0 con 0x5619fae8a770 2024-04-16T05:38:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10e5d9b640 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae8a770 0x5619fae99020 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f10d800c010 tx=0x7f10d8005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ce9d98de60473a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10d5ffb640 1 -- 172.21.15.86:0/862085331 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d802e020 con 0x5619fae8a770 2024-04-16T05:38:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.659+0000 7f10d5ffb640 1 -- 172.21.15.86:0/862085331 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f10d8003dd0 con 0x5619fae8a770 2024-04-16T05:38:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10f7bce740 1 -- 172.21.15.86:0/862085331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10e8053070 con 0x5619fae8a770 2024-04-16T05:38:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10d5ffb640 1 -- 172.21.15.86:0/862085331 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10d8039050 con 0x5619fae8a770 2024-04-16T05:38:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10f7bce740 10 client.16674 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10f7bce740 1 -- 172.21.15.86:0/862085331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619faf09740 con 0x5619fae8a770 2024-04-16T05:38:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10d5ffb640 1 -- 172.21.15.86:0/862085331 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e8053070 con 0x5619fae8a770 2024-04-16T05:38:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10d5ffb640 1 -- 172.21.15.86:0/862085331 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f10d8011740 con 0x5619fae8a770 2024-04-16T05:38:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10f7bce740 10 client.16674 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10f7bce740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10f7bce740 20 client.16674 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10f7bce740 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619faf09c90 0x5619faf0c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10f7bce740 4 client.16674 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:38:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10f7bce740 1 -- 172.21.15.86:0/862085331 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619fadf6a10 con 0x5619faf09c90 2024-04-16T05:38:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.660+0000 7f10ee59c640 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619faf09c90 0x5619faf0c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10d5ffb640 10 client.16674 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10ee59c640 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619faf09c90 0x5619faf0c0b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea21787a4e57038f server_cookie=65ec76a97714ba4a in_seq=0 out_seq=0 2024-04-16T05:38:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10d5ffb640 1 -- 172.21.15.86:0/862085331 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5619fadf6a10 con 0x5619faf09c90 2024-04-16T05:38:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10d5ffb640 10 client.16674 handle_command_reply: tid=0 2024-04-16T05:38:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 client.16674 shutdown 2024-04-16T05:38:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 -- 172.21.15.86:0/862085331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae8a770 msgr2=0x5619fae99020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae8a770 0x5619fae99020 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f10d800c010 tx=0x7f10d8005bf0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 -- 172.21.15.86:0/862085331 shutdown_connections 2024-04-16T05:38:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619faf09c90 0x5619faf0c0b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619fae93350 0x5619faf01c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619fae8a770 0x5619fae99020 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10d5ffb640 0 client.16674 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 --2- 172.21.15.86:0/862085331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619fae89e50 0x5619fae98ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 -- 172.21.15.86:0/862085331 >> 172.21.15.86:0/862085331 conn(0x5619fae84580 msgr2=0x5619fae83760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 -- 172.21.15.86:0/862085331 shutdown_connections 2024-04-16T05:38:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 1 -- 172.21.15.86:0/862085331 wait complete. 2024-04-16T05:38:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.661+0000 7f10f7bce740 20 client.16674 trim_cache size 0 max 16384 2024-04-16T05:38:38.670 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:38:38.670 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:38:38.670 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:38:38.670 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:38:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 -- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f10dc0a3d50 msgr2=0x7f10dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f10dc0a3d50 0x7f10dc0a6210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f10e8032530 tx=0x7f10e8032d70 comp rx=0 tx=0).stop 2024-04-16T05:38:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 -- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101250 msgr2=0x7f10f0102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101250 0x7f10f0102c00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f10e000d950 tx=0x7f10e000de10 comp rx=0 tx=0).stop 2024-04-16T05:38:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 -- 172.21.15.86:0/1686480241 shutdown_connections 2024-04-16T05:38:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f10dc0a3d50 0x7f10dc0a6210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10f01070e0 0x7f10f0103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f0101bf0 0x7f10f0103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 --2- 172.21.15.86:0/1686480241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f0101250 0x7f10f0102c00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 -- 172.21.15.86:0/1686480241 >> 172.21.15.86:0/1686480241 conn(0x7f10f0076d00 msgr2=0x7f10f0100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 -- 172.21.15.86:0/1686480241 shutdown_connections 2024-04-16T05:38:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.662+0000 7f10f4fe2640 1 -- 172.21.15.86:0/1686480241 wait complete. 2024-04-16T05:38:38.673 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:38:38.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:38:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.780+0000 7f7edd87e640 1 -- 172.21.15.86:0/4179043304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 msgr2=0x7f7ed81026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.780+0000 7f7edd87e640 1 --2- 172.21.15.86:0/4179043304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 0x7f7ed81026d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009ca0 tx=0x7f7ec802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.780+0000 7f7edd87e640 1 -- 172.21.15.86:0/4179043304 shutdown_connections 2024-04-16T05:38:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.780+0000 7f7edd87e640 1 --2- 172.21.15.86:0/4179043304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8107330 0x7f7ed810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.780+0000 7f7edd87e640 1 --2- 172.21.15.86:0/4179043304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed8102ca0 0x7f7ed8106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.780+0000 7f7edd87e640 1 --2- 172.21.15.86:0/4179043304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 0x7f7ed81026d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.780+0000 7f7edd87e640 1 -- 172.21.15.86:0/4179043304 >> 172.21.15.86:0/4179043304 conn(0x7f7ed8079190 msgr2=0x7f7ed80795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.780+0000 7f7edd87e640 1 -- 172.21.15.86:0/4179043304 shutdown_connections 2024-04-16T05:38:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.781+0000 7f7edd87e640 1 -- 172.21.15.86:0/4179043304 wait complete. 2024-04-16T05:38:38.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.781+0000 7f7edd87e640 1 Processor -- start 2024-04-16T05:38:38.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.781+0000 7f7edd87e640 1 -- start start 2024-04-16T05:38:38.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.781+0000 7f7edd87e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 0x7f7ed819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.781+0000 7f7edd87e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed8107330 0x7f7ed819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.781+0000 7f7edd87e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8198710 0x7f7ed8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.781+0000 7f7edd87e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8112410 con 0x7f7ed81022f0 2024-04-16T05:38:38.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.781+0000 7f7edd87e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8112290 con 0x7f7ed8107330 2024-04-16T05:38:38.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.781+0000 7f7edd87e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8112590 con 0x7f7ed8198710 2024-04-16T05:38:38.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 0x7f7ed819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 0x7f7ed819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39494/0 (socket says 172.21.15.86:39494) 2024-04-16T05:38:38.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed6ffd640 1 -- 172.21.15.86:0/791419476 learned_addr learned my addr 172.21.15.86:0/791419476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:38.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed77fe640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8198710 0x7f7ed8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed67fc640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed8107330 0x7f7ed819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed6ffd640 1 -- 172.21.15.86:0/791419476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8198710 msgr2=0x7f7ed8198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed6ffd640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8198710 0x7f7ed8198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed6ffd640 1 -- 172.21.15.86:0/791419476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed8107330 msgr2=0x7f7ed819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:38.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed6ffd640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed8107330 0x7f7ed819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed6ffd640 1 -- 172.21.15.86:0/791419476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed8199310 con 0x7f7ed81022f0 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed77fe640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8198710 0x7f7ed8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7ed6ffd640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 0x7f7ed819e650 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8005e30 tx=0x7f7ec8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7edc87c640 1 -- 172.21.15.86:0/791419476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8039040 con 0x7f7ed81022f0 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed81995a0 con 0x7f7ed81022f0 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7edc87c640 1 -- 172.21.15.86:0/791419476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8044070 con 0x7f7ed81022f0 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7edc87c640 1 -- 172.21.15.86:0/791419476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80042a0 con 0x7f7ed81022f0 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.782+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81a5730 con 0x7f7ed81022f0 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.783+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed8103730 con 0x7f7ed81022f0 2024-04-16T05:38:38.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.784+0000 7f7edc87c640 1 -- 172.21.15.86:0/791419476 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ec8004440 con 0x7f7ed81022f0 2024-04-16T05:38:38.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.785+0000 7f7edc87c640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7ec40a3d20 0x7f7ec40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:38.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.785+0000 7f7ed67fc640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7ec40a3d20 0x7f7ec40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:38.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.785+0000 7f7edc87c640 1 -- 172.21.15.86:0/791419476 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7ec80f2e80 con 0x7f7ed81022f0 2024-04-16T05:38:38.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.785+0000 7f7ed67fc640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7ec40a3d20 0x7f7ec40a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec0005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:38.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:38.789+0000 7f7edc87c640 1 -- 172.21.15.86:0/791419476 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ec80bf2c0 con 0x7f7ed81022f0 2024-04-16T05:38:38.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:38 smithi086 ceph-mon[32028]: pgmap v103: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:38.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:38 smithi196 ceph-mon[36373]: pgmap v103: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.006+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed810a530 con 0x7f7ed81022f0 2024-04-16T05:38:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.006+0000 7f7edc87c640 1 -- 172.21.15.86:0/791419476 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f7ec80bebc0 con 0x7f7ed81022f0 2024-04-16T05:38:39.005 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:38:39.005 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:38:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:38:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7ec40a3d20 msgr2=0x7f7ec40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7ec40a3d20 0x7f7ec40a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec0005850 comp rx=0 tx=0).stop 2024-04-16T05:38:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 msgr2=0x7f7ed819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 0x7f7ed819e650 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8005e30 tx=0x7f7ec8032f20 comp rx=0 tx=0).stop 2024-04-16T05:38:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 shutdown_connections 2024-04-16T05:38:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8198710 0x7f7ed8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f7ec40a3d20 0x7f7ec40a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed8107330 0x7f7ed819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 --2- 172.21.15.86:0/791419476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed81022f0 0x7f7ed819e650 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.008+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 >> 172.21.15.86:0/791419476 conn(0x7f7ed8079190 msgr2=0x7f7ed81003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.009+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 shutdown_connections 2024-04-16T05:38:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.009+0000 7f7edd87e640 1 -- 172.21.15.86:0/791419476 wait complete. 2024-04-16T05:38:39.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:38:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.126+0000 7fa196d1c640 1 -- 172.21.15.86:0/4228517197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 msgr2=0x7fa19010dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.126+0000 7fa196d1c640 1 --2- 172.21.15.86:0/4228517197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 0x7fa19010dcf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa188009ca0 tx=0x7fa18802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.126+0000 7fa196d1c640 1 -- 172.21.15.86:0/4228517197 shutdown_connections 2024-04-16T05:38:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.126+0000 7fa196d1c640 1 --2- 172.21.15.86:0/4228517197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 0x7fa19010dcf0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.126+0000 7fa196d1c640 1 --2- 172.21.15.86:0/4228517197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa190101250 0x7fa1901016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.126+0000 7fa196d1c640 1 --2- 172.21.15.86:0/4228517197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa190107040 0x7fa190107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.126+0000 7fa196d1c640 1 -- 172.21.15.86:0/4228517197 >> 172.21.15.86:0/4228517197 conn(0x7fa190076d00 msgr2=0x7fa190077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.126+0000 7fa196d1c640 1 -- 172.21.15.86:0/4228517197 shutdown_connections 2024-04-16T05:38:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.126+0000 7fa196d1c640 1 -- 172.21.15.86:0/4228517197 wait complete. 2024-04-16T05:38:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa196d1c640 1 Processor -- start 2024-04-16T05:38:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa196d1c640 1 -- start start 2024-04-16T05:38:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa196d1c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa190101250 0x7fa1901a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa196d1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 0x7fa1901a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa196d1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa190107040 0x7fa19019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa196d1c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901112f0 con 0x7fa190101bf0 2024-04-16T05:38:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa196d1c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa190111170 con 0x7fa190101250 2024-04-16T05:38:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa196d1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa190111470 con 0x7fa190107040 2024-04-16T05:38:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 0x7fa1901a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 0x7fa1901a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39510/0 (socket says 172.21.15.86:39510) 2024-04-16T05:38:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa18ffff640 1 -- 172.21.15.86:0/2213185909 learned_addr learned my addr 172.21.15.86:0/2213185909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.127+0000 7fa194ab8640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa190101250 0x7fa1901a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa1952b9640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa190107040 0x7fa19019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa18ffff640 1 -- 172.21.15.86:0/2213185909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa190107040 msgr2=0x7fa19019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa18ffff640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa190107040 0x7fa19019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa18ffff640 1 -- 172.21.15.86:0/2213185909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa190101250 msgr2=0x7fa1901a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa18ffff640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa190101250 0x7fa1901a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa18ffff640 1 -- 172.21.15.86:0/2213185909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19019b010 con 0x7fa190101bf0 2024-04-16T05:38:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa18ffff640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 0x7fa1901a0c20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa18400d770 tx=0x7fa18400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa18dffb640 1 -- 172.21.15.86:0/2213185909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184017070 con 0x7fa190101bf0 2024-04-16T05:38:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa196d1c640 1 -- 172.21.15.86:0/2213185909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19019b300 con 0x7fa190101bf0 2024-04-16T05:38:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa18dffb640 1 -- 172.21.15.86:0/2213185909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa184005500 con 0x7fa190101bf0 2024-04-16T05:38:39.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa18dffb640 1 -- 172.21.15.86:0/2213185909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184004e00 con 0x7fa190101bf0 2024-04-16T05:38:39.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.128+0000 7fa196d1c640 1 -- 172.21.15.86:0/2213185909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1901a75a0 con 0x7fa190101bf0 2024-04-16T05:38:39.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.130+0000 7fa18dffb640 1 -- 172.21.15.86:0/2213185909 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa184004fa0 con 0x7fa190101bf0 2024-04-16T05:38:39.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1999e9740 1 Processor -- start 2024-04-16T05:38:39.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa18dffb640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa1780a3d20 0x7fa1780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1999e9740 1 -- start start 2024-04-16T05:38:39.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1952b9640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa1780a3d20 0x7fa1780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 0x5612f0427240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0430360 0x5612f0432750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1999e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612f038c0e0 con 0x5612f0427780 2024-04-16T05:38:39.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1999e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612f0329520 con 0x5612f0426e60 2024-04-16T05:38:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1999e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612f03880a0 con 0x5612f0430360 2024-04-16T05:38:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa18dffb640 1 -- 172.21.15.86:0/2213185909 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa1840d05f0 con 0x7fa190101bf0 2024-04-16T05:38:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39512/0 (socket says 172.21.15.86:39512) 2024-04-16T05:38:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa18ffff640 1 -- 172.21.15.86:0/3670713528 learned_addr learned my addr 172.21.15.86:0/3670713528 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa1952b9640 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0430360 0x5612f0432750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa18ffff640 1 -- 172.21.15.86:0/3670713528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0430360 msgr2=0x5612f0432750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.131+0000 7fa194ab8640 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 0x5612f0427240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa18ffff640 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0430360 0x5612f0432750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa18ffff640 1 -- 172.21.15.86:0/3670713528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 msgr2=0x5612f0427240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa18ffff640 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 0x5612f0427240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa18ffff640 1 -- 172.21.15.86:0/3670713528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f03518d0 con 0x5612f0427780 2024-04-16T05:38:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa1952b9640 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0430360 0x5612f0432750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa1952b9640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa1780a3d20 0x7fa1780a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa188005e30 tx=0x7fa1880098a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa18ffff640 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042fe20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa18400d740 tx=0x7fa18409c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc8ef9e04b68038 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa176ffd640 1 -- 172.21.15.86:0/3670713528 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184020040 con 0x5612f0427780 2024-04-16T05:38:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa176ffd640 1 -- 172.21.15.86:0/3670713528 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1840a4e90 con 0x5612f0427780 2024-04-16T05:38:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa176ffd640 1 -- 172.21.15.86:0/3670713528 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840cde50 con 0x5612f0427780 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa1999e9740 1 -- 172.21.15.86:0/3670713528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 msgr2=0x5612f042fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.132+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042fe20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa18400d740 tx=0x7fa18409c920 comp rx=0 tx=0).stop 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 -- 172.21.15.86:0/3670713528 shutdown_connections 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0430360 0x5612f0432750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042fe20 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3670713528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 0x5612f0427240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 -- 172.21.15.86:0/3670713528 >> 172.21.15.86:0/3670713528 conn(0x5612f0421590 msgr2=0x5612f0421970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 -- 172.21.15.86:0/3670713528 shutdown_connections 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 -- 172.21.15.86:0/3670713528 wait complete. 2024-04-16T05:38:39.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 -- start start 2024-04-16T05:38:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 Processor -- start 2024-04-16T05:38:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0426e60 0x5612f042d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0430360 0x5612f042def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612f0432fa0 con 0x5612f0427780 2024-04-16T05:38:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612f03454d0 con 0x5612f0430360 2024-04-16T05:38:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612f0433120 con 0x5612f0426e60 2024-04-16T05:38:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa1999e9740 10 client.0 _ll_register_callbacks cb 0x5612f032a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39526/0 (socket says 172.21.15.86:39526) 2024-04-16T05:38:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.133+0000 7fa18ffff640 1 -- 172.21.15.86:0/3389808946 learned_addr learned my addr 172.21.15.86:0/3389808946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa194ab8640 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0426e60 0x5612f042d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa1952b9640 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0430360 0x5612f042def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa18ffff640 1 -- 172.21.15.86:0/3389808946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0426e60 msgr2=0x5612f042d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa18ffff640 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0426e60 0x5612f042d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa18ffff640 1 -- 172.21.15.86:0/3389808946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0430360 msgr2=0x5612f042def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa18ffff640 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0430360 0x5612f042def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa18ffff640 1 -- 172.21.15.86:0/3389808946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f009e3f0 con 0x5612f0427780 2024-04-16T05:38:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa18ffff640 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042d9b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa184012040 tx=0x7fa184009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299eef345c59b0ab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa16ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa16ffff640 1 -- 172.21.15.86:0/3389808946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184080060 con 0x5612f0427780 2024-04-16T05:38:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa16ffff640 1 -- 172.21.15.86:0/3389808946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa1840d0240 con 0x5612f0427780 2024-04-16T05:38:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa1999e9740 1 -- 172.21.15.86:0/3389808946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1840a4e90 con 0x5612f0427780 2024-04-16T05:38:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa16ffff640 1 -- 172.21.15.86:0/3389808946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1840bb070 con 0x5612f0427780 2024-04-16T05:38:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa1999e9740 10 client.16692 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa1999e9740 1 -- 172.21.15.86:0/3389808946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612f01687d0 con 0x5612f0427780 2024-04-16T05:38:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa16ffff640 1 -- 172.21.15.86:0/3389808946 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1840a4e90 con 0x5612f0427780 2024-04-16T05:38:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa16ffff640 1 -- 172.21.15.86:0/3389808946 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa1840a0120 con 0x5612f0427780 2024-04-16T05:38:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa1999e9740 10 client.16692 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.134+0000 7fa1999e9740 10 client.16692 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:38:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.135+0000 7fa1999e9740 20 client.16692 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.135+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5612f0440170 0x5612f0442560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.135+0000 7fa1999e9740 4 client.16692 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.135+0000 7fa1999e9740 1 -- 172.21.15.86:0/3389808946 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612f0399b00 con 0x5612f0440170 2024-04-16T05:38:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.135+0000 7fa194ab8640 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5612f0440170 0x5612f0442560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.135+0000 7fa16ffff640 10 client.16692 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:38:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.135+0000 7fa194ab8640 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5612f0440170 0x5612f0442560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1fa8fefca1558e43 server_cookie=d94076de4979a727 in_seq=0 out_seq=0 2024-04-16T05:38:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa16ffff640 1 -- 172.21.15.86:0/3389808946 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612f0399b00 con 0x5612f0440170 2024-04-16T05:38:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa16ffff640 10 client.16692 handle_command_reply: tid=0 2024-04-16T05:38:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 client.16692 shutdown 2024-04-16T05:38:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 -- 172.21.15.86:0/3389808946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 msgr2=0x5612f042d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042d9b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa184012040 tx=0x7fa184009c30 comp rx=0 tx=0).stop 2024-04-16T05:38:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 -- 172.21.15.86:0/3389808946 shutdown_connections 2024-04-16T05:38:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5612f0440170 0x5612f0442560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0430360 0x5612f042def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa16ffff640 0 client.16692 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:38:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0427780 0x5612f042d9b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3389808946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0426e60 0x5612f042d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 -- 172.21.15.86:0/3389808946 >> 172.21.15.86:0/3389808946 conn(0x5612f0421590 msgr2=0x5612f04232e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 -- 172.21.15.86:0/3389808946 shutdown_connections 2024-04-16T05:38:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 1 -- 172.21.15.86:0/3389808946 wait complete. 2024-04-16T05:38:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.136+0000 7fa1999e9740 20 client.16692 trim_cache size 0 max 16384 2024-04-16T05:38:39.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.149+0000 7fa1999e9740 1 Processor -- start 2024-04-16T05:38:39.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.149+0000 7fa1999e9740 1 -- start start 2024-04-16T05:38:39.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.149+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0426e60 0x5612f042dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.149+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 0x5612f0447b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0430360 0x5612f04480d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa1999e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901b15a0 con 0x5612f0426e60 2024-04-16T05:38:39.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa1999e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901b30e0 con 0x5612f0430360 2024-04-16T05:38:39.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa1999e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901b1c70 con 0x5612f0427780 2024-04-16T05:38:39.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa194ab8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0426e60 0x5612f042dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa194ab8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0426e60 0x5612f042dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39536/0 (socket says 172.21.15.86:39536) 2024-04-16T05:38:39.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa194ab8640 1 -- 172.21.15.86:0/3291445479 learned_addr learned my addr 172.21.15.86:0/3291445479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa194ab8640 1 -- 172.21.15.86:0/3291445479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 msgr2=0x5612f0447b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa194ab8640 1 --2- 172.21.15.86:0/3291445479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 0x5612f0447b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa194ab8640 1 -- 172.21.15.86:0/3291445479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0430360 msgr2=0x5612f04480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa194ab8640 1 --2- 172.21.15.86:0/3291445479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0430360 0x5612f04480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa194ab8640 1 -- 172.21.15.86:0/3291445479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f041bb40 con 0x5612f0426e60 2024-04-16T05:38:39.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa194ab8640 1 --2- 172.21.15.86:0/3291445479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0426e60 0x5612f042dd10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa180004590 tx=0x7fa18000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600abed9c2c62221 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa1767fc640 1 -- 172.21.15.86:0/3291445479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18001f070 con 0x5612f0426e60 2024-04-16T05:38:39.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa1767fc640 1 -- 172.21.15.86:0/3291445479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa18001bdc0 con 0x5612f0426e60 2024-04-16T05:38:39.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.150+0000 7fa1767fc640 1 -- 172.21.15.86:0/3291445479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1800126b0 con 0x5612f0426e60 2024-04-16T05:38:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 -- 172.21.15.86:0/3291445479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0426e60 msgr2=0x5612f042dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3291445479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0426e60 0x5612f042dd10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa180004590 tx=0x7fa18000b390 comp rx=0 tx=0).stop 2024-04-16T05:38:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 -- 172.21.15.86:0/3291445479 shutdown_connections 2024-04-16T05:38:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3291445479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0430360 0x5612f04480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3291445479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 0x5612f0447b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 --2- 172.21.15.86:0/3291445479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0426e60 0x5612f042dd10 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 -- 172.21.15.86:0/3291445479 >> 172.21.15.86:0/3291445479 conn(0x5612f0421590 msgr2=0x5612f0420730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 -- 172.21.15.86:0/3291445479 shutdown_connections 2024-04-16T05:38:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 -- 172.21.15.86:0/3291445479 wait complete. 2024-04-16T05:38:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 -- start start 2024-04-16T05:38:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.151+0000 7fa1999e9740 1 Processor -- start 2024-04-16T05:38:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 0x5612f04388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 0x5612f0433c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1999e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0430360 0x5612f04341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1999e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901b2a10 con 0x5612f0430360 2024-04-16T05:38:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1999e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901b2340 con 0x5612f0426e60 2024-04-16T05:38:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1999e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901af560 con 0x5612f0427780 2024-04-16T05:38:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1999e9740 10 client.0 _ll_register_callbacks cb 0x7fa18409be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1952b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0430360 0x5612f04341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1952b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0430360 0x5612f04341c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39550/0 (socket says 172.21.15.86:39550) 2024-04-16T05:38:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1952b9640 1 -- 172.21.15.86:0/195694292 learned_addr learned my addr 172.21.15.86:0/195694292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa18ffff640 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 0x5612f0433c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa194ab8640 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 0x5612f04388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1952b9640 1 -- 172.21.15.86:0/195694292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 msgr2=0x5612f0433c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1952b9640 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 0x5612f0433c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1952b9640 1 -- 172.21.15.86:0/195694292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 msgr2=0x5612f04388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1952b9640 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 0x5612f04388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1952b9640 1 -- 172.21.15.86:0/195694292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f04349c0 con 0x5612f0430360 2024-04-16T05:38:39.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa194ab8640 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 0x5612f04388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa18ffff640 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 0x5612f0433c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1952b9640 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0430360 0x5612f04341c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa188009ca0 tx=0x7fa1880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57dc48ef1f35967 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa16ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa16ffff640 1 -- 172.21.15.86:0/195694292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188039040 con 0x5612f0430360 2024-04-16T05:38:39.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa16ffff640 1 -- 172.21.15.86:0/195694292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa188003b80 con 0x5612f0430360 2024-04-16T05:38:39.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.152+0000 7fa1999e9740 1 -- 172.21.15.86:0/195694292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa18001bdc0 con 0x5612f0430360 2024-04-16T05:38:39.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa16ffff640 1 -- 172.21.15.86:0/195694292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa188003d80 con 0x5612f0430360 2024-04-16T05:38:39.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa1999e9740 10 client.16698 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa1999e9740 1 -- 172.21.15.86:0/195694292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612f04a6430 con 0x5612f0430360 2024-04-16T05:38:39.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa16ffff640 1 -- 172.21.15.86:0/195694292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa18001bdc0 con 0x5612f0430360 2024-04-16T05:38:39.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa16ffff640 1 -- 172.21.15.86:0/195694292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa18803f070 con 0x5612f0430360 2024-04-16T05:38:39.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa1999e9740 10 client.16698 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa1999e9740 10 client.16698 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:38:39.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa1999e9740 20 client.16698 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa1999e9740 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5612f04a6c30 0x5612f04a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa1999e9740 4 client.16698 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:39.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa1999e9740 1 -- 172.21.15.86:0/195694292 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612f0393970 con 0x5612f04a6c30 2024-04-16T05:38:39.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.153+0000 7fa194ab8640 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5612f04a6c30 0x5612f04a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa194ab8640 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5612f04a6c30 0x5612f04a9050 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d92d78a90be6cb1c server_cookie=684e39892e0f6fd6 in_seq=0 out_seq=0 2024-04-16T05:38:39.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa16ffff640 10 client.16698 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:38:39.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa16ffff640 1 -- 172.21.15.86:0/195694292 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612f0393970 con 0x5612f04a6c30 2024-04-16T05:38:39.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa16ffff640 10 client.16698 handle_command_reply: tid=0 2024-04-16T05:38:39.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 client.16698 shutdown 2024-04-16T05:38:39.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 -- 172.21.15.86:0/195694292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0430360 msgr2=0x5612f04341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0430360 0x5612f04341c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa188009ca0 tx=0x7fa1880095c0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 -- 172.21.15.86:0/195694292 shutdown_connections 2024-04-16T05:38:39.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5612f04a6c30 0x5612f04a9050 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612f0430360 0x5612f04341c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612f0427780 0x5612f0433c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa16ffff640 0 client.16698 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:38:39.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 --2- 172.21.15.86:0/195694292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f0426e60 0x5612f04388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 -- 172.21.15.86:0/195694292 >> 172.21.15.86:0/195694292 conn(0x5612f0421590 msgr2=0x5612f042ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 -- 172.21.15.86:0/195694292 shutdown_connections 2024-04-16T05:38:39.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 1 -- 172.21.15.86:0/195694292 wait complete. 2024-04-16T05:38:39.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.154+0000 7fa1999e9740 20 client.16698 trim_cache size 0 max 16384 2024-04-16T05:38:39.164 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:39.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 -- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa1780a3d20 msgr2=0x7fa1780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa1780a3d20 0x7fa1780a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa188005e30 tx=0x7fa1880098a0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 -- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 msgr2=0x7fa1901a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 0x7fa1901a0c20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa18400d770 tx=0x7fa18400dc30 comp rx=0 tx=0).stop 2024-04-16T05:38:39.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 -- 172.21.15.86:0/2213185909 shutdown_connections 2024-04-16T05:38:39.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa190107040 0x7fa19019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190101bf0 0x7fa1901a0c20 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa1780a3d20 0x7fa1780a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 --2- 172.21.15.86:0/2213185909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa190101250 0x7fa1901a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 -- 172.21.15.86:0/2213185909 >> 172.21.15.86:0/2213185909 conn(0x7fa190076d00 msgr2=0x7fa19006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 -- 172.21.15.86:0/2213185909 shutdown_connections 2024-04-16T05:38:39.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.155+0000 7fa196d1c640 1 -- 172.21.15.86:0/2213185909 wait complete. 2024-04-16T05:38:39.167 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:38:39.168 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:38 smithi194 ceph-mon[35865]: pgmap v103: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:39.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.275+0000 7f73eca6a640 1 -- 172.21.15.86:0/91931358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 msgr2=0x7f73e8101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.275+0000 7f73eca6a640 1 --2- 172.21.15.86:0/91931358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 0x7f73e8101510 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009900 tx=0x7f73d802bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:39.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.275+0000 7f73eca6a640 1 -- 172.21.15.86:0/91931358 shutdown_connections 2024-04-16T05:38:39.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.275+0000 7f73eca6a640 1 --2- 172.21.15.86:0/91931358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e8106200 0x7f73e810ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.275+0000 7f73eca6a640 1 --2- 172.21.15.86:0/91931358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73e8101ae0 0x7f73e8105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.275+0000 7f73eca6a640 1 --2- 172.21.15.86:0/91931358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 0x7f73e8101510 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.275+0000 7f73eca6a640 1 -- 172.21.15.86:0/91931358 >> 172.21.15.86:0/91931358 conn(0x7f73e80780b0 msgr2=0x7f73e80784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.275+0000 7f73eca6a640 1 -- 172.21.15.86:0/91931358 shutdown_connections 2024-04-16T05:38:39.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.275+0000 7f73eca6a640 1 -- 172.21.15.86:0/91931358 wait complete. 2024-04-16T05:38:39.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73eca6a640 1 Processor -- start 2024-04-16T05:38:39.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73eca6a640 1 -- start start 2024-04-16T05:38:39.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73eca6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 0x7f73e81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73eca6a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73e8106200 0x7f73e8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73eca6a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e819a1b0 0x7f73e81a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73eca6a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81112c0 con 0x7f73e8101130 2024-04-16T05:38:39.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73eca6a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8111140 con 0x7f73e8106200 2024-04-16T05:38:39.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73eca6a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8111440 con 0x7f73e819a1b0 2024-04-16T05:38:39.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 0x7f73e81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 0x7f73e81996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39558/0 (socket says 172.21.15.86:39558) 2024-04-16T05:38:39.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e659c640 1 -- 172.21.15.86:0/2518824022 learned_addr learned my addr 172.21.15.86:0/2518824022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e659c640 1 -- 172.21.15.86:0/2518824022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e819a1b0 msgr2=0x7f73e81a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e659c640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e819a1b0 0x7f73e81a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e5d9b640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73e8106200 0x7f73e8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e659c640 1 -- 172.21.15.86:0/2518824022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73e8106200 msgr2=0x7f73e8199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e659c640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73e8106200 0x7f73e8199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e659c640 1 -- 172.21.15.86:0/2518824022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e81a4f30 con 0x7f73e8101130 2024-04-16T05:38:39.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73e659c640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 0x7f73e81996e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006010 tx=0x7f73d802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.276+0000 7f73df7fe640 1 -- 172.21.15.86:0/2518824022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8041070 con 0x7f73e8101130 2024-04-16T05:38:39.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.277+0000 7f73eca6a640 1 -- 172.21.15.86:0/2518824022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e81a51c0 con 0x7f73e8101130 2024-04-16T05:38:39.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.277+0000 7f73df7fe640 1 -- 172.21.15.86:0/2518824022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d8004450 con 0x7f73e8101130 2024-04-16T05:38:39.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.277+0000 7f73df7fe640 1 -- 172.21.15.86:0/2518824022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8003e00 con 0x7f73e8101130 2024-04-16T05:38:39.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.277+0000 7f73eca6a640 1 -- 172.21.15.86:0/2518824022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e81a56b0 con 0x7f73e8101130 2024-04-16T05:38:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.279+0000 7f73df7fe640 1 -- 172.21.15.86:0/2518824022 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73d8044070 con 0x7f73e8101130 2024-04-16T05:38:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.279+0000 7f73ef658740 1 Processor -- start 2024-04-16T05:38:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.279+0000 7f73ef658740 1 -- start start 2024-04-16T05:38:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.279+0000 7f73df7fe640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f73d40a3d20 0x7f73d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73ef658740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa314e60 0x55b9aa315240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73ef658740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73ef658740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa31e360 0x55b9aa320750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73ef658740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b9aa27a0e0 con 0x55b9aa315780 2024-04-16T05:38:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73ef658740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b9aa217520 con 0x55b9aa314e60 2024-04-16T05:38:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73ef658740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b9aa2760a0 con 0x55b9aa31e360 2024-04-16T05:38:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e6d9d640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f73d40a3d20 0x7f73d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73df7fe640 1 -- 172.21.15.86:0/2518824022 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f73d80f2730 con 0x7f73e8101130 2024-04-16T05:38:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39574/0 (socket says 172.21.15.86:39574) 2024-04-16T05:38:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e5d9b640 1 -- 172.21.15.86:0/3049757740 learned_addr learned my addr 172.21.15.86:0/3049757740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e659c640 1 --2- 172.21.15.86:0/3049757740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa314e60 0x55b9aa315240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e6d9d640 1 --2- 172.21.15.86:0/3049757740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa31e360 0x55b9aa320750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e5d9b640 1 -- 172.21.15.86:0/3049757740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa31e360 msgr2=0x55b9aa320750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e5d9b640 1 --2- 172.21.15.86:0/3049757740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa31e360 0x55b9aa320750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e5d9b640 1 -- 172.21.15.86:0/3049757740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa314e60 msgr2=0x55b9aa315240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e5d9b640 1 --2- 172.21.15.86:0/3049757740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa314e60 0x55b9aa315240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e5d9b640 1 -- 172.21.15.86:0/3049757740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9aa23f8d0 con 0x55b9aa315780 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e6d9d640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f73d40a3d20 0x7f73d40a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f73e0005c40 tx=0x7f73e0015900 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73e5d9b640 1 --2- 172.21.15.86:0/3049757740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31de20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009bf0 tx=0x7f73d0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10fd9868ae81b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73dcff9640 1 -- 172.21.15.86:0/3049757740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0004020 con 0x55b9aa315780 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73dcff9640 1 -- 172.21.15.86:0/3049757740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d00041c0 con 0x55b9aa315780 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.280+0000 7f73dcff9640 1 -- 172.21.15.86:0/3049757740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d00126b0 con 0x55b9aa315780 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.281+0000 7f73ef658740 1 -- 172.21.15.86:0/3049757740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 msgr2=0x55b9aa31de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.281+0000 7f73ef658740 1 --2- 172.21.15.86:0/3049757740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31de20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009bf0 tx=0x7f73d0010740 comp rx=0 tx=0).stop 2024-04-16T05:38:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.281+0000 7f73ef658740 1 -- 172.21.15.86:0/3049757740 shutdown_connections 2024-04-16T05:38:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.281+0000 7f73ef658740 1 --2- 172.21.15.86:0/3049757740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa31e360 0x55b9aa320750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.281+0000 7f73ef658740 1 --2- 172.21.15.86:0/3049757740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31de20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.281+0000 7f73ef658740 1 --2- 172.21.15.86:0/3049757740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa314e60 0x55b9aa315240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.281+0000 7f73ef658740 1 -- 172.21.15.86:0/3049757740 >> 172.21.15.86:0/3049757740 conn(0x55b9aa30f590 msgr2=0x55b9aa30f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.281+0000 7f73ef658740 1 -- 172.21.15.86:0/3049757740 shutdown_connections 2024-04-16T05:38:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.281+0000 7f73ef658740 1 -- 172.21.15.86:0/3049757740 wait complete. 2024-04-16T05:38:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73ef658740 1 -- start start 2024-04-16T05:38:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73ef658740 1 Processor -- start 2024-04-16T05:38:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73ef658740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 0x55b9aa31b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73ef658740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73ef658740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa31e360 0x55b9aa31be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73ef658740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b9aa321060 con 0x55b9aa315780 2024-04-16T05:38:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73ef658740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b9aa2334d0 con 0x55b9aa31e360 2024-04-16T05:38:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73ef658740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b9aa3211e0 con 0x55b9aa314e60 2024-04-16T05:38:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73ef658740 10 client.0 _ll_register_callbacks cb 0x55b9aa2182c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39580/0 (socket says 172.21.15.86:39580) 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e5d9b640 1 -- 172.21.15.86:0/988398941 learned_addr learned my addr 172.21.15.86:0/988398941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e659c640 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 0x55b9aa31b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e6d9d640 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa31e360 0x55b9aa31be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e5d9b640 1 -- 172.21.15.86:0/988398941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 msgr2=0x55b9aa31b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e5d9b640 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 0x55b9aa31b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e5d9b640 1 -- 172.21.15.86:0/988398941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa31e360 msgr2=0x55b9aa31be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e5d9b640 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa31e360 0x55b9aa31be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e5d9b640 1 -- 172.21.15.86:0/988398941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a9f8c3f0 con 0x55b9aa315780 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.282+0000 7f73e659c640 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 0x55b9aa31b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73e5d9b640 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31b940 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f73d0003df0 tx=0x7f73d00045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cae69a20a7adb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73cdffb640 1 -- 172.21.15.86:0/988398941 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d00035f0 con 0x55b9aa315780 2024-04-16T05:38:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73cdffb640 1 -- 172.21.15.86:0/988398941 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f73d0003790 con 0x55b9aa315780 2024-04-16T05:38:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73ef658740 1 -- 172.21.15.86:0/988398941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d00041c0 con 0x55b9aa315780 2024-04-16T05:38:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73cdffb640 1 -- 172.21.15.86:0/988398941 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f73d0031050 con 0x55b9aa315780 2024-04-16T05:38:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73ef658740 10 client.16710 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73ef658740 1 -- 172.21.15.86:0/988398941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9aa0567d0 con 0x55b9aa315780 2024-04-16T05:38:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73cdffb640 1 -- 172.21.15.86:0/988398941 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d00041c0 con 0x55b9aa315780 2024-04-16T05:38:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73cdffb640 1 -- 172.21.15.86:0/988398941 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f73d000f040 con 0x55b9aa315780 2024-04-16T05:38:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73ef658740 10 client.16710 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73ef658740 10 client.16710 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:38:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73ef658740 20 client.16710 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73ef658740 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b9aa32e1e0 0x55b9aa3305d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73ef658740 4 client.16710 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.283+0000 7f73ef658740 1 -- 172.21.15.86:0/988398941 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9aa287b00 con 0x55b9aa32e1e0 2024-04-16T05:38:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.284+0000 7f73e659c640 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b9aa32e1e0 0x55b9aa3305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.284+0000 7f73cdffb640 10 client.16710 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:38:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.284+0000 7f73e659c640 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b9aa32e1e0 0x55b9aa3305d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f0f992fbde5f3a29 server_cookie=f04330ccd568cad8 in_seq=0 out_seq=0 2024-04-16T05:38:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73cdffb640 1 -- 172.21.15.86:0/988398941 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9aa287b00 con 0x55b9aa32e1e0 2024-04-16T05:38:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73cdffb640 10 client.16710 handle_command_reply: tid=0 2024-04-16T05:38:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 client.16710 shutdown 2024-04-16T05:38:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 -- 172.21.15.86:0/988398941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 msgr2=0x55b9aa31b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31b940 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f73d0003df0 tx=0x7f73d00045a0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 -- 172.21.15.86:0/988398941 shutdown_connections 2024-04-16T05:38:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa31e360 0x55b9aa31be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa315780 0x55b9aa31b940 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b9aa32e1e0 0x55b9aa3305d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 --2- 172.21.15.86:0/988398941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 0x55b9aa31b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73cdffb640 0 client.16710 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:38:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 -- 172.21.15.86:0/988398941 >> 172.21.15.86:0/988398941 conn(0x55b9aa30f590 msgr2=0x55b9aa3112e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 -- 172.21.15.86:0/988398941 shutdown_connections 2024-04-16T05:38:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 1 -- 172.21.15.86:0/988398941 wait complete. 2024-04-16T05:38:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.285+0000 7f73ef658740 20 client.16710 trim_cache size 0 max 16384 2024-04-16T05:38:39.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73ef658740 1 Processor -- start 2024-04-16T05:38:39.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73ef658740 1 -- start start 2024-04-16T05:38:39.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73ef658740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa314e60 0x55b9aa31a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73ef658740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa315780 0x55b9aa335d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73ef658740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa31e360 0x55b9aa3362a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73ef658740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81af490 con 0x55b9aa314e60 2024-04-16T05:38:39.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73ef658740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81b0fd0 con 0x55b9aa31e360 2024-04-16T05:38:39.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73ef658740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81afb60 con 0x55b9aa315780 2024-04-16T05:38:39.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa314e60 0x55b9aa31a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa314e60 0x55b9aa31a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39594/0 (socket says 172.21.15.86:39594) 2024-04-16T05:38:39.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73e659c640 1 -- 172.21.15.86:0/3683968297 learned_addr learned my addr 172.21.15.86:0/3683968297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.298+0000 7f73e659c640 1 -- 172.21.15.86:0/3683968297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa315780 msgr2=0x55b9aa335d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73e659c640 1 --2- 172.21.15.86:0/3683968297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa315780 0x55b9aa335d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73e659c640 1 -- 172.21.15.86:0/3683968297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa31e360 msgr2=0x55b9aa3362a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73e659c640 1 --2- 172.21.15.86:0/3683968297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa31e360 0x55b9aa3362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73e659c640 1 -- 172.21.15.86:0/3683968297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9aa311760 con 0x55b9aa314e60 2024-04-16T05:38:39.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73e659c640 1 --2- 172.21.15.86:0/3683968297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa314e60 0x55b9aa31a580 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f73d80ba890 tx=0x7f73d802cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af69fe67f1e9008 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73cffff640 1 -- 172.21.15.86:0/3683968297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d804b040 con 0x55b9aa314e60 2024-04-16T05:38:39.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73cffff640 1 -- 172.21.15.86:0/3683968297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d80be430 con 0x55b9aa314e60 2024-04-16T05:38:39.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73cffff640 1 -- 172.21.15.86:0/3683968297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80baaa0 con 0x55b9aa314e60 2024-04-16T05:38:39.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73ef658740 1 -- 172.21.15.86:0/3683968297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa314e60 msgr2=0x55b9aa31a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73ef658740 1 --2- 172.21.15.86:0/3683968297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa314e60 0x55b9aa31a580 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f73d80ba890 tx=0x7f73d802cde0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73ef658740 1 -- 172.21.15.86:0/3683968297 shutdown_connections 2024-04-16T05:38:39.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73ef658740 1 --2- 172.21.15.86:0/3683968297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa31e360 0x55b9aa3362a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73ef658740 1 --2- 172.21.15.86:0/3683968297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa315780 0x55b9aa335d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73ef658740 1 --2- 172.21.15.86:0/3683968297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa314e60 0x55b9aa31a580 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73ef658740 1 -- 172.21.15.86:0/3683968297 >> 172.21.15.86:0/3683968297 conn(0x55b9aa30f590 msgr2=0x55b9aa30f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73ef658740 1 -- 172.21.15.86:0/3683968297 shutdown_connections 2024-04-16T05:38:39.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.299+0000 7f73ef658740 1 -- 172.21.15.86:0/3683968297 wait complete. 2024-04-16T05:38:39.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73ef658740 1 -- start start 2024-04-16T05:38:39.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73ef658740 1 Processor -- start 2024-04-16T05:38:39.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73ef658740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 0x55b9aa326540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73ef658740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa315780 0x55b9aa326a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73ef658740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa31e360 0x55b9aa317aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73ef658740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81b0900 con 0x55b9aa31e360 2024-04-16T05:38:39.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73ef658740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81b0230 con 0x55b9aa315780 2024-04-16T05:38:39.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73ef658740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81ad450 con 0x55b9aa314e60 2024-04-16T05:38:39.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73ef658740 10 client.0 _ll_register_callbacks cb 0x7f73d8005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa31e360 0x55b9aa317aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa31e360 0x55b9aa317aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39608/0 (socket says 172.21.15.86:39608) 2024-04-16T05:38:39.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e6d9d640 1 -- 172.21.15.86:0/1501766063 learned_addr learned my addr 172.21.15.86:0/1501766063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e6d9d640 1 -- 172.21.15.86:0/1501766063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 msgr2=0x55b9aa326540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e659c640 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 0x55b9aa326540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e6d9d640 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 0x55b9aa326540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e6d9d640 1 -- 172.21.15.86:0/1501766063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa315780 msgr2=0x55b9aa326a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e6d9d640 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa315780 0x55b9aa326a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e6d9d640 1 -- 172.21.15.86:0/1501766063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9aa3182d0 con 0x55b9aa31e360 2024-04-16T05:38:39.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.300+0000 7f73e6d9d640 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa31e360 0x55b9aa317aa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f73e0015300 tx=0x7f73e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864c423b57f1ab5a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73cdffb640 1 -- 172.21.15.86:0/1501766063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0024030 con 0x55b9aa31e360 2024-04-16T05:38:39.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73cdffb640 1 -- 172.21.15.86:0/1501766063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f73e0020610 con 0x55b9aa31e360 2024-04-16T05:38:39.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73ef658740 1 -- 172.21.15.86:0/1501766063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d80be430 con 0x55b9aa31e360 2024-04-16T05:38:39.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73cdffb640 1 -- 172.21.15.86:0/1501766063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f73e0031050 con 0x55b9aa31e360 2024-04-16T05:38:39.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73ef658740 10 client.16716 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73ef658740 1 -- 172.21.15.86:0/1501766063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9aa398420 con 0x55b9aa31e360 2024-04-16T05:38:39.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73cdffb640 1 -- 172.21.15.86:0/1501766063 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d80be430 con 0x55b9aa31e360 2024-04-16T05:38:39.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73cdffb640 1 -- 172.21.15.86:0/1501766063 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f73e001f040 con 0x55b9aa31e360 2024-04-16T05:38:39.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73ef658740 10 client.16716 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73ef658740 10 client.16716 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:38:39.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73ef658740 20 client.16716 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73ef658740 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b9aa398bc0 0x55b9aa39b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73ef658740 4 client.16716 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73ef658740 1 -- 172.21.15.86:0/1501766063 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9aa281970 con 0x55b9aa398bc0 2024-04-16T05:38:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.301+0000 7f73e659c640 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b9aa398bc0 0x55b9aa39b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73cdffb640 10 client.16716 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:38:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73e659c640 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b9aa398bc0 0x55b9aa39b000 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=126cc170fa6d1237 server_cookie=2c3009a45ad2a7ad in_seq=0 out_seq=0 2024-04-16T05:38:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73cdffb640 1 -- 172.21.15.86:0/1501766063 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9aa281970 con 0x55b9aa398bc0 2024-04-16T05:38:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73cdffb640 10 client.16716 handle_command_reply: tid=0 2024-04-16T05:38:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 client.16716 shutdown 2024-04-16T05:38:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 -- 172.21.15.86:0/1501766063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa31e360 msgr2=0x55b9aa317aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa31e360 0x55b9aa317aa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f73e0015300 tx=0x7f73e0004a40 comp rx=0 tx=0).stop 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 -- 172.21.15.86:0/1501766063 shutdown_connections 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9aa31e360 0x55b9aa317aa0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9aa315780 0x55b9aa326a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55b9aa398bc0 0x55b9aa39b000 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 --2- 172.21.15.86:0/1501766063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9aa314e60 0x55b9aa326540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73cdffb640 0 client.16716 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 -- 172.21.15.86:0/1501766063 >> 172.21.15.86:0/1501766063 conn(0x55b9aa30f590 msgr2=0x55b9aa310750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 -- 172.21.15.86:0/1501766063 shutdown_connections 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 1 -- 172.21.15.86:0/1501766063 wait complete. 2024-04-16T05:38:39.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.302+0000 7f73ef658740 20 client.16716 trim_cache size 0 max 16384 2024-04-16T05:38:39.313 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:39.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 -- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f73d40a3d20 msgr2=0x7f73d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f73d40a3d20 0x7f73d40a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f73e0005c40 tx=0x7f73e0015900 comp rx=0 tx=0).stop 2024-04-16T05:38:39.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 -- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 msgr2=0x7f73e81996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 0x7f73e81996e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006010 tx=0x7f73d802cb30 comp rx=0 tx=0).stop 2024-04-16T05:38:39.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 -- 172.21.15.86:0/2518824022 shutdown_connections 2024-04-16T05:38:39.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e819a1b0 0x7f73e81a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f73d40a3d20 0x7f73d40a61e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73e8106200 0x7f73e8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 --2- 172.21.15.86:0/2518824022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8101130 0x7f73e81996e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 -- 172.21.15.86:0/2518824022 >> 172.21.15.86:0/2518824022 conn(0x7f73e80780b0 msgr2=0x7f73e80fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 -- 172.21.15.86:0/2518824022 shutdown_connections 2024-04-16T05:38:39.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.303+0000 7f73eca6a640 1 -- 172.21.15.86:0/2518824022 wait complete. 2024-04-16T05:38:39.316 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:38:39.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.423+0000 7f18efb96640 1 -- 172.21.15.86:0/2217882115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e810c630 msgr2=0x7f18e810ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.423+0000 7f18efb96640 1 --2- 172.21.15.86:0/2217882115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e810c630 0x7f18e810ea60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009ca0 tx=0x7f18e002c180 comp rx=0 tx=0).stop 2024-04-16T05:38:39.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.423+0000 7f18efb96640 1 -- 172.21.15.86:0/2217882115 shutdown_connections 2024-04-16T05:38:39.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.423+0000 7f18efb96640 1 --2- 172.21.15.86:0/2217882115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e810c630 0x7f18e810ea60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.423+0000 7f18efb96640 1 --2- 172.21.15.86:0/2217882115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e8103b80 0x7f18e810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.423+0000 7f18efb96640 1 --2- 172.21.15.86:0/2217882115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18e8103260 0x7f18e8103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.423+0000 7f18efb96640 1 -- 172.21.15.86:0/2217882115 >> 172.21.15.86:0/2217882115 conn(0x7f18e8077280 msgr2=0x7f18e8077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.423+0000 7f18efb96640 1 -- 172.21.15.86:0/2217882115 shutdown_connections 2024-04-16T05:38:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.423+0000 7f18efb96640 1 -- 172.21.15.86:0/2217882115 wait complete. 2024-04-16T05:38:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18efb96640 1 Processor -- start 2024-04-16T05:38:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18efb96640 1 -- start start 2024-04-16T05:38:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18efb96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e8103260 0x7f18e819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18efb96640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e8103b80 0x7f18e819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18efb96640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18e810c630 0x7f18e81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18efb96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18e8112380 con 0x7f18e8103260 2024-04-16T05:38:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18efb96640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f18e8112200 con 0x7f18e8103b80 2024-04-16T05:38:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18efb96640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18e8112500 con 0x7f18e810c630 2024-04-16T05:38:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18ed932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e8103260 0x7f18e819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18ed932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e8103260 0x7f18e819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39626/0 (socket says 172.21.15.86:39626) 2024-04-16T05:38:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18ed932640 1 -- 172.21.15.86:0/1109609581 learned_addr learned my addr 172.21.15.86:0/1109609581 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18ed131640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e8103b80 0x7f18e819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18ed932640 1 -- 172.21.15.86:0/1109609581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18e810c630 msgr2=0x7f18e81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18ed932640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18e810c630 0x7f18e81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18ed932640 1 -- 172.21.15.86:0/1109609581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e8103b80 msgr2=0x7f18e819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18ed932640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e8103b80 0x7f18e819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.424+0000 7f18ed932640 1 -- 172.21.15.86:0/1109609581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e8198d40 con 0x7f18e8103260 2024-04-16T05:38:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.425+0000 7f18ed932640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e8103260 0x7f18e819e410 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f18e400c920 tx=0x7f18e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.425+0000 7f18deffd640 1 -- 172.21.15.86:0/1109609581 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4007c70 con 0x7f18e8103260 2024-04-16T05:38:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.425+0000 7f18deffd640 1 -- 172.21.15.86:0/1109609581 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e4007e10 con 0x7f18e8103260 2024-04-16T05:38:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.425+0000 7f18efb96640 1 -- 172.21.15.86:0/1109609581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e8199030 con 0x7f18e8103260 2024-04-16T05:38:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.425+0000 7f18deffd640 1 -- 172.21.15.86:0/1109609581 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4018890 con 0x7f18e8103260 2024-04-16T05:38:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.425+0000 7f18efb96640 1 -- 172.21.15.86:0/1109609581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e81a52d0 con 0x7f18e8103260 2024-04-16T05:38:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.427+0000 7f18deffd640 1 -- 172.21.15.86:0/1109609581 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18e4018a70 con 0x7f18e8103260 2024-04-16T05:38:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18f2776740 1 Processor -- start 2024-04-16T05:38:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18f2776740 1 -- start start 2024-04-16T05:38:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18f2776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e080453240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18f2776740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080453780 0x55e08045be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18f2776740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e08045c360 0x55e08045e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18f2776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e0803b80e0 con 0x55e080452e60 2024-04-16T05:38:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18f2776740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e080355520 con 0x55e08045c360 2024-04-16T05:38:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18f2776740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e0803b40a0 con 0x55e080453780 2024-04-16T05:38:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18ee133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e080453240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18deffd640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f18d40a3d20 0x7f18d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18ee133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e080453240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39642/0 (socket says 172.21.15.86:39642) 2024-04-16T05:38:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18ee133640 1 -- 172.21.15.86:0/1332349251 learned_addr learned my addr 172.21.15.86:0/1332349251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18ee133640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f18d40a3d20 0x7f18d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18ed131640 1 --2- 172.21.15.86:0/1332349251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e08045c360 0x55e08045e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18deffd640 1 -- 172.21.15.86:0/1109609581 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f18e40d0da0 con 0x7f18e8103260 2024-04-16T05:38:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.428+0000 7f18ed932640 1 --2- 172.21.15.86:0/1332349251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080453780 0x55e08045be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.429+0000 7f18ee133640 1 -- 172.21.15.86:0/1332349251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080453780 msgr2=0x55e08045be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.429+0000 7f18ee133640 1 --2- 172.21.15.86:0/1332349251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080453780 0x55e08045be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.429+0000 7f18ee133640 1 -- 172.21.15.86:0/1332349251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e08045c360 msgr2=0x55e08045e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.429+0000 7f18ee133640 1 --2- 172.21.15.86:0/1332349251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e08045c360 0x55e08045e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.429+0000 7f18ee133640 1 -- 172.21.15.86:0/1332349251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e08037d8d0 con 0x55e080452e60 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.429+0000 7f18ee133640 1 --2- 172.21.15.86:0/1332349251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e080453240 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f18e002c790 tx=0x7f18e0003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7b2cc4af841cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.429+0000 7f18d3fff640 1 -- 172.21.15.86:0/1332349251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0004610 con 0x55e080452e60 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.429+0000 7f18d3fff640 1 -- 172.21.15.86:0/1332349251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e003a070 con 0x55e080452e60 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.429+0000 7f18d3fff640 1 -- 172.21.15.86:0/1332349251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e00346a0 con 0x55e080452e60 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 -- 172.21.15.86:0/1332349251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 msgr2=0x55e080453240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 --2- 172.21.15.86:0/1332349251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e080453240 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f18e002c790 tx=0x7f18e0003a70 comp rx=0 tx=0).stop 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18ee133640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f18d40a3d20 0x7f18d40a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18e004a000 tx=0x7f18e0048000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 -- 172.21.15.86:0/1332349251 shutdown_connections 2024-04-16T05:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 --2- 172.21.15.86:0/1332349251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e08045c360 0x55e08045e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 --2- 172.21.15.86:0/1332349251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080453780 0x55e08045be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 --2- 172.21.15.86:0/1332349251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e080453240 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 -- 172.21.15.86:0/1332349251 >> 172.21.15.86:0/1332349251 conn(0x55e08044d590 msgr2=0x55e08044d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 -- 172.21.15.86:0/1332349251 shutdown_connections 2024-04-16T05:38:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 -- 172.21.15.86:0/1332349251 wait complete. 2024-04-16T05:38:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 -- start start 2024-04-16T05:38:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.430+0000 7f18f2776740 1 Processor -- start 2024-04-16T05:38:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18f2776740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080452e60 0x55e080459460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18f2776740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e0804599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18f2776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e08045c360 0x55e080459ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18f2776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e08045efa0 con 0x55e08045c360 2024-04-16T05:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18f2776740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e0803714d0 con 0x55e080453780 2024-04-16T05:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18f2776740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e08045f120 con 0x55e080452e60 2024-04-16T05:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18f2776740 10 client.0 _ll_register_callbacks cb 0x55e0803562c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ee133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e08045c360 0x55e080459ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ee133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e08045c360 0x55e080459ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39644/0 (socket says 172.21.15.86:39644) 2024-04-16T05:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ee133640 1 -- 172.21.15.86:0/1694212014 learned_addr learned my addr 172.21.15.86:0/1694212014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ed131640 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e0804599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ed932640 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080452e60 0x55e080459460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ee133640 1 -- 172.21.15.86:0/1694212014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080452e60 msgr2=0x55e080459460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ee133640 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080452e60 0x55e080459460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ee133640 1 -- 172.21.15.86:0/1694212014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 msgr2=0x55e0804599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ee133640 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e0804599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ee133640 1 -- 172.21.15.86:0/1694212014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0800ca3f0 con 0x55e08045c360 2024-04-16T05:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ed932640 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080452e60 0x55e080459460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18ee133640 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e08045c360 0x55e080459ee0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f18e0032d40 tx=0x7f18e0003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c8861e569967b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18d17fa640 1 -- 172.21.15.86:0/1694212014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0039040 con 0x55e08045c360 2024-04-16T05:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18f2776740 1 -- 172.21.15.86:0/1694212014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e003a070 con 0x55e08045c360 2024-04-16T05:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.431+0000 7f18d17fa640 1 -- 172.21.15.86:0/1694212014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f18e0032670 con 0x55e08045c360 2024-04-16T05:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18d17fa640 1 -- 172.21.15.86:0/1694212014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18e0062050 con 0x55e08045c360 2024-04-16T05:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18f2776740 10 client.16728 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18f2776740 1 -- 172.21.15.86:0/1694212014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0801947d0 con 0x55e08045c360 2024-04-16T05:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18d17fa640 1 -- 172.21.15.86:0/1694212014 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e003a070 con 0x55e08045c360 2024-04-16T05:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18d17fa640 1 -- 172.21.15.86:0/1694212014 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f18e0034990 con 0x55e08045c360 2024-04-16T05:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18f2776740 10 client.16728 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18f2776740 10 client.16728 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18f2776740 20 client.16728 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18f2776740 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e08046c170 0x55e08046e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18f2776740 4 client.16728 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18f2776740 1 -- 172.21.15.86:0/1694212014 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0803c5b00 con 0x55e08046c170 2024-04-16T05:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.432+0000 7f18ed932640 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e08046c170 0x55e08046e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.433+0000 7f18d17fa640 10 client.16728 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.433+0000 7f18ed932640 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e08046c170 0x55e08046e560 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4d4e69ab78fc714 server_cookie=7b21745e3639c55d in_seq=0 out_seq=0 2024-04-16T05:38:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.433+0000 7f18d17fa640 1 -- 172.21.15.86:0/1694212014 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0803c5b00 con 0x55e08046c170 2024-04-16T05:38:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.433+0000 7f18d17fa640 10 client.16728 handle_command_reply: tid=0 2024-04-16T05:38:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.433+0000 7f18f2776740 1 client.16728 shutdown 2024-04-16T05:38:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 -- 172.21.15.86:0/1694212014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e08045c360 msgr2=0x55e080459ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e08045c360 0x55e080459ee0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f18e0032d40 tx=0x7f18e0003b60 comp rx=0 tx=0).stop 2024-04-16T05:38:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 -- 172.21.15.86:0/1694212014 shutdown_connections 2024-04-16T05:38:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e08046c170 0x55e08046e560 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e08045c360 0x55e080459ee0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18d17fa640 0 client.16728 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:38:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e0804599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 --2- 172.21.15.86:0/1694212014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e080452e60 0x55e080459460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 -- 172.21.15.86:0/1694212014 >> 172.21.15.86:0/1694212014 conn(0x55e08044d590 msgr2=0x55e08044f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 -- 172.21.15.86:0/1694212014 shutdown_connections 2024-04-16T05:38:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 1 -- 172.21.15.86:0/1694212014 wait complete. 2024-04-16T05:38:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.434+0000 7f18f2776740 20 client.16728 trim_cache size 0 max 16384 2024-04-16T05:38:39.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.447+0000 7f18f2776740 1 Processor -- start 2024-04-16T05:38:39.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.447+0000 7f18f2776740 1 -- start start 2024-04-16T05:38:39.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.447+0000 7f18f2776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.447+0000 7f18f2776740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e0804485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.447+0000 7f18f2776740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 0x55e080473bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.447+0000 7f18f2776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18e81af2f0 con 0x55e080452e60 2024-04-16T05:38:39.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.447+0000 7f18f2776740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f18e81b0e30 con 0x55e080453780 2024-04-16T05:38:39.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.447+0000 7f18f2776740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18e81af9c0 con 0x55e08045c360 2024-04-16T05:38:39.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39648/0 (socket says 172.21.15.86:39648) 2024-04-16T05:38:39.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed932640 1 -- 172.21.15.86:0/796901792 learned_addr learned my addr 172.21.15.86:0/796901792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed131640 1 --2- 172.21.15.86:0/796901792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e0804485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed932640 1 -- 172.21.15.86:0/796901792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 msgr2=0x55e080473bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ee133640 1 --2- 172.21.15.86:0/796901792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 0x55e080473bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed932640 1 --2- 172.21.15.86:0/796901792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 0x55e080473bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed932640 1 -- 172.21.15.86:0/796901792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 msgr2=0x55e0804485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed932640 1 --2- 172.21.15.86:0/796901792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e0804485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed932640 1 -- 172.21.15.86:0/796901792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e080458410 con 0x55e080452e60 2024-04-16T05:38:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18ed932640 1 --2- 172.21.15.86:0/796901792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045bb90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f18e409c960 tx=0x7f18e4098cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e178947fe0f5065 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18d37fe640 1 -- 172.21.15.86:0/796901792 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e40b5070 con 0x55e080452e60 2024-04-16T05:38:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18d37fe640 1 -- 172.21.15.86:0/796901792 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e40a0d80 con 0x55e080452e60 2024-04-16T05:38:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.448+0000 7f18d37fe640 1 -- 172.21.15.86:0/796901792 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e40ce400 con 0x55e080452e60 2024-04-16T05:38:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 -- 172.21.15.86:0/796901792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 msgr2=0x55e08045bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 --2- 172.21.15.86:0/796901792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045bb90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f18e409c960 tx=0x7f18e4098cb0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 -- 172.21.15.86:0/796901792 shutdown_connections 2024-04-16T05:38:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 --2- 172.21.15.86:0/796901792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 0x55e080473bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 --2- 172.21.15.86:0/796901792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e0804485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 --2- 172.21.15.86:0/796901792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045bb90 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 -- 172.21.15.86:0/796901792 >> 172.21.15.86:0/796901792 conn(0x55e08044d590 msgr2=0x55e08044c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 -- 172.21.15.86:0/796901792 shutdown_connections 2024-04-16T05:38:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 -- 172.21.15.86:0/796901792 wait complete. 2024-04-16T05:38:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 -- start start 2024-04-16T05:38:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 Processor -- start 2024-04-16T05:38:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.449+0000 7f18f2776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18f2776740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e080460400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18f2776740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 0x55e080460940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18f2776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18e81b0760 con 0x55e080452e60 2024-04-16T05:38:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18f2776740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f18e81b0090 con 0x55e080453780 2024-04-16T05:38:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18f2776740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18e81ad2b0 con 0x55e08045c360 2024-04-16T05:38:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18f2776740 10 client.0 _ll_register_callbacks cb 0x7f18d8009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39660/0 (socket says 172.21.15.86:39660) 2024-04-16T05:38:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed932640 1 -- 172.21.15.86:0/1649970583 learned_addr learned my addr 172.21.15.86:0/1649970583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ee133640 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 0x55e080460940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed131640 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e080460400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed932640 1 -- 172.21.15.86:0/1649970583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 msgr2=0x55e080460940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed932640 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 0x55e080460940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed932640 1 -- 172.21.15.86:0/1649970583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 msgr2=0x55e080460400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed932640 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e080460400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed932640 1 -- 172.21.15.86:0/1649970583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0804d1470 con 0x55e080452e60 2024-04-16T05:38:39.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed131640 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e080460400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:39.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ee133640 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 0x55e080460940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:39.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18ed932640 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045fec0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f18e4087010 tx=0x7f18e400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a799d5b02e1479 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18d17fa640 1 -- 172.21.15.86:0/1649970583 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e40b5070 con 0x55e080452e60 2024-04-16T05:38:39.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18d17fa640 1 -- 172.21.15.86:0/1649970583 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f18e4099040 con 0x55e080452e60 2024-04-16T05:38:39.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18f2776740 1 -- 172.21.15.86:0/1649970583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e40a0d80 con 0x55e080452e60 2024-04-16T05:38:39.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18d17fa640 1 -- 172.21.15.86:0/1649970583 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18e40a5640 con 0x55e080452e60 2024-04-16T05:38:39.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18f2776740 10 client.16734 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18f2776740 1 -- 172.21.15.86:0/1649970583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0804d2770 con 0x55e080452e60 2024-04-16T05:38:39.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.450+0000 7f18d17fa640 1 -- 172.21.15.86:0/1649970583 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e40a0d80 con 0x55e080452e60 2024-04-16T05:38:39.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18d17fa640 1 -- 172.21.15.86:0/1649970583 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f18e40bbb90 con 0x55e080452e60 2024-04-16T05:38:39.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18f2776740 10 client.16734 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18f2776740 10 client.16734 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:38:39.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18f2776740 20 client.16734 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18f2776740 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e0804d2ee0 0x55e0804d5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18f2776740 4 client.16734 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:39.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18f2776740 1 -- 172.21.15.86:0/1649970583 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0803bf970 con 0x55e0804d2ee0 2024-04-16T05:38:39.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18ed131640 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e0804d2ee0 0x55e0804d5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18d17fa640 10 client.16734 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:38:39.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.451+0000 7f18ed131640 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e0804d2ee0 0x55e0804d5300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2432bb486c80a33d server_cookie=c7ed9996604984ec in_seq=0 out_seq=0 2024-04-16T05:38:39.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18d17fa640 1 -- 172.21.15.86:0/1649970583 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0803bf970 con 0x55e0804d2ee0 2024-04-16T05:38:39.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18d17fa640 10 client.16734 handle_command_reply: tid=0 2024-04-16T05:38:39.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 client.16734 shutdown 2024-04-16T05:38:39.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 -- 172.21.15.86:0/1649970583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 msgr2=0x55e08045fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045fec0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f18e4087010 tx=0x7f18e400fb30 comp rx=0 tx=0).stop 2024-04-16T05:38:39.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 -- 172.21.15.86:0/1649970583 shutdown_connections 2024-04-16T05:38:39.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e08045c360 0x55e080460940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e080453780 0x55e080460400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e0804d2ee0 0x55e0804d5300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 --2- 172.21.15.86:0/1649970583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e080452e60 0x55e08045fec0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18d17fa640 0 client.16734 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:38:39.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 -- 172.21.15.86:0/1649970583 >> 172.21.15.86:0/1649970583 conn(0x55e08044d590 msgr2=0x55e080459250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 -- 172.21.15.86:0/1649970583 shutdown_connections 2024-04-16T05:38:39.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 1 -- 172.21.15.86:0/1649970583 wait complete. 2024-04-16T05:38:39.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18f2776740 20 client.16734 trim_cache size 0 max 16384 2024-04-16T05:38:39.462 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:39.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 -- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f18d40a3d20 msgr2=0x7f18d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f18d40a3d20 0x7f18d40a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18e004a000 tx=0x7f18e0048000 comp rx=0 tx=0).stop 2024-04-16T05:38:39.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 -- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e8103260 msgr2=0x7f18e819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e8103260 0x7f18e819e410 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f18e400c920 tx=0x7f18e400cde0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 -- 172.21.15.86:0/1109609581 shutdown_connections 2024-04-16T05:38:39.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18e810c630 0x7f18e81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f18d40a3d20 0x7f18d40a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e8103b80 0x7f18e819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 --2- 172.21.15.86:0/1109609581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e8103260 0x7f18e819e410 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.452+0000 7f18efb96640 1 -- 172.21.15.86:0/1109609581 >> 172.21.15.86:0/1109609581 conn(0x7f18e8077280 msgr2=0x7f18e80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.453+0000 7f18efb96640 1 -- 172.21.15.86:0/1109609581 shutdown_connections 2024-04-16T05:38:39.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.453+0000 7f18efb96640 1 -- 172.21.15.86:0/1109609581 wait complete. 2024-04-16T05:38:39.464 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:38:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 -- 172.21.15.86:0/2536753822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac106200 msgr2=0x7facac10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 --2- 172.21.15.86:0/2536753822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac106200 0x7facac10ddd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7faca4009ca0 tx=0x7faca402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 -- 172.21.15.86:0/2536753822 shutdown_connections 2024-04-16T05:38:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 --2- 172.21.15.86:0/2536753822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac106200 0x7facac10ddd0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 --2- 172.21.15.86:0/2536753822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facac101ae0 0x7facac105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 --2- 172.21.15.86:0/2536753822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facac101130 0x7facac101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 -- 172.21.15.86:0/2536753822 >> 172.21.15.86:0/2536753822 conn(0x7facac0780b0 msgr2=0x7facac0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 -- 172.21.15.86:0/2536753822 shutdown_connections 2024-04-16T05:38:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 -- 172.21.15.86:0/2536753822 wait complete. 2024-04-16T05:38:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.575+0000 7facb284a640 1 Processor -- start 2024-04-16T05:38:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb284a640 1 -- start start 2024-04-16T05:38:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb284a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facac101130 0x7facac19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb284a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facac106200 0x7facac19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb284a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac198730 0x7facac198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb284a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facac112410 con 0x7facac198730 2024-04-16T05:38:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb284a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7facac112290 con 0x7facac106200 2024-04-16T05:38:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb284a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facac112590 con 0x7facac101130 2024-04-16T05:38:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb0de7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac198730 0x7facac198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb0de7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac198730 0x7facac198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39668/0 (socket says 172.21.15.86:39668) 2024-04-16T05:38:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb0de7640 1 -- 172.21.15.86:0/909067329 learned_addr learned my addr 172.21.15.86:0/909067329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb0de7640 1 -- 172.21.15.86:0/909067329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facac101130 msgr2=0x7facac19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb0de7640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facac101130 0x7facac19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb0de7640 1 -- 172.21.15.86:0/909067329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facac106200 msgr2=0x7facac19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb0de7640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facac106200 0x7facac19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb0de7640 1 -- 172.21.15.86:0/909067329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac199330 con 0x7facac198730 2024-04-16T05:38:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7facb0de7640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac198730 0x7facac198bb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7faca402c700 tx=0x7faca4035b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.576+0000 7faca97fa640 1 -- 172.21.15.86:0/909067329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4039040 con 0x7facac198730 2024-04-16T05:38:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.577+0000 7facb284a640 1 -- 172.21.15.86:0/909067329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac1995c0 con 0x7facac198730 2024-04-16T05:38:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.577+0000 7faca97fa640 1 -- 172.21.15.86:0/909067329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faca4044070 con 0x7facac198730 2024-04-16T05:38:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.577+0000 7faca97fa640 1 -- 172.21.15.86:0/909067329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca40043a0 con 0x7facac198730 2024-04-16T05:38:39.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.577+0000 7facb284a640 1 -- 172.21.15.86:0/909067329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac1a56b0 con 0x7facac198730 2024-04-16T05:38:39.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7faca97fa640 1 -- 172.21.15.86:0/909067329 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faca4004540 con 0x7facac198730 2024-04-16T05:38:39.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facb5437740 1 Processor -- start 2024-04-16T05:38:39.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facb5437740 1 -- start start 2024-04-16T05:38:39.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facb5437740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b803240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facb5437740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b80be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facb5437740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b80e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facb5437740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55956b7680e0 con 0x55956b803780 2024-04-16T05:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facb5437740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55956b705520 con 0x55956b802e60 2024-04-16T05:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facb5437740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55956b7640a0 con 0x55956b80c360 2024-04-16T05:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b80be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b80be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39680/0 (socket says 172.21.15.86:39680) 2024-04-16T05:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facabfff640 1 -- 172.21.15.86:0/3364483498 learned_addr learned my addr 172.21.15.86:0/3364483498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7faca97fa640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fac980a3d20 0x7fac980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.579+0000 7facab7fe640 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b803240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facab7fe640 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b80e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facb0de7640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fac980a3d20 0x7fac980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facabfff640 1 -- 172.21.15.86:0/3364483498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 msgr2=0x55956b80e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facabfff640 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b80e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facabfff640 1 -- 172.21.15.86:0/3364483498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 msgr2=0x55956b803240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facabfff640 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b803240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facabfff640 1 -- 172.21.15.86:0/3364483498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956b72d8d0 con 0x55956b803780 2024-04-16T05:38:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facab7fe640 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b80e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7faca97fa640 1 -- 172.21.15.86:0/909067329 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faca40f3030 con 0x7facac198730 2024-04-16T05:38:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facabfff640 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b80be20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00a9f0 tx=0x7fac9c00aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acba7769591c18b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7faca27fc640 1 -- 172.21.15.86:0/3364483498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00cde0 con 0x55956b803780 2024-04-16T05:38:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7facb0de7640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fac980a3d20 0x7fac980a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faca40bb1d0 tx=0x7faca4049000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7faca27fc640 1 -- 172.21.15.86:0/3364483498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9c00f930 con 0x55956b803780 2024-04-16T05:38:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.580+0000 7faca27fc640 1 -- 172.21.15.86:0/3364483498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00e950 con 0x55956b803780 2024-04-16T05:38:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 -- 172.21.15.86:0/3364483498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 msgr2=0x55956b80be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b80be20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00a9f0 tx=0x7fac9c00aeb0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 -- 172.21.15.86:0/3364483498 shutdown_connections 2024-04-16T05:38:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b80e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b80be20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 --2- 172.21.15.86:0/3364483498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b803240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 -- 172.21.15.86:0/3364483498 >> 172.21.15.86:0/3364483498 conn(0x55956b7fd590 msgr2=0x55956b7fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 -- 172.21.15.86:0/3364483498 shutdown_connections 2024-04-16T05:38:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 -- 172.21.15.86:0/3364483498 wait complete. 2024-04-16T05:38:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 -- start start 2024-04-16T05:38:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 Processor -- start 2024-04-16T05:38:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b8093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.581+0000 7facb5437740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b8098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facb5437740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b809e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facb5437740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55956b80f060 con 0x55956b803780 2024-04-16T05:38:39.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facb5437740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55956b7214d0 con 0x55956b802e60 2024-04-16T05:38:39.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facb5437740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55956b80f1e0 con 0x55956b80c360 2024-04-16T05:38:39.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facb5437740 10 client.0 _ll_register_callbacks cb 0x55956b7062c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b8098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b8098f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39696/0 (socket says 172.21.15.86:39696) 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facabfff640 1 -- 172.21.15.86:0/1081775878 learned_addr learned my addr 172.21.15.86:0/1081775878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facab7fe640 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b809e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facab7fe640 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b8093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facabfff640 1 -- 172.21.15.86:0/1081775878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 msgr2=0x55956b809e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facabfff640 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b809e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facabfff640 1 -- 172.21.15.86:0/1081775878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 msgr2=0x55956b8093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facabfff640 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b8093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facabfff640 1 -- 172.21.15.86:0/1081775878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956b47a3f0 con 0x55956b803780 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facab7fe640 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b8093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facabfff640 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b8098f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00fd10 tx=0x7fac9c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb0583d08242b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7fac8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7fac8b7fe640 1 -- 172.21.15.86:0/1081775878 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00e690 con 0x55956b803780 2024-04-16T05:38:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7fac8b7fe640 1 -- 172.21.15.86:0/1081775878 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fac9c00e830 con 0x55956b803780 2024-04-16T05:38:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facb5437740 1 -- 172.21.15.86:0/1081775878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac9c00f930 con 0x55956b803780 2024-04-16T05:38:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7fac8b7fe640 1 -- 172.21.15.86:0/1081775878 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac9c030050 con 0x55956b803780 2024-04-16T05:38:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facb5437740 10 client.16746 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.582+0000 7facb5437740 1 -- 172.21.15.86:0/1081775878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55956b5447d0 con 0x55956b803780 2024-04-16T05:38:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.583+0000 7fac8b7fe640 1 -- 172.21.15.86:0/1081775878 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9c00f930 con 0x55956b803780 2024-04-16T05:38:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.583+0000 7fac8b7fe640 1 -- 172.21.15.86:0/1081775878 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac9c02b030 con 0x55956b803780 2024-04-16T05:38:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.583+0000 7facb5437740 10 client.16746 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.583+0000 7facb5437740 10 client.16746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:38:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.583+0000 7facb5437740 20 client.16746 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.583+0000 7facb5437740 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55956b81c1c0 0x55956b81e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.583+0000 7facb5437740 4 client.16746 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.583+0000 7facb5437740 1 -- 172.21.15.86:0/1081775878 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55956b775b00 con 0x55956b81c1c0 2024-04-16T05:38:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.583+0000 7facab7fe640 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55956b81c1c0 0x55956b81e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.584+0000 7fac8b7fe640 10 client.16746 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:38:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.584+0000 7facab7fe640 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55956b81c1c0 0x55956b81e5b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=382926aa4aaf29b1 server_cookie=1efb854c29eb0f0c in_seq=0 out_seq=0 2024-04-16T05:38:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.584+0000 7fac8b7fe640 1 -- 172.21.15.86:0/1081775878 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55956b775b00 con 0x55956b81c1c0 2024-04-16T05:38:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.584+0000 7fac8b7fe640 10 client.16746 handle_command_reply: tid=0 2024-04-16T05:38:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.584+0000 7facb5437740 1 client.16746 shutdown 2024-04-16T05:38:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.584+0000 7facb5437740 1 -- 172.21.15.86:0/1081775878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 msgr2=0x55956b8098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b8098f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00fd10 tx=0x7fac9c00fd40 comp rx=0 tx=0).stop 2024-04-16T05:38:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 1 -- 172.21.15.86:0/1081775878 shutdown_connections 2024-04-16T05:38:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55956b81c1c0 0x55956b81e5b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b809e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b8098f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7fac8b7fe640 0 client.16746 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:38:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 1 --2- 172.21.15.86:0/1081775878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b8093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 1 -- 172.21.15.86:0/1081775878 >> 172.21.15.86:0/1081775878 conn(0x55956b7fd590 msgr2=0x55956b7ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 1 -- 172.21.15.86:0/1081775878 shutdown_connections 2024-04-16T05:38:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 1 -- 172.21.15.86:0/1081775878 wait complete. 2024-04-16T05:38:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.585+0000 7facb5437740 20 client.16746 trim_cache size 0 max 16384 2024-04-16T05:38:39.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facb5437740 1 Processor -- start 2024-04-16T05:38:39.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facb5437740 1 -- start start 2024-04-16T05:38:39.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facb5437740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b802e60 0x55956b823b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facb5437740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b803780 0x55956b824090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facb5437740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b7fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facb5437740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facac1af550 con 0x55956b802e60 2024-04-16T05:38:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facb5437740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b1090 con 0x55956b803780 2024-04-16T05:38:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facb5437740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facac1afc20 con 0x55956b80c360 2024-04-16T05:38:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b802e60 0x55956b823b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b802e60 0x55956b823b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39698/0 (socket says 172.21.15.86:39698) 2024-04-16T05:38:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facab7fe640 1 -- 172.21.15.86:0/559484877 learned_addr learned my addr 172.21.15.86:0/559484877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facabfff640 1 --2- 172.21.15.86:0/559484877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b803780 0x55956b824090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facab7fe640 1 -- 172.21.15.86:0/559484877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 msgr2=0x55956b7fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facab7fe640 1 --2- 172.21.15.86:0/559484877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b7fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facab7fe640 1 -- 172.21.15.86:0/559484877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b803780 msgr2=0x55956b824090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facab7fe640 1 --2- 172.21.15.86:0/559484877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b803780 0x55956b824090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facab7fe640 1 -- 172.21.15.86:0/559484877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956b80eac0 con 0x55956b802e60 2024-04-16T05:38:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facabfff640 1 --2- 172.21.15.86:0/559484877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b803780 0x55956b824090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.598+0000 7facab7fe640 1 --2- 172.21.15.86:0/559484877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b802e60 0x55956b823b50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fac94001010 tx=0x7fac94001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2165435c78be3225 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7faca1ffb640 1 -- 172.21.15.86:0/559484877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94016a30 con 0x55956b802e60 2024-04-16T05:38:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7faca1ffb640 1 -- 172.21.15.86:0/559484877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9401d070 con 0x55956b802e60 2024-04-16T05:38:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7faca1ffb640 1 -- 172.21.15.86:0/559484877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940065f0 con 0x55956b802e60 2024-04-16T05:38:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 -- 172.21.15.86:0/559484877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b802e60 msgr2=0x55956b823b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 --2- 172.21.15.86:0/559484877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b802e60 0x55956b823b50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fac94001010 tx=0x7fac94001040 comp rx=0 tx=0).stop 2024-04-16T05:38:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 -- 172.21.15.86:0/559484877 shutdown_connections 2024-04-16T05:38:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 --2- 172.21.15.86:0/559484877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b7fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 --2- 172.21.15.86:0/559484877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b803780 0x55956b824090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 --2- 172.21.15.86:0/559484877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b802e60 0x55956b823b50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 -- 172.21.15.86:0/559484877 >> 172.21.15.86:0/559484877 conn(0x55956b7fd590 msgr2=0x55956b7fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 -- 172.21.15.86:0/559484877 shutdown_connections 2024-04-16T05:38:39.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 -- 172.21.15.86:0/559484877 wait complete. 2024-04-16T05:38:39.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.599+0000 7facb5437740 1 -- start start 2024-04-16T05:38:39.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facb5437740 1 Processor -- start 2024-04-16T05:38:39.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facb5437740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b814a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facb5437740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b805e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facb5437740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b8065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facb5437740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b09c0 con 0x55956b803780 2024-04-16T05:38:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facb5437740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b02f0 con 0x55956b802e60 2024-04-16T05:38:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facb5437740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facac1ad510 con 0x55956b80c360 2024-04-16T05:38:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facb5437740 10 client.0 _ll_register_callbacks cb 0x7fac9c00a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b805e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b805e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39702/0 (socket says 172.21.15.86:39702) 2024-04-16T05:38:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facabfff640 1 -- 172.21.15.86:0/2084792005 learned_addr learned my addr 172.21.15.86:0/2084792005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facab7fe640 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b814a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facabfff640 1 -- 172.21.15.86:0/2084792005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 msgr2=0x55956b8065c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facabfff640 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b8065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facabfff640 1 -- 172.21.15.86:0/2084792005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 msgr2=0x55956b814a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facabfff640 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b814a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facabfff640 1 -- 172.21.15.86:0/2084792005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956b8813c0 con 0x55956b803780 2024-04-16T05:38:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facab7fe640 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b814a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.600+0000 7facabfff640 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b805e80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00aef0 tx=0x7fac9c00a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fa4f3ceeeadfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7fac8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7fac8b7fe640 1 -- 172.21.15.86:0/2084792005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c038070 con 0x55956b803780 2024-04-16T05:38:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7fac8b7fe640 1 -- 172.21.15.86:0/2084792005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fac9c00f860 con 0x55956b803780 2024-04-16T05:38:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facb5437740 1 -- 172.21.15.86:0/2084792005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac9401d070 con 0x55956b803780 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7fac8b7fe640 1 -- 172.21.15.86:0/2084792005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac9c044050 con 0x55956b803780 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facb5437740 10 client.16752 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facb5437740 1 -- 172.21.15.86:0/2084792005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55956b8824f0 con 0x55956b803780 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7fac8b7fe640 1 -- 172.21.15.86:0/2084792005 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9401d070 con 0x55956b803780 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7fac8b7fe640 1 -- 172.21.15.86:0/2084792005 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac9c02b030 con 0x55956b803780 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facb5437740 10 client.16752 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facb5437740 10 client.16752 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facb5437740 20 client.16752 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facb5437740 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55956b882c70 0x55956b8850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facb5437740 4 client.16752 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:39.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facb5437740 1 -- 172.21.15.86:0/2084792005 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55956b76f970 con 0x55956b882c70 2024-04-16T05:38:39.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.601+0000 7facab7fe640 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55956b882c70 0x55956b8850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7fac8b7fe640 10 client.16752 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:38:39.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facab7fe640 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55956b882c70 0x55956b8850b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38cf743894713f67 server_cookie=c237d3a7b5753ca0 in_seq=0 out_seq=0 2024-04-16T05:38:39.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7fac8b7fe640 1 -- 172.21.15.86:0/2084792005 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55956b76f970 con 0x55956b882c70 2024-04-16T05:38:39.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7fac8b7fe640 10 client.16752 handle_command_reply: tid=0 2024-04-16T05:38:39.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 client.16752 shutdown 2024-04-16T05:38:39.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 -- 172.21.15.86:0/2084792005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 msgr2=0x55956b805e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b805e80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00aef0 tx=0x7fac9c00a280 comp rx=0 tx=0).stop 2024-04-16T05:38:39.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 -- 172.21.15.86:0/2084792005 shutdown_connections 2024-04-16T05:38:39.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55956b882c70 0x55956b8850b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55956b80c360 0x55956b8065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7fac8b7fe640 0 client.16752 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:38:39.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956b803780 0x55956b805e80 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 --2- 172.21.15.86:0/2084792005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55956b802e60 0x55956b814a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 -- 172.21.15.86:0/2084792005 >> 172.21.15.86:0/2084792005 conn(0x55956b7fd590 msgr2=0x55956b80aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 -- 172.21.15.86:0/2084792005 shutdown_connections 2024-04-16T05:38:39.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 1 -- 172.21.15.86:0/2084792005 wait complete. 2024-04-16T05:38:39.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.602+0000 7facb5437740 20 client.16752 trim_cache size 0 max 16384 2024-04-16T05:38:39.612 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:39.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 -- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fac980a3d20 msgr2=0x7fac980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fac980a3d20 0x7fac980a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faca40bb1d0 tx=0x7faca4049000 comp rx=0 tx=0).stop 2024-04-16T05:38:39.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 -- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac198730 msgr2=0x7facac198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac198730 0x7facac198bb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7faca402c700 tx=0x7faca4035b60 comp rx=0 tx=0).stop 2024-04-16T05:38:39.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 -- 172.21.15.86:0/909067329 shutdown_connections 2024-04-16T05:38:39.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facac198730 0x7facac198bb0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fac980a3d20 0x7fac980a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facac106200 0x7facac19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 --2- 172.21.15.86:0/909067329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facac101130 0x7facac19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 -- 172.21.15.86:0/909067329 >> 172.21.15.86:0/909067329 conn(0x7facac0780b0 msgr2=0x7facac0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 -- 172.21.15.86:0/909067329 shutdown_connections 2024-04-16T05:38:39.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.603+0000 7facb284a640 1 -- 172.21.15.86:0/909067329 wait complete. 2024-04-16T05:38:39.615 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:38:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.723+0000 7f227c84a640 1 -- 172.21.15.86:0/1947131010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2278101af0 msgr2=0x7f2278105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.723+0000 7f227c84a640 1 --2- 172.21.15.86:0/1947131010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2278101af0 0x7f2278105b30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2260009900 tx=0x7f226002bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.723+0000 7f227c84a640 1 -- 172.21.15.86:0/1947131010 shutdown_connections 2024-04-16T05:38:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.723+0000 7f227c84a640 1 --2- 172.21.15.86:0/1947131010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278106210 0x7f227810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.723+0000 7f227c84a640 1 --2- 172.21.15.86:0/1947131010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2278101af0 0x7f2278105b30 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.723+0000 7f227c84a640 1 --2- 172.21.15.86:0/1947131010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2278101140 0x7f2278101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.723+0000 7f227c84a640 1 -- 172.21.15.86:0/1947131010 >> 172.21.15.86:0/1947131010 conn(0x7f2278077280 msgr2=0x7f2278077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.723+0000 7f227c84a640 1 -- 172.21.15.86:0/1947131010 shutdown_connections 2024-04-16T05:38:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.723+0000 7f227c84a640 1 -- 172.21.15.86:0/1947131010 wait complete. 2024-04-16T05:38:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.724+0000 7f227c84a640 1 Processor -- start 2024-04-16T05:38:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.724+0000 7f227c84a640 1 -- start start 2024-04-16T05:38:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.724+0000 7f227c84a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2278101140 0x7f227819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.724+0000 7f227c84a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278106210 0x7f227819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.724+0000 7f227c84a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22781986b0 0x7f2278198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.724+0000 7f227c84a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22781112a0 con 0x7f22781986b0 2024-04-16T05:38:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.724+0000 7f227c84a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2278111120 con 0x7f2278101140 2024-04-16T05:38:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.724+0000 7f227c84a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2278111420 con 0x7f2278106210 2024-04-16T05:38:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.724+0000 7f2276d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22781986b0 0x7f2278198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f2276d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22781986b0 0x7f2278198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39718/0 (socket says 172.21.15.86:39718) 2024-04-16T05:38:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f2276d9d640 1 -- 172.21.15.86:0/3507610752 learned_addr learned my addr 172.21.15.86:0/3507610752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f227659c640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2278101140 0x7f227819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f226ffff640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278106210 0x7f227819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f2276d9d640 1 -- 172.21.15.86:0/3507610752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278106210 msgr2=0x7f227819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f2276d9d640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278106210 0x7f227819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f2276d9d640 1 -- 172.21.15.86:0/3507610752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2278101140 msgr2=0x7f227819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f2276d9d640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2278101140 0x7f227819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f2276d9d640 1 -- 172.21.15.86:0/3507610752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22781992b0 con 0x7f22781986b0 2024-04-16T05:38:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f226ffff640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278106210 0x7f227819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f227659c640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2278101140 0x7f227819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f2276d9d640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22781986b0 0x7f2278198b30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f227000eaa0 tx=0x7f227000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f226f7fe640 1 -- 172.21.15.86:0/3507610752 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227000cdb0 con 0x7f22781986b0 2024-04-16T05:38:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f226f7fe640 1 -- 172.21.15.86:0/3507610752 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2270005020 con 0x7f22781986b0 2024-04-16T05:38:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f227c84a640 1 -- 172.21.15.86:0/3507610752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22781a5390 con 0x7f22781986b0 2024-04-16T05:38:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f226f7fe640 1 -- 172.21.15.86:0/3507610752 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22700116b0 con 0x7f22781986b0 2024-04-16T05:38:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.725+0000 7f227c84a640 1 -- 172.21.15.86:0/3507610752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22781a5740 con 0x7f22781986b0 2024-04-16T05:38:39.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.727+0000 7f226f7fe640 1 -- 172.21.15.86:0/3507610752 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2270005560 con 0x7f22781986b0 2024-04-16T05:38:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f227f438740 1 Processor -- start 2024-04-16T05:38:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f227f438740 1 -- start start 2024-04-16T05:38:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f227f438740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9767e60 0x55cbd9768240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f226f7fe640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f22640a3da0 0x7f22640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f227f438740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd9770e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f227f438740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9771360 0x55cbd9773750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f227f438740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cbd96cd0e0 con 0x55cbd9767e60 2024-04-16T05:38:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f227f438740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cbd966a520 con 0x55cbd9771360 2024-04-16T05:38:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f227f438740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbd96c90a0 con 0x55cbd9768780 2024-04-16T05:38:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f226ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9767e60 0x55cbd9768240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f227659c640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f22640a3da0 0x7f22640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f226ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9767e60 0x55cbd9768240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39728/0 (socket says 172.21.15.86:39728) 2024-04-16T05:38:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f226ffff640 1 -- 172.21.15.86:0/3378311759 learned_addr learned my addr 172.21.15.86:0/3378311759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.728+0000 7f226f7fe640 1 -- 172.21.15.86:0/3507610752 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f22700d0e30 con 0x7f22781986b0 2024-04-16T05:38:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226ffff640 1 --2- 172.21.15.86:0/3378311759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd9770e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f2276d9d640 1 --2- 172.21.15.86:0/3378311759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9771360 0x55cbd9773750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226ffff640 1 -- 172.21.15.86:0/3378311759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 msgr2=0x55cbd9770e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226ffff640 1 --2- 172.21.15.86:0/3378311759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd9770e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226ffff640 1 -- 172.21.15.86:0/3378311759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9771360 msgr2=0x55cbd9773750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226ffff640 1 --2- 172.21.15.86:0/3378311759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9771360 0x55cbd9773750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226ffff640 1 -- 172.21.15.86:0/3378311759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd96928d0 con 0x55cbd9767e60 2024-04-16T05:38:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f227659c640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f22640a3da0 0x7f22640a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f22680058c0 tx=0x7f2268005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226ffff640 1 --2- 172.21.15.86:0/3378311759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9767e60 0x55cbd9768240 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2260005ec0 tx=0x7f2260005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733520adfd163ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226cff9640 1 -- 172.21.15.86:0/3378311759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260046020 con 0x55cbd9767e60 2024-04-16T05:38:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226cff9640 1 -- 172.21.15.86:0/3378311759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2260005040 con 0x55cbd9767e60 2024-04-16T05:38:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.729+0000 7f226cff9640 1 -- 172.21.15.86:0/3378311759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260004950 con 0x55cbd9767e60 2024-04-16T05:38:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 -- 172.21.15.86:0/3378311759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9767e60 msgr2=0x55cbd9768240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 --2- 172.21.15.86:0/3378311759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9767e60 0x55cbd9768240 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2260005ec0 tx=0x7f2260005b50 comp rx=0 tx=0).stop 2024-04-16T05:38:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 -- 172.21.15.86:0/3378311759 shutdown_connections 2024-04-16T05:38:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 --2- 172.21.15.86:0/3378311759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9771360 0x55cbd9773750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 --2- 172.21.15.86:0/3378311759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd9770e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 --2- 172.21.15.86:0/3378311759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9767e60 0x55cbd9768240 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 -- 172.21.15.86:0/3378311759 >> 172.21.15.86:0/3378311759 conn(0x55cbd9762590 msgr2=0x55cbd9762970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 -- 172.21.15.86:0/3378311759 shutdown_connections 2024-04-16T05:38:39.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 -- 172.21.15.86:0/3378311759 wait complete. 2024-04-16T05:38:39.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 -- start start 2024-04-16T05:38:39.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 Processor -- start 2024-04-16T05:38:39.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd976e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd976e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd976eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cbd9773fa0 con 0x55cbd9771360 2024-04-16T05:38:39.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.730+0000 7f227f438740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cbd96864d0 con 0x55cbd9767e60 2024-04-16T05:38:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f227f438740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbd9774120 con 0x55cbd9768780 2024-04-16T05:38:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f227f438740 10 client.0 _ll_register_callbacks cb 0x55cbd966b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f2276d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd976eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f2276d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd976eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39738/0 (socket says 172.21.15.86:39738) 2024-04-16T05:38:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f2276d9d640 1 -- 172.21.15.86:0/144312145 learned_addr learned my addr 172.21.15.86:0/144312145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f226ffff640 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd976e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f227659c640 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd976e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f2276d9d640 1 -- 172.21.15.86:0/144312145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 msgr2=0x55cbd976e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f2276d9d640 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd976e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f2276d9d640 1 -- 172.21.15.86:0/144312145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 msgr2=0x55cbd976e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f2276d9d640 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd976e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f2276d9d640 1 -- 172.21.15.86:0/144312145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd93df3f0 con 0x55cbd9771360 2024-04-16T05:38:39.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f226ffff640 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd976e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:39.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f227659c640 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd976e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:39.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f2276d9d640 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd976eee0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f227009c720 tx=0x7f227009d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc9af7dbd1c9fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f225dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f225dffb640 1 -- 172.21.15.86:0/144312145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227001f040 con 0x55cbd9771360 2024-04-16T05:38:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f225dffb640 1 -- 172.21.15.86:0/144312145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f227009cc80 con 0x55cbd9771360 2024-04-16T05:38:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f227f438740 1 -- 172.21.15.86:0/144312145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2260005040 con 0x55cbd9771360 2024-04-16T05:38:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f225dffb640 1 -- 172.21.15.86:0/144312145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f22700ce650 con 0x55cbd9771360 2024-04-16T05:38:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f227f438740 10 client.16764 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f227f438740 1 -- 172.21.15.86:0/144312145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbd94a97d0 con 0x55cbd9771360 2024-04-16T05:38:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.731+0000 7f225dffb640 1 -- 172.21.15.86:0/144312145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2260005040 con 0x55cbd9771360 2024-04-16T05:38:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f225dffb640 1 -- 172.21.15.86:0/144312145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f227000f040 con 0x55cbd9771360 2024-04-16T05:38:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f227f438740 10 client.16764 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f227f438740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f227f438740 20 client.16764 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f227f438740 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cbd97812b0 0x55cbd97836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f227f438740 4 client.16764 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f227f438740 1 -- 172.21.15.86:0/144312145 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbd96dab00 con 0x55cbd97812b0 2024-04-16T05:38:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f227659c640 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cbd97812b0 0x55cbd97836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f225dffb640 10 client.16764 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.732+0000 7f227659c640 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cbd97812b0 0x55cbd97836e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c414fee96c8643f8 server_cookie=2723c2809cdea617 in_seq=0 out_seq=0 2024-04-16T05:38:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f225dffb640 1 -- 172.21.15.86:0/144312145 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbd96dab00 con 0x55cbd97812b0 2024-04-16T05:38:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f225dffb640 10 client.16764 handle_command_reply: tid=0 2024-04-16T05:38:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 client.16764 shutdown 2024-04-16T05:38:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 -- 172.21.15.86:0/144312145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 msgr2=0x55cbd976eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd976eee0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f227009c720 tx=0x7f227009d1e0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 -- 172.21.15.86:0/144312145 shutdown_connections 2024-04-16T05:38:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cbd97812b0 0x55cbd97836e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd976eee0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd976e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 --2- 172.21.15.86:0/144312145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd976e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f225dffb640 0 client.16764 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 -- 172.21.15.86:0/144312145 >> 172.21.15.86:0/144312145 conn(0x55cbd9762590 msgr2=0x55cbd97642e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 -- 172.21.15.86:0/144312145 shutdown_connections 2024-04-16T05:38:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.733+0000 7f227f438740 1 -- 172.21.15.86:0/144312145 wait complete. 2024-04-16T05:38:39.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.734+0000 7f227f438740 20 client.16764 trim_cache size 0 max 16384 2024-04-16T05:38:39.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f227f438740 1 Processor -- start 2024-04-16T05:38:39.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f227f438740 1 -- start start 2024-04-16T05:38:39.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f227f438740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd9770b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f227f438740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd977cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f227f438740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd97615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f227f438740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22781af420 con 0x55cbd9771360 2024-04-16T05:38:39.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f227f438740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22781b0f60 con 0x55cbd9767e60 2024-04-16T05:38:39.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f227f438740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22781afaf0 con 0x55cbd9768780 2024-04-16T05:38:39.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f2276d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd97615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f2276d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd97615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39746/0 (socket says 172.21.15.86:39746) 2024-04-16T05:38:39.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f2276d9d640 1 -- 172.21.15.86:0/417356095 learned_addr learned my addr 172.21.15.86:0/417356095 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f2276d9d640 1 -- 172.21.15.86:0/417356095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 msgr2=0x55cbd977cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f2276d9d640 1 --2- 172.21.15.86:0/417356095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd977cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f2276d9d640 1 -- 172.21.15.86:0/417356095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 msgr2=0x55cbd9770b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f2276d9d640 1 --2- 172.21.15.86:0/417356095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd9770b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f2276d9d640 1 -- 172.21.15.86:0/417356095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd97671e0 con 0x55cbd9771360 2024-04-16T05:38:39.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f2276d9d640 1 --2- 172.21.15.86:0/417356095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd97615b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f22700993a0 tx=0x7f227009c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757af0c05ebdd9e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f225ffff640 1 -- 172.21.15.86:0/417356095 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22700aa1f0 con 0x55cbd9771360 2024-04-16T05:38:39.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f225ffff640 1 -- 172.21.15.86:0/417356095 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22700bd090 con 0x55cbd9771360 2024-04-16T05:38:39.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.747+0000 7f225ffff640 1 -- 172.21.15.86:0/417356095 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227009cee0 con 0x55cbd9771360 2024-04-16T05:38:39.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 -- 172.21.15.86:0/417356095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 msgr2=0x55cbd97615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 --2- 172.21.15.86:0/417356095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd97615b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f22700993a0 tx=0x7f227009c560 comp rx=0 tx=0).stop 2024-04-16T05:38:39.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 -- 172.21.15.86:0/417356095 shutdown_connections 2024-04-16T05:38:39.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 --2- 172.21.15.86:0/417356095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9771360 0x55cbd97615b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 --2- 172.21.15.86:0/417356095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9768780 0x55cbd977cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 --2- 172.21.15.86:0/417356095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd9770b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 -- 172.21.15.86:0/417356095 >> 172.21.15.86:0/417356095 conn(0x55cbd9762590 msgr2=0x55cbd9760ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 -- 172.21.15.86:0/417356095 shutdown_connections 2024-04-16T05:38:39.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 -- 172.21.15.86:0/417356095 wait complete. 2024-04-16T05:38:39.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 -- start start 2024-04-16T05:38:39.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.748+0000 7f227f438740 1 Processor -- start 2024-04-16T05:38:39.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd9779600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9768780 0x55cbd97749c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9771360 0x55cbd9774f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22781b0890 con 0x55cbd9768780 2024-04-16T05:38:39.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22781b01c0 con 0x55cbd9767e60 2024-04-16T05:38:39.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22781ad3e0 con 0x55cbd9771360 2024-04-16T05:38:39.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 10 client.0 _ll_register_callbacks cb 0x7f2270009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:39.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f226ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9768780 0x55cbd97749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f226ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9768780 0x55cbd97749c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39752/0 (socket says 172.21.15.86:39752) 2024-04-16T05:38:39.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f226ffff640 1 -- 172.21.15.86:0/260543996 learned_addr learned my addr 172.21.15.86:0/260543996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:39.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227659c640 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd9779600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f226ffff640 1 -- 172.21.15.86:0/260543996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9771360 msgr2=0x55cbd9774f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f226ffff640 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9771360 0x55cbd9774f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f226ffff640 1 -- 172.21.15.86:0/260543996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 msgr2=0x55cbd9779600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:39.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f226ffff640 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd9779600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f226ffff640 1 -- 172.21.15.86:0/260543996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd9775540 con 0x55cbd9768780 2024-04-16T05:38:39.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f226ffff640 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9768780 0x55cbd97749c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f226002cfd0 tx=0x7f2260004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab4a16dc6d46e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:39.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f225dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:39.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f225dffb640 1 -- 172.21.15.86:0/260543996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226004e020 con 0x55cbd9768780 2024-04-16T05:38:39.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f225dffb640 1 -- 172.21.15.86:0/260543996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2260005250 con 0x55cbd9768780 2024-04-16T05:38:39.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 1 -- 172.21.15.86:0/260543996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22700bd090 con 0x55cbd9768780 2024-04-16T05:38:39.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f225dffb640 1 -- 172.21.15.86:0/260543996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2260059050 con 0x55cbd9768780 2024-04-16T05:38:39.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 10 client.16770 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.749+0000 7f227f438740 1 -- 172.21.15.86:0/260543996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbd97e7650 con 0x55cbd9768780 2024-04-16T05:38:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f225dffb640 1 -- 172.21.15.86:0/260543996 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22700bd090 con 0x55cbd9768780 2024-04-16T05:38:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f225dffb640 1 -- 172.21.15.86:0/260543996 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f22600035f0 con 0x55cbd9768780 2024-04-16T05:38:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f227f438740 10 client.16770 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f227f438740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f227f438740 20 client.16770 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f227f438740 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cbd97e7a00 0x55cbd97e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f227f438740 4 client.16770 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f227f438740 1 -- 172.21.15.86:0/260543996 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbd96d4970 con 0x55cbd97e7a00 2024-04-16T05:38:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f227659c640 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cbd97e7a00 0x55cbd97e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f225dffb640 10 client.16770 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.750+0000 7f227659c640 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cbd97e7a00 0x55cbd97e9df0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15f1db7d1b736da server_cookie=ec245d99de6ca0e0 in_seq=0 out_seq=0 2024-04-16T05:38:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f225dffb640 1 -- 172.21.15.86:0/260543996 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbd96d4970 con 0x55cbd97e7a00 2024-04-16T05:38:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f225dffb640 10 client.16770 handle_command_reply: tid=0 2024-04-16T05:38:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 client.16770 shutdown 2024-04-16T05:38:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 -- 172.21.15.86:0/260543996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9768780 msgr2=0x55cbd97749c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9768780 0x55cbd97749c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f226002cfd0 tx=0x7f2260004250 comp rx=0 tx=0).stop 2024-04-16T05:38:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 -- 172.21.15.86:0/260543996 shutdown_connections 2024-04-16T05:38:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55cbd97e7a00 0x55cbd97e9df0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbd9771360 0x55cbd9774f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbd9768780 0x55cbd97749c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f225dffb640 0 client.16770 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 --2- 172.21.15.86:0/260543996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cbd9767e60 0x55cbd9779600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 -- 172.21.15.86:0/260543996 >> 172.21.15.86:0/260543996 conn(0x55cbd9762590 msgr2=0x55cbd9761730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 -- 172.21.15.86:0/260543996 shutdown_connections 2024-04-16T05:38:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 1 -- 172.21.15.86:0/260543996 wait complete. 2024-04-16T05:38:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227f438740 20 client.16770 trim_cache size 0 max 16384 2024-04-16T05:38:39.761 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:38:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 -- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f22640a3da0 msgr2=0x7f22640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f22640a3da0 0x7f22640a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f22680058c0 tx=0x7f2268005850 comp rx=0 tx=0).stop 2024-04-16T05:38:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 -- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22781986b0 msgr2=0x7f2278198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22781986b0 0x7f2278198b30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f227000eaa0 tx=0x7f227000ef60 comp rx=0 tx=0).stop 2024-04-16T05:38:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 -- 172.21.15.86:0/3507610752 shutdown_connections 2024-04-16T05:38:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22781986b0 0x7f2278198b30 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278106210 0x7f227819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f22640a3da0 0x7f22640a6260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 --2- 172.21.15.86:0/3507610752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2278101140 0x7f227819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 -- 172.21.15.86:0/3507610752 >> 172.21.15.86:0/3507610752 conn(0x7f2278077280 msgr2=0x7f227806aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 -- 172.21.15.86:0/3507610752 shutdown_connections 2024-04-16T05:38:39.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:39.751+0000 7f227c84a640 1 -- 172.21.15.86:0/3507610752 wait complete. 2024-04-16T05:38:39.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:39 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/791419476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:39.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:39 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/791419476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:40.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:39 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/791419476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:40.765 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:38:40.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:38:40.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:38:40.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.887+0000 7f869af9f640 1 -- 172.21.15.86:0/2776564409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694101000 msgr2=0x7f86941013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:40.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.887+0000 7f869af9f640 1 --2- 172.21.15.86:0/2776564409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694101000 0x7f86941013e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8690009900 tx=0x7f869002bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:40.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.887+0000 7f869af9f640 1 -- 172.21.15.86:0/2776564409 shutdown_connections 2024-04-16T05:38:40.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.887+0000 7f869af9f640 1 --2- 172.21.15.86:0/2776564409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86941060d0 0x7f869410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.887+0000 7f869af9f640 1 --2- 172.21.15.86:0/2776564409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86941019b0 0x7f86941059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.887+0000 7f869af9f640 1 --2- 172.21.15.86:0/2776564409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694101000 0x7f86941013e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.887+0000 7f869af9f640 1 -- 172.21.15.86:0/2776564409 >> 172.21.15.86:0/2776564409 conn(0x7f86940fc880 msgr2=0x7f86940feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:40.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.887+0000 7f869af9f640 1 -- 172.21.15.86:0/2776564409 shutdown_connections 2024-04-16T05:38:40.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.887+0000 7f869af9f640 1 -- 172.21.15.86:0/2776564409 wait complete. 2024-04-16T05:38:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869af9f640 1 Processor -- start 2024-04-16T05:38:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869af9f640 1 -- start start 2024-04-16T05:38:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869af9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8694101000 0x7f8694102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869af9f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86941060d0 0x7f8694102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869af9f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694103510 0x7f86941a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869af9f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8694111180 con 0x7f8694103510 2024-04-16T05:38:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869af9f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8694111000 con 0x7f86941060d0 2024-04-16T05:38:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869af9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8694111300 con 0x7f8694101000 2024-04-16T05:38:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869a79e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694103510 0x7f86941a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869a79e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694103510 0x7f86941a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35942/0 (socket says 172.21.15.86:35942) 2024-04-16T05:38:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869a79e640 1 -- 172.21.15.86:0/2903848751 learned_addr learned my addr 172.21.15.86:0/2903848751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869979c640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86941060d0 0x7f8694102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f8699f9d640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8694101000 0x7f8694102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869a79e640 1 -- 172.21.15.86:0/2903848751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8694101000 msgr2=0x7f8694102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869a79e640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8694101000 0x7f8694102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869a79e640 1 -- 172.21.15.86:0/2903848751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86941060d0 msgr2=0x7f8694102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869a79e640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86941060d0 0x7f8694102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869a79e640 1 -- 172.21.15.86:0/2903848751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86941a7060 con 0x7f8694103510 2024-04-16T05:38:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.888+0000 7f869a79e640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694103510 0x7f86941a6960 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f868c00cce0 tx=0x7f868c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.889+0000 7f868affd640 1 -- 172.21.15.86:0/2903848751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c00f450 con 0x7f8694103510 2024-04-16T05:38:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.889+0000 7f868affd640 1 -- 172.21.15.86:0/2903848751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f868c00fad0 con 0x7f8694103510 2024-04-16T05:38:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.889+0000 7f869af9f640 1 -- 172.21.15.86:0/2903848751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86941a7350 con 0x7f8694103510 2024-04-16T05:38:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.889+0000 7f868affd640 1 -- 172.21.15.86:0/2903848751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c018c10 con 0x7f8694103510 2024-04-16T05:38:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.889+0000 7f869af9f640 1 -- 172.21.15.86:0/2903848751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86941a7850 con 0x7f8694103510 2024-04-16T05:38:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.891+0000 7f868affd640 1 -- 172.21.15.86:0/2903848751 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f868c00f5f0 con 0x7f8694103510 2024-04-16T05:38:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.891+0000 7f869ede2740 1 Processor -- start 2024-04-16T05:38:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869ede2740 1 -- start start 2024-04-16T05:38:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869ede2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869ede2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d08c5e0 0x55ea2d094c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869ede2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d0951c0 0x55ea2d0975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869ede2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea2cff0e90 con 0x55ea2d0951c0 2024-04-16T05:38:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869ede2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ea2cf8e4c0 con 0x55ea2d08bcc0 2024-04-16T05:38:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869ede2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea2cfece50 con 0x55ea2d08c5e0 2024-04-16T05:38:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f868affd640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f86800a3d50 0x7f86800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d0951c0 0x55ea2d0975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d0951c0 0x55ea2d0975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35956/0 (socket says 172.21.15.86:35956) 2024-04-16T05:38:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 -- 172.21.15.86:0/2577898633 learned_addr learned my addr 172.21.15.86:0/2577898633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869a79e640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f86800a3d50 0x7f86800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 --2- 172.21.15.86:0/2577898633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f8699f9d640 1 --2- 172.21.15.86:0/2577898633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d08c5e0 0x55ea2d094c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 -- 172.21.15.86:0/2577898633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d08c5e0 msgr2=0x55ea2d094c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 --2- 172.21.15.86:0/2577898633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d08c5e0 0x55ea2d094c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f868affd640 1 -- 172.21.15.86:0/2903848751 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f868c0d1280 con 0x7f8694103510 2024-04-16T05:38:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 -- 172.21.15.86:0/2577898633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08bcc0 msgr2=0x55ea2d08c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 --2- 172.21.15.86:0/2577898633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 -- 172.21.15.86:0/2577898633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea2cdcb7e0 con 0x55ea2d0951c0 2024-04-16T05:38:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.892+0000 7f869979c640 1 --2- 172.21.15.86:0/2577898633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d0951c0 0x55ea2d0975b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f868400cfd0 tx=0x7f868400c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b66777ffb01bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f869a79e640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f86800a3d50 0x7f86800a6210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f868c09cb70 tx=0x7f868c06e8b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f867ffff640 1 -- 172.21.15.86:0/2577898633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684019030 con 0x55ea2d0951c0 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f867ffff640 1 -- 172.21.15.86:0/2577898633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8684005500 con 0x55ea2d0951c0 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f867ffff640 1 -- 172.21.15.86:0/2577898633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684011780 con 0x55ea2d0951c0 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f869ede2740 1 -- 172.21.15.86:0/2577898633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d0951c0 msgr2=0x55ea2d0975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f869ede2740 1 --2- 172.21.15.86:0/2577898633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d0951c0 0x55ea2d0975b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f868400cfd0 tx=0x7f868400c670 comp rx=0 tx=0).stop 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f869ede2740 1 -- 172.21.15.86:0/2577898633 shutdown_connections 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f869ede2740 1 --2- 172.21.15.86:0/2577898633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d0951c0 0x55ea2d0975b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f869ede2740 1 --2- 172.21.15.86:0/2577898633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d08c5e0 0x55ea2d094c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f869ede2740 1 --2- 172.21.15.86:0/2577898633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f869ede2740 1 -- 172.21.15.86:0/2577898633 >> 172.21.15.86:0/2577898633 conn(0x55ea2d0863f0 msgr2=0x55ea2d0867d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:40.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.893+0000 7f869ede2740 1 -- 172.21.15.86:0/2577898633 shutdown_connections 2024-04-16T05:38:40.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 1 -- 172.21.15.86:0/2577898633 wait complete. 2024-04-16T05:38:40.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 1 -- start start 2024-04-16T05:38:40.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 1 Processor -- start 2024-04-16T05:38:40.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d092290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 0x55ea2d0927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d0951c0 0x55ea2d092d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea2d097fb0 con 0x55ea2d08bcc0 2024-04-16T05:38:40.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ea2cf79410 con 0x55ea2d08c5e0 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea2d098130 con 0x55ea2d0951c0 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869ede2740 10 client.0 _ll_register_callbacks cb 0x55ea2cf74b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869979c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d092290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869979c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d092290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35968/0 (socket says 172.21.15.86:35968) 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.894+0000 7f869979c640 1 -- 172.21.15.86:0/1371835422 learned_addr learned my addr 172.21.15.86:0/1371835422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f8699f9d640 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 0x55ea2d0927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869979c640 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d0951c0 0x55ea2d092d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869979c640 1 -- 172.21.15.86:0/1371835422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d0951c0 msgr2=0x55ea2d092d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869979c640 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d0951c0 0x55ea2d092d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869979c640 1 -- 172.21.15.86:0/1371835422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 msgr2=0x55ea2d0927d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869979c640 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 0x55ea2d0927d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869979c640 1 -- 172.21.15.86:0/1371835422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea2ce2bf70 con 0x55ea2d08bcc0 2024-04-16T05:38:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869979c640 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d092290 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f868400c460 tx=0x7f8684004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb79f16377c8f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f867d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f867d7fa640 1 -- 172.21.15.86:0/1371835422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684026070 con 0x55ea2d08bcc0 2024-04-16T05:38:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869ede2740 1 -- 172.21.15.86:0/1371835422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8684005500 con 0x55ea2d08bcc0 2024-04-16T05:38:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f867d7fa640 1 -- 172.21.15.86:0/1371835422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8684011520 con 0x55ea2d08bcc0 2024-04-16T05:38:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f867d7fa640 1 -- 172.21.15.86:0/1371835422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f868402f050 con 0x55ea2d08bcc0 2024-04-16T05:38:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869ede2740 10 client.16782 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:40.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f869ede2740 1 -- 172.21.15.86:0/1371835422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea2ccc3170 con 0x55ea2d08bcc0 2024-04-16T05:38:40.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.895+0000 7f867d7fa640 1 -- 172.21.15.86:0/1371835422 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8684005500 con 0x55ea2d08bcc0 2024-04-16T05:38:40.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f867d7fa640 1 -- 172.21.15.86:0/1371835422 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8684029020 con 0x55ea2d08bcc0 2024-04-16T05:38:40.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f869ede2740 10 client.16782 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:40.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f869ede2740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:40.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f869ede2740 20 client.16782 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:40.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f869ede2740 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ea2d0a54c0 0x55ea2d0a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f869ede2740 4 client.16782 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:40.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f869ede2740 1 -- 172.21.15.86:0/1371835422 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea2cffe8b0 con 0x55ea2d0a54c0 2024-04-16T05:38:40.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f8699f9d640 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ea2d0a54c0 0x55ea2d0a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f867d7fa640 10 client.16782 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:40.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.896+0000 7f8699f9d640 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ea2d0a54c0 0x55ea2d0a78b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53591eecb2d49bf server_cookie=892d2747422907c6 in_seq=0 out_seq=0 2024-04-16T05:38:40.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f867d7fa640 1 -- 172.21.15.86:0/1371835422 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea2cffe8b0 con 0x55ea2d0a54c0 2024-04-16T05:38:40.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f867d7fa640 10 client.16782 handle_command_reply: tid=0 2024-04-16T05:38:40.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 client.16782 shutdown 2024-04-16T05:38:40.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 -- 172.21.15.86:0/1371835422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 msgr2=0x55ea2d092290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:40.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d092290 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f868400c460 tx=0x7f8684004df0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 -- 172.21.15.86:0/1371835422 shutdown_connections 2024-04-16T05:38:40.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ea2d0a54c0 0x55ea2d0a78b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d0951c0 0x55ea2d092d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 0x55ea2d0927d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f867d7fa640 0 client.16782 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:40.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 --2- 172.21.15.86:0/1371835422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d092290 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 -- 172.21.15.86:0/1371835422 >> 172.21.15.86:0/1371835422 conn(0x55ea2d0863f0 msgr2=0x55ea2d088140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:40.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.897+0000 7f869ede2740 1 -- 172.21.15.86:0/1371835422 shutdown_connections 2024-04-16T05:38:40.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.898+0000 7f869ede2740 1 -- 172.21.15.86:0/1371835422 wait complete. 2024-04-16T05:38:40.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.898+0000 7f869ede2740 20 client.16782 trim_cache size 0 max 16384 2024-04-16T05:38:40.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f869ede2740 1 Processor -- start 2024-04-16T05:38:40.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f869ede2740 1 -- start start 2024-04-16T05:38:40.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f869ede2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d08bcc0 0x55ea2d0a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f869ede2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08c5e0 0x55ea2d092b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f869ede2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d0951c0 0x55ea2d085450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f869ede2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86941b1410 con 0x55ea2d08c5e0 2024-04-16T05:38:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f869ede2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f86941b2f50 con 0x55ea2d0951c0 2024-04-16T05:38:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f869ede2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86941b1ae0 con 0x55ea2d08bcc0 2024-04-16T05:38:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f8699f9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08c5e0 0x55ea2d092b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f8699f9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08c5e0 0x55ea2d092b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35970/0 (socket says 172.21.15.86:35970) 2024-04-16T05:38:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f8699f9d640 1 -- 172.21.15.86:0/3136174096 learned_addr learned my addr 172.21.15.86:0/3136174096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.915+0000 7f869979c640 1 --2- 172.21.15.86:0/3136174096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d0951c0 0x55ea2d085450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f8699f9d640 1 -- 172.21.15.86:0/3136174096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d08bcc0 msgr2=0x55ea2d0a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f8699f9d640 1 --2- 172.21.15.86:0/3136174096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d08bcc0 0x55ea2d0a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f8699f9d640 1 -- 172.21.15.86:0/3136174096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d0951c0 msgr2=0x55ea2d085450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f8699f9d640 1 --2- 172.21.15.86:0/3136174096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d0951c0 0x55ea2d085450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f8699f9d640 1 -- 172.21.15.86:0/3136174096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea2d097920 con 0x55ea2d08c5e0 2024-04-16T05:38:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f869979c640 1 --2- 172.21.15.86:0/3136174096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d0951c0 0x55ea2d085450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f8699f9d640 1 --2- 172.21.15.86:0/3136174096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08c5e0 0x55ea2d092b30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8690035000 tx=0x7f8690000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b8bb1aa52654b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f867f7fe640 1 -- 172.21.15.86:0/3136174096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690004340 con 0x55ea2d08c5e0 2024-04-16T05:38:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f867f7fe640 1 -- 172.21.15.86:0/3136174096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869002cd60 con 0x55ea2d08c5e0 2024-04-16T05:38:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f867f7fe640 1 -- 172.21.15.86:0/3136174096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690004920 con 0x55ea2d08c5e0 2024-04-16T05:38:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f869ede2740 1 -- 172.21.15.86:0/3136174096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08c5e0 msgr2=0x55ea2d092b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.916+0000 7f869ede2740 1 --2- 172.21.15.86:0/3136174096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08c5e0 0x55ea2d092b30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8690035000 tx=0x7f8690000bc0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 -- 172.21.15.86:0/3136174096 shutdown_connections 2024-04-16T05:38:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 --2- 172.21.15.86:0/3136174096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d0951c0 0x55ea2d085450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 --2- 172.21.15.86:0/3136174096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08c5e0 0x55ea2d092b30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 --2- 172.21.15.86:0/3136174096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d08bcc0 0x55ea2d0a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 -- 172.21.15.86:0/3136174096 >> 172.21.15.86:0/3136174096 conn(0x55ea2d0863f0 msgr2=0x55ea2d0880e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 -- 172.21.15.86:0/3136174096 shutdown_connections 2024-04-16T05:38:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 -- 172.21.15.86:0/3136174096 wait complete. 2024-04-16T05:38:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 -- start start 2024-04-16T05:38:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 Processor -- start 2024-04-16T05:38:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 0x55ea2d08f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d0951c0 0x55ea2d10a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86941b2880 con 0x55ea2d08bcc0 2024-04-16T05:38:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f86941b21b0 con 0x55ea2d08c5e0 2024-04-16T05:38:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86941af3d0 con 0x55ea2d0951c0 2024-04-16T05:38:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869ede2740 10 client.0 _ll_register_callbacks cb 0x7f8684013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869979c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869979c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35980/0 (socket says 172.21.15.86:35980) 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.917+0000 7f869979c640 1 -- 172.21.15.86:0/3675941929 learned_addr learned my addr 172.21.15.86:0/3675941929 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f8699f9d640 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 0x55ea2d08f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869979c640 1 -- 172.21.15.86:0/3675941929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d0951c0 msgr2=0x55ea2d10a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869979c640 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d0951c0 0x55ea2d10a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869979c640 1 -- 172.21.15.86:0/3675941929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 msgr2=0x55ea2d08f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869979c640 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 0x55ea2d08f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869979c640 1 -- 172.21.15.86:0/3675941929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea2d10adf0 con 0x55ea2d08bcc0 2024-04-16T05:38:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f8699f9d640 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 0x55ea2d08f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869979c640 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08f330 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f868401e9d0 tx=0x7f8684014f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ad671d8de74d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f867d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f867d7fa640 1 -- 172.21.15.86:0/3675941929 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684037070 con 0x55ea2d08bcc0 2024-04-16T05:38:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869ede2740 1 -- 172.21.15.86:0/3675941929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f869002cd60 con 0x55ea2d08bcc0 2024-04-16T05:38:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f867d7fa640 1 -- 172.21.15.86:0/3675941929 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8684005500 con 0x55ea2d08bcc0 2024-04-16T05:38:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f867d7fa640 1 -- 172.21.15.86:0/3675941929 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8684043050 con 0x55ea2d08bcc0 2024-04-16T05:38:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869ede2740 10 client.16788 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869ede2740 1 -- 172.21.15.86:0/3675941929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea2d10bee0 con 0x55ea2d08bcc0 2024-04-16T05:38:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f867d7fa640 1 -- 172.21.15.86:0/3675941929 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869002cd60 con 0x55ea2d08bcc0 2024-04-16T05:38:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f867d7fa640 1 -- 172.21.15.86:0/3675941929 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8684029050 con 0x55ea2d08bcc0 2024-04-16T05:38:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869ede2740 10 client.16788 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869ede2740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.918+0000 7f869ede2740 20 client.16788 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.919+0000 7f869ede2740 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ea2d10c5e0 0x55ea2d10ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.919+0000 7f869ede2740 4 client.16788 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:38:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.919+0000 7f869ede2740 1 -- 172.21.15.86:0/3675941929 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ea2cff8720 con 0x55ea2d10c5e0 2024-04-16T05:38:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.919+0000 7f8699f9d640 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ea2d10c5e0 0x55ea2d10ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.919+0000 7f867d7fa640 10 client.16788 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:40.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.919+0000 7f8699f9d640 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ea2d10c5e0 0x55ea2d10ea00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df5fed6ea860c4f server_cookie=cb949163df5b9438 in_seq=0 out_seq=0 2024-04-16T05:38:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.920+0000 7f867d7fa640 1 -- 172.21.15.86:0/3675941929 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ea2cff8720 con 0x55ea2d10c5e0 2024-04-16T05:38:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.920+0000 7f867d7fa640 10 client.16788 handle_command_reply: tid=0 2024-04-16T05:38:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.920+0000 7f869ede2740 1 client.16788 shutdown 2024-04-16T05:38:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.920+0000 7f869ede2740 1 -- 172.21.15.86:0/3675941929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 msgr2=0x55ea2d08f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.920+0000 7f869ede2740 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08f330 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f868401e9d0 tx=0x7f8684014f30 comp rx=0 tx=0).stop 2024-04-16T05:38:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.920+0000 7f869ede2740 1 -- 172.21.15.86:0/3675941929 shutdown_connections 2024-04-16T05:38:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.920+0000 7f869ede2740 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ea2d10c5e0 0x55ea2d10ea00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.920+0000 7f869ede2740 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea2d0951c0 0x55ea2d10a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869ede2740 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ea2d08c5e0 0x55ea2d08f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f867d7fa640 0 client.16788 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869ede2740 1 --2- 172.21.15.86:0/3675941929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea2d08bcc0 0x55ea2d08f330 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869ede2740 1 -- 172.21.15.86:0/3675941929 >> 172.21.15.86:0/3675941929 conn(0x55ea2d0863f0 msgr2=0x55ea2d091090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869ede2740 1 -- 172.21.15.86:0/3675941929 shutdown_connections 2024-04-16T05:38:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869ede2740 1 -- 172.21.15.86:0/3675941929 wait complete. 2024-04-16T05:38:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869ede2740 20 client.16788 trim_cache size 0 max 16384 2024-04-16T05:38:40.929 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:38:40.929 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:38:40.929 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "e48bee4e-bd81-4bc8-acbb-0e9791c865c0", 2024-04-16T05:38:40.929 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:38:40.929 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:38:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 -- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f86800a3d50 msgr2=0x7f86800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f86800a3d50 0x7f86800a6210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f868c09cb70 tx=0x7f868c06e8b0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 -- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694103510 msgr2=0x7f86941a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694103510 0x7f86941a6960 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f868c00cce0 tx=0x7f868c00a620 comp rx=0 tx=0).stop 2024-04-16T05:38:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 -- 172.21.15.86:0/2903848751 shutdown_connections 2024-04-16T05:38:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8694103510 0x7f86941a6960 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f86800a3d50 0x7f86800a6210 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86941060d0 0x7f8694102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 --2- 172.21.15.86:0/2903848751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8694101000 0x7f8694102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 -- 172.21.15.86:0/2903848751 >> 172.21.15.86:0/2903848751 conn(0x7f86940fc880 msgr2=0x7f86940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 -- 172.21.15.86:0/2903848751 shutdown_connections 2024-04-16T05:38:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:40.921+0000 7f869af9f640 1 -- 172.21.15.86:0/2903848751 wait complete. 2024-04-16T05:38:40.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:38:40.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:38:40.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:40 smithi086 ceph-mon[32028]: pgmap v104: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:38:40.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:40 smithi196 ceph-mon[36373]: pgmap v104: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:38:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.040+0000 7f70e3d15640 1 -- 172.21.15.86:0/460727959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 msgr2=0x7f70dc100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.040+0000 7f70e3d15640 1 --2- 172.21.15.86:0/460727959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 0x7f70dc100ad0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00aa40 tx=0x7f70cc02cf50 comp rx=0 tx=0).stop 2024-04-16T05:38:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.040+0000 7f70e3d15640 1 -- 172.21.15.86:0/460727959 shutdown_connections 2024-04-16T05:38:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.040+0000 7f70e3d15640 1 --2- 172.21.15.86:0/460727959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70dc101010 0x7f70dc10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.040+0000 7f70e3d15640 1 --2- 172.21.15.86:0/460727959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 0x7f70dc100ad0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.040+0000 7f70e3d15640 1 --2- 172.21.15.86:0/460727959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc106660 0x7f70dc106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.040+0000 7f70e3d15640 1 -- 172.21.15.86:0/460727959 >> 172.21.15.86:0/460727959 conn(0x7f70dc077490 msgr2=0x7f70dc0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 -- 172.21.15.86:0/460727959 shutdown_connections 2024-04-16T05:38:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 -- 172.21.15.86:0/460727959 wait complete. 2024-04-16T05:38:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 Processor -- start 2024-04-16T05:38:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 -- start start 2024-04-16T05:38:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 0x7f70dc19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc101010 0x7f70dc19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70dc106660 0x7f70dc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc110190 con 0x7f70dc100650 2024-04-16T05:38:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc110010 con 0x7f70dc106660 2024-04-16T05:38:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e3d15640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc110310 con 0x7f70dc101010 2024-04-16T05:38:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e1ab1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 0x7f70dc19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e1ab1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 0x7f70dc19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36006/0 (socket says 172.21.15.86:36006) 2024-04-16T05:38:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.041+0000 7f70e1ab1640 1 -- 172.21.15.86:0/748187262 learned_addr learned my addr 172.21.15.86:0/748187262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e12b0640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc101010 0x7f70dc19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e1ab1640 1 -- 172.21.15.86:0/748187262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc101010 msgr2=0x7f70dc19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc101010 0x7f70dc19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e22b2640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70dc106660 0x7f70dc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e1ab1640 1 -- 172.21.15.86:0/748187262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70dc106660 msgr2=0x7f70dc198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70dc106660 0x7f70dc198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e1ab1640 1 -- 172.21.15.86:0/748187262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc198fe0 con 0x7f70dc100650 2024-04-16T05:38:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e12b0640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc101010 0x7f70dc19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e22b2640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70dc106660 0x7f70dc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 0x7f70dc19e660 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b4a0 tx=0x7f70d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70d2ffd640 1 -- 172.21.15.86:0/748187262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8014070 con 0x7f70dc100650 2024-04-16T05:38:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70d2ffd640 1 -- 172.21.15.86:0/748187262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d8010af0 con 0x7f70dc100650 2024-04-16T05:38:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e3d15640 1 -- 172.21.15.86:0/748187262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc1992d0 con 0x7f70dc100650 2024-04-16T05:38:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70d2ffd640 1 -- 172.21.15.86:0/748187262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8017d10 con 0x7f70dc100650 2024-04-16T05:38:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.042+0000 7f70e3d15640 1 -- 172.21.15.86:0/748187262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc1a55e0 con 0x7f70dc100650 2024-04-16T05:38:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.044+0000 7f70d2ffd640 1 -- 172.21.15.86:0/748187262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f70d8017070 con 0x7f70dc100650 2024-04-16T05:38:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.044+0000 7f70e69dd740 1 Processor -- start 2024-04-16T05:38:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e69dd740 1 -- start start 2024-04-16T05:38:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac76e50 0x55bb3ac77230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac77770 0x55bb3ac7fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac80350 0x55bb3ac82740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e69dd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb3abdc180 con 0x55bb3ac80350 2024-04-16T05:38:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e69dd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ab79360 con 0x55bb3ac77770 2024-04-16T05:38:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e69dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb3abd8140 con 0x55bb3ac76e50 2024-04-16T05:38:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e12b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac80350 0x55bb3ac82740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e12b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac80350 0x55bb3ac82740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36008/0 (socket says 172.21.15.86:36008) 2024-04-16T05:38:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e12b0640 1 -- 172.21.15.86:0/3609461474 learned_addr learned my addr 172.21.15.86:0/3609461474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e22b2640 1 --2- 172.21.15.86:0/3609461474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac76e50 0x55bb3ac77230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70d2ffd640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f70c80a3d20 0x7f70c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/3609461474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac77770 0x55bb3ac7fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e12b0640 1 -- 172.21.15.86:0/3609461474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac76e50 msgr2=0x55bb3ac77230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e12b0640 1 --2- 172.21.15.86:0/3609461474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac76e50 0x55bb3ac77230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e22b2640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f70c80a3d20 0x7f70c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e12b0640 1 -- 172.21.15.86:0/3609461474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac77770 msgr2=0x55bb3ac7fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e12b0640 1 --2- 172.21.15.86:0/3609461474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac77770 0x55bb3ac7fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.045+0000 7f70e12b0640 1 -- 172.21.15.86:0/3609461474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3a9b87d0 con 0x55bb3ac80350 2024-04-16T05:38:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.046+0000 7f70d2ffd640 1 -- 172.21.15.86:0/748187262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f70d80cfe90 con 0x7f70dc100650 2024-04-16T05:38:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.046+0000 7f70e12b0640 1 --2- 172.21.15.86:0/3609461474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac80350 0x55bb3ac82740 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f70cc037040 tx=0x7f70cc0354c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf8cc55067b3ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.046+0000 7f70c7fff640 1 -- 172.21.15.86:0/3609461474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc02db00 con 0x55bb3ac80350 2024-04-16T05:38:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.046+0000 7f70e22b2640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f70c80a3d20 0x7f70c80a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f70d4005fd0 tx=0x7f70d4004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.046+0000 7f70c7fff640 1 -- 172.21.15.86:0/3609461474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70cc02dca0 con 0x55bb3ac80350 2024-04-16T05:38:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.046+0000 7f70c7fff640 1 -- 172.21.15.86:0/3609461474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc004920 con 0x55bb3ac80350 2024-04-16T05:38:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 -- 172.21.15.86:0/3609461474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac80350 msgr2=0x55bb3ac82740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 --2- 172.21.15.86:0/3609461474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac80350 0x55bb3ac82740 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f70cc037040 tx=0x7f70cc0354c0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 -- 172.21.15.86:0/3609461474 shutdown_connections 2024-04-16T05:38:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 --2- 172.21.15.86:0/3609461474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac80350 0x55bb3ac82740 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 --2- 172.21.15.86:0/3609461474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac77770 0x55bb3ac7fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 --2- 172.21.15.86:0/3609461474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac76e50 0x55bb3ac77230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 -- 172.21.15.86:0/3609461474 >> 172.21.15.86:0/3609461474 conn(0x55bb3ac71580 msgr2=0x55bb3ac71960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 -- 172.21.15.86:0/3609461474 shutdown_connections 2024-04-16T05:38:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 -- 172.21.15.86:0/3609461474 wait complete. 2024-04-16T05:38:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 -- start start 2024-04-16T05:38:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 Processor -- start 2024-04-16T05:38:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.047+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac7d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac80350 0x55bb3ac7dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e69dd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ac830e0 con 0x55bb3ac76e50 2024-04-16T05:38:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e69dd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ab952c0 con 0x55bb3ac80350 2024-04-16T05:38:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e69dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ac83260 con 0x55bb3ac77770 2024-04-16T05:38:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e69dd740 10 client.0 _ll_register_callbacks cb 0x55bb3ab7c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e1ab1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e1ab1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36010/0 (socket says 172.21.15.86:36010) 2024-04-16T05:38:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e1ab1640 1 -- 172.21.15.86:0/1456530184 learned_addr learned my addr 172.21.15.86:0/1456530184 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e12b0640 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac7d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e22b2640 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac80350 0x55bb3ac7dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e1ab1640 1 -- 172.21.15.86:0/1456530184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 msgr2=0x55bb3ac7d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac7d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e1ab1640 1 -- 172.21.15.86:0/1456530184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac80350 msgr2=0x55bb3ac7dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac80350 0x55bb3ac7dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e1ab1640 1 -- 172.21.15.86:0/1456530184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3a8ee3f0 con 0x55bb3ac76e50 2024-04-16T05:38:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e12b0640 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac7d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e22b2640 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac80350 0x55bb3ac7dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7d2d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f70d800be60 tx=0x7f70d809b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cfc2f632d35c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70c57fa640 1 -- 172.21.15.86:0/1456530184 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d801e070 con 0x55bb3ac76e50 2024-04-16T05:38:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70c57fa640 1 -- 172.21.15.86:0/1456530184 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f70d8010ca0 con 0x55bb3ac76e50 2024-04-16T05:38:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e69dd740 1 -- 172.21.15.86:0/1456530184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70cc02dca0 con 0x55bb3ac76e50 2024-04-16T05:38:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70c57fa640 1 -- 172.21.15.86:0/1456530184 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70d8009230 con 0x55bb3ac76e50 2024-04-16T05:38:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e69dd740 10 client.16800 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70e69dd740 1 -- 172.21.15.86:0/1456530184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb3a8ae0f0 con 0x55bb3ac76e50 2024-04-16T05:38:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.048+0000 7f70c57fa640 1 -- 172.21.15.86:0/1456530184 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70cc02dca0 con 0x55bb3ac76e50 2024-04-16T05:38:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70c57fa640 1 -- 172.21.15.86:0/1456530184 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f70d80cd4f0 con 0x55bb3ac76e50 2024-04-16T05:38:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70e69dd740 10 client.16800 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70e69dd740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70e69dd740 20 client.16800 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70e69dd740 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb3ac90500 0x55bb3ac92950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70e69dd740 4 client.16800 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70e69dd740 1 -- 172.21.15.86:0/1456530184 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb3abe9ba0 con 0x55bb3ac90500 2024-04-16T05:38:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70e12b0640 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb3ac90500 0x55bb3ac92950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70e12b0640 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb3ac90500 0x55bb3ac92950 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa4c7e7afed9536f server_cookie=e845c60ed88a795 in_seq=0 out_seq=0 2024-04-16T05:38:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.049+0000 7f70c57fa640 10 client.16800 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70c57fa640 1 -- 172.21.15.86:0/1456530184 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb3abe9ba0 con 0x55bb3ac90500 2024-04-16T05:38:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70c57fa640 10 client.16800 handle_command_reply: tid=0 2024-04-16T05:38:41.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 client.16800 shutdown 2024-04-16T05:38:41.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 -- 172.21.15.86:0/1456530184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 msgr2=0x55bb3ac7d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:41.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7d2d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f70d800be60 tx=0x7f70d809b530 comp rx=0 tx=0).stop 2024-04-16T05:38:41.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 -- 172.21.15.86:0/1456530184 shutdown_connections 2024-04-16T05:38:41.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb3ac90500 0x55bb3ac92950 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac80350 0x55bb3ac7dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac7d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 --2- 172.21.15.86:0/1456530184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7d2d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70c57fa640 0 client.16800 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:41.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 -- 172.21.15.86:0/1456530184 >> 172.21.15.86:0/1456530184 conn(0x55bb3ac71580 msgr2=0x55bb3ac732d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:41.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 -- 172.21.15.86:0/1456530184 shutdown_connections 2024-04-16T05:38:41.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 1 -- 172.21.15.86:0/1456530184 wait complete. 2024-04-16T05:38:41.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.050+0000 7f70e69dd740 20 client.16800 trim_cache size 0 max 16384 2024-04-16T05:38:41.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e69dd740 1 Processor -- start 2024-04-16T05:38:41.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e69dd740 1 -- start start 2024-04-16T05:38:41.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac71580 0x55bb3ac8bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac98220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e69dd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1af4d0 con 0x55bb3ac71580 2024-04-16T05:38:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e69dd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1b1010 con 0x55bb3ac76e50 2024-04-16T05:38:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e69dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1afba0 con 0x55bb3ac77770 2024-04-16T05:38:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e1ab1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac71580 0x55bb3ac8bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e1ab1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac71580 0x55bb3ac8bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36012/0 (socket says 172.21.15.86:36012) 2024-04-16T05:38:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e1ab1640 1 -- 172.21.15.86:0/2459388129 learned_addr learned my addr 172.21.15.86:0/2459388129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.068+0000 7f70e12b0640 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70e22b2640 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac98220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70e1ab1640 1 -- 172.21.15.86:0/2459388129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 msgr2=0x55bb3ac98220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:41.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac98220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70e1ab1640 1 -- 172.21.15.86:0/2459388129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac76e50 msgr2=0x55bb3ac7db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:41.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70e1ab1640 1 -- 172.21.15.86:0/2459388129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3ac713b0 con 0x55bb3ac71580 2024-04-16T05:38:41.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70e12b0640 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:41.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac71580 0x55bb3ac8bfa0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f70d809f040 tx=0x7f70d809a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6005c4acb7f547 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:41.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70c77fe640 1 -- 172.21.15.86:0/2459388129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d801e070 con 0x55bb3ac71580 2024-04-16T05:38:41.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70c77fe640 1 -- 172.21.15.86:0/2459388129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d809c2d0 con 0x55bb3ac71580 2024-04-16T05:38:41.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.069+0000 7f70c77fe640 1 -- 172.21.15.86:0/2459388129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8010dc0 con 0x55bb3ac71580 2024-04-16T05:38:41.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 -- 172.21.15.86:0/2459388129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac71580 msgr2=0x55bb3ac8bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:41.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac71580 0x55bb3ac8bfa0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f70d809f040 tx=0x7f70d809a0e0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 -- 172.21.15.86:0/2459388129 shutdown_connections 2024-04-16T05:38:41.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac98220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac76e50 0x55bb3ac7db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 --2- 172.21.15.86:0/2459388129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac71580 0x55bb3ac8bfa0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 -- 172.21.15.86:0/2459388129 >> 172.21.15.86:0/2459388129 conn(0x55bb3ac7bf10 msgr2=0x55bb3ac70760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:41.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 -- 172.21.15.86:0/2459388129 shutdown_connections 2024-04-16T05:38:41.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 -- 172.21.15.86:0/2459388129 wait complete. 2024-04-16T05:38:41.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 -- start start 2024-04-16T05:38:41.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 Processor -- start 2024-04-16T05:38:41.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac71580 0x55bb3ac86190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac866d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac86c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1b0940 con 0x55bb3ac76e50 2024-04-16T05:38:41.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1b0270 con 0x55bb3ac71580 2024-04-16T05:38:41.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1ad490 con 0x55bb3ac77770 2024-04-16T05:38:41.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.070+0000 7f70e69dd740 10 client.0 _ll_register_callbacks cb 0x7f70d8017eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:41.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e12b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac866d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e12b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac866d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36026/0 (socket says 172.21.15.86:36026) 2024-04-16T05:38:41.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e12b0640 1 -- 172.21.15.86:0/224356859 learned_addr learned my addr 172.21.15.86:0/224356859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:38:41.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e22b2640 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac86c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e12b0640 1 -- 172.21.15.86:0/224356859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 msgr2=0x55bb3ac86c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:41.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac71580 0x55bb3ac86190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e12b0640 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac86c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e12b0640 1 -- 172.21.15.86:0/224356859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac71580 msgr2=0x55bb3ac86190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:41.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e12b0640 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac71580 0x55bb3ac86190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e12b0640 1 -- 172.21.15.86:0/224356859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3acf5d40 con 0x55bb3ac76e50 2024-04-16T05:38:41.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e12b0640 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac866d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005ab0 tx=0x7f70cc004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4d64709ec85d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:41.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:38:41.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70c57fa640 1 -- 172.21.15.86:0/224356859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc053060 con 0x55bb3ac76e50 2024-04-16T05:38:41.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70c57fa640 1 -- 172.21.15.86:0/224356859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f70cc035c70 con 0x55bb3ac76e50 2024-04-16T05:38:41.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e69dd740 1 -- 172.21.15.86:0/224356859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d809c2d0 con 0x55bb3ac76e50 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70c57fa640 1 -- 172.21.15.86:0/224356859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70cc05a050 con 0x55bb3ac76e50 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e69dd740 10 client.16806 fetch_fsmap learned FSMap version 29 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70e69dd740 1 -- 172.21.15.86:0/224356859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb3acf7050 con 0x55bb3ac76e50 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.071+0000 7f70c57fa640 1 -- 172.21.15.86:0/224356859 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d809c2d0 con 0x55bb3ac76e50 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70c57fa640 1 -- 172.21.15.86:0/224356859 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f70cc00b040 con 0x55bb3ac76e50 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e69dd740 10 client.16806 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e69dd740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e69dd740 20 client.16806 populate_metadata read hostname 'smithi086' 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e69dd740 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb3ac80350 0x55bb3acf93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:41.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e69dd740 4 client.16806 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:38:41.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e69dd740 1 -- 172.21.15.86:0/224356859 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb3abe3a10 con 0x55bb3ac80350 2024-04-16T05:38:41.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb3ac80350 0x55bb3acf93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:41.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70c57fa640 10 client.16806 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:41.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e1ab1640 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb3ac80350 0x55bb3acf93f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c190bfef061532f server_cookie=19a61091647b9a85 in_seq=0 out_seq=0 2024-04-16T05:38:41.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70c57fa640 1 -- 172.21.15.86:0/224356859 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bb3abe3a10 con 0x55bb3ac80350 2024-04-16T05:38:41.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70c57fa640 10 client.16806 handle_command_reply: tid=0 2024-04-16T05:38:41.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e69dd740 1 client.16806 shutdown 2024-04-16T05:38:41.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.072+0000 7f70e69dd740 1 -- 172.21.15.86:0/224356859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 msgr2=0x55bb3ac866d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:41.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac866d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005ab0 tx=0x7f70cc004370 comp rx=0 tx=0).stop 2024-04-16T05:38:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 1 -- 172.21.15.86:0/224356859 shutdown_connections 2024-04-16T05:38:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb3ac80350 0x55bb3acf93f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb3ac77770 0x55bb3ac86c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3ac76e50 0x55bb3ac866d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70c57fa640 0 client.16806 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:38:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 1 --2- 172.21.15.86:0/224356859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb3ac71580 0x55bb3ac86190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 1 -- 172.21.15.86:0/224356859 >> 172.21.15.86:0/224356859 conn(0x55bb3ac7bf10 msgr2=0x55bb3ac70760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 1 -- 172.21.15.86:0/224356859 shutdown_connections 2024-04-16T05:38:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 1 -- 172.21.15.86:0/224356859 wait complete. 2024-04-16T05:38:41.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e69dd740 20 client.16806 trim_cache size 0 max 16384 2024-04-16T05:38:41.082 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:38:41.082 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:38:41.082 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:38:41.082 INFO:teuthology.orchestra.run.smithi086.stdout: "e48bee4e-bd81-4bc8-acbb-0e9791c865c0": { 2024-04-16T05:38:41.082 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:38:41.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e48bee4e-bd81-4bc8-acbb-0e9791c865c0", 2024-04-16T05:38:41.083 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:38:41.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:38:41.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:38:41.083 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:38:41.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 -- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f70c80a3d20 msgr2=0x7f70c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:41.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f70c80a3d20 0x7f70c80a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f70d4005fd0 tx=0x7f70d4004ab0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 -- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 msgr2=0x7f70dc19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:41.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 0x7f70dc19e660 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b4a0 tx=0x7f70d800b960 comp rx=0 tx=0).stop 2024-04-16T05:38:41.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 -- 172.21.15.86:0/748187262 shutdown_connections 2024-04-16T05:38:41.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f70c80a3d20 0x7f70c80a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70dc106660 0x7f70dc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc101010 0x7f70dc19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 --2- 172.21.15.86:0/748187262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc100650 0x7f70dc19e660 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:41.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 -- 172.21.15.86:0/748187262 >> 172.21.15.86:0/748187262 conn(0x7f70dc077490 msgr2=0x7f70dc1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:41.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 -- 172.21.15.86:0/748187262 shutdown_connections 2024-04-16T05:38:41.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:38:41.073+0000 7f70e3d15640 1 -- 172.21.15.86:0/748187262 wait complete. 2024-04-16T05:38:41.086 INFO:tasks.cephfs.filesystem:scrub status for tag:e48bee4e-bd81-4bc8-acbb-0e9791c865c0 - {'path': '/', 'tag': 'e48bee4e-bd81-4bc8-acbb-0e9791c865c0', 'options': 'recursive,force'} 2024-04-16T05:38:41.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:40 smithi194 ceph-mon[35865]: pgmap v104: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:38:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:41 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:38:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:41 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:41 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:38:41.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:41 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:42.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:41 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:38:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:41 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:41 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:38:42.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:41 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:42.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:41 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:38:42.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:41 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:42.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:41 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:38:42.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:41 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:42.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:42 smithi196 ceph-mon[36373]: pgmap v105: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:43.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:42 smithi194 ceph-mon[35865]: pgmap v105: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:43.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:42 smithi086 ceph-mon[32028]: pgmap v105: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:44 smithi196 ceph-mon[36373]: pgmap v106: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:38:44.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:44 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:38:45.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:44 smithi194 ceph-mon[35865]: pgmap v106: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:38:45.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:44 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:38:45.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:44 smithi086 ceph-mon[32028]: pgmap v106: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:38:45.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:44 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:38:46.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:46 smithi196 ceph-mon[36373]: pgmap v107: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:38:47.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:46 smithi194 ceph-mon[35865]: pgmap v107: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:38:47.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:46 smithi086 ceph-mon[32028]: pgmap v107: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:38:48.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:48 smithi196 ceph-mon[36373]: pgmap v108: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:38:49.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:48 smithi194 ceph-mon[35865]: pgmap v108: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:38:49.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:48 smithi086 ceph-mon[32028]: pgmap v108: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:38:51.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:50 smithi194 ceph-mon[35865]: pgmap v109: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:38:51.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:50 smithi086 ceph-mon[32028]: pgmap v109: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:38:51.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:50 smithi196 ceph-mon[36373]: pgmap v109: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:38:53.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:52 smithi194 ceph-mon[35865]: pgmap v110: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:38:53.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:52 smithi086 ceph-mon[32028]: pgmap v110: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:38:53.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:52 smithi196 ceph-mon[36373]: pgmap v110: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:38:55.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:54 smithi194 ceph-mon[35865]: pgmap v111: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:38:55.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:54 smithi086 ceph-mon[32028]: pgmap v111: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:38:55.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:54 smithi196 ceph-mon[36373]: pgmap v111: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:38:57.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:56 smithi194 ceph-mon[35865]: pgmap v112: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:57.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:56 smithi086 ceph-mon[32028]: pgmap v112: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:57.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:56 smithi196 ceph-mon[36373]: pgmap v112: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:59.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:38:58 smithi194 ceph-mon[35865]: pgmap v113: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:59.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:38:58 smithi086 ceph-mon[32028]: pgmap v113: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:38:59.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:38:58 smithi196 ceph-mon[36373]: pgmap v113: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:01.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:00 smithi194 ceph-mon[35865]: pgmap v114: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:01.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:00 smithi086 ceph-mon[32028]: pgmap v114: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:01.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:00 smithi196 ceph-mon[36373]: pgmap v114: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:03.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:02 smithi194 ceph-mon[35865]: pgmap v115: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:03.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:02 smithi086 ceph-mon[32028]: pgmap v115: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:03.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:02 smithi196 ceph-mon[36373]: pgmap v115: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:05.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:04 smithi194 ceph-mon[35865]: pgmap v116: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:39:05.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:04 smithi086 ceph-mon[32028]: pgmap v116: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:39:05.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:04 smithi196 ceph-mon[36373]: pgmap v116: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:39:07.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:06 smithi194 ceph-mon[35865]: pgmap v117: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:07.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:06 smithi086 ceph-mon[32028]: pgmap v117: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:07.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:06 smithi196 ceph-mon[36373]: pgmap v117: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:09.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:08 smithi194 ceph-mon[35865]: pgmap v118: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:09.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:08 smithi086 ceph-mon[32028]: pgmap v118: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:09.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:08 smithi196 ceph-mon[36373]: pgmap v118: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:10.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:09 smithi194 ceph-mon[35865]: pgmap v119: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:39:10.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:09 smithi086 ceph-mon[32028]: pgmap v119: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:39:10.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:09 smithi196 ceph-mon[36373]: pgmap v119: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:39:11.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:39:11.088 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:39:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.208+0000 7feb701bd640 1 -- 172.21.15.86:0/3098138832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 msgr2=0x7feb68102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.208+0000 7feb701bd640 1 --2- 172.21.15.86:0/3098138832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 0x7feb68102620 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7feb64009900 tx=0x7feb6402bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.208+0000 7feb701bd640 1 -- 172.21.15.86:0/3098138832 shutdown_connections 2024-04-16T05:39:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.208+0000 7feb701bd640 1 --2- 172.21.15.86:0/3098138832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb68107310 0x7feb6810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.208+0000 7feb701bd640 1 --2- 172.21.15.86:0/3098138832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb68102bf0 0x7feb68106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.208+0000 7feb701bd640 1 --2- 172.21.15.86:0/3098138832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 0x7feb68102620 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.208+0000 7feb701bd640 1 -- 172.21.15.86:0/3098138832 >> 172.21.15.86:0/3098138832 conn(0x7feb680fda40 msgr2=0x7feb680ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.208+0000 7feb701bd640 1 -- 172.21.15.86:0/3098138832 shutdown_connections 2024-04-16T05:39:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.208+0000 7feb701bd640 1 -- 172.21.15.86:0/3098138832 wait complete. 2024-04-16T05:39:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb701bd640 1 Processor -- start 2024-04-16T05:39:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb701bd640 1 -- start start 2024-04-16T05:39:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb701bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 0x7feb6819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb701bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb68107310 0x7feb6819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb701bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb681986c0 0x7feb68198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb701bd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb68112420 con 0x7feb68102240 2024-04-16T05:39:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb701bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb681122a0 con 0x7feb68107310 2024-04-16T05:39:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb701bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb681125a0 con 0x7feb681986c0 2024-04-16T05:39:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb6df59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 0x7feb6819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb6df59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 0x7feb6819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58146/0 (socket says 172.21.15.86:58146) 2024-04-16T05:39:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb6df59640 1 -- 172.21.15.86:0/2199504490 learned_addr learned my addr 172.21.15.86:0/2199504490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb6e75a640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb681986c0 0x7feb68198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb6d758640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb68107310 0x7feb6819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb6df59640 1 -- 172.21.15.86:0/2199504490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb681986c0 msgr2=0x7feb68198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb6df59640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb681986c0 0x7feb68198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb6df59640 1 -- 172.21.15.86:0/2199504490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb68107310 msgr2=0x7feb6819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.209+0000 7feb6df59640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb68107310 0x7feb6819eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.210+0000 7feb6df59640 1 -- 172.21.15.86:0/2199504490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb681992c0 con 0x7feb68102240 2024-04-16T05:39:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.210+0000 7feb6df59640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 0x7feb6819e600 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7feb6402c240 tx=0x7feb64004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.210+0000 7feb5effd640 1 -- 172.21.15.86:0/2199504490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64043020 con 0x7feb68102240 2024-04-16T05:39:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.210+0000 7feb5effd640 1 -- 172.21.15.86:0/2199504490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb640054e0 con 0x7feb68102240 2024-04-16T05:39:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.210+0000 7feb701bd640 1 -- 172.21.15.86:0/2199504490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb68199550 con 0x7feb68102240 2024-04-16T05:39:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.210+0000 7feb5effd640 1 -- 172.21.15.86:0/2199504490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64003cc0 con 0x7feb68102240 2024-04-16T05:39:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.210+0000 7feb701bd640 1 -- 172.21.15.86:0/2199504490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb681a56f0 con 0x7feb68102240 2024-04-16T05:39:11.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.212+0000 7feb5effd640 1 -- 172.21.15.86:0/2199504490 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb64046020 con 0x7feb68102240 2024-04-16T05:39:11.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.212+0000 7feb72da9740 1 Processor -- start 2024-04-16T05:39:11.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb72da9740 1 -- start start 2024-04-16T05:39:11.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb72da9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea2e50 0x55fa8eea3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb72da9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 0x55fa8eeabe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb72da9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb72da9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa8ee08180 con 0x55fa8eeac350 2024-04-16T05:39:11.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb72da9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fa8eda5360 con 0x55fa8eea2e50 2024-04-16T05:39:11.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb72da9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa8ee04140 con 0x55fa8eea3770 2024-04-16T05:39:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6d758640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6d758640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58152/0 (socket says 172.21.15.86:58152) 2024-04-16T05:39:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6d758640 1 -- 172.21.15.86:0/1417436091 learned_addr learned my addr 172.21.15.86:0/1417436091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6e75a640 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea2e50 0x55fa8eea3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb5effd640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb540a3d20 0x7feb540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6df59640 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 0x55fa8eeabe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6d758640 1 -- 172.21.15.86:0/1417436091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 msgr2=0x55fa8eeabe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6d758640 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 0x55fa8eeabe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6d758640 1 -- 172.21.15.86:0/1417436091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea2e50 msgr2=0x55fa8eea3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6d758640 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea2e50 0x55fa8eea3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6d758640 1 -- 172.21.15.86:0/1417436091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8ebe47d0 con 0x55fa8eeac350 2024-04-16T05:39:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6df59640 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 0x55fa8eeabe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.213+0000 7feb6e75a640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb540a3d20 0x7feb540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.214+0000 7feb5effd640 1 -- 172.21.15.86:0/2199504490 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feb640f2c50 con 0x7feb68102240 2024-04-16T05:39:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.214+0000 7feb6d758640 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeae740 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7feb5800bae0 tx=0x7feb5800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407849b1b07c6a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.214+0000 7feb53fff640 1 -- 172.21.15.86:0/1417436091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58017420 con 0x55fa8eeac350 2024-04-16T05:39:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.214+0000 7feb53fff640 1 -- 172.21.15.86:0/1417436091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb58017aa0 con 0x55fa8eeac350 2024-04-16T05:39:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.214+0000 7feb53fff640 1 -- 172.21.15.86:0/1417436091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5800fae0 con 0x55fa8eeac350 2024-04-16T05:39:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 -- 172.21.15.86:0/1417436091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 msgr2=0x55fa8eeae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb6e75a640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb540a3d20 0x7feb540a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb60005fd0 tx=0x7feb60004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeae740 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7feb5800bae0 tx=0x7feb5800d340 comp rx=0 tx=0).stop 2024-04-16T05:39:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 -- 172.21.15.86:0/1417436091 shutdown_connections 2024-04-16T05:39:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeae740 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 0x55fa8eeabe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 --2- 172.21.15.86:0/1417436091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea2e50 0x55fa8eea3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 -- 172.21.15.86:0/1417436091 >> 172.21.15.86:0/1417436091 conn(0x55fa8ee9d580 msgr2=0x55fa8ee9d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 -- 172.21.15.86:0/1417436091 shutdown_connections 2024-04-16T05:39:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 -- 172.21.15.86:0/1417436091 wait complete. 2024-04-16T05:39:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 -- start start 2024-04-16T05:39:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.215+0000 7feb72da9740 1 Processor -- start 2024-04-16T05:39:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb72da9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea2e50 0x55fa8eea9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb72da9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea3770 0x55fa8eea98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb72da9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 0x55fa8eea9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb72da9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa8eeaf140 con 0x55fa8eea3770 2024-04-16T05:39:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb72da9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fa8edc12c0 con 0x55fa8eeac350 2024-04-16T05:39:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb72da9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa8eeaf2c0 con 0x55fa8eea2e50 2024-04-16T05:39:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb72da9740 10 client.0 _ll_register_callbacks cb 0x55fa8eda8090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6d758640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea3770 0x55fa8eea98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6d758640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea3770 0x55fa8eea98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58158/0 (socket says 172.21.15.86:58158) 2024-04-16T05:39:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6d758640 1 -- 172.21.15.86:0/2888021097 learned_addr learned my addr 172.21.15.86:0/2888021097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6e75a640 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 0x55fa8eea9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6df59640 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea2e50 0x55fa8eea9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6d758640 1 -- 172.21.15.86:0/2888021097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea2e50 msgr2=0x55fa8eea9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6d758640 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea2e50 0x55fa8eea9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6d758640 1 -- 172.21.15.86:0/2888021097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 msgr2=0x55fa8eea9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6d758640 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 0x55fa8eea9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6d758640 1 -- 172.21.15.86:0/2888021097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa8eb1a3f0 con 0x55fa8eea3770 2024-04-16T05:39:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb6d758640 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea3770 0x55fa8eea98c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7feb58019040 tx=0x7feb58017f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70f332ab1969465 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb517fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb517fa640 1 -- 172.21.15.86:0/2888021097 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58027020 con 0x55fa8eea3770 2024-04-16T05:39:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb517fa640 1 -- 172.21.15.86:0/2888021097 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feb58017830 con 0x55fa8eea3770 2024-04-16T05:39:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.216+0000 7feb72da9740 1 -- 172.21.15.86:0/2888021097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb58017aa0 con 0x55fa8eea3770 2024-04-16T05:39:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb517fa640 1 -- 172.21.15.86:0/2888021097 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb58032050 con 0x55fa8eea3770 2024-04-16T05:39:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb72da9740 10 client.16818 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb72da9740 1 -- 172.21.15.86:0/2888021097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa8eada0f0 con 0x55fa8eea3770 2024-04-16T05:39:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb517fa640 1 -- 172.21.15.86:0/2888021097 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb58017aa0 con 0x55fa8eea3770 2024-04-16T05:39:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb517fa640 1 -- 172.21.15.86:0/2888021097 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7feb58023960 con 0x55fa8eea3770 2024-04-16T05:39:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb72da9740 10 client.16818 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb72da9740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb72da9740 20 client.16818 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb72da9740 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa8eebc720 0x55fa8eebeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb72da9740 4 client.16818 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb72da9740 1 -- 172.21.15.86:0/2888021097 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa8ee15ba0 con 0x55fa8eebc720 2024-04-16T05:39:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.217+0000 7feb6df59640 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa8eebc720 0x55fa8eebeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.218+0000 7feb517fa640 10 client.16818 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.218+0000 7feb6df59640 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa8eebc720 0x55fa8eebeb10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2781ce70d87e49 server_cookie=3ca57e3c3770347f in_seq=0 out_seq=0 2024-04-16T05:39:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.218+0000 7feb517fa640 1 -- 172.21.15.86:0/2888021097 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa8ee15ba0 con 0x55fa8eebc720 2024-04-16T05:39:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.218+0000 7feb517fa640 10 client.16818 handle_command_reply: tid=0 2024-04-16T05:39:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.218+0000 7feb72da9740 1 client.16818 shutdown 2024-04-16T05:39:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.218+0000 7feb72da9740 1 -- 172.21.15.86:0/2888021097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea3770 msgr2=0x55fa8eea98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.218+0000 7feb72da9740 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea3770 0x55fa8eea98c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7feb58019040 tx=0x7feb58017f00 comp rx=0 tx=0).stop 2024-04-16T05:39:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb72da9740 1 -- 172.21.15.86:0/2888021097 shutdown_connections 2024-04-16T05:39:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb72da9740 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa8eebc720 0x55fa8eebeb10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb72da9740 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 0x55fa8eea9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb72da9740 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea3770 0x55fa8eea98c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb517fa640 0 client.16818 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb72da9740 1 --2- 172.21.15.86:0/2888021097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea2e50 0x55fa8eea9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb72da9740 1 -- 172.21.15.86:0/2888021097 >> 172.21.15.86:0/2888021097 conn(0x55fa8ee9d580 msgr2=0x55fa8ee9f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb72da9740 1 -- 172.21.15.86:0/2888021097 shutdown_connections 2024-04-16T05:39:11.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb72da9740 1 -- 172.21.15.86:0/2888021097 wait complete. 2024-04-16T05:39:11.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.219+0000 7feb72da9740 20 client.16818 trim_cache size 0 max 16384 2024-04-16T05:39:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb72da9740 1 Processor -- start 2024-04-16T05:39:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb72da9740 1 -- start start 2024-04-16T05:39:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb72da9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea2e50 0x55fa8eeaba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb72da9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 0x55fa8eeb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb72da9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 0x55fa8ee9c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb72da9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb681af2e0 con 0x55fa8eea2e50 2024-04-16T05:39:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb72da9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb681b0e20 con 0x55fa8eeac350 2024-04-16T05:39:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb72da9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb681af9b0 con 0x55fa8eea3770 2024-04-16T05:39:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6df59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea2e50 0x55fa8eeaba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6df59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea2e50 0x55fa8eeaba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58160/0 (socket says 172.21.15.86:58160) 2024-04-16T05:39:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6df59640 1 -- 172.21.15.86:0/510934350 learned_addr learned my addr 172.21.15.86:0/510934350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6df59640 1 -- 172.21.15.86:0/510934350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 msgr2=0x55fa8eeb8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6df59640 1 --2- 172.21.15.86:0/510934350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 0x55fa8eeb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6df59640 1 -- 172.21.15.86:0/510934350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 msgr2=0x55fa8ee9c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6e75a640 1 --2- 172.21.15.86:0/510934350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 0x55fa8ee9c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6df59640 1 --2- 172.21.15.86:0/510934350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 0x55fa8ee9c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6df59640 1 -- 172.21.15.86:0/510934350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8eeae740 con 0x55fa8eea2e50 2024-04-16T05:39:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.236+0000 7feb6e75a640 1 --2- 172.21.15.86:0/510934350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 0x55fa8ee9c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb6df59640 1 --2- 172.21.15.86:0/510934350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea2e50 0x55fa8eeaba40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7feb640096c0 tx=0x7feb640a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2ed07bb03a958 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb537fe640 1 -- 172.21.15.86:0/510934350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640be990 con 0x55fa8eea2e50 2024-04-16T05:39:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb537fe640 1 -- 172.21.15.86:0/510934350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb640c2d40 con 0x55fa8eea2e50 2024-04-16T05:39:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb537fe640 1 -- 172.21.15.86:0/510934350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640bcac0 con 0x55fa8eea2e50 2024-04-16T05:39:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb72da9740 1 -- 172.21.15.86:0/510934350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea2e50 msgr2=0x55fa8eeaba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb72da9740 1 --2- 172.21.15.86:0/510934350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea2e50 0x55fa8eeaba40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7feb640096c0 tx=0x7feb640a9040 comp rx=0 tx=0).stop 2024-04-16T05:39:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb72da9740 1 -- 172.21.15.86:0/510934350 shutdown_connections 2024-04-16T05:39:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb72da9740 1 --2- 172.21.15.86:0/510934350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eeac350 0x55fa8ee9c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb72da9740 1 --2- 172.21.15.86:0/510934350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea3770 0x55fa8eeb8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb72da9740 1 --2- 172.21.15.86:0/510934350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eea2e50 0x55fa8eeaba40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb72da9740 1 -- 172.21.15.86:0/510934350 >> 172.21.15.86:0/510934350 conn(0x55fa8ee9d580 msgr2=0x55fa8ee9bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb72da9740 1 -- 172.21.15.86:0/510934350 shutdown_connections 2024-04-16T05:39:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.237+0000 7feb72da9740 1 -- 172.21.15.86:0/510934350 wait complete. 2024-04-16T05:39:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb72da9740 1 -- start start 2024-04-16T05:39:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb72da9740 1 Processor -- start 2024-04-16T05:39:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb72da9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea2e50 0x55fa8eeb4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb72da9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea3770 0x55fa8eeb1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb72da9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeb2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb72da9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb681b0750 con 0x55fa8eeac350 2024-04-16T05:39:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb72da9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb681b0080 con 0x55fa8eea3770 2024-04-16T05:39:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb72da9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb681ad2a0 con 0x55fa8eea2e50 2024-04-16T05:39:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb72da9740 10 client.0 _ll_register_callbacks cb 0x7feb5800b930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb6e75a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeb2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb6e75a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeb2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58166/0 (socket says 172.21.15.86:58166) 2024-04-16T05:39:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb6e75a640 1 -- 172.21.15.86:0/618567108 learned_addr learned my addr 172.21.15.86:0/618567108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb6d758640 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea3770 0x55fa8eeb1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.238+0000 7feb6e75a640 1 -- 172.21.15.86:0/618567108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea2e50 msgr2=0x55fa8eeb4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb6e75a640 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea2e50 0x55fa8eeb4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb6e75a640 1 -- 172.21.15.86:0/618567108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea3770 msgr2=0x55fa8eeb1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb6e75a640 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea3770 0x55fa8eeb1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb6e75a640 1 -- 172.21.15.86:0/618567108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa8eeb2a10 con 0x55fa8eeac350 2024-04-16T05:39:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb6e75a640 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeb2200 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7feb6001fa90 tx=0x7feb60004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f6e6d19b7c9853 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb517fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb517fa640 1 -- 172.21.15.86:0/618567108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60012bc0 con 0x55fa8eeac350 2024-04-16T05:39:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb517fa640 1 -- 172.21.15.86:0/618567108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feb60012d60 con 0x55fa8eeac350 2024-04-16T05:39:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb72da9740 1 -- 172.21.15.86:0/618567108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb640c2d40 con 0x55fa8eeac350 2024-04-16T05:39:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb517fa640 1 -- 172.21.15.86:0/618567108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb6001d470 con 0x55fa8eeac350 2024-04-16T05:39:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb72da9740 10 client.16824 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb72da9740 1 -- 172.21.15.86:0/618567108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa8ef22a60 con 0x55fa8eeac350 2024-04-16T05:39:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb517fa640 1 -- 172.21.15.86:0/618567108 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb640c2d40 con 0x55fa8eeac350 2024-04-16T05:39:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb517fa640 1 -- 172.21.15.86:0/618567108 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7feb6000f040 con 0x55fa8eeac350 2024-04-16T05:39:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb72da9740 10 client.16824 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb72da9740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb72da9740 20 client.16824 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb72da9740 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa8ef23260 0x55fa8ef25680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb72da9740 4 client.16824 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:39:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.239+0000 7feb72da9740 1 -- 172.21.15.86:0/618567108 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa8ee0fa10 con 0x55fa8ef23260 2024-04-16T05:39:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb6df59640 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa8ef23260 0x55fa8ef25680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb517fa640 10 client.16824 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb6df59640 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa8ef23260 0x55fa8ef25680 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53cb67d74f1bfdf server_cookie=395d0b16b192b7ca in_seq=0 out_seq=0 2024-04-16T05:39:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb517fa640 1 -- 172.21.15.86:0/618567108 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fa8ee0fa10 con 0x55fa8ef23260 2024-04-16T05:39:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb517fa640 10 client.16824 handle_command_reply: tid=0 2024-04-16T05:39:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb72da9740 1 client.16824 shutdown 2024-04-16T05:39:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb72da9740 1 -- 172.21.15.86:0/618567108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 msgr2=0x55fa8eeb2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb72da9740 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeb2200 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7feb6001fa90 tx=0x7feb60004ab0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb72da9740 1 -- 172.21.15.86:0/618567108 shutdown_connections 2024-04-16T05:39:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb72da9740 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa8ef23260 0x55fa8ef25680 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb72da9740 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8eeac350 0x55fa8eeb2200 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb72da9740 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa8eea3770 0x55fa8eeb1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb72da9740 1 --2- 172.21.15.86:0/618567108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa8eea2e50 0x55fa8eeb4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb517fa640 0 client.16824 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:11.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.240+0000 7feb72da9740 1 -- 172.21.15.86:0/618567108 >> 172.21.15.86:0/618567108 conn(0x55fa8ee9d580 msgr2=0x55fa8eea9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb72da9740 1 -- 172.21.15.86:0/618567108 shutdown_connections 2024-04-16T05:39:11.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb72da9740 1 -- 172.21.15.86:0/618567108 wait complete. 2024-04-16T05:39:11.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb72da9740 20 client.16824 trim_cache size 0 max 16384 2024-04-16T05:39:11.249 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:39:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:39:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:39:11.250 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:39:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 -- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb540a3d20 msgr2=0x7feb540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb540a3d20 0x7feb540a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb60005fd0 tx=0x7feb60004a40 comp rx=0 tx=0).stop 2024-04-16T05:39:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 -- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 msgr2=0x7feb6819e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 0x7feb6819e600 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7feb6402c240 tx=0x7feb64004430 comp rx=0 tx=0).stop 2024-04-16T05:39:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 -- 172.21.15.86:0/2199504490 shutdown_connections 2024-04-16T05:39:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb681986c0 0x7feb68198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb540a3d20 0x7feb540a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb68107310 0x7feb6819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 --2- 172.21.15.86:0/2199504490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb68102240 0x7feb6819e600 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 -- 172.21.15.86:0/2199504490 >> 172.21.15.86:0/2199504490 conn(0x7feb680fda40 msgr2=0x7feb680fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 -- 172.21.15.86:0/2199504490 shutdown_connections 2024-04-16T05:39:11.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.241+0000 7feb701bd640 1 -- 172.21.15.86:0/2199504490 wait complete. 2024-04-16T05:39:11.252 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:39:11.253 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:39:11.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 -- 172.21.15.86:0/3350096277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 msgr2=0x7fa61c101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 --2- 172.21.15.86:0/3350096277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 0x7fa61c101530 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009ca0 tx=0x7fa60c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 -- 172.21.15.86:0/3350096277 shutdown_connections 2024-04-16T05:39:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 --2- 172.21.15.86:0/3350096277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa61c106220 0x7fa61c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 --2- 172.21.15.86:0/3350096277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c101b00 0x7fa61c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 --2- 172.21.15.86:0/3350096277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 0x7fa61c101530 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 -- 172.21.15.86:0/3350096277 >> 172.21.15.86:0/3350096277 conn(0x7fa61c078000 msgr2=0x7fa61c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 -- 172.21.15.86:0/3350096277 shutdown_connections 2024-04-16T05:39:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 -- 172.21.15.86:0/3350096277 wait complete. 2024-04-16T05:39:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 Processor -- start 2024-04-16T05:39:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.359+0000 7fa621488640 1 -- start start 2024-04-16T05:39:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa621488640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 0x7fa61c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa621488640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c106220 0x7fa61c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa621488640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa61c19a210 0x7fa61c1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa621488640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1112e0 con 0x7fa61c101150 2024-04-16T05:39:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa621488640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c111160 con 0x7fa61c106220 2024-04-16T05:39:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa621488640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c111460 con 0x7fa61c19a210 2024-04-16T05:39:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 0x7fa61c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 0x7fa61c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58178/0 (socket says 172.21.15.86:58178) 2024-04-16T05:39:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61affd640 1 -- 172.21.15.86:0/404783007 learned_addr learned my addr 172.21.15.86:0/404783007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61a7fc640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c106220 0x7fa61c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61affd640 1 -- 172.21.15.86:0/404783007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa61c19a210 msgr2=0x7fa61c1a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61affd640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa61c19a210 0x7fa61c1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61affd640 1 -- 172.21.15.86:0/404783007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c106220 msgr2=0x7fa61c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61affd640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c106220 0x7fa61c199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61affd640 1 -- 172.21.15.86:0/404783007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c1a4f90 con 0x7fa61c101150 2024-04-16T05:39:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61a7fc640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c106220 0x7fa61c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa61affd640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 0x7fa61c199740 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa60c005e30 tx=0x7fa60c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa613fff640 1 -- 172.21.15.86:0/404783007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c041020 con 0x7fa61c101150 2024-04-16T05:39:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c1a5220 con 0x7fa61c101150 2024-04-16T05:39:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa613fff640 1 -- 172.21.15.86:0/404783007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa60c034d90 con 0x7fa61c101150 2024-04-16T05:39:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa613fff640 1 -- 172.21.15.86:0/404783007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c039040 con 0x7fa61c101150 2024-04-16T05:39:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.360+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1a5760 con 0x7fa61c101150 2024-04-16T05:39:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.361+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61c068e20 con 0x7fa61c101150 2024-04-16T05:39:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.362+0000 7fa613fff640 1 -- 172.21.15.86:0/404783007 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa60c044020 con 0x7fa61c101150 2024-04-16T05:39:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.363+0000 7fa613fff640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6080a3d20 0x7fa6080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.363+0000 7fa61a7fc640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6080a3d20 0x7fa6080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.363+0000 7fa613fff640 1 -- 172.21.15.86:0/404783007 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa60c0f29b0 con 0x7fa61c101150 2024-04-16T05:39:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.363+0000 7fa61a7fc640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6080a3d20 0x7fa6080a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa604005d00 tx=0x7fa604009550 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.368+0000 7fa613fff640 1 -- 172.21.15.86:0/404783007 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa60c04b020 con 0x7fa61c101150 2024-04-16T05:39:11.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.587+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa61c101b00 con 0x7fa61c101150 2024-04-16T05:39:11.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.588+0000 7fa613fff640 1 -- 172.21.15.86:0/404783007 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fa60c0bedf0 con 0x7fa61c101150 2024-04-16T05:39:11.587 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:39:11.587 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:39:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:39:11.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6080a3d20 msgr2=0x7fa6080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6080a3d20 0x7fa6080a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa604005d00 tx=0x7fa604009550 comp rx=0 tx=0).stop 2024-04-16T05:39:11.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 msgr2=0x7fa61c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 0x7fa61c199740 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa60c005e30 tx=0x7fa60c032f20 comp rx=0 tx=0).stop 2024-04-16T05:39:11.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 shutdown_connections 2024-04-16T05:39:11.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa61c19a210 0x7fa61c1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6080a3d20 0x7fa6080a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c106220 0x7fa61c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 --2- 172.21.15.86:0/404783007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c101150 0x7fa61c199740 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 >> 172.21.15.86:0/404783007 conn(0x7fa61c078000 msgr2=0x7fa61c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 shutdown_connections 2024-04-16T05:39:11.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.590+0000 7fa621488640 1 -- 172.21.15.86:0/404783007 wait complete. 2024-04-16T05:39:11.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:39:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.709+0000 7f070b1a8640 1 -- 172.21.15.86:0/572751073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 msgr2=0x7f070410eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.709+0000 7f070b1a8640 1 --2- 172.21.15.86:0/572751073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 0x7f070410eef0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009ca0 tx=0x7f06fc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.709+0000 7f070b1a8640 1 -- 172.21.15.86:0/572751073 shutdown_connections 2024-04-16T05:39:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.709+0000 7f070b1a8640 1 --2- 172.21.15.86:0/572751073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 0x7f070410eef0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.709+0000 7f070b1a8640 1 --2- 172.21.15.86:0/572751073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0704102c00 0x7f0704106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.709+0000 7f070b1a8640 1 --2- 172.21.15.86:0/572751073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0704102250 0x7f0704102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.709+0000 7f070b1a8640 1 -- 172.21.15.86:0/572751073 >> 172.21.15.86:0/572751073 conn(0x7f0704079490 msgr2=0x7f07040798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.709+0000 7f070b1a8640 1 -- 172.21.15.86:0/572751073 shutdown_connections 2024-04-16T05:39:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.709+0000 7f070b1a8640 1 -- 172.21.15.86:0/572751073 wait complete. 2024-04-16T05:39:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f070b1a8640 1 Processor -- start 2024-04-16T05:39:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f070b1a8640 1 -- start start 2024-04-16T05:39:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f070b1a8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0704102250 0x7f070419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f070b1a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 0x7f070419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f070b1a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0704198650 0x7f0704198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f070b1a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0704112430 con 0x7f0704107320 2024-04-16T05:39:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f070b1a8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07041122b0 con 0x7f0704102250 2024-04-16T05:39:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f070b1a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07041125b0 con 0x7f0704198650 2024-04-16T05:39:11.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0703fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 0x7f070419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0703fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 0x7f070419ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58206/0 (socket says 172.21.15.86:58206) 2024-04-16T05:39:11.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0703fff640 1 -- 172.21.15.86:0/3163762013 learned_addr learned my addr 172.21.15.86:0/3163762013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0708f44640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0704102250 0x7f070419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0709745640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0704198650 0x7f0704198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0703fff640 1 -- 172.21.15.86:0/3163762013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0704198650 msgr2=0x7f0704198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0703fff640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0704198650 0x7f0704198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0703fff640 1 -- 172.21.15.86:0/3163762013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0704102250 msgr2=0x7f070419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0703fff640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0704102250 0x7f070419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0703fff640 1 -- 172.21.15.86:0/3163762013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0704199250 con 0x7f0704107320 2024-04-16T05:39:11.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.710+0000 7f0708f44640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0704102250 0x7f070419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:11.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.711+0000 7f0703fff640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 0x7f070419ead0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f06f000b450 tx=0x7f06f000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.711+0000 7f0701ffb640 1 -- 172.21.15.86:0/3163762013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0015020 con 0x7f0704107320 2024-04-16T05:39:11.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.711+0000 7f070b1a8640 1 -- 172.21.15.86:0/3163762013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07041a5330 con 0x7f0704107320 2024-04-16T05:39:11.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.711+0000 7f0701ffb640 1 -- 172.21.15.86:0/3163762013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0011d70 con 0x7f0704107320 2024-04-16T05:39:11.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.711+0000 7f0701ffb640 1 -- 172.21.15.86:0/3163762013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f000f340 con 0x7f0704107320 2024-04-16T05:39:11.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.711+0000 7f070b1a8640 1 -- 172.21.15.86:0/3163762013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07041a56e0 con 0x7f0704107320 2024-04-16T05:39:11.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.713+0000 7f0701ffb640 1 -- 172.21.15.86:0/3163762013 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f06f0018020 con 0x7f0704107320 2024-04-16T05:39:11.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.713+0000 7f070dd96740 1 Processor -- start 2024-04-16T05:39:11.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f070dd96740 1 -- start start 2024-04-16T05:39:11.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f070dd96740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df6b10 0x557b77df6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f070dd96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df7430 0x557b77e04230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0701ffb640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f06f40a3d20 0x7f06f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f070dd96740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77e04770 0x557b77e06b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f070dd96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b77d5c0e0 con 0x557b77df7430 2024-04-16T05:39:11.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f070dd96740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557b77cf9520 con 0x557b77df6b10 2024-04-16T05:39:11.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f070dd96740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b77d580a0 con 0x557b77e04770 2024-04-16T05:39:11.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0708f44640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f06f40a3d20 0x7f06f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0703fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df7430 0x557b77e04230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0703fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df7430 0x557b77e04230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58220/0 (socket says 172.21.15.86:58220) 2024-04-16T05:39:11.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0703fff640 1 -- 172.21.15.86:0/4035900458 learned_addr learned my addr 172.21.15.86:0/4035900458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0709745640 1 --2- 172.21.15.86:0/4035900458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df6b10 0x557b77df6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0701ffb640 1 -- 172.21.15.86:0/3163762013 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f06f00d05f0 con 0x7f0704107320 2024-04-16T05:39:11.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0703fff640 1 -- 172.21.15.86:0/4035900458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77e04770 msgr2=0x557b77e06b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0709745640 1 --2- 172.21.15.86:0/4035900458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77e04770 0x557b77e06b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0703fff640 1 --2- 172.21.15.86:0/4035900458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77e04770 0x557b77e06b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0703fff640 1 -- 172.21.15.86:0/4035900458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df6b10 msgr2=0x557b77df6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0703fff640 1 --2- 172.21.15.86:0/4035900458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df6b10 0x557b77df6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0703fff640 1 -- 172.21.15.86:0/4035900458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b77d218d0 con 0x557b77df7430 2024-04-16T05:39:11.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0708f44640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f06f40a3d20 0x7f06f40a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f06f800abe0 tx=0x7f06f800a5c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.714+0000 7f0703fff640 1 --2- 172.21.15.86:0/4035900458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df7430 0x557b77e04230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f06f00cdb40 tx=0x7f06f0010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f13d91283d28e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f06eeffd640 1 -- 172.21.15.86:0/4035900458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0009280 con 0x557b77df7430 2024-04-16T05:39:11.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f06eeffd640 1 -- 172.21.15.86:0/4035900458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f001d120 con 0x557b77df7430 2024-04-16T05:39:11.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f06eeffd640 1 -- 172.21.15.86:0/4035900458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f00cdc50 con 0x557b77df7430 2024-04-16T05:39:11.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f070dd96740 1 -- 172.21.15.86:0/4035900458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df7430 msgr2=0x557b77e04230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f070dd96740 1 --2- 172.21.15.86:0/4035900458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df7430 0x557b77e04230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f06f00cdb40 tx=0x7f06f0010970 comp rx=0 tx=0).stop 2024-04-16T05:39:11.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f070dd96740 1 -- 172.21.15.86:0/4035900458 shutdown_connections 2024-04-16T05:39:11.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f070dd96740 1 --2- 172.21.15.86:0/4035900458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77e04770 0x557b77e06b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f070dd96740 1 --2- 172.21.15.86:0/4035900458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df7430 0x557b77e04230 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f070dd96740 1 --2- 172.21.15.86:0/4035900458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df6b10 0x557b77df6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f070dd96740 1 -- 172.21.15.86:0/4035900458 >> 172.21.15.86:0/4035900458 conn(0x557b77df1590 msgr2=0x557b77df1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f070dd96740 1 -- 172.21.15.86:0/4035900458 shutdown_connections 2024-04-16T05:39:11.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.715+0000 7f070dd96740 1 -- 172.21.15.86:0/4035900458 wait complete. 2024-04-16T05:39:11.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f070dd96740 1 -- start start 2024-04-16T05:39:11.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f070dd96740 1 Processor -- start 2024-04-16T05:39:11.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f070dd96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e01710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f070dd96740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df7430 0x557b77e01c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f070dd96740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77e04770 0x557b77e02190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f070dd96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b77e074f0 con 0x557b77df6b10 2024-04-16T05:39:11.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f070dd96740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557b77d154d0 con 0x557b77df7430 2024-04-16T05:39:11.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f070dd96740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b77e07670 con 0x557b77e04770 2024-04-16T05:39:11.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f070dd96740 10 client.0 _ll_register_callbacks cb 0x557b77cfa2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:11.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f0708f44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e01710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f0708f44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e01710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58222/0 (socket says 172.21.15.86:58222) 2024-04-16T05:39:11.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f0708f44640 1 -- 172.21.15.86:0/1293888159 learned_addr learned my addr 172.21.15.86:0/1293888159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f0703fff640 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df7430 0x557b77e01c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f0708f44640 1 -- 172.21.15.86:0/1293888159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77e04770 msgr2=0x557b77e02190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f0708f44640 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77e04770 0x557b77e02190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f0708f44640 1 -- 172.21.15.86:0/1293888159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df7430 msgr2=0x557b77e01c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.716+0000 7f0708f44640 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df7430 0x557b77e01c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f0708f44640 1 -- 172.21.15.86:0/1293888159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b77a6e3f0 con 0x557b77df6b10 2024-04-16T05:39:11.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f0703fff640 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df7430 0x557b77e01c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:11.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f0708f44640 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e01710 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f06f8010990 tx=0x7f06f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf9bd48a20dd2ec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f06e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:11.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f06e3fff640 1 -- 172.21.15.86:0/1293888159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8021070 con 0x557b77df6b10 2024-04-16T05:39:11.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f06e3fff640 1 -- 172.21.15.86:0/1293888159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f06f800fcd0 con 0x557b77df6b10 2024-04-16T05:39:11.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f070dd96740 1 -- 172.21.15.86:0/1293888159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557b77d69b00 con 0x557b77df6b10 2024-04-16T05:39:11.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f06e3fff640 1 -- 172.21.15.86:0/1293888159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06f802d050 con 0x557b77df6b10 2024-04-16T05:39:11.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f070dd96740 10 client.16842 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:11.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f070dd96740 1 -- 172.21.15.86:0/1293888159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557b77b387d0 con 0x557b77df6b10 2024-04-16T05:39:11.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f06e3fff640 1 -- 172.21.15.86:0/1293888159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557b77d69b00 con 0x557b77df6b10 2024-04-16T05:39:11.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f06e3fff640 1 -- 172.21.15.86:0/1293888159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06f8025570 con 0x557b77df6b10 2024-04-16T05:39:11.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f070dd96740 10 client.16842 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:11.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f070dd96740 10 client.16842 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:39:11.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f070dd96740 20 client.16842 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:11.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f070dd96740 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x557b77e14760 0x557b77e16c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f070dd96740 4 client.16842 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:11.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.717+0000 7f070dd96740 1 -- 172.21.15.86:0/1293888159 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b77d63970 con 0x557b77e14760 2024-04-16T05:39:11.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.718+0000 7f0703fff640 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x557b77e14760 0x557b77e16c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.718+0000 7f06e3fff640 10 client.16842 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:39:11.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.718+0000 7f0703fff640 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x557b77e14760 0x557b77e16c00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c14fa62da312a86e server_cookie=3a81591719fb869b in_seq=0 out_seq=0 2024-04-16T05:39:11.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f06e3fff640 1 -- 172.21.15.86:0/1293888159 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b77d63970 con 0x557b77e14760 2024-04-16T05:39:11.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f06e3fff640 10 client.16842 handle_command_reply: tid=0 2024-04-16T05:39:11.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 client.16842 shutdown 2024-04-16T05:39:11.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 -- 172.21.15.86:0/1293888159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 msgr2=0x557b77e01710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e01710 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f06f8010990 tx=0x7f06f80046c0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 -- 172.21.15.86:0/1293888159 shutdown_connections 2024-04-16T05:39:11.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x557b77e14760 0x557b77e16c00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77e04770 0x557b77e02190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df7430 0x557b77e01c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f06e3fff640 0 client.16842 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:39:11.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 --2- 172.21.15.86:0/1293888159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e01710 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 -- 172.21.15.86:0/1293888159 >> 172.21.15.86:0/1293888159 conn(0x557b77df1590 msgr2=0x557b77df32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 -- 172.21.15.86:0/1293888159 shutdown_connections 2024-04-16T05:39:11.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 1 -- 172.21.15.86:0/1293888159 wait complete. 2024-04-16T05:39:11.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.719+0000 7f070dd96740 20 client.16842 trim_cache size 0 max 16384 2024-04-16T05:39:11.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.732+0000 7f070dd96740 1 Processor -- start 2024-04-16T05:39:11.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.732+0000 7f070dd96740 1 -- start start 2024-04-16T05:39:11.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.732+0000 7f070dd96740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df6b10 0x557b77e03e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.732+0000 7f070dd96740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77df7430 0x557b77e10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.732+0000 7f070dd96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77e04770 0x557b77df3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.732+0000 7f070dd96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ad220 con 0x557b77e04770 2024-04-16T05:39:11.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.732+0000 7f070dd96740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07041aed60 con 0x557b77df6b10 2024-04-16T05:39:11.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.732+0000 7f070dd96740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ad8f0 con 0x557b77df7430 2024-04-16T05:39:11.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f0709745640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77e04770 0x557b77df3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f0709745640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77e04770 0x557b77df3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58234/0 (socket says 172.21.15.86:58234) 2024-04-16T05:39:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f0709745640 1 -- 172.21.15.86:0/4244063628 learned_addr learned my addr 172.21.15.86:0/4244063628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f0709745640 1 -- 172.21.15.86:0/4244063628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77df7430 msgr2=0x557b77e10450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f0709745640 1 --2- 172.21.15.86:0/4244063628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77df7430 0x557b77e10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f0709745640 1 -- 172.21.15.86:0/4244063628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df6b10 msgr2=0x557b77e03e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f0709745640 1 --2- 172.21.15.86:0/4244063628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df6b10 0x557b77e03e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f0709745640 1 -- 172.21.15.86:0/4244063628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b77e006c0 con 0x557b77e04770 2024-04-16T05:39:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f0709745640 1 --2- 172.21.15.86:0/4244063628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77e04770 0x557b77df3270 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f06fc02c700 tx=0x7f06fc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4923f0e488e103a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f06ee7fc640 1 -- 172.21.15.86:0/4244063628 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc045030 con 0x557b77e04770 2024-04-16T05:39:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f06ee7fc640 1 -- 172.21.15.86:0/4244063628 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc03a040 con 0x557b77e04770 2024-04-16T05:39:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.733+0000 7f06ee7fc640 1 -- 172.21.15.86:0/4244063628 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc039070 con 0x557b77e04770 2024-04-16T05:39:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 -- 172.21.15.86:0/4244063628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77e04770 msgr2=0x557b77df3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 --2- 172.21.15.86:0/4244063628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77e04770 0x557b77df3270 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f06fc02c700 tx=0x7f06fc009800 comp rx=0 tx=0).stop 2024-04-16T05:39:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 -- 172.21.15.86:0/4244063628 shutdown_connections 2024-04-16T05:39:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 --2- 172.21.15.86:0/4244063628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77e04770 0x557b77df3270 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 --2- 172.21.15.86:0/4244063628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77df7430 0x557b77e10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 --2- 172.21.15.86:0/4244063628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77df6b10 0x557b77e03e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 -- 172.21.15.86:0/4244063628 >> 172.21.15.86:0/4244063628 conn(0x557b77df1590 msgr2=0x557b77df1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 -- 172.21.15.86:0/4244063628 shutdown_connections 2024-04-16T05:39:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 -- 172.21.15.86:0/4244063628 wait complete. 2024-04-16T05:39:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 -- start start 2024-04-16T05:39:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 Processor -- start 2024-04-16T05:39:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e08760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77df7430 0x557b77e08ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77e04770 0x557b77e79d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ae690 con 0x557b77df6b10 2024-04-16T05:39:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07041adfc0 con 0x557b77e04770 2024-04-16T05:39:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.734+0000 7f070dd96740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ab1e0 con 0x557b77df7430 2024-04-16T05:39:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f070dd96740 10 client.0 _ll_register_callbacks cb 0x7f06f8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0708f44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e08760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0708f44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e08760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58242/0 (socket says 172.21.15.86:58242) 2024-04-16T05:39:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0708f44640 1 -- 172.21.15.86:0/4212238690 learned_addr learned my addr 172.21.15.86:0/4212238690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0703fff640 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77df7430 0x557b77e08ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0709745640 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77e04770 0x557b77e79d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0708f44640 1 -- 172.21.15.86:0/4212238690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77df7430 msgr2=0x557b77e08ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0708f44640 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77df7430 0x557b77e08ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0708f44640 1 -- 172.21.15.86:0/4212238690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77e04770 msgr2=0x557b77e79d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0708f44640 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77e04770 0x557b77e79d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0708f44640 1 -- 172.21.15.86:0/4212238690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b77e7a280 con 0x557b77df6b10 2024-04-16T05:39:11.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f0708f44640 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e08760 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f06f80106e0 tx=0x7f06f800e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615a861276c7a34e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f06e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:11.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f06e3fff640 1 -- 172.21.15.86:0/4212238690 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8035020 con 0x557b77df6b10 2024-04-16T05:39:11.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f070dd96740 1 -- 172.21.15.86:0/4212238690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06fc03a040 con 0x557b77df6b10 2024-04-16T05:39:11.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f06e3fff640 1 -- 172.21.15.86:0/4212238690 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f06f800fcd0 con 0x557b77df6b10 2024-04-16T05:39:11.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f06e3fff640 1 -- 172.21.15.86:0/4212238690 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06f8040050 con 0x557b77df6b10 2024-04-16T05:39:11.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f070dd96740 10 client.16848 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:11.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f070dd96740 1 -- 172.21.15.86:0/4212238690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557b77e7b3b0 con 0x557b77df6b10 2024-04-16T05:39:11.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.735+0000 7f06e3fff640 1 -- 172.21.15.86:0/4212238690 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc03a040 con 0x557b77df6b10 2024-04-16T05:39:11.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f06e3fff640 1 -- 172.21.15.86:0/4212238690 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06f8005040 con 0x557b77df6b10 2024-04-16T05:39:11.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f070dd96740 10 client.16848 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:11.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f070dd96740 10 client.16848 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:39:11.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f070dd96740 20 client.16848 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:11.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f070dd96740 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x557b77e7bb80 0x557b77e7dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f070dd96740 4 client.16848 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:11.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f070dd96740 1 -- 172.21.15.86:0/4212238690 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b77cdf6b0 con 0x557b77e7bb80 2024-04-16T05:39:11.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f0703fff640 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x557b77e7bb80 0x557b77e7dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f06e3fff640 10 client.16848 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:39:11.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.736+0000 7f0703fff640 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x557b77e7bb80 0x557b77e7dfa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c078de377ca9b6d0 server_cookie=cc39727eee3577af in_seq=0 out_seq=0 2024-04-16T05:39:11.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f06e3fff640 1 -- 172.21.15.86:0/4212238690 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b77cdf6b0 con 0x557b77e7bb80 2024-04-16T05:39:11.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f06e3fff640 10 client.16848 handle_command_reply: tid=0 2024-04-16T05:39:11.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 client.16848 shutdown 2024-04-16T05:39:11.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 -- 172.21.15.86:0/4212238690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 msgr2=0x557b77e08760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e08760 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f06f80106e0 tx=0x7f06f800e860 comp rx=0 tx=0).stop 2024-04-16T05:39:11.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 -- 172.21.15.86:0/4212238690 shutdown_connections 2024-04-16T05:39:11.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x557b77e7bb80 0x557b77e7dfa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557b77e04770 0x557b77e79d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f06e3fff640 0 client.16848 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:39:11.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b77df7430 0x557b77e08ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 --2- 172.21.15.86:0/4212238690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b77df6b10 0x557b77e08760 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 -- 172.21.15.86:0/4212238690 >> 172.21.15.86:0/4212238690 conn(0x557b77df1590 msgr2=0x557b77df0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 -- 172.21.15.86:0/4212238690 shutdown_connections 2024-04-16T05:39:11.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 1 -- 172.21.15.86:0/4212238690 wait complete. 2024-04-16T05:39:11.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070dd96740 20 client.16848 trim_cache size 0 max 16384 2024-04-16T05:39:11.748 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:11.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 -- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f06f40a3d20 msgr2=0x7f06f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f06f40a3d20 0x7f06f40a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f06f800abe0 tx=0x7f06f800a5c0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 -- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 msgr2=0x7f070419ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 0x7f070419ead0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f06f000b450 tx=0x7f06f000b910 comp rx=0 tx=0).stop 2024-04-16T05:39:11.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 -- 172.21.15.86:0/3163762013 shutdown_connections 2024-04-16T05:39:11.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0704198650 0x7f0704198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0704107320 0x7f070419ead0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f06f40a3d20 0x7f06f40a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 --2- 172.21.15.86:0/3163762013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0704102250 0x7f070419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.737+0000 7f070b1a8640 1 -- 172.21.15.86:0/3163762013 >> 172.21.15.86:0/3163762013 conn(0x7f0704079490 msgr2=0x7f07040ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.738+0000 7f070b1a8640 1 -- 172.21.15.86:0/3163762013 shutdown_connections 2024-04-16T05:39:11.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.738+0000 7f070b1a8640 1 -- 172.21.15.86:0/3163762013 wait complete. 2024-04-16T05:39:11.752 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:39:11.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.859+0000 7f1b23ca0640 1 -- 172.21.15.86:0/2276604272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c106200 msgr2=0x7f1b1c10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.859+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/2276604272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c106200 0x7f1b1c10ddd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009900 tx=0x7f1b1402bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:11.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.859+0000 7f1b23ca0640 1 -- 172.21.15.86:0/2276604272 shutdown_connections 2024-04-16T05:39:11.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.859+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/2276604272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c106200 0x7f1b1c10ddd0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.859+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/2276604272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b1c101ae0 0x7f1b1c105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.859+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/2276604272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b1c101130 0x7f1b1c101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.859+0000 7f1b23ca0640 1 -- 172.21.15.86:0/2276604272 >> 172.21.15.86:0/2276604272 conn(0x7f1b1c0780b0 msgr2=0x7f1b1c0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.859+0000 7f1b23ca0640 1 -- 172.21.15.86:0/2276604272 shutdown_connections 2024-04-16T05:39:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.859+0000 7f1b23ca0640 1 -- 172.21.15.86:0/2276604272 wait complete. 2024-04-16T05:39:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b23ca0640 1 Processor -- start 2024-04-16T05:39:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b23ca0640 1 -- start start 2024-04-16T05:39:11.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b23ca0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b1c101130 0x7f1b1c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b23ca0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b1c106200 0x7f1b1c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b23ca0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c1986e0 0x7f1b1c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b23ca0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c111310 con 0x7f1b1c1986e0 2024-04-16T05:39:11.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b23ca0640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c111190 con 0x7f1b1c101130 2024-04-16T05:39:11.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b23ca0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c111490 con 0x7f1b1c106200 2024-04-16T05:39:11.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2223d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c1986e0 0x7f1b1c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2223d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c1986e0 0x7f1b1c198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58258/0 (socket says 172.21.15.86:58258) 2024-04-16T05:39:11.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2223d640 1 -- 172.21.15.86:0/259994200 learned_addr learned my addr 172.21.15.86:0/259994200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b21a3c640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b1c101130 0x7f1b1c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2123b640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b1c106200 0x7f1b1c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2223d640 1 -- 172.21.15.86:0/259994200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b1c106200 msgr2=0x7f1b1c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2223d640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b1c106200 0x7f1b1c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2223d640 1 -- 172.21.15.86:0/259994200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b1c101130 msgr2=0x7f1b1c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2223d640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b1c101130 0x7f1b1c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2223d640 1 -- 172.21.15.86:0/259994200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c1992e0 con 0x7f1b1c1986e0 2024-04-16T05:39:11.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2123b640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b1c106200 0x7f1b1c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:11.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b2223d640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c1986e0 0x7f1b1c198b60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009230 tx=0x7f1b1402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.860+0000 7f1b1affd640 1 -- 172.21.15.86:0/259994200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14041070 con 0x7f1b1c1986e0 2024-04-16T05:39:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.861+0000 7f1b23ca0640 1 -- 172.21.15.86:0/259994200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c199570 con 0x7f1b1c1986e0 2024-04-16T05:39:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.861+0000 7f1b1affd640 1 -- 172.21.15.86:0/259994200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b14004450 con 0x7f1b1c1986e0 2024-04-16T05:39:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.861+0000 7f1b1affd640 1 -- 172.21.15.86:0/259994200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14034440 con 0x7f1b1c1986e0 2024-04-16T05:39:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.861+0000 7f1b23ca0640 1 -- 172.21.15.86:0/259994200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c1a5700 con 0x7f1b1c1986e0 2024-04-16T05:39:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.863+0000 7f1b1affd640 1 -- 172.21.15.86:0/259994200 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1b14044070 con 0x7f1b1c1986e0 2024-04-16T05:39:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.863+0000 7f1b26885740 1 Processor -- start 2024-04-16T05:39:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b26885740 1 -- start start 2024-04-16T05:39:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b26885740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe2e60 0x5581fefe3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b26885740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe3780 0x5581fefebe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b26885740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 0x5581fefee750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b26885740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581fef480e0 con 0x5581fefe2e60 2024-04-16T05:39:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b26885740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5581feee5520 con 0x5581fefe3780 2024-04-16T05:39:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b26885740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581fef440a0 con 0x5581fefec360 2024-04-16T05:39:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b1affd640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b040a3d20 0x7f1b040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2123b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe2e60 0x5581fefe3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2123b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe2e60 0x5581fefe3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58264/0 (socket says 172.21.15.86:58264) 2024-04-16T05:39:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2123b640 1 -- 172.21.15.86:0/2100995409 learned_addr learned my addr 172.21.15.86:0/2100995409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b21a3c640 1 --2- 172.21.15.86:0/2100995409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe3780 0x5581fefebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2223d640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b040a3d20 0x7f1b040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2123b640 1 --2- 172.21.15.86:0/2100995409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 0x5581fefee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2123b640 1 -- 172.21.15.86:0/2100995409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 msgr2=0x5581fefee750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2123b640 1 --2- 172.21.15.86:0/2100995409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 0x5581fefee750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2123b640 1 -- 172.21.15.86:0/2100995409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe3780 msgr2=0x5581fefebe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2123b640 1 --2- 172.21.15.86:0/2100995409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe3780 0x5581fefebe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b2123b640 1 -- 172.21.15.86:0/2100995409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581fef0d8d0 con 0x5581fefe2e60 2024-04-16T05:39:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.864+0000 7f1b1affd640 1 -- 172.21.15.86:0/259994200 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1b140f2670 con 0x7f1b1c1986e0 2024-04-16T05:39:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.865+0000 7f1b2223d640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b040a3d20 0x7f1b040a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b140c6ab0 tx=0x7f1b140b9ef0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.865+0000 7f1b2123b640 1 --2- 172.21.15.86:0/2100995409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe2e60 0x5581fefe3240 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005e00 tx=0x7f1b10009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c75cfb803599208 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.865+0000 7f1b03fff640 1 -- 172.21.15.86:0/2100995409 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100150f0 con 0x5581fefe2e60 2024-04-16T05:39:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.865+0000 7f1b03fff640 1 -- 172.21.15.86:0/2100995409 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b100035f0 con 0x5581fefe2e60 2024-04-16T05:39:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.865+0000 7f1b03fff640 1 -- 172.21.15.86:0/2100995409 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10013700 con 0x5581fefe2e60 2024-04-16T05:39:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 -- 172.21.15.86:0/2100995409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe2e60 msgr2=0x5581fefe3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 --2- 172.21.15.86:0/2100995409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe2e60 0x5581fefe3240 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005e00 tx=0x7f1b10009670 comp rx=0 tx=0).stop 2024-04-16T05:39:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 -- 172.21.15.86:0/2100995409 shutdown_connections 2024-04-16T05:39:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 --2- 172.21.15.86:0/2100995409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 0x5581fefee750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 --2- 172.21.15.86:0/2100995409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe3780 0x5581fefebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 --2- 172.21.15.86:0/2100995409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe2e60 0x5581fefe3240 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 -- 172.21.15.86:0/2100995409 >> 172.21.15.86:0/2100995409 conn(0x5581fefdd590 msgr2=0x5581fefdd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 -- 172.21.15.86:0/2100995409 shutdown_connections 2024-04-16T05:39:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 -- 172.21.15.86:0/2100995409 wait complete. 2024-04-16T05:39:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 -- start start 2024-04-16T05:39:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 Processor -- start 2024-04-16T05:39:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefe2e60 0x5581fefe9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefec360 0x5581fefe9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581fefeefa0 con 0x5581fefe3780 2024-04-16T05:39:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5581fef014d0 con 0x5581fefec360 2024-04-16T05:39:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.866+0000 7f1b26885740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581fefef120 con 0x5581fefe2e60 2024-04-16T05:39:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b26885740 10 client.0 _ll_register_callbacks cb 0x5581feee62c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b21a3c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b21a3c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58268/0 (socket says 172.21.15.86:58268) 2024-04-16T05:39:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b21a3c640 1 -- 172.21.15.86:0/792971443 learned_addr learned my addr 172.21.15.86:0/792971443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b2123b640 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefe2e60 0x5581fefe9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b2123b640 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefec360 0x5581fefe9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b21a3c640 1 -- 172.21.15.86:0/792971443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefe2e60 msgr2=0x5581fefe9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b21a3c640 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefe2e60 0x5581fefe9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b21a3c640 1 -- 172.21.15.86:0/792971443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefec360 msgr2=0x5581fefe9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b21a3c640 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefec360 0x5581fefe9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b21a3c640 1 -- 172.21.15.86:0/792971443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581fec5a3f0 con 0x5581fefe3780 2024-04-16T05:39:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b2123b640 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefec360 0x5581fefe9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b21a3c640 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe9950 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00ab60 tx=0x7f1b0c011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ff63be37e01664 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b017fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b017fa640 1 -- 172.21.15.86:0/792971443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c01a070 con 0x5581fefe3780 2024-04-16T05:39:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b017fa640 1 -- 172.21.15.86:0/792971443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1b0c016450 con 0x5581fefe3780 2024-04-16T05:39:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b26885740 1 -- 172.21.15.86:0/792971443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b100035f0 con 0x5581fefe3780 2024-04-16T05:39:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b017fa640 1 -- 172.21.15.86:0/792971443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b0c026050 con 0x5581fefe3780 2024-04-16T05:39:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b26885740 10 client.16860 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.867+0000 7f1b26885740 1 -- 172.21.15.86:0/792971443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581fed247d0 con 0x5581fefe3780 2024-04-16T05:39:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b017fa640 1 -- 172.21.15.86:0/792971443 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b100035f0 con 0x5581fefe3780 2024-04-16T05:39:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b017fa640 1 -- 172.21.15.86:0/792971443 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b0c01e780 con 0x5581fefe3780 2024-04-16T05:39:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b26885740 10 client.16860 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b26885740 10 client.16860 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:39:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b26885740 20 client.16860 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b26885740 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5581feffc310 0x5581feffe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b26885740 4 client.16860 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b26885740 1 -- 172.21.15.86:0/792971443 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581fef55b00 con 0x5581feffc310 2024-04-16T05:39:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b2123b640 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5581feffc310 0x5581feffe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b2123b640 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5581feffc310 0x5581feffe750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=56fa5b8041329476 server_cookie=d1e54ed883a4fbdf in_seq=0 out_seq=0 2024-04-16T05:39:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.868+0000 7f1b017fa640 10 client.16860 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:39:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b017fa640 1 -- 172.21.15.86:0/792971443 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581fef55b00 con 0x5581feffc310 2024-04-16T05:39:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b017fa640 10 client.16860 handle_command_reply: tid=0 2024-04-16T05:39:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 client.16860 shutdown 2024-04-16T05:39:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 -- 172.21.15.86:0/792971443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 msgr2=0x5581fefe9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe9950 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00ab60 tx=0x7f1b0c011580 comp rx=0 tx=0).stop 2024-04-16T05:39:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 -- 172.21.15.86:0/792971443 shutdown_connections 2024-04-16T05:39:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefec360 0x5581fefe9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe9950 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5581feffc310 0x5581feffe750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 --2- 172.21.15.86:0/792971443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefe2e60 0x5581fefe9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b017fa640 0 client.16860 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:39:11.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 -- 172.21.15.86:0/792971443 >> 172.21.15.86:0/792971443 conn(0x5581fefdd590 msgr2=0x5581fefdf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 -- 172.21.15.86:0/792971443 shutdown_connections 2024-04-16T05:39:11.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.869+0000 7f1b26885740 1 -- 172.21.15.86:0/792971443 wait complete. 2024-04-16T05:39:11.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.870+0000 7f1b26885740 20 client.16860 trim_cache size 0 max 16384 2024-04-16T05:39:11.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b26885740 1 Processor -- start 2024-04-16T05:39:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b26885740 1 -- start start 2024-04-16T05:39:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b26885740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefe2e60 0x5581fefebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b26885740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe3780 0x5581ff004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b26885740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefec360 0x5581ff004560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b26885740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1ad180 con 0x5581fefec360 2024-04-16T05:39:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b26885740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1aecc0 con 0x5581fefe3780 2024-04-16T05:39:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b26885740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1ad850 con 0x5581fefe2e60 2024-04-16T05:39:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b2123b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefec360 0x5581ff004560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b2123b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefec360 0x5581ff004560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58270/0 (socket says 172.21.15.86:58270) 2024-04-16T05:39:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b2123b640 1 -- 172.21.15.86:0/361345881 learned_addr learned my addr 172.21.15.86:0/361345881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b2123b640 1 -- 172.21.15.86:0/361345881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefe2e60 msgr2=0x5581fefebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b2123b640 1 --2- 172.21.15.86:0/361345881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefe2e60 0x5581fefebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b2123b640 1 -- 172.21.15.86:0/361345881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe3780 msgr2=0x5581ff004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b2123b640 1 --2- 172.21.15.86:0/361345881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe3780 0x5581ff004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b2123b640 1 -- 172.21.15.86:0/361345881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581fefdd0c0 con 0x5581fefec360 2024-04-16T05:39:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b2123b640 1 --2- 172.21.15.86:0/361345881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefec360 0x5581ff004560 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005d00 tx=0x7f1b10003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8766ba24b4f4d27b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b037fe640 1 -- 172.21.15.86:0/361345881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10031070 con 0x5581fefec360 2024-04-16T05:39:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b037fe640 1 -- 172.21.15.86:0/361345881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b1002cbd0 con 0x5581fefec360 2024-04-16T05:39:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.883+0000 7f1b037fe640 1 -- 172.21.15.86:0/361345881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1002c670 con 0x5581fefec360 2024-04-16T05:39:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 -- 172.21.15.86:0/361345881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefec360 msgr2=0x5581ff004560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 --2- 172.21.15.86:0/361345881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefec360 0x5581ff004560 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005d00 tx=0x7f1b10003f00 comp rx=0 tx=0).stop 2024-04-16T05:39:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 -- 172.21.15.86:0/361345881 shutdown_connections 2024-04-16T05:39:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 --2- 172.21.15.86:0/361345881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefec360 0x5581ff004560 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 --2- 172.21.15.86:0/361345881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe3780 0x5581ff004020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 --2- 172.21.15.86:0/361345881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefe2e60 0x5581fefebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 -- 172.21.15.86:0/361345881 >> 172.21.15.86:0/361345881 conn(0x5581fefdd590 msgr2=0x5581fefdc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 -- 172.21.15.86:0/361345881 shutdown_connections 2024-04-16T05:39:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 -- 172.21.15.86:0/361345881 wait complete. 2024-04-16T05:39:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 -- start start 2024-04-16T05:39:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.884+0000 7f1b26885740 1 Processor -- start 2024-04-16T05:39:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b26885740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe2e60 0x5581fefe60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b26885740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b26885740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 0x5581fefe6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b26885740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1ae5f0 con 0x5581fefe3780 2024-04-16T05:39:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b26885740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1adf20 con 0x5581fefe2e60 2024-04-16T05:39:11.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b26885740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1ab140 con 0x5581fefec360 2024-04-16T05:39:11.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b26885740 10 client.0 _ll_register_callbacks cb 0x7f1b0c009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:11.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b21a3c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b21a3c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58278/0 (socket says 172.21.15.86:58278) 2024-04-16T05:39:11.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b21a3c640 1 -- 172.21.15.86:0/1079697447 learned_addr learned my addr 172.21.15.86:0/1079697447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:11.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b2123b640 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 0x5581fefe6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b21a3c640 1 -- 172.21.15.86:0/1079697447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 msgr2=0x5581fefe6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b21a3c640 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 0x5581fefe6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b21a3c640 1 -- 172.21.15.86:0/1079697447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe2e60 msgr2=0x5581fefe60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b2123b640 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe2e60 0x5581fefe60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b21a3c640 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe2e60 0x5581fefe60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b21a3c640 1 -- 172.21.15.86:0/1079697447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ff061890 con 0x5581fefe3780 2024-04-16T05:39:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b21a3c640 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe65e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c01eba0 tx=0x7f1b0c00a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6014fd7503c5cd9f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.885+0000 7f1b017fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b017fa640 1 -- 172.21.15.86:0/1079697447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c02f060 con 0x5581fefe3780 2024-04-16T05:39:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b017fa640 1 -- 172.21.15.86:0/1079697447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1b0c01e7c0 con 0x5581fefe3780 2024-04-16T05:39:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b26885740 1 -- 172.21.15.86:0/1079697447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b1002cbd0 con 0x5581fefe3780 2024-04-16T05:39:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b017fa640 1 -- 172.21.15.86:0/1079697447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b0c036050 con 0x5581fefe3780 2024-04-16T05:39:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b26885740 10 client.16866 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b26885740 1 -- 172.21.15.86:0/1079697447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581ff062af0 con 0x5581fefe3780 2024-04-16T05:39:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b017fa640 1 -- 172.21.15.86:0/1079697447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b1002cbd0 con 0x5581fefe3780 2024-04-16T05:39:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b017fa640 1 -- 172.21.15.86:0/1079697447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b0c01f440 con 0x5581fefe3780 2024-04-16T05:39:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b26885740 10 client.16866 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b26885740 10 client.16866 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:39:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b26885740 20 client.16866 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b26885740 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5581ff063230 0x5581ff065650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b26885740 4 client.16866 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:11.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b26885740 1 -- 172.21.15.86:0/1079697447 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581fef4f970 con 0x5581ff063230 2024-04-16T05:39:11.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.886+0000 7f1b2123b640 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5581ff063230 0x5581ff065650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:11.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b017fa640 10 client.16866 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:39:11.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b2123b640 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5581ff063230 0x5581ff065650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c40e033657b4e933 server_cookie=cb4c5828d531d7e0 in_seq=0 out_seq=0 2024-04-16T05:39:11.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b017fa640 1 -- 172.21.15.86:0/1079697447 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581fef4f970 con 0x5581ff063230 2024-04-16T05:39:11.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b017fa640 10 client.16866 handle_command_reply: tid=0 2024-04-16T05:39:11.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 client.16866 shutdown 2024-04-16T05:39:11.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 -- 172.21.15.86:0/1079697447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 msgr2=0x5581fefe65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe65e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c01eba0 tx=0x7f1b0c00a750 comp rx=0 tx=0).stop 2024-04-16T05:39:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 -- 172.21.15.86:0/1079697447 shutdown_connections 2024-04-16T05:39:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5581ff063230 0x5581ff065650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581fefec360 0x5581fefe6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581fefe3780 0x5581fefe65e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b017fa640 0 client.16866 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:39:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 --2- 172.21.15.86:0/1079697447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581fefe2e60 0x5581fefe60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 -- 172.21.15.86:0/1079697447 >> 172.21.15.86:0/1079697447 conn(0x5581fefdd590 msgr2=0x5581fefe9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 -- 172.21.15.86:0/1079697447 shutdown_connections 2024-04-16T05:39:11.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 1 -- 172.21.15.86:0/1079697447 wait complete. 2024-04-16T05:39:11.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.887+0000 7f1b26885740 20 client.16866 trim_cache size 0 max 16384 2024-04-16T05:39:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:11.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 -- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b040a3d20 msgr2=0x7f1b040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b040a3d20 0x7f1b040a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b140c6ab0 tx=0x7f1b140b9ef0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 -- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c1986e0 msgr2=0x7f1b1c198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c1986e0 0x7f1b1c198b60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009230 tx=0x7f1b1402cbf0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 -- 172.21.15.86:0/259994200 shutdown_connections 2024-04-16T05:39:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1c1986e0 0x7f1b1c198b60 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b1c106200 0x7f1b1c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1b040a3d20 0x7f1b040a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 --2- 172.21.15.86:0/259994200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b1c101130 0x7f1b1c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 -- 172.21.15.86:0/259994200 >> 172.21.15.86:0/259994200 conn(0x7f1b1c0780b0 msgr2=0x7f1b1c0693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 -- 172.21.15.86:0/259994200 shutdown_connections 2024-04-16T05:39:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:11.888+0000 7f1b23ca0640 1 -- 172.21.15.86:0/259994200 wait complete. 2024-04-16T05:39:11.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:39:12.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 -- 172.21.15.86:0/3941944086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54101130 msgr2=0x7fba54101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/3941944086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54101130 0x7fba54101510 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fba50009900 tx=0x7fba5002bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:12.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 -- 172.21.15.86:0/3941944086 shutdown_connections 2024-04-16T05:39:12.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/3941944086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba54106200 0x7fba5410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/3941944086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba54101ae0 0x7fba54105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/3941944086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54101130 0x7fba54101510 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 -- 172.21.15.86:0/3941944086 >> 172.21.15.86:0/3941944086 conn(0x7fba540780b0 msgr2=0x7fba540784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 -- 172.21.15.86:0/3941944086 shutdown_connections 2024-04-16T05:39:12.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 -- 172.21.15.86:0/3941944086 wait complete. 2024-04-16T05:39:12.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.008+0000 7fba5c0e4640 1 Processor -- start 2024-04-16T05:39:12.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5c0e4640 1 -- start start 2024-04-16T05:39:12.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5c0e4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba54101130 0x7fba5419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5c0e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba54106200 0x7fba5419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5c0e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54197650 0x7fba54197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5c0e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba54111310 con 0x7fba54197650 2024-04-16T05:39:12.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5c0e4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba54111190 con 0x7fba54101130 2024-04-16T05:39:12.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5c0e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba54111490 con 0x7fba54106200 2024-04-16T05:39:12.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5a681640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54197650 0x7fba54197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5a681640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54197650 0x7fba54197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58286/0 (socket says 172.21.15.86:58286) 2024-04-16T05:39:12.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5a681640 1 -- 172.21.15.86:0/1857953947 learned_addr learned my addr 172.21.15.86:0/1857953947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba59e80640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba54101130 0x7fba5419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5967f640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba54106200 0x7fba5419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5a681640 1 -- 172.21.15.86:0/1857953947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba54106200 msgr2=0x7fba5419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5a681640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba54106200 0x7fba5419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5a681640 1 -- 172.21.15.86:0/1857953947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba54101130 msgr2=0x7fba5419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5a681640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba54101130 0x7fba5419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5a681640 1 -- 172.21.15.86:0/1857953947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba54198250 con 0x7fba54197650 2024-04-16T05:39:12.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba59e80640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba54101130 0x7fba5419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:12.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5a681640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54197650 0x7fba54197ad0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00eaa0 tx=0x7fba4c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba4affd640 1 -- 172.21.15.86:0/1857953947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00cdb0 con 0x7fba54197650 2024-04-16T05:39:12.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba4affd640 1 -- 172.21.15.86:0/1857953947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c005020 con 0x7fba54197650 2024-04-16T05:39:12.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba5c0e4640 1 -- 172.21.15.86:0/1857953947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba541a4330 con 0x7fba54197650 2024-04-16T05:39:12.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.009+0000 7fba4affd640 1 -- 172.21.15.86:0/1857953947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0116b0 con 0x7fba54197650 2024-04-16T05:39:12.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.010+0000 7fba5c0e4640 1 -- 172.21.15.86:0/1857953947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba541a46e0 con 0x7fba54197650 2024-04-16T05:39:12.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.012+0000 7fba4affd640 1 -- 172.21.15.86:0/1857953947 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba4c005560 con 0x7fba54197650 2024-04-16T05:39:12.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.012+0000 7fba5edaf740 1 Processor -- start 2024-04-16T05:39:12.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.012+0000 7fba5edaf740 1 -- start start 2024-04-16T05:39:12.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.012+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a6f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.012+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 0x565283a77e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.012+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a78360 0x565283a7a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.012+0000 7fba5edaf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5652839d40e0 con 0x565283a6ee60 2024-04-16T05:39:12.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.012+0000 7fba5edaf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x565283971520 con 0x565283a6f780 2024-04-16T05:39:12.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.012+0000 7fba5edaf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5652839d00a0 con 0x565283a78360 2024-04-16T05:39:12.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a6f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a6f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58290/0 (socket says 172.21.15.86:58290) 2024-04-16T05:39:12.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 -- 172.21.15.86:0/800410481 learned_addr learned my addr 172.21.15.86:0/800410481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba4affd640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba400a3d20 0x7fba400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 --2- 172.21.15.86:0/800410481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a78360 0x565283a7a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5a681640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba400a3d20 0x7fba400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 -- 172.21.15.86:0/800410481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a78360 msgr2=0x565283a7a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 --2- 172.21.15.86:0/800410481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a78360 0x565283a7a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 -- 172.21.15.86:0/800410481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 msgr2=0x565283a77e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 --2- 172.21.15.86:0/800410481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 0x565283a77e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 -- 172.21.15.86:0/800410481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652839998d0 con 0x565283a6ee60 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba4affd640 1 -- 172.21.15.86:0/1857953947 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fba4c0d0eb0 con 0x7fba54197650 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5967f640 1 --2- 172.21.15.86:0/800410481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a6f240 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fba4400be70 tx=0x7fba4400b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54142b0d1fdd7599 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba3ffff640 1 -- 172.21.15.86:0/800410481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4400fe60 con 0x565283a6ee60 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba5a681640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba400a3d20 0x7fba400a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fba4c098f00 tx=0x7fba4c0875d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba3ffff640 1 -- 172.21.15.86:0/800410481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba44010040 con 0x565283a6ee60 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.013+0000 7fba3ffff640 1 -- 172.21.15.86:0/800410481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44016900 con 0x565283a6ee60 2024-04-16T05:39:12.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 -- 172.21.15.86:0/800410481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 msgr2=0x565283a6f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 --2- 172.21.15.86:0/800410481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a6f240 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fba4400be70 tx=0x7fba4400b400 comp rx=0 tx=0).stop 2024-04-16T05:39:12.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 -- 172.21.15.86:0/800410481 shutdown_connections 2024-04-16T05:39:12.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 --2- 172.21.15.86:0/800410481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a78360 0x565283a7a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 --2- 172.21.15.86:0/800410481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 0x565283a77e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 --2- 172.21.15.86:0/800410481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a6f240 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 -- 172.21.15.86:0/800410481 >> 172.21.15.86:0/800410481 conn(0x565283a69590 msgr2=0x565283a69970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 -- 172.21.15.86:0/800410481 shutdown_connections 2024-04-16T05:39:12.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 -- 172.21.15.86:0/800410481 wait complete. 2024-04-16T05:39:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 -- start start 2024-04-16T05:39:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.014+0000 7fba5edaf740 1 Processor -- start 2024-04-16T05:39:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a6ee60 0x565283a75390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6f780 0x565283a758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a78360 0x565283a75e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba5edaf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565283a7b000 con 0x565283a6f780 2024-04-16T05:39:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba5edaf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56528398d4d0 con 0x565283a78360 2024-04-16T05:39:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba5edaf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565283a7b180 con 0x565283a6ee60 2024-04-16T05:39:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba5edaf740 10 client.0 _ll_register_callbacks cb 0x5652839722c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:12.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba59e80640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6f780 0x565283a758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba59e80640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6f780 0x565283a758d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58300/0 (socket says 172.21.15.86:58300) 2024-04-16T05:39:12.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba59e80640 1 -- 172.21.15.86:0/3926101115 learned_addr learned my addr 172.21.15.86:0/3926101115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba5967f640 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a6ee60 0x565283a75390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba59e80640 1 -- 172.21.15.86:0/3926101115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a6ee60 msgr2=0x565283a75390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba59e80640 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a6ee60 0x565283a75390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba59e80640 1 -- 172.21.15.86:0/3926101115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a78360 msgr2=0x565283a75e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba59e80640 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a78360 0x565283a75e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba59e80640 1 -- 172.21.15.86:0/3926101115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652836e63f0 con 0x565283a6f780 2024-04-16T05:39:12.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba59e80640 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6f780 0x565283a758d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fba50009900 tx=0x7fba50009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a853ed359accdae server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.015+0000 7fba3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:12.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba3d7fa640 1 -- 172.21.15.86:0/3926101115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50041070 con 0x565283a6f780 2024-04-16T05:39:12.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba3d7fa640 1 -- 172.21.15.86:0/3926101115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fba50005050 con 0x565283a6f780 2024-04-16T05:39:12.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba5edaf740 1 -- 172.21.15.86:0/3926101115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba44010040 con 0x565283a6f780 2024-04-16T05:39:12.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba3d7fa640 1 -- 172.21.15.86:0/3926101115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba5004d050 con 0x565283a6f780 2024-04-16T05:39:12.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba5edaf740 10 client.16878 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:12.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba5edaf740 1 -- 172.21.15.86:0/3926101115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5652837b07d0 con 0x565283a6f780 2024-04-16T05:39:12.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba3d7fa640 1 -- 172.21.15.86:0/3926101115 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba44010040 con 0x565283a6f780 2024-04-16T05:39:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba3d7fa640 1 -- 172.21.15.86:0/3926101115 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba500035f0 con 0x565283a6f780 2024-04-16T05:39:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba5edaf740 10 client.16878 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba5edaf740 10 client.16878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:39:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba5edaf740 20 client.16878 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x565283a88290 0x565283a8a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba5edaf740 4 client.16878 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.016+0000 7fba5edaf740 1 -- 172.21.15.86:0/3926101115 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652839e1b00 con 0x565283a88290 2024-04-16T05:39:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.017+0000 7fba5967f640 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x565283a88290 0x565283a8a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.017+0000 7fba5967f640 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x565283a88290 0x565283a8a6d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9475a5a91e03f7e server_cookie=1ca4aee60d1a9f84 in_seq=0 out_seq=0 2024-04-16T05:39:12.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.017+0000 7fba3d7fa640 10 client.16878 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:39:12.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba3d7fa640 1 -- 172.21.15.86:0/3926101115 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652839e1b00 con 0x565283a88290 2024-04-16T05:39:12.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba3d7fa640 10 client.16878 handle_command_reply: tid=0 2024-04-16T05:39:12.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 client.16878 shutdown 2024-04-16T05:39:12.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 -- 172.21.15.86:0/3926101115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6f780 msgr2=0x565283a758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6f780 0x565283a758d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fba50009900 tx=0x7fba50009630 comp rx=0 tx=0).stop 2024-04-16T05:39:12.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 -- 172.21.15.86:0/3926101115 shutdown_connections 2024-04-16T05:39:12.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a78360 0x565283a75e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x565283a88290 0x565283a8a6d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6f780 0x565283a758d0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba3d7fa640 0 client.16878 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:39:12.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3926101115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a6ee60 0x565283a75390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 -- 172.21.15.86:0/3926101115 >> 172.21.15.86:0/3926101115 conn(0x565283a69590 msgr2=0x565283a6b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 -- 172.21.15.86:0/3926101115 shutdown_connections 2024-04-16T05:39:12.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 1 -- 172.21.15.86:0/3926101115 wait complete. 2024-04-16T05:39:12.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.018+0000 7fba5edaf740 20 client.16878 trim_cache size 0 max 16384 2024-04-16T05:39:12.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.031+0000 7fba5edaf740 1 Processor -- start 2024-04-16T05:39:12.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.031+0000 7fba5edaf740 1 -- start start 2024-04-16T05:39:12.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a77ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a6f780 0x565283a84030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a78360 0x565283a8ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5edaf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba541ae300 con 0x565283a6ee60 2024-04-16T05:39:12.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5edaf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba541afe40 con 0x565283a78360 2024-04-16T05:39:12.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5edaf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba541ae9d0 con 0x565283a6f780 2024-04-16T05:39:12.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5967f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a77ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5967f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a77ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58306/0 (socket says 172.21.15.86:58306) 2024-04-16T05:39:12.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5967f640 1 -- 172.21.15.86:0/3132413187 learned_addr learned my addr 172.21.15.86:0/3132413187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5967f640 1 -- 172.21.15.86:0/3132413187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a6f780 msgr2=0x565283a84030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5967f640 1 --2- 172.21.15.86:0/3132413187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a6f780 0x565283a84030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5967f640 1 -- 172.21.15.86:0/3132413187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a78360 msgr2=0x565283a8ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5967f640 1 --2- 172.21.15.86:0/3132413187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a78360 0x565283a8ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5967f640 1 -- 172.21.15.86:0/3132413187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565283a70f90 con 0x565283a6ee60 2024-04-16T05:39:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba5967f640 1 --2- 172.21.15.86:0/3132413187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a77ac0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fba440164b0 tx=0x7fba44016440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d789502a11ba17b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba3f7fe640 1 -- 172.21.15.86:0/3132413187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4402e070 con 0x565283a6ee60 2024-04-16T05:39:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba3f7fe640 1 -- 172.21.15.86:0/3132413187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4400b470 con 0x565283a6ee60 2024-04-16T05:39:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.032+0000 7fba3f7fe640 1 -- 172.21.15.86:0/3132413187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44016970 con 0x565283a6ee60 2024-04-16T05:39:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 -- 172.21.15.86:0/3132413187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 msgr2=0x565283a77ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3132413187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a77ac0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fba440164b0 tx=0x7fba44016440 comp rx=0 tx=0).stop 2024-04-16T05:39:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 -- 172.21.15.86:0/3132413187 shutdown_connections 2024-04-16T05:39:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3132413187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a78360 0x565283a8ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3132413187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a6f780 0x565283a84030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 --2- 172.21.15.86:0/3132413187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a77ac0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 -- 172.21.15.86:0/3132413187 >> 172.21.15.86:0/3132413187 conn(0x565283a69590 msgr2=0x565283a645c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 -- 172.21.15.86:0/3132413187 shutdown_connections 2024-04-16T05:39:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 -- 172.21.15.86:0/3132413187 wait complete. 2024-04-16T05:39:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 -- start start 2024-04-16T05:39:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.033+0000 7fba5edaf740 1 Processor -- start 2024-04-16T05:39:12.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 0x565283a72720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a78360 0x565283a72c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba541af770 con 0x565283a6ee60 2024-04-16T05:39:12.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba541af0a0 con 0x565283a6f780 2024-04-16T05:39:12.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba541ac2c0 con 0x565283a78360 2024-04-16T05:39:12.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 10 client.0 _ll_register_callbacks cb 0x7fba50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:12.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5967f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5967f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58308/0 (socket says 172.21.15.86:58308) 2024-04-16T05:39:12.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5967f640 1 -- 172.21.15.86:0/142757237 learned_addr learned my addr 172.21.15.86:0/142757237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba59e80640 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 0x565283a72720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5967f640 1 -- 172.21.15.86:0/142757237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a78360 msgr2=0x565283a72c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5967f640 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a78360 0x565283a72c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5967f640 1 -- 172.21.15.86:0/142757237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 msgr2=0x565283a72720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5967f640 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 0x565283a72720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5967f640 1 -- 172.21.15.86:0/142757237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565283aed970 con 0x565283a6ee60 2024-04-16T05:39:12.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba59e80640 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 0x565283a72720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:12.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5967f640 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a721e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fba4400bc00 tx=0x7fba4400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85da3f14df600e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:12.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba3d7fa640 1 -- 172.21.15.86:0/142757237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4402e050 con 0x565283a6ee60 2024-04-16T05:39:12.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba3d7fa640 1 -- 172.21.15.86:0/142757237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fba44028d80 con 0x565283a6ee60 2024-04-16T05:39:12.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 1 -- 172.21.15.86:0/142757237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba4400b470 con 0x565283a6ee60 2024-04-16T05:39:12.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba3d7fa640 1 -- 172.21.15.86:0/142757237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba4403f050 con 0x565283a6ee60 2024-04-16T05:39:12.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 10 client.16884 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:12.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.034+0000 7fba5edaf740 1 -- 172.21.15.86:0/142757237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565283aeec50 con 0x565283a6ee60 2024-04-16T05:39:12.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba3d7fa640 1 -- 172.21.15.86:0/142757237 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4400b470 con 0x565283a6ee60 2024-04-16T05:39:12.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba3d7fa640 1 -- 172.21.15.86:0/142757237 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba440166f0 con 0x565283a6ee60 2024-04-16T05:39:12.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba5edaf740 10 client.16884 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:12.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba5edaf740 10 client.16884 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:39:12.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba5edaf740 20 client.16884 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:12.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba5edaf740 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x565283aef020 0x565283af1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba5edaf740 4 client.16884 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:12.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba5edaf740 1 -- 172.21.15.86:0/142757237 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652839db970 con 0x565283aef020 2024-04-16T05:39:12.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba59e80640 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x565283aef020 0x565283af1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba59e80640 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x565283aef020 0x565283af1410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d0f6818e8b993bf server_cookie=f9853ca7ce9152a5 in_seq=0 out_seq=0 2024-04-16T05:39:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.035+0000 7fba3d7fa640 10 client.16884 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:39:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba3d7fa640 1 -- 172.21.15.86:0/142757237 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652839db970 con 0x565283aef020 2024-04-16T05:39:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba3d7fa640 10 client.16884 handle_command_reply: tid=0 2024-04-16T05:39:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 client.16884 shutdown 2024-04-16T05:39:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 -- 172.21.15.86:0/142757237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 msgr2=0x565283a721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a721e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fba4400bc00 tx=0x7fba4400b940 comp rx=0 tx=0).stop 2024-04-16T05:39:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 -- 172.21.15.86:0/142757237 shutdown_connections 2024-04-16T05:39:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565283a78360 0x565283a72c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565283a6f780 0x565283a72720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x565283aef020 0x565283af1410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 --2- 172.21.15.86:0/142757237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565283a6ee60 0x565283a721e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 -- 172.21.15.86:0/142757237 >> 172.21.15.86:0/142757237 conn(0x565283a69590 msgr2=0x565283a75330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba3d7fa640 0 client.16884 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:39:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 -- 172.21.15.86:0/142757237 shutdown_connections 2024-04-16T05:39:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 1 -- 172.21.15.86:0/142757237 wait complete. 2024-04-16T05:39:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5edaf740 20 client.16884 trim_cache size 0 max 16384 2024-04-16T05:39:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5c0e4640 1 -- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba400a3d20 msgr2=0x7fba400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba400a3d20 0x7fba400a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fba4c098f00 tx=0x7fba4c0875d0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5c0e4640 1 -- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54197650 msgr2=0x7fba54197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.036+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54197650 0x7fba54197ad0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00eaa0 tx=0x7fba4c00ef60 comp rx=0 tx=0).stop 2024-04-16T05:39:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.037+0000 7fba5c0e4640 1 -- 172.21.15.86:0/1857953947 shutdown_connections 2024-04-16T05:39:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.037+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba54197650 0x7fba54197ad0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.037+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba54106200 0x7fba5419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.037+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba400a3d20 0x7fba400a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.037+0000 7fba5c0e4640 1 --2- 172.21.15.86:0/1857953947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba54101130 0x7fba5419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.037+0000 7fba5c0e4640 1 -- 172.21.15.86:0/1857953947 >> 172.21.15.86:0/1857953947 conn(0x7fba540780b0 msgr2=0x7fba540ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.037+0000 7fba5c0e4640 1 -- 172.21.15.86:0/1857953947 shutdown_connections 2024-04-16T05:39:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.037+0000 7fba5c0e4640 1 -- 172.21.15.86:0/1857953947 wait complete. 2024-04-16T05:39:12.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:39:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.156+0000 7f1c1feba640 1 -- 172.21.15.86:0/3727618485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18101670 msgr2=0x7f1c18101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.156+0000 7f1c1feba640 1 --2- 172.21.15.86:0/3727618485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18101670 0x7f1c18101ad0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009900 tx=0x7f1c0802bc80 comp rx=0 tx=0).stop 2024-04-16T05:39:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.156+0000 7f1c1feba640 1 -- 172.21.15.86:0/3727618485 shutdown_connections 2024-04-16T05:39:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.156+0000 7f1c1feba640 1 --2- 172.21.15.86:0/3727618485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c18102010 0x7f1c1810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.156+0000 7f1c1feba640 1 --2- 172.21.15.86:0/3727618485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18101670 0x7f1c18101ad0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.156+0000 7f1c1feba640 1 --2- 172.21.15.86:0/3727618485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c18105ed0 0x7f1c181062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.156+0000 7f1c1feba640 1 -- 172.21.15.86:0/3727618485 >> 172.21.15.86:0/3727618485 conn(0x7f1c180774c0 msgr2=0x7f1c180fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.156+0000 7f1c1feba640 1 -- 172.21.15.86:0/3727618485 shutdown_connections 2024-04-16T05:39:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.156+0000 7f1c1feba640 1 -- 172.21.15.86:0/3727618485 wait complete. 2024-04-16T05:39:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1feba640 1 Processor -- start 2024-04-16T05:39:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1feba640 1 -- start start 2024-04-16T05:39:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1feba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c18101670 0x7f1c1819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1feba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18102010 0x7f1c1819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1feba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c18105ed0 0x7f1c181986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1feba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181124f0 con 0x7f1c18102010 2024-04-16T05:39:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1feba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18112370 con 0x7f1c18101670 2024-04-16T05:39:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1feba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18112670 con 0x7f1c18105ed0 2024-04-16T05:39:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1d455640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18102010 0x7f1c1819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1d455640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18102010 0x7f1c1819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58336/0 (socket says 172.21.15.86:58336) 2024-04-16T05:39:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1d455640 1 -- 172.21.15.86:0/2816044212 learned_addr learned my addr 172.21.15.86:0/2816044212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c18101670 0x7f1c1819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1d455640 1 -- 172.21.15.86:0/2816044212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c18105ed0 msgr2=0x7f1c181986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1d455640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c18105ed0 0x7f1c181986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1d455640 1 -- 172.21.15.86:0/2816044212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c18101670 msgr2=0x7f1c1819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1d455640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c18101670 0x7f1c1819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1d455640 1 -- 172.21.15.86:0/2816044212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c18198fb0 con 0x7f1c18102010 2024-04-16T05:39:12.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c1d455640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18102010 0x7f1c1819eb70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1c08006010 tx=0x7f1c080043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.157+0000 7f1c0effd640 1 -- 172.21.15.86:0/2816044212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08041070 con 0x7f1c18102010 2024-04-16T05:39:12.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.158+0000 7f1c1feba640 1 -- 172.21.15.86:0/2816044212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c18199240 con 0x7f1c18102010 2024-04-16T05:39:12.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.158+0000 7f1c0effd640 1 -- 172.21.15.86:0/2816044212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c0802ccb0 con 0x7f1c18102010 2024-04-16T05:39:12.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.158+0000 7f1c0effd640 1 -- 172.21.15.86:0/2816044212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08003e20 con 0x7f1c18102010 2024-04-16T05:39:12.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.158+0000 7f1c1feba640 1 -- 172.21.15.86:0/2816044212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181a55b0 con 0x7f1c18102010 2024-04-16T05:39:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.160+0000 7f1c0effd640 1 -- 172.21.15.86:0/2816044212 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c08044070 con 0x7f1c18102010 2024-04-16T05:39:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.160+0000 7f1c22b80740 1 Processor -- start 2024-04-16T05:39:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.160+0000 7f1c22b80740 1 -- start start 2024-04-16T05:39:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ebb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c22b80740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5c0e200e0 con 0x55f5c0ec4360 2024-04-16T05:39:12.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c22b80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f5c0dbd520 con 0x55f5c0ebb780 2024-04-16T05:39:12.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c22b80740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5c0e1c0a0 con 0x55f5c0ebae60 2024-04-16T05:39:12.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1d455640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c0effd640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1c040a3d20 0x7f1c040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1dc56640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1d455640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58344/0 (socket says 172.21.15.86:58344) 2024-04-16T05:39:12.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1d455640 1 -- 172.21.15.86:0/1732217175 learned_addr learned my addr 172.21.15.86:0/1732217175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1e457640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1c040a3d20 0x7f1c040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1e457640 1 --2- 172.21.15.86:0/1732217175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ebb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c0effd640 1 -- 172.21.15.86:0/2816044212 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1c080f2630 con 0x7f1c18102010 2024-04-16T05:39:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1dc56640 1 -- 172.21.15.86:0/1732217175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 msgr2=0x55f5c0ebb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/1732217175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ebb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1dc56640 1 -- 172.21.15.86:0/1732217175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 msgr2=0x55f5c0ec6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/1732217175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1dc56640 1 -- 172.21.15.86:0/1732217175 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c0de58d0 con 0x55f5c0ebb780 2024-04-16T05:39:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1e457640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1c040a3d20 0x7f1c040a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005c40 tx=0x7f1c100094a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.161+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/1732217175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec3e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ab40 tx=0x7f1c1400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ea981fa1aadf65 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c03fff640 1 -- 172.21.15.86:0/1732217175 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14004020 con 0x55f5c0ebb780 2024-04-16T05:39:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c03fff640 1 -- 172.21.15.86:0/1732217175 <== mon.2 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c140041c0 con 0x55f5c0ebb780 2024-04-16T05:39:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c03fff640 1 -- 172.21.15.86:0/1732217175 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14018800 con 0x55f5c0ebb780 2024-04-16T05:39:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c22b80740 1 -- 172.21.15.86:0/1732217175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 msgr2=0x55f5c0ec3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c22b80740 1 --2- 172.21.15.86:0/1732217175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec3e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ab40 tx=0x7f1c1400f3c0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c22b80740 1 -- 172.21.15.86:0/1732217175 shutdown_connections 2024-04-16T05:39:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c22b80740 1 --2- 172.21.15.86:0/1732217175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c22b80740 1 --2- 172.21.15.86:0/1732217175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec3e20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c22b80740 1 --2- 172.21.15.86:0/1732217175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ebb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.162+0000 7f1c22b80740 1 -- 172.21.15.86:0/1732217175 >> 172.21.15.86:0/1732217175 conn(0x55f5c0eb5590 msgr2=0x55f5c0eb5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 -- 172.21.15.86:0/1732217175 shutdown_connections 2024-04-16T05:39:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 -- 172.21.15.86:0/1732217175 wait complete. 2024-04-16T05:39:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 -- start start 2024-04-16T05:39:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 Processor -- start 2024-04-16T05:39:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5c0ec70c0 con 0x55f5c0ec4360 2024-04-16T05:39:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f5c0dd94d0 con 0x55f5c0ebb780 2024-04-16T05:39:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5c0ec7240 con 0x55f5c0ebae60 2024-04-16T05:39:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.163+0000 7f1c22b80740 10 client.0 _ll_register_callbacks cb 0x55f5c0dbe2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1e457640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1e457640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58348/0 (socket says 172.21.15.86:58348) 2024-04-16T05:39:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1e457640 1 -- 172.21.15.86:0/2922358648 learned_addr learned my addr 172.21.15.86:0/2922358648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1d455640 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1e457640 1 -- 172.21.15.86:0/2922358648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 msgr2=0x55f5c0ec1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1e457640 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1e457640 1 -- 172.21.15.86:0/2922358648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 msgr2=0x55f5c0ec18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1e457640 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1e457640 1 -- 172.21.15.86:0/2922358648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c0b323f0 con 0x55f5c0ec4360 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c1e457640 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec1de0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1c10012490 tx=0x7f1c10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448df17af39b0b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c017fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c017fa640 1 -- 172.21.15.86:0/2922358648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10010cd0 con 0x55f5c0ec4360 2024-04-16T05:39:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c22b80740 1 -- 172.21.15.86:0/2922358648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c140041c0 con 0x55f5c0ec4360 2024-04-16T05:39:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c017fa640 1 -- 172.21.15.86:0/2922358648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1c10011470 con 0x55f5c0ec4360 2024-04-16T05:39:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c017fa640 1 -- 172.21.15.86:0/2922358648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c1002b620 con 0x55f5c0ec4360 2024-04-16T05:39:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c22b80740 10 client.16896 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c22b80740 1 -- 172.21.15.86:0/2922358648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5c0bfc7d0 con 0x55f5c0ec4360 2024-04-16T05:39:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.164+0000 7f1c017fa640 1 -- 172.21.15.86:0/2922358648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c140041c0 con 0x55f5c0ec4360 2024-04-16T05:39:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.165+0000 7f1c017fa640 1 -- 172.21.15.86:0/2922358648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1c10026430 con 0x55f5c0ec4360 2024-04-16T05:39:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.165+0000 7f1c22b80740 10 client.16896 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.165+0000 7f1c22b80740 10 client.16896 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:39:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.165+0000 7f1c22b80740 20 client.16896 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.165+0000 7f1c22b80740 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55f5c0ed47b0 0x55f5c0ed6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.165+0000 7f1c22b80740 4 client.16896 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.165+0000 7f1c22b80740 1 -- 172.21.15.86:0/2922358648 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c0e2db00 con 0x55f5c0ed47b0 2024-04-16T05:39:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.165+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55f5c0ed47b0 0x55f5c0ed6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c017fa640 10 client.16896 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:39:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55f5c0ed47b0 0x55f5c0ed6c30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f7f25f16b2c50bd server_cookie=769a640c49d51304 in_seq=0 out_seq=0 2024-04-16T05:39:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c017fa640 1 -- 172.21.15.86:0/2922358648 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5c0e2db00 con 0x55f5c0ed47b0 2024-04-16T05:39:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c017fa640 10 client.16896 handle_command_reply: tid=0 2024-04-16T05:39:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c22b80740 1 client.16896 shutdown 2024-04-16T05:39:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c22b80740 1 -- 172.21.15.86:0/2922358648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 msgr2=0x55f5c0ec1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c22b80740 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec1de0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1c10012490 tx=0x7f1c10004a40 comp rx=0 tx=0).stop 2024-04-16T05:39:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c22b80740 1 -- 172.21.15.86:0/2922358648 shutdown_connections 2024-04-16T05:39:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c22b80740 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55f5c0ed47b0 0x55f5c0ed6c30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c22b80740 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ec4360 0x55f5c0ec1de0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c22b80740 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.166+0000 7f1c22b80740 1 --2- 172.21.15.86:0/2922358648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.167+0000 7f1c22b80740 1 -- 172.21.15.86:0/2922358648 >> 172.21.15.86:0/2922358648 conn(0x55f5c0eb5590 msgr2=0x55f5c0eb72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.167+0000 7f1c017fa640 0 client.16896 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:39:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.167+0000 7f1c22b80740 1 -- 172.21.15.86:0/2922358648 shutdown_connections 2024-04-16T05:39:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.167+0000 7f1c22b80740 1 -- 172.21.15.86:0/2922358648 wait complete. 2024-04-16T05:39:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.167+0000 7f1c22b80740 20 client.16896 trim_cache size 0 max 16384 2024-04-16T05:39:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c22b80740 1 Processor -- start 2024-04-16T05:39:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c22b80740 1 -- start start 2024-04-16T05:39:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ed0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ec4360 0x55f5c0eb45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c22b80740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181af3e0 con 0x55f5c0ebb780 2024-04-16T05:39:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c22b80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181b0f20 con 0x55f5c0ec4360 2024-04-16T05:39:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c22b80740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181afab0 con 0x55f5c0ebae60 2024-04-16T05:39:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c1d455640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ed0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c1d455640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ed0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58358/0 (socket says 172.21.15.86:58358) 2024-04-16T05:39:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c1d455640 1 -- 172.21.15.86:0/3274194720 learned_addr learned my addr 172.21.15.86:0/3274194720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c1d455640 1 -- 172.21.15.86:0/3274194720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 msgr2=0x55f5c0ec3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c1d455640 1 --2- 172.21.15.86:0/3274194720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c1d455640 1 -- 172.21.15.86:0/3274194720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ec4360 msgr2=0x55f5c0eb45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c1d455640 1 --2- 172.21.15.86:0/3274194720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ec4360 0x55f5c0eb45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c1d455640 1 -- 172.21.15.86:0/3274194720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c0ebb5b0 con 0x55f5c0ebb780 2024-04-16T05:39:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c1d455640 1 --2- 172.21.15.86:0/3274194720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ed0070 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1c080be2c0 tx=0x7f1c08005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64908e1f0ac376 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c037fe640 1 -- 172.21.15.86:0/3274194720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0804b040 con 0x55f5c0ebb780 2024-04-16T05:39:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c037fe640 1 -- 172.21.15.86:0/3274194720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c080bea30 con 0x55f5c0ebb780 2024-04-16T05:39:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.180+0000 7f1c037fe640 1 -- 172.21.15.86:0/3274194720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c080efc50 con 0x55f5c0ebb780 2024-04-16T05:39:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 -- 172.21.15.86:0/3274194720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 msgr2=0x55f5c0ed0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 --2- 172.21.15.86:0/3274194720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ed0070 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1c080be2c0 tx=0x7f1c08005700 comp rx=0 tx=0).stop 2024-04-16T05:39:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 -- 172.21.15.86:0/3274194720 shutdown_connections 2024-04-16T05:39:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 --2- 172.21.15.86:0/3274194720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ec4360 0x55f5c0eb45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 --2- 172.21.15.86:0/3274194720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ed0070 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 --2- 172.21.15.86:0/3274194720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 -- 172.21.15.86:0/3274194720 >> 172.21.15.86:0/3274194720 conn(0x55f5c0eb5590 msgr2=0x55f5c0eb3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 -- 172.21.15.86:0/3274194720 shutdown_connections 2024-04-16T05:39:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 -- 172.21.15.86:0/3274194720 wait complete. 2024-04-16T05:39:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 -- start start 2024-04-16T05:39:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.181+0000 7f1c22b80740 1 Processor -- start 2024-04-16T05:39:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c22b80740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ec4360 0x55f5c0f38da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c22b80740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181b0850 con 0x55f5c0ebb780 2024-04-16T05:39:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c22b80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181b0180 con 0x55f5c0ebae60 2024-04-16T05:39:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c22b80740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181ad3a0 con 0x55f5c0ec4360 2024-04-16T05:39:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c22b80740 10 client.0 _ll_register_callbacks cb 0x7f1c10012140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1d455640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1d455640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58374/0 (socket says 172.21.15.86:58374) 2024-04-16T05:39:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1d455640 1 -- 172.21.15.86:0/146159113 learned_addr learned my addr 172.21.15.86:0/146159113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1d455640 1 -- 172.21.15.86:0/146159113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ec4360 msgr2=0x55f5c0f38da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1d455640 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ec4360 0x55f5c0f38da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1d455640 1 -- 172.21.15.86:0/146159113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebae60 msgr2=0x55f5c0ec7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1d455640 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1d455640 1 -- 172.21.15.86:0/146159113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c0f393b0 con 0x55f5c0ebb780 2024-04-16T05:39:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c1d455640 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec7cd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1c080be030 tx=0x7f1c08049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491a420a9163c81e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c017fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.182+0000 7f1c017fa640 1 -- 172.21.15.86:0/146159113 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0804b070 con 0x55f5c0ebb780 2024-04-16T05:39:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c017fa640 1 -- 172.21.15.86:0/146159113 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1c080c6d10 con 0x55f5c0ebb780 2024-04-16T05:39:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c22b80740 1 -- 172.21.15.86:0/146159113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c080bea30 con 0x55f5c0ebb780 2024-04-16T05:39:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c017fa640 1 -- 172.21.15.86:0/146159113 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c080c6fa0 con 0x55f5c0ebb780 2024-04-16T05:39:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c22b80740 10 client.16902 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c22b80740 1 -- 172.21.15.86:0/146159113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5c0f3a6b0 con 0x55f5c0ebb780 2024-04-16T05:39:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c017fa640 1 -- 172.21.15.86:0/146159113 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c080bea30 con 0x55f5c0ebb780 2024-04-16T05:39:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c017fa640 1 -- 172.21.15.86:0/146159113 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1c080bd780 con 0x55f5c0ebb780 2024-04-16T05:39:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c22b80740 10 client.16902 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c22b80740 10 client.16902 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:39:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c22b80740 20 client.16902 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c22b80740 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55f5c0f3ae30 0x55f5c0f3d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c22b80740 4 client.16902 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c22b80740 1 -- 172.21.15.86:0/146159113 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5c0e27970 con 0x55f5c0f3ae30 2024-04-16T05:39:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.183+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55f5c0f3ae30 0x55f5c0f3d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c1dc56640 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55f5c0f3ae30 0x55f5c0f3d250 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31f054816ab76875 server_cookie=46fe0a6084c2830 in_seq=0 out_seq=0 2024-04-16T05:39:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c017fa640 10 client.16902 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:39:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c017fa640 1 -- 172.21.15.86:0/146159113 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5c0e27970 con 0x55f5c0f3ae30 2024-04-16T05:39:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c017fa640 10 client.16902 handle_command_reply: tid=0 2024-04-16T05:39:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 client.16902 shutdown 2024-04-16T05:39:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 -- 172.21.15.86:0/146159113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 msgr2=0x55f5c0ec7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec7cd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1c080be030 tx=0x7f1c08049000 comp rx=0 tx=0).stop 2024-04-16T05:39:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 -- 172.21.15.86:0/146159113 shutdown_connections 2024-04-16T05:39:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55f5c0f3ae30 0x55f5c0f3d250 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c0ec4360 0x55f5c0f38da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c0ebb780 0x55f5c0ec7cd0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 --2- 172.21.15.86:0/146159113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f5c0ebae60 0x55f5c0ec7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c017fa640 0 client.16902 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:39:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 -- 172.21.15.86:0/146159113 >> 172.21.15.86:0/146159113 conn(0x55f5c0eb5590 msgr2=0x55f5c0eb4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 -- 172.21.15.86:0/146159113 shutdown_connections 2024-04-16T05:39:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 1 -- 172.21.15.86:0/146159113 wait complete. 2024-04-16T05:39:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.184+0000 7f1c22b80740 20 client.16902 trim_cache size 0 max 16384 2024-04-16T05:39:12.193 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:12.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 -- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1c040a3d20 msgr2=0x7f1c040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1c040a3d20 0x7f1c040a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005c40 tx=0x7f1c100094a0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 -- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18102010 msgr2=0x7f1c1819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18102010 0x7f1c1819eb70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1c08006010 tx=0x7f1c080043b0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 -- 172.21.15.86:0/2816044212 shutdown_connections 2024-04-16T05:39:12.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c18105ed0 0x7f1c181986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c18102010 0x7f1c1819eb70 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1c040a3d20 0x7f1c040a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 --2- 172.21.15.86:0/2816044212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c18101670 0x7f1c1819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 -- 172.21.15.86:0/2816044212 >> 172.21.15.86:0/2816044212 conn(0x7f1c180774c0 msgr2=0x7f1c18103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 -- 172.21.15.86:0/2816044212 shutdown_connections 2024-04-16T05:39:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.185+0000 7f1c1feba640 1 -- 172.21.15.86:0/2816044212 wait complete. 2024-04-16T05:39:12.195 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:12 smithi086 ceph-mon[32028]: pgmap v120: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:12.195 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:12 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/404783007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:12.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:39:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.304+0000 7fd620efb640 1 -- 172.21.15.86:0/1996625702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c107320 msgr2=0x7fd61c10eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.304+0000 7fd620efb640 1 --2- 172.21.15.86:0/1996625702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c107320 0x7fd61c10eef0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ca0 tx=0x7fd61402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.304+0000 7fd620efb640 1 -- 172.21.15.86:0/1996625702 shutdown_connections 2024-04-16T05:39:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.304+0000 7fd620efb640 1 --2- 172.21.15.86:0/1996625702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c107320 0x7fd61c10eef0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.304+0000 7fd620efb640 1 --2- 172.21.15.86:0/1996625702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd61c102c00 0x7fd61c106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.304+0000 7fd620efb640 1 --2- 172.21.15.86:0/1996625702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c102250 0x7fd61c102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.304+0000 7fd620efb640 1 -- 172.21.15.86:0/1996625702 >> 172.21.15.86:0/1996625702 conn(0x7fd61c079490 msgr2=0x7fd61c0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.304+0000 7fd620efb640 1 -- 172.21.15.86:0/1996625702 shutdown_connections 2024-04-16T05:39:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.304+0000 7fd620efb640 1 -- 172.21.15.86:0/1996625702 wait complete. 2024-04-16T05:39:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd620efb640 1 Processor -- start 2024-04-16T05:39:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd620efb640 1 -- start start 2024-04-16T05:39:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd620efb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c102250 0x7fd61c074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd620efb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c107320 0x7fd61c074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd620efb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd61c074d80 0x7fd61c1a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd620efb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1123c0 con 0x7fd61c102250 2024-04-16T05:39:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd620efb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c112240 con 0x7fd61c074d80 2024-04-16T05:39:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd620efb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c112540 con 0x7fd61c107320 2024-04-16T05:39:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd61a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c102250 0x7fd61c074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd61a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c102250 0x7fd61c074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58398/0 (socket says 172.21.15.86:58398) 2024-04-16T05:39:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd61a59c640 1 -- 172.21.15.86:0/2805775820 learned_addr learned my addr 172.21.15.86:0/2805775820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd61ad9d640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd61c074d80 0x7fd61c1a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd611d9b640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c107320 0x7fd61c074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd61a59c640 1 -- 172.21.15.86:0/2805775820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c107320 msgr2=0x7fd61c074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd61a59c640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c107320 0x7fd61c074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd61a59c640 1 -- 172.21.15.86:0/2805775820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd61c074d80 msgr2=0x7fd61c1a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd61a59c640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd61c074d80 0x7fd61c1a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd61a59c640 1 -- 172.21.15.86:0/2805775820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c1a70f0 con 0x7fd61c102250 2024-04-16T05:39:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.305+0000 7fd611d9b640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c107320 0x7fd61c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.306+0000 7fd61a59c640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c102250 0x7fd61c074300 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c890 tx=0x7fd60c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.306+0000 7fd613fff640 1 -- 172.21.15.86:0/2805775820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c004220 con 0x7fd61c102250 2024-04-16T05:39:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.306+0000 7fd620efb640 1 -- 172.21.15.86:0/2805775820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c1a73e0 con 0x7fd61c102250 2024-04-16T05:39:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.306+0000 7fd613fff640 1 -- 172.21.15.86:0/2805775820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60c0043c0 con 0x7fd61c102250 2024-04-16T05:39:12.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.306+0000 7fd613fff640 1 -- 172.21.15.86:0/2805775820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c004a00 con 0x7fd61c102250 2024-04-16T05:39:12.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.306+0000 7fd620efb640 1 -- 172.21.15.86:0/2805775820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a7980 con 0x7fd61c102250 2024-04-16T05:39:12.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.308+0000 7fd613fff640 1 -- 172.21.15.86:0/2805775820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd60c0079b0 con 0x7fd61c102250 2024-04-16T05:39:12.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.308+0000 7fd623bbd740 1 Processor -- start 2024-04-16T05:39:12.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.308+0000 7fd623bbd740 1 -- start start 2024-04-16T05:39:12.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 0x562b3153a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b31542e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31545750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd623bbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b3149f0e0 con 0x562b31543360 2024-04-16T05:39:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd623bbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562b3143c520 con 0x562b3153a780 2024-04-16T05:39:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd623bbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b3149b0a0 con 0x562b31539e60 2024-04-16T05:39:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd613fff640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6080a3d20 0x7fd6080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd611d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31545750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd61ad9d640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6080a3d20 0x7fd6080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd611d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31545750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58412/0 (socket says 172.21.15.86:58412) 2024-04-16T05:39:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd611d9b640 1 -- 172.21.15.86:0/3459815253 learned_addr learned my addr 172.21.15.86:0/3459815253 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd613fff640 1 -- 172.21.15.86:0/2805775820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd60c0d0900 con 0x7fd61c102250 2024-04-16T05:39:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd61a59c640 1 --2- 172.21.15.86:0/3459815253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b31542e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd61ad9d640 1 --2- 172.21.15.86:0/3459815253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 0x562b3153a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd611d9b640 1 -- 172.21.15.86:0/3459815253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 msgr2=0x562b3153a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd611d9b640 1 --2- 172.21.15.86:0/3459815253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 0x562b3153a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd611d9b640 1 -- 172.21.15.86:0/3459815253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 msgr2=0x562b31542e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd611d9b640 1 --2- 172.21.15.86:0/3459815253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b31542e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd611d9b640 1 -- 172.21.15.86:0/3459815253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b314648d0 con 0x562b31543360 2024-04-16T05:39:12.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd61ad9d640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6080a3d20 0x7fd6080a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd614006010 tx=0x7fd61402ce40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd611d9b640 1 --2- 172.21.15.86:0/3459815253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31545750 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd604009b70 tx=0x7fd60400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8f5e406aafa069 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd610d99640 1 -- 172.21.15.86:0/3459815253 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604016070 con 0x562b31543360 2024-04-16T05:39:12.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.309+0000 7fd610d99640 1 -- 172.21.15.86:0/3459815253 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd604005500 con 0x562b31543360 2024-04-16T05:39:12.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd610d99640 1 -- 172.21.15.86:0/3459815253 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604011470 con 0x562b31543360 2024-04-16T05:39:12.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd623bbd740 1 -- 172.21.15.86:0/3459815253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 msgr2=0x562b31545750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3459815253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31545750 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd604009b70 tx=0x7fd60400e6c0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd623bbd740 1 -- 172.21.15.86:0/3459815253 shutdown_connections 2024-04-16T05:39:12.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3459815253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31545750 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3459815253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b31542e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3459815253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 0x562b3153a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd623bbd740 1 -- 172.21.15.86:0/3459815253 >> 172.21.15.86:0/3459815253 conn(0x562b31534590 msgr2=0x562b31534970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd623bbd740 1 -- 172.21.15.86:0/3459815253 shutdown_connections 2024-04-16T05:39:12.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.310+0000 7fd623bbd740 1 -- 172.21.15.86:0/3459815253 wait complete. 2024-04-16T05:39:12.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd623bbd740 1 -- start start 2024-04-16T05:39:12.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd623bbd740 1 Processor -- start 2024-04-16T05:39:12.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b31539e60 0x562b31540380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3153a780 0x562b315408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31543360 0x562b31540e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd623bbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b31546060 con 0x562b3153a780 2024-04-16T05:39:12.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd623bbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562b314584d0 con 0x562b31539e60 2024-04-16T05:39:12.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd623bbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b315461e0 con 0x562b31543360 2024-04-16T05:39:12.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd623bbd740 10 client.0 _ll_register_callbacks cb 0x562b3143d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:12.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd611d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3153a780 0x562b315408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd611d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3153a780 0x562b315408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58424/0 (socket says 172.21.15.86:58424) 2024-04-16T05:39:12.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd611d9b640 1 -- 172.21.15.86:0/3579204247 learned_addr learned my addr 172.21.15.86:0/3579204247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd61a59c640 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b31539e60 0x562b31540380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd61ad9d640 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31543360 0x562b31540e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd611d9b640 1 -- 172.21.15.86:0/3579204247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31543360 msgr2=0x562b31540e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd611d9b640 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31543360 0x562b31540e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd611d9b640 1 -- 172.21.15.86:0/3579204247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b31539e60 msgr2=0x562b31540380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd611d9b640 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b31539e60 0x562b31540380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd611d9b640 1 -- 172.21.15.86:0/3579204247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b311b13f0 con 0x562b3153a780 2024-04-16T05:39:12.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd611d9b640 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3153a780 0x562b315408c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd604000c00 tx=0x7fd6040048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12440adc193a0bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.311+0000 7fd601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:12.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd601ffb640 1 -- 172.21.15.86:0/3579204247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604016040 con 0x562b3153a780 2024-04-16T05:39:12.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd601ffb640 1 -- 172.21.15.86:0/3579204247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd604004020 con 0x562b3153a780 2024-04-16T05:39:12.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd623bbd740 1 -- 172.21.15.86:0/3579204247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd604005500 con 0x562b3153a780 2024-04-16T05:39:12.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd601ffb640 1 -- 172.21.15.86:0/3579204247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd60402e050 con 0x562b3153a780 2024-04-16T05:39:12.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd623bbd740 10 client.16914 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:12.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd623bbd740 1 -- 172.21.15.86:0/3579204247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b3127b7d0 con 0x562b3153a780 2024-04-16T05:39:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd601ffb640 1 -- 172.21.15.86:0/3579204247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd604005500 con 0x562b3153a780 2024-04-16T05:39:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd601ffb640 1 -- 172.21.15.86:0/3579204247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd60400f040 con 0x562b3153a780 2024-04-16T05:39:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd623bbd740 10 client.16914 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd623bbd740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd623bbd740 20 client.16914 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562b315531e0 0x562b315555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd623bbd740 4 client.16914 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd623bbd740 1 -- 172.21.15.86:0/3579204247 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b314acb00 con 0x562b315531e0 2024-04-16T05:39:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.312+0000 7fd61a59c640 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562b315531e0 0x562b315555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.313+0000 7fd601ffb640 10 client.16914 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.313+0000 7fd61a59c640 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562b315531e0 0x562b315555d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0e4c2cad1fda19 server_cookie=bcd2a020dcd5fb6a in_seq=0 out_seq=0 2024-04-16T05:39:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd601ffb640 1 -- 172.21.15.86:0/3579204247 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b314acb00 con 0x562b315531e0 2024-04-16T05:39:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd601ffb640 10 client.16914 handle_command_reply: tid=0 2024-04-16T05:39:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 client.16914 shutdown 2024-04-16T05:39:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 -- 172.21.15.86:0/3579204247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3153a780 msgr2=0x562b315408c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3153a780 0x562b315408c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd604000c00 tx=0x7fd6040048f0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 -- 172.21.15.86:0/3579204247 shutdown_connections 2024-04-16T05:39:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562b315531e0 0x562b315555d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31543360 0x562b31540e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3153a780 0x562b315408c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd601ffb640 0 client.16914 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3579204247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b31539e60 0x562b31540380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 -- 172.21.15.86:0/3579204247 >> 172.21.15.86:0/3579204247 conn(0x562b31534590 msgr2=0x562b315362e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 -- 172.21.15.86:0/3579204247 shutdown_connections 2024-04-16T05:39:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 1 -- 172.21.15.86:0/3579204247 wait complete. 2024-04-16T05:39:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.314+0000 7fd623bbd740 20 client.16914 trim_cache size 0 max 16384 2024-04-16T05:39:12.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd623bbd740 1 Processor -- start 2024-04-16T05:39:12.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd623bbd740 1 -- start start 2024-04-16T05:39:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31539e60 0x562b3154f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b31540a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31543360 0x562b3155ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd623bbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b16e0 con 0x562b31539e60 2024-04-16T05:39:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd623bbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b3220 con 0x562b3153a780 2024-04-16T05:39:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd623bbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b1db0 con 0x562b31543360 2024-04-16T05:39:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd61a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31539e60 0x562b3154f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd61a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31539e60 0x562b3154f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58432/0 (socket says 172.21.15.86:58432) 2024-04-16T05:39:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd61a59c640 1 -- 172.21.15.86:0/502178319 learned_addr learned my addr 172.21.15.86:0/502178319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd61a59c640 1 -- 172.21.15.86:0/502178319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31543360 msgr2=0x562b3155ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd61a59c640 1 --2- 172.21.15.86:0/502178319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31543360 0x562b3155ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd61a59c640 1 -- 172.21.15.86:0/502178319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 msgr2=0x562b31540a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd61a59c640 1 --2- 172.21.15.86:0/502178319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b31540a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.327+0000 7fd61a59c640 1 -- 172.21.15.86:0/502178319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b3152eb40 con 0x562b31539e60 2024-04-16T05:39:12.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd61a59c640 1 --2- 172.21.15.86:0/502178319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31539e60 0x562b3154f010 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd60c09c4c0 tx=0x7fd60c0d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345c6d10ee87f2ec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd603fff640 1 -- 172.21.15.86:0/502178319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01d070 con 0x562b31539e60 2024-04-16T05:39:12.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd603fff640 1 -- 172.21.15.86:0/502178319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60c09c530 con 0x562b31539e60 2024-04-16T05:39:12.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd603fff640 1 -- 172.21.15.86:0/502178319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c081e60 con 0x562b31539e60 2024-04-16T05:39:12.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd623bbd740 1 -- 172.21.15.86:0/502178319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31539e60 msgr2=0x562b3154f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd623bbd740 1 --2- 172.21.15.86:0/502178319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31539e60 0x562b3154f010 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd60c09c4c0 tx=0x7fd60c0d0040 comp rx=0 tx=0).stop 2024-04-16T05:39:12.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd623bbd740 1 -- 172.21.15.86:0/502178319 shutdown_connections 2024-04-16T05:39:12.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd623bbd740 1 --2- 172.21.15.86:0/502178319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31543360 0x562b3155ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd623bbd740 1 --2- 172.21.15.86:0/502178319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b31540a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd623bbd740 1 --2- 172.21.15.86:0/502178319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31539e60 0x562b3154f010 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd623bbd740 1 -- 172.21.15.86:0/502178319 >> 172.21.15.86:0/502178319 conn(0x562b31534590 msgr2=0x562b31540340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd623bbd740 1 -- 172.21.15.86:0/502178319 shutdown_connections 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.328+0000 7fd623bbd740 1 -- 172.21.15.86:0/502178319 wait complete. 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd623bbd740 1 -- start start 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd623bbd740 1 Processor -- start 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 0x562b315470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b315475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd623bbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31547b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd623bbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b2b50 con 0x562b31543360 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd623bbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b2480 con 0x562b3153a780 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd623bbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1af6a0 con 0x562b31539e60 2024-04-16T05:39:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd623bbd740 10 client.0 _ll_register_callbacks cb 0x7fd60c0053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31547b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31547b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58448/0 (socket says 172.21.15.86:58448) 2024-04-16T05:39:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61ad9d640 1 -- 172.21.15.86:0/3756654494 learned_addr learned my addr 172.21.15.86:0/3756654494 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61a59c640 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 0x562b315470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61ad9d640 1 -- 172.21.15.86:0/3756654494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 msgr2=0x562b315470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61ad9d640 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 0x562b315470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61ad9d640 1 -- 172.21.15.86:0/3756654494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 msgr2=0x562b315475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd611d9b640 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b315475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61ad9d640 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b315475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61ad9d640 1 -- 172.21.15.86:0/3756654494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b315b8660 con 0x562b31543360 2024-04-16T05:39:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd61ad9d640 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31547b20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ca0 tx=0x7fd6140095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd13f9d6e0a312e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.329+0000 7fd601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd601ffb640 1 -- 172.21.15.86:0/3756654494 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614052020 con 0x562b31543360 2024-04-16T05:39:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd601ffb640 1 -- 172.21.15.86:0/3756654494 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd614003d80 con 0x562b31543360 2024-04-16T05:39:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd623bbd740 1 -- 172.21.15.86:0/3756654494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60c09c530 con 0x562b31543360 2024-04-16T05:39:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd601ffb640 1 -- 172.21.15.86:0/3756654494 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd61405d050 con 0x562b31543360 2024-04-16T05:39:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd623bbd740 10 client.16920 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd623bbd740 1 -- 172.21.15.86:0/3756654494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b315b9970 con 0x562b31543360 2024-04-16T05:39:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd601ffb640 1 -- 172.21.15.86:0/3756654494 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60c09c530 con 0x562b31543360 2024-04-16T05:39:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd601ffb640 1 -- 172.21.15.86:0/3756654494 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6140347d0 con 0x562b31543360 2024-04-16T05:39:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd623bbd740 10 client.16920 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd623bbd740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd623bbd740 20 client.16920 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562b315ba1a0 0x562b315bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd623bbd740 4 client.16920 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.330+0000 7fd623bbd740 1 -- 172.21.15.86:0/3756654494 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b314a6970 con 0x562b315ba1a0 2024-04-16T05:39:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd61a59c640 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562b315ba1a0 0x562b315bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd601ffb640 10 client.16920 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd61a59c640 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562b315ba1a0 0x562b315bc5c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0f00e5ffcd587c server_cookie=3b4287942c9cee83 in_seq=0 out_seq=0 2024-04-16T05:39:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd601ffb640 1 -- 172.21.15.86:0/3756654494 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b314a6970 con 0x562b315ba1a0 2024-04-16T05:39:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd601ffb640 10 client.16920 handle_command_reply: tid=0 2024-04-16T05:39:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 client.16920 shutdown 2024-04-16T05:39:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 -- 172.21.15.86:0/3756654494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 msgr2=0x562b31547b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31547b20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ca0 tx=0x7fd6140095c0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 -- 172.21.15.86:0/3756654494 shutdown_connections 2024-04-16T05:39:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562b315ba1a0 0x562b315bc5c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b31543360 0x562b31547b20 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b3153a780 0x562b315475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 --2- 172.21.15.86:0/3756654494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b31539e60 0x562b315470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd601ffb640 0 client.16920 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 -- 172.21.15.86:0/3756654494 >> 172.21.15.86:0/3756654494 conn(0x562b31534590 msgr2=0x562b31540bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 -- 172.21.15.86:0/3756654494 shutdown_connections 2024-04-16T05:39:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.331+0000 7fd623bbd740 1 -- 172.21.15.86:0/3756654494 wait complete. 2024-04-16T05:39:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd623bbd740 20 client.16920 trim_cache size 0 max 16384 2024-04-16T05:39:12.341 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:12.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 -- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6080a3d20 msgr2=0x7fd6080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6080a3d20 0x7fd6080a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd614006010 tx=0x7fd61402ce40 comp rx=0 tx=0).stop 2024-04-16T05:39:12.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 -- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c102250 msgr2=0x7fd61c074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c102250 0x7fd61c074300 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c890 tx=0x7fd60c00cd50 comp rx=0 tx=0).stop 2024-04-16T05:39:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 -- 172.21.15.86:0/2805775820 shutdown_connections 2024-04-16T05:39:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6080a3d20 0x7fd6080a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd61c074d80 0x7fd61c1a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c107320 0x7fd61c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 --2- 172.21.15.86:0/2805775820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd61c102250 0x7fd61c074300 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 -- 172.21.15.86:0/2805775820 >> 172.21.15.86:0/2805775820 conn(0x7fd61c079490 msgr2=0x7fd61c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 -- 172.21.15.86:0/2805775820 shutdown_connections 2024-04-16T05:39:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:12.332+0000 7fd620efb640 1 -- 172.21.15.86:0/2805775820 wait complete. 2024-04-16T05:39:12.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:12 smithi194 ceph-mon[35865]: pgmap v120: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:12.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:12 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/404783007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:12 smithi196 ceph-mon[36373]: pgmap v120: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:12 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/404783007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:13.345 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:39:13.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:39:13.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:39:13.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.468+0000 7f38609bd640 1 -- 172.21.15.86:0/3270078539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c101af0 msgr2=0x7f385c105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.468+0000 7f38609bd640 1 --2- 172.21.15.86:0/3270078539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c101af0 0x7f385c105b30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f384c009900 tx=0x7f384c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:13.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.468+0000 7f38609bd640 1 -- 172.21.15.86:0/3270078539 shutdown_connections 2024-04-16T05:39:13.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.468+0000 7f38609bd640 1 --2- 172.21.15.86:0/3270078539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f385c106210 0x7f385c10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.468+0000 7f38609bd640 1 --2- 172.21.15.86:0/3270078539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c101af0 0x7f385c105b30 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.468+0000 7f38609bd640 1 --2- 172.21.15.86:0/3270078539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f385c101140 0x7f385c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.468+0000 7f38609bd640 1 -- 172.21.15.86:0/3270078539 >> 172.21.15.86:0/3270078539 conn(0x7f385c078000 msgr2=0x7f385c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.468+0000 7f38609bd640 1 -- 172.21.15.86:0/3270078539 shutdown_connections 2024-04-16T05:39:13.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.468+0000 7f38609bd640 1 -- 172.21.15.86:0/3270078539 wait complete. 2024-04-16T05:39:13.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f38609bd640 1 Processor -- start 2024-04-16T05:39:13.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f38609bd640 1 -- start start 2024-04-16T05:39:13.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f38609bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f385c101140 0x7f385c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f38609bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c106210 0x7f385c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f38609bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f385c19a2d0 0x7f385c1a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f38609bd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f385c111380 con 0x7f385c106210 2024-04-16T05:39:13.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f38609bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f385c111200 con 0x7f385c19a2d0 2024-04-16T05:39:13.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f38609bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f385c111500 con 0x7f385c101140 2024-04-16T05:39:13.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c106210 0x7f385c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c106210 0x7f385c199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58456/0 (socket says 172.21.15.86:58456) 2024-04-16T05:39:13.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f3859d9b640 1 -- 172.21.15.86:0/4288298648 learned_addr learned my addr 172.21.15.86:0/4288298648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f385a59c640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f385c101140 0x7f385c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f385ad9d640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f385c19a2d0 0x7f385c1a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f3859d9b640 1 -- 172.21.15.86:0/4288298648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f385c101140 msgr2=0x7f385c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f3859d9b640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f385c101140 0x7f385c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f3859d9b640 1 -- 172.21.15.86:0/4288298648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f385c19a2d0 msgr2=0x7f385c1a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f3859d9b640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f385c19a2d0 0x7f385c1a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f3859d9b640 1 -- 172.21.15.86:0/4288298648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c1a5050 con 0x7f385c106210 2024-04-16T05:39:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f385a59c640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f385c101140 0x7f385c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.469+0000 7f385ad9d640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f385c19a2d0 0x7f385c1a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.470+0000 7f3859d9b640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c106210 0x7f385c199d40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f384c000c00 tx=0x7f384c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.470+0000 7f384b7fe640 1 -- 172.21.15.86:0/4288298648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c041070 con 0x7f385c106210 2024-04-16T05:39:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.470+0000 7f384b7fe640 1 -- 172.21.15.86:0/4288298648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c004450 con 0x7f385c106210 2024-04-16T05:39:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.470+0000 7f38609bd640 1 -- 172.21.15.86:0/4288298648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c1a52e0 con 0x7f385c106210 2024-04-16T05:39:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.470+0000 7f384b7fe640 1 -- 172.21.15.86:0/4288298648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c034440 con 0x7f385c106210 2024-04-16T05:39:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.470+0000 7f38609bd640 1 -- 172.21.15.86:0/4288298648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c1a57d0 con 0x7f385c106210 2024-04-16T05:39:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.472+0000 7f384b7fe640 1 -- 172.21.15.86:0/4288298648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f384c044070 con 0x7f385c106210 2024-04-16T05:39:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.472+0000 7f38635aa740 1 Processor -- start 2024-04-16T05:39:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f38635aa740 1 -- start start 2024-04-16T05:39:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f38635aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6d4ab0 0x55b22e6d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f38635aa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f38635aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f38635aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b22e639e90 con 0x55b22e6ddfb0 2024-04-16T05:39:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f38635aa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b22e5d74c0 con 0x55b22e6d53d0 2024-04-16T05:39:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f38635aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b22e635e50 con 0x55b22e6d4ab0 2024-04-16T05:39:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f384b7fe640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38440a3d20 0x7f38440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58460/0 (socket says 172.21.15.86:58460) 2024-04-16T05:39:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f3859d9b640 1 -- 172.21.15.86:0/288507873 learned_addr learned my addr 172.21.15.86:0/288507873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f385a59c640 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f385ad9d640 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6d4ab0 0x55b22e6d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f385ad9d640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38440a3d20 0x7f38440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f384b7fe640 1 -- 172.21.15.86:0/4288298648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f384c0f28b0 con 0x7f385c106210 2024-04-16T05:39:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f3859d9b640 1 -- 172.21.15.86:0/288507873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6d4ab0 msgr2=0x55b22e6d4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f3859d9b640 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6d4ab0 0x55b22e6d4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f3859d9b640 1 -- 172.21.15.86:0/288507873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 msgr2=0x55b22e6dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f3859d9b640 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f3859d9b640 1 -- 172.21.15.86:0/288507873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22e4147e0 con 0x55b22e6ddfb0 2024-04-16T05:39:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.473+0000 7f385a59c640 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:39:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.474+0000 7f385ad9d640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38440a3d20 0x7f38440a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3854009440 tx=0x7f38540093d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.474+0000 7f3859d9b640 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e03a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f384c0efd10 tx=0x7f384c0ba9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e332209c5330376 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.474+0000 7f3848ff9640 1 -- 172.21.15.86:0/288507873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c04b030 con 0x55b22e6ddfb0 2024-04-16T05:39:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.474+0000 7f3848ff9640 1 -- 172.21.15.86:0/288507873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c0becf0 con 0x55b22e6ddfb0 2024-04-16T05:39:13.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.474+0000 7f3848ff9640 1 -- 172.21.15.86:0/288507873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c0eff10 con 0x55b22e6ddfb0 2024-04-16T05:39:13.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 -- 172.21.15.86:0/288507873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 msgr2=0x55b22e6e03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e03a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f384c0efd10 tx=0x7f384c0ba9c0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 -- 172.21.15.86:0/288507873 shutdown_connections 2024-04-16T05:39:13.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e03a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 --2- 172.21.15.86:0/288507873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6d4ab0 0x55b22e6d4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 -- 172.21.15.86:0/288507873 >> 172.21.15.86:0/288507873 conn(0x55b22e6cf1e0 msgr2=0x55b22e6cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 -- 172.21.15.86:0/288507873 shutdown_connections 2024-04-16T05:39:13.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 -- 172.21.15.86:0/288507873 wait complete. 2024-04-16T05:39:13.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 -- start start 2024-04-16T05:39:13.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 Processor -- start 2024-04-16T05:39:13.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d4ab0 0x55b22e6db150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d53d0 0x55b22e6db690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 0x55b22e6dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b22e639e90 con 0x55b22e6d53d0 2024-04-16T05:39:13.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b22e5c2410 con 0x55b22e6d4ab0 2024-04-16T05:39:13.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b22e6e0b60 con 0x55b22e6ddfb0 2024-04-16T05:39:13.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.475+0000 7f38635aa740 10 client.0 _ll_register_callbacks cb 0x55b22e5bdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:13.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d53d0 0x55b22e6db690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d53d0 0x55b22e6db690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58466/0 (socket says 172.21.15.86:58466) 2024-04-16T05:39:13.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3859d9b640 1 -- 172.21.15.86:0/3293210911 learned_addr learned my addr 172.21.15.86:0/3293210911 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f385ad9d640 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 0x55b22e6dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f385a59c640 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d4ab0 0x55b22e6db150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3859d9b640 1 -- 172.21.15.86:0/3293210911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 msgr2=0x55b22e6dbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3859d9b640 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 0x55b22e6dbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3859d9b640 1 -- 172.21.15.86:0/3293210911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d4ab0 msgr2=0x55b22e6db150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3859d9b640 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d4ab0 0x55b22e6db150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3859d9b640 1 -- 172.21.15.86:0/3293210911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22e474f70 con 0x55b22e6d53d0 2024-04-16T05:39:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f385a59c640 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d4ab0 0x55b22e6db150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3859d9b640 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d53d0 0x55b22e6db690 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f384c0bafc0 tx=0x7f384c0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539720a27f63f2d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3841ffb640 1 -- 172.21.15.86:0/3293210911 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c04b070 con 0x55b22e6d53d0 2024-04-16T05:39:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3841ffb640 1 -- 172.21.15.86:0/3293210911 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f384c0bea50 con 0x55b22e6d53d0 2024-04-16T05:39:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f38635aa740 1 -- 172.21.15.86:0/3293210911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f384c0becf0 con 0x55b22e6d53d0 2024-04-16T05:39:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3841ffb640 1 -- 172.21.15.86:0/3293210911 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f384c003800 con 0x55b22e6d53d0 2024-04-16T05:39:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f38635aa740 10 client.16932 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f38635aa740 1 -- 172.21.15.86:0/3293210911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b22e30c170 con 0x55b22e6d53d0 2024-04-16T05:39:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.476+0000 7f3841ffb640 1 -- 172.21.15.86:0/3293210911 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c0becf0 con 0x55b22e6d53d0 2024-04-16T05:39:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f3841ffb640 1 -- 172.21.15.86:0/3293210911 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f384c0f2ce0 con 0x55b22e6d53d0 2024-04-16T05:39:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f38635aa740 10 client.16932 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f38635aa740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f38635aa740 20 client.16932 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f38635aa740 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b22e6edf80 0x55b22e6f0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f38635aa740 4 client.16932 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f38635aa740 1 -- 172.21.15.86:0/3293210911 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b22e6478b0 con 0x55b22e6edf80 2024-04-16T05:39:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f385a59c640 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b22e6edf80 0x55b22e6f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f385a59c640 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b22e6edf80 0x55b22e6f0370 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e46b2f5ce04f400 server_cookie=5722ce8affc23528 in_seq=0 out_seq=0 2024-04-16T05:39:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.477+0000 7f3841ffb640 10 client.16932 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f3841ffb640 1 -- 172.21.15.86:0/3293210911 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b22e6478b0 con 0x55b22e6edf80 2024-04-16T05:39:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f3841ffb640 10 client.16932 handle_command_reply: tid=0 2024-04-16T05:39:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 client.16932 shutdown 2024-04-16T05:39:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 -- 172.21.15.86:0/3293210911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d53d0 msgr2=0x55b22e6db690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d53d0 0x55b22e6db690 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f384c0bafc0 tx=0x7f384c0057e0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 -- 172.21.15.86:0/3293210911 shutdown_connections 2024-04-16T05:39:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b22e6edf80 0x55b22e6f0370 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 0x55b22e6dbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d53d0 0x55b22e6db690 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f3841ffb640 0 client.16932 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 --2- 172.21.15.86:0/3293210911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d4ab0 0x55b22e6db150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 -- 172.21.15.86:0/3293210911 >> 172.21.15.86:0/3293210911 conn(0x55b22e6cf1e0 msgr2=0x55b22e6d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 -- 172.21.15.86:0/3293210911 shutdown_connections 2024-04-16T05:39:13.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.478+0000 7f38635aa740 1 -- 172.21.15.86:0/3293210911 wait complete. 2024-04-16T05:39:13.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.479+0000 7f38635aa740 20 client.16932 trim_cache size 0 max 16384 2024-04-16T05:39:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f38635aa740 1 Processor -- start 2024-04-16T05:39:13.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f38635aa740 1 -- start start 2024-04-16T05:39:13.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f38635aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d4ab0 0x55b22e6e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f38635aa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f38635aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 0x55b22e6f6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f38635aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f385c1af450 con 0x55b22e6d4ab0 2024-04-16T05:39:13.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f38635aa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f385c1b0f90 con 0x55b22e6d53d0 2024-04-16T05:39:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f38635aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f385c1afb20 con 0x55b22e6ddfb0 2024-04-16T05:39:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f385a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d4ab0 0x55b22e6e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f385a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d4ab0 0x55b22e6e98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58474/0 (socket says 172.21.15.86:58474) 2024-04-16T05:39:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f385a59c640 1 -- 172.21.15.86:0/323615809 learned_addr learned my addr 172.21.15.86:0/323615809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f385a59c640 1 -- 172.21.15.86:0/323615809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 msgr2=0x55b22e6f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f385ad9d640 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 0x55b22e6f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f3859d9b640 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f385a59c640 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 0x55b22e6f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f385a59c640 1 -- 172.21.15.86:0/323615809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 msgr2=0x55b22e6f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f385a59c640 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.496+0000 7f385a59c640 1 -- 172.21.15.86:0/323615809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22e6d51f0 con 0x55b22e6d4ab0 2024-04-16T05:39:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f385ad9d640 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 0x55b22e6f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f3859d9b640 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f385a59c640 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d4ab0 0x55b22e6e98a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3850007c40 tx=0x7f3850007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e96a9fdb08be784 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f3843fff640 1 -- 172.21.15.86:0/323615809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850015900 con 0x55b22e6d4ab0 2024-04-16T05:39:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f3843fff640 1 -- 172.21.15.86:0/323615809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3850023430 con 0x55b22e6d4ab0 2024-04-16T05:39:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f3843fff640 1 -- 172.21.15.86:0/323615809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38500135e0 con 0x55b22e6d4ab0 2024-04-16T05:39:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f38635aa740 1 -- 172.21.15.86:0/323615809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d4ab0 msgr2=0x55b22e6e98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f38635aa740 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d4ab0 0x55b22e6e98a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3850007c40 tx=0x7f3850007d30 comp rx=0 tx=0).stop 2024-04-16T05:39:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f38635aa740 1 -- 172.21.15.86:0/323615809 shutdown_connections 2024-04-16T05:39:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f38635aa740 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6ddfb0 0x55b22e6f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f38635aa740 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f38635aa740 1 --2- 172.21.15.86:0/323615809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6d4ab0 0x55b22e6e98a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f38635aa740 1 -- 172.21.15.86:0/323615809 >> 172.21.15.86:0/323615809 conn(0x55b22e6cf1e0 msgr2=0x55b22e6d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f38635aa740 1 -- 172.21.15.86:0/323615809 shutdown_connections 2024-04-16T05:39:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.497+0000 7f38635aa740 1 -- 172.21.15.86:0/323615809 wait complete. 2024-04-16T05:39:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f38635aa740 1 -- start start 2024-04-16T05:39:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f38635aa740 1 Processor -- start 2024-04-16T05:39:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f38635aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6d4ab0 0x55b22e6e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f38635aa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f38635aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f38635aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f385c1b08c0 con 0x55b22e6ddfb0 2024-04-16T05:39:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f38635aa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f385c1b01f0 con 0x55b22e6d53d0 2024-04-16T05:39:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f38635aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f385c1ad410 con 0x55b22e6d4ab0 2024-04-16T05:39:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f38635aa740 10 client.0 _ll_register_callbacks cb 0x7f38500071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f385ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f385ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58482/0 (socket says 172.21.15.86:58482) 2024-04-16T05:39:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f385ad9d640 1 -- 172.21.15.86:0/2299521450 learned_addr learned my addr 172.21.15.86:0/2299521450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f3859d9b640 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f385ad9d640 1 -- 172.21.15.86:0/2299521450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6d4ab0 msgr2=0x55b22e6e5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f385ad9d640 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6d4ab0 0x55b22e6e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f385ad9d640 1 -- 172.21.15.86:0/2299521450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 msgr2=0x55b22e6e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f385ad9d640 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6e2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.498+0000 7f385ad9d640 1 -- 172.21.15.86:0/2299521450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22e6e2ca0 con 0x55b22e6ddfb0 2024-04-16T05:39:13.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f385ad9d640 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e2760 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3854009e10 tx=0x7f3854004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becb1cc116716da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f3841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:13.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f3841ffb640 1 -- 172.21.15.86:0/2299521450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385402b060 con 0x55b22e6ddfb0 2024-04-16T05:39:13.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f3841ffb640 1 -- 172.21.15.86:0/2299521450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3854019d80 con 0x55b22e6ddfb0 2024-04-16T05:39:13.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f38635aa740 1 -- 172.21.15.86:0/2299521450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3850023430 con 0x55b22e6ddfb0 2024-04-16T05:39:13.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f3841ffb640 1 -- 172.21.15.86:0/2299521450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3854032050 con 0x55b22e6ddfb0 2024-04-16T05:39:13.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f38635aa740 10 client.16938 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:13.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f38635aa740 1 -- 172.21.15.86:0/2299521450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b22e755040 con 0x55b22e6ddfb0 2024-04-16T05:39:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f3841ffb640 1 -- 172.21.15.86:0/2299521450 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3850023430 con 0x55b22e6ddfb0 2024-04-16T05:39:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f3841ffb640 1 -- 172.21.15.86:0/2299521450 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38540254e0 con 0x55b22e6ddfb0 2024-04-16T05:39:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f38635aa740 10 client.16938 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f38635aa740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f38635aa740 20 client.16938 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f38635aa740 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b22e6edf80 0x55b22e7578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f38635aa740 4 client.16938 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:39:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.499+0000 7f38635aa740 1 -- 172.21.15.86:0/2299521450 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b22e641720 con 0x55b22e6edf80 2024-04-16T05:39:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.500+0000 7f385a59c640 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b22e6edf80 0x55b22e7578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.500+0000 7f385a59c640 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b22e6edf80 0x55b22e7578e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df09644c4711f4f server_cookie=c7e9f5d68271310e in_seq=0 out_seq=0 2024-04-16T05:39:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.500+0000 7f3841ffb640 10 client.16938 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.501+0000 7f3841ffb640 1 -- 172.21.15.86:0/2299521450 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b22e641720 con 0x55b22e6edf80 2024-04-16T05:39:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.501+0000 7f3841ffb640 10 client.16938 handle_command_reply: tid=0 2024-04-16T05:39:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.501+0000 7f38635aa740 1 client.16938 shutdown 2024-04-16T05:39:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 -- 172.21.15.86:0/2299521450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 msgr2=0x55b22e6e2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e2760 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3854009e10 tx=0x7f3854004a40 comp rx=0 tx=0).stop 2024-04-16T05:39:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 -- 172.21.15.86:0/2299521450 shutdown_connections 2024-04-16T05:39:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55b22e6edf80 0x55b22e7578e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b22e6ddfb0 0x55b22e6e2760 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f3841ffb640 0 client.16938 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b22e6d53d0 0x55b22e6e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 --2- 172.21.15.86:0/2299521450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b22e6d4ab0 0x55b22e6e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 -- 172.21.15.86:0/2299521450 >> 172.21.15.86:0/2299521450 conn(0x55b22e6cf1e0 msgr2=0x55b22e6dc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 -- 172.21.15.86:0/2299521450 shutdown_connections 2024-04-16T05:39:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 1 -- 172.21.15.86:0/2299521450 wait complete. 2024-04-16T05:39:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38635aa740 20 client.16938 trim_cache size 0 max 16384 2024-04-16T05:39:13.512 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:39:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:39:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "dcb7ba51-9841-4f8a-b570-f7c4301b4ed1", 2024-04-16T05:39:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:39:13.513 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:39:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 -- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38440a3d20 msgr2=0x7f38440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38440a3d20 0x7f38440a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3854009440 tx=0x7f38540093d0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 -- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c106210 msgr2=0x7f385c199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c106210 0x7f385c199d40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f384c000c00 tx=0x7f384c02cc70 comp rx=0 tx=0).stop 2024-04-16T05:39:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 -- 172.21.15.86:0/4288298648 shutdown_connections 2024-04-16T05:39:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38440a3d20 0x7f38440a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f385c19a2d0 0x7f385c1a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385c106210 0x7f385c199d40 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 --2- 172.21.15.86:0/4288298648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f385c101140 0x7f385c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 -- 172.21.15.86:0/4288298648 >> 172.21.15.86:0/4288298648 conn(0x7f385c078000 msgr2=0x7f385c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 -- 172.21.15.86:0/4288298648 shutdown_connections 2024-04-16T05:39:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.502+0000 7f38609bd640 1 -- 172.21.15.86:0/4288298648 wait complete. 2024-04-16T05:39:13.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:39:13.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:39:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.624+0000 7f37f1893640 1 -- 172.21.15.86:0/2674978299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec107320 msgr2=0x7f37ec10eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.624+0000 7f37f1893640 1 --2- 172.21.15.86:0/2674978299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec107320 0x7f37ec10eef0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009ca0 tx=0x7f37e402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.624+0000 7f37f1893640 1 -- 172.21.15.86:0/2674978299 shutdown_connections 2024-04-16T05:39:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.624+0000 7f37f1893640 1 --2- 172.21.15.86:0/2674978299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec107320 0x7f37ec10eef0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.624+0000 7f37f1893640 1 --2- 172.21.15.86:0/2674978299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ec06b2a0 0x7f37ec106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.624+0000 7f37f1893640 1 --2- 172.21.15.86:0/2674978299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37ec06a8f0 0x7f37ec06acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.624+0000 7f37f1893640 1 -- 172.21.15.86:0/2674978299 >> 172.21.15.86:0/2674978299 conn(0x7f37ec077f60 msgr2=0x7f37ec078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.624+0000 7f37f1893640 1 -- 172.21.15.86:0/2674978299 shutdown_connections 2024-04-16T05:39:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.624+0000 7f37f1893640 1 -- 172.21.15.86:0/2674978299 wait complete. 2024-04-16T05:39:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37f1893640 1 Processor -- start 2024-04-16T05:39:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37f1893640 1 -- start start 2024-04-16T05:39:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37f1893640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec06a8f0 0x7f37ec100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37f1893640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ec107320 0x7f37ec101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37f1893640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37ec1019b0 0x7f37ec1a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37f1893640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1123b0 con 0x7f37ec06a8f0 2024-04-16T05:39:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37f1893640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec112230 con 0x7f37ec1019b0 2024-04-16T05:39:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37f1893640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec112530 con 0x7f37ec107320 2024-04-16T05:39:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec06a8f0 0x7f37ec100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec06a8f0 0x7f37ec100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58496/0 (socket says 172.21.15.86:58496) 2024-04-16T05:39:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37eaffd640 1 -- 172.21.15.86:0/410984728 learned_addr learned my addr 172.21.15.86:0/410984728 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37ec1019b0 0x7f37ec1a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37e27fc640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ec107320 0x7f37ec101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.625+0000 7f37eaffd640 1 -- 172.21.15.86:0/410984728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ec107320 msgr2=0x7f37ec101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37eaffd640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ec107320 0x7f37ec101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37eaffd640 1 -- 172.21.15.86:0/410984728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37ec1019b0 msgr2=0x7f37ec1a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37eaffd640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37ec1019b0 0x7f37ec1a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37eaffd640 1 -- 172.21.15.86:0/410984728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec1a70c0 con 0x7f37ec06a8f0 2024-04-16T05:39:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37ec1019b0 0x7f37ec1a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37e27fc640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ec107320 0x7f37ec101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37eaffd640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec06a8f0 0x7f37ec100f30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d950 tx=0x7f37dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37f0891640 1 -- 172.21.15.86:0/410984728 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc017070 con 0x7f37ec06a8f0 2024-04-16T05:39:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37f0891640 1 -- 172.21.15.86:0/410984728 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc009c30 con 0x7f37ec06a8f0 2024-04-16T05:39:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37f1893640 1 -- 172.21.15.86:0/410984728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec1a73b0 con 0x7f37ec06a8f0 2024-04-16T05:39:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37f0891640 1 -- 172.21.15.86:0/410984728 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00f6b0 con 0x7f37ec06a8f0 2024-04-16T05:39:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.626+0000 7f37f1893640 1 -- 172.21.15.86:0/410984728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec1a7900 con 0x7f37ec06a8f0 2024-04-16T05:39:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.628+0000 7f37f0891640 1 -- 172.21.15.86:0/410984728 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f37dc0109d0 con 0x7f37ec06a8f0 2024-04-16T05:39:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.628+0000 7f37f4122740 1 Processor -- start 2024-04-16T05:39:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37f4122740 1 -- start start 2024-04-16T05:39:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37f4122740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026be50 0x55d42026c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37f4122740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 0x55d420274e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37f4122740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 0x55d420277740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37f4122740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d4201d1180 con 0x55d42026be50 2024-04-16T05:39:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37f4122740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d42016e360 con 0x55d42026c770 2024-04-16T05:39:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37f4122740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4201cd140 con 0x55d420275350 2024-04-16T05:39:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37f0891640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37d80a3d20 0x7f37d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026be50 0x55d42026c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026be50 0x55d42026c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58510/0 (socket says 172.21.15.86:58510) 2024-04-16T05:39:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 -- 172.21.15.86:0/923895371 learned_addr learned my addr 172.21.15.86:0/923895371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eaffd640 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 0x55d420274e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37d80a3d20 0x7f37d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37f0891640 1 -- 172.21.15.86:0/410984728 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37dc0d0870 con 0x7f37ec06a8f0 2024-04-16T05:39:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37e27fc640 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 0x55d420277740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 -- 172.21.15.86:0/923895371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 msgr2=0x55d420277740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 0x55d420277740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 -- 172.21.15.86:0/923895371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 msgr2=0x55d420274e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 0x55d420274e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 -- 172.21.15.86:0/923895371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41ffad7d0 con 0x55d42026be50 2024-04-16T05:39:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37e27fc640 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 0x55d420277740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.629+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026be50 0x55d42026c230 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009ca0 tx=0x7f37e4032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b3d04955913e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37d80a3d20 0x7f37d80a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f37e4036000 tx=0x7f37e4041000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37e1ffb640 1 -- 172.21.15.86:0/923895371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4051070 con 0x55d42026be50 2024-04-16T05:39:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37e1ffb640 1 -- 172.21.15.86:0/923895371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4048070 con 0x55d42026be50 2024-04-16T05:39:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37e1ffb640 1 -- 172.21.15.86:0/923895371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4039040 con 0x55d42026be50 2024-04-16T05:39:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37f4122740 1 -- 172.21.15.86:0/923895371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026be50 msgr2=0x55d42026c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37f4122740 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026be50 0x55d42026c230 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009ca0 tx=0x7f37e4032fc0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37f4122740 1 -- 172.21.15.86:0/923895371 shutdown_connections 2024-04-16T05:39:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37f4122740 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 0x55d420277740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37f4122740 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 0x55d420274e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37f4122740 1 --2- 172.21.15.86:0/923895371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026be50 0x55d42026c230 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37f4122740 1 -- 172.21.15.86:0/923895371 >> 172.21.15.86:0/923895371 conn(0x55d420266580 msgr2=0x55d420266960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37f4122740 1 -- 172.21.15.86:0/923895371 shutdown_connections 2024-04-16T05:39:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.630+0000 7f37f4122740 1 -- 172.21.15.86:0/923895371 wait complete. 2024-04-16T05:39:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37f4122740 1 -- start start 2024-04-16T05:39:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37f4122740 1 Processor -- start 2024-04-16T05:39:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37f4122740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42026be50 0x55d420272340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37f4122740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d420272880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37f4122740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d420275350 0x55d420272dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37f4122740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d4202780e0 con 0x55d42026c770 2024-04-16T05:39:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37f4122740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d42018a2c0 con 0x55d420275350 2024-04-16T05:39:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37f4122740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d420278260 con 0x55d42026be50 2024-04-16T05:39:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37f4122740 10 client.0 _ll_register_callbacks cb 0x55d420171090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37e27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d420272880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37e27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d420272880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58512/0 (socket says 172.21.15.86:58512) 2024-04-16T05:39:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37e27fc640 1 -- 172.21.15.86:0/2030065313 learned_addr learned my addr 172.21.15.86:0/2030065313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37eaffd640 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42026be50 0x55d420272340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.631+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d420275350 0x55d420272dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37e27fc640 1 -- 172.21.15.86:0/2030065313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42026be50 msgr2=0x55d420272340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37e27fc640 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42026be50 0x55d420272340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37e27fc640 1 -- 172.21.15.86:0/2030065313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d420275350 msgr2=0x55d420272dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37e27fc640 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d420275350 0x55d420272dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37e27fc640 1 -- 172.21.15.86:0/2030065313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d41fee33f0 con 0x55d42026c770 2024-04-16T05:39:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37e27fc640 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d420272880 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f37d400e6b0 tx=0x7f37d400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be372d2bb8136306 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37caffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37caffd640 1 -- 172.21.15.86:0/2030065313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d401c060 con 0x55d42026c770 2024-04-16T05:39:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37caffd640 1 -- 172.21.15.86:0/2030065313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37d4009a80 con 0x55d42026c770 2024-04-16T05:39:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37f4122740 1 -- 172.21.15.86:0/2030065313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e4048070 con 0x55d42026c770 2024-04-16T05:39:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37caffd640 1 -- 172.21.15.86:0/2030065313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37d4023050 con 0x55d42026c770 2024-04-16T05:39:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37f4122740 10 client.16950 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37f4122740 1 -- 172.21.15.86:0/2030065313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d41fea30f0 con 0x55d42026c770 2024-04-16T05:39:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37caffd640 1 -- 172.21.15.86:0/2030065313 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4048070 con 0x55d42026c770 2024-04-16T05:39:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37caffd640 1 -- 172.21.15.86:0/2030065313 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37d400f420 con 0x55d42026c770 2024-04-16T05:39:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37f4122740 10 client.16950 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37f4122740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37f4122740 20 client.16950 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37f4122740 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d4202856d0 0x55d420287ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37f4122740 4 client.16950 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.632+0000 7f37f4122740 1 -- 172.21.15.86:0/2030065313 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4201deba0 con 0x55d4202856d0 2024-04-16T05:39:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.633+0000 7f37eaffd640 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d4202856d0 0x55d420287ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.633+0000 7f37eaffd640 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d4202856d0 0x55d420287ac0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adaecc60a71f34bd server_cookie=d77b7e7a7b6eee2 in_seq=0 out_seq=0 2024-04-16T05:39:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.633+0000 7f37caffd640 10 client.16950 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37caffd640 1 -- 172.21.15.86:0/2030065313 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4201deba0 con 0x55d4202856d0 2024-04-16T05:39:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37caffd640 10 client.16950 handle_command_reply: tid=0 2024-04-16T05:39:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 client.16950 shutdown 2024-04-16T05:39:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 -- 172.21.15.86:0/2030065313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 msgr2=0x55d420272880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d420272880 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f37d400e6b0 tx=0x7f37d400eb70 comp rx=0 tx=0).stop 2024-04-16T05:39:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 -- 172.21.15.86:0/2030065313 shutdown_connections 2024-04-16T05:39:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d4202856d0 0x55d420287ac0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d420275350 0x55d420272dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d420272880 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 --2- 172.21.15.86:0/2030065313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42026be50 0x55d420272340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37caffd640 0 client.16950 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 -- 172.21.15.86:0/2030065313 >> 172.21.15.86:0/2030065313 conn(0x55d420266580 msgr2=0x55d4202682d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 -- 172.21.15.86:0/2030065313 shutdown_connections 2024-04-16T05:39:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 1 -- 172.21.15.86:0/2030065313 wait complete. 2024-04-16T05:39:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.634+0000 7f37f4122740 20 client.16950 trim_cache size 0 max 16384 2024-04-16T05:39:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.651+0000 7f37f4122740 1 Processor -- start 2024-04-16T05:39:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.651+0000 7f37f4122740 1 -- start start 2024-04-16T05:39:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37f4122740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42026be50 0x55d420274a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37f4122740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 0x55d420280fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37f4122740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d420275350 0x55d42028d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37f4122740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b1610 con 0x55d420275350 2024-04-16T05:39:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37f4122740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b3150 con 0x55d42026c770 2024-04-16T05:39:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37f4122740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b1ce0 con 0x55d42026be50 2024-04-16T05:39:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d420275350 0x55d42028d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d420275350 0x55d42028d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58522/0 (socket says 172.21.15.86:58522) 2024-04-16T05:39:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37eb7fe640 1 -- 172.21.15.86:0/800354375 learned_addr learned my addr 172.21.15.86:0/800354375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37e27fc640 1 --2- 172.21.15.86:0/800354375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 0x55d420280fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37eb7fe640 1 -- 172.21.15.86:0/800354375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42026be50 msgr2=0x55d420274a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/800354375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42026be50 0x55d420274a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37eb7fe640 1 -- 172.21.15.86:0/800354375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 msgr2=0x55d420280fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/800354375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 0x55d420280fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37eb7fe640 1 -- 172.21.15.86:0/800354375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d420268870 con 0x55d420275350 2024-04-16T05:39:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37e27fc640 1 --2- 172.21.15.86:0/800354375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 0x55d420280fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37eb7fe640 1 --2- 172.21.15.86:0/800354375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d420275350 0x55d42028d3d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009ca0 tx=0x7f37e4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93ae29f8d00e1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37e17fa640 1 -- 172.21.15.86:0/800354375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e405a070 con 0x55d420275350 2024-04-16T05:39:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37e17fa640 1 -- 172.21.15.86:0/800354375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4048050 con 0x55d420275350 2024-04-16T05:39:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.652+0000 7f37e17fa640 1 -- 172.21.15.86:0/800354375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4039070 con 0x55d420275350 2024-04-16T05:39:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 -- 172.21.15.86:0/800354375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d420275350 msgr2=0x55d42028d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 --2- 172.21.15.86:0/800354375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d420275350 0x55d42028d3d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009ca0 tx=0x7f37e4009540 comp rx=0 tx=0).stop 2024-04-16T05:39:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 -- 172.21.15.86:0/800354375 shutdown_connections 2024-04-16T05:39:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 --2- 172.21.15.86:0/800354375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d420275350 0x55d42028d3d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 --2- 172.21.15.86:0/800354375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026c770 0x55d420280fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 --2- 172.21.15.86:0/800354375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42026be50 0x55d420274a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 -- 172.21.15.86:0/800354375 >> 172.21.15.86:0/800354375 conn(0x55d420266580 msgr2=0x55d420265760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 -- 172.21.15.86:0/800354375 shutdown_connections 2024-04-16T05:39:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 -- 172.21.15.86:0/800354375 wait complete. 2024-04-16T05:39:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.653+0000 7f37f4122740 1 -- start start 2024-04-16T05:39:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37f4122740 1 Processor -- start 2024-04-16T05:39:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37f4122740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026be50 0x55d42027d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37f4122740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d42027b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37f4122740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 0x55d42027be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37f4122740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b2a80 con 0x55d42026c770 2024-04-16T05:39:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37f4122740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b23b0 con 0x55d42026be50 2024-04-16T05:39:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37f4122740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1af5d0 con 0x55d420275350 2024-04-16T05:39:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37f4122740 10 client.0 _ll_register_callbacks cb 0x7f37d400c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37e27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d42027b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37e27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d42027b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58524/0 (socket says 172.21.15.86:58524) 2024-04-16T05:39:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37e27fc640 1 -- 172.21.15.86:0/3995934473 learned_addr learned my addr 172.21.15.86:0/3995934473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37e27fc640 1 -- 172.21.15.86:0/3995934473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 msgr2=0x55d42027be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37e27fc640 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 0x55d42027be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37e27fc640 1 -- 172.21.15.86:0/3995934473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026be50 msgr2=0x55d42027d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37e27fc640 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026be50 0x55d42027d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37e27fc640 1 -- 172.21.15.86:0/3995934473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42027c3b0 con 0x55d42026c770 2024-04-16T05:39:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37e27fc640 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d42027b930 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f37d4000c00 tx=0x7f37d40097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505c6f15879175be server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.654+0000 7f37caffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37caffd640 1 -- 172.21.15.86:0/3995934473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d402b070 con 0x55d42026c770 2024-04-16T05:39:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37caffd640 1 -- 172.21.15.86:0/3995934473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37d4005040 con 0x55d42026c770 2024-04-16T05:39:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37f4122740 1 -- 172.21.15.86:0/3995934473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e4048050 con 0x55d42026c770 2024-04-16T05:39:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37caffd640 1 -- 172.21.15.86:0/3995934473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37d401c020 con 0x55d42026c770 2024-04-16T05:39:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37f4122740 10 client.16956 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37f4122740 1 -- 172.21.15.86:0/3995934473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d4202ec8e0 con 0x55d42026c770 2024-04-16T05:39:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37caffd640 1 -- 172.21.15.86:0/3995934473 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4048050 con 0x55d42026c770 2024-04-16T05:39:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37caffd640 1 -- 172.21.15.86:0/3995934473 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37d4009c70 con 0x55d42026c770 2024-04-16T05:39:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37f4122740 10 client.16956 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37f4122740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37f4122740 20 client.16956 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37f4122740 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d4202ed090 0x55d4202ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37f4122740 4 client.16956 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:39:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.655+0000 7f37f4122740 1 -- 172.21.15.86:0/3995934473 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4201d8a10 con 0x55d4202ed090 2024-04-16T05:39:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37eaffd640 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d4202ed090 0x55d4202ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37eaffd640 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d4202ed090 0x55d4202ef4b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79929aeaa9a8cde1 server_cookie=1b0fe0edf2b12c35 in_seq=0 out_seq=0 2024-04-16T05:39:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37caffd640 10 client.16956 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37caffd640 1 -- 172.21.15.86:0/3995934473 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d4201d8a10 con 0x55d4202ed090 2024-04-16T05:39:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37caffd640 10 client.16956 handle_command_reply: tid=0 2024-04-16T05:39:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 client.16956 shutdown 2024-04-16T05:39:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 -- 172.21.15.86:0/3995934473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 msgr2=0x55d42027b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d42027b930 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f37d4000c00 tx=0x7f37d40097b0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 -- 172.21.15.86:0/3995934473 shutdown_connections 2024-04-16T05:39:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d4202ed090 0x55d4202ef4b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d420275350 0x55d42027be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d42026c770 0x55d42027b930 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37caffd640 0 client.16956 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 --2- 172.21.15.86:0/3995934473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d42026be50 0x55d42027d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 -- 172.21.15.86:0/3995934473 >> 172.21.15.86:0/3995934473 conn(0x55d420266580 msgr2=0x55d420264ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 -- 172.21.15.86:0/3995934473 shutdown_connections 2024-04-16T05:39:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.656+0000 7f37f4122740 1 -- 172.21.15.86:0/3995934473 wait complete. 2024-04-16T05:39:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f4122740 20 client.16956 trim_cache size 0 max 16384 2024-04-16T05:39:13.667 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:39:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:39:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:39:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "dcb7ba51-9841-4f8a-b570-f7c4301b4ed1": { 2024-04-16T05:39:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:39:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "dcb7ba51-9841-4f8a-b570-f7c4301b4ed1", 2024-04-16T05:39:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:39:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:39:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:39:13.668 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 -- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37d80a3d20 msgr2=0x7f37d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37d80a3d20 0x7f37d80a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f37e4036000 tx=0x7f37e4041000 comp rx=0 tx=0).stop 2024-04-16T05:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 -- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec06a8f0 msgr2=0x7f37ec100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec06a8f0 0x7f37ec100f30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d950 tx=0x7f37dc00de10 comp rx=0 tx=0).stop 2024-04-16T05:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 -- 172.21.15.86:0/410984728 shutdown_connections 2024-04-16T05:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37d80a3d20 0x7f37d80a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37ec1019b0 0x7f37ec1a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ec107320 0x7f37ec101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 --2- 172.21.15.86:0/410984728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec06a8f0 0x7f37ec100f30 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 -- 172.21.15.86:0/410984728 >> 172.21.15.86:0/410984728 conn(0x7f37ec077f60 msgr2=0x7f37ec0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 -- 172.21.15.86:0/410984728 shutdown_connections 2024-04-16T05:39:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:13.657+0000 7f37f1893640 1 -- 172.21.15.86:0/410984728 wait complete. 2024-04-16T05:39:13.671 INFO:tasks.cephfs.filesystem:scrub status for tag:dcb7ba51-9841-4f8a-b570-f7c4301b4ed1 - {'path': '/', 'tag': 'dcb7ba51-9841-4f8a-b570-f7c4301b4ed1', 'options': 'recursive,force'} 2024-04-16T05:39:14.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:14 smithi194 ceph-mon[35865]: pgmap v121: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:39:14.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:14 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:39:14.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:14 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:14.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:14 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:39:14.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:14 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:14.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:14 smithi086 ceph-mon[32028]: pgmap v121: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:39:14.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:14 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:39:14.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:14 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:14.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:14 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:39:14.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:14 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:14.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:14 smithi196 ceph-mon[36373]: pgmap v121: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:39:14.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:14 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:39:14.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:14 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:14.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:14 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:39:14.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:14 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:15.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:15 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:39:15.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:15 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:39:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:15 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:39:16.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:16 smithi194 ceph-mon[35865]: pgmap v122: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:39:16.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:16 smithi086 ceph-mon[32028]: pgmap v122: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:39:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:16 smithi196 ceph-mon[36373]: pgmap v122: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:39:18.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:18 smithi194 ceph-mon[35865]: pgmap v123: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:39:18.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:18 smithi086 ceph-mon[32028]: pgmap v123: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:39:18.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:18 smithi196 ceph-mon[36373]: pgmap v123: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:39:20.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:20 smithi194 ceph-mon[35865]: pgmap v124: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-16T05:39:20.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:20 smithi086 ceph-mon[32028]: pgmap v124: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-16T05:39:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:20 smithi196 ceph-mon[36373]: pgmap v124: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-16T05:39:22.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:22 smithi194 ceph-mon[35865]: pgmap v125: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-16T05:39:22.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:22 smithi086 ceph-mon[32028]: pgmap v125: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-16T05:39:22.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:22 smithi196 ceph-mon[36373]: pgmap v125: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-16T05:39:24.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:24 smithi194 ceph-mon[35865]: pgmap v126: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-16T05:39:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:24 smithi086 ceph-mon[32028]: pgmap v126: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-16T05:39:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:24 smithi196 ceph-mon[36373]: pgmap v126: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-16T05:39:26.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:26 smithi194 ceph-mon[35865]: pgmap v127: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:26.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:26 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:39:26.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:26 smithi086 ceph-mon[32028]: pgmap v127: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:26.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:26 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:39:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:26 smithi196 ceph-mon[36373]: pgmap v127: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:26 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:39:27.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:27 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:39:27.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:27 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:39:27.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:27 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:39:27.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:27 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:39:27.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:27 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:39:27.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:27 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:39:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:27 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:39:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:27 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:39:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:27 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:39:28.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:28 smithi086 ceph-mon[32028]: pgmap v128: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:28 smithi196 ceph-mon[36373]: pgmap v128: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:28 smithi194 ceph-mon[35865]: pgmap v128: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:30 smithi086 ceph-mon[32028]: pgmap v129: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:39:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:30 smithi196 ceph-mon[36373]: pgmap v129: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:39:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:30 smithi194 ceph-mon[35865]: pgmap v129: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:39:32.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:32 smithi086 ceph-mon[32028]: pgmap v130: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:32.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:32 smithi196 ceph-mon[36373]: pgmap v130: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:32.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:32 smithi194 ceph-mon[35865]: pgmap v130: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:34.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:34 smithi086 ceph-mon[32028]: pgmap v131: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:34.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:34 smithi196 ceph-mon[36373]: pgmap v131: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:34.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:34 smithi194 ceph-mon[35865]: pgmap v131: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:36 smithi086 ceph-mon[32028]: pgmap v132: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:36 smithi196 ceph-mon[36373]: pgmap v132: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:36.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:36 smithi194 ceph-mon[35865]: pgmap v132: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:38.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:38 smithi196 ceph-mon[36373]: pgmap v133: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:38.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:38 smithi194 ceph-mon[35865]: pgmap v133: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:38.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:38 smithi086 ceph-mon[32028]: pgmap v133: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:40.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:40 smithi196 ceph-mon[36373]: pgmap v134: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:40.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:40 smithi194 ceph-mon[35865]: pgmap v134: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:40.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:40 smithi086 ceph-mon[32028]: pgmap v134: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:42.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:42 smithi196 ceph-mon[36373]: pgmap v135: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:42.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:42 smithi086 ceph-mon[32028]: pgmap v135: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:42.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:42 smithi194 ceph-mon[35865]: pgmap v135: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:43.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:39:43.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:39:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 -- 172.21.15.86:0/378273448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb244103bb0 msgr2=0x7fb24410c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 --2- 172.21.15.86:0/378273448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb244103bb0 0x7fb24410c090 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb238009900 tx=0x7fb23802bd60 comp rx=0 tx=0).stop 2024-04-16T05:39:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 -- 172.21.15.86:0/378273448 shutdown_connections 2024-04-16T05:39:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 --2- 172.21.15.86:0/378273448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb24410c5d0 0x7fb24410ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 --2- 172.21.15.86:0/378273448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb244103bb0 0x7fb24410c090 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 --2- 172.21.15.86:0/378273448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb244103290 0x7fb244103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 -- 172.21.15.86:0/378273448 >> 172.21.15.86:0/378273448 conn(0x7fb244077280 msgr2=0x7fb244077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:43.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 -- 172.21.15.86:0/378273448 shutdown_connections 2024-04-16T05:39:43.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 -- 172.21.15.86:0/378273448 wait complete. 2024-04-16T05:39:43.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 Processor -- start 2024-04-16T05:39:43.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.794+0000 7fb249d15640 1 -- start start 2024-04-16T05:39:43.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb249d15640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb244103290 0x7fb24419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb249d15640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb244103bb0 0x7fb24419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb249d15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24410c5d0 0x7fb2441986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb249d15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2441123e0 con 0x7fb24410c5d0 2024-04-16T05:39:43.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb249d15640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb244112260 con 0x7fb244103290 2024-04-16T05:39:43.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb249d15640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb244112560 con 0x7fb244103bb0 2024-04-16T05:39:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb243fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24410c5d0 0x7fb2441986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb243fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24410c5d0 0x7fb2441986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52584/0 (socket says 172.21.15.86:52584) 2024-04-16T05:39:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb243fff640 1 -- 172.21.15.86:0/43452654 learned_addr learned my addr 172.21.15.86:0/43452654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb2437fe640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb244103290 0x7fb24419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb242ffd640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb244103bb0 0x7fb24419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb243fff640 1 -- 172.21.15.86:0/43452654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb244103bb0 msgr2=0x7fb24419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb243fff640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb244103bb0 0x7fb24419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb243fff640 1 -- 172.21.15.86:0/43452654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb244103290 msgr2=0x7fb24419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb243fff640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb244103290 0x7fb24419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb243fff640 1 -- 172.21.15.86:0/43452654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb244198fa0 con 0x7fb24410c5d0 2024-04-16T05:39:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb243fff640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24410c5d0 0x7fb2441986e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00eaa0 tx=0x7fb23c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.795+0000 7fb240ff9640 1 -- 172.21.15.86:0/43452654 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00cdb0 con 0x7fb24410c5d0 2024-04-16T05:39:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.796+0000 7fb249d15640 1 -- 172.21.15.86:0/43452654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb244199290 con 0x7fb24410c5d0 2024-04-16T05:39:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.796+0000 7fb240ff9640 1 -- 172.21.15.86:0/43452654 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb23c005020 con 0x7fb24410c5d0 2024-04-16T05:39:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.796+0000 7fb240ff9640 1 -- 172.21.15.86:0/43452654 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c0116b0 con 0x7fb24410c5d0 2024-04-16T05:39:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.796+0000 7fb249d15640 1 -- 172.21.15.86:0/43452654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2441a55a0 con 0x7fb24410c5d0 2024-04-16T05:39:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.798+0000 7fb240ff9640 1 -- 172.21.15.86:0/43452654 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb23c005560 con 0x7fb24410c5d0 2024-04-16T05:39:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb24c9d9740 1 Processor -- start 2024-04-16T05:39:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb24c9d9740 1 -- start start 2024-04-16T05:39:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404abe50 0x55e6404ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 0x55e6404b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e640411180 con 0x55e6404ac770 2024-04-16T05:39:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e6403ae360 con 0x55e6404abe50 2024-04-16T05:39:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e64040d140 con 0x55e6404b5350 2024-04-16T05:39:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb2437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb2437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52596/0 (socket says 172.21.15.86:52596) 2024-04-16T05:39:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb2437fe640 1 -- 172.21.15.86:0/2068468144 learned_addr learned my addr 172.21.15.86:0/2068468144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb240ff9640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb22c0a3d20 0x7fb22c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb242ffd640 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404abe50 0x55e6404ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb242ffd640 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 0x55e6404b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb2437fe640 1 -- 172.21.15.86:0/2068468144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 msgr2=0x55e6404b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb2437fe640 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 0x55e6404b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb2437fe640 1 -- 172.21.15.86:0/2068468144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404abe50 msgr2=0x55e6404ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb2437fe640 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404abe50 0x55e6404ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb2437fe640 1 -- 172.21.15.86:0/2068468144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6401ed7d0 con 0x55e6404ac770 2024-04-16T05:39:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb242ffd640 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 0x55e6404b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb243fff640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb22c0a3d20 0x7fb22c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.799+0000 7fb240ff9640 1 -- 172.21.15.86:0/43452654 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb23c0d0d70 con 0x7fb24410c5d0 2024-04-16T05:39:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb2437fe640 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404b4e10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb23400ac20 tx=0x7fb23400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7355b5cdb2d0b619 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb229ffb640 1 -- 172.21.15.86:0/2068468144 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23401d070 con 0x55e6404ac770 2024-04-16T05:39:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb229ffb640 1 -- 172.21.15.86:0/2068468144 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb234018b20 con 0x55e6404ac770 2024-04-16T05:39:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb229ffb640 1 -- 172.21.15.86:0/2068468144 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234010c00 con 0x55e6404ac770 2024-04-16T05:39:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb243fff640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb22c0a3d20 0x7fb22c0a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb23c09c930 tx=0x7fb23c087040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb24c9d9740 1 -- 172.21.15.86:0/2068468144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 msgr2=0x55e6404b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404b4e10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb23400ac20 tx=0x7fb23400f790 comp rx=0 tx=0).stop 2024-04-16T05:39:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb24c9d9740 1 -- 172.21.15.86:0/2068468144 shutdown_connections 2024-04-16T05:39:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 0x55e6404b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404b4e10 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.800+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/2068468144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404abe50 0x55e6404ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 -- 172.21.15.86:0/2068468144 >> 172.21.15.86:0/2068468144 conn(0x55e6404a6580 msgr2=0x55e6404a6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 -- 172.21.15.86:0/2068468144 shutdown_connections 2024-04-16T05:39:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 -- 172.21.15.86:0/2068468144 wait complete. 2024-04-16T05:39:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 -- start start 2024-04-16T05:39:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 Processor -- start 2024-04-16T05:39:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404abe50 0x55e6404b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404ac770 0x55e6404b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404b5350 0x55e6404b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6404b8140 con 0x55e6404b5350 2024-04-16T05:39:43.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e6403ca2c0 con 0x55e6404ac770 2024-04-16T05:39:43.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e6404b82c0 con 0x55e6404abe50 2024-04-16T05:39:43.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.801+0000 7fb24c9d9740 10 client.0 _ll_register_callbacks cb 0x55e6403b1090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:43.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404b5350 0x55e6404b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404b5350 0x55e6404b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52610/0 (socket says 172.21.15.86:52610) 2024-04-16T05:39:43.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 -- 172.21.15.86:0/232025165 learned_addr learned my addr 172.21.15.86:0/232025165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:43.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404abe50 0x55e6404b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb2437fe640 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404ac770 0x55e6404b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 -- 172.21.15.86:0/232025165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404abe50 msgr2=0x55e6404b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404abe50 0x55e6404b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 -- 172.21.15.86:0/232025165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404ac770 msgr2=0x55e6404b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404ac770 0x55e6404b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 -- 172.21.15.86:0/232025165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6401233f0 con 0x55e6404b5350 2024-04-16T05:39:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb242ffd640 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404b5350 0x55e6404b2e00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb238005ab0 tx=0x7fb23802bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e98a8fdf6b100a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb222ffd640 1 -- 172.21.15.86:0/232025165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23804a060 con 0x55e6404b5350 2024-04-16T05:39:43.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb222ffd640 1 -- 172.21.15.86:0/232025165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb2380038c0 con 0x55e6404b5350 2024-04-16T05:39:43.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb24c9d9740 1 -- 172.21.15.86:0/232025165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb234018b20 con 0x55e6404b5350 2024-04-16T05:39:43.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb222ffd640 1 -- 172.21.15.86:0/232025165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb238003ac0 con 0x55e6404b5350 2024-04-16T05:39:43.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb24c9d9740 10 client.16968 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:43.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb24c9d9740 1 -- 172.21.15.86:0/232025165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e6400e30f0 con 0x55e6404b5350 2024-04-16T05:39:43.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.802+0000 7fb222ffd640 1 -- 172.21.15.86:0/232025165 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb234018b20 con 0x55e6404b5350 2024-04-16T05:39:43.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb222ffd640 1 -- 172.21.15.86:0/232025165 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb238043c90 con 0x55e6404b5350 2024-04-16T05:39:43.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb24c9d9740 10 client.16968 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:43.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb24c9d9740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:43.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb24c9d9740 20 client.16968 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:43.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e6404c5640 0x55e6404c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb24c9d9740 4 client.16968 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:43.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb24c9d9740 1 -- 172.21.15.86:0/232025165 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e64041eba0 con 0x55e6404c5640 2024-04-16T05:39:43.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb2437fe640 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e6404c5640 0x55e6404c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb222ffd640 10 client.16968 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:43.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.803+0000 7fb2437fe640 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e6404c5640 0x55e6404c7a30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59007c5f79ca33ff server_cookie=2a86261af04783b9 in_seq=0 out_seq=0 2024-04-16T05:39:43.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb222ffd640 1 -- 172.21.15.86:0/232025165 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e64041eba0 con 0x55e6404c5640 2024-04-16T05:39:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb222ffd640 10 client.16968 handle_command_reply: tid=0 2024-04-16T05:39:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 client.16968 shutdown 2024-04-16T05:39:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 -- 172.21.15.86:0/232025165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404b5350 msgr2=0x55e6404b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404b5350 0x55e6404b2e00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb238005ab0 tx=0x7fb23802bd60 comp rx=0 tx=0).stop 2024-04-16T05:39:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 -- 172.21.15.86:0/232025165 shutdown_connections 2024-04-16T05:39:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e6404c5640 0x55e6404c7a30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404b5350 0x55e6404b2e00 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb222ffd640 0 client.16968 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404ac770 0x55e6404b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/232025165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404abe50 0x55e6404b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 -- 172.21.15.86:0/232025165 >> 172.21.15.86:0/232025165 conn(0x55e6404a6580 msgr2=0x55e6404a82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 -- 172.21.15.86:0/232025165 shutdown_connections 2024-04-16T05:39:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.804+0000 7fb24c9d9740 1 -- 172.21.15.86:0/232025165 wait complete. 2024-04-16T05:39:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.805+0000 7fb24c9d9740 20 client.16968 trim_cache size 0 max 16384 2024-04-16T05:39:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb24c9d9740 1 Processor -- start 2024-04-16T05:39:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb24c9d9740 1 -- start start 2024-04-16T05:39:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404abe50 0x55e6404b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404c1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 0x55e6404b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2441af530 con 0x55e6404ac770 2024-04-16T05:39:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2441b1070 con 0x55e6404abe50 2024-04-16T05:39:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2441afc00 con 0x55e6404b5350 2024-04-16T05:39:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb2437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb2437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404c1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52620/0 (socket says 172.21.15.86:52620) 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb2437fe640 1 -- 172.21.15.86:0/479105565 learned_addr learned my addr 172.21.15.86:0/479105565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb2437fe640 1 -- 172.21.15.86:0/479105565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 msgr2=0x55e6404b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb2437fe640 1 --2- 172.21.15.86:0/479105565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 0x55e6404b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb2437fe640 1 -- 172.21.15.86:0/479105565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404abe50 msgr2=0x55e6404b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.822+0000 7fb2437fe640 1 --2- 172.21.15.86:0/479105565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404abe50 0x55e6404b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb2437fe640 1 -- 172.21.15.86:0/479105565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6404c80b0 con 0x55e6404ac770 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb2437fe640 1 --2- 172.21.15.86:0/479105565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404c1000 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb23400ac20 tx=0x7fb2340106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da12128048b48ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb2297fa640 1 -- 172.21.15.86:0/479105565 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234030070 con 0x55e6404ac770 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb2297fa640 1 -- 172.21.15.86:0/479105565 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2340184c0 con 0x55e6404ac770 2024-04-16T05:39:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb2297fa640 1 -- 172.21.15.86:0/479105565 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23401d040 con 0x55e6404ac770 2024-04-16T05:39:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb24c9d9740 1 -- 172.21.15.86:0/479105565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 msgr2=0x55e6404c1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/479105565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404c1000 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb23400ac20 tx=0x7fb2340106c0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb24c9d9740 1 -- 172.21.15.86:0/479105565 shutdown_connections 2024-04-16T05:39:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/479105565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404b5350 0x55e6404b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/479105565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404c1000 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/479105565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404abe50 0x55e6404b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb24c9d9740 1 -- 172.21.15.86:0/479105565 >> 172.21.15.86:0/479105565 conn(0x55e6404a6580 msgr2=0x55e6404a5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.823+0000 7fb24c9d9740 1 -- 172.21.15.86:0/479105565 shutdown_connections 2024-04-16T05:39:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 1 -- 172.21.15.86:0/479105565 wait complete. 2024-04-16T05:39:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 1 -- start start 2024-04-16T05:39:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 1 Processor -- start 2024-04-16T05:39:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404abe50 0x55e6404ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404b5350 0x55e64052df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2441b09a0 con 0x55e6404ac770 2024-04-16T05:39:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2441b02d0 con 0x55e6404b5350 2024-04-16T05:39:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2441ad4f0 con 0x55e6404abe50 2024-04-16T05:39:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb24c9d9740 10 client.0 _ll_register_callbacks cb 0x7fb238009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb2437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb2437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404bac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52626/0 (socket says 172.21.15.86:52626) 2024-04-16T05:39:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.824+0000 7fb2437fe640 1 -- 172.21.15.86:0/110924220 learned_addr learned my addr 172.21.15.86:0/110924220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb242ffd640 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404b5350 0x55e64052df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb2437fe640 1 -- 172.21.15.86:0/110924220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404abe50 msgr2=0x55e6404ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb2437fe640 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404abe50 0x55e6404ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb2437fe640 1 -- 172.21.15.86:0/110924220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404b5350 msgr2=0x55e64052df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb2437fe640 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404b5350 0x55e64052df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb2437fe640 1 -- 172.21.15.86:0/110924220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e64052e510 con 0x55e6404ac770 2024-04-16T05:39:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb242ffd640 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404b5350 0x55e64052df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb2437fe640 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404bac10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb23400abf0 tx=0x7fb23400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac4c8e82fb3d97a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb222ffd640 1 -- 172.21.15.86:0/110924220 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234030070 con 0x55e6404ac770 2024-04-16T05:39:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb222ffd640 1 -- 172.21.15.86:0/110924220 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb234018930 con 0x55e6404ac770 2024-04-16T05:39:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb24c9d9740 1 -- 172.21.15.86:0/110924220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2340184c0 con 0x55e6404ac770 2024-04-16T05:39:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb222ffd640 1 -- 172.21.15.86:0/110924220 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb23401d050 con 0x55e6404ac770 2024-04-16T05:39:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb24c9d9740 10 client.16974 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb24c9d9740 1 -- 172.21.15.86:0/110924220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e64052f820 con 0x55e6404ac770 2024-04-16T05:39:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb222ffd640 1 -- 172.21.15.86:0/110924220 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2340184c0 con 0x55e6404ac770 2024-04-16T05:39:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.825+0000 7fb222ffd640 1 -- 172.21.15.86:0/110924220 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb23402b040 con 0x55e6404ac770 2024-04-16T05:39:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb24c9d9740 10 client.16974 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb24c9d9740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb24c9d9740 20 client.16974 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e64052fc40 0x55e640532050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb24c9d9740 4 client.16974 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:39:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb24c9d9740 1 -- 172.21.15.86:0/110924220 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e640418a10 con 0x55e64052fc40 2024-04-16T05:39:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb242ffd640 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e64052fc40 0x55e640532050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb242ffd640 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e64052fc40 0x55e640532050 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff9831e8c3936d3 server_cookie=4b76bf8594560118 in_seq=0 out_seq=0 2024-04-16T05:39:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb222ffd640 10 client.16974 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb222ffd640 1 -- 172.21.15.86:0/110924220 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e640418a10 con 0x55e64052fc40 2024-04-16T05:39:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb222ffd640 10 client.16974 handle_command_reply: tid=0 2024-04-16T05:39:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.826+0000 7fb24c9d9740 1 client.16974 shutdown 2024-04-16T05:39:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 -- 172.21.15.86:0/110924220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 msgr2=0x55e6404bac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404bac10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb23400abf0 tx=0x7fb23400f790 comp rx=0 tx=0).stop 2024-04-16T05:39:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 -- 172.21.15.86:0/110924220 shutdown_connections 2024-04-16T05:39:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e64052fc40 0x55e640532050 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e6404b5350 0x55e64052df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb222ffd640 0 client.16974 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6404ac770 0x55e6404bac10 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 --2- 172.21.15.86:0/110924220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6404abe50 0x55e6404ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 -- 172.21.15.86:0/110924220 >> 172.21.15.86:0/110924220 conn(0x55e6404a6580 msgr2=0x55e6404b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 -- 172.21.15.86:0/110924220 shutdown_connections 2024-04-16T05:39:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 1 -- 172.21.15.86:0/110924220 wait complete. 2024-04-16T05:39:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb24c9d9740 20 client.16974 trim_cache size 0 max 16384 2024-04-16T05:39:43.836 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:39:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:39:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:39:43.836 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:39:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 -- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb22c0a3d20 msgr2=0x7fb22c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb22c0a3d20 0x7fb22c0a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb23c09c930 tx=0x7fb23c087040 comp rx=0 tx=0).stop 2024-04-16T05:39:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 -- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24410c5d0 msgr2=0x7fb2441986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24410c5d0 0x7fb2441986e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00eaa0 tx=0x7fb23c00ef60 comp rx=0 tx=0).stop 2024-04-16T05:39:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 -- 172.21.15.86:0/43452654 shutdown_connections 2024-04-16T05:39:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24410c5d0 0x7fb2441986e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb244103bb0 0x7fb24419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb22c0a3d20 0x7fb22c0a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 --2- 172.21.15.86:0/43452654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb244103290 0x7fb24419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.827+0000 7fb249d15640 1 -- 172.21.15.86:0/43452654 >> 172.21.15.86:0/43452654 conn(0x7fb244077280 msgr2=0x7fb2440fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.828+0000 7fb249d15640 1 -- 172.21.15.86:0/43452654 shutdown_connections 2024-04-16T05:39:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.828+0000 7fb249d15640 1 -- 172.21.15.86:0/43452654 wait complete. 2024-04-16T05:39:43.839 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:39:43.839 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:39:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.945+0000 7ff71bbd4640 1 -- 172.21.15.86:0/2336706656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714101b70 msgr2=0x7ff714105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.945+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/2336706656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714101b70 0x7ff714105b60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff704009900 tx=0x7ff70402bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.945+0000 7ff71bbd4640 1 -- 172.21.15.86:0/2336706656 shutdown_connections 2024-04-16T05:39:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.945+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/2336706656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff714106240 0x7ff71410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.945+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/2336706656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714101b70 0x7ff714105b60 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.945+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/2336706656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7141011c0 0x7ff7141015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.945+0000 7ff71bbd4640 1 -- 172.21.15.86:0/2336706656 >> 172.21.15.86:0/2336706656 conn(0x7ff714078060 msgr2=0x7ff714078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.945+0000 7ff71bbd4640 1 -- 172.21.15.86:0/2336706656 shutdown_connections 2024-04-16T05:39:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.945+0000 7ff71bbd4640 1 -- 172.21.15.86:0/2336706656 wait complete. 2024-04-16T05:39:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71bbd4640 1 Processor -- start 2024-04-16T05:39:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71bbd4640 1 -- start start 2024-04-16T05:39:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71bbd4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7141011c0 0x7ff71419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71bbd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714106240 0x7ff71419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71bbd4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff714198750 0x7ff714198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71bbd4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff714112410 con 0x7ff714106240 2024-04-16T05:39:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71bbd4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff714112290 con 0x7ff7141011c0 2024-04-16T05:39:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71bbd4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff714112590 con 0x7ff714198750 2024-04-16T05:39:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71916f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714106240 0x7ff71419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71916f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714106240 0x7ff71419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52634/0 (socket says 172.21.15.86:52634) 2024-04-16T05:39:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71916f640 1 -- 172.21.15.86:0/795642981 learned_addr learned my addr 172.21.15.86:0/795642981 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff719970640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7141011c0 0x7ff71419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71916f640 1 -- 172.21.15.86:0/795642981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff714198750 msgr2=0x7ff714198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71916f640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff714198750 0x7ff714198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71916f640 1 -- 172.21.15.86:0/795642981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7141011c0 msgr2=0x7ff71419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71916f640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7141011c0 0x7ff71419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71916f640 1 -- 172.21.15.86:0/795642981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff714199350 con 0x7ff714106240 2024-04-16T05:39:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff719970640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7141011c0 0x7ff71419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.946+0000 7ff71916f640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714106240 0x7ff71419eb80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff704036040 tx=0x7ff70402cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.947+0000 7ff70affd640 1 -- 172.21.15.86:0/795642981 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704041070 con 0x7ff714106240 2024-04-16T05:39:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.947+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7141995e0 con 0x7ff714106240 2024-04-16T05:39:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.947+0000 7ff70affd640 1 -- 172.21.15.86:0/795642981 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff704004450 con 0x7ff714106240 2024-04-16T05:39:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.947+0000 7ff70affd640 1 -- 172.21.15.86:0/795642981 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704003dc0 con 0x7ff714106240 2024-04-16T05:39:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.947+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7141a5720 con 0x7ff714106240 2024-04-16T05:39:43.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.948+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff714102640 con 0x7ff714106240 2024-04-16T05:39:43.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.949+0000 7ff70affd640 1 -- 172.21.15.86:0/795642981 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff704044070 con 0x7ff714106240 2024-04-16T05:39:43.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.950+0000 7ff70affd640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7000a3d20 0x7ff7000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:43.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.950+0000 7ff719970640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7000a3d20 0x7ff7000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:43.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.950+0000 7ff70affd640 1 -- 172.21.15.86:0/795642981 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff7040f2650 con 0x7ff714106240 2024-04-16T05:39:43.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.950+0000 7ff719970640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7000a3d20 0x7ff7000a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005fd0 tx=0x7ff70c009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:43.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:43.955+0000 7ff70affd640 1 -- 172.21.15.86:0/795642981 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7040bea90 con 0x7ff714106240 2024-04-16T05:39:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.171+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff714101b70 con 0x7ff714106240 2024-04-16T05:39:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.172+0000 7ff70affd640 1 -- 172.21.15.86:0/795642981 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7ff7040be390 con 0x7ff714106240 2024-04-16T05:39:44.171 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:39:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:39:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:39:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7000a3d20 msgr2=0x7ff7000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7000a3d20 0x7ff7000a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005fd0 tx=0x7ff70c009500 comp rx=0 tx=0).stop 2024-04-16T05:39:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714106240 msgr2=0x7ff71419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714106240 0x7ff71419eb80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff704036040 tx=0x7ff70402cb20 comp rx=0 tx=0).stop 2024-04-16T05:39:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 shutdown_connections 2024-04-16T05:39:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff714198750 0x7ff714198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff714106240 0x7ff71419eb80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff7000a3d20 0x7ff7000a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 --2- 172.21.15.86:0/795642981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7141011c0 0x7ff71419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 >> 172.21.15.86:0/795642981 conn(0x7ff714078060 msgr2=0x7ff7140ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 shutdown_connections 2024-04-16T05:39:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.174+0000 7ff71bbd4640 1 -- 172.21.15.86:0/795642981 wait complete. 2024-04-16T05:39:44.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:39:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.292+0000 7fbc04baa640 1 -- 172.21.15.86:0/596406416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00100770 msgr2=0x7fbc00100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.292+0000 7fbc04baa640 1 --2- 172.21.15.86:0/596406416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00100770 0x7fbc00100b50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009900 tx=0x7fbbf002bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.292+0000 7fbc04baa640 1 -- 172.21.15.86:0/596406416 shutdown_connections 2024-04-16T05:39:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.292+0000 7fbc04baa640 1 --2- 172.21.15.86:0/596406416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc00107330 0x7fbc0010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.292+0000 7fbc04baa640 1 --2- 172.21.15.86:0/596406416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00101120 0x7fbc00106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.292+0000 7fbc04baa640 1 --2- 172.21.15.86:0/596406416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00100770 0x7fbc00100b50 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.292+0000 7fbc04baa640 1 -- 172.21.15.86:0/596406416 >> 172.21.15.86:0/596406416 conn(0x7fbc00077450 msgr2=0x7fbc000fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.292+0000 7fbc04baa640 1 -- 172.21.15.86:0/596406416 shutdown_connections 2024-04-16T05:39:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.292+0000 7fbc04baa640 1 -- 172.21.15.86:0/596406416 wait complete. 2024-04-16T05:39:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbc04baa640 1 Processor -- start 2024-04-16T05:39:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbc04baa640 1 -- start start 2024-04-16T05:39:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbc04baa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00100770 0x7fbc001016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbc04baa640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc00107330 0x7fbc00101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbc04baa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00102160 0x7fbc001a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbc04baa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001123e0 con 0x7fbc00102160 2024-04-16T05:39:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbc04baa640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00112260 con 0x7fbc00107330 2024-04-16T05:39:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbc04baa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00112560 con 0x7fbc00100770 2024-04-16T05:39:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00102160 0x7fbc001a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00102160 0x7fbc001a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52658/0 (socket says 172.21.15.86:52658) 2024-04-16T05:39:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfed9d640 1 -- 172.21.15.86:0/2883723527 learned_addr learned my addr 172.21.15.86:0/2883723527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc00107330 0x7fbc00101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfed9d640 1 -- 172.21.15.86:0/2883723527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00100770 msgr2=0x7fbc001016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfed9d640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00100770 0x7fbc001016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfed9d640 1 -- 172.21.15.86:0/2883723527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc00107330 msgr2=0x7fbc00101c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfed9d640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc00107330 0x7fbc00101c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfed9d640 1 -- 172.21.15.86:0/2883723527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc001a71f0 con 0x7fbc00102160 2024-04-16T05:39:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.293+0000 7fbbfed9d640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00102160 0x7fbc001a6af0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800eaa0 tx=0x7fbbf800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.294+0000 7fbbef7fe640 1 -- 172.21.15.86:0/2883723527 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800cdb0 con 0x7fbc00102160 2024-04-16T05:39:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.294+0000 7fbbef7fe640 1 -- 172.21.15.86:0/2883723527 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf8005020 con 0x7fbc00102160 2024-04-16T05:39:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.294+0000 7fbc04baa640 1 -- 172.21.15.86:0/2883723527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc001a74e0 con 0x7fbc00102160 2024-04-16T05:39:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.294+0000 7fbbef7fe640 1 -- 172.21.15.86:0/2883723527 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80116b0 con 0x7fbc00102160 2024-04-16T05:39:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.294+0000 7fbc04baa640 1 -- 172.21.15.86:0/2883723527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc001a79e0 con 0x7fbc00102160 2024-04-16T05:39:44.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.296+0000 7fbbef7fe640 1 -- 172.21.15.86:0/2883723527 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbf8005560 con 0x7fbc00102160 2024-04-16T05:39:44.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.296+0000 7fbc0778b740 1 Processor -- start 2024-04-16T05:39:44.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.296+0000 7fbc0778b740 1 -- start start 2024-04-16T05:39:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 0x55c6c4b96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4b9ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4ba1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbc0778b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6c4afb0e0 con 0x55c6c4b96780 2024-04-16T05:39:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbc0778b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c6c4a98520 con 0x55c6c4b9f360 2024-04-16T05:39:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbc0778b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6c4af70a0 con 0x55c6c4b95e60 2024-04-16T05:39:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4b9ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4b9ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52660/0 (socket says 172.21.15.86:52660) 2024-04-16T05:39:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfe59c640 1 -- 172.21.15.86:0/1435614455 learned_addr learned my addr 172.21.15.86:0/1435614455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4ba1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbef7fe640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbbe80a3d50 0x7fbbe80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 0x55c6c4b96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfed9d640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbbe80a3d50 0x7fbbe80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfe59c640 1 -- 172.21.15.86:0/1435614455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 msgr2=0x55c6c4b96240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfe59c640 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 0x55c6c4b96240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfe59c640 1 -- 172.21.15.86:0/1435614455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 msgr2=0x55c6c4ba1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfe59c640 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4ba1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfe59c640 1 -- 172.21.15.86:0/1435614455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c4ac08d0 con 0x55c6c4b96780 2024-04-16T05:39:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 0x55c6c4b96240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbef7fe640 1 -- 172.21.15.86:0/2883723527 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbbf80d0de0 con 0x7fbc00102160 2024-04-16T05:39:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfe59c640 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4b9ee20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0036040 tx=0x7fbbf0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55ac3a05e5b7bb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbecff9640 1 -- 172.21.15.86:0/1435614455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf002c880 con 0x55c6c4b96780 2024-04-16T05:39:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.297+0000 7fbbfed9d640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbbe80a3d50 0x7fbbe80a6210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80993e0 tx=0x7fbbf80bd040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbbecff9640 1 -- 172.21.15.86:0/1435614455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf0042070 con 0x55c6c4b96780 2024-04-16T05:39:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbbecff9640 1 -- 172.21.15.86:0/1435614455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00046c0 con 0x55c6c4b96780 2024-04-16T05:39:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbc0778b740 1 -- 172.21.15.86:0/1435614455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 msgr2=0x55c6c4b9ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbc0778b740 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4b9ee20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0036040 tx=0x7fbbf0005d50 comp rx=0 tx=0).stop 2024-04-16T05:39:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbc0778b740 1 -- 172.21.15.86:0/1435614455 shutdown_connections 2024-04-16T05:39:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbc0778b740 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4ba1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbc0778b740 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4b9ee20 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbc0778b740 1 --2- 172.21.15.86:0/1435614455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 0x55c6c4b96240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbc0778b740 1 -- 172.21.15.86:0/1435614455 >> 172.21.15.86:0/1435614455 conn(0x55c6c4b90590 msgr2=0x55c6c4b90970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbc0778b740 1 -- 172.21.15.86:0/1435614455 shutdown_connections 2024-04-16T05:39:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.298+0000 7fbc0778b740 1 -- 172.21.15.86:0/1435614455 wait complete. 2024-04-16T05:39:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbc0778b740 1 -- start start 2024-04-16T05:39:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbc0778b740 1 Processor -- start 2024-04-16T05:39:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b95e60 0x55c6c4b9c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b96780 0x55c6c4b9c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4b9ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbc0778b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6c4ba2060 con 0x55c6c4b95e60 2024-04-16T05:39:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbc0778b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c6c4ab44d0 con 0x55c6c4b9f360 2024-04-16T05:39:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbc0778b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6c4ba21e0 con 0x55c6c4b96780 2024-04-16T05:39:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbc0778b740 10 client.0 _ll_register_callbacks cb 0x55c6c4a992c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b95e60 0x55c6c4b9c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b95e60 0x55c6c4b9c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52668/0 (socket says 172.21.15.86:52668) 2024-04-16T05:39:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 -- 172.21.15.86:0/659087187 learned_addr learned my addr 172.21.15.86:0/659087187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4b9ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 -- 172.21.15.86:0/659087187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b96780 msgr2=0x55c6c4b9c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b96780 0x55c6c4b9c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 -- 172.21.15.86:0/659087187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 msgr2=0x55c6c4b9ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4b9ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 -- 172.21.15.86:0/659087187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c480d3f0 con 0x55c6c4b95e60 2024-04-16T05:39:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b95e60 0x55c6c4b9c400 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400ef80 tx=0x7fbbf4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a3a7c1c02a922e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.299+0000 7fbbe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/659087187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4005020 con 0x55c6c4b95e60 2024-04-16T05:39:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/659087187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbbf40051c0 con 0x55c6c4b95e60 2024-04-16T05:39:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbc0778b740 1 -- 172.21.15.86:0/659087187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf0042070 con 0x55c6c4b95e60 2024-04-16T05:39:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/659087187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbf4027050 con 0x55c6c4b95e60 2024-04-16T05:39:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbc0778b740 10 client.16992 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbc0778b740 1 -- 172.21.15.86:0/659087187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6c48d77d0 con 0x55c6c4b95e60 2024-04-16T05:39:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/659087187 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf0042070 con 0x55c6c4b95e60 2024-04-16T05:39:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/659087187 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbbf400f040 con 0x55c6c4b95e60 2024-04-16T05:39:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbc0778b740 10 client.16992 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbc0778b740 10 client.16992 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:39:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbc0778b740 20 client.16992 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbc0778b740 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55c6c4baf390 0x55c6c4bb17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbc0778b740 4 client.16992 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.300+0000 7fbc0778b740 1 -- 172.21.15.86:0/659087187 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6c4b08b00 con 0x55c6c4baf390 2024-04-16T05:39:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.301+0000 7fbbfe59c640 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55c6c4baf390 0x55c6c4bb17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.301+0000 7fbbe5ffb640 10 client.16992 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:39:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.301+0000 7fbbfe59c640 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55c6c4baf390 0x55c6c4bb17f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a7185871c2600fb server_cookie=ac4ee5ac2e5825a7 in_seq=0 out_seq=0 2024-04-16T05:39:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/659087187 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6c4b08b00 con 0x55c6c4baf390 2024-04-16T05:39:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbbe5ffb640 10 client.16992 handle_command_reply: tid=0 2024-04-16T05:39:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 client.16992 shutdown 2024-04-16T05:39:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 -- 172.21.15.86:0/659087187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b95e60 msgr2=0x55c6c4b9c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b95e60 0x55c6c4b9c400 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400ef80 tx=0x7fbbf4004730 comp rx=0 tx=0).stop 2024-04-16T05:39:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 -- 172.21.15.86:0/659087187 shutdown_connections 2024-04-16T05:39:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55c6c4baf390 0x55c6c4bb17f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4b9ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b96780 0x55c6c4b9c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 --2- 172.21.15.86:0/659087187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b95e60 0x55c6c4b9c400 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbbe5ffb640 0 client.16992 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:39:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 -- 172.21.15.86:0/659087187 >> 172.21.15.86:0/659087187 conn(0x55c6c4b90590 msgr2=0x55c6c4b922e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 -- 172.21.15.86:0/659087187 shutdown_connections 2024-04-16T05:39:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 1 -- 172.21.15.86:0/659087187 wait complete. 2024-04-16T05:39:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.302+0000 7fbc0778b740 20 client.16992 trim_cache size 0 max 16384 2024-04-16T05:39:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbc0778b740 1 Processor -- start 2024-04-16T05:39:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbc0778b740 1 -- start start 2024-04-16T05:39:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b95e60 0x55c6c4bab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b96780 0x55c6c4b9e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b9f360 0x55c6c4bb70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbc0778b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001b16f0 con 0x55c6c4b9f360 2024-04-16T05:39:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbc0778b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001b3230 con 0x55c6c4b95e60 2024-04-16T05:39:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbc0778b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001b1dc0 con 0x55c6c4b96780 2024-04-16T05:39:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbbfdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b9f360 0x55c6c4bb70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbbfdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b9f360 0x55c6c4bb70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52684/0 (socket says 172.21.15.86:52684) 2024-04-16T05:39:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.315+0000 7fbbfdd9b640 1 -- 172.21.15.86:0/3242112990 learned_addr learned my addr 172.21.15.86:0/3242112990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbbfdd9b640 1 -- 172.21.15.86:0/3242112990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b96780 msgr2=0x55c6c4b9e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/3242112990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b96780 0x55c6c4b9e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbbfdd9b640 1 -- 172.21.15.86:0/3242112990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b95e60 msgr2=0x55c6c4bab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/3242112990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b95e60 0x55c6c4bab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbbfdd9b640 1 -- 172.21.15.86:0/3242112990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c4bb1e70 con 0x55c6c4b9f360 2024-04-16T05:39:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/3242112990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b9f360 0x55c6c4bb70c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4010fd0 tx=0x7fbbf401f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b30eb12a470731a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbbe7fff640 1 -- 172.21.15.86:0/3242112990 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4005850 con 0x55c6c4b9f360 2024-04-16T05:39:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbbe7fff640 1 -- 172.21.15.86:0/3242112990 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf4024070 con 0x55c6c4b9f360 2024-04-16T05:39:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbbe7fff640 1 -- 172.21.15.86:0/3242112990 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4020880 con 0x55c6c4b9f360 2024-04-16T05:39:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbc0778b740 1 -- 172.21.15.86:0/3242112990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b9f360 msgr2=0x55c6c4bb70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbc0778b740 1 --2- 172.21.15.86:0/3242112990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b9f360 0x55c6c4bb70c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4010fd0 tx=0x7fbbf401f610 comp rx=0 tx=0).stop 2024-04-16T05:39:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbc0778b740 1 -- 172.21.15.86:0/3242112990 shutdown_connections 2024-04-16T05:39:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbc0778b740 1 --2- 172.21.15.86:0/3242112990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b9f360 0x55c6c4bb70c0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbc0778b740 1 --2- 172.21.15.86:0/3242112990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b96780 0x55c6c4b9e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbc0778b740 1 --2- 172.21.15.86:0/3242112990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b95e60 0x55c6c4bab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbc0778b740 1 -- 172.21.15.86:0/3242112990 >> 172.21.15.86:0/3242112990 conn(0x55c6c4b90590 msgr2=0x55c6c4b9db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbc0778b740 1 -- 172.21.15.86:0/3242112990 shutdown_connections 2024-04-16T05:39:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.316+0000 7fbc0778b740 1 -- 172.21.15.86:0/3242112990 wait complete. 2024-04-16T05:39:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbc0778b740 1 -- start start 2024-04-16T05:39:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbc0778b740 1 Processor -- start 2024-04-16T05:39:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 0x55c6c4ba30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4ba3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbc0778b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4ba3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbc0778b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001b2b60 con 0x55c6c4b96780 2024-04-16T05:39:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbc0778b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001b2490 con 0x55c6c4b9f360 2024-04-16T05:39:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbc0778b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001af6b0 con 0x55c6c4b95e60 2024-04-16T05:39:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbc0778b740 10 client.0 _ll_register_callbacks cb 0x7fbbf400ec90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4ba3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4ba3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52690/0 (socket says 172.21.15.86:52690) 2024-04-16T05:39:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfe59c640 1 -- 172.21.15.86:0/2340744283 learned_addr learned my addr 172.21.15.86:0/2340744283 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 0x55c6c4ba30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfe59c640 1 -- 172.21.15.86:0/2340744283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 msgr2=0x55c6c4ba30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfe59c640 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 0x55c6c4ba30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfe59c640 1 -- 172.21.15.86:0/2340744283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 msgr2=0x55c6c4ba3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfe59c640 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4ba3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfe59c640 1 -- 172.21.15.86:0/2340744283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c4c14a90 con 0x55c6c4b96780 2024-04-16T05:39:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbfe59c640 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4ba3600 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009900 tx=0x7fbbf002c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e9cecbb2693388 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.317+0000 7fbbe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/2340744283 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf004a070 con 0x55c6c4b96780 2024-04-16T05:39:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/2340744283 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbbf002cc90 con 0x55c6c4b96780 2024-04-16T05:39:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbc0778b740 1 -- 172.21.15.86:0/2340744283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf4024070 con 0x55c6c4b96780 2024-04-16T05:39:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/2340744283 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbf0004030 con 0x55c6c4b96780 2024-04-16T05:39:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbc0778b740 10 client.16998 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbc0778b740 1 -- 172.21.15.86:0/2340744283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6c4c15d70 con 0x55c6c4b96780 2024-04-16T05:39:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/2340744283 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf4024070 con 0x55c6c4b96780 2024-04-16T05:39:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/2340744283 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbbf004b3f0 con 0x55c6c4b96780 2024-04-16T05:39:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbc0778b740 10 client.16998 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbc0778b740 10 client.16998 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:39:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbc0778b740 20 client.16998 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbc0778b740 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55c6c4c161c0 0x55c6c4c185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbc0778b740 4 client.16998 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbc0778b740 1 -- 172.21.15.86:0/2340744283 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6c4b02970 con 0x55c6c4c161c0 2024-04-16T05:39:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.318+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55c6c4c161c0 0x55c6c4c185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbbe5ffb640 10 client.16998 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:39:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbbfdd9b640 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55c6c4c161c0 0x55c6c4c185b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7da315a245383197 server_cookie=c4355fe907854e1a in_seq=0 out_seq=0 2024-04-16T05:39:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbbe5ffb640 1 -- 172.21.15.86:0/2340744283 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6c4b02970 con 0x55c6c4c161c0 2024-04-16T05:39:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbbe5ffb640 10 client.16998 handle_command_reply: tid=0 2024-04-16T05:39:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 client.16998 shutdown 2024-04-16T05:39:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 -- 172.21.15.86:0/2340744283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 msgr2=0x55c6c4ba3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4ba3600 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009900 tx=0x7fbbf002c770 comp rx=0 tx=0).stop 2024-04-16T05:39:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 -- 172.21.15.86:0/2340744283 shutdown_connections 2024-04-16T05:39:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55c6c4c161c0 0x55c6c4c185b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c4b9f360 0x55c6c4ba3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbbe5ffb640 0 client.16998 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:39:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c4b96780 0x55c6c4ba3600 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 --2- 172.21.15.86:0/2340744283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c4b95e60 0x55c6c4ba30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 -- 172.21.15.86:0/2340744283 >> 172.21.15.86:0/2340744283 conn(0x55c6c4b90590 msgr2=0x55c6c4b9cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 -- 172.21.15.86:0/2340744283 shutdown_connections 2024-04-16T05:39:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 1 -- 172.21.15.86:0/2340744283 wait complete. 2024-04-16T05:39:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.319+0000 7fbc0778b740 20 client.16998 trim_cache size 0 max 16384 2024-04-16T05:39:44.328 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 -- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbbe80a3d50 msgr2=0x7fbbe80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbbe80a3d50 0x7fbbe80a6210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80993e0 tx=0x7fbbf80bd040 comp rx=0 tx=0).stop 2024-04-16T05:39:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 -- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00102160 msgr2=0x7fbc001a6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00102160 0x7fbc001a6af0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800eaa0 tx=0x7fbbf800ef60 comp rx=0 tx=0).stop 2024-04-16T05:39:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 -- 172.21.15.86:0/2883723527 shutdown_connections 2024-04-16T05:39:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc00102160 0x7fbc001a6af0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fbbe80a3d50 0x7fbbe80a6210 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc00107330 0x7fbc00101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 --2- 172.21.15.86:0/2883723527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00100770 0x7fbc001016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 -- 172.21.15.86:0/2883723527 >> 172.21.15.86:0/2883723527 conn(0x7fbc00077450 msgr2=0x7fbc001055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 -- 172.21.15.86:0/2883723527 shutdown_connections 2024-04-16T05:39:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.320+0000 7fbc04baa640 1 -- 172.21.15.86:0/2883723527 wait complete. 2024-04-16T05:39:44.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:39:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 -- 172.21.15.86:0/1530327440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e8106f80 msgr2=0x7ff6e806a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 --2- 172.21.15.86:0/1530327440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e8106f80 0x7ff6e806a6a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009ca0 tx=0x7ff6e402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 -- 172.21.15.86:0/1530327440 shutdown_connections 2024-04-16T05:39:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 --2- 172.21.15.86:0/1530327440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e8104a10 0x7ff6e8104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 --2- 172.21.15.86:0/1530327440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e806abe0 0x7ff6e806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 --2- 172.21.15.86:0/1530327440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e8106f80 0x7ff6e806a6a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 -- 172.21.15.86:0/1530327440 >> 172.21.15.86:0/1530327440 conn(0x7ff6e8077cf0 msgr2=0x7ff6e8078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 -- 172.21.15.86:0/1530327440 shutdown_connections 2024-04-16T05:39:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 -- 172.21.15.86:0/1530327440 wait complete. 2024-04-16T05:39:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 Processor -- start 2024-04-16T05:39:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 -- start start 2024-04-16T05:39:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806abe0 0x7ff6e819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.439+0000 7ff6ef959640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e8104a10 0x7ff6e819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ef959640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e8106f80 0x7ff6e81a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ef959640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e8118920 con 0x7ff6e806abe0 2024-04-16T05:39:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ef959640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81187a0 con 0x7ff6e8106f80 2024-04-16T05:39:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ef959640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e8118aa0 con 0x7ff6e8104a10 2024-04-16T05:39:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ed6f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806abe0 0x7ff6e819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ed6f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806abe0 0x7ff6e819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52706/0 (socket says 172.21.15.86:52706) 2024-04-16T05:39:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ed6f5640 1 -- 172.21.15.86:0/184889388 learned_addr learned my addr 172.21.15.86:0/184889388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6edef6640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e8106f80 0x7ff6e81a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e8104a10 0x7ff6e819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ed6f5640 1 -- 172.21.15.86:0/184889388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e8104a10 msgr2=0x7ff6e819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e8104a10 0x7ff6e819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ed6f5640 1 -- 172.21.15.86:0/184889388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e8106f80 msgr2=0x7ff6e81a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e8106f80 0x7ff6e81a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ed6f5640 1 -- 172.21.15.86:0/184889388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e81a3d60 con 0x7ff6e806abe0 2024-04-16T05:39:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6edef6640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e8106f80 0x7ff6e81a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e8104a10 0x7ff6e819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806abe0 0x7ff6e819edc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40098d0 tx=0x7ff6e4035b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6de7fc640 1 -- 172.21.15.86:0/184889388 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4039040 con 0x7ff6e806abe0 2024-04-16T05:39:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6de7fc640 1 -- 172.21.15.86:0/184889388 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e4044070 con 0x7ff6e806abe0 2024-04-16T05:39:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ef959640 1 -- 172.21.15.86:0/184889388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e81a4050 con 0x7ff6e806abe0 2024-04-16T05:39:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6de7fc640 1 -- 172.21.15.86:0/184889388 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4004370 con 0x7ff6e806abe0 2024-04-16T05:39:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.440+0000 7ff6ef959640 1 -- 172.21.15.86:0/184889388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e81ab8e0 con 0x7ff6e806abe0 2024-04-16T05:39:44.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.442+0000 7ff6de7fc640 1 -- 172.21.15.86:0/184889388 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff6e4004510 con 0x7ff6e806abe0 2024-04-16T05:39:44.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6f2626740 1 Processor -- start 2024-04-16T05:39:44.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6f2626740 1 -- start start 2024-04-16T05:39:44.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x560681206240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681206780 0x56068120ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56068120f360 0x560681211750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6f2626740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56068116b0e0 con 0x56068120f360 2024-04-16T05:39:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6f2626740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560681108520 con 0x560681205e60 2024-04-16T05:39:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6f2626740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5606811670a0 con 0x560681206780 2024-04-16T05:39:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6de7fc640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6ecef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56068120f360 0x560681211750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6ecef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56068120f360 0x560681211750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52720/0 (socket says 172.21.15.86:52720) 2024-04-16T05:39:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6ecef4640 1 -- 172.21.15.86:0/1213722351 learned_addr learned my addr 172.21.15.86:0/1213722351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6edef6640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6de7fc640 1 -- 172.21.15.86:0/184889388 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff6e40f30a0 con 0x7ff6e806abe0 2024-04-16T05:39:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6edef6640 1 --2- 172.21.15.86:0/1213722351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x560681206240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6ecef4640 1 -- 172.21.15.86:0/1213722351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681206780 msgr2=0x56068120ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/1213722351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681206780 0x56068120ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.443+0000 7ff6ecef4640 1 -- 172.21.15.86:0/1213722351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 msgr2=0x560681206240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.444+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/1213722351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x560681206240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.444+0000 7ff6ecef4640 1 -- 172.21.15.86:0/1213722351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606811308d0 con 0x56068120f360 2024-04-16T05:39:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.444+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/1213722351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56068120f360 0x560681211750 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800a9d0 tx=0x7ff6d800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24286a69225ea43 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.444+0000 7ff6edef6640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0005d50 tx=0x7ff6e0005f10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.444+0000 7ff6d37fe640 1 -- 172.21.15.86:0/1213722351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8015070 con 0x56068120f360 2024-04-16T05:39:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.444+0000 7ff6d37fe640 1 -- 172.21.15.86:0/1213722351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d800cc60 con 0x56068120f360 2024-04-16T05:39:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.444+0000 7ff6d37fe640 1 -- 172.21.15.86:0/1213722351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80105b0 con 0x56068120f360 2024-04-16T05:39:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 -- 172.21.15.86:0/1213722351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56068120f360 msgr2=0x560681211750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 --2- 172.21.15.86:0/1213722351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56068120f360 0x560681211750 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800a9d0 tx=0x7ff6d800ae90 comp rx=0 tx=0).stop 2024-04-16T05:39:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 -- 172.21.15.86:0/1213722351 shutdown_connections 2024-04-16T05:39:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 --2- 172.21.15.86:0/1213722351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56068120f360 0x560681211750 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 --2- 172.21.15.86:0/1213722351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681206780 0x56068120ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 --2- 172.21.15.86:0/1213722351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x560681206240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 -- 172.21.15.86:0/1213722351 >> 172.21.15.86:0/1213722351 conn(0x560681200590 msgr2=0x560681200970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 -- 172.21.15.86:0/1213722351 shutdown_connections 2024-04-16T05:39:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 -- 172.21.15.86:0/1213722351 wait complete. 2024-04-16T05:39:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 -- start start 2024-04-16T05:39:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 Processor -- start 2024-04-16T05:39:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x56068120c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068120c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56068120f360 0x56068120cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560681212000 con 0x560681206780 2024-04-16T05:39:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5606811244d0 con 0x560681205e60 2024-04-16T05:39:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560681212180 con 0x56068120f360 2024-04-16T05:39:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.445+0000 7ff6f2626740 10 client.0 _ll_register_callbacks cb 0x5606811092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ecef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068120c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ecef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068120c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52724/0 (socket says 172.21.15.86:52724) 2024-04-16T05:39:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ecef4640 1 -- 172.21.15.86:0/3374679433 learned_addr learned my addr 172.21.15.86:0/3374679433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6edef6640 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56068120f360 0x56068120cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x56068120c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ecef4640 1 -- 172.21.15.86:0/3374679433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56068120f360 msgr2=0x56068120cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56068120f360 0x56068120cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ecef4640 1 -- 172.21.15.86:0/3374679433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 msgr2=0x56068120c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x56068120c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ecef4640 1 -- 172.21.15.86:0/3374679433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560680e7d3f0 con 0x560681206780 2024-04-16T05:39:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068120c8b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8008010 tx=0x7ff6d8011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bb3507284084aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/3374679433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8020070 con 0x560681206780 2024-04-16T05:39:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6f2626740 1 -- 172.21.15.86:0/3374679433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d800cc60 con 0x560681206780 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/3374679433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff6d801a660 con 0x560681206780 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/3374679433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff6d8015040 con 0x560681206780 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6f2626740 10 client.17010 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6f2626740 1 -- 172.21.15.86:0/3374679433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560680f477d0 con 0x560681206780 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.446+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/3374679433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d800cc60 con 0x560681206780 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/3374679433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff6d8024890 con 0x560681206780 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6f2626740 10 client.17010 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6f2626740 10 client.17010 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6f2626740 20 client.17010 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6f2626740 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56068121f220 0x560681221610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6f2626740 4 client.17010 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6f2626740 1 -- 172.21.15.86:0/3374679433 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560681178b00 con 0x56068121f220 2024-04-16T05:39:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56068121f220 0x560681221610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6d0ff9640 10 client.17010 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:39:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.447+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56068121f220 0x560681221610 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc039d23f2784a93 server_cookie=8174a2ca9b06fe5e in_seq=0 out_seq=0 2024-04-16T05:39:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/3374679433 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560681178b00 con 0x56068121f220 2024-04-16T05:39:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6d0ff9640 10 client.17010 handle_command_reply: tid=0 2024-04-16T05:39:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 client.17010 shutdown 2024-04-16T05:39:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 -- 172.21.15.86:0/3374679433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 msgr2=0x56068120c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068120c8b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8008010 tx=0x7ff6d8011be0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 -- 172.21.15.86:0/3374679433 shutdown_connections 2024-04-16T05:39:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x56068121f220 0x560681221610 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56068120f360 0x56068120cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068120c8b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 --2- 172.21.15.86:0/3374679433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x56068120c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6d0ff9640 0 client.17010 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:39:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 -- 172.21.15.86:0/3374679433 >> 172.21.15.86:0/3374679433 conn(0x560681200590 msgr2=0x5606812022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 -- 172.21.15.86:0/3374679433 shutdown_connections 2024-04-16T05:39:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.448+0000 7ff6f2626740 1 -- 172.21.15.86:0/3374679433 wait complete. 2024-04-16T05:39:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.449+0000 7ff6f2626740 20 client.17010 trim_cache size 0 max 16384 2024-04-16T05:39:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.461+0000 7ff6f2626740 1 Processor -- start 2024-04-16T05:39:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6f2626740 1 -- start start 2024-04-16T05:39:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681205e60 0x56068120eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068121b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56068120f360 0x5606811ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6f2626740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b59a0 con 0x560681206780 2024-04-16T05:39:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6f2626740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b74e0 con 0x56068120f360 2024-04-16T05:39:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6f2626740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b6070 con 0x560681205e60 2024-04-16T05:39:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ecef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068121b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ecef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068121b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52738/0 (socket says 172.21.15.86:52738) 2024-04-16T05:39:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ecef4640 1 -- 172.21.15.86:0/2072876863 learned_addr learned my addr 172.21.15.86:0/2072876863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6edef6640 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56068120f360 0x5606811ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681205e60 0x56068120eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ecef4640 1 -- 172.21.15.86:0/2072876863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681205e60 msgr2=0x56068120eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681205e60 0x56068120eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ecef4640 1 -- 172.21.15.86:0/2072876863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56068120f360 msgr2=0x5606811ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56068120f360 0x5606811ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ecef4640 1 -- 172.21.15.86:0/2072876863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606812000c0 con 0x560681206780 2024-04-16T05:39:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681205e60 0x56068120eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068121b070 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8004810 tx=0x7ff6d800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cc77294c181d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6d2ffd640 1 -- 172.21.15.86:0/2072876863 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8020050 con 0x560681206780 2024-04-16T05:39:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6d2ffd640 1 -- 172.21.15.86:0/2072876863 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d8032070 con 0x560681206780 2024-04-16T05:39:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.462+0000 7ff6d2ffd640 1 -- 172.21.15.86:0/2072876863 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8023720 con 0x560681206780 2024-04-16T05:39:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 -- 172.21.15.86:0/2072876863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 msgr2=0x56068121b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068121b070 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8004810 tx=0x7ff6d800ae90 comp rx=0 tx=0).stop 2024-04-16T05:39:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 -- 172.21.15.86:0/2072876863 shutdown_connections 2024-04-16T05:39:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56068120f360 0x5606811ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x56068121b070 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2072876863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560681205e60 0x56068120eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 -- 172.21.15.86:0/2072876863 >> 172.21.15.86:0/2072876863 conn(0x560681200590 msgr2=0x5606811feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 -- 172.21.15.86:0/2072876863 shutdown_connections 2024-04-16T05:39:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 -- 172.21.15.86:0/2072876863 wait complete. 2024-04-16T05:39:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 -- start start 2024-04-16T05:39:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.463+0000 7ff6f2626740 1 Processor -- start 2024-04-16T05:39:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x560681209120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x560681209660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6f2626740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56068120f360 0x560681209ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6f2626740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b6e10 con 0x560681206780 2024-04-16T05:39:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6f2626740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b6740 con 0x560681205e60 2024-04-16T05:39:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6f2626740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b3960 con 0x56068120f360 2024-04-16T05:39:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6f2626740 10 client.0 _ll_register_callbacks cb 0x7ff6d800a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ecef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x560681209660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ecef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x560681209660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52744/0 (socket says 172.21.15.86:52744) 2024-04-16T05:39:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ecef4640 1 -- 172.21.15.86:0/2919010615 learned_addr learned my addr 172.21.15.86:0/2919010615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x560681209120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ecef4640 1 -- 172.21.15.86:0/2919010615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56068120f360 msgr2=0x560681209ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56068120f360 0x560681209ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ecef4640 1 -- 172.21.15.86:0/2919010615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 msgr2=0x560681209120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x560681209120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ecef4640 1 -- 172.21.15.86:0/2919010615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606812845e0 con 0x560681206780 2024-04-16T05:39:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6ecef4640 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x560681209660 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8024dd0 tx=0x7ff6d80235e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dca0b86604f855d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/2919010615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d803c020 con 0x560681206780 2024-04-16T05:39:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/2919010615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff6d8023720 con 0x560681206780 2024-04-16T05:39:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.464+0000 7ff6f2626740 1 -- 172.21.15.86:0/2919010615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d8032070 con 0x560681206780 2024-04-16T05:39:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/2919010615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff6d8047050 con 0x560681206780 2024-04-16T05:39:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6f2626740 10 client.17016 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6f2626740 1 -- 172.21.15.86:0/2919010615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560681285830 con 0x560681206780 2024-04-16T05:39:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/2919010615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d8032070 con 0x560681206780 2024-04-16T05:39:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/2919010615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff6d801a660 con 0x560681206780 2024-04-16T05:39:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6f2626740 10 client.17016 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6f2626740 10 client.17016 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:39:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6f2626740 20 client.17016 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560681285ef0 0x560681288310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6f2626740 4 client.17016 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6f2626740 1 -- 172.21.15.86:0/2919010615 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560681172970 con 0x560681285ef0 2024-04-16T05:39:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.465+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560681285ef0 0x560681288310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6ed6f5640 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560681285ef0 0x560681288310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=674317852e6fcf5 server_cookie=5c53a6477e460c78 in_seq=0 out_seq=0 2024-04-16T05:39:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6d0ff9640 10 client.17016 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:39:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6d0ff9640 1 -- 172.21.15.86:0/2919010615 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560681172970 con 0x560681285ef0 2024-04-16T05:39:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6d0ff9640 10 client.17016 handle_command_reply: tid=0 2024-04-16T05:39:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 client.17016 shutdown 2024-04-16T05:39:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 -- 172.21.15.86:0/2919010615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 msgr2=0x560681209660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x560681209660 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8024dd0 tx=0x7ff6d80235e0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 -- 172.21.15.86:0/2919010615 shutdown_connections 2024-04-16T05:39:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560681285ef0 0x560681288310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56068120f360 0x560681209ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560681206780 0x560681209660 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6d0ff9640 0 client.17016 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:39:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 --2- 172.21.15.86:0/2919010615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560681205e60 0x560681209120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 -- 172.21.15.86:0/2919010615 >> 172.21.15.86:0/2919010615 conn(0x560681200590 msgr2=0x56068120cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 -- 172.21.15.86:0/2919010615 shutdown_connections 2024-04-16T05:39:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 1 -- 172.21.15.86:0/2919010615 wait complete. 2024-04-16T05:39:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.466+0000 7ff6f2626740 20 client.17016 trim_cache size 0 max 16384 2024-04-16T05:39:44.476 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 -- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6d40a3d20 msgr2=0x7ff6d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0005d50 tx=0x7ff6e0005f10 comp rx=0 tx=0).stop 2024-04-16T05:39:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 -- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806abe0 msgr2=0x7ff6e819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806abe0 0x7ff6e819edc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40098d0 tx=0x7ff6e4035b60 comp rx=0 tx=0).stop 2024-04-16T05:39:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 -- 172.21.15.86:0/184889388 shutdown_connections 2024-04-16T05:39:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e8106f80 0x7ff6e81a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e8104a10 0x7ff6e819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 --2- 172.21.15.86:0/184889388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806abe0 0x7ff6e819edc0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 -- 172.21.15.86:0/184889388 >> 172.21.15.86:0/184889388 conn(0x7ff6e8077cf0 msgr2=0x7ff6e8103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 -- 172.21.15.86:0/184889388 shutdown_connections 2024-04-16T05:39:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.467+0000 7ff6ef959640 1 -- 172.21.15.86:0/184889388 wait complete. 2024-04-16T05:39:44.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:44 smithi086 ceph-mon[32028]: pgmap v136: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:44.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:44 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/795642981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:44.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:39:44.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:44 smithi196 ceph-mon[36373]: pgmap v136: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:44.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:44 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/795642981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.587+0000 7fd4f12fb640 1 -- 172.21.15.86:0/4066108646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec102c00 msgr2=0x7fd4ec106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.587+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/4066108646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec102c00 0x7fd4ec106c40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009900 tx=0x7fd4e002bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.587+0000 7fd4f12fb640 1 -- 172.21.15.86:0/4066108646 shutdown_connections 2024-04-16T05:39:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.587+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/4066108646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec107320 0x7fd4ec10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.587+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/4066108646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec102c00 0x7fd4ec106c40 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.587+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/4066108646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ec102250 0x7fd4ec102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.587+0000 7fd4f12fb640 1 -- 172.21.15.86:0/4066108646 >> 172.21.15.86:0/4066108646 conn(0x7fd4ec079490 msgr2=0x7fd4ec0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.587+0000 7fd4f12fb640 1 -- 172.21.15.86:0/4066108646 shutdown_connections 2024-04-16T05:39:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.587+0000 7fd4f12fb640 1 -- 172.21.15.86:0/4066108646 wait complete. 2024-04-16T05:39:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4f12fb640 1 Processor -- start 2024-04-16T05:39:44.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4f12fb640 1 -- start start 2024-04-16T05:39:44.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4f12fb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec102250 0x7fd4ec19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4f12fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec107320 0x7fd4ec19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4f12fb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ec198750 0x7fd4ec198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4f12fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec112430 con 0x7fd4ec107320 2024-04-16T05:39:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4f12fb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1122b0 con 0x7fd4ec102250 2024-04-16T05:39:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4f12fb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1125b0 con 0x7fd4ec198750 2024-04-16T05:39:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4ea7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec107320 0x7fd4ec19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4ea7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec107320 0x7fd4ec19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52758/0 (socket says 172.21.15.86:52758) 2024-04-16T05:39:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4ea7fc640 1 -- 172.21.15.86:0/3164481309 learned_addr learned my addr 172.21.15.86:0/3164481309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.588+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ec198750 0x7fd4ec198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4ea7fc640 1 -- 172.21.15.86:0/3164481309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ec198750 msgr2=0x7fd4ec198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ec198750 0x7fd4ec198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4ea7fc640 1 -- 172.21.15.86:0/3164481309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec102250 msgr2=0x7fd4ec19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4eaffd640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec102250 0x7fd4ec19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec102250 0x7fd4ec19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4ea7fc640 1 -- 172.21.15.86:0/3164481309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec199350 con 0x7fd4ec107320 2024-04-16T05:39:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4eaffd640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec102250 0x7fd4ec19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec107320 0x7fd4ec19eb80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0036040 tx=0x7fd4e002ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4d3fff640 1 -- 172.21.15.86:0/3164481309 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0041070 con 0x7fd4ec107320 2024-04-16T05:39:44.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4d3fff640 1 -- 172.21.15.86:0/3164481309 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e00054e0 con 0x7fd4ec107320 2024-04-16T05:39:44.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4f12fb640 1 -- 172.21.15.86:0/3164481309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec1995e0 con 0x7fd4ec107320 2024-04-16T05:39:44.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4d3fff640 1 -- 172.21.15.86:0/3164481309 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0034440 con 0x7fd4ec107320 2024-04-16T05:39:44.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.589+0000 7fd4f12fb640 1 -- 172.21.15.86:0/3164481309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec1a56d0 con 0x7fd4ec107320 2024-04-16T05:39:44.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.591+0000 7fd4d3fff640 1 -- 172.21.15.86:0/3164481309 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd4e0044070 con 0x7fd4ec107320 2024-04-16T05:39:44.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.591+0000 7fd4f3fbd740 1 Processor -- start 2024-04-16T05:39:44.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.591+0000 7fd4f3fbd740 1 -- start start 2024-04-16T05:39:44.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb774e60 0x555deb775240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb775780 0x555deb77de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb780750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555deb6da0e0 con 0x555deb77e360 2024-04-16T05:39:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555deb677520 con 0x555deb774e60 2024-04-16T05:39:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555deb6d60a0 con 0x555deb775780 2024-04-16T05:39:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4ea7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb780750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4ea7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb780750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52762/0 (socket says 172.21.15.86:52762) 2024-04-16T05:39:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4ea7fc640 1 -- 172.21.15.86:0/3611235643 learned_addr learned my addr 172.21.15.86:0/3611235643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/3611235643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb774e60 0x555deb775240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4d3fff640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4d40a3d20 0x7fd4d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4eaffd640 1 --2- 172.21.15.86:0/3611235643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb775780 0x555deb77de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4d40a3d20 0x7fd4d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4ea7fc640 1 -- 172.21.15.86:0/3611235643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb775780 msgr2=0x555deb77de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/3611235643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb775780 0x555deb77de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4ea7fc640 1 -- 172.21.15.86:0/3611235643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb774e60 msgr2=0x555deb775240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/3611235643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb774e60 0x555deb775240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4ea7fc640 1 -- 172.21.15.86:0/3611235643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555deb69f8d0 con 0x555deb77e360 2024-04-16T05:39:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.592+0000 7fd4d3fff640 1 -- 172.21.15.86:0/3164481309 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd4e00f2730 con 0x7fd4ec107320 2024-04-16T05:39:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.593+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/3611235643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb780750 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00baed0 tx=0x7fd4e004d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fff14d0a788ba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.593+0000 7fd4d17fa640 1 -- 172.21.15.86:0/3611235643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00bde60 con 0x555deb77e360 2024-04-16T05:39:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.593+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4d40a3d20 0x7fd4d40a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009fd0 tx=0x7fd4e4004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.593+0000 7fd4d17fa640 1 -- 172.21.15.86:0/3611235643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e0005690 con 0x555deb77e360 2024-04-16T05:39:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.593+0000 7fd4d17fa640 1 -- 172.21.15.86:0/3611235643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0004c50 con 0x555deb77e360 2024-04-16T05:39:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.593+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/3611235643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 msgr2=0x555deb780750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.593+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/3611235643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb780750 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00baed0 tx=0x7fd4e004d000 comp rx=0 tx=0).stop 2024-04-16T05:39:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/3611235643 shutdown_connections 2024-04-16T05:39:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/3611235643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb780750 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/3611235643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb775780 0x555deb77de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/3611235643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb774e60 0x555deb775240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/3611235643 >> 172.21.15.86:0/3611235643 conn(0x555deb76f590 msgr2=0x555deb76f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/3611235643 shutdown_connections 2024-04-16T05:39:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/3611235643 wait complete. 2024-04-16T05:39:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 -- start start 2024-04-16T05:39:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 Processor -- start 2024-04-16T05:39:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb774e60 0x555deb77b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb77b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb77bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555deb780f10 con 0x555deb77e360 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555deb6934d0 con 0x555deb775780 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555deb781090 con 0x555deb774e60 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4f3fbd740 10 client.0 _ll_register_callbacks cb 0x555deb6782c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb77bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb77bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52768/0 (socket says 172.21.15.86:52768) 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4eb7fe640 1 -- 172.21.15.86:0/610716482 learned_addr learned my addr 172.21.15.86:0/610716482 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4eaffd640 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb774e60 0x555deb77b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.594+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb77b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4eb7fe640 1 -- 172.21.15.86:0/610716482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb774e60 msgr2=0x555deb77b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb774e60 0x555deb77b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4eb7fe640 1 -- 172.21.15.86:0/610716482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 msgr2=0x555deb77b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb77b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4eb7fe640 1 -- 172.21.15.86:0/610716482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555deb3ec3f0 con 0x555deb77e360 2024-04-16T05:39:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb77bdf0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400f040 tx=0x7fd4e4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d6bd0e9ed47b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4ceff5640 1 -- 172.21.15.86:0/610716482 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4020070 con 0x555deb77e360 2024-04-16T05:39:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4ceff5640 1 -- 172.21.15.86:0/610716482 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd4e4025660 con 0x555deb77e360 2024-04-16T05:39:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/610716482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555deb6e7b00 con 0x555deb77e360 2024-04-16T05:39:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4ceff5640 1 -- 172.21.15.86:0/610716482 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd4e402e050 con 0x555deb77e360 2024-04-16T05:39:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4f3fbd740 10 client.17028 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/610716482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555deb4b67d0 con 0x555deb77e360 2024-04-16T05:39:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4ceff5640 1 -- 172.21.15.86:0/610716482 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555deb6e7b00 con 0x555deb77e360 2024-04-16T05:39:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4ceff5640 1 -- 172.21.15.86:0/610716482 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd4e401a390 con 0x555deb77e360 2024-04-16T05:39:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4f3fbd740 10 client.17028 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.595+0000 7fd4f3fbd740 10 client.17028 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:39:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.596+0000 7fd4f3fbd740 20 client.17028 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.596+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x555deb78e040 0x555deb790450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.596+0000 7fd4f3fbd740 4 client.17028 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.596+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/610716482 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555deb6e1970 con 0x555deb78e040 2024-04-16T05:39:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.596+0000 7fd4eaffd640 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x555deb78e040 0x555deb790450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.596+0000 7fd4ceff5640 10 client.17028 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:39:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.596+0000 7fd4eaffd640 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x555deb78e040 0x555deb790450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d2cfcf0d472cd05 server_cookie=33457c02608bd81d in_seq=0 out_seq=0 2024-04-16T05:39:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4ceff5640 1 -- 172.21.15.86:0/610716482 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555deb6e1970 con 0x555deb78e040 2024-04-16T05:39:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4ceff5640 10 client.17028 handle_command_reply: tid=0 2024-04-16T05:39:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 client.17028 shutdown 2024-04-16T05:39:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/610716482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 msgr2=0x555deb77bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb77bdf0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400f040 tx=0x7fd4e4004b50 comp rx=0 tx=0).stop 2024-04-16T05:39:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/610716482 shutdown_connections 2024-04-16T05:39:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x555deb78e040 0x555deb790450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb77bdf0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb77b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4ceff5640 0 client.17028 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:39:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/610716482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb774e60 0x555deb77b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/610716482 >> 172.21.15.86:0/610716482 conn(0x555deb76f590 msgr2=0x555deb7712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/610716482 shutdown_connections 2024-04-16T05:39:44.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/610716482 wait complete. 2024-04-16T05:39:44.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.597+0000 7fd4f3fbd740 20 client.17028 trim_cache size 0 max 16384 2024-04-16T05:39:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4f3fbd740 1 Processor -- start 2024-04-16T05:39:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4f3fbd740 1 -- start start 2024-04-16T05:39:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb774e60 0x555deb77daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb789d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb76a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1af4a0 con 0x555deb77e360 2024-04-16T05:39:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1b0fe0 con 0x555deb775780 2024-04-16T05:39:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1afb70 con 0x555deb774e60 2024-04-16T05:39:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb76a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb76a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52778/0 (socket says 172.21.15.86:52778) 2024-04-16T05:39:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.610+0000 7fd4eb7fe640 1 -- 172.21.15.86:0/801871691 learned_addr learned my addr 172.21.15.86:0/801871691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.611+0000 7fd4eb7fe640 1 -- 172.21.15.86:0/801871691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb774e60 msgr2=0x555deb77daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.611+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/801871691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb774e60 0x555deb77daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.611+0000 7fd4eb7fe640 1 -- 172.21.15.86:0/801871691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 msgr2=0x555deb789d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.611+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/801871691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb789d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.611+0000 7fd4eb7fe640 1 -- 172.21.15.86:0/801871691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555deb77a320 con 0x555deb77e360 2024-04-16T05:39:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.611+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/801871691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb76a5c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4000f80 tx=0x7fd4e4009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af61e997cce24a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.611+0000 7fd4d0ff9640 1 -- 172.21.15.86:0/801871691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4020040 con 0x555deb77e360 2024-04-16T05:39:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.611+0000 7fd4d0ff9640 1 -- 172.21.15.86:0/801871691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e4037070 con 0x555deb77e360 2024-04-16T05:39:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.611+0000 7fd4d0ff9640 1 -- 172.21.15.86:0/801871691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4015050 con 0x555deb77e360 2024-04-16T05:39:44.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/801871691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 msgr2=0x555deb76a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/801871691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb76a5c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4000f80 tx=0x7fd4e4009cb0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/801871691 shutdown_connections 2024-04-16T05:39:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/801871691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb77e360 0x555deb76a5c0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/801871691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb789d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/801871691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb774e60 0x555deb77daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/801871691 >> 172.21.15.86:0/801871691 conn(0x555deb76f590 msgr2=0x555deb76e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/801871691 shutdown_connections 2024-04-16T05:39:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/801871691 wait complete. 2024-04-16T05:39:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 -- start start 2024-04-16T05:39:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 Processor -- start 2024-04-16T05:39:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb774e60 0x555deb786440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb777be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb77e360 0x555deb778120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1b0910 con 0x555deb774e60 2024-04-16T05:39:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1b0240 con 0x555deb775780 2024-04-16T05:39:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1ad460 con 0x555deb77e360 2024-04-16T05:39:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4f3fbd740 10 client.0 _ll_register_callbacks cb 0x7fd4e400c500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb774e60 0x555deb786440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb774e60 0x555deb786440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52792/0 (socket says 172.21.15.86:52792) 2024-04-16T05:39:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.612+0000 7fd4eaffd640 1 -- 172.21.15.86:0/4039584045 learned_addr learned my addr 172.21.15.86:0/4039584045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4eaffd640 1 -- 172.21.15.86:0/4039584045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb77e360 msgr2=0x555deb778120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb77e360 0x555deb778120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4eaffd640 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb77e360 0x555deb778120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb777be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4eaffd640 1 -- 172.21.15.86:0/4039584045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 msgr2=0x555deb777be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4eaffd640 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb777be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4eaffd640 1 -- 172.21.15.86:0/4039584045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555deb778760 con 0x555deb774e60 2024-04-16T05:39:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4eb7fe640 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb77e360 0x555deb778120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb777be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4eaffd640 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb774e60 0x555deb786440 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005fd0 tx=0x7fd4dc012980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25323ac17496a9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4ceff5640 1 -- 172.21.15.86:0/4039584045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc01f070 con 0x555deb774e60 2024-04-16T05:39:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4ceff5640 1 -- 172.21.15.86:0/4039584045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd4dc01b4c0 con 0x555deb774e60 2024-04-16T05:39:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/4039584045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e4037070 con 0x555deb774e60 2024-04-16T05:39:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4ceff5640 1 -- 172.21.15.86:0/4039584045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd4dc02b050 con 0x555deb774e60 2024-04-16T05:39:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4f3fbd740 10 client.17034 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/4039584045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555deb7f4420 con 0x555deb774e60 2024-04-16T05:39:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4ceff5640 1 -- 172.21.15.86:0/4039584045 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e4037070 con 0x555deb774e60 2024-04-16T05:39:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4ceff5640 1 -- 172.21.15.86:0/4039584045 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd4dc00f040 con 0x555deb774e60 2024-04-16T05:39:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4f3fbd740 10 client.17034 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4f3fbd740 10 client.17034 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:39:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4f3fbd740 20 client.17034 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x555deb7f4b50 0x555deb7f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4f3fbd740 4 client.17034 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.613+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/4039584045 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555deb65d6b0 con 0x555deb7f4b50 2024-04-16T05:39:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x555deb7f4b50 0x555deb7f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4ea7fc640 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x555deb7f4b50 0x555deb7f6f70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31f60b5858ee47a1 server_cookie=c276a887ee8d65a0 in_seq=0 out_seq=0 2024-04-16T05:39:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4ceff5640 10 client.17034 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:39:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4ceff5640 1 -- 172.21.15.86:0/4039584045 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555deb65d6b0 con 0x555deb7f4b50 2024-04-16T05:39:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4ceff5640 10 client.17034 handle_command_reply: tid=0 2024-04-16T05:39:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4f3fbd740 1 client.17034 shutdown 2024-04-16T05:39:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/4039584045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb774e60 msgr2=0x555deb786440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb774e60 0x555deb786440 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005fd0 tx=0x7fd4dc012980 comp rx=0 tx=0).stop 2024-04-16T05:39:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/4039584045 shutdown_connections 2024-04-16T05:39:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555deb77e360 0x555deb778120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555deb775780 0x555deb777be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x555deb7f4b50 0x555deb7f6f70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4f3fbd740 1 --2- 172.21.15.86:0/4039584045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555deb774e60 0x555deb786440 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4ceff5640 0 client.17034 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:39:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.614+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/4039584045 >> 172.21.15.86:0/4039584045 conn(0x555deb76f590 msgr2=0x555deb76dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/4039584045 shutdown_connections 2024-04-16T05:39:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f3fbd740 1 -- 172.21.15.86:0/4039584045 wait complete. 2024-04-16T05:39:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f3fbd740 20 client.17034 trim_cache size 0 max 16384 2024-04-16T05:39:44.624 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 -- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4d40a3d20 msgr2=0x7fd4d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4d40a3d20 0x7fd4d40a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009fd0 tx=0x7fd4e4004a40 comp rx=0 tx=0).stop 2024-04-16T05:39:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 -- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec107320 msgr2=0x7fd4ec19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec107320 0x7fd4ec19eb80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0036040 tx=0x7fd4e002ce20 comp rx=0 tx=0).stop 2024-04-16T05:39:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 -- 172.21.15.86:0/3164481309 shutdown_connections 2024-04-16T05:39:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ec198750 0x7fd4ec198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec107320 0x7fd4ec19eb80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd4d40a3d20 0x7fd4d40a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 --2- 172.21.15.86:0/3164481309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec102250 0x7fd4ec19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 -- 172.21.15.86:0/3164481309 >> 172.21.15.86:0/3164481309 conn(0x7fd4ec079490 msgr2=0x7fd4ec0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 -- 172.21.15.86:0/3164481309 shutdown_connections 2024-04-16T05:39:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.615+0000 7fd4f12fb640 1 -- 172.21.15.86:0/3164481309 wait complete. 2024-04-16T05:39:44.627 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:39:44.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:44 smithi194 ceph-mon[35865]: pgmap v136: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:39:44.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:44 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/795642981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:44.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.735+0000 7fcd34c50640 1 -- 172.21.15.86:0/1143081434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 msgr2=0x7fcd30101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.735+0000 7fcd34c50640 1 --2- 172.21.15.86:0/1143081434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 0x7fcd30101540 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009900 tx=0x7fcd2002bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:44.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.735+0000 7fcd34c50640 1 -- 172.21.15.86:0/1143081434 shutdown_connections 2024-04-16T05:39:44.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.735+0000 7fcd34c50640 1 --2- 172.21.15.86:0/1143081434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd30106230 0x7fcd3010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.735+0000 7fcd34c50640 1 --2- 172.21.15.86:0/1143081434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd30101b10 0x7fcd30105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.735+0000 7fcd34c50640 1 --2- 172.21.15.86:0/1143081434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 0x7fcd30101540 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.735+0000 7fcd34c50640 1 -- 172.21.15.86:0/1143081434 >> 172.21.15.86:0/1143081434 conn(0x7fcd30077280 msgr2=0x7fcd30077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.735+0000 7fcd34c50640 1 -- 172.21.15.86:0/1143081434 shutdown_connections 2024-04-16T05:39:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.735+0000 7fcd34c50640 1 -- 172.21.15.86:0/1143081434 wait complete. 2024-04-16T05:39:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd34c50640 1 Processor -- start 2024-04-16T05:39:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd34c50640 1 -- start start 2024-04-16T05:39:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd34c50640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 0x7fcd3019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd34c50640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd30106230 0x7fcd3019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd34c50640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd301987e0 0x7fcd30198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd34c50640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd301113c0 con 0x7fcd30101160 2024-04-16T05:39:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd34c50640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30111240 con 0x7fcd301987e0 2024-04-16T05:39:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd34c50640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30111540 con 0x7fcd30106230 2024-04-16T05:39:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd2e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 0x7fcd3019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd2e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 0x7fcd3019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52808/0 (socket says 172.21.15.86:52808) 2024-04-16T05:39:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.736+0000 7fcd2e59c640 1 -- 172.21.15.86:0/3075803712 learned_addr learned my addr 172.21.15.86:0/3075803712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd30106230 0x7fcd3019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd2e59c640 1 -- 172.21.15.86:0/3075803712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd30106230 msgr2=0x7fcd3019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd30106230 0x7fcd3019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd301987e0 0x7fcd30198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd2e59c640 1 -- 172.21.15.86:0/3075803712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd301987e0 msgr2=0x7fcd30198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd301987e0 0x7fcd30198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd2e59c640 1 -- 172.21.15.86:0/3075803712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd30199370 con 0x7fcd30101160 2024-04-16T05:39:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd30106230 0x7fcd3019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 0x7fcd3019e670 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fcd20006010 tx=0x7fcd2002cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd1f7fe640 1 -- 172.21.15.86:0/3075803712 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20041070 con 0x7fcd30101160 2024-04-16T05:39:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd1f7fe640 1 -- 172.21.15.86:0/3075803712 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd200054e0 con 0x7fcd30101160 2024-04-16T05:39:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd34c50640 1 -- 172.21.15.86:0/3075803712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd30199600 con 0x7fcd30101160 2024-04-16T05:39:44.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd1f7fe640 1 -- 172.21.15.86:0/3075803712 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20034440 con 0x7fcd30101160 2024-04-16T05:39:44.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.737+0000 7fcd34c50640 1 -- 172.21.15.86:0/3075803712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd301a57b0 con 0x7fcd30101160 2024-04-16T05:39:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.739+0000 7fcd1f7fe640 1 -- 172.21.15.86:0/3075803712 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd20044070 con 0x7fcd30101160 2024-04-16T05:39:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.739+0000 7fcd37830740 1 Processor -- start 2024-04-16T05:39:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd37830740 1 -- start start 2024-04-16T05:39:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd37830740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd57f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd37830740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd584b0 0x559d8cd65230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd37830740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd67b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd37830740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d8ccbd0e0 con 0x559d8cd65770 2024-04-16T05:39:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd37830740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d8cc5a520 con 0x559d8cd584b0 2024-04-16T05:39:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd37830740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559d8ccb90a0 con 0x559d8cd57b90 2024-04-16T05:39:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd1f7fe640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd180a3d20 0x7fcd180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd67b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd67b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52814/0 (socket says 172.21.15.86:52814) 2024-04-16T05:39:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/1107779729 learned_addr learned my addr 172.21.15.86:0/1107779729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd180a3d20 0x7fcd180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd1f7fe640 1 -- 172.21.15.86:0/3075803712 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcd200f27c0 con 0x7fcd30101160 2024-04-16T05:39:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd584b0 0x559d8cd65230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd57f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/1107779729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 msgr2=0x559d8cd57f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd57f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/1107779729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd584b0 msgr2=0x559d8cd65230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd584b0 0x559d8cd65230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/1107779729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d8cc828d0 con 0x559d8cd65770 2024-04-16T05:39:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.740+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd584b0 0x559d8cd65230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.741+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd180a3d20 0x7fcd180a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcd28014390 tx=0x7fcd28014790 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.741+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd67b60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400a970 tx=0x7fcd2400ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4e5b5d736800a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.741+0000 7fcd1cff9640 1 -- 172.21.15.86:0/1107779729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24015070 con 0x559d8cd65770 2024-04-16T05:39:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.741+0000 7fcd1cff9640 1 -- 172.21.15.86:0/1107779729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2400cc00 con 0x559d8cd65770 2024-04-16T05:39:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.741+0000 7fcd1cff9640 1 -- 172.21.15.86:0/1107779729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd240185c0 con 0x559d8cd65770 2024-04-16T05:39:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.741+0000 7fcd37830740 1 -- 172.21.15.86:0/1107779729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 msgr2=0x559d8cd67b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.741+0000 7fcd37830740 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd67b60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400a970 tx=0x7fcd2400ae30 comp rx=0 tx=0).stop 2024-04-16T05:39:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 -- 172.21.15.86:0/1107779729 shutdown_connections 2024-04-16T05:39:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd67b60 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd584b0 0x559d8cd65230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 --2- 172.21.15.86:0/1107779729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd57f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 -- 172.21.15.86:0/1107779729 >> 172.21.15.86:0/1107779729 conn(0x559d8cd52590 msgr2=0x559d8cd52970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 -- 172.21.15.86:0/1107779729 shutdown_connections 2024-04-16T05:39:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 -- 172.21.15.86:0/1107779729 wait complete. 2024-04-16T05:39:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 -- start start 2024-04-16T05:39:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 Processor -- start 2024-04-16T05:39:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd62c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 0x559d8cd631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d8cd68530 con 0x559d8cd584b0 2024-04-16T05:39:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d8cc764d0 con 0x559d8cd65770 2024-04-16T05:39:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559d8cd686b0 con 0x559d8cd57b90 2024-04-16T05:39:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd37830740 10 client.0 _ll_register_callbacks cb 0x559d8cc5b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd62c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd62c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52826/0 (socket says 172.21.15.86:52826) 2024-04-16T05:39:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.742+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/905787479 learned_addr learned my addr 172.21.15.86:0/905787479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/905787479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 msgr2=0x559d8cd62720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd62720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 0x559d8cd631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/905787479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 msgr2=0x559d8cd631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 0x559d8cd631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/905787479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d8c9cf3f0 con 0x559d8cd584b0 2024-04-16T05:39:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd62720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 0x559d8cd631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd62c60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400a120 tx=0x7fcd2400a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd12745b98e48c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd15ffb640 1 -- 172.21.15.86:0/905787479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24020070 con 0x559d8cd584b0 2024-04-16T05:39:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd15ffb640 1 -- 172.21.15.86:0/905787479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcd2401a450 con 0x559d8cd584b0 2024-04-16T05:39:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd37830740 1 -- 172.21.15.86:0/905787479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd2400cc00 con 0x559d8cd584b0 2024-04-16T05:39:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd15ffb640 1 -- 172.21.15.86:0/905787479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd24015040 con 0x559d8cd584b0 2024-04-16T05:39:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd37830740 10 client.17046 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd37830740 1 -- 172.21.15.86:0/905787479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d8ca997d0 con 0x559d8cd584b0 2024-04-16T05:39:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd15ffb640 1 -- 172.21.15.86:0/905787479 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2400cc00 con 0x559d8cd584b0 2024-04-16T05:39:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.743+0000 7fcd15ffb640 1 -- 172.21.15.86:0/905787479 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd24023720 con 0x559d8cd584b0 2024-04-16T05:39:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.744+0000 7fcd37830740 10 client.17046 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.744+0000 7fcd37830740 10 client.17046 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:39:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.744+0000 7fcd37830740 20 client.17046 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.744+0000 7fcd37830740 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559d8cd75550 0x559d8cd77940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.744+0000 7fcd37830740 4 client.17046 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:44.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.744+0000 7fcd37830740 1 -- 172.21.15.86:0/905787479 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d8cccab00 con 0x559d8cd75550 2024-04-16T05:39:44.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.744+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559d8cd75550 0x559d8cd77940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.744+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559d8cd75550 0x559d8cd77940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df8b8f0008880d70 server_cookie=36a2c294949ef7c4 in_seq=0 out_seq=0 2024-04-16T05:39:44.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.744+0000 7fcd15ffb640 10 client.17046 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:39:44.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd15ffb640 1 -- 172.21.15.86:0/905787479 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d8cccab00 con 0x559d8cd75550 2024-04-16T05:39:44.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd15ffb640 10 client.17046 handle_command_reply: tid=0 2024-04-16T05:39:44.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 client.17046 shutdown 2024-04-16T05:39:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 -- 172.21.15.86:0/905787479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 msgr2=0x559d8cd62c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd62c60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400a120 tx=0x7fcd2400a230 comp rx=0 tx=0).stop 2024-04-16T05:39:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 -- 172.21.15.86:0/905787479 shutdown_connections 2024-04-16T05:39:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559d8cd75550 0x559d8cd77940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 0x559d8cd631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd15ffb640 0 client.17046 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:39:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd62c60 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 --2- 172.21.15.86:0/905787479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd62720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 -- 172.21.15.86:0/905787479 >> 172.21.15.86:0/905787479 conn(0x559d8cd52590 msgr2=0x559d8cd542e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 -- 172.21.15.86:0/905787479 shutdown_connections 2024-04-16T05:39:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 1 -- 172.21.15.86:0/905787479 wait complete. 2024-04-16T05:39:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.745+0000 7fcd37830740 20 client.17046 trim_cache size 0 max 16384 2024-04-16T05:39:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.758+0000 7fcd37830740 1 Processor -- start 2024-04-16T05:39:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd37830740 1 -- start start 2024-04-16T05:39:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd37830740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd57b90 0x559d8cd64e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd37830740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd584b0 0x559d8cd71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd37830740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd7cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd37830740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd301ad250 con 0x559d8cd65770 2024-04-16T05:39:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd37830740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd301aed90 con 0x559d8cd57b90 2024-04-16T05:39:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd37830740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd301ad920 con 0x559d8cd584b0 2024-04-16T05:39:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd7cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd7cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52830/0 (socket says 172.21.15.86:52830) 2024-04-16T05:39:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2ed9d640 1 -- 172.21.15.86:0/1236826162 learned_addr learned my addr 172.21.15.86:0/1236826162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/1236826162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd57b90 0x559d8cd64e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2ed9d640 1 -- 172.21.15.86:0/1236826162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd584b0 msgr2=0x559d8cd71430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/1236826162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd584b0 0x559d8cd71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2ed9d640 1 -- 172.21.15.86:0/1236826162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd57b90 msgr2=0x559d8cd64e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/1236826162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd57b90 0x559d8cd64e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2ed9d640 1 -- 172.21.15.86:0/1236826162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d8cd523c0 con 0x559d8cd65770 2024-04-16T05:39:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/1236826162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd7cfa0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcd28016520 tx=0x7fcd28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4da0c719b691821 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd17fff640 1 -- 172.21.15.86:0/1236826162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28028070 con 0x559d8cd65770 2024-04-16T05:39:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd17fff640 1 -- 172.21.15.86:0/1236826162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd28023470 con 0x559d8cd65770 2024-04-16T05:39:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.759+0000 7fcd17fff640 1 -- 172.21.15.86:0/1236826162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd280215d0 con 0x559d8cd65770 2024-04-16T05:39:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 -- 172.21.15.86:0/1236826162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 msgr2=0x559d8cd7cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 --2- 172.21.15.86:0/1236826162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd7cfa0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcd28016520 tx=0x7fcd28004a40 comp rx=0 tx=0).stop 2024-04-16T05:39:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 -- 172.21.15.86:0/1236826162 shutdown_connections 2024-04-16T05:39:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 --2- 172.21.15.86:0/1236826162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd65770 0x559d8cd7cfa0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 --2- 172.21.15.86:0/1236826162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd584b0 0x559d8cd71430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 --2- 172.21.15.86:0/1236826162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd57b90 0x559d8cd64e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 -- 172.21.15.86:0/1236826162 >> 172.21.15.86:0/1236826162 conn(0x559d8cd52590 msgr2=0x559d8cd4d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 -- 172.21.15.86:0/1236826162 shutdown_connections 2024-04-16T05:39:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 -- 172.21.15.86:0/1236826162 wait complete. 2024-04-16T05:39:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 -- start start 2024-04-16T05:39:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.760+0000 7fcd37830740 1 Processor -- start 2024-04-16T05:39:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd6de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd5f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 0x559d8cd5f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd301ae6c0 con 0x559d8cd584b0 2024-04-16T05:39:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd301adff0 con 0x559d8cd65770 2024-04-16T05:39:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd301ab210 con 0x559d8cd57b90 2024-04-16T05:39:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 10 client.0 _ll_register_callbacks cb 0x7fcd2400a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd5f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd5f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52844/0 (socket says 172.21.15.86:52844) 2024-04-16T05:39:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/222531771 learned_addr learned my addr 172.21.15.86:0/222531771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2ed9d640 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 0x559d8cd5f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd6de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/222531771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 msgr2=0x559d8cd6de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd6de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/222531771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 msgr2=0x559d8cd5f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 0x559d8cd5f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2dd9b640 1 -- 172.21.15.86:0/222531771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d8cdda810 con 0x559d8cd584b0 2024-04-16T05:39:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd2dd9b640 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd5f210 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcd24006cd0 tx=0x7fcd240117f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9388a0798fd1b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd15ffb640 1 -- 172.21.15.86:0/222531771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24020040 con 0x559d8cd584b0 2024-04-16T05:39:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 1 -- 172.21.15.86:0/222531771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd28023470 con 0x559d8cd584b0 2024-04-16T05:39:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd15ffb640 1 -- 172.21.15.86:0/222531771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcd240233b0 con 0x559d8cd584b0 2024-04-16T05:39:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd15ffb640 1 -- 172.21.15.86:0/222531771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd24038050 con 0x559d8cd584b0 2024-04-16T05:39:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 10 client.17052 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.761+0000 7fcd37830740 1 -- 172.21.15.86:0/222531771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d8cddb930 con 0x559d8cd584b0 2024-04-16T05:39:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd15ffb640 1 -- 172.21.15.86:0/222531771 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd28023470 con 0x559d8cd584b0 2024-04-16T05:39:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd15ffb640 1 -- 172.21.15.86:0/222531771 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd2401aaa0 con 0x559d8cd584b0 2024-04-16T05:39:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd37830740 10 client.17052 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd37830740 10 client.17052 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:39:44.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd37830740 20 client.17052 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd37830740 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559d8cddc0b0 0x559d8cdde4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd37830740 4 client.17052 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:44.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd37830740 1 -- 172.21.15.86:0/222531771 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d8ccc4970 con 0x559d8cddc0b0 2024-04-16T05:39:44.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559d8cddc0b0 0x559d8cdde4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd2e59c640 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559d8cddc0b0 0x559d8cdde4d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=773be0509b19cb51 server_cookie=7cda5b1fd3c17f90 in_seq=0 out_seq=0 2024-04-16T05:39:44.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.762+0000 7fcd15ffb640 10 client.17052 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:39:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd15ffb640 1 -- 172.21.15.86:0/222531771 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d8ccc4970 con 0x559d8cddc0b0 2024-04-16T05:39:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd15ffb640 10 client.17052 handle_command_reply: tid=0 2024-04-16T05:39:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 client.17052 shutdown 2024-04-16T05:39:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 -- 172.21.15.86:0/222531771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 msgr2=0x559d8cd5f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd5f210 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcd24006cd0 tx=0x7fcd240117f0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 -- 172.21.15.86:0/222531771 shutdown_connections 2024-04-16T05:39:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559d8cddc0b0 0x559d8cdde4d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d8cd65770 0x559d8cd5f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd15ffb640 0 client.17052 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:39:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d8cd584b0 0x559d8cd5f210 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 --2- 172.21.15.86:0/222531771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d8cd57b90 0x559d8cd6de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 -- 172.21.15.86:0/222531771 >> 172.21.15.86:0/222531771 conn(0x559d8cd52590 msgr2=0x559d8cd50ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 -- 172.21.15.86:0/222531771 shutdown_connections 2024-04-16T05:39:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 1 -- 172.21.15.86:0/222531771 wait complete. 2024-04-16T05:39:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd37830740 20 client.17052 trim_cache size 0 max 16384 2024-04-16T05:39:44.776 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 -- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd180a3d20 msgr2=0x7fcd180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd180a3d20 0x7fcd180a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcd28014390 tx=0x7fcd28014790 comp rx=0 tx=0).stop 2024-04-16T05:39:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 -- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 msgr2=0x7fcd3019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 0x7fcd3019e670 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fcd20006010 tx=0x7fcd2002cd70 comp rx=0 tx=0).stop 2024-04-16T05:39:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 -- 172.21.15.86:0/3075803712 shutdown_connections 2024-04-16T05:39:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd180a3d20 0x7fcd180a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd301987e0 0x7fcd30198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd30106230 0x7fcd3019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 --2- 172.21.15.86:0/3075803712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd30101160 0x7fcd3019e670 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.763+0000 7fcd34c50640 1 -- 172.21.15.86:0/3075803712 >> 172.21.15.86:0/3075803712 conn(0x7fcd30077280 msgr2=0x7fcd3006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.764+0000 7fcd34c50640 1 -- 172.21.15.86:0/3075803712 shutdown_connections 2024-04-16T05:39:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.764+0000 7fcd34c50640 1 -- 172.21.15.86:0/3075803712 wait complete. 2024-04-16T05:39:44.780 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:39:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.887+0000 7f8989288640 1 -- 172.21.15.86:0/3355594144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984102c00 msgr2=0x7f8984106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.887+0000 7f8989288640 1 --2- 172.21.15.86:0/3355594144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984102c00 0x7f8984106c40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f896c009900 tx=0x7f896c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.887+0000 7f8989288640 1 -- 172.21.15.86:0/3355594144 shutdown_connections 2024-04-16T05:39:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.887+0000 7f8989288640 1 --2- 172.21.15.86:0/3355594144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8984107320 0x7f898410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.887+0000 7f8989288640 1 --2- 172.21.15.86:0/3355594144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984102c00 0x7f8984106c40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.887+0000 7f8989288640 1 --2- 172.21.15.86:0/3355594144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8984102250 0x7f8984102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.887+0000 7f8989288640 1 -- 172.21.15.86:0/3355594144 >> 172.21.15.86:0/3355594144 conn(0x7f8984079490 msgr2=0x7f89840798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.887+0000 7f8989288640 1 -- 172.21.15.86:0/3355594144 shutdown_connections 2024-04-16T05:39:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.887+0000 7f8989288640 1 -- 172.21.15.86:0/3355594144 wait complete. 2024-04-16T05:39:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f8989288640 1 Processor -- start 2024-04-16T05:39:44.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f8989288640 1 -- start start 2024-04-16T05:39:44.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f8989288640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8984102250 0x7f898419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f8989288640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984107320 0x7f898419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f8989288640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8984198670 0x7f8984198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f8989288640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8984112390 con 0x7f8984107320 2024-04-16T05:39:44.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f8989288640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8984112210 con 0x7f8984198670 2024-04-16T05:39:44.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f8989288640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8984112510 con 0x7f8984102250 2024-04-16T05:39:44.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f897a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984107320 0x7f898419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f897a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984107320 0x7f898419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52850/0 (socket says 172.21.15.86:52850) 2024-04-16T05:39:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f897a7fc640 1 -- 172.21.15.86:0/3863870245 learned_addr learned my addr 172.21.15.86:0/3863870245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f8982ffd640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8984102250 0x7f898419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.888+0000 7f89837fe640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8984198670 0x7f8984198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f897a7fc640 1 -- 172.21.15.86:0/3863870245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8984102250 msgr2=0x7f898419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f897a7fc640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8984102250 0x7f898419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f897a7fc640 1 -- 172.21.15.86:0/3863870245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8984198670 msgr2=0x7f8984198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f897a7fc640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8984198670 0x7f8984198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f897a7fc640 1 -- 172.21.15.86:0/3863870245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8984199270 con 0x7f8984107320 2024-04-16T05:39:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f8982ffd640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8984102250 0x7f898419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f897a7fc640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984107320 0x7f898419eaf0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f896c036040 tx=0x7f896c02ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f8980ff9640 1 -- 172.21.15.86:0/3863870245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c041070 con 0x7f8984107320 2024-04-16T05:39:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f8980ff9640 1 -- 172.21.15.86:0/3863870245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f896c0054e0 con 0x7f8984107320 2024-04-16T05:39:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f8989288640 1 -- 172.21.15.86:0/3863870245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8984199500 con 0x7f8984107320 2024-04-16T05:39:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f8980ff9640 1 -- 172.21.15.86:0/3863870245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c034440 con 0x7f8984107320 2024-04-16T05:39:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.889+0000 7f8989288640 1 -- 172.21.15.86:0/3863870245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89841a5640 con 0x7f8984107320 2024-04-16T05:39:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.891+0000 7f8980ff9640 1 -- 172.21.15.86:0/3863870245 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f896c044070 con 0x7f8984107320 2024-04-16T05:39:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.891+0000 7f898be7b740 1 Processor -- start 2024-04-16T05:39:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.891+0000 7f898be7b740 1 -- start start 2024-04-16T05:39:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f898be7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d8e60 0x5622201d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f898be7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201e1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f898be7b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 0x5622201e4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f898be7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56222013e0e0 con 0x5622201d9780 2024-04-16T05:39:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f898be7b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5622200db520 con 0x5622201e2360 2024-04-16T05:39:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f898be7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56222013a0a0 con 0x5622201d8e60 2024-04-16T05:39:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8982ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201e1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8982ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201e1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52862/0 (socket says 172.21.15.86:52862) 2024-04-16T05:39:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8982ffd640 1 -- 172.21.15.86:0/2264743543 learned_addr learned my addr 172.21.15.86:0/2264743543 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8980ff9640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89700a3d20 0x7f89700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f897a7fc640 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 0x5622201e4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f89837fe640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89700a3d20 0x7f89700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8982ffd640 1 -- 172.21.15.86:0/2264743543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d8e60 msgr2=0x5622201d9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f89837fe640 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d8e60 0x5622201d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8982ffd640 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d8e60 0x5622201d9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8982ffd640 1 -- 172.21.15.86:0/2264743543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 msgr2=0x5622201e4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8982ffd640 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 0x5622201e4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8982ffd640 1 -- 172.21.15.86:0/2264743543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622201038d0 con 0x5622201d9780 2024-04-16T05:39:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8980ff9640 1 -- 172.21.15.86:0/3863870245 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f896c0f2730 con 0x7f8984107320 2024-04-16T05:39:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f89837fe640 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d8e60 0x5622201d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.892+0000 7f8982ffd640 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201e1e20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8974008c60 tx=0x7f897400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a93d90df6e083e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.893+0000 7f89837fe640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89700a3d20 0x7f89700a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f897c005c40 tx=0x7f897c009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.893+0000 7f89797fa640 1 -- 172.21.15.86:0/2264743543 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974014070 con 0x5622201d9780 2024-04-16T05:39:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.893+0000 7f89797fa640 1 -- 172.21.15.86:0/2264743543 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89740047a0 con 0x5622201d9780 2024-04-16T05:39:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.893+0000 7f89797fa640 1 -- 172.21.15.86:0/2264743543 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974010740 con 0x5622201d9780 2024-04-16T05:39:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.893+0000 7f898be7b740 1 -- 172.21.15.86:0/2264743543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 msgr2=0x5622201e1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.893+0000 7f898be7b740 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201e1e20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8974008c60 tx=0x7f897400e720 comp rx=0 tx=0).stop 2024-04-16T05:39:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.893+0000 7f898be7b740 1 -- 172.21.15.86:0/2264743543 shutdown_connections 2024-04-16T05:39:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.893+0000 7f898be7b740 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 0x5622201e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.893+0000 7f898be7b740 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201e1e20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 --2- 172.21.15.86:0/2264743543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d8e60 0x5622201d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 -- 172.21.15.86:0/2264743543 >> 172.21.15.86:0/2264743543 conn(0x5622201d3590 msgr2=0x5622201d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 -- 172.21.15.86:0/2264743543 shutdown_connections 2024-04-16T05:39:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 -- 172.21.15.86:0/2264743543 wait complete. 2024-04-16T05:39:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 -- start start 2024-04-16T05:39:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 Processor -- start 2024-04-16T05:39:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d8e60 0x5622201df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 0x5622201dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622201e5000 con 0x5622201d9780 2024-04-16T05:39:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5622200f74d0 con 0x5622201d8e60 2024-04-16T05:39:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622201e5180 con 0x5622201e2360 2024-04-16T05:39:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f898be7b740 10 client.0 _ll_register_callbacks cb 0x5622200dc2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f897a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f897a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201df7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52864/0 (socket says 172.21.15.86:52864) 2024-04-16T05:39:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f897a7fc640 1 -- 172.21.15.86:0/1846083698 learned_addr learned my addr 172.21.15.86:0/1846083698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f8982ffd640 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d8e60 0x5622201df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.894+0000 7f89837fe640 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 0x5622201dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f897a7fc640 1 -- 172.21.15.86:0/1846083698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 msgr2=0x5622201dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f897a7fc640 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 0x5622201dfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f897a7fc640 1 -- 172.21.15.86:0/1846083698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d8e60 msgr2=0x5622201df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f897a7fc640 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d8e60 0x5622201df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f897a7fc640 1 -- 172.21.15.86:0/1846083698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221fe503f0 con 0x5622201d9780 2024-04-16T05:39:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f8982ffd640 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d8e60 0x5622201df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f897a7fc640 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201df7f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f896c04c000 tx=0x7f896c08ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2292369cd06e2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f896a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f896a7fc640 1 -- 172.21.15.86:0/1846083698 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c04b070 con 0x5622201d9780 2024-04-16T05:39:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f896a7fc640 1 -- 172.21.15.86:0/1846083698 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f896c0efd90 con 0x5622201d9780 2024-04-16T05:39:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f898be7b740 1 -- 172.21.15.86:0/1846083698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89740047a0 con 0x5622201d9780 2024-04-16T05:39:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f896a7fc640 1 -- 172.21.15.86:0/1846083698 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f896c0eff90 con 0x5622201d9780 2024-04-16T05:39:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f898be7b740 10 client.17064 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f898be7b740 1 -- 172.21.15.86:0/1846083698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56221ff1a7d0 con 0x5622201d9780 2024-04-16T05:39:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f896a7fc640 1 -- 172.21.15.86:0/1846083698 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89740047a0 con 0x5622201d9780 2024-04-16T05:39:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f896a7fc640 1 -- 172.21.15.86:0/1846083698 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f896c08b080 con 0x5622201d9780 2024-04-16T05:39:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f898be7b740 10 client.17064 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f898be7b740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f898be7b740 20 client.17064 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f898be7b740 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5622201f21e0 0x5622201f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f898be7b740 4 client.17064 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.895+0000 7f898be7b740 1 -- 172.21.15.86:0/1846083698 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56222014bb00 con 0x5622201f21e0 2024-04-16T05:39:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.896+0000 7f8982ffd640 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5622201f21e0 0x5622201f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.896+0000 7f896a7fc640 10 client.17064 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.896+0000 7f8982ffd640 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5622201f21e0 0x5622201f46a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cebb55679ee0dd2 server_cookie=73393d2b98daaf82 in_seq=0 out_seq=0 2024-04-16T05:39:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f896a7fc640 1 -- 172.21.15.86:0/1846083698 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56222014bb00 con 0x5622201f21e0 2024-04-16T05:39:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f896a7fc640 10 client.17064 handle_command_reply: tid=0 2024-04-16T05:39:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 client.17064 shutdown 2024-04-16T05:39:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 -- 172.21.15.86:0/1846083698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 msgr2=0x5622201df7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201df7f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f896c04c000 tx=0x7f896c08ceb0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 -- 172.21.15.86:0/1846083698 shutdown_connections 2024-04-16T05:39:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5622201f21e0 0x5622201f46a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 0x5622201dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d9780 0x5622201df7f0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f896a7fc640 0 client.17064 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 --2- 172.21.15.86:0/1846083698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d8e60 0x5622201df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 -- 172.21.15.86:0/1846083698 >> 172.21.15.86:0/1846083698 conn(0x5622201d3590 msgr2=0x5622201d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 -- 172.21.15.86:0/1846083698 shutdown_connections 2024-04-16T05:39:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 1 -- 172.21.15.86:0/1846083698 wait complete. 2024-04-16T05:39:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.897+0000 7f898be7b740 20 client.17064 trim_cache size 0 max 16384 2024-04-16T05:39:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.910+0000 7f898be7b740 1 Processor -- start 2024-04-16T05:39:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.910+0000 7f898be7b740 1 -- start start 2024-04-16T05:39:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f898be7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f898be7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d9780 0x5622201deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f898be7b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 0x5622201f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f898be7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89841af330 con 0x5622201d8e60 2024-04-16T05:39:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f898be7b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89841b0e70 con 0x5622201e2360 2024-04-16T05:39:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f898be7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89841afa00 con 0x5622201d9780 2024-04-16T05:39:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8982ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8982ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52878/0 (socket says 172.21.15.86:52878) 2024-04-16T05:39:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8982ffd640 1 -- 172.21.15.86:0/3757166168 learned_addr learned my addr 172.21.15.86:0/3757166168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f89837fe640 1 --2- 172.21.15.86:0/3757166168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 0x5622201f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8982ffd640 1 -- 172.21.15.86:0/3757166168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d9780 msgr2=0x5622201deb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f897a7fc640 1 --2- 172.21.15.86:0/3757166168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d9780 0x5622201deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8982ffd640 1 --2- 172.21.15.86:0/3757166168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d9780 0x5622201deb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8982ffd640 1 -- 172.21.15.86:0/3757166168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 msgr2=0x5622201f9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8982ffd640 1 --2- 172.21.15.86:0/3757166168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 0x5622201f9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8982ffd640 1 -- 172.21.15.86:0/3757166168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622201cec30 con 0x5622201d8e60 2024-04-16T05:39:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8982ffd640 1 --2- 172.21.15.86:0/3757166168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201ee0c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8974008c60 tx=0x7f8974008fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af0de1b8b49aa54 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8978ff9640 1 -- 172.21.15.86:0/3757166168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974014040 con 0x5622201d8e60 2024-04-16T05:39:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8978ff9640 1 -- 172.21.15.86:0/3757166168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897402d070 con 0x5622201d8e60 2024-04-16T05:39:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.911+0000 7f8978ff9640 1 -- 172.21.15.86:0/3757166168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89740239f0 con 0x5622201d8e60 2024-04-16T05:39:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 -- 172.21.15.86:0/3757166168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 msgr2=0x5622201ee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 --2- 172.21.15.86:0/3757166168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201ee0c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8974008c60 tx=0x7f8974008fa0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 -- 172.21.15.86:0/3757166168 shutdown_connections 2024-04-16T05:39:44.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 --2- 172.21.15.86:0/3757166168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201e2360 0x5622201f9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 --2- 172.21.15.86:0/3757166168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201d9780 0x5622201deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 --2- 172.21.15.86:0/3757166168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201ee0c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 -- 172.21.15.86:0/3757166168 >> 172.21.15.86:0/3757166168 conn(0x5622201d3590 msgr2=0x5622201d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 -- 172.21.15.86:0/3757166168 shutdown_connections 2024-04-16T05:39:44.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 -- 172.21.15.86:0/3757166168 wait complete. 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 -- start start 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.912+0000 7f898be7b740 1 Processor -- start 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f898be7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201e5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f898be7b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d9780 0x5622201e5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f898be7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 0x5622202507e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f898be7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89841b07a0 con 0x5622201d8e60 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f898be7b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89841b00d0 con 0x5622201d9780 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f898be7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89841ad2f0 con 0x5622201e2360 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f898be7b740 10 client.0 _ll_register_callbacks cb 0x7f896c0ba0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f8982ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f8982ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201e5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52880/0 (socket says 172.21.15.86:52880) 2024-04-16T05:39:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f8982ffd640 1 -- 172.21.15.86:0/2068856669 learned_addr learned my addr 172.21.15.86:0/2068856669 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f89837fe640 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 0x5622202507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f8982ffd640 1 -- 172.21.15.86:0/2068856669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 msgr2=0x5622202507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f8982ffd640 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 0x5622202507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f897a7fc640 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d9780 0x5622201e5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f8982ffd640 1 -- 172.21.15.86:0/2068856669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d9780 msgr2=0x5622201e5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f8982ffd640 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d9780 0x5622201e5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f8982ffd640 1 -- 172.21.15.86:0/2068856669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562220250e20 con 0x5622201d8e60 2024-04-16T05:39:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f89837fe640 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 0x5622202507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f8982ffd640 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201e5820 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8974004640 tx=0x7f8974008400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89eb7a035d836887 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f896a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f896a7fc640 1 -- 172.21.15.86:0/2068856669 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974035030 con 0x5622201d8e60 2024-04-16T05:39:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f896a7fc640 1 -- 172.21.15.86:0/2068856669 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8974004020 con 0x5622201d8e60 2024-04-16T05:39:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f898be7b740 1 -- 172.21.15.86:0/2068856669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f897402d070 con 0x5622201d8e60 2024-04-16T05:39:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f896a7fc640 1 -- 172.21.15.86:0/2068856669 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8974014040 con 0x5622201d8e60 2024-04-16T05:39:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.913+0000 7f898be7b740 10 client.17070 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f898be7b740 1 -- 172.21.15.86:0/2068856669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562220258190 con 0x5622201d8e60 2024-04-16T05:39:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f896a7fc640 1 -- 172.21.15.86:0/2068856669 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897402d070 con 0x5622201d8e60 2024-04-16T05:39:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f896a7fc640 1 -- 172.21.15.86:0/2068856669 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f89740238f0 con 0x5622201d8e60 2024-04-16T05:39:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f898be7b740 10 client.17070 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f898be7b740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f898be7b740 20 client.17070 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f898be7b740 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562220258700 0x56222025ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f898be7b740 4 client.17070 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f898be7b740 1 -- 172.21.15.86:0/2068856669 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562220145970 con 0x562220258700 2024-04-16T05:39:44.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.914+0000 7f897a7fc640 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562220258700 0x56222025ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:44.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f896a7fc640 10 client.17070 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:44.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f897a7fc640 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562220258700 0x56222025ab20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd30a450943abbb server_cookie=3392094da2fa1119 in_seq=0 out_seq=0 2024-04-16T05:39:44.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f896a7fc640 1 -- 172.21.15.86:0/2068856669 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562220145970 con 0x562220258700 2024-04-16T05:39:44.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f896a7fc640 10 client.17070 handle_command_reply: tid=0 2024-04-16T05:39:44.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 client.17070 shutdown 2024-04-16T05:39:44.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 -- 172.21.15.86:0/2068856669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 msgr2=0x5622201e5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201e5820 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8974004640 tx=0x7f8974008400 comp rx=0 tx=0).stop 2024-04-16T05:39:44.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 -- 172.21.15.86:0/2068856669 shutdown_connections 2024-04-16T05:39:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562220258700 0x56222025ab20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622201e2360 0x5622202507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622201d9780 0x5622201e5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 --2- 172.21.15.86:0/2068856669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622201d8e60 0x5622201e5820 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f896a7fc640 0 client.17070 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 -- 172.21.15.86:0/2068856669 >> 172.21.15.86:0/2068856669 conn(0x5622201d3590 msgr2=0x5622201d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 -- 172.21.15.86:0/2068856669 shutdown_connections 2024-04-16T05:39:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 1 -- 172.21.15.86:0/2068856669 wait complete. 2024-04-16T05:39:44.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f898be7b740 20 client.17070 trim_cache size 0 max 16384 2024-04-16T05:39:44.926 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:39:44.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.915+0000 7f8989288640 1 -- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89700a3d20 msgr2=0x7f89700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89700a3d20 0x7f89700a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f897c005c40 tx=0x7f897c009500 comp rx=0 tx=0).stop 2024-04-16T05:39:44.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 -- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984107320 msgr2=0x7f898419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:44.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984107320 0x7f898419eaf0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f896c036040 tx=0x7f896c02ce20 comp rx=0 tx=0).stop 2024-04-16T05:39:44.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 -- 172.21.15.86:0/3863870245 shutdown_connections 2024-04-16T05:39:44.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f89700a3d20 0x7f89700a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8984198670 0x7f8984198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8984107320 0x7f898419eaf0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 --2- 172.21.15.86:0/3863870245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8984102250 0x7f898419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:44.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 -- 172.21.15.86:0/3863870245 >> 172.21.15.86:0/3863870245 conn(0x7f8984079490 msgr2=0x7f89840ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:44.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 -- 172.21.15.86:0/3863870245 shutdown_connections 2024-04-16T05:39:44.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:44.916+0000 7f8989288640 1 -- 172.21.15.86:0/3863870245 wait complete. 2024-04-16T05:39:45.930 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:39:45.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:39:45.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:39:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.052+0000 7f79be174640 1 -- 172.21.15.86:0/3541313539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103ba0 msgr2=0x7f79b810c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.052+0000 7f79be174640 1 --2- 172.21.15.86:0/3541313539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103ba0 0x7f79b810c080 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009900 tx=0x7f79ac02bd60 comp rx=0 tx=0).stop 2024-04-16T05:39:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.052+0000 7f79be174640 1 -- 172.21.15.86:0/3541313539 shutdown_connections 2024-04-16T05:39:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.052+0000 7f79be174640 1 --2- 172.21.15.86:0/3541313539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b810c5c0 0x7f79b810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.052+0000 7f79be174640 1 --2- 172.21.15.86:0/3541313539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103ba0 0x7f79b810c080 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.052+0000 7f79be174640 1 --2- 172.21.15.86:0/3541313539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79b8103280 0x7f79b8103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.052+0000 7f79be174640 1 -- 172.21.15.86:0/3541313539 >> 172.21.15.86:0/3541313539 conn(0x7f79b8078000 msgr2=0x7f79b8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.052+0000 7f79be174640 1 -- 172.21.15.86:0/3541313539 shutdown_connections 2024-04-16T05:39:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.052+0000 7f79be174640 1 -- 172.21.15.86:0/3541313539 wait complete. 2024-04-16T05:39:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.053+0000 7f79be174640 1 Processor -- start 2024-04-16T05:39:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.053+0000 7f79be174640 1 -- start start 2024-04-16T05:39:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.053+0000 7f79be174640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103280 0x7f79b8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.053+0000 7f79be174640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b8103ba0 0x7f79b8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.053+0000 7f79be174640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79b810c5c0 0x7f79b8101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.053+0000 7f79be174640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81123d0 con 0x7f79b8103280 2024-04-16T05:39:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.053+0000 7f79be174640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f79b8112250 con 0x7f79b810c5c0 2024-04-16T05:39:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.053+0000 7f79be174640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79b8112550 con 0x7f79b8103ba0 2024-04-16T05:39:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103280 0x7f79b8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103280 0x7f79b8100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52896/0 (socket says 172.21.15.86:52896) 2024-04-16T05:39:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b77fe640 1 -- 172.21.15.86:0/3307217621 learned_addr learned my addr 172.21.15.86:0/3307217621 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b8103ba0 0x7f79b8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b7fff640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79b810c5c0 0x7f79b8101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b77fe640 1 -- 172.21.15.86:0/3307217621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b8103ba0 msgr2=0x7f79b8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b77fe640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b8103ba0 0x7f79b8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b77fe640 1 -- 172.21.15.86:0/3307217621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79b810c5c0 msgr2=0x7f79b8101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b77fe640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79b810c5c0 0x7f79b8101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b77fe640 1 -- 172.21.15.86:0/3307217621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b81a6e10 con 0x7f79b8103280 2024-04-16T05:39:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b77fe640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103280 0x7f79b8100ed0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f79a800eab0 tx=0x7f79a800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b4ff9640 1 -- 172.21.15.86:0/3307217621 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a800ce40 con 0x7f79b8103280 2024-04-16T05:39:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b4ff9640 1 -- 172.21.15.86:0/3307217621 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a8011950 con 0x7f79b8103280 2024-04-16T05:39:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79be174640 1 -- 172.21.15.86:0/3307217621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b81a7100 con 0x7f79b8103280 2024-04-16T05:39:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79b4ff9640 1 -- 172.21.15.86:0/3307217621 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8010ae0 con 0x7f79b8103280 2024-04-16T05:39:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.054+0000 7f79be174640 1 -- 172.21.15.86:0/3307217621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b81a7670 con 0x7f79b8103280 2024-04-16T05:39:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.057+0000 7f79b4ff9640 1 -- 172.21.15.86:0/3307217621 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f79a8011b00 con 0x7f79b8103280 2024-04-16T05:39:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.057+0000 7f79c0e31740 1 Processor -- start 2024-04-16T05:39:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.057+0000 7f79c0e31740 1 -- start start 2024-04-16T05:39:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266aab0 0x56222266ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222673a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 0x5622226763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79c0e31740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622225cfe90 con 0x56222266b3d0 2024-04-16T05:39:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79c0e31740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56222256d4c0 con 0x56222266aab0 2024-04-16T05:39:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79c0e31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622225cbe50 con 0x562222673fb0 2024-04-16T05:39:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222673a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222673a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52902/0 (socket says 172.21.15.86:52902) 2024-04-16T05:39:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b77fe640 1 -- 172.21.15.86:0/3239461109 learned_addr learned my addr 172.21.15.86:0/3239461109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b7fff640 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266aab0 0x56222266ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b4ff9640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79a00a3da0 0x7f79a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 0x5622226763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b7fff640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79a00a3da0 0x7f79a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b77fe640 1 -- 172.21.15.86:0/3239461109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 msgr2=0x5622226763a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b77fe640 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 0x5622226763a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b77fe640 1 -- 172.21.15.86:0/3239461109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266aab0 msgr2=0x56222266ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b77fe640 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266aab0 0x56222266ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b77fe640 1 -- 172.21.15.86:0/3239461109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622223aa7e0 con 0x56222266b3d0 2024-04-16T05:39:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 0x5622226763a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b4ff9640 1 -- 172.21.15.86:0/3307217621 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f79a80d11b0 con 0x7f79b8103280 2024-04-16T05:39:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b77fe640 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222673a70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f79a80997b0 tx=0x7f79a809ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcc345b0fd41259 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.058+0000 7f79b7fff640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79a00a3da0 0x7f79a00a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79b0005fd0 tx=0x7f79b0004730 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f799dffb640 1 -- 172.21.15.86:0/3239461109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a809c740 con 0x56222266b3d0 2024-04-16T05:39:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f799dffb640 1 -- 172.21.15.86:0/3239461109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a809d4e0 con 0x56222266b3d0 2024-04-16T05:39:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f799dffb640 1 -- 172.21.15.86:0/3239461109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8099450 con 0x56222266b3d0 2024-04-16T05:39:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f79c0e31740 1 -- 172.21.15.86:0/3239461109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 msgr2=0x562222673a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222673a70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f79a80997b0 tx=0x7f79a809ca50 comp rx=0 tx=0).stop 2024-04-16T05:39:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f79c0e31740 1 -- 172.21.15.86:0/3239461109 shutdown_connections 2024-04-16T05:39:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 0x5622226763a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222673a70 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3239461109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266aab0 0x56222266ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f79c0e31740 1 -- 172.21.15.86:0/3239461109 >> 172.21.15.86:0/3239461109 conn(0x5622226651e0 msgr2=0x5622226655c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f79c0e31740 1 -- 172.21.15.86:0/3239461109 shutdown_connections 2024-04-16T05:39:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.059+0000 7f79c0e31740 1 -- 172.21.15.86:0/3239461109 wait complete. 2024-04-16T05:39:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.060+0000 7f79c0e31740 1 -- start start 2024-04-16T05:39:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.060+0000 7f79c0e31740 1 Processor -- start 2024-04-16T05:39:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.060+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266aab0 0x562222671090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266b3d0 0x5622226715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562222673fb0 0x562222671b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79c0e31740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562222676d40 con 0x56222266aab0 2024-04-16T05:39:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79c0e31740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562222558410 con 0x562222673fb0 2024-04-16T05:39:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79c0e31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562222676ec0 con 0x56222266b3d0 2024-04-16T05:39:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266aab0 0x562222671090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266aab0 0x562222671090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52910/0 (socket says 172.21.15.86:52910) 2024-04-16T05:39:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b77fe640 1 -- 172.21.15.86:0/1439125048 learned_addr learned my addr 172.21.15.86:0/1439125048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266b3d0 0x5622226715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79c0e31740 10 client.0 _ll_register_callbacks cb 0x562222553b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b7fff640 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562222673fb0 0x562222671b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b77fe640 1 -- 172.21.15.86:0/1439125048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266b3d0 msgr2=0x5622226715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b77fe640 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266b3d0 0x5622226715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b77fe640 1 -- 172.21.15.86:0/1439125048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562222673fb0 msgr2=0x562222671b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b77fe640 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562222673fb0 0x562222671b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b77fe640 1 -- 172.21.15.86:0/1439125048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222240af70 con 0x56222266aab0 2024-04-16T05:39:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79b77fe640 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266aab0 0x562222671090 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f79a8010580 tx=0x7f79a8099680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd4c19c8a96d0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f7996ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f7996ffd640 1 -- 172.21.15.86:0/1439125048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a809c740 con 0x56222266aab0 2024-04-16T05:39:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f7996ffd640 1 -- 172.21.15.86:0/1439125048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f79a809d240 con 0x56222266aab0 2024-04-16T05:39:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f79c0e31740 1 -- 172.21.15.86:0/1439125048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a809d4e0 con 0x56222266aab0 2024-04-16T05:39:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.061+0000 7f7996ffd640 1 -- 172.21.15.86:0/1439125048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79a80bb050 con 0x56222266aab0 2024-04-16T05:39:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f79c0e31740 10 client.17082 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f79c0e31740 1 -- 172.21.15.86:0/1439125048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622222a2170 con 0x56222266aab0 2024-04-16T05:39:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f7996ffd640 1 -- 172.21.15.86:0/1439125048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a809d4e0 con 0x56222266aab0 2024-04-16T05:39:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f7996ffd640 1 -- 172.21.15.86:0/1439125048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79a80cece0 con 0x56222266aab0 2024-04-16T05:39:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f79c0e31740 10 client.17082 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f79c0e31740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f79c0e31740 20 client.17082 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f79c0e31740 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562222684170 0x562222686560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f79c0e31740 4 client.17082 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f79c0e31740 1 -- 172.21.15.86:0/1439125048 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622225dd8b0 con 0x562222684170 2024-04-16T05:39:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.062+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562222684170 0x562222686560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.063+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562222684170 0x562222686560 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfe50fdbaf7cefa server_cookie=531fa8efba9b013d in_seq=0 out_seq=0 2024-04-16T05:39:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.063+0000 7f7996ffd640 10 client.17082 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.063+0000 7f7996ffd640 1 -- 172.21.15.86:0/1439125048 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622225dd8b0 con 0x562222684170 2024-04-16T05:39:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.063+0000 7f7996ffd640 10 client.17082 handle_command_reply: tid=0 2024-04-16T05:39:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.063+0000 7f79c0e31740 1 client.17082 shutdown 2024-04-16T05:39:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.063+0000 7f79c0e31740 1 -- 172.21.15.86:0/1439125048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266aab0 msgr2=0x562222671090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.063+0000 7f79c0e31740 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266aab0 0x562222671090 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f79a8010580 tx=0x7f79a8099680 comp rx=0 tx=0).stop 2024-04-16T05:39:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f79c0e31740 1 -- 172.21.15.86:0/1439125048 shutdown_connections 2024-04-16T05:39:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f79c0e31740 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x562222684170 0x562222686560 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f79c0e31740 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562222673fb0 0x562222671b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f79c0e31740 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266b3d0 0x5622226715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f79c0e31740 1 --2- 172.21.15.86:0/1439125048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266aab0 0x562222671090 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f7996ffd640 0 client.17082 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f79c0e31740 1 -- 172.21.15.86:0/1439125048 >> 172.21.15.86:0/1439125048 conn(0x5622226651e0 msgr2=0x562222666f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f79c0e31740 1 -- 172.21.15.86:0/1439125048 shutdown_connections 2024-04-16T05:39:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f79c0e31740 1 -- 172.21.15.86:0/1439125048 wait complete. 2024-04-16T05:39:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.064+0000 7f79c0e31740 20 client.17082 trim_cache size 0 max 16384 2024-04-16T05:39:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.081+0000 7f79c0e31740 1 Processor -- start 2024-04-16T05:39:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.081+0000 7f79c0e31740 1 -- start start 2024-04-16T05:39:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.081+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266aab0 0x5622226737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.081+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222671930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.081+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 0x56222268bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.081+0000 7f79c0e31740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81b1700 con 0x56222266b3d0 2024-04-16T05:39:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.081+0000 7f79c0e31740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81b3240 con 0x56222266aab0 2024-04-16T05:39:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.081+0000 7f79c0e31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81b1dd0 con 0x562222673fb0 2024-04-16T05:39:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.081+0000 7f79b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222671930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f79b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222671930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52912/0 (socket says 172.21.15.86:52912) 2024-04-16T05:39:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f79b6ffd640 1 -- 172.21.15.86:0/2756822276 learned_addr learned my addr 172.21.15.86:0/2756822276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f79b7fff640 1 --2- 172.21.15.86:0/2756822276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 0x56222268bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f79b6ffd640 1 -- 172.21.15.86:0/2756822276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 msgr2=0x56222268bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/2756822276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 0x56222268bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f79b6ffd640 1 -- 172.21.15.86:0/2756822276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266aab0 msgr2=0x5622226737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/2756822276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266aab0 0x5622226737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f79b6ffd640 1 -- 172.21.15.86:0/2756822276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562222676710 con 0x56222266b3d0 2024-04-16T05:39:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/2756822276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222671930 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f79ac006010 tx=0x7f79ac02dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586aec9245f8bb24 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f799d7fa640 1 -- 172.21.15.86:0/2756822276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac04e070 con 0x56222266b3d0 2024-04-16T05:39:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f799d7fa640 1 -- 172.21.15.86:0/2756822276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ac02dda0 con 0x56222266b3d0 2024-04-16T05:39:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.082+0000 7f799d7fa640 1 -- 172.21.15.86:0/2756822276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac004840 con 0x56222266b3d0 2024-04-16T05:39:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 -- 172.21.15.86:0/2756822276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 msgr2=0x562222671930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 --2- 172.21.15.86:0/2756822276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222671930 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f79ac006010 tx=0x7f79ac02dbf0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 -- 172.21.15.86:0/2756822276 shutdown_connections 2024-04-16T05:39:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 --2- 172.21.15.86:0/2756822276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562222673fb0 0x56222268bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 --2- 172.21.15.86:0/2756822276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56222266b3d0 0x562222671930 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 --2- 172.21.15.86:0/2756822276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266aab0 0x5622226737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 -- 172.21.15.86:0/2756822276 >> 172.21.15.86:0/2756822276 conn(0x5622226651e0 msgr2=0x5622226661e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 -- 172.21.15.86:0/2756822276 shutdown_connections 2024-04-16T05:39:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 -- 172.21.15.86:0/2756822276 wait complete. 2024-04-16T05:39:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 -- start start 2024-04-16T05:39:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 Processor -- start 2024-04-16T05:39:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266aab0 0x56222267c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266b3d0 0x56222267c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562222673fb0 0x5622226791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81b2b70 con 0x562222673fb0 2024-04-16T05:39:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81b24a0 con 0x56222266b3d0 2024-04-16T05:39:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.083+0000 7f79c0e31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81af6c0 con 0x56222266aab0 2024-04-16T05:39:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79c0e31740 10 client.0 _ll_register_callbacks cb 0x7f79b000c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562222673fb0 0x5622226791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562222673fb0 0x5622226791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52926/0 (socket says 172.21.15.86:52926) 2024-04-16T05:39:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b7fff640 1 -- 172.21.15.86:0/3735901 learned_addr learned my addr 172.21.15.86:0/3735901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b77fe640 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266aab0 0x56222267c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b7fff640 1 -- 172.21.15.86:0/3735901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266aab0 msgr2=0x56222267c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b7fff640 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266aab0 0x56222267c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b7fff640 1 -- 172.21.15.86:0/3735901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266b3d0 msgr2=0x56222267c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b6ffd640 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266b3d0 0x56222267c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b7fff640 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266b3d0 0x56222267c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b7fff640 1 -- 172.21.15.86:0/3735901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562222679a00 con 0x562222673fb0 2024-04-16T05:39:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79b7fff640 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562222673fb0 0x5622226791d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f79b001ed70 tx=0x7f79b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7724e734533355f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f7996ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f7996ffd640 1 -- 172.21.15.86:0/3735901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0018070 con 0x562222673fb0 2024-04-16T05:39:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f7996ffd640 1 -- 172.21.15.86:0/3735901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f79b0005020 con 0x562222673fb0 2024-04-16T05:39:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79c0e31740 1 -- 172.21.15.86:0/3735901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79ac02dda0 con 0x562222673fb0 2024-04-16T05:39:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f7996ffd640 1 -- 172.21.15.86:0/3735901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79b0005220 con 0x562222673fb0 2024-04-16T05:39:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79c0e31740 10 client.17088 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.084+0000 7f79c0e31740 1 -- 172.21.15.86:0/3735901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622226eb070 con 0x562222673fb0 2024-04-16T05:39:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f7996ffd640 1 -- 172.21.15.86:0/3735901 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ac02dda0 con 0x562222673fb0 2024-04-16T05:39:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f7996ffd640 1 -- 172.21.15.86:0/3735901 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79b000f810 con 0x562222673fb0 2024-04-16T05:39:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f79c0e31740 10 client.17088 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f79c0e31740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f79c0e31740 20 client.17088 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5622226eb870 0x5622226edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f79c0e31740 4 client.17088 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:39:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f79c0e31740 1 -- 172.21.15.86:0/3735901 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5622225d7720 con 0x5622226eb870 2024-04-16T05:39:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f79b77fe640 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5622226eb870 0x5622226edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f7996ffd640 10 client.17088 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.085+0000 7f79b77fe640 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5622226eb870 0x5622226edc90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7670f42c099ada74 server_cookie=7f25d3571c16a876 in_seq=0 out_seq=0 2024-04-16T05:39:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.086+0000 7f7996ffd640 1 -- 172.21.15.86:0/3735901 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5622225d7720 con 0x5622226eb870 2024-04-16T05:39:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.086+0000 7f7996ffd640 10 client.17088 handle_command_reply: tid=0 2024-04-16T05:39:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.086+0000 7f79c0e31740 1 client.17088 shutdown 2024-04-16T05:39:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 -- 172.21.15.86:0/3735901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562222673fb0 msgr2=0x5622226791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562222673fb0 0x5622226791d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f79b001ed70 tx=0x7f79b00046c0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 -- 172.21.15.86:0/3735901 shutdown_connections 2024-04-16T05:39:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5622226eb870 0x5622226edc90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562222673fb0 0x5622226791d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56222266b3d0 0x56222267c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 --2- 172.21.15.86:0/3735901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222266aab0 0x56222267c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 -- 172.21.15.86:0/3735901 >> 172.21.15.86:0/3735901 conn(0x5622226651e0 msgr2=0x562222666a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f7996ffd640 0 client.17088 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 -- 172.21.15.86:0/3735901 shutdown_connections 2024-04-16T05:39:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 1 -- 172.21.15.86:0/3735901 wait complete. 2024-04-16T05:39:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79c0e31740 20 client.17088 trim_cache size 0 max 16384 2024-04-16T05:39:46.095 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:39:46.096 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:39:46.096 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "9d7a11fc-df0b-48d5-9684-89c2babee4ef", 2024-04-16T05:39:46.096 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:39:46.096 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:39:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 -- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79a00a3da0 msgr2=0x7f79a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79a00a3da0 0x7f79a00a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79b0005fd0 tx=0x7f79b0004730 comp rx=0 tx=0).stop 2024-04-16T05:39:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 -- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103280 msgr2=0x7f79b8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103280 0x7f79b8100ed0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f79a800eab0 tx=0x7f79a800ef70 comp rx=0 tx=0).stop 2024-04-16T05:39:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 -- 172.21.15.86:0/3307217621 shutdown_connections 2024-04-16T05:39:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f79a00a3da0 0x7f79a00a6260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79b810c5c0 0x7f79b8101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b8103ba0 0x7f79b8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 --2- 172.21.15.86:0/3307217621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b8103280 0x7f79b8100ed0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.087+0000 7f79be174640 1 -- 172.21.15.86:0/3307217621 >> 172.21.15.86:0/3307217621 conn(0x7f79b8078000 msgr2=0x7f79b80fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.088+0000 7f79be174640 1 -- 172.21.15.86:0/3307217621 shutdown_connections 2024-04-16T05:39:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.088+0000 7f79be174640 1 -- 172.21.15.86:0/3307217621 wait complete. 2024-04-16T05:39:46.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:39:46.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:39:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 -- 172.21.15.86:0/2265603844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101250 msgr2=0x7f27f41016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/2265603844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101250 0x7f27f41016b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009930 tx=0x7f27dc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 -- 172.21.15.86:0/2265603844 shutdown_connections 2024-04-16T05:39:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/2265603844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27f4101bf0 0x7f27f410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/2265603844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101250 0x7f27f41016b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/2265603844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f41070e0 0x7f27f41074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 -- 172.21.15.86:0/2265603844 >> 172.21.15.86:0/2265603844 conn(0x7f27f4076d00 msgr2=0x7f27f4077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 -- 172.21.15.86:0/2265603844 shutdown_connections 2024-04-16T05:39:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 -- 172.21.15.86:0/2265603844 wait complete. 2024-04-16T05:39:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 Processor -- start 2024-04-16T05:39:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.207+0000 7f27f9fa8640 1 -- start start 2024-04-16T05:39:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f9fa8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f4101250 0x7f27f419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f9fa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101bf0 0x7f27f419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f9fa8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27f41070e0 0x7f27f4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f9fa8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41112e0 con 0x7f27f4101bf0 2024-04-16T05:39:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f9fa8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4111160 con 0x7f27f41070e0 2024-04-16T05:39:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f9fa8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4111460 con 0x7f27f4101250 2024-04-16T05:39:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101bf0 0x7f27f419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101bf0 0x7f27f419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52944/0 (socket says 172.21.15.86:52944) 2024-04-16T05:39:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27eaffd640 1 -- 172.21.15.86:0/4152055027 learned_addr learned my addr 172.21.15.86:0/4152055027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f3fff640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27f41070e0 0x7f27f4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27eaffd640 1 -- 172.21.15.86:0/4152055027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f4101250 msgr2=0x7f27f419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27eaffd640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f4101250 0x7f27f419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27eaffd640 1 -- 172.21.15.86:0/4152055027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27f41070e0 msgr2=0x7f27f4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27eaffd640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27f41070e0 0x7f27f4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27eaffd640 1 -- 172.21.15.86:0/4152055027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f4199000 con 0x7f27f4101bf0 2024-04-16T05:39:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27eaffd640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101bf0 0x7f27f419eb70 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009230 tx=0x7f27dc02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f17fa640 1 -- 172.21.15.86:0/4152055027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc041070 con 0x7f27f4101bf0 2024-04-16T05:39:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f17fa640 1 -- 172.21.15.86:0/4152055027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc004450 con 0x7f27f4101bf0 2024-04-16T05:39:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f9fa8640 1 -- 172.21.15.86:0/4152055027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f4199290 con 0x7f27f4101bf0 2024-04-16T05:39:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.208+0000 7f27f17fa640 1 -- 172.21.15.86:0/4152055027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc034440 con 0x7f27f4101bf0 2024-04-16T05:39:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.209+0000 7f27f9fa8640 1 -- 172.21.15.86:0/4152055027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f41a53d0 con 0x7f27f4101bf0 2024-04-16T05:39:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27f17fa640 1 -- 172.21.15.86:0/4152055027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27dc044070 con 0x7f27f4101bf0 2024-04-16T05:39:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27fcb8c740 1 Processor -- start 2024-04-16T05:39:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27fcb8c740 1 -- start start 2024-04-16T05:39:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 0x55fa063cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa06325180 con 0x55fa063bfe50 2024-04-16T05:39:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fa062c2360 con 0x55fa063c0770 2024-04-16T05:39:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa06321140 con 0x55fa063c9350 2024-04-16T05:39:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.211+0000 7f27f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52948/0 (socket says 172.21.15.86:52948) 2024-04-16T05:39:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f3fff640 1 -- 172.21.15.86:0/637118277 learned_addr learned my addr 172.21.15.86:0/637118277 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27eaffd640 1 --2- 172.21.15.86:0/637118277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 0x55fa063cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f37fe640 1 --2- 172.21.15.86:0/637118277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f17fa640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27e00a3d20 0x7f27e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f3fff640 1 -- 172.21.15.86:0/637118277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 msgr2=0x55fa063cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f3fff640 1 --2- 172.21.15.86:0/637118277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 0x55fa063cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f3fff640 1 -- 172.21.15.86:0/637118277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 msgr2=0x55fa063c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f3fff640 1 --2- 172.21.15.86:0/637118277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f3fff640 1 -- 172.21.15.86:0/637118277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa061017d0 con 0x55fa063bfe50 2024-04-16T05:39:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f17fa640 1 -- 172.21.15.86:0/4152055027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f27dc0f2560 con 0x7f27f4101bf0 2024-04-16T05:39:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f3fff640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27e00a3d20 0x7f27e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27f3fff640 1 --2- 172.21.15.86:0/637118277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c0230 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0103f0 tx=0x7f27ec0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0adc0735bce40a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27e9ffb640 1 -- 172.21.15.86:0/637118277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec01c900 con 0x55fa063bfe50 2024-04-16T05:39:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27e9ffb640 1 -- 172.21.15.86:0/637118277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ec01b3d0 con 0x55fa063bfe50 2024-04-16T05:39:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.212+0000 7f27e9ffb640 1 -- 172.21.15.86:0/637118277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0255e0 con 0x55fa063bfe50 2024-04-16T05:39:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 -- 172.21.15.86:0/637118277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 msgr2=0x55fa063c0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/637118277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c0230 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0103f0 tx=0x7f27ec0108b0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27f3fff640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27e00a3d20 0x7f27e00a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f27ec020000 tx=0x7f27ec01d000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 -- 172.21.15.86:0/637118277 shutdown_connections 2024-04-16T05:39:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/637118277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 0x55fa063cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/637118277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/637118277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c0230 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 -- 172.21.15.86:0/637118277 >> 172.21.15.86:0/637118277 conn(0x55fa063ba580 msgr2=0x55fa063ba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 -- 172.21.15.86:0/637118277 shutdown_connections 2024-04-16T05:39:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 -- 172.21.15.86:0/637118277 wait complete. 2024-04-16T05:39:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.213+0000 7f27fcb8c740 1 -- start start 2024-04-16T05:39:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27fcb8c740 1 Processor -- start 2024-04-16T05:39:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 0x55fa063c6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c9350 0x55fa063c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa063cc140 con 0x55fa063c9350 2024-04-16T05:39:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fa062de2c0 con 0x55fa063c0770 2024-04-16T05:39:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa063cc2c0 con 0x55fa063bfe50 2024-04-16T05:39:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27fcb8c740 10 client.0 _ll_register_callbacks cb 0x55fa062c5090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c9350 0x55fa063c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c9350 0x55fa063c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52952/0 (socket says 172.21.15.86:52952) 2024-04-16T05:39:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27f3fff640 1 -- 172.21.15.86:0/886644902 learned_addr learned my addr 172.21.15.86:0/886644902 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27eaffd640 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27f37fe640 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 0x55fa063c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27f3fff640 1 -- 172.21.15.86:0/886644902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 msgr2=0x55fa063c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27f3fff640 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 0x55fa063c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27f3fff640 1 -- 172.21.15.86:0/886644902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 msgr2=0x55fa063c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27f3fff640 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27f3fff640 1 -- 172.21.15.86:0/886644902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa060373f0 con 0x55fa063c9350 2024-04-16T05:39:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.214+0000 7f27eaffd640 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27f3fff640 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c9350 0x55fa063c6de0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f27ec01bd80 tx=0x7f27ec01b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ff327792986751 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27d2ffd640 1 -- 172.21.15.86:0/886644902 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec009b90 con 0x55fa063c9350 2024-04-16T05:39:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27d2ffd640 1 -- 172.21.15.86:0/886644902 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f27ec01ca90 con 0x55fa063c9350 2024-04-16T05:39:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27fcb8c740 1 -- 172.21.15.86:0/886644902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27ec01b3d0 con 0x55fa063c9350 2024-04-16T05:39:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27d2ffd640 1 -- 172.21.15.86:0/886644902 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27ec03d050 con 0x55fa063c9350 2024-04-16T05:39:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27fcb8c740 10 client.17100 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27fcb8c740 1 -- 172.21.15.86:0/886644902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa05ff70f0 con 0x55fa063c9350 2024-04-16T05:39:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27d2ffd640 1 -- 172.21.15.86:0/886644902 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ec01b3d0 con 0x55fa063c9350 2024-04-16T05:39:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27d2ffd640 1 -- 172.21.15.86:0/886644902 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f27ec00f040 con 0x55fa063c9350 2024-04-16T05:39:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27fcb8c740 10 client.17100 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27fcb8c740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27fcb8c740 20 client.17100 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa063d9650 0x55fa063dba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27fcb8c740 4 client.17100 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.215+0000 7f27fcb8c740 1 -- 172.21.15.86:0/886644902 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa06332ba0 con 0x55fa063d9650 2024-04-16T05:39:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.216+0000 7f27f37fe640 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa063d9650 0x55fa063dba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.216+0000 7f27d2ffd640 10 client.17100 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.216+0000 7f27f37fe640 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa063d9650 0x55fa063dba40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4276013799a312 server_cookie=a679718fc439f9ee in_seq=0 out_seq=0 2024-04-16T05:39:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27d2ffd640 1 -- 172.21.15.86:0/886644902 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa06332ba0 con 0x55fa063d9650 2024-04-16T05:39:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27d2ffd640 10 client.17100 handle_command_reply: tid=0 2024-04-16T05:39:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 client.17100 shutdown 2024-04-16T05:39:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 -- 172.21.15.86:0/886644902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c9350 msgr2=0x55fa063c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c9350 0x55fa063c6de0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f27ec01bd80 tx=0x7f27ec01b830 comp rx=0 tx=0).stop 2024-04-16T05:39:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 -- 172.21.15.86:0/886644902 shutdown_connections 2024-04-16T05:39:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa063d9650 0x55fa063dba40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c9350 0x55fa063c6de0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27d2ffd640 0 client.17100 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/886644902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 0x55fa063c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 -- 172.21.15.86:0/886644902 >> 172.21.15.86:0/886644902 conn(0x55fa063ba580 msgr2=0x55fa063bc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 -- 172.21.15.86:0/886644902 shutdown_connections 2024-04-16T05:39:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 1 -- 172.21.15.86:0/886644902 wait complete. 2024-04-16T05:39:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.217+0000 7f27fcb8c740 20 client.17100 trim_cache size 0 max 16384 2024-04-16T05:39:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27fcb8c740 1 Processor -- start 2024-04-16T05:39:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27fcb8c740 1 -- start start 2024-04-16T05:39:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 0x55fa063c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41af420 con 0x55fa063bfe50 2024-04-16T05:39:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41b0f60 con 0x55fa063c0770 2024-04-16T05:39:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41afaf0 con 0x55fa063c9350 2024-04-16T05:39:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52964/0 (socket says 172.21.15.86:52964) 2024-04-16T05:39:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f37fe640 1 -- 172.21.15.86:0/2871363922 learned_addr learned my addr 172.21.15.86:0/2871363922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27eaffd640 1 --2- 172.21.15.86:0/2871363922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f3fff640 1 --2- 172.21.15.86:0/2871363922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 0x55fa063c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f37fe640 1 -- 172.21.15.86:0/2871363922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 msgr2=0x55fa063c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f37fe640 1 --2- 172.21.15.86:0/2871363922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 0x55fa063c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f37fe640 1 -- 172.21.15.86:0/2871363922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 msgr2=0x55fa063d5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f37fe640 1 --2- 172.21.15.86:0/2871363922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063d5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f37fe640 1 -- 172.21.15.86:0/2871363922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa063bb150 con 0x55fa063bfe50 2024-04-16T05:39:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.235+0000 7f27f37fe640 1 --2- 172.21.15.86:0/2871363922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c8a20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009c40 tx=0x7f27e4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee9f1ecc66ffcb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.236+0000 7f27e97fa640 1 -- 172.21.15.86:0/2871363922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e401bbc0 con 0x55fa063bfe50 2024-04-16T05:39:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.236+0000 7f27e97fa640 1 -- 172.21.15.86:0/2871363922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e4020070 con 0x55fa063bfe50 2024-04-16T05:39:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.236+0000 7f27e97fa640 1 -- 172.21.15.86:0/2871363922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4010460 con 0x55fa063bfe50 2024-04-16T05:39:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 -- 172.21.15.86:0/2871363922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 msgr2=0x55fa063c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/2871363922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c8a20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009c40 tx=0x7f27e4009620 comp rx=0 tx=0).stop 2024-04-16T05:39:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 -- 172.21.15.86:0/2871363922 shutdown_connections 2024-04-16T05:39:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/2871363922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063c9350 0x55fa063c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/2871363922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c0770 0x55fa063d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/2871363922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063bfe50 0x55fa063c8a20 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 -- 172.21.15.86:0/2871363922 >> 172.21.15.86:0/2871363922 conn(0x55fa063ba580 msgr2=0x55fa063b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 -- 172.21.15.86:0/2871363922 shutdown_connections 2024-04-16T05:39:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 -- 172.21.15.86:0/2871363922 wait complete. 2024-04-16T05:39:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 -- start start 2024-04-16T05:39:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 Processor -- start 2024-04-16T05:39:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 0x55fa063ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c0770 0x55fa063cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c9350 0x55fa0643db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41b0890 con 0x55fa063c0770 2024-04-16T05:39:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41b01c0 con 0x55fa063c9350 2024-04-16T05:39:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.237+0000 7f27fcb8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41ad3e0 con 0x55fa063bfe50 2024-04-16T05:39:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27fcb8c740 10 client.0 _ll_register_callbacks cb 0x7f27ec01f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:46.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c0770 0x55fa063cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c0770 0x55fa063cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52970/0 (socket says 172.21.15.86:52970) 2024-04-16T05:39:46.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27eaffd640 1 -- 172.21.15.86:0/4007099344 learned_addr learned my addr 172.21.15.86:0/4007099344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:39:46.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27f37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 0x55fa063ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27f3fff640 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c9350 0x55fa0643db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27eaffd640 1 -- 172.21.15.86:0/4007099344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 msgr2=0x55fa063ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27eaffd640 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 0x55fa063ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27eaffd640 1 -- 172.21.15.86:0/4007099344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c9350 msgr2=0x55fa0643db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:46.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27eaffd640 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c9350 0x55fa0643db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27eaffd640 1 -- 172.21.15.86:0/4007099344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0643e050 con 0x55fa063c0770 2024-04-16T05:39:46.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27eaffd640 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c0770 0x55fa063cea20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009430 tx=0x7f27dc0f28c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83747ade9ac82a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:46.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:39:46.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27d2ffd640 1 -- 172.21.15.86:0/4007099344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc04b040 con 0x55fa063c0770 2024-04-16T05:39:46.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27d2ffd640 1 -- 172.21.15.86:0/4007099344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f27dc0f21b0 con 0x55fa063c0770 2024-04-16T05:39:46.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27fcb8c740 1 -- 172.21.15.86:0/4007099344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e4020070 con 0x55fa063c0770 2024-04-16T05:39:46.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27d2ffd640 1 -- 172.21.15.86:0/4007099344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27dc0bdc00 con 0x55fa063c0770 2024-04-16T05:39:46.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27fcb8c740 10 client.17106 fetch_fsmap learned FSMap version 29 2024-04-16T05:39:46.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27fcb8c740 1 -- 172.21.15.86:0/4007099344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa0643f1a0 con 0x55fa063c0770 2024-04-16T05:39:46.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.238+0000 7f27d2ffd640 1 -- 172.21.15.86:0/4007099344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e4020070 con 0x55fa063c0770 2024-04-16T05:39:46.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27d2ffd640 1 -- 172.21.15.86:0/4007099344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f27dc0c6a80 con 0x55fa063c0770 2024-04-16T05:39:46.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27fcb8c740 10 client.17106 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:39:46.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27fcb8c740 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:46.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27fcb8c740 20 client.17106 populate_metadata read hostname 'smithi086' 2024-04-16T05:39:46.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa0643f970 0x55fa06441d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:46.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27fcb8c740 4 client.17106 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:39:46.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27fcb8c740 1 -- 172.21.15.86:0/4007099344 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa0632ca10 con 0x55fa0643f970 2024-04-16T05:39:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27f37fe640 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa0643f970 0x55fa06441d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27d2ffd640 10 client.17106 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27f37fe640 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa0643f970 0x55fa06441d90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc7eb407f3029b server_cookie=1dc8cae55bde770b in_seq=0 out_seq=0 2024-04-16T05:39:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27d2ffd640 1 -- 172.21.15.86:0/4007099344 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fa0632ca10 con 0x55fa0643f970 2024-04-16T05:39:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27d2ffd640 10 client.17106 handle_command_reply: tid=0 2024-04-16T05:39:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.239+0000 7f27fcb8c740 1 client.17106 shutdown 2024-04-16T05:39:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 -- 172.21.15.86:0/4007099344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c0770 msgr2=0x55fa063cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c0770 0x55fa063cea20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009430 tx=0x7f27dc0f28c0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 -- 172.21.15.86:0/4007099344 shutdown_connections 2024-04-16T05:39:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55fa0643f970 0x55fa06441d90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fa063c9350 0x55fa0643db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27d2ffd640 0 client.17106 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:39:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa063c0770 0x55fa063cea20 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 --2- 172.21.15.86:0/4007099344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa063bfe50 0x55fa063ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 -- 172.21.15.86:0/4007099344 >> 172.21.15.86:0/4007099344 conn(0x55fa063ba580 msgr2=0x55fa063b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 -- 172.21.15.86:0/4007099344 shutdown_connections 2024-04-16T05:39:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 1 -- 172.21.15.86:0/4007099344 wait complete. 2024-04-16T05:39:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27fcb8c740 20 client.17106 trim_cache size 0 max 16384 2024-04-16T05:39:46.249 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:39:46.249 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:39:46.249 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:39:46.249 INFO:teuthology.orchestra.run.smithi086.stdout: "9d7a11fc-df0b-48d5-9684-89c2babee4ef": { 2024-04-16T05:39:46.249 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:39:46.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "9d7a11fc-df0b-48d5-9684-89c2babee4ef", 2024-04-16T05:39:46.249 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:39:46.250 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:39:46.250 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:39:46.250 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:39:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 -- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27e00a3d20 msgr2=0x7f27e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27e00a3d20 0x7f27e00a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f27ec020000 tx=0x7f27ec01d000 comp rx=0 tx=0).stop 2024-04-16T05:39:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 -- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101bf0 msgr2=0x7f27f419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101bf0 0x7f27f419eb70 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009230 tx=0x7f27dc02cba0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 -- 172.21.15.86:0/4152055027 shutdown_connections 2024-04-16T05:39:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27e00a3d20 0x7f27e00a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27f41070e0 0x7f27f4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f4101bf0 0x7f27f419eb70 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 --2- 172.21.15.86:0/4152055027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f4101250 0x7f27f419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.240+0000 7f27f9fa8640 1 -- 172.21.15.86:0/4152055027 >> 172.21.15.86:0/4152055027 conn(0x7f27f4076d00 msgr2=0x7f27f406a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.241+0000 7f27f9fa8640 1 -- 172.21.15.86:0/4152055027 shutdown_connections 2024-04-16T05:39:46.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:39:46.241+0000 7f27f9fa8640 1 -- 172.21.15.86:0/4152055027 wait complete. 2024-04-16T05:39:46.253 INFO:tasks.cephfs.filesystem:scrub status for tag:9d7a11fc-df0b-48d5-9684-89c2babee4ef - {'path': '/', 'tag': '9d7a11fc-df0b-48d5-9684-89c2babee4ef', 'options': 'recursive,force'} 2024-04-16T05:39:46.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:46 smithi196 ceph-mon[36373]: pgmap v137: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:46.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:46 smithi194 ceph-mon[35865]: pgmap v137: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:46.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:46 smithi086 ceph-mon[32028]: pgmap v137: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:47.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:47 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:39:47.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:47 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:47.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:47 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:39:47.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:47 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:47.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:47 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:39:47.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:47 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:47.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:47 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:39:47.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:47 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:47.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:47 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:39:47.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:47 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:47.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:47 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:39:47.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:47 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:48.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:48 smithi194 ceph-mon[35865]: pgmap v138: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:48.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:48 smithi086 ceph-mon[32028]: pgmap v138: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:48.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:48 smithi196 ceph-mon[36373]: pgmap v138: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:39:50.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:50 smithi196 ceph-mon[36373]: pgmap v139: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:39:50.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:50 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:39:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:50 smithi194 ceph-mon[35865]: pgmap v139: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:39:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:50 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:39:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:50 smithi086 ceph-mon[32028]: pgmap v139: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:39:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:50 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:39:52.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:52 smithi194 ceph-mon[35865]: pgmap v140: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:52.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:52 smithi086 ceph-mon[32028]: pgmap v140: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:52 smithi196 ceph-mon[36373]: pgmap v140: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:54.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:54 smithi194 ceph-mon[35865]: pgmap v141: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:39:54.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:54 smithi086 ceph-mon[32028]: pgmap v141: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:39:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:54 smithi196 ceph-mon[36373]: pgmap v141: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:39:56.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:56 smithi194 ceph-mon[35865]: pgmap v142: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:56.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:56 smithi086 ceph-mon[32028]: pgmap v142: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:56 smithi196 ceph-mon[36373]: pgmap v142: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:58.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:39:58 smithi194 ceph-mon[35865]: pgmap v143: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:58.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:39:58 smithi086 ceph-mon[32028]: pgmap v143: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:39:58.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:39:58 smithi196 ceph-mon[36373]: pgmap v143: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:00.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:00 smithi194 ceph-mon[35865]: pgmap v144: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:00.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:00 smithi194 ceph-mon[35865]: overall HEALTH_OK 2024-04-16T05:40:00.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:00 smithi086 ceph-mon[32028]: pgmap v144: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:00.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:00 smithi086 ceph-mon[32028]: overall HEALTH_OK 2024-04-16T05:40:00.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:00 smithi196 ceph-mon[36373]: pgmap v144: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:00.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:00 smithi196 ceph-mon[36373]: overall HEALTH_OK 2024-04-16T05:40:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:02 smithi194 ceph-mon[35865]: pgmap v145: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:02 smithi086 ceph-mon[32028]: pgmap v145: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:02 smithi196 ceph-mon[36373]: pgmap v145: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:04.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:04 smithi194 ceph-mon[35865]: pgmap v146: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:40:04.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:04 smithi086 ceph-mon[32028]: pgmap v146: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:40:04.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:04 smithi196 ceph-mon[36373]: pgmap v146: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:40:06.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:06 smithi194 ceph-mon[35865]: pgmap v147: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:06.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:06 smithi086 ceph-mon[32028]: pgmap v147: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:06 smithi196 ceph-mon[36373]: pgmap v147: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:08.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:08 smithi194 ceph-mon[35865]: pgmap v148: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:08.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:08 smithi086 ceph-mon[32028]: pgmap v148: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:08.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:08 smithi196 ceph-mon[36373]: pgmap v148: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:10.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:10 smithi194 ceph-mon[35865]: pgmap v149: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:40:10.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:10 smithi086 ceph-mon[32028]: pgmap v149: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:40:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:10 smithi196 ceph-mon[36373]: pgmap v149: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:40:12.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:12 smithi194 ceph-mon[35865]: pgmap v150: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:12.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:12 smithi086 ceph-mon[32028]: pgmap v150: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:12.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:12 smithi196 ceph-mon[36373]: pgmap v150: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:14.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:14 smithi194 ceph-mon[35865]: pgmap v151: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:40:14.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:14 smithi086 ceph-mon[32028]: pgmap v151: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:40:14.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:14 smithi196 ceph-mon[36373]: pgmap v151: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:40:16.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:40:16.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:40:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.380+0000 7f4cb9d72640 1 -- 172.21.15.86:0/3855664671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 msgr2=0x7f4cb4101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.380+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/3855664671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 0x7f4cb4101520 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009900 tx=0x7f4ca402bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:16.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.380+0000 7f4cb9d72640 1 -- 172.21.15.86:0/3855664671 shutdown_connections 2024-04-16T05:40:16.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.380+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/3855664671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb4106210 0x7f4cb410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.380+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/3855664671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb4101af0 0x7f4cb4105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.380+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/3855664671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 0x7f4cb4101520 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.380+0000 7f4cb9d72640 1 -- 172.21.15.86:0/3855664671 >> 172.21.15.86:0/3855664671 conn(0x7f4cb4078000 msgr2=0x7f4cb4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.380+0000 7f4cb9d72640 1 -- 172.21.15.86:0/3855664671 shutdown_connections 2024-04-16T05:40:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.380+0000 7f4cb9d72640 1 -- 172.21.15.86:0/3855664671 wait complete. 2024-04-16T05:40:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb9d72640 1 Processor -- start 2024-04-16T05:40:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb9d72640 1 -- start start 2024-04-16T05:40:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb9d72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 0x7f4cb410e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb9d72640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb4106210 0x7f4cb410ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb9d72640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb410f1f0 0x7f4cb41a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb9d72640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41123e0 con 0x7f4cb4101140 2024-04-16T05:40:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb9d72640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4112260 con 0x7f4cb410f1f0 2024-04-16T05:40:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb9d72640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4112560 con 0x7f4cb4106210 2024-04-16T05:40:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 0x7f4cb410e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 0x7f4cb410e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39674/0 (socket says 172.21.15.86:39674) 2024-04-16T05:40:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb37fe640 1 -- 172.21.15.86:0/920169429 learned_addr learned my addr 172.21.15.86:0/920169429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.381+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb4106210 0x7f4cb410ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb410f1f0 0x7f4cb41a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb37fe640 1 -- 172.21.15.86:0/920169429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb4106210 msgr2=0x7f4cb410ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb4106210 0x7f4cb410ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb37fe640 1 -- 172.21.15.86:0/920169429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb410f1f0 msgr2=0x7f4cb41a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb410f1f0 0x7f4cb41a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb37fe640 1 -- 172.21.15.86:0/920169429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb41a70d0 con 0x7f4cb4101140 2024-04-16T05:40:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb4106210 0x7f4cb410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb410f1f0 0x7f4cb41a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 0x7f4cb410e770 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4006010 tx=0x7f4ca402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb0ff9640 1 -- 172.21.15.86:0/920169429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4041070 con 0x7f4cb4101140 2024-04-16T05:40:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb0ff9640 1 -- 172.21.15.86:0/920169429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ca4004450 con 0x7f4cb4101140 2024-04-16T05:40:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb9d72640 1 -- 172.21.15.86:0/920169429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb41a7360 con 0x7f4cb4101140 2024-04-16T05:40:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb0ff9640 1 -- 172.21.15.86:0/920169429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4034440 con 0x7f4cb4101140 2024-04-16T05:40:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.382+0000 7f4cb9d72640 1 -- 172.21.15.86:0/920169429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb41a7850 con 0x7f4cb4101140 2024-04-16T05:40:16.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.385+0000 7f4cb0ff9640 1 -- 172.21.15.86:0/920169429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ca4044070 con 0x7f4cb4101140 2024-04-16T05:40:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.385+0000 7f4cbca37740 1 Processor -- start 2024-04-16T05:40:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.385+0000 7f4cbca37740 1 -- start start 2024-04-16T05:40:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.385+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734795e50 0x561734796230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.385+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734796770 0x56173479ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.385+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.385+0000 7f4cbca37740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5617346fb180 con 0x56173479f350 2024-04-16T05:40:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.385+0000 7f4cbca37740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561734698360 con 0x561734796770 2024-04-16T05:40:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.385+0000 7f4cbca37740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5617346f7140 con 0x561734795e50 2024-04-16T05:40:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39686/0 (socket says 172.21.15.86:39686) 2024-04-16T05:40:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb2ffd640 1 -- 172.21.15.86:0/670816789 learned_addr learned my addr 172.21.15.86:0/670816789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734795e50 0x561734796230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb0ff9640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4ca00a3d50 0x7f4ca00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb2ffd640 1 -- 172.21.15.86:0/670816789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734795e50 msgr2=0x561734796230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734795e50 0x561734796230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734796770 0x56173479ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb2ffd640 1 -- 172.21.15.86:0/670816789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734796770 msgr2=0x56173479ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4ca00a3d50 0x7f4ca00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734796770 0x56173479ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb2ffd640 1 -- 172.21.15.86:0/670816789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617344d77d0 con 0x56173479f350 2024-04-16T05:40:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734796770 0x56173479ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb0ff9640 1 -- 172.21.15.86:0/920169429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4ca40f2720 con 0x7f4cb4101140 2024-04-16T05:40:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347a1740 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c008c00 tx=0x7f4c9c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d9e59248b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4ca9ffb640 1 -- 172.21.15.86:0/670816789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c014070 con 0x56173479f350 2024-04-16T05:40:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4ca00a3d50 0x7f4ca00a6210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4cac005fd0 tx=0x7f4cac004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4ca9ffb640 1 -- 172.21.15.86:0/670816789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c9c0047a0 con 0x56173479f350 2024-04-16T05:40:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.386+0000 7f4ca9ffb640 1 -- 172.21.15.86:0/670816789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c010760 con 0x56173479f350 2024-04-16T05:40:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 -- 172.21.15.86:0/670816789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 msgr2=0x5617347a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347a1740 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c008c00 tx=0x7f4c9c00e720 comp rx=0 tx=0).stop 2024-04-16T05:40:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 -- 172.21.15.86:0/670816789 shutdown_connections 2024-04-16T05:40:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347a1740 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734796770 0x56173479ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 --2- 172.21.15.86:0/670816789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734795e50 0x561734796230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 -- 172.21.15.86:0/670816789 >> 172.21.15.86:0/670816789 conn(0x561734790580 msgr2=0x561734790960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 -- 172.21.15.86:0/670816789 shutdown_connections 2024-04-16T05:40:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 -- 172.21.15.86:0/670816789 wait complete. 2024-04-16T05:40:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.387+0000 7f4cbca37740 1 -- start start 2024-04-16T05:40:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cbca37740 1 Processor -- start 2024-04-16T05:40:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 0x56173479c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x56173479c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x56173479ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cbca37740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5617347a2140 con 0x56173479f350 2024-04-16T05:40:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cbca37740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5617346b42c0 con 0x561734795e50 2024-04-16T05:40:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cbca37740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5617347a22c0 con 0x561734796770 2024-04-16T05:40:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cbca37740 10 client.0 _ll_register_callbacks cb 0x56173469b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x56173479ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x56173479ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39700/0 (socket says 172.21.15.86:39700) 2024-04-16T05:40:16.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb3fff640 1 -- 172.21.15.86:0/1567908515 learned_addr learned my addr 172.21.15.86:0/1567908515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 0x56173479c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x56173479c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb3fff640 1 -- 172.21.15.86:0/1567908515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 msgr2=0x56173479c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x56173479c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb3fff640 1 -- 172.21.15.86:0/1567908515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 msgr2=0x56173479c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 0x56173479c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.388+0000 7f4cb3fff640 1 -- 172.21.15.86:0/1567908515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173440d3f0 con 0x56173479f350 2024-04-16T05:40:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x56173479ce30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0138b0 tx=0x7f4cac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9046c8456cf3d420 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4c92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4c92ffd640 1 -- 172.21.15.86:0/1567908515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac004d20 con 0x56173479f350 2024-04-16T05:40:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4c92ffd640 1 -- 172.21.15.86:0/1567908515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4cac01e8e0 con 0x56173479f350 2024-04-16T05:40:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cbca37740 1 -- 172.21.15.86:0/1567908515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c9c0047a0 con 0x56173479f350 2024-04-16T05:40:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4c92ffd640 1 -- 172.21.15.86:0/1567908515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4cac01eae0 con 0x56173479f350 2024-04-16T05:40:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cbca37740 10 client.17118 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cbca37740 1 -- 172.21.15.86:0/1567908515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5617343cd0f0 con 0x56173479f350 2024-04-16T05:40:16.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4c92ffd640 1 -- 172.21.15.86:0/1567908515 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c9c0047a0 con 0x56173479f350 2024-04-16T05:40:16.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4c92ffd640 1 -- 172.21.15.86:0/1567908515 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4cac00f040 con 0x56173479f350 2024-04-16T05:40:16.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cbca37740 10 client.17118 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:16.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cbca37740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:16.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cbca37740 20 client.17118 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:16.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cbca37740 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5617347af6d0 0x5617347b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cbca37740 4 client.17118 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:16.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.389+0000 7f4cbca37740 1 -- 172.21.15.86:0/1567908515 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561734708ba0 con 0x5617347af6d0 2024-04-16T05:40:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.390+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5617347af6d0 0x5617347b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.390+0000 7f4c92ffd640 10 client.17118 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.390+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5617347af6d0 0x5617347b1b20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b4fbcf9b2c0ebef server_cookie=871da6cb3aef32d in_seq=0 out_seq=0 2024-04-16T05:40:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4c92ffd640 1 -- 172.21.15.86:0/1567908515 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561734708ba0 con 0x5617347af6d0 2024-04-16T05:40:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4c92ffd640 10 client.17118 handle_command_reply: tid=0 2024-04-16T05:40:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 client.17118 shutdown 2024-04-16T05:40:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 -- 172.21.15.86:0/1567908515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 msgr2=0x56173479ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x56173479ce30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0138b0 tx=0x7f4cac004ab0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 -- 172.21.15.86:0/1567908515 shutdown_connections 2024-04-16T05:40:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5617347af6d0 0x5617347b1b20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x56173479ce30 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x56173479c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4c92ffd640 0 client.17118 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 --2- 172.21.15.86:0/1567908515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 0x56173479c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 -- 172.21.15.86:0/1567908515 >> 172.21.15.86:0/1567908515 conn(0x561734790580 msgr2=0x5617347922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 -- 172.21.15.86:0/1567908515 shutdown_connections 2024-04-16T05:40:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 1 -- 172.21.15.86:0/1567908515 wait complete. 2024-04-16T05:40:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.391+0000 7f4cbca37740 20 client.17118 trim_cache size 0 max 16384 2024-04-16T05:40:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.408+0000 7f4cbca37740 1 Processor -- start 2024-04-16T05:40:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.408+0000 7f4cbca37740 1 -- start start 2024-04-16T05:40:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.408+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 0x56173479eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.408+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x5617347ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.408+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.408+0000 7f4cbca37740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41b1510 con 0x56173479f350 2024-04-16T05:40:16.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.408+0000 7f4cbca37740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41b3050 con 0x561734795e50 2024-04-16T05:40:16.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.408+0000 7f4cbca37740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41b1be0 con 0x561734796770 2024-04-16T05:40:16.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347b73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39702/0 (socket says 172.21.15.86:39702) 2024-04-16T05:40:16.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb3fff640 1 -- 172.21.15.86:0/2298745054 learned_addr learned my addr 172.21.15.86:0/2298745054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/2298745054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x5617347ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/2298745054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 0x56173479eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb3fff640 1 -- 172.21.15.86:0/2298745054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 msgr2=0x5617347ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/2298745054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x5617347ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb3fff640 1 -- 172.21.15.86:0/2298745054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 msgr2=0x56173479eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/2298745054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 0x56173479eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb3fff640 1 -- 172.21.15.86:0/2298745054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617347965a0 con 0x56173479f350 2024-04-16T05:40:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/2298745054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347b73f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4cac01e9c0 tx=0x7f4cac01e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b27d8bf1d70be7e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4ca97fa640 1 -- 172.21.15.86:0/2298745054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac038020 con 0x56173479f350 2024-04-16T05:40:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4ca97fa640 1 -- 172.21.15.86:0/2298745054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cac023070 con 0x56173479f350 2024-04-16T05:40:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.409+0000 7f4ca97fa640 1 -- 172.21.15.86:0/2298745054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0052a0 con 0x56173479f350 2024-04-16T05:40:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 -- 172.21.15.86:0/2298745054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 msgr2=0x5617347b73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 --2- 172.21.15.86:0/2298745054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347b73f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4cac01e9c0 tx=0x7f4cac01e9f0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 -- 172.21.15.86:0/2298745054 shutdown_connections 2024-04-16T05:40:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 --2- 172.21.15.86:0/2298745054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56173479f350 0x5617347b73f0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 --2- 172.21.15.86:0/2298745054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x5617347ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 --2- 172.21.15.86:0/2298745054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561734795e50 0x56173479eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 -- 172.21.15.86:0/2298745054 >> 172.21.15.86:0/2298745054 conn(0x561734790580 msgr2=0x56173478f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 -- 172.21.15.86:0/2298745054 shutdown_connections 2024-04-16T05:40:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 -- 172.21.15.86:0/2298745054 wait complete. 2024-04-16T05:40:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 -- start start 2024-04-16T05:40:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.410+0000 7f4cbca37740 1 Processor -- start 2024-04-16T05:40:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561734795e50 0x5617347a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x5617347a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173479f350 0x5617347a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41b2980 con 0x561734795e50 2024-04-16T05:40:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41b22b0 con 0x56173479f350 2024-04-16T05:40:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41af4d0 con 0x561734796770 2024-04-16T05:40:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 10 client.0 _ll_register_callbacks cb 0x7f4cac009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561734795e50 0x5617347a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561734795e50 0x5617347a51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39704/0 (socket says 172.21.15.86:39704) 2024-04-16T05:40:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb37fe640 1 -- 172.21.15.86:0/3470023987 learned_addr learned my addr 172.21.15.86:0/3470023987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x5617347a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173479f350 0x5617347a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb37fe640 1 -- 172.21.15.86:0/3470023987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 msgr2=0x5617347a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x5617347a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb37fe640 1 -- 172.21.15.86:0/3470023987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173479f350 msgr2=0x5617347a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173479f350 0x5617347a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb37fe640 1 -- 172.21.15.86:0/3470023987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561734814f10 con 0x561734795e50 2024-04-16T05:40:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb3fff640 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173479f350 0x5617347a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cb37fe640 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561734795e50 0x5617347a51c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40efc70 tx=0x7f4ca408ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9712dc71524433d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4c92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4c92ffd640 1 -- 172.21.15.86:0/3470023987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca404b040 con 0x561734795e50 2024-04-16T05:40:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4c92ffd640 1 -- 172.21.15.86:0/3470023987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4ca40035f0 con 0x561734795e50 2024-04-16T05:40:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 1 -- 172.21.15.86:0/3470023987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cac023070 con 0x561734795e50 2024-04-16T05:40:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4c92ffd640 1 -- 172.21.15.86:0/3470023987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ca40be460 con 0x561734795e50 2024-04-16T05:40:16.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 10 client.17124 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:16.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.411+0000 7f4cbca37740 1 -- 172.21.15.86:0/3470023987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561734816200 con 0x561734795e50 2024-04-16T05:40:16.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4c92ffd640 1 -- 172.21.15.86:0/3470023987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cac023070 con 0x561734795e50 2024-04-16T05:40:16.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4c92ffd640 1 -- 172.21.15.86:0/3470023987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4ca40f2030 con 0x561734795e50 2024-04-16T05:40:16.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4cbca37740 10 client.17124 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:16.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4cbca37740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:16.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4cbca37740 20 client.17124 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:16.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4cbca37740 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5617348168d0 0x561734818cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4cbca37740 4 client.17124 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:40:16.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4cbca37740 1 -- 172.21.15.86:0/3470023987 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561734702a10 con 0x5617348168d0 2024-04-16T05:40:16.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5617348168d0 0x561734818cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4c92ffd640 10 client.17124 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:16.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.412+0000 7f4cb2ffd640 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5617348168d0 0x561734818cf0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc7f6989b0167a1 server_cookie=f18fe45cf57763cd in_seq=0 out_seq=0 2024-04-16T05:40:16.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4c92ffd640 1 -- 172.21.15.86:0/3470023987 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561734702a10 con 0x5617348168d0 2024-04-16T05:40:16.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4c92ffd640 10 client.17124 handle_command_reply: tid=0 2024-04-16T05:40:16.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 client.17124 shutdown 2024-04-16T05:40:16.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 -- 172.21.15.86:0/3470023987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561734795e50 msgr2=0x5617347a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561734795e50 0x5617347a51c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40efc70 tx=0x7f4ca408ce10 comp rx=0 tx=0).stop 2024-04-16T05:40:16.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 -- 172.21.15.86:0/3470023987 shutdown_connections 2024-04-16T05:40:16.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5617348168d0 0x561734818cf0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173479f350 0x5617347a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561734796770 0x5617347a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4c92ffd640 0 client.17124 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:16.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 --2- 172.21.15.86:0/3470023987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561734795e50 0x5617347a51c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 -- 172.21.15.86:0/3470023987 >> 172.21.15.86:0/3470023987 conn(0x561734790580 msgr2=0x56173479c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 -- 172.21.15.86:0/3470023987 shutdown_connections 2024-04-16T05:40:16.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 1 -- 172.21.15.86:0/3470023987 wait complete. 2024-04-16T05:40:16.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cbca37740 20 client.17124 trim_cache size 0 max 16384 2024-04-16T05:40:16.425 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:40:16.425 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:40:16.426 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:40:16.426 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:40:16.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cb9d72640 1 -- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4ca00a3d50 msgr2=0x7f4ca00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4ca00a3d50 0x7f4ca00a6210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4cac005fd0 tx=0x7f4cac004a40 comp rx=0 tx=0).stop 2024-04-16T05:40:16.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cb9d72640 1 -- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 msgr2=0x7f4cb410e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 0x7f4cb410e770 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4006010 tx=0x7f4ca402cc70 comp rx=0 tx=0).stop 2024-04-16T05:40:16.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cb9d72640 1 -- 172.21.15.86:0/920169429 shutdown_connections 2024-04-16T05:40:16.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4ca00a3d50 0x7f4ca00a6210 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.413+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb410f1f0 0x7f4cb41a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.414+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb4106210 0x7f4cb410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.414+0000 7f4cb9d72640 1 --2- 172.21.15.86:0/920169429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb4101140 0x7f4cb410e770 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.414+0000 7f4cb9d72640 1 -- 172.21.15.86:0/920169429 >> 172.21.15.86:0/920169429 conn(0x7f4cb4078000 msgr2=0x7f4cb40fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.414+0000 7f4cb9d72640 1 -- 172.21.15.86:0/920169429 shutdown_connections 2024-04-16T05:40:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.414+0000 7f4cb9d72640 1 -- 172.21.15.86:0/920169429 wait complete. 2024-04-16T05:40:16.429 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:40:16.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:40:16.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.536+0000 7f4c19da8640 1 -- 172.21.15.86:0/3153855087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102bf0 msgr2=0x7f4c14106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.536+0000 7f4c19da8640 1 --2- 172.21.15.86:0/3153855087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102bf0 0x7f4c14106c30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009900 tx=0x7f4c0802bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:16.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.536+0000 7f4c19da8640 1 -- 172.21.15.86:0/3153855087 shutdown_connections 2024-04-16T05:40:16.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.536+0000 7f4c19da8640 1 --2- 172.21.15.86:0/3153855087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c14107310 0x7f4c1410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.536+0000 7f4c19da8640 1 --2- 172.21.15.86:0/3153855087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102bf0 0x7f4c14106c30 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.536+0000 7f4c19da8640 1 --2- 172.21.15.86:0/3153855087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c14102240 0x7f4c14102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.536+0000 7f4c19da8640 1 -- 172.21.15.86:0/3153855087 >> 172.21.15.86:0/3153855087 conn(0x7f4c140793c0 msgr2=0x7f4c140797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 -- 172.21.15.86:0/3153855087 shutdown_connections 2024-04-16T05:40:16.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 -- 172.21.15.86:0/3153855087 wait complete. 2024-04-16T05:40:16.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 Processor -- start 2024-04-16T05:40:16.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 -- start start 2024-04-16T05:40:16.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102240 0x7f4c1419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c14107310 0x7f4c1419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c14198700 0x7f4c14198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14112380 con 0x7f4c14102240 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14112200 con 0x7f4c14198700 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.537+0000 7f4c19da8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14112500 con 0x7f4c14107310 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102240 0x7f4c1419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102240 0x7f4c1419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39728/0 (socket says 172.21.15.86:39728) 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c137fe640 1 -- 172.21.15.86:0/4085079949 learned_addr learned my addr 172.21.15.86:0/4085079949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c12ffd640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c14107310 0x7f4c1419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c137fe640 1 -- 172.21.15.86:0/4085079949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c14107310 msgr2=0x7f4c1419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c137fe640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c14107310 0x7f4c1419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c137fe640 1 -- 172.21.15.86:0/4085079949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c14198700 msgr2=0x7f4c14198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c137fe640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c14198700 0x7f4c14198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c137fe640 1 -- 172.21.15.86:0/4085079949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c14199300 con 0x7f4c14102240 2024-04-16T05:40:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c12ffd640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c14107310 0x7f4c1419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c137fe640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102240 0x7f4c1419e640 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400d950 tx=0x7f4c0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c10ff9640 1 -- 172.21.15.86:0/4085079949 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04017070 con 0x7f4c14102240 2024-04-16T05:40:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c10ff9640 1 -- 172.21.15.86:0/4085079949 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c04009c30 con 0x7f4c14102240 2024-04-16T05:40:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c141a53e0 con 0x7f4c14102240 2024-04-16T05:40:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c10ff9640 1 -- 172.21.15.86:0/4085079949 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400f6b0 con 0x7f4c14102240 2024-04-16T05:40:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.538+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c141a5790 con 0x7f4c14102240 2024-04-16T05:40:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.539+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c14103710 con 0x7f4c14102240 2024-04-16T05:40:16.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.540+0000 7f4c10ff9640 1 -- 172.21.15.86:0/4085079949 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4c040109d0 con 0x7f4c14102240 2024-04-16T05:40:16.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.541+0000 7f4c10ff9640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4bfc0a3d20 0x7f4bfc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.541+0000 7f4c12ffd640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4bfc0a3d20 0x7f4bfc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.541+0000 7f4c10ff9640 1 -- 172.21.15.86:0/4085079949 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4c040d0850 con 0x7f4c14102240 2024-04-16T05:40:16.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.542+0000 7f4c12ffd640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4bfc0a3d20 0x7f4bfc0a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006010 tx=0x7f4c0802c710 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.546+0000 7f4c10ff9640 1 -- 172.21.15.86:0/4085079949 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c04009de0 con 0x7f4c14102240 2024-04-16T05:40:16.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:16 smithi194 ceph-mon[35865]: pgmap v152: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:40:16.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:16 smithi086 ceph-mon[32028]: pgmap v152: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:40:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:16 smithi196 ceph-mon[36373]: pgmap v152: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:40:16.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.765+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c14102bf0 con 0x7f4c14102240 2024-04-16T05:40:16.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.766+0000 7f4c10ff9640 1 -- 172.21.15.86:0/4085079949 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f4c0409cb80 con 0x7f4c14102240 2024-04-16T05:40:16.765 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:40:16.765 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:40:16.766 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:40:16.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4bfc0a3d20 msgr2=0x7f4bfc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4bfc0a3d20 0x7f4bfc0a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006010 tx=0x7f4c0802c710 comp rx=0 tx=0).stop 2024-04-16T05:40:16.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102240 msgr2=0x7f4c1419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102240 0x7f4c1419e640 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400d950 tx=0x7f4c0400de10 comp rx=0 tx=0).stop 2024-04-16T05:40:16.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 shutdown_connections 2024-04-16T05:40:16.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4bfc0a3d20 0x7f4bfc0a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c14198700 0x7f4c14198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c14107310 0x7f4c1419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 --2- 172.21.15.86:0/4085079949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c14102240 0x7f4c1419e640 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 >> 172.21.15.86:0/4085079949 conn(0x7f4c140793c0 msgr2=0x7f4c14100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 shutdown_connections 2024-04-16T05:40:16.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.768+0000 7f4c19da8640 1 -- 172.21.15.86:0/4085079949 wait complete. 2024-04-16T05:40:16.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:40:16.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.887+0000 7f74409a1640 1 -- 172.21.15.86:0/1709978334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c100560 msgr2=0x7f743c100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.887+0000 7f74409a1640 1 --2- 172.21.15.86:0/1709978334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c100560 0x7f743c100940 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7430009900 tx=0x7f743002bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.887+0000 7f74409a1640 1 -- 172.21.15.86:0/1709978334 shutdown_connections 2024-04-16T05:40:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.887+0000 7f74409a1640 1 --2- 172.21.15.86:0/1709978334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c107170 0x7f743c10ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.887+0000 7f74409a1640 1 --2- 172.21.15.86:0/1709978334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743c100f10 0x7f743c106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.887+0000 7f74409a1640 1 --2- 172.21.15.86:0/1709978334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c100560 0x7f743c100940 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.887+0000 7f74409a1640 1 -- 172.21.15.86:0/1709978334 >> 172.21.15.86:0/1709978334 conn(0x7f743c077280 msgr2=0x7f743c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.887+0000 7f74409a1640 1 -- 172.21.15.86:0/1709978334 shutdown_connections 2024-04-16T05:40:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.887+0000 7f74409a1640 1 -- 172.21.15.86:0/1709978334 wait complete. 2024-04-16T05:40:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f74409a1640 1 Processor -- start 2024-04-16T05:40:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f74409a1640 1 -- start start 2024-04-16T05:40:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f74409a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c100560 0x7f743c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f74409a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c107170 0x7f743c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f74409a1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743c19cfc0 0x7f743c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f74409a1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1121e0 con 0x7f743c107170 2024-04-16T05:40:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f74409a1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f743c112060 con 0x7f743c19cfc0 2024-04-16T05:40:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f74409a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f743c112360 con 0x7f743c100560 2024-04-16T05:40:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f743affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c107170 0x7f743c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f743affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c107170 0x7f743c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39744/0 (socket says 172.21.15.86:39744) 2024-04-16T05:40:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f743affd640 1 -- 172.21.15.86:0/3119312176 learned_addr learned my addr 172.21.15.86:0/3119312176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f743b7fe640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c100560 0x7f743c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.888+0000 7f743bfff640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743c19cfc0 0x7f743c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f743affd640 1 -- 172.21.15.86:0/3119312176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c100560 msgr2=0x7f743c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f743affd640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c100560 0x7f743c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f743affd640 1 -- 172.21.15.86:0/3119312176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743c19cfc0 msgr2=0x7f743c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f743affd640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743c19cfc0 0x7f743c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f743affd640 1 -- 172.21.15.86:0/3119312176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c19dab0 con 0x7f743c107170 2024-04-16T05:40:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f743affd640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c107170 0x7f743c198c40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f742c00d950 tx=0x7f742c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f7438ff9640 1 -- 172.21.15.86:0/3119312176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c017070 con 0x7f743c107170 2024-04-16T05:40:16.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f7438ff9640 1 -- 172.21.15.86:0/3119312176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f742c009c30 con 0x7f743c107170 2024-04-16T05:40:16.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f74409a1640 1 -- 172.21.15.86:0/3119312176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c19dda0 con 0x7f743c107170 2024-04-16T05:40:16.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f7438ff9640 1 -- 172.21.15.86:0/3119312176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c0047a0 con 0x7f743c107170 2024-04-16T05:40:16.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.889+0000 7f74409a1640 1 -- 172.21.15.86:0/3119312176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c1a5590 con 0x7f743c107170 2024-04-16T05:40:16.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.891+0000 7f7438ff9640 1 -- 172.21.15.86:0/3119312176 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f742c004020 con 0x7f743c107170 2024-04-16T05:40:16.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f74447df740 1 Processor -- start 2024-04-16T05:40:16.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f74447df740 1 -- start start 2024-04-16T05:40:16.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f7438ff9640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74240a3d20 0x7f74240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f74447df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f74447df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 0x55627e7f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f74447df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 0x55627e7f8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f74447df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55627e7520e0 con 0x55627e7ed030 2024-04-16T05:40:16.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f74447df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55627e6ef520 con 0x55627e7ed950 2024-04-16T05:40:16.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f74447df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55627e74e0a0 con 0x55627e7f6530 2024-04-16T05:40:16.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743bfff640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74240a3d20 0x7f74240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f7438ff9640 1 -- 172.21.15.86:0/3119312176 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f742c0d0760 con 0x7f743c107170 2024-04-16T05:40:16.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7ed410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39748/0 (socket says 172.21.15.86:39748) 2024-04-16T05:40:16.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743b7fe640 1 -- 172.21.15.86:0/886402071 learned_addr learned my addr 172.21.15.86:0/886402071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743affd640 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 0x55627e7f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743bfff640 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 0x55627e7f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743b7fe640 1 -- 172.21.15.86:0/886402071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 msgr2=0x55627e7f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743b7fe640 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 0x55627e7f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743b7fe640 1 -- 172.21.15.86:0/886402071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 msgr2=0x55627e7f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743b7fe640 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 0x55627e7f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743b7fe640 1 -- 172.21.15.86:0/886402071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55627e7178d0 con 0x55627e7ed030 2024-04-16T05:40:16.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743bfff640 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 0x55627e7f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:16.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743bfff640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74240a3d20 0x7f74240a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7434005fd0 tx=0x7f7434009a60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.892+0000 7f743b7fe640 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7ed410 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f743002c170 tx=0x7f743002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeff11f29bed084a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f7421ffb640 1 -- 172.21.15.86:0/886402071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430042070 con 0x55627e7ed030 2024-04-16T05:40:16.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f7421ffb640 1 -- 172.21.15.86:0/886402071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f743002ca80 con 0x55627e7ed030 2024-04-16T05:40:16.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f7421ffb640 1 -- 172.21.15.86:0/886402071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430005180 con 0x55627e7ed030 2024-04-16T05:40:16.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f74447df740 1 -- 172.21.15.86:0/886402071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 msgr2=0x55627e7ed410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f74447df740 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7ed410 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f743002c170 tx=0x7f743002c5d0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f74447df740 1 -- 172.21.15.86:0/886402071 shutdown_connections 2024-04-16T05:40:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f74447df740 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 0x55627e7f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f74447df740 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 0x55627e7f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f74447df740 1 --2- 172.21.15.86:0/886402071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7ed410 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f74447df740 1 -- 172.21.15.86:0/886402071 >> 172.21.15.86:0/886402071 conn(0x55627e7e7760 msgr2=0x55627e7e7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f74447df740 1 -- 172.21.15.86:0/886402071 shutdown_connections 2024-04-16T05:40:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.893+0000 7f74447df740 1 -- 172.21.15.86:0/886402071 wait complete. 2024-04-16T05:40:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f74447df740 1 -- start start 2024-04-16T05:40:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f74447df740 1 Processor -- start 2024-04-16T05:40:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f74447df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7ed030 0x55627e7f3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f74447df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 0x55627e7f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f74447df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7f6530 0x55627e7f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f74447df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55627e7f92c0 con 0x55627e7f6530 2024-04-16T05:40:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f74447df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55627e70b4d0 con 0x55627e7ed950 2024-04-16T05:40:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f74447df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55627e7f9440 con 0x55627e7ed030 2024-04-16T05:40:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f74447df740 10 client.0 _ll_register_callbacks cb 0x55627e6f02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7f6530 0x55627e7f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7f6530 0x55627e7f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39756/0 (socket says 172.21.15.86:39756) 2024-04-16T05:40:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743bfff640 1 -- 172.21.15.86:0/3547324772 learned_addr learned my addr 172.21.15.86:0/3547324772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743affd640 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 0x55627e7f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743b7fe640 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7ed030 0x55627e7f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743bfff640 1 -- 172.21.15.86:0/3547324772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7ed030 msgr2=0x55627e7f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743bfff640 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7ed030 0x55627e7f3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743bfff640 1 -- 172.21.15.86:0/3547324772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 msgr2=0x55627e7f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743bfff640 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 0x55627e7f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.894+0000 7f743bfff640 1 -- 172.21.15.86:0/3547324772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55627e4643f0 con 0x55627e7f6530 2024-04-16T05:40:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f743bfff640 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7f6530 0x55627e7f3fc0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7434013cb0 tx=0x7f7434004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c1f3a33dc0546 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f741affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:16.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f741affd640 1 -- 172.21.15.86:0/3547324772 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434020650 con 0x55627e7f6530 2024-04-16T05:40:16.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f741affd640 1 -- 172.21.15.86:0/3547324772 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f74340207f0 con 0x55627e7f6530 2024-04-16T05:40:16.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f74447df740 1 -- 172.21.15.86:0/3547324772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f743002ca80 con 0x55627e7f6530 2024-04-16T05:40:16.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f741affd640 1 -- 172.21.15.86:0/3547324772 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7434031050 con 0x55627e7f6530 2024-04-16T05:40:16.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f74447df740 10 client.17142 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:16.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f74447df740 1 -- 172.21.15.86:0/3547324772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55627e52e7d0 con 0x55627e7f6530 2024-04-16T05:40:16.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f741affd640 1 -- 172.21.15.86:0/3547324772 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f743002ca80 con 0x55627e7f6530 2024-04-16T05:40:16.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f741affd640 1 -- 172.21.15.86:0/3547324772 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f743401d040 con 0x55627e7f6530 2024-04-16T05:40:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f74447df740 10 client.17142 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f74447df740 10 client.17142 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:40:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f74447df740 20 client.17142 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f74447df740 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55627e806540 0x55627e808980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f74447df740 4 client.17142 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.895+0000 7f74447df740 1 -- 172.21.15.86:0/3547324772 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55627e75fb00 con 0x55627e806540 2024-04-16T05:40:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.896+0000 7f743b7fe640 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55627e806540 0x55627e808980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.896+0000 7f741affd640 10 client.17142 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:40:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.896+0000 7f743b7fe640 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55627e806540 0x55627e808980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1b7c26707992a30 server_cookie=7c4698d40c9f360 in_seq=0 out_seq=0 2024-04-16T05:40:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f741affd640 1 -- 172.21.15.86:0/3547324772 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55627e75fb00 con 0x55627e806540 2024-04-16T05:40:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f741affd640 10 client.17142 handle_command_reply: tid=0 2024-04-16T05:40:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 client.17142 shutdown 2024-04-16T05:40:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 -- 172.21.15.86:0/3547324772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7f6530 msgr2=0x55627e7f3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7f6530 0x55627e7f3fc0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7434013cb0 tx=0x7f7434004ab0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 -- 172.21.15.86:0/3547324772 shutdown_connections 2024-04-16T05:40:16.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55627e806540 0x55627e808980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7f6530 0x55627e7f3fc0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed950 0x55627e7f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f741affd640 0 client.17142 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:40:16.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 --2- 172.21.15.86:0/3547324772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7ed030 0x55627e7f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 -- 172.21.15.86:0/3547324772 >> 172.21.15.86:0/3547324772 conn(0x55627e7e7760 msgr2=0x55627e7e94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 -- 172.21.15.86:0/3547324772 shutdown_connections 2024-04-16T05:40:16.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.897+0000 7f74447df740 1 -- 172.21.15.86:0/3547324772 wait complete. 2024-04-16T05:40:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.898+0000 7f74447df740 20 client.17142 trim_cache size 0 max 16384 2024-04-16T05:40:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.910+0000 7f74447df740 1 Processor -- start 2024-04-16T05:40:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.910+0000 7f74447df740 1 -- start start 2024-04-16T05:40:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f74447df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f74447df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7ed950 0x55627e8021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f74447df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7f6530 0x55627e80e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f74447df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1af1f0 con 0x55627e7ed030 2024-04-16T05:40:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f74447df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b0d30 con 0x55627e7f6530 2024-04-16T05:40:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f74447df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1af8c0 con 0x55627e7ed950 2024-04-16T05:40:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f743b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f743b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39772/0 (socket says 172.21.15.86:39772) 2024-04-16T05:40:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f743b7fe640 1 -- 172.21.15.86:0/547740399 learned_addr learned my addr 172.21.15.86:0/547740399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f743b7fe640 1 -- 172.21.15.86:0/547740399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7ed950 msgr2=0x55627e8021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f743b7fe640 1 --2- 172.21.15.86:0/547740399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7ed950 0x55627e8021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f743b7fe640 1 -- 172.21.15.86:0/547740399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7f6530 msgr2=0x55627e80e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f743b7fe640 1 --2- 172.21.15.86:0/547740399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7f6530 0x55627e80e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f743b7fe640 1 -- 172.21.15.86:0/547740399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55627e805090 con 0x55627e7ed030 2024-04-16T05:40:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f743b7fe640 1 --2- 172.21.15.86:0/547740399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7f5c70 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f7430009900 tx=0x7f743002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e88fae20c1f8a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f74217fa640 1 -- 172.21.15.86:0/547740399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430042040 con 0x55627e7ed030 2024-04-16T05:40:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f74217fa640 1 -- 172.21.15.86:0/547740399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7430004030 con 0x55627e7ed030 2024-04-16T05:40:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.911+0000 7f74217fa640 1 -- 172.21.15.86:0/547740399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743004f910 con 0x55627e7ed030 2024-04-16T05:40:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 -- 172.21.15.86:0/547740399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 msgr2=0x55627e7f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 --2- 172.21.15.86:0/547740399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7f5c70 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f7430009900 tx=0x7f743002bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 -- 172.21.15.86:0/547740399 shutdown_connections 2024-04-16T05:40:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 --2- 172.21.15.86:0/547740399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7f6530 0x55627e80e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 --2- 172.21.15.86:0/547740399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7ed950 0x55627e8021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 --2- 172.21.15.86:0/547740399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed030 0x55627e7f5c70 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 -- 172.21.15.86:0/547740399 >> 172.21.15.86:0/547740399 conn(0x55627e7e7760 msgr2=0x55627e7e2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 -- 172.21.15.86:0/547740399 shutdown_connections 2024-04-16T05:40:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 -- 172.21.15.86:0/547740399 wait complete. 2024-04-16T05:40:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 -- start start 2024-04-16T05:40:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.912+0000 7f74447df740 1 Processor -- start 2024-04-16T05:40:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed030 0x55627e7fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed950 0x55627e7fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 0x55627e7efd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b0660 con 0x55627e7ed950 2024-04-16T05:40:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1aff90 con 0x55627e7ed030 2024-04-16T05:40:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1ad1b0 con 0x55627e7f6530 2024-04-16T05:40:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 10 client.0 _ll_register_callbacks cb 0x7f7434009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f743affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed950 0x55627e7fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f743affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed950 0x55627e7fec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39788/0 (socket says 172.21.15.86:39788) 2024-04-16T05:40:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f743affd640 1 -- 172.21.15.86:0/3957300982 learned_addr learned my addr 172.21.15.86:0/3957300982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f743affd640 1 -- 172.21.15.86:0/3957300982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 msgr2=0x55627e7efd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f743affd640 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 0x55627e7efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f743affd640 1 -- 172.21.15.86:0/3957300982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed030 msgr2=0x55627e7fe6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f743affd640 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed030 0x55627e7fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f743affd640 1 -- 172.21.15.86:0/3957300982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55627e7f0550 con 0x55627e7ed950 2024-04-16T05:40:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f743affd640 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed950 0x55627e7fec00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f742c0987f0 tx=0x7f742c09be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d735a2e427df70cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f741affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f741affd640 1 -- 172.21.15.86:0/3957300982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c0cdfc0 con 0x55627e7ed950 2024-04-16T05:40:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f741affd640 1 -- 172.21.15.86:0/3957300982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f742c0d03b0 con 0x55627e7ed950 2024-04-16T05:40:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 1 -- 172.21.15.86:0/3957300982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7430004030 con 0x55627e7ed950 2024-04-16T05:40:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f741affd640 1 -- 172.21.15.86:0/3957300982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f742c0d05b0 con 0x55627e7ed950 2024-04-16T05:40:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 10 client.17148 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f74447df740 1 -- 172.21.15.86:0/3957300982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55627e86c7c0 con 0x55627e7ed950 2024-04-16T05:40:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.913+0000 7f741affd640 1 -- 172.21.15.86:0/3957300982 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7430004030 con 0x55627e7ed950 2024-04-16T05:40:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f741affd640 1 -- 172.21.15.86:0/3957300982 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f742c0bb1a0 con 0x55627e7ed950 2024-04-16T05:40:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f74447df740 10 client.17148 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f74447df740 10 client.17148 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:40:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f74447df740 20 client.17148 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:16.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f74447df740 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55627e86cfc0 0x55627e86f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:16.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f74447df740 4 client.17148 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:16.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f74447df740 1 -- 172.21.15.86:0/3957300982 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55627e759970 con 0x55627e86cfc0 2024-04-16T05:40:16.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f743b7fe640 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55627e86cfc0 0x55627e86f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:16.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f741affd640 10 client.17148 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:40:16.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f743b7fe640 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55627e86cfc0 0x55627e86f3e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2596878064da54da server_cookie=966d7f6303d8e176 in_seq=0 out_seq=0 2024-04-16T05:40:16.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f741affd640 1 -- 172.21.15.86:0/3957300982 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55627e759970 con 0x55627e86cfc0 2024-04-16T05:40:16.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f741affd640 10 client.17148 handle_command_reply: tid=0 2024-04-16T05:40:16.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.914+0000 7f74447df740 1 client.17148 shutdown 2024-04-16T05:40:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 -- 172.21.15.86:0/3957300982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed950 msgr2=0x55627e7fec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed950 0x55627e7fec00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f742c0987f0 tx=0x7f742c09be20 comp rx=0 tx=0).stop 2024-04-16T05:40:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 -- 172.21.15.86:0/3957300982 shutdown_connections 2024-04-16T05:40:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55627e86cfc0 0x55627e86f3e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55627e7f6530 0x55627e7efd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55627e7ed950 0x55627e7fec00 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f741affd640 0 client.17148 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:40:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 --2- 172.21.15.86:0/3957300982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55627e7ed030 0x55627e7fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 -- 172.21.15.86:0/3957300982 >> 172.21.15.86:0/3957300982 conn(0x55627e7e7760 msgr2=0x55627e7e60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 -- 172.21.15.86:0/3957300982 shutdown_connections 2024-04-16T05:40:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 1 -- 172.21.15.86:0/3957300982 wait complete. 2024-04-16T05:40:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74447df740 20 client.17148 trim_cache size 0 max 16384 2024-04-16T05:40:16.925 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 -- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74240a3d20 msgr2=0x7f74240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74240a3d20 0x7f74240a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7434005fd0 tx=0x7f7434009a60 comp rx=0 tx=0).stop 2024-04-16T05:40:16.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 -- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c107170 msgr2=0x7f743c198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:16.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c107170 0x7f743c198c40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f742c00d950 tx=0x7f742c00de10 comp rx=0 tx=0).stop 2024-04-16T05:40:16.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 -- 172.21.15.86:0/3119312176 shutdown_connections 2024-04-16T05:40:16.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f74240a3d20 0x7f74240a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743c19cfc0 0x7f743c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c107170 0x7f743c198c40 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 --2- 172.21.15.86:0/3119312176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c100560 0x7f743c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 -- 172.21.15.86:0/3119312176 >> 172.21.15.86:0/3119312176 conn(0x7f743c077280 msgr2=0x7f743c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 -- 172.21.15.86:0/3119312176 shutdown_connections 2024-04-16T05:40:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:16.915+0000 7f74409a1640 1 -- 172.21.15.86:0/3119312176 wait complete. 2024-04-16T05:40:16.928 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:40:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 -- 172.21.15.86:0/2063483643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4101650 msgr2=0x7f4fe4101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 --2- 172.21.15.86:0/2063483643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4101650 0x7f4fe4101ad0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009900 tx=0x7f4fe002bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:17.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 -- 172.21.15.86:0/2063483643 shutdown_connections 2024-04-16T05:40:17.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 --2- 172.21.15.86:0/2063483643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe4102010 0x7f4fe410ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 --2- 172.21.15.86:0/2063483643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4101650 0x7f4fe4101ad0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 --2- 172.21.15.86:0/2063483643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe4105f30 0x7f4fe4106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 -- 172.21.15.86:0/2063483643 >> 172.21.15.86:0/2063483643 conn(0x7f4fe4077450 msgr2=0x7f4fe40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 -- 172.21.15.86:0/2063483643 shutdown_connections 2024-04-16T05:40:17.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 -- 172.21.15.86:0/2063483643 wait complete. 2024-04-16T05:40:17.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.036+0000 7f4feb406640 1 Processor -- start 2024-04-16T05:40:17.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4feb406640 1 -- start start 2024-04-16T05:40:17.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4feb406640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe4101650 0x7f4fe4197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4feb406640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe4102010 0x7f4fe4197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4feb406640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4105f30 0x7f4fe419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4feb406640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4111290 con 0x7f4fe4105f30 2024-04-16T05:40:17.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4feb406640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4111110 con 0x7f4fe4102010 2024-04-16T05:40:17.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4feb406640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4111410 con 0x7f4fe4101650 2024-04-16T05:40:17.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe99a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4105f30 0x7f4fe419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe99a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4105f30 0x7f4fe419bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39806/0 (socket says 172.21.15.86:39806) 2024-04-16T05:40:17.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe99a3640 1 -- 172.21.15.86:0/3439275332 learned_addr learned my addr 172.21.15.86:0/3439275332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe4101650 0x7f4fe4197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe4102010 0x7f4fe4197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe99a3640 1 -- 172.21.15.86:0/3439275332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe4101650 msgr2=0x7f4fe4197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe4101650 0x7f4fe4197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe99a3640 1 -- 172.21.15.86:0/3439275332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe4102010 msgr2=0x7f4fe4197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe4102010 0x7f4fe4197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe99a3640 1 -- 172.21.15.86:0/3439275332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe419c700 con 0x7f4fe4105f30 2024-04-16T05:40:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4105f30 0x7f4fe419bfd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00cad0 tx=0x7f4fdc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fd67fc640 1 -- 172.21.15.86:0/3439275332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc00ae20 con 0x7f4fe4105f30 2024-04-16T05:40:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4fd67fc640 1 -- 172.21.15.86:0/3439275332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc005020 con 0x7f4fe4105f30 2024-04-16T05:40:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.037+0000 7f4feb406640 1 -- 172.21.15.86:0/3439275332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe419c9f0 con 0x7f4fe4105f30 2024-04-16T05:40:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.038+0000 7f4fd67fc640 1 -- 172.21.15.86:0/3439275332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc00f6f0 con 0x7f4fe4105f30 2024-04-16T05:40:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.038+0000 7f4feb406640 1 -- 172.21.15.86:0/3439275332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe41a4280 con 0x7f4fe4105f30 2024-04-16T05:40:17.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.040+0000 7f4fd67fc640 1 -- 172.21.15.86:0/3439275332 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4fdc0051d0 con 0x7f4fe4105f30 2024-04-16T05:40:17.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.040+0000 7f4fedff2740 1 Processor -- start 2024-04-16T05:40:17.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.040+0000 7f4fedff2740 1 -- start start 2024-04-16T05:40:17.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.040+0000 7f4fd67fc640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4fd00a3d20 0x7f4fd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.040+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852070 0x5575c5852450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.040+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852990 0x5575c585b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c585d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fedff2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575c57b50e0 con 0x5575c585b570 2024-04-16T05:40:17.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fedff2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5575c5752520 con 0x5575c5852070 2024-04-16T05:40:17.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fedff2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575c57b10a0 con 0x5575c5852990 2024-04-16T05:40:17.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe99a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c585d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4fd00a3d20 0x7f4fd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe91a2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852070 0x5575c5852450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe99a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c585d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39810/0 (socket says 172.21.15.86:39810) 2024-04-16T05:40:17.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fd67fc640 1 -- 172.21.15.86:0/3439275332 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4fdc0d0cf0 con 0x7f4fe4105f30 2024-04-16T05:40:17.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe99a3640 1 -- 172.21.15.86:0/4179837787 learned_addr learned my addr 172.21.15.86:0/4179837787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/4179837787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852990 0x5575c585b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe99a3640 1 -- 172.21.15.86:0/4179837787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852990 msgr2=0x5575c585b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/4179837787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852990 0x5575c585b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe99a3640 1 -- 172.21.15.86:0/4179837787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852070 msgr2=0x5575c5852450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/4179837787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852070 0x5575c5852450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe99a3640 1 -- 172.21.15.86:0/4179837787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575c577a8d0 con 0x5575c585b570 2024-04-16T05:40:17.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4fd00a3d20 0x7f4fd00a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40ff9e0 tx=0x7f4fe00042c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/4179837787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c585d960 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc012040 tx=0x7f4fdc09c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4790b0653a17fc33 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fcf7fe640 1 -- 172.21.15.86:0/4179837787 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc01d070 con 0x5575c585b570 2024-04-16T05:40:17.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fcf7fe640 1 -- 172.21.15.86:0/4179837787 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc09c920 con 0x5575c585b570 2024-04-16T05:40:17.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.041+0000 7f4fcf7fe640 1 -- 172.21.15.86:0/4179837787 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc085e60 con 0x5575c585b570 2024-04-16T05:40:17.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 -- 172.21.15.86:0/4179837787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 msgr2=0x5575c585d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 --2- 172.21.15.86:0/4179837787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c585d960 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc012040 tx=0x7f4fdc09c560 comp rx=0 tx=0).stop 2024-04-16T05:40:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 -- 172.21.15.86:0/4179837787 shutdown_connections 2024-04-16T05:40:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 --2- 172.21.15.86:0/4179837787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c585d960 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 --2- 172.21.15.86:0/4179837787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852990 0x5575c585b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 --2- 172.21.15.86:0/4179837787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852070 0x5575c5852450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 -- 172.21.15.86:0/4179837787 >> 172.21.15.86:0/4179837787 conn(0x5575c584c7a0 msgr2=0x5575c584cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 -- 172.21.15.86:0/4179837787 shutdown_connections 2024-04-16T05:40:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 -- 172.21.15.86:0/4179837787 wait complete. 2024-04-16T05:40:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 -- start start 2024-04-16T05:40:17.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.042+0000 7f4fedff2740 1 Processor -- start 2024-04-16T05:40:17.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c58584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c58589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 0x5575c5858f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fedff2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575c585e360 con 0x5575c5852990 2024-04-16T05:40:17.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fedff2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5575c576e4d0 con 0x5575c585b570 2024-04-16T05:40:17.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fedff2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575c585e4e0 con 0x5575c5852070 2024-04-16T05:40:17.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fedff2740 10 client.0 _ll_register_callbacks cb 0x5575c57532c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:17.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe89a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c58589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe89a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c58589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39814/0 (socket says 172.21.15.86:39814) 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe89a1640 1 -- 172.21.15.86:0/982368818 learned_addr learned my addr 172.21.15.86:0/982368818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c58584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 0x5575c5858f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe89a1640 1 -- 172.21.15.86:0/982368818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 msgr2=0x5575c58584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c58584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe89a1640 1 -- 172.21.15.86:0/982368818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 msgr2=0x5575c5858f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 0x5575c5858f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe89a1640 1 -- 172.21.15.86:0/982368818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575c54c73f0 con 0x5575c5852990 2024-04-16T05:40:17.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c58584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:17.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c58589f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009900 tx=0x7f4fe0044040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae378ffd4ac117ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.043+0000 7f4fccff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:17.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fccff9640 1 -- 172.21.15.86:0/982368818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0005110 con 0x5575c5852990 2024-04-16T05:40:17.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fccff9640 1 -- 172.21.15.86:0/982368818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4fe00052b0 con 0x5575c5852990 2024-04-16T05:40:17.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fedff2740 1 -- 172.21.15.86:0/982368818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fdc09c920 con 0x5575c5852990 2024-04-16T05:40:17.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fccff9640 1 -- 172.21.15.86:0/982368818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4fe005c050 con 0x5575c5852990 2024-04-16T05:40:17.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fedff2740 10 client.17160 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:17.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fedff2740 1 -- 172.21.15.86:0/982368818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5575c55917d0 con 0x5575c5852990 2024-04-16T05:40:17.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fccff9640 1 -- 172.21.15.86:0/982368818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc09c920 con 0x5575c5852990 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fccff9640 1 -- 172.21.15.86:0/982368818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4fe0003c90 con 0x5575c5852990 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fedff2740 10 client.17160 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fedff2740 10 client.17160 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fedff2740 20 client.17160 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fedff2740 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5575c586b480 0x5575c586d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fedff2740 4 client.17160 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fedff2740 1 -- 172.21.15.86:0/982368818 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575c57c2b00 con 0x5575c586b480 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.044+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5575c586b480 0x5575c586d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.045+0000 7f4fccff9640 10 client.17160 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:40:17.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.045+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5575c586b480 0x5575c586d940 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9acacbf8852b980 server_cookie=2aef0f48bded33cb in_seq=0 out_seq=0 2024-04-16T05:40:17.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fccff9640 1 -- 172.21.15.86:0/982368818 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575c57c2b00 con 0x5575c586b480 2024-04-16T05:40:17.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fccff9640 10 client.17160 handle_command_reply: tid=0 2024-04-16T05:40:17.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 client.17160 shutdown 2024-04-16T05:40:17.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 -- 172.21.15.86:0/982368818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 msgr2=0x5575c58589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c58589f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009900 tx=0x7f4fe0044040 comp rx=0 tx=0).stop 2024-04-16T05:40:17.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 -- 172.21.15.86:0/982368818 shutdown_connections 2024-04-16T05:40:17.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 0x5575c5858f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c58589f0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5575c586b480 0x5575c586d940 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 --2- 172.21.15.86:0/982368818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c58584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fccff9640 0 client.17160 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:40:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 -- 172.21.15.86:0/982368818 >> 172.21.15.86:0/982368818 conn(0x5575c584c7a0 msgr2=0x5575c5849730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 -- 172.21.15.86:0/982368818 shutdown_connections 2024-04-16T05:40:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 1 -- 172.21.15.86:0/982368818 wait complete. 2024-04-16T05:40:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.046+0000 7f4fedff2740 20 client.17160 trim_cache size 0 max 16384 2024-04-16T05:40:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.060+0000 7f4fedff2740 1 Processor -- start 2024-04-16T05:40:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.060+0000 7f4fedff2740 1 -- start start 2024-04-16T05:40:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c585abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c5867220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 0x5575c5873210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fedff2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41ad2b0 con 0x5575c5852990 2024-04-16T05:40:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fedff2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41aedf0 con 0x5575c585b570 2024-04-16T05:40:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fedff2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41ad980 con 0x5575c5852070 2024-04-16T05:40:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe89a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c5867220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe89a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c5867220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39820/0 (socket says 172.21.15.86:39820) 2024-04-16T05:40:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe89a1640 1 -- 172.21.15.86:0/772544305 learned_addr learned my addr 172.21.15.86:0/772544305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c585abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 0x5575c5873210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe89a1640 1 -- 172.21.15.86:0/772544305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 msgr2=0x5575c585abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c585abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe89a1640 1 -- 172.21.15.86:0/772544305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 msgr2=0x5575c5873210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 0x5575c5873210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe89a1640 1 -- 172.21.15.86:0/772544305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575c584c5d0 con 0x5575c5852990 2024-04-16T05:40:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c585abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c5867220 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0004350 tx=0x7f4fe00058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c38440d7316f300 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fceffd640 1 -- 172.21.15.86:0/772544305 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0005110 con 0x5575c5852990 2024-04-16T05:40:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fceffd640 1 -- 172.21.15.86:0/772544305 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe0053070 con 0x5575c5852990 2024-04-16T05:40:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.061+0000 7f4fceffd640 1 -- 172.21.15.86:0/772544305 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe004da70 con 0x5575c5852990 2024-04-16T05:40:17.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 -- 172.21.15.86:0/772544305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 msgr2=0x5575c5867220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c5867220 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0004350 tx=0x7f4fe00058f0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 -- 172.21.15.86:0/772544305 shutdown_connections 2024-04-16T05:40:17.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c585b570 0x5575c5873210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c5852990 0x5575c5867220 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 --2- 172.21.15.86:0/772544305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c585abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 -- 172.21.15.86:0/772544305 >> 172.21.15.86:0/772544305 conn(0x5575c584c7a0 msgr2=0x5575c5849730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 -- 172.21.15.86:0/772544305 shutdown_connections 2024-04-16T05:40:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 -- 172.21.15.86:0/772544305 wait complete. 2024-04-16T05:40:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.062+0000 7f4fedff2740 1 -- start start 2024-04-16T05:40:17.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fedff2740 1 Processor -- start 2024-04-16T05:40:17.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c5855160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852990 0x5575c58556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fedff2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c5855be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fedff2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41ae720 con 0x5575c585b570 2024-04-16T05:40:17.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fedff2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41ae050 con 0x5575c5852990 2024-04-16T05:40:17.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fedff2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41ab270 con 0x5575c5852070 2024-04-16T05:40:17.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fedff2740 10 client.0 _ll_register_callbacks cb 0x7f4fe002cc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:17.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe99a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c5855be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe99a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c5855be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39822/0 (socket says 172.21.15.86:39822) 2024-04-16T05:40:17.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe99a3640 1 -- 172.21.15.86:0/1431499353 learned_addr learned my addr 172.21.15.86:0/1431499353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c5855160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe99a3640 1 -- 172.21.15.86:0/1431499353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 msgr2=0x5575c5855160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c5855160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852990 0x5575c58556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe99a3640 1 -- 172.21.15.86:0/1431499353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852990 msgr2=0x5575c58556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852990 0x5575c58556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe99a3640 1 -- 172.21.15.86:0/1431499353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575c58d0a80 con 0x5575c585b570 2024-04-16T05:40:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe89a1640 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852990 0x5575c58556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fe99a3640 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c5855be0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc09c5d0 tx=0x7f4fdc09c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155702283141b0c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.063+0000 7f4fccff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fccff9640 1 -- 172.21.15.86:0/1431499353 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc01d070 con 0x5575c585b570 2024-04-16T05:40:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fccff9640 1 -- 172.21.15.86:0/1431499353 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4fdc0ce350 con 0x5575c585b570 2024-04-16T05:40:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fedff2740 1 -- 172.21.15.86:0/1431499353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe0053070 con 0x5575c585b570 2024-04-16T05:40:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fccff9640 1 -- 172.21.15.86:0/1431499353 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4fdc085e60 con 0x5575c585b570 2024-04-16T05:40:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fedff2740 10 client.17166 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fedff2740 1 -- 172.21.15.86:0/1431499353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5575c58d1ca0 con 0x5575c585b570 2024-04-16T05:40:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fccff9640 1 -- 172.21.15.86:0/1431499353 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe0053070 con 0x5575c585b570 2024-04-16T05:40:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fccff9640 1 -- 172.21.15.86:0/1431499353 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4fdc09bd70 con 0x5575c585b570 2024-04-16T05:40:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fedff2740 10 client.17166 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fedff2740 10 client.17166 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:40:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fedff2740 20 client.17166 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fedff2740 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5575c58d1fa0 0x5575c58d4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fedff2740 4 client.17166 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fedff2740 1 -- 172.21.15.86:0/1431499353 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575c57bc970 con 0x5575c58d1fa0 2024-04-16T05:40:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.064+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5575c58d1fa0 0x5575c58d4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fccff9640 10 client.17166 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:40:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fe91a2640 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5575c58d1fa0 0x5575c58d4390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c37e3cde9ba6ccd5 server_cookie=3b447b01acbe54ac in_seq=0 out_seq=0 2024-04-16T05:40:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fccff9640 1 -- 172.21.15.86:0/1431499353 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575c57bc970 con 0x5575c58d1fa0 2024-04-16T05:40:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fccff9640 10 client.17166 handle_command_reply: tid=0 2024-04-16T05:40:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 client.17166 shutdown 2024-04-16T05:40:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 -- 172.21.15.86:0/1431499353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 msgr2=0x5575c5855be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c5855be0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc09c5d0 tx=0x7f4fdc09c9a0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 -- 172.21.15.86:0/1431499353 shutdown_connections 2024-04-16T05:40:17.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575c585b570 0x5575c5855be0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575c5852990 0x5575c58556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5575c58d1fa0 0x5575c58d4390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 --2- 172.21.15.86:0/1431499353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575c5852070 0x5575c5855160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fccff9640 0 client.17166 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:40:17.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 -- 172.21.15.86:0/1431499353 >> 172.21.15.86:0/1431499353 conn(0x5575c584c7a0 msgr2=0x5575c5858470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 -- 172.21.15.86:0/1431499353 shutdown_connections 2024-04-16T05:40:17.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 1 -- 172.21.15.86:0/1431499353 wait complete. 2024-04-16T05:40:17.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.065+0000 7f4fedff2740 20 client.17166 trim_cache size 0 max 16384 2024-04-16T05:40:17.076 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:17.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 -- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4fd00a3d20 msgr2=0x7f4fd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4fd00a3d20 0x7f4fd00a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40ff9e0 tx=0x7f4fe00042c0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 -- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4105f30 msgr2=0x7f4fe419bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4105f30 0x7f4fe419bfd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00cad0 tx=0x7f4fdc00cf90 comp rx=0 tx=0).stop 2024-04-16T05:40:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 -- 172.21.15.86:0/3439275332 shutdown_connections 2024-04-16T05:40:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fe4105f30 0x7f4fe419bfd0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4fd00a3d20 0x7f4fd00a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fe4102010 0x7f4fe4197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 --2- 172.21.15.86:0/3439275332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe4101650 0x7f4fe4197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 -- 172.21.15.86:0/3439275332 >> 172.21.15.86:0/3439275332 conn(0x7f4fe4077450 msgr2=0x7f4fe4103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 -- 172.21.15.86:0/3439275332 shutdown_connections 2024-04-16T05:40:17.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.066+0000 7f4feb406640 1 -- 172.21.15.86:0/3439275332 wait complete. 2024-04-16T05:40:17.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:40:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.187+0000 7fb4a7590640 1 -- 172.21.15.86:0/912253394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 msgr2=0x7fb4a0106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.187+0000 7fb4a7590640 1 --2- 172.21.15.86:0/912253394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 0x7fb4a0106810 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb498009ca0 tx=0x7fb49802c230 comp rx=0 tx=0).stop 2024-04-16T05:40:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.187+0000 7fb4a7590640 1 -- 172.21.15.86:0/912253394 shutdown_connections 2024-04-16T05:40:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.187+0000 7fb4a7590640 1 --2- 172.21.15.86:0/912253394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 0x7fb4a0106810 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.187+0000 7fb4a7590640 1 --2- 172.21.15.86:0/912253394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4a0101980 0x7fb4a0105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.187+0000 7fb4a7590640 1 --2- 172.21.15.86:0/912253394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a0101060 0x7fb4a0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.187+0000 7fb4a7590640 1 -- 172.21.15.86:0/912253394 >> 172.21.15.86:0/912253394 conn(0x7fb4a00780b0 msgr2=0x7fb4a00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.187+0000 7fb4a7590640 1 -- 172.21.15.86:0/912253394 shutdown_connections 2024-04-16T05:40:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.187+0000 7fb4a7590640 1 -- 172.21.15.86:0/912253394 wait complete. 2024-04-16T05:40:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a7590640 1 Processor -- start 2024-04-16T05:40:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a7590640 1 -- start start 2024-04-16T05:40:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a7590640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4a0101060 0x7fb4a019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a7590640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a0101980 0x7fb4a019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a7590640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 0x7fb4a01a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a7590640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01189b0 con 0x7fb4a0106380 2024-04-16T05:40:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a7590640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0118830 con 0x7fb4a0101060 2024-04-16T05:40:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a7590640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0118b30 con 0x7fb4a0101980 2024-04-16T05:40:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a5b2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 0x7fb4a01a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a5b2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 0x7fb4a01a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39852/0 (socket says 172.21.15.86:39852) 2024-04-16T05:40:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a5b2d640 1 -- 172.21.15.86:0/2546211123 learned_addr learned my addr 172.21.15.86:0/2546211123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a0101980 0x7fb4a019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a5b2d640 1 -- 172.21.15.86:0/2546211123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a0101980 msgr2=0x7fb4a019f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a532c640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4a0101060 0x7fb4a019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a5b2d640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a0101980 0x7fb4a019f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a5b2d640 1 -- 172.21.15.86:0/2546211123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4a0101060 msgr2=0x7fb4a019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a5b2d640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4a0101060 0x7fb4a019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a5b2d640 1 -- 172.21.15.86:0/2546211123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a01a3e00 con 0x7fb4a0106380 2024-04-16T05:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a0101980 0x7fb4a019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.188+0000 7fb4a532c640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4a0101060 0x7fb4a019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.189+0000 7fb4a5b2d640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 0x7fb4a01a36d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb498005e30 tx=0x7fb498035e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.189+0000 7fb4967fc640 1 -- 172.21.15.86:0/2546211123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498039040 con 0x7fb4a0106380 2024-04-16T05:40:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.189+0000 7fb4a7590640 1 -- 172.21.15.86:0/2546211123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a01a4090 con 0x7fb4a0106380 2024-04-16T05:40:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.189+0000 7fb4967fc640 1 -- 172.21.15.86:0/2546211123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb498044070 con 0x7fb4a0106380 2024-04-16T05:40:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.189+0000 7fb4967fc640 1 -- 172.21.15.86:0/2546211123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980043a0 con 0x7fb4a0106380 2024-04-16T05:40:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.189+0000 7fb4a7590640 1 -- 172.21.15.86:0/2546211123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a01ab980 con 0x7fb4a0106380 2024-04-16T05:40:17.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.191+0000 7fb4967fc640 1 -- 172.21.15.86:0/2546211123 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb498004540 con 0x7fb4a0106380 2024-04-16T05:40:17.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.191+0000 7fb4aa178740 1 Processor -- start 2024-04-16T05:40:17.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4aa178740 1 -- start start 2024-04-16T05:40:17.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4967fc640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb48c0a3d20 0x7fb48c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf4e60 0x5648bbbf5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf5780 0x5648bbbfde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 0x5648bbc00750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4aa178740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648bbb5a0e0 con 0x5648bbbf5780 2024-04-16T05:40:17.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4aa178740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5648bbaf7520 con 0x5648bbbfe360 2024-04-16T05:40:17.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4aa178740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648bbb560a0 con 0x5648bbbf4e60 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb48c0a3d20 0x7fb48c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4967fc640 1 -- 172.21.15.86:0/2546211123 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb4980f3030 con 0x7fb4a0106380 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a4b2b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf5780 0x5648bbbfde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a4b2b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf5780 0x5648bbbfde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39858/0 (socket says 172.21.15.86:39858) 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a4b2b640 1 -- 172.21.15.86:0/1699696526 learned_addr learned my addr 172.21.15.86:0/1699696526 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a532c640 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf4e60 0x5648bbbf5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a5b2d640 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 0x5648bbc00750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a532c640 1 -- 172.21.15.86:0/1699696526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 msgr2=0x5648bbc00750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a532c640 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 0x5648bbc00750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a532c640 1 -- 172.21.15.86:0/1699696526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf5780 msgr2=0x5648bbbfde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a532c640 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf5780 0x5648bbbfde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a532c640 1 -- 172.21.15.86:0/1699696526 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648bbb1f8d0 con 0x5648bbbf4e60 2024-04-16T05:40:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf5780 0x5648bbbfde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:40:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.192+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb48c0a3d20 0x7fb48c0a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb490013840 tx=0x7fb49000ba80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb4a532c640 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf4e60 0x5648bbbf5240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00bab0 tx=0x7fb49c00bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=838c6d88e2a6d56f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb48b7fe640 1 -- 172.21.15.86:0/1699696526 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c017420 con 0x5648bbbf4e60 2024-04-16T05:40:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb48b7fe640 1 -- 172.21.15.86:0/1699696526 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c017aa0 con 0x5648bbbf4e60 2024-04-16T05:40:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb48b7fe640 1 -- 172.21.15.86:0/1699696526 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c00fa10 con 0x5648bbbf4e60 2024-04-16T05:40:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb4aa178740 1 -- 172.21.15.86:0/1699696526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf4e60 msgr2=0x5648bbbf5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf4e60 0x5648bbbf5240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00bab0 tx=0x7fb49c00bf70 comp rx=0 tx=0).stop 2024-04-16T05:40:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb4aa178740 1 -- 172.21.15.86:0/1699696526 shutdown_connections 2024-04-16T05:40:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 0x5648bbc00750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf5780 0x5648bbbfde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.193+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1699696526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf4e60 0x5648bbbf5240 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 -- 172.21.15.86:0/1699696526 >> 172.21.15.86:0/1699696526 conn(0x5648bbbef590 msgr2=0x5648bbbef970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 -- 172.21.15.86:0/1699696526 shutdown_connections 2024-04-16T05:40:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 -- 172.21.15.86:0/1699696526 wait complete. 2024-04-16T05:40:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 -- start start 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 Processor -- start 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 0x5648bbbfb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 0x5648bbbfbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648bbc00fa0 con 0x5648bbbf4e60 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5648bbb134d0 con 0x5648bbbfe360 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648bbc01120 con 0x5648bbbf5780 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4aa178740 10 client.0 _ll_register_callbacks cb 0x5648bbaf82c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.194+0000 7fb4a532c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a532c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39864/0 (socket says 172.21.15.86:39864) 2024-04-16T05:40:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a532c640 1 -- 172.21.15.86:0/3448763075 learned_addr learned my addr 172.21.15.86:0/3448763075 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 0x5648bbbfb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a5b2d640 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 0x5648bbbfbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a532c640 1 -- 172.21.15.86:0/3448763075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 msgr2=0x5648bbbfb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a532c640 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 0x5648bbbfb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a532c640 1 -- 172.21.15.86:0/3448763075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 msgr2=0x5648bbbfbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a532c640 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 0x5648bbbfbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a532c640 1 -- 172.21.15.86:0/3448763075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648bb86c3f0 con 0x5648bbbf4e60 2024-04-16T05:40:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4a532c640 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfb410 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb49c017fa0 tx=0x7fb49c005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991490bfc6068011 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb488ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb488ff9640 1 -- 172.21.15.86:0/3448763075 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c027020 con 0x5648bbbf4e60 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb488ff9640 1 -- 172.21.15.86:0/3448763075 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb49c017830 con 0x5648bbbf4e60 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4aa178740 1 -- 172.21.15.86:0/3448763075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49c017aa0 con 0x5648bbbf4e60 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb488ff9640 1 -- 172.21.15.86:0/3448763075 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb49c032050 con 0x5648bbbf4e60 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4aa178740 10 client.17178 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb4aa178740 1 -- 172.21.15.86:0/3448763075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648bb9367d0 con 0x5648bbbf4e60 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.195+0000 7fb488ff9640 1 -- 172.21.15.86:0/3448763075 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c017aa0 con 0x5648bbbf4e60 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb488ff9640 1 -- 172.21.15.86:0/3448763075 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb49c0239c0 con 0x5648bbbf4e60 2024-04-16T05:40:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb4aa178740 10 client.17178 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb4aa178740 10 client.17178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:40:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb4aa178740 20 client.17178 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5648bbc0e230 0x5648bbc10620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb4aa178740 4 client.17178 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb4aa178740 1 -- 172.21.15.86:0/3448763075 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648bbb67b00 con 0x5648bbc0e230 2024-04-16T05:40:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5648bbc0e230 0x5648bbc10620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb488ff9640 10 client.17178 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:40:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.196+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5648bbc0e230 0x5648bbc10620 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1176f36a6af22fda server_cookie=4f3d1899ab4100bf in_seq=0 out_seq=0 2024-04-16T05:40:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb488ff9640 1 -- 172.21.15.86:0/3448763075 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648bbb67b00 con 0x5648bbc0e230 2024-04-16T05:40:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb488ff9640 10 client.17178 handle_command_reply: tid=0 2024-04-16T05:40:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 client.17178 shutdown 2024-04-16T05:40:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 -- 172.21.15.86:0/3448763075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 msgr2=0x5648bbbfb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfb410 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb49c017fa0 tx=0x7fb49c005ed0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 -- 172.21.15.86:0/3448763075 shutdown_connections 2024-04-16T05:40:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbfe360 0x5648bbbfbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 0x5648bbbfb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5648bbc0e230 0x5648bbc10620 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3448763075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfb410 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb488ff9640 0 client.17178 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:40:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 -- 172.21.15.86:0/3448763075 >> 172.21.15.86:0/3448763075 conn(0x5648bbbef590 msgr2=0x5648bbbf12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 -- 172.21.15.86:0/3448763075 shutdown_connections 2024-04-16T05:40:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 1 -- 172.21.15.86:0/3448763075 wait complete. 2024-04-16T05:40:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.197+0000 7fb4aa178740 20 client.17178 trim_cache size 0 max 16384 2024-04-16T05:40:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4aa178740 1 Processor -- start 2024-04-16T05:40:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4aa178740 1 -- start start 2024-04-16T05:40:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf5780 0x5648bbbfa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbfe360 0x5648bbbee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4aa178740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b5a00 con 0x5648bbbf4e60 2024-04-16T05:40:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4aa178740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b7540 con 0x5648bbbf5780 2024-04-16T05:40:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4aa178740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b60d0 con 0x5648bbbfe360 2024-04-16T05:40:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a532c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a532c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39872/0 (socket says 172.21.15.86:39872) 2024-04-16T05:40:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a532c640 1 -- 172.21.15.86:0/3780806576 learned_addr learned my addr 172.21.15.86:0/3780806576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a532c640 1 -- 172.21.15.86:0/3780806576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbfe360 msgr2=0x5648bbbee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a532c640 1 --2- 172.21.15.86:0/3780806576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbfe360 0x5648bbbee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/3780806576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf5780 0x5648bbbfa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a532c640 1 -- 172.21.15.86:0/3780806576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf5780 msgr2=0x5648bbbfa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a532c640 1 --2- 172.21.15.86:0/3780806576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf5780 0x5648bbbfa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a532c640 1 -- 172.21.15.86:0/3780806576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648bbbeac30 con 0x5648bbbf4e60 2024-04-16T05:40:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/3780806576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf5780 0x5648bbbfa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.211+0000 7fb4a532c640 1 --2- 172.21.15.86:0/3780806576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfdb40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb49c017fd0 tx=0x7fb49c00b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837145ff75924cba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb48affd640 1 -- 172.21.15.86:0/3780806576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c027070 con 0x5648bbbf4e60 2024-04-16T05:40:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb48affd640 1 -- 172.21.15.86:0/3780806576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c021420 con 0x5648bbbf4e60 2024-04-16T05:40:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb48affd640 1 -- 172.21.15.86:0/3780806576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c00f790 con 0x5648bbbf4e60 2024-04-16T05:40:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb4aa178740 1 -- 172.21.15.86:0/3780806576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 msgr2=0x5648bbbfdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3780806576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfdb40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb49c017fd0 tx=0x7fb49c00b3f0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb4aa178740 1 -- 172.21.15.86:0/3780806576 shutdown_connections 2024-04-16T05:40:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3780806576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbfe360 0x5648bbbee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3780806576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf5780 0x5648bbbfa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb4aa178740 1 --2- 172.21.15.86:0/3780806576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbf4e60 0x5648bbbfdb40 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb4aa178740 1 -- 172.21.15.86:0/3780806576 >> 172.21.15.86:0/3780806576 conn(0x5648bbbef590 msgr2=0x5648bbbedea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb4aa178740 1 -- 172.21.15.86:0/3780806576 shutdown_connections 2024-04-16T05:40:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.212+0000 7fb4aa178740 1 -- 172.21.15.86:0/3780806576 wait complete. 2024-04-16T05:40:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4aa178740 1 -- start start 2024-04-16T05:40:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4aa178740 1 Processor -- start 2024-04-16T05:40:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf4e60 0x5648bbc06480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 0x5648bbc069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4aa178740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbfe360 0x5648bbbf7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4aa178740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b6e70 con 0x5648bbbfe360 2024-04-16T05:40:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4aa178740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b67a0 con 0x5648bbbf4e60 2024-04-16T05:40:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4aa178740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b39c0 con 0x5648bbbf5780 2024-04-16T05:40:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4aa178740 10 client.0 _ll_register_callbacks cb 0x7fb49000bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a5b2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbfe360 0x5648bbbf7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a5b2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbfe360 0x5648bbbf7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39886/0 (socket says 172.21.15.86:39886) 2024-04-16T05:40:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a5b2d640 1 -- 172.21.15.86:0/1511660637 learned_addr learned my addr 172.21.15.86:0/1511660637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a4b2b640 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 0x5648bbc069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a532c640 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf4e60 0x5648bbc06480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a5b2d640 1 -- 172.21.15.86:0/1511660637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 msgr2=0x5648bbc069c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a5b2d640 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 0x5648bbc069c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a5b2d640 1 -- 172.21.15.86:0/1511660637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf4e60 msgr2=0x5648bbc06480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a5b2d640 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf4e60 0x5648bbc06480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a5b2d640 1 -- 172.21.15.86:0/1511660637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648bbbf7db0 con 0x5648bbbfe360 2024-04-16T05:40:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.213+0000 7fb4a532c640 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf4e60 0x5648bbc06480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4a5b2d640 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbfe360 0x5648bbbf7740 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb4980beb00 tx=0x7fb4980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bd509770fc1e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb488ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb488ff9640 1 -- 172.21.15.86:0/1511660637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980c31b0 con 0x5648bbbfe360 2024-04-16T05:40:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb488ff9640 1 -- 172.21.15.86:0/1511660637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb4980923c0 con 0x5648bbbfe360 2024-04-16T05:40:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4aa178740 1 -- 172.21.15.86:0/1511660637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49c021420 con 0x5648bbbfe360 2024-04-16T05:40:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb488ff9640 1 -- 172.21.15.86:0/1511660637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4980c79e0 con 0x5648bbbfe360 2024-04-16T05:40:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4aa178740 10 client.17184 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4aa178740 1 -- 172.21.15.86:0/1511660637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648bbc73e90 con 0x5648bbbfe360 2024-04-16T05:40:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb488ff9640 1 -- 172.21.15.86:0/1511660637 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c021420 con 0x5648bbbfe360 2024-04-16T05:40:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb488ff9640 1 -- 172.21.15.86:0/1511660637 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb49803c030 con 0x5648bbbfe360 2024-04-16T05:40:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4aa178740 10 client.17184 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4aa178740 10 client.17184 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:40:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4aa178740 20 client.17184 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5648bbc745e0 0x5648bbc769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4aa178740 4 client.17184 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4aa178740 1 -- 172.21.15.86:0/1511660637 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648bbb61970 con 0x5648bbc745e0 2024-04-16T05:40:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.214+0000 7fb4a532c640 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5648bbc745e0 0x5648bbc769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb488ff9640 10 client.17184 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:40:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4a532c640 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5648bbc745e0 0x5648bbc769d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d584ee783e7803d7 server_cookie=1a5a5c9d5b83bab6 in_seq=0 out_seq=0 2024-04-16T05:40:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb488ff9640 1 -- 172.21.15.86:0/1511660637 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648bbb61970 con 0x5648bbc745e0 2024-04-16T05:40:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb488ff9640 10 client.17184 handle_command_reply: tid=0 2024-04-16T05:40:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 client.17184 shutdown 2024-04-16T05:40:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 -- 172.21.15.86:0/1511660637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbfe360 msgr2=0x5648bbbf7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbfe360 0x5648bbbf7740 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb4980beb00 tx=0x7fb4980048f0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 -- 172.21.15.86:0/1511660637 shutdown_connections 2024-04-16T05:40:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x5648bbc745e0 0x5648bbc769d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648bbbfe360 0x5648bbbf7740 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb488ff9640 0 client.17184 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:40:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648bbbf5780 0x5648bbc069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 --2- 172.21.15.86:0/1511660637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648bbbf4e60 0x5648bbc06480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 -- 172.21.15.86:0/1511660637 >> 172.21.15.86:0/1511660637 conn(0x5648bbbef590 msgr2=0x5648bbbee730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 -- 172.21.15.86:0/1511660637 shutdown_connections 2024-04-16T05:40:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 1 -- 172.21.15.86:0/1511660637 wait complete. 2024-04-16T05:40:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.215+0000 7fb4aa178740 20 client.17184 trim_cache size 0 max 16384 2024-04-16T05:40:17.225 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:17.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 -- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb48c0a3d20 msgr2=0x7fb48c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb48c0a3d20 0x7fb48c0a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb490013840 tx=0x7fb49000ba80 comp rx=0 tx=0).stop 2024-04-16T05:40:17.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 -- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 msgr2=0x7fb4a01a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 0x7fb4a01a36d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb498005e30 tx=0x7fb498035e00 comp rx=0 tx=0).stop 2024-04-16T05:40:17.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 -- 172.21.15.86:0/2546211123 shutdown_connections 2024-04-16T05:40:17.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0106380 0x7fb4a01a36d0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a0101980 0x7fb4a019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb48c0a3d20 0x7fb48c0a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 --2- 172.21.15.86:0/2546211123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4a0101060 0x7fb4a019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 -- 172.21.15.86:0/2546211123 >> 172.21.15.86:0/2546211123 conn(0x7fb4a00780b0 msgr2=0x7fb4a0103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 -- 172.21.15.86:0/2546211123 shutdown_connections 2024-04-16T05:40:17.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.216+0000 7fb4a7590640 1 -- 172.21.15.86:0/2546211123 wait complete. 2024-04-16T05:40:17.228 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:40:17.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.336+0000 7f2ea0042640 1 -- 172.21.15.86:0/3922235103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98107320 msgr2=0x7f2e9810eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.336+0000 7f2ea0042640 1 --2- 172.21.15.86:0/3922235103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98107320 0x7f2e9810eef0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009900 tx=0x7f2e9002bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:17.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 -- 172.21.15.86:0/3922235103 shutdown_connections 2024-04-16T05:40:17.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 --2- 172.21.15.86:0/3922235103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98107320 0x7f2e9810eef0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 --2- 172.21.15.86:0/3922235103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e98102c00 0x7f2e98106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 --2- 172.21.15.86:0/3922235103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98102250 0x7f2e98102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 -- 172.21.15.86:0/3922235103 >> 172.21.15.86:0/3922235103 conn(0x7f2e98079490 msgr2=0x7f2e980798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 -- 172.21.15.86:0/3922235103 shutdown_connections 2024-04-16T05:40:17.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 -- 172.21.15.86:0/3922235103 wait complete. 2024-04-16T05:40:17.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 Processor -- start 2024-04-16T05:40:17.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 -- start start 2024-04-16T05:40:17.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e98102250 0x7f2e98074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98107320 0x7f2e98074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98074d80 0x7f2e981a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981123e0 con 0x7f2e98074d80 2024-04-16T05:40:17.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98112260 con 0x7f2e98107320 2024-04-16T05:40:17.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.337+0000 7f2ea0042640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98112560 con 0x7f2e98102250 2024-04-16T05:40:17.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9e5df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98074d80 0x7f2e981a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9e5df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98074d80 0x7f2e981a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39910/0 (socket says 172.21.15.86:39910) 2024-04-16T05:40:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9e5df640 1 -- 172.21.15.86:0/367071785 learned_addr learned my addr 172.21.15.86:0/367071785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9ddde640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e98102250 0x7f2e98074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9e5df640 1 -- 172.21.15.86:0/367071785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e98102250 msgr2=0x7f2e98074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9e5df640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e98102250 0x7f2e98074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9e5df640 1 -- 172.21.15.86:0/367071785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98107320 msgr2=0x7f2e98074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98107320 0x7f2e98074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9e5df640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98107320 0x7f2e98074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9e5df640 1 -- 172.21.15.86:0/367071785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e981a7110 con 0x7f2e98074d80 2024-04-16T05:40:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e9e5df640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98074d80 0x7f2e981a6a10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009230 tx=0x7f2e9002cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e8effd640 1 -- 172.21.15.86:0/367071785 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90041070 con 0x7f2e98074d80 2024-04-16T05:40:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e8effd640 1 -- 172.21.15.86:0/367071785 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e90004450 con 0x7f2e98074d80 2024-04-16T05:40:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2ea0042640 1 -- 172.21.15.86:0/367071785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e981a73a0 con 0x7f2e98074d80 2024-04-16T05:40:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2e8effd640 1 -- 172.21.15.86:0/367071785 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90034440 con 0x7f2e98074d80 2024-04-16T05:40:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.338+0000 7f2ea0042640 1 -- 172.21.15.86:0/367071785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e981a7890 con 0x7f2e98074d80 2024-04-16T05:40:17.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.340+0000 7f2e8effd640 1 -- 172.21.15.86:0/367071785 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e90044070 con 0x7f2e98074d80 2024-04-16T05:40:17.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2ea2c20740 1 Processor -- start 2024-04-16T05:40:17.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2ea2c20740 1 -- start start 2024-04-16T05:40:17.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebade60 0x56321ebae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebae780 0x56321ebb6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebb7360 0x56321ebb9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56321eb130e0 con 0x56321ebade60 2024-04-16T05:40:17.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56321eab0520 con 0x56321ebae780 2024-04-16T05:40:17.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56321eb0f0a0 con 0x56321ebb7360 2024-04-16T05:40:17.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2e9d5dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebade60 0x56321ebae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2e9d5dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebade60 0x56321ebae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39922/0 (socket says 172.21.15.86:39922) 2024-04-16T05:40:17.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/323449544 learned_addr learned my addr 172.21.15.86:0/323449544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2e8effd640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2e840a3d50 0x7f2e840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2e9e5df640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2e840a3d50 0x7f2e840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.341+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/323449544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebae780 0x56321ebb6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/323449544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebb7360 msgr2=0x56321ebb9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/323449544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebb7360 0x56321ebb9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/323449544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebae780 msgr2=0x56321ebb6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/323449544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebae780 0x56321ebb6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/323449544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321ead88d0 con 0x56321ebade60 2024-04-16T05:40:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/323449544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebae780 0x56321ebb6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e8effd640 1 -- 172.21.15.86:0/367071785 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2e900f2610 con 0x7f2e98074d80 2024-04-16T05:40:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/323449544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebade60 0x56321ebae240 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009b40 tx=0x7f2e88010310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787d82450df2bac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e9e5df640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2e840a3d50 0x7f2e840a6210 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e900be2e0 tx=0x7f2e90005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e83fff640 1 -- 172.21.15.86:0/323449544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88016070 con 0x56321ebade60 2024-04-16T05:40:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e83fff640 1 -- 172.21.15.86:0/323449544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e88010d40 con 0x56321ebade60 2024-04-16T05:40:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.342+0000 7f2e83fff640 1 -- 172.21.15.86:0/323449544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88011740 con 0x56321ebade60 2024-04-16T05:40:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 -- 172.21.15.86:0/323449544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebade60 msgr2=0x56321ebae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/323449544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebade60 0x56321ebae240 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009b40 tx=0x7f2e88010310 comp rx=0 tx=0).stop 2024-04-16T05:40:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 -- 172.21.15.86:0/323449544 shutdown_connections 2024-04-16T05:40:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/323449544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebb7360 0x56321ebb9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/323449544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebae780 0x56321ebb6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/323449544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebade60 0x56321ebae240 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 -- 172.21.15.86:0/323449544 >> 172.21.15.86:0/323449544 conn(0x56321eba8590 msgr2=0x56321eba8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 -- 172.21.15.86:0/323449544 shutdown_connections 2024-04-16T05:40:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 -- 172.21.15.86:0/323449544 wait complete. 2024-04-16T05:40:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 -- start start 2024-04-16T05:40:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.343+0000 7f2ea2c20740 1 Processor -- start 2024-04-16T05:40:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebb4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 0x56321ebb4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56321ebba060 con 0x56321ebb7360 2024-04-16T05:40:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56321eacc4d0 con 0x56321ebade60 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56321ebba1e0 con 0x56321ebae780 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2ea2c20740 10 client.0 _ll_register_callbacks cb 0x56321eab12c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39934/0 (socket says 172.21.15.86:39934) 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/1321101720 learned_addr learned my addr 172.21.15.86:0/1321101720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebb4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 0x56321ebb4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/1321101720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 msgr2=0x56321ebb4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 0x56321ebb4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/1321101720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 msgr2=0x56321ebb4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebb4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/1321101720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321e8253f0 con 0x56321ebb7360 2024-04-16T05:40:17.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb4ea0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2e88010fd0 tx=0x7f2e880114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547bf77edb7ce4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e817fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:17.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e817fa640 1 -- 172.21.15.86:0/1321101720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88016070 con 0x56321ebb7360 2024-04-16T05:40:17.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2e817fa640 1 -- 172.21.15.86:0/1321101720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2e88012990 con 0x56321ebb7360 2024-04-16T05:40:17.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.344+0000 7f2ea2c20740 1 -- 172.21.15.86:0/1321101720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e88010d40 con 0x56321ebb7360 2024-04-16T05:40:17.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2e817fa640 1 -- 172.21.15.86:0/1321101720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e88031050 con 0x56321ebb7360 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2ea2c20740 10 client.17196 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2ea2c20740 1 -- 172.21.15.86:0/1321101720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56321e8ef7d0 con 0x56321ebb7360 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2e817fa640 1 -- 172.21.15.86:0/1321101720 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e88010d40 con 0x56321ebb7360 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2e817fa640 1 -- 172.21.15.86:0/1321101720 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e88003b90 con 0x56321ebb7360 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2ea2c20740 10 client.17196 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2ea2c20740 10 client.17196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2ea2c20740 20 client.17196 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56321ebc7200 0x56321ebc95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2ea2c20740 4 client.17196 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2ea2c20740 1 -- 172.21.15.86:0/1321101720 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56321eb20b00 con 0x56321ebc7200 2024-04-16T05:40:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.345+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56321ebc7200 0x56321ebc95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.346+0000 7f2e817fa640 10 client.17196 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:40:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.346+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56321ebc7200 0x56321ebc95f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0f4fa51e2eac259 server_cookie=fe52db58ea35c9bc in_seq=0 out_seq=0 2024-04-16T05:40:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.346+0000 7f2e817fa640 1 -- 172.21.15.86:0/1321101720 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56321eb20b00 con 0x56321ebc7200 2024-04-16T05:40:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.346+0000 7f2e817fa640 10 client.17196 handle_command_reply: tid=0 2024-04-16T05:40:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.346+0000 7f2ea2c20740 1 client.17196 shutdown 2024-04-16T05:40:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.346+0000 7f2ea2c20740 1 -- 172.21.15.86:0/1321101720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 msgr2=0x56321ebb4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.346+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb4ea0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2e88010fd0 tx=0x7f2e880114b0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2ea2c20740 1 -- 172.21.15.86:0/1321101720 shutdown_connections 2024-04-16T05:40:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56321ebc7200 0x56321ebc95f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb4ea0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 0x56321ebb4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2e817fa640 0 client.17196 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:40:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/1321101720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebb4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2ea2c20740 1 -- 172.21.15.86:0/1321101720 >> 172.21.15.86:0/1321101720 conn(0x56321eba8590 msgr2=0x56321ebaa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2ea2c20740 1 -- 172.21.15.86:0/1321101720 shutdown_connections 2024-04-16T05:40:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2ea2c20740 1 -- 172.21.15.86:0/1321101720 wait complete. 2024-04-16T05:40:17.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.347+0000 7f2ea2c20740 20 client.17196 trim_cache size 0 max 16384 2024-04-16T05:40:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2ea2c20740 1 Processor -- start 2024-04-16T05:40:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2ea2c20740 1 -- start start 2024-04-16T05:40:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebb3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebae780 0x56321ebced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebb7360 0x56321ebcf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b15c0 con 0x56321ebae780 2024-04-16T05:40:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b3100 con 0x56321ebade60 2024-04-16T05:40:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b1c90 con 0x56321ebb7360 2024-04-16T05:40:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9ddde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebae780 0x56321ebced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9ddde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebae780 0x56321ebced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39936/0 (socket says 172.21.15.86:39936) 2024-04-16T05:40:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9ddde640 1 -- 172.21.15.86:0/80331458 learned_addr learned my addr 172.21.15.86:0/80331458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9ddde640 1 -- 172.21.15.86:0/80331458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebb7360 msgr2=0x56321ebcf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/80331458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebb7360 0x56321ebcf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/80331458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebb3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9ddde640 1 -- 172.21.15.86:0/80331458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 msgr2=0x56321ebb3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/80331458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebb3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9ddde640 1 -- 172.21.15.86:0/80331458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321ebb9ac0 con 0x56321ebae780 2024-04-16T05:40:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/80331458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebb3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.360+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/80331458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebae780 0x56321ebced30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x56321ebb3c80 tx=0x7f2e94009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49d7d5aacae41b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2e837fe640 1 -- 172.21.15.86:0/80331458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9401f020 con 0x56321ebae780 2024-04-16T05:40:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2e837fe640 1 -- 172.21.15.86:0/80331458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e9400fdd0 con 0x56321ebae780 2024-04-16T05:40:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2e837fe640 1 -- 172.21.15.86:0/80331458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9401b800 con 0x56321ebae780 2024-04-16T05:40:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 -- 172.21.15.86:0/80331458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebae780 msgr2=0x56321ebced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/80331458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebae780 0x56321ebced30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x56321ebb3c80 tx=0x7f2e94009760 comp rx=0 tx=0).stop 2024-04-16T05:40:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 -- 172.21.15.86:0/80331458 shutdown_connections 2024-04-16T05:40:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/80331458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebb7360 0x56321ebcf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/80331458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebae780 0x56321ebced30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/80331458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebb3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 -- 172.21.15.86:0/80331458 >> 172.21.15.86:0/80331458 conn(0x56321eba8590 msgr2=0x56321ebaeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 -- 172.21.15.86:0/80331458 shutdown_connections 2024-04-16T05:40:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 -- 172.21.15.86:0/80331458 wait complete. 2024-04-16T05:40:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.361+0000 7f2ea2c20740 1 -- start start 2024-04-16T05:40:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2ea2c20740 1 Processor -- start 2024-04-16T05:40:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebbf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 0x56321ebbfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2ea2c20740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b2a30 con 0x56321ebb7360 2024-04-16T05:40:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b2360 con 0x56321ebade60 2024-04-16T05:40:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2ea2c20740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981af580 con 0x56321ebae780 2024-04-16T05:40:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2ea2c20740 10 client.0 _ll_register_callbacks cb 0x7f2e88009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2e9d5dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2e9d5dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39938/0 (socket says 172.21.15.86:39938) 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/3111449239 learned_addr learned my addr 172.21.15.86:0/3111449239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 0x56321ebbfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/3111449239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 msgr2=0x56321ebbfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 0x56321ebbfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/3111449239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 msgr2=0x56321ebbf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebbf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.362+0000 7f2e9d5dd640 1 -- 172.21.15.86:0/3111449239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321ebb1450 con 0x56321ebb7360 2024-04-16T05:40:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2e9d5dd640 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb0c50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2e880113f0 tx=0x7f2e880117e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d22db20a2364e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2e817fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2e817fa640 1 -- 172.21.15.86:0/3111449239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8803a060 con 0x56321ebb7360 2024-04-16T05:40:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2e817fa640 1 -- 172.21.15.86:0/3111449239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2e88011ce0 con 0x56321ebb7360 2024-04-16T05:40:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2ea2c20740 1 -- 172.21.15.86:0/3111449239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9400fdd0 con 0x56321ebb7360 2024-04-16T05:40:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2e817fa640 1 -- 172.21.15.86:0/3111449239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e88041050 con 0x56321ebb7360 2024-04-16T05:40:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2ea2c20740 10 client.17202 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2ea2c20740 1 -- 172.21.15.86:0/3111449239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56321ec2d130 con 0x56321ebb7360 2024-04-16T05:40:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2e817fa640 1 -- 172.21.15.86:0/3111449239 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e9400fdd0 con 0x56321ebb7360 2024-04-16T05:40:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2e817fa640 1 -- 172.21.15.86:0/3111449239 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e88012770 con 0x56321ebb7360 2024-04-16T05:40:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2ea2c20740 10 client.17202 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2ea2c20740 10 client.17202 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:40:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2ea2c20740 20 client.17202 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56321ec2d880 0x56321ec2fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2ea2c20740 4 client.17202 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.363+0000 7f2ea2c20740 1 -- 172.21.15.86:0/3111449239 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56321eb1a970 con 0x56321ec2d880 2024-04-16T05:40:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56321ec2d880 0x56321ec2fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2e817fa640 10 client.17202 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:40:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2e9ddde640 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56321ec2d880 0x56321ec2fca0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4aa988f65650306 server_cookie=80580782a54c426a in_seq=0 out_seq=0 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2e817fa640 1 -- 172.21.15.86:0/3111449239 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56321eb1a970 con 0x56321ec2d880 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2e817fa640 10 client.17202 handle_command_reply: tid=0 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 client.17202 shutdown 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 -- 172.21.15.86:0/3111449239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 msgr2=0x56321ebb0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb0c50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2e880113f0 tx=0x7f2e880117e0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 -- 172.21.15.86:0/3111449239 shutdown_connections 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56321ec2d880 0x56321ec2fca0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321ebb7360 0x56321ebb0c50 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56321ebae780 0x56321ebbfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 --2- 172.21.15.86:0/3111449239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56321ebade60 0x56321ebbf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2e817fa640 0 client.17202 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:40:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 -- 172.21.15.86:0/3111449239 >> 172.21.15.86:0/3111449239 conn(0x56321eba8590 msgr2=0x56321ebb5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 -- 172.21.15.86:0/3111449239 shutdown_connections 2024-04-16T05:40:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.364+0000 7f2ea2c20740 1 -- 172.21.15.86:0/3111449239 wait complete. 2024-04-16T05:40:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea2c20740 20 client.17202 trim_cache size 0 max 16384 2024-04-16T05:40:17.373 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 -- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2e840a3d50 msgr2=0x7f2e840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2e840a3d50 0x7f2e840a6210 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e900be2e0 tx=0x7f2e90005fd0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 -- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98074d80 msgr2=0x7f2e981a6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98074d80 0x7f2e981a6a10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009230 tx=0x7f2e9002cbf0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 -- 172.21.15.86:0/367071785 shutdown_connections 2024-04-16T05:40:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98074d80 0x7f2e981a6a10 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2e840a3d50 0x7f2e840a6210 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98107320 0x7f2e98074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 --2- 172.21.15.86:0/367071785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e98102250 0x7f2e98074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 -- 172.21.15.86:0/367071785 >> 172.21.15.86:0/367071785 conn(0x7f2e98079490 msgr2=0x7f2e980ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 -- 172.21.15.86:0/367071785 shutdown_connections 2024-04-16T05:40:17.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.365+0000 7f2ea0042640 1 -- 172.21.15.86:0/367071785 wait complete. 2024-04-16T05:40:17.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:17 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4085079949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:17.377 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:40:17.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.484+0000 7f3680dba640 1 -- 172.21.15.86:0/3090799989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101ae0 msgr2=0x7f367c105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.484+0000 7f3680dba640 1 --2- 172.21.15.86:0/3090799989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101ae0 0x7f367c105b20 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f3670009900 tx=0x7f367002bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:17.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.485+0000 7f3680dba640 1 -- 172.21.15.86:0/3090799989 shutdown_connections 2024-04-16T05:40:17.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.485+0000 7f3680dba640 1 --2- 172.21.15.86:0/3090799989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f367c106200 0x7f367c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.485+0000 7f3680dba640 1 --2- 172.21.15.86:0/3090799989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101ae0 0x7f367c105b20 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.485+0000 7f3680dba640 1 --2- 172.21.15.86:0/3090799989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f367c101130 0x7f367c101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.485+0000 7f3680dba640 1 -- 172.21.15.86:0/3090799989 >> 172.21.15.86:0/3090799989 conn(0x7f367c0780b0 msgr2=0x7f367c0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.485+0000 7f3680dba640 1 -- 172.21.15.86:0/3090799989 shutdown_connections 2024-04-16T05:40:17.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.485+0000 7f3680dba640 1 -- 172.21.15.86:0/3090799989 wait complete. 2024-04-16T05:40:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.485+0000 7f3680dba640 1 Processor -- start 2024-04-16T05:40:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.485+0000 7f3680dba640 1 -- start start 2024-04-16T05:40:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f3680dba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101130 0x7f367c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f3680dba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f367c106200 0x7f367c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f3680dba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f367c198770 0x7f367c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f3680dba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f367c1123d0 con 0x7f367c101130 2024-04-16T05:40:17.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f3680dba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f367c112250 con 0x7f367c198770 2024-04-16T05:40:17.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f3680dba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f367c112550 con 0x7f367c106200 2024-04-16T05:40:17.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101130 0x7f367c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101130 0x7f367c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39952/0 (socket says 172.21.15.86:39952) 2024-04-16T05:40:17.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367a59c640 1 -- 172.21.15.86:0/2709983038 learned_addr learned my addr 172.21.15.86:0/2709983038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367ad9d640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f367c198770 0x7f367c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367a59c640 1 -- 172.21.15.86:0/2709983038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f367c106200 msgr2=0x7f367c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367a59c640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f367c106200 0x7f367c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367a59c640 1 -- 172.21.15.86:0/2709983038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f367c198770 msgr2=0x7f367c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367a59c640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f367c198770 0x7f367c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367a59c640 1 -- 172.21.15.86:0/2709983038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c199370 con 0x7f367c101130 2024-04-16T05:40:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f367a59c640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101130 0x7f367c19e660 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f366c00ea80 tx=0x7f366c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f366b7fe640 1 -- 172.21.15.86:0/2709983038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c00ce40 con 0x7f367c101130 2024-04-16T05:40:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f366b7fe640 1 -- 172.21.15.86:0/2709983038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366c005020 con 0x7f367c101130 2024-04-16T05:40:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f3680dba640 1 -- 172.21.15.86:0/2709983038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367c1a5400 con 0x7f367c101130 2024-04-16T05:40:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f366b7fe640 1 -- 172.21.15.86:0/2709983038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c011760 con 0x7f367c101130 2024-04-16T05:40:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.486+0000 7f3680dba640 1 -- 172.21.15.86:0/2709983038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367c1a57b0 con 0x7f367c101130 2024-04-16T05:40:17.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f366b7fe640 1 -- 172.21.15.86:0/2709983038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f366c0051d0 con 0x7f367c101130 2024-04-16T05:40:17.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f36839a0740 1 Processor -- start 2024-04-16T05:40:17.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f36839a0740 1 -- start start 2024-04-16T05:40:17.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f36839a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b43240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f36839a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b4be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f36839a0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b4e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f36839a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b12aa80e0 con 0x556b12b42e60 2024-04-16T05:40:17.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f36839a0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556b12a45520 con 0x556b12b4c360 2024-04-16T05:40:17.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f36839a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556b12aa40a0 con 0x556b12b43780 2024-04-16T05:40:17.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.489+0000 7f366b7fe640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f36640a3d20 0x7f36640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b43240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b43240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39954/0 (socket says 172.21.15.86:39954) 2024-04-16T05:40:17.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 -- 172.21.15.86:0/2672059349 learned_addr learned my addr 172.21.15.86:0/2672059349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f36640a3d20 0x7f36640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f3679d9b640 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b4e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367a59c640 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b4be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f366b7fe640 1 -- 172.21.15.86:0/2709983038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f366c0d1260 con 0x7f367c101130 2024-04-16T05:40:17.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 -- 172.21.15.86:0/2672059349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 msgr2=0x556b12b4be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b4be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 -- 172.21.15.86:0/2672059349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 msgr2=0x556b12b4e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b4e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 -- 172.21.15.86:0/2672059349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b12a6d8d0 con 0x556b12b42e60 2024-04-16T05:40:17.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f3679d9b640 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b4e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:40:17.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b43240 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3674014bc0 tx=0x7f36740113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2162e7a4432086f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f367ad9d640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f36640a3d20 0x7f36640a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f36740118e0 tx=0x7f3674016000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f3668ff9640 1 -- 172.21.15.86:0/2672059349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674022070 con 0x556b12b42e60 2024-04-16T05:40:17.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f3668ff9640 1 -- 172.21.15.86:0/2672059349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3674014590 con 0x556b12b42e60 2024-04-16T05:40:17.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.490+0000 7f3668ff9640 1 -- 172.21.15.86:0/2672059349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674009a90 con 0x556b12b42e60 2024-04-16T05:40:17.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 -- 172.21.15.86:0/2672059349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 msgr2=0x556b12b43240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b43240 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3674014bc0 tx=0x7f36740113e0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 -- 172.21.15.86:0/2672059349 shutdown_connections 2024-04-16T05:40:17.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b4e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b4be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 --2- 172.21.15.86:0/2672059349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b43240 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 -- 172.21.15.86:0/2672059349 >> 172.21.15.86:0/2672059349 conn(0x556b12b3d590 msgr2=0x556b12b3d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 -- 172.21.15.86:0/2672059349 shutdown_connections 2024-04-16T05:40:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 -- 172.21.15.86:0/2672059349 wait complete. 2024-04-16T05:40:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.491+0000 7f36839a0740 1 -- start start 2024-04-16T05:40:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f36839a0740 1 Processor -- start 2024-04-16T05:40:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f36839a0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b42e60 0x556b12b49500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f36839a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b49a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f36839a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b4c360 0x556b12b49f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f36839a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b12b4efa0 con 0x556b12b4c360 2024-04-16T05:40:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f36839a0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556b12a614d0 con 0x556b12b42e60 2024-04-16T05:40:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f36839a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556b12b4f120 con 0x556b12b43780 2024-04-16T05:40:17.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f36839a0740 10 client.0 _ll_register_callbacks cb 0x556b12a462c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:17.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b4c360 0x556b12b49f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b4c360 0x556b12b49f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39968/0 (socket says 172.21.15.86:39968) 2024-04-16T05:40:17.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367ad9d640 1 -- 172.21.15.86:0/2804509349 learned_addr learned my addr 172.21.15.86:0/2804509349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367a59c640 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b42e60 0x556b12b49500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367ad9d640 1 -- 172.21.15.86:0/2804509349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 msgr2=0x556b12b49a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367ad9d640 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b49a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367ad9d640 1 -- 172.21.15.86:0/2804509349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b42e60 msgr2=0x556b12b49500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367ad9d640 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b42e60 0x556b12b49500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367ad9d640 1 -- 172.21.15.86:0/2804509349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b127ba3f0 con 0x556b12b4c360 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367a59c640 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b42e60 0x556b12b49500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f367ad9d640 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b4c360 0x556b12b49f80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3674015fd0 tx=0x7f36740056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb4e7431ca48151 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f3661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f3661ffb640 1 -- 172.21.15.86:0/2804509349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674022040 con 0x556b12b4c360 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.492+0000 7f3661ffb640 1 -- 172.21.15.86:0/2804509349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3674009c40 con 0x556b12b4c360 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f36839a0740 1 -- 172.21.15.86:0/2804509349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3674014590 con 0x556b12b4c360 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f3661ffb640 1 -- 172.21.15.86:0/2804509349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f367403a050 con 0x556b12b4c360 2024-04-16T05:40:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f36839a0740 10 client.17214 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f36839a0740 1 -- 172.21.15.86:0/2804509349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b128847d0 con 0x556b12b4c360 2024-04-16T05:40:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f3661ffb640 1 -- 172.21.15.86:0/2804509349 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3674014590 con 0x556b12b4c360 2024-04-16T05:40:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f3661ffb640 1 -- 172.21.15.86:0/2804509349 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3674003c60 con 0x556b12b4c360 2024-04-16T05:40:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f36839a0740 10 client.17214 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f36839a0740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f36839a0740 20 client.17214 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f36839a0740 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556b12b5c3c0 0x556b12b5e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f36839a0740 4 client.17214 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f36839a0740 1 -- 172.21.15.86:0/2804509349 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b12ab5b00 con 0x556b12b5c3c0 2024-04-16T05:40:17.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.493+0000 7f367a59c640 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556b12b5c3c0 0x556b12b5e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.494+0000 7f3661ffb640 10 client.17214 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:17.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.494+0000 7f367a59c640 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556b12b5c3c0 0x556b12b5e7b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a29291a9fef0f4b server_cookie=a8e1283b88aece1e in_seq=0 out_seq=0 2024-04-16T05:40:17.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.494+0000 7f3661ffb640 1 -- 172.21.15.86:0/2804509349 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b12ab5b00 con 0x556b12b5c3c0 2024-04-16T05:40:17.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.494+0000 7f3661ffb640 10 client.17214 handle_command_reply: tid=0 2024-04-16T05:40:17.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.494+0000 7f36839a0740 1 client.17214 shutdown 2024-04-16T05:40:17.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 -- 172.21.15.86:0/2804509349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b4c360 msgr2=0x556b12b49f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b4c360 0x556b12b49f80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3674015fd0 tx=0x7f36740056b0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 -- 172.21.15.86:0/2804509349 shutdown_connections 2024-04-16T05:40:17.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556b12b5c3c0 0x556b12b5e7b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b4c360 0x556b12b49f80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b49a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f3661ffb640 0 client.17214 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:17.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 --2- 172.21.15.86:0/2804509349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b42e60 0x556b12b49500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 -- 172.21.15.86:0/2804509349 >> 172.21.15.86:0/2804509349 conn(0x556b12b3d590 msgr2=0x556b12b3f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 -- 172.21.15.86:0/2804509349 shutdown_connections 2024-04-16T05:40:17.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 1 -- 172.21.15.86:0/2804509349 wait complete. 2024-04-16T05:40:17.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.495+0000 7f36839a0740 20 client.17214 trim_cache size 0 max 16384 2024-04-16T05:40:17.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f36839a0740 1 Processor -- start 2024-04-16T05:40:17.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f36839a0740 1 -- start start 2024-04-16T05:40:17.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f36839a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b4bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f36839a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b57f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f36839a0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b48660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f36839a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f367c1af500 con 0x556b12b42e60 2024-04-16T05:40:17.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f36839a0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f367c1b1040 con 0x556b12b4c360 2024-04-16T05:40:17.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f36839a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f367c1afbd0 con 0x556b12b43780 2024-04-16T05:40:17.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f367a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b4bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f367a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b4bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39970/0 (socket says 172.21.15.86:39970) 2024-04-16T05:40:17.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f367a59c640 1 -- 172.21.15.86:0/4259129927 learned_addr learned my addr 172.21.15.86:0/4259129927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f367a59c640 1 -- 172.21.15.86:0/4259129927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 msgr2=0x556b12b57f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f367a59c640 1 --2- 172.21.15.86:0/4259129927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b57f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f367a59c640 1 -- 172.21.15.86:0/4259129927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 msgr2=0x556b12b48660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f367a59c640 1 --2- 172.21.15.86:0/4259129927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b48660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f367a59c640 1 -- 172.21.15.86:0/4259129927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b12b38c30 con 0x556b12b42e60 2024-04-16T05:40:17.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.508+0000 7f367a59c640 1 --2- 172.21.15.86:0/4259129927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b4bc30 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f366c00efd0 tx=0x7f366c0d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb88f01748754171 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f3663fff640 1 -- 172.21.15.86:0/4259129927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c01f040 con 0x556b12b42e60 2024-04-16T05:40:17.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f3663fff640 1 -- 172.21.15.86:0/4259129927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366c0a1240 con 0x556b12b42e60 2024-04-16T05:40:17.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f3663fff640 1 -- 172.21.15.86:0/4259129927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c099690 con 0x556b12b42e60 2024-04-16T05:40:17.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 -- 172.21.15.86:0/4259129927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 msgr2=0x556b12b4bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 --2- 172.21.15.86:0/4259129927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b4bc30 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f366c00efd0 tx=0x7f366c0d1040 comp rx=0 tx=0).stop 2024-04-16T05:40:17.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 -- 172.21.15.86:0/4259129927 shutdown_connections 2024-04-16T05:40:17.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 --2- 172.21.15.86:0/4259129927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b48660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 --2- 172.21.15.86:0/4259129927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b43780 0x556b12b57f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 --2- 172.21.15.86:0/4259129927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b42e60 0x556b12b4bc30 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 -- 172.21.15.86:0/4259129927 >> 172.21.15.86:0/4259129927 conn(0x556b12b3d590 msgr2=0x556b12b494c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 -- 172.21.15.86:0/4259129927 shutdown_connections 2024-04-16T05:40:17.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 -- 172.21.15.86:0/4259129927 wait complete. 2024-04-16T05:40:17.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 -- start start 2024-04-16T05:40:17.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.509+0000 7f36839a0740 1 Processor -- start 2024-04-16T05:40:17.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f36839a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b42e60 0x556b12b548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f36839a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b43780 0x556b12b4fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f36839a0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f36839a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f367c1b0970 con 0x556b12b43780 2024-04-16T05:40:17.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f36839a0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f367c1b02a0 con 0x556b12b4c360 2024-04-16T05:40:17.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f36839a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f367c1ad4c0 con 0x556b12b42e60 2024-04-16T05:40:17.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f36839a0740 10 client.0 _ll_register_callbacks cb 0x7f3670009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:17.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3679d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b43780 0x556b12b4fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3679d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b43780 0x556b12b4fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39982/0 (socket says 172.21.15.86:39982) 2024-04-16T05:40:17.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3679d9b640 1 -- 172.21.15.86:0/965690766 learned_addr learned my addr 172.21.15.86:0/965690766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:17.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f367ad9d640 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f367a59c640 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b42e60 0x556b12b548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3679d9b640 1 -- 172.21.15.86:0/965690766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b42e60 msgr2=0x556b12b548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3679d9b640 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b42e60 0x556b12b548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3679d9b640 1 -- 172.21.15.86:0/965690766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 msgr2=0x556b12b501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:17.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3679d9b640 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3679d9b640 1 -- 172.21.15.86:0/965690766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b12b50a10 con 0x556b12b43780 2024-04-16T05:40:17.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f367a59c640 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b42e60 0x556b12b548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:17.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3679d9b640 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b43780 0x556b12b4fc60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3670005ab0 tx=0x7f36700056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd10ac9e2f4d75a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:17.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:17.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3661ffb640 1 -- 172.21.15.86:0/965690766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3670042070 con 0x556b12b43780 2024-04-16T05:40:17.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f3661ffb640 1 -- 172.21.15.86:0/965690766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3670004450 con 0x556b12b43780 2024-04-16T05:40:17.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.510+0000 7f36839a0740 1 -- 172.21.15.86:0/965690766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f366c0a1240 con 0x556b12b43780 2024-04-16T05:40:17.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f3661ffb640 1 -- 172.21.15.86:0/965690766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f367004e050 con 0x556b12b43780 2024-04-16T05:40:17.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f36839a0740 10 client.17220 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:17.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f36839a0740 1 -- 172.21.15.86:0/965690766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b12bc2750 con 0x556b12b43780 2024-04-16T05:40:17.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f3661ffb640 1 -- 172.21.15.86:0/965690766 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366c0a1240 con 0x556b12b43780 2024-04-16T05:40:17.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f3661ffb640 1 -- 172.21.15.86:0/965690766 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3670047640 con 0x556b12b43780 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f36839a0740 10 client.17220 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f36839a0740 10 client.17220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f36839a0740 20 client.17220 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f36839a0740 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556b12bc2ed0 0x556b12bc52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f36839a0740 4 client.17220 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f36839a0740 1 -- 172.21.15.86:0/965690766 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b12aaf970 con 0x556b12bc2ed0 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.511+0000 7f367a59c640 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556b12bc2ed0 0x556b12bc52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f3661ffb640 10 client.17220 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f367a59c640 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556b12bc2ed0 0x556b12bc52f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b2105c23d809c64 server_cookie=c2e57bf244c6025b in_seq=0 out_seq=0 2024-04-16T05:40:17.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f3661ffb640 1 -- 172.21.15.86:0/965690766 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b12aaf970 con 0x556b12bc2ed0 2024-04-16T05:40:17.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f3661ffb640 10 client.17220 handle_command_reply: tid=0 2024-04-16T05:40:17.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 client.17220 shutdown 2024-04-16T05:40:17.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 -- 172.21.15.86:0/965690766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b43780 msgr2=0x556b12b4fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b43780 0x556b12b4fc60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3670005ab0 tx=0x7f36700056d0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 -- 172.21.15.86:0/965690766 shutdown_connections 2024-04-16T05:40:17.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556b12bc2ed0 0x556b12bc52f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b12b4c360 0x556b12b501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f3661ffb640 0 client.17220 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:17.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b12b43780 0x556b12b4fc60 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 --2- 172.21.15.86:0/965690766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b12b42e60 0x556b12b548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 -- 172.21.15.86:0/965690766 >> 172.21.15.86:0/965690766 conn(0x556b12b3d590 msgr2=0x556b12b3bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 -- 172.21.15.86:0/965690766 shutdown_connections 2024-04-16T05:40:17.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 1 -- 172.21.15.86:0/965690766 wait complete. 2024-04-16T05:40:17.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f36839a0740 20 client.17220 trim_cache size 0 max 16384 2024-04-16T05:40:17.522 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:17.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f3680dba640 1 -- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f36640a3d20 msgr2=0x7f36640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f3680dba640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f36640a3d20 0x7f36640a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f36740118e0 tx=0x7f3674016000 comp rx=0 tx=0).stop 2024-04-16T05:40:17.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f3680dba640 1 -- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101130 msgr2=0x7f367c19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:17.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.512+0000 7f3680dba640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101130 0x7f367c19e660 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f366c00ea80 tx=0x7f366c00ef40 comp rx=0 tx=0).stop 2024-04-16T05:40:17.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.513+0000 7f3680dba640 1 -- 172.21.15.86:0/2709983038 shutdown_connections 2024-04-16T05:40:17.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.513+0000 7f3680dba640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f36640a3d20 0x7f36640a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.513+0000 7f3680dba640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f367c198770 0x7f367c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.513+0000 7f3680dba640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f367c106200 0x7f367c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.513+0000 7f3680dba640 1 --2- 172.21.15.86:0/2709983038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f367c101130 0x7f367c19e660 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:17.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.513+0000 7f3680dba640 1 -- 172.21.15.86:0/2709983038 >> 172.21.15.86:0/2709983038 conn(0x7f367c0780b0 msgr2=0x7f367c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:17.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.513+0000 7f3680dba640 1 -- 172.21.15.86:0/2709983038 shutdown_connections 2024-04-16T05:40:17.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:17.513+0000 7f3680dba640 1 -- 172.21.15.86:0/2709983038 wait complete. 2024-04-16T05:40:17.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:17 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4085079949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:17.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:17 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4085079949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:18.525 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:40:18.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:40:18.526 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:40:18.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:18 smithi086 ceph-mon[32028]: pgmap v153: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:40:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.648+0000 7fd5bd663640 1 -- 172.21.15.86:0/4097828292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8107320 msgr2=0x7fd5b810eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.648+0000 7fd5bd663640 1 --2- 172.21.15.86:0/4097828292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8107320 0x7fd5b810eef0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009ca0 tx=0x7fd5b002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.648+0000 7fd5bd663640 1 -- 172.21.15.86:0/4097828292 shutdown_connections 2024-04-16T05:40:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.648+0000 7fd5bd663640 1 --2- 172.21.15.86:0/4097828292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8107320 0x7fd5b810eef0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.648+0000 7fd5bd663640 1 --2- 172.21.15.86:0/4097828292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b806b2a0 0x7fd5b8106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.648+0000 7fd5bd663640 1 --2- 172.21.15.86:0/4097828292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5b806a8f0 0x7fd5b806acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.648+0000 7fd5bd663640 1 -- 172.21.15.86:0/4097828292 >> 172.21.15.86:0/4097828292 conn(0x7fd5b8077f60 msgr2=0x7fd5b8078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.648+0000 7fd5bd663640 1 -- 172.21.15.86:0/4097828292 shutdown_connections 2024-04-16T05:40:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.648+0000 7fd5bd663640 1 -- 172.21.15.86:0/4097828292 wait complete. 2024-04-16T05:40:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5bd663640 1 Processor -- start 2024-04-16T05:40:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5bd663640 1 -- start start 2024-04-16T05:40:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5bd663640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5b806a8f0 0x7fd5b819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5bd663640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b8107320 0x7fd5b819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5bd663640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8198690 0x7fd5b8198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5bd663640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81123b0 con 0x7fd5b8198690 2024-04-16T05:40:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5bd663640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b8112230 con 0x7fd5b806a8f0 2024-04-16T05:40:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5bd663640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b8112530 con 0x7fd5b8107320 2024-04-16T05:40:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8198690 0x7fd5b8198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8198690 0x7fd5b8198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40014/0 (socket says 172.21.15.86:40014) 2024-04-16T05:40:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5b77fe640 1 -- 172.21.15.86:0/1261314751 learned_addr learned my addr 172.21.15.86:0/1261314751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.649+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b8107320 0x7fd5b819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5b806a8f0 0x7fd5b819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b77fe640 1 -- 172.21.15.86:0/1261314751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b8107320 msgr2=0x7fd5b819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b77fe640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b8107320 0x7fd5b819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b77fe640 1 -- 172.21.15.86:0/1261314751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5b806a8f0 msgr2=0x7fd5b819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b77fe640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5b806a8f0 0x7fd5b819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b77fe640 1 -- 172.21.15.86:0/1261314751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b8199290 con 0x7fd5b8198690 2024-04-16T05:40:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5b806a8f0 0x7fd5b819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b77fe640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8198690 0x7fd5b8198b10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd5b002c700 tx=0x7fd5b0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/1261314751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0039040 con 0x7fd5b8198690 2024-04-16T05:40:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/1261314751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b0044070 con 0x7fd5b8198690 2024-04-16T05:40:18.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5bd663640 1 -- 172.21.15.86:0/1261314751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b8199520 con 0x7fd5b8198690 2024-04-16T05:40:18.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/1261314751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00042a0 con 0x7fd5b8198690 2024-04-16T05:40:18.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.650+0000 7fd5bd663640 1 -- 172.21.15.86:0/1261314751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b81a5610 con 0x7fd5b8198690 2024-04-16T05:40:18.660 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:18 smithi194 ceph-mon[35865]: pgmap v153: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:40:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.652+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/1261314751 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd5b0004440 con 0x7fd5b8198690 2024-04-16T05:40:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5c024f740 1 Processor -- start 2024-04-16T05:40:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5c024f740 1 -- start start 2024-04-16T05:40:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bcfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 0x559542bd8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd8fb0 0x559542bdb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5c024f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559542b34e90 con 0x559542bcfab0 2024-04-16T05:40:18.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5c024f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559542ad24c0 con 0x559542bd03d0 2024-04-16T05:40:18.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5c024f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559542b30e50 con 0x559542bd8fb0 2024-04-16T05:40:18.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5b4ff9640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bcfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bcfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40026/0 (socket says 172.21.15.86:40026) 2024-04-16T05:40:18.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.653+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/736371530 learned_addr learned my addr 172.21.15.86:0/736371530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5b77fe640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/736371530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 0x559542bd8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/736371530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd8fb0 msgr2=0x559542bdb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/1261314751 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd5b00f2e40 con 0x7fd5b8198690 2024-04-16T05:40:18.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/736371530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd8fb0 0x559542bdb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/736371530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 msgr2=0x559542bd8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/736371530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 0x559542bd8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/736371530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954290f7e0 con 0x559542bcfab0 2024-04-16T05:40:18.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/736371530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 0x559542bd8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:18.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/736371530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bcfe90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000bc90 tx=0x7fd5a00127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22655c4fcf8a040f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5b77fe640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0048000 tx=0x7fd5b008d4f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5ad7fa640 1 -- 172.21.15.86:0/736371530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00120d0 con 0x559542bcfab0 2024-04-16T05:40:18.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5ad7fa640 1 -- 172.21.15.86:0/736371530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5a0016070 con 0x559542bcfab0 2024-04-16T05:40:18.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.654+0000 7fd5ad7fa640 1 -- 172.21.15.86:0/736371530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0010430 con 0x559542bcfab0 2024-04-16T05:40:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 -- 172.21.15.86:0/736371530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 msgr2=0x559542bcfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 --2- 172.21.15.86:0/736371530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bcfe90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000bc90 tx=0x7fd5a00127c0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 -- 172.21.15.86:0/736371530 shutdown_connections 2024-04-16T05:40:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 --2- 172.21.15.86:0/736371530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd8fb0 0x559542bdb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 --2- 172.21.15.86:0/736371530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 0x559542bd8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 --2- 172.21.15.86:0/736371530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bcfe90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 -- 172.21.15.86:0/736371530 >> 172.21.15.86:0/736371530 conn(0x559542bca1e0 msgr2=0x559542bca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 -- 172.21.15.86:0/736371530 shutdown_connections 2024-04-16T05:40:18.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 -- 172.21.15.86:0/736371530 wait complete. 2024-04-16T05:40:18.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 -- start start 2024-04-16T05:40:18.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.655+0000 7fd5c024f740 1 Processor -- start 2024-04-16T05:40:18.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 0x559542bd6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd8fb0 0x559542bd6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5c024f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559542bdbda0 con 0x559542bcfab0 2024-04-16T05:40:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5c024f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559542abd410 con 0x559542bd03d0 2024-04-16T05:40:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5c024f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559542bdbf20 con 0x559542bd8fb0 2024-04-16T05:40:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5c024f740 10 client.0 _ll_register_callbacks cb 0x559542ab8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40028/0 (socket says 172.21.15.86:40028) 2024-04-16T05:40:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/2326231823 learned_addr learned my addr 172.21.15.86:0/2326231823 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd8fb0 0x559542bd6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 0x559542bd6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/2326231823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd8fb0 msgr2=0x559542bd6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd8fb0 0x559542bd6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/2326231823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 msgr2=0x559542bd6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 0x559542bd6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/2326231823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55954296ff70 con 0x559542bcfab0 2024-04-16T05:40:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd6050 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0003b20 tx=0x7fd5a0003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f37af9f24f7d4b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd59e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd59e7fc640 1 -- 172.21.15.86:0/2326231823 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0003e70 con 0x559542bcfab0 2024-04-16T05:40:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd59e7fc640 1 -- 172.21.15.86:0/2326231823 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd5a0025040 con 0x559542bcfab0 2024-04-16T05:40:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.656+0000 7fd5c024f740 1 -- 172.21.15.86:0/2326231823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5a0016070 con 0x559542bcfab0 2024-04-16T05:40:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd59e7fc640 1 -- 172.21.15.86:0/2326231823 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5a0031050 con 0x559542bcfab0 2024-04-16T05:40:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd5c024f740 10 client.17232 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd5c024f740 1 -- 172.21.15.86:0/2326231823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559542807170 con 0x559542bcfab0 2024-04-16T05:40:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd59e7fc640 1 -- 172.21.15.86:0/2326231823 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5a0016070 con 0x559542bcfab0 2024-04-16T05:40:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd59e7fc640 1 -- 172.21.15.86:0/2326231823 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd5a0028070 con 0x559542bcfab0 2024-04-16T05:40:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd5c024f740 10 client.17232 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd5c024f740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd5c024f740 20 client.17232 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd5c024f740 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x559542be9220 0x559542beb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd5c024f740 4 client.17232 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd5c024f740 1 -- 172.21.15.86:0/2326231823 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559542b428b0 con 0x559542be9220 2024-04-16T05:40:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.657+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x559542be9220 0x559542beb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.658+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x559542be9220 0x559542beb610 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e20e3feb22d774f1 server_cookie=b4a4d09a0e65e100 in_seq=0 out_seq=0 2024-04-16T05:40:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.658+0000 7fd59e7fc640 10 client.17232 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.658+0000 7fd59e7fc640 1 -- 172.21.15.86:0/2326231823 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559542b428b0 con 0x559542be9220 2024-04-16T05:40:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.658+0000 7fd59e7fc640 10 client.17232 handle_command_reply: tid=0 2024-04-16T05:40:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.658+0000 7fd5c024f740 1 client.17232 shutdown 2024-04-16T05:40:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.658+0000 7fd5c024f740 1 -- 172.21.15.86:0/2326231823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 msgr2=0x559542bd6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.658+0000 7fd5c024f740 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd6050 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0003b20 tx=0x7fd5a0003980 comp rx=0 tx=0).stop 2024-04-16T05:40:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd5c024f740 1 -- 172.21.15.86:0/2326231823 shutdown_connections 2024-04-16T05:40:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd5c024f740 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x559542be9220 0x559542beb610 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd5c024f740 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd8fb0 0x559542bd6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd5c024f740 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd03d0 0x559542bd6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd59e7fc640 0 client.17232 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd5c024f740 1 --2- 172.21.15.86:0/2326231823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd6050 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd5c024f740 1 -- 172.21.15.86:0/2326231823 >> 172.21.15.86:0/2326231823 conn(0x559542bca1e0 msgr2=0x559542bcbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd5c024f740 1 -- 172.21.15.86:0/2326231823 shutdown_connections 2024-04-16T05:40:18.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd5c024f740 1 -- 172.21.15.86:0/2326231823 wait complete. 2024-04-16T05:40:18.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.659+0000 7fd5c024f740 20 client.17232 trim_cache size 0 max 16384 2024-04-16T05:40:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.676+0000 7fd5c024f740 1 Processor -- start 2024-04-16T05:40:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.676+0000 7fd5c024f740 1 -- start start 2024-04-16T05:40:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.676+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.676+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd03d0 0x559542bcbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.676+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd8fb0 0x559542bcc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.676+0000 7fd5c024f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81af3c0 con 0x559542bcfab0 2024-04-16T05:40:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.676+0000 7fd5c024f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81b0f00 con 0x559542bd8fb0 2024-04-16T05:40:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.676+0000 7fd5c024f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81afa90 con 0x559542bd03d0 2024-04-16T05:40:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40034/0 (socket says 172.21.15.86:40034) 2024-04-16T05:40:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/3927784650 learned_addr learned my addr 172.21.15.86:0/3927784650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/3927784650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd03d0 msgr2=0x559542bcbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/3927784650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd03d0 0x559542bcbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/3927784650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd8fb0 msgr2=0x559542bcc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/3927784650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd8fb0 0x559542bcc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/3927784650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559542bdbe20 con 0x559542bcfab0 2024-04-16T05:40:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/3927784650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd68f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0012800 tx=0x7fd5a0003720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89ac809311dcbdc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5acff9640 1 -- 172.21.15.86:0/3927784650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000bd80 con 0x559542bcfab0 2024-04-16T05:40:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5acff9640 1 -- 172.21.15.86:0/3927784650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5a0016070 con 0x559542bcfab0 2024-04-16T05:40:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.677+0000 7fd5acff9640 1 -- 172.21.15.86:0/3927784650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00053f0 con 0x559542bcfab0 2024-04-16T05:40:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 -- 172.21.15.86:0/3927784650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 msgr2=0x559542bd68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 --2- 172.21.15.86:0/3927784650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd68f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0012800 tx=0x7fd5a0003720 comp rx=0 tx=0).stop 2024-04-16T05:40:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 -- 172.21.15.86:0/3927784650 shutdown_connections 2024-04-16T05:40:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 --2- 172.21.15.86:0/3927784650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bd8fb0 0x559542bcc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 --2- 172.21.15.86:0/3927784650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd03d0 0x559542bcbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 --2- 172.21.15.86:0/3927784650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bcfab0 0x559542bd68f0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 -- 172.21.15.86:0/3927784650 >> 172.21.15.86:0/3927784650 conn(0x559542bca1e0 msgr2=0x559542bcb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 -- 172.21.15.86:0/3927784650 shutdown_connections 2024-04-16T05:40:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 -- 172.21.15.86:0/3927784650 wait complete. 2024-04-16T05:40:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 -- start start 2024-04-16T05:40:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 Processor -- start 2024-04-16T05:40:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bcfab0 0x559542bed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd03d0 0x559542bed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.678+0000 7fd5c024f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bd8fb0 0x559542beda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5c024f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81b0830 con 0x559542bd8fb0 2024-04-16T05:40:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5c024f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81b0160 con 0x559542bcfab0 2024-04-16T05:40:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5c024f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81ad380 con 0x559542bd03d0 2024-04-16T05:40:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5c024f740 10 client.0 _ll_register_callbacks cb 0x7fd5a000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bd8fb0 0x559542beda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bd8fb0 0x559542beda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40048/0 (socket says 172.21.15.86:40048) 2024-04-16T05:40:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/1378418183 learned_addr learned my addr 172.21.15.86:0/1378418183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/1378418183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd03d0 msgr2=0x559542bed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd03d0 0x559542bed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/1378418183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bcfab0 msgr2=0x559542bed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bcfab0 0x559542bed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5ae7fc640 1 -- 172.21.15.86:0/1378418183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559542c4f410 con 0x559542bd8fb0 2024-04-16T05:40:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5ae7fc640 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bd8fb0 0x559542beda90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0003670 tx=0x7fd5a000bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb88d0ce9869caed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd59e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd59e7fc640 1 -- 172.21.15.86:0/1378418183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00039c0 con 0x559542bd8fb0 2024-04-16T05:40:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd59e7fc640 1 -- 172.21.15.86:0/1378418183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd5a0010430 con 0x559542bd8fb0 2024-04-16T05:40:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5c024f740 1 -- 172.21.15.86:0/1378418183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5a0016070 con 0x559542bd8fb0 2024-04-16T05:40:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd59e7fc640 1 -- 172.21.15.86:0/1378418183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5a0044050 con 0x559542bd8fb0 2024-04-16T05:40:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5c024f740 10 client.17238 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.679+0000 7fd5c024f740 1 -- 172.21.15.86:0/1378418183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559542c50650 con 0x559542bd8fb0 2024-04-16T05:40:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd59e7fc640 1 -- 172.21.15.86:0/1378418183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5a0016070 con 0x559542bd8fb0 2024-04-16T05:40:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd59e7fc640 1 -- 172.21.15.86:0/1378418183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd5a003f030 con 0x559542bd8fb0 2024-04-16T05:40:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd5c024f740 10 client.17238 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd5c024f740 10 client.17238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd5c024f740 20 client.17238 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd5c024f740 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x559542c50d90 0x559542c531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd5c024f740 4 client.17238 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:40:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd5c024f740 1 -- 172.21.15.86:0/1378418183 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559542b3c720 con 0x559542c50d90 2024-04-16T05:40:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x559542c50d90 0x559542c531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x559542c50d90 0x559542c531b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c45a6fb30b825b server_cookie=54fd1f761845d7f in_seq=0 out_seq=0 2024-04-16T05:40:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.680+0000 7fd59e7fc640 10 client.17238 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.681+0000 7fd59e7fc640 1 -- 172.21.15.86:0/1378418183 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559542b3c720 con 0x559542c50d90 2024-04-16T05:40:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.681+0000 7fd59e7fc640 10 client.17238 handle_command_reply: tid=0 2024-04-16T05:40:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.681+0000 7fd5c024f740 1 client.17238 shutdown 2024-04-16T05:40:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.681+0000 7fd5c024f740 1 -- 172.21.15.86:0/1378418183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bd8fb0 msgr2=0x559542beda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.681+0000 7fd5c024f740 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bd8fb0 0x559542beda90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0003670 tx=0x7fd5a000bfb0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5c024f740 1 -- 172.21.15.86:0/1378418183 shutdown_connections 2024-04-16T05:40:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5c024f740 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x559542c50d90 0x559542c531b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5c024f740 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559542bd8fb0 0x559542beda90 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5c024f740 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559542bd03d0 0x559542bed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd59e7fc640 0 client.17238 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5c024f740 1 --2- 172.21.15.86:0/1378418183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559542bcfab0 0x559542bed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5c024f740 1 -- 172.21.15.86:0/1378418183 >> 172.21.15.86:0/1378418183 conn(0x559542bca1e0 msgr2=0x559542bcbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5c024f740 1 -- 172.21.15.86:0/1378418183 shutdown_connections 2024-04-16T05:40:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5c024f740 1 -- 172.21.15.86:0/1378418183 wait complete. 2024-04-16T05:40:18.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5c024f740 20 client.17238 trim_cache size 0 max 16384 2024-04-16T05:40:18.692 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:40:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:40:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "c3d0b082-dd52-4616-905a-80720da0424e", 2024-04-16T05:40:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:40:18.693 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:40:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 -- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd5a40a3d20 msgr2=0x7fd5a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0048000 tx=0x7fd5b008d4f0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 -- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8198690 msgr2=0x7fd5b8198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8198690 0x7fd5b8198b10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd5b002c700 tx=0x7fd5b0032f20 comp rx=0 tx=0).stop 2024-04-16T05:40:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 -- 172.21.15.86:0/1261314751 shutdown_connections 2024-04-16T05:40:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b8198690 0x7fd5b8198b10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b8107320 0x7fd5b819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 --2- 172.21.15.86:0/1261314751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5b806a8f0 0x7fd5b819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 -- 172.21.15.86:0/1261314751 >> 172.21.15.86:0/1261314751 conn(0x7fd5b8077f60 msgr2=0x7fd5b80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 -- 172.21.15.86:0/1261314751 shutdown_connections 2024-04-16T05:40:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.682+0000 7fd5bd663640 1 -- 172.21.15.86:0/1261314751 wait complete. 2024-04-16T05:40:18.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:40:18.696 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:40:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:18 smithi196 ceph-mon[36373]: pgmap v153: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:40:18.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 -- 172.21.15.86:0/3360681758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0714103280 msgr2=0x7f0714103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 --2- 172.21.15.86:0/3360681758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0714103280 0x7f0714103660 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f070c009ca0 tx=0x7f070c02c230 comp rx=0 tx=0).stop 2024-04-16T05:40:18.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 -- 172.21.15.86:0/3360681758 shutdown_connections 2024-04-16T05:40:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 --2- 172.21.15.86:0/3360681758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f071410c5c0 0x7f071410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 --2- 172.21.15.86:0/3360681758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0714103ba0 0x7f071410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 --2- 172.21.15.86:0/3360681758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0714103280 0x7f0714103660 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 -- 172.21.15.86:0/3360681758 >> 172.21.15.86:0/3360681758 conn(0x7f0714078000 msgr2=0x7f0714078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 -- 172.21.15.86:0/3360681758 shutdown_connections 2024-04-16T05:40:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 -- 172.21.15.86:0/3360681758 wait complete. 2024-04-16T05:40:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 Processor -- start 2024-04-16T05:40:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.843+0000 7f07193a8640 1 -- start start 2024-04-16T05:40:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07193a8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0714103280 0x7f071419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07193a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0714103ba0 0x7f071419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07193a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071410c5c0 0x7f07141987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07193a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07141123a0 con 0x7f071410c5c0 2024-04-16T05:40:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07193a8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0714112220 con 0x7f0714103280 2024-04-16T05:40:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07193a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0714112520 con 0x7f0714103ba0 2024-04-16T05:40:18.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071410c5c0 0x7f07141987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071410c5c0 0x7f07141987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40056/0 (socket says 172.21.15.86:40056) 2024-04-16T05:40:18.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07137fe640 1 -- 172.21.15.86:0/1369934323 learned_addr learned my addr 172.21.15.86:0/1369934323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f0712ffd640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0714103280 0x7f071419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07127fc640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0714103ba0 0x7f071419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07137fe640 1 -- 172.21.15.86:0/1369934323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0714103ba0 msgr2=0x7f071419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07137fe640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0714103ba0 0x7f071419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07137fe640 1 -- 172.21.15.86:0/1369934323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0714103280 msgr2=0x7f071419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07137fe640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0714103280 0x7f071419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07137fe640 1 -- 172.21.15.86:0/1369934323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0714198f70 con 0x7f071410c5c0 2024-04-16T05:40:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07137fe640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071410c5c0 0x7f07141987a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f070400c970 tx=0x7f070400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f070bfff640 1 -- 172.21.15.86:0/1369934323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704007bb0 con 0x7f071410c5c0 2024-04-16T05:40:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f070bfff640 1 -- 172.21.15.86:0/1369934323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0704007d50 con 0x7f071410c5c0 2024-04-16T05:40:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07193a8640 1 -- 172.21.15.86:0/1369934323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0714199260 con 0x7f071410c5c0 2024-04-16T05:40:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f070bfff640 1 -- 172.21.15.86:0/1369934323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070400f450 con 0x7f071410c5c0 2024-04-16T05:40:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.844+0000 7f07193a8640 1 -- 172.21.15.86:0/1369934323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07141a5480 con 0x7f071410c5c0 2024-04-16T05:40:18.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.846+0000 7f070bfff640 1 -- 172.21.15.86:0/1369934323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0704004f20 con 0x7f071410c5c0 2024-04-16T05:40:18.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f071bf94740 1 Processor -- start 2024-04-16T05:40:18.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f071bf94740 1 -- start start 2024-04-16T05:40:18.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f071bf94740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x5613862c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f071bf94740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x5613862c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f071bf94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f071bf94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561386225180 con 0x5613862c9350 2024-04-16T05:40:18.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f071bf94740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5613861c2360 con 0x5613862c0770 2024-04-16T05:40:18.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f071bf94740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561386221140 con 0x5613862bfe50 2024-04-16T05:40:18.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f070bfff640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f07000a3d20 0x7f07000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f07127fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f07127fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40072/0 (socket says 172.21.15.86:40072) 2024-04-16T05:40:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f07127fc640 1 -- 172.21.15.86:0/590254243 learned_addr learned my addr 172.21.15.86:0/590254243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.847+0000 7f07137fe640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f07000a3d20 0x7f07000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07127fc640 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x5613862c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f0712ffd640 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x5613862c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f070bfff640 1 -- 172.21.15.86:0/1369934323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f07040d0850 con 0x7f071410c5c0 2024-04-16T05:40:18.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07127fc640 1 -- 172.21.15.86:0/590254243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 msgr2=0x5613862c0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07127fc640 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x5613862c0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07127fc640 1 -- 172.21.15.86:0/590254243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 msgr2=0x5613862c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07127fc640 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x5613862c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07127fc640 1 -- 172.21.15.86:0/590254243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613860017d0 con 0x5613862c9350 2024-04-16T05:40:18.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f0712ffd640 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x5613862c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:18.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07127fc640 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862cb740 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00cfc0 tx=0x7f06fc00c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447a4ce8e0b6818b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07137fe640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f07000a3d20 0x7f07000a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f070409c410 tx=0x7f0704091910 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07097fa640 1 -- 172.21.15.86:0/590254243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc018030 con 0x5613862c9350 2024-04-16T05:40:18.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07097fa640 1 -- 172.21.15.86:0/590254243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc005020 con 0x5613862c9350 2024-04-16T05:40:18.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.848+0000 7f07097fa640 1 -- 172.21.15.86:0/590254243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc004ab0 con 0x5613862c9350 2024-04-16T05:40:18.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 -- 172.21.15.86:0/590254243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 msgr2=0x5613862cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862cb740 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00cfc0 tx=0x7f06fc00c5a0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 -- 172.21.15.86:0/590254243 shutdown_connections 2024-04-16T05:40:18.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862cb740 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x5613862c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 --2- 172.21.15.86:0/590254243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x5613862c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 -- 172.21.15.86:0/590254243 >> 172.21.15.86:0/590254243 conn(0x5613862ba580 msgr2=0x5613862ba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 -- 172.21.15.86:0/590254243 shutdown_connections 2024-04-16T05:40:18.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 -- 172.21.15.86:0/590254243 wait complete. 2024-04-16T05:40:18.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 -- start start 2024-04-16T05:40:18.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.849+0000 7f071bf94740 1 Processor -- start 2024-04-16T05:40:18.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862bfe50 0x5613862c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x5613862c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862c9350 0x5613862c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5613862cc140 con 0x5613862bfe50 2024-04-16T05:40:18.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5613861de2c0 con 0x5613862c0770 2024-04-16T05:40:18.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5613862cc2c0 con 0x5613862c9350 2024-04-16T05:40:18.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 10 client.0 _ll_register_callbacks cb 0x5613861c5090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:18.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f07127fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862bfe50 0x5613862c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f07127fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862bfe50 0x5613862c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40088/0 (socket says 172.21.15.86:40088) 2024-04-16T05:40:18.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f07127fc640 1 -- 172.21.15.86:0/3513565113 learned_addr learned my addr 172.21.15.86:0/3513565113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f07127fc640 1 -- 172.21.15.86:0/3513565113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862c9350 msgr2=0x5613862c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f07127fc640 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862c9350 0x5613862c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f07127fc640 1 -- 172.21.15.86:0/3513565113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 msgr2=0x5613862c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f07127fc640 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x5613862c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f07127fc640 1 -- 172.21.15.86:0/3513565113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561385f373f0 con 0x5613862bfe50 2024-04-16T05:40:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f07127fc640 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862bfe50 0x5613862c6420 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f06fc005710 tx=0x7f06fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78933c9232b8c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f06fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f06fa7fc640 1 -- 172.21.15.86:0/3513565113 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc018040 con 0x5613862bfe50 2024-04-16T05:40:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f06fa7fc640 1 -- 172.21.15.86:0/3513565113 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f06fc004b70 con 0x5613862bfe50 2024-04-16T05:40:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 1 -- 172.21.15.86:0/3513565113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06fc005020 con 0x5613862bfe50 2024-04-16T05:40:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f06fa7fc640 1 -- 172.21.15.86:0/3513565113 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06fc02e050 con 0x5613862bfe50 2024-04-16T05:40:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 10 client.17250 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:18.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.850+0000 7f071bf94740 1 -- 172.21.15.86:0/3513565113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561385ef70f0 con 0x5613862bfe50 2024-04-16T05:40:18.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f06fa7fc640 1 -- 172.21.15.86:0/3513565113 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc005020 con 0x5613862bfe50 2024-04-16T05:40:18.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f06fa7fc640 1 -- 172.21.15.86:0/3513565113 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06fc01c410 con 0x5613862bfe50 2024-04-16T05:40:18.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f071bf94740 10 client.17250 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:18.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f071bf94740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:18.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f071bf94740 20 client.17250 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:18.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f071bf94740 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5613862d9560 0x5613862db950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f071bf94740 4 client.17250 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f071bf94740 1 -- 172.21.15.86:0/3513565113 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561386232ba0 con 0x5613862d9560 2024-04-16T05:40:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f0712ffd640 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5613862d9560 0x5613862db950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f06fa7fc640 10 client.17250 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.851+0000 7f0712ffd640 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5613862d9560 0x5613862db950 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe9cc64d52f81ba server_cookie=4dab418d160d7006 in_seq=0 out_seq=0 2024-04-16T05:40:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f06fa7fc640 1 -- 172.21.15.86:0/3513565113 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561386232ba0 con 0x5613862d9560 2024-04-16T05:40:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f06fa7fc640 10 client.17250 handle_command_reply: tid=0 2024-04-16T05:40:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 client.17250 shutdown 2024-04-16T05:40:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 -- 172.21.15.86:0/3513565113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862bfe50 msgr2=0x5613862c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862bfe50 0x5613862c6420 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f06fc005710 tx=0x7f06fc0046c0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 -- 172.21.15.86:0/3513565113 shutdown_connections 2024-04-16T05:40:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5613862d9560 0x5613862db950 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862c9350 0x5613862c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x5613862c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 --2- 172.21.15.86:0/3513565113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862bfe50 0x5613862c6420 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f06fa7fc640 0 client.17250 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:18.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 -- 172.21.15.86:0/3513565113 >> 172.21.15.86:0/3513565113 conn(0x5613862ba580 msgr2=0x5613862bc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 -- 172.21.15.86:0/3513565113 shutdown_connections 2024-04-16T05:40:18.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 1 -- 172.21.15.86:0/3513565113 wait complete. 2024-04-16T05:40:18.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.852+0000 7f071bf94740 20 client.17250 trim_cache size 0 max 16384 2024-04-16T05:40:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f071bf94740 1 Processor -- start 2024-04-16T05:40:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f071bf94740 1 -- start start 2024-04-16T05:40:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f071bf94740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x5613862d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f071bf94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c0770 0x5613862e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f071bf94740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c9350 0x5613862e1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f071bf94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07141af540 con 0x5613862c0770 2024-04-16T05:40:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f071bf94740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07141b1080 con 0x5613862c9350 2024-04-16T05:40:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f071bf94740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07141afc10 con 0x5613862bfe50 2024-04-16T05:40:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f0712ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c0770 0x5613862e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f0712ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c0770 0x5613862e0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40100/0 (socket says 172.21.15.86:40100) 2024-04-16T05:40:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f0712ffd640 1 -- 172.21.15.86:0/4139303724 learned_addr learned my addr 172.21.15.86:0/4139303724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f0712ffd640 1 -- 172.21.15.86:0/4139303724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 msgr2=0x5613862d5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f0712ffd640 1 --2- 172.21.15.86:0/4139303724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x5613862d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f0712ffd640 1 -- 172.21.15.86:0/4139303724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c9350 msgr2=0x5613862e1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f0712ffd640 1 --2- 172.21.15.86:0/4139303724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c9350 0x5613862e1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f0712ffd640 1 -- 172.21.15.86:0/4139303724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613862b4b70 con 0x5613862c0770 2024-04-16T05:40:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.870+0000 7f0712ffd640 1 --2- 172.21.15.86:0/4139303724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c0770 0x5613862e0ef0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f070c02c770 tx=0x7f070c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d3758926467028 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f0708ff9640 1 -- 172.21.15.86:0/4139303724 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c04a070 con 0x5613862c0770 2024-04-16T05:40:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f0708ff9640 1 -- 172.21.15.86:0/4139303724 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c03a040 con 0x5613862c0770 2024-04-16T05:40:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f0708ff9640 1 -- 172.21.15.86:0/4139303724 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c039040 con 0x5613862c0770 2024-04-16T05:40:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f071bf94740 1 -- 172.21.15.86:0/4139303724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c0770 msgr2=0x5613862e0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f071bf94740 1 --2- 172.21.15.86:0/4139303724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c0770 0x5613862e0ef0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f070c02c770 tx=0x7f070c005f10 comp rx=0 tx=0).stop 2024-04-16T05:40:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f071bf94740 1 -- 172.21.15.86:0/4139303724 shutdown_connections 2024-04-16T05:40:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f071bf94740 1 --2- 172.21.15.86:0/4139303724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c9350 0x5613862e1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f071bf94740 1 --2- 172.21.15.86:0/4139303724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c0770 0x5613862e0ef0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f071bf94740 1 --2- 172.21.15.86:0/4139303724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x5613862d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f071bf94740 1 -- 172.21.15.86:0/4139303724 >> 172.21.15.86:0/4139303724 conn(0x5613862ba580 msgr2=0x5613862b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f071bf94740 1 -- 172.21.15.86:0/4139303724 shutdown_connections 2024-04-16T05:40:18.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.871+0000 7f071bf94740 1 -- 172.21.15.86:0/4139303724 wait complete. 2024-04-16T05:40:18.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f071bf94740 1 -- start start 2024-04-16T05:40:18.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f071bf94740 1 Processor -- start 2024-04-16T05:40:18.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f071bf94740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x56138633f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f071bf94740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x56138633f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f071bf94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f071bf94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07141b09b0 con 0x5613862c9350 2024-04-16T05:40:18.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f071bf94740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07141b02e0 con 0x5613862c0770 2024-04-16T05:40:18.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f071bf94740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07141ad500 con 0x5613862bfe50 2024-04-16T05:40:18.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f071bf94740 10 client.0 _ll_register_callbacks cb 0x7f06fc00cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:18.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f07127fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f07127fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862d0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40114/0 (socket says 172.21.15.86:40114) 2024-04-16T05:40:18.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f07127fc640 1 -- 172.21.15.86:0/1130087091 learned_addr learned my addr 172.21.15.86:0/1130087091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:18.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f0712ffd640 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x56138633f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f07127fc640 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x56138633f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f07127fc640 1 -- 172.21.15.86:0/1130087091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 msgr2=0x56138633f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f07127fc640 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x56138633f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f07127fc640 1 -- 172.21.15.86:0/1130087091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 msgr2=0x56138633f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:18.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f07127fc640 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x56138633f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f07127fc640 1 -- 172.21.15.86:0/1130087091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613862d0ef0 con 0x5613862c9350 2024-04-16T05:40:18.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.872+0000 7f0712ffd640 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x56138633f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:18.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f07127fc640 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862d0800 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0058c0 tx=0x7f06fc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ca06429b9c265f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:18.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f06fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:18.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f06fa7fc640 1 -- 172.21.15.86:0/1130087091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc018070 con 0x5613862c9350 2024-04-16T05:40:18.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f06fa7fc640 1 -- 172.21.15.86:0/1130087091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f06fc00c9f0 con 0x5613862c9350 2024-04-16T05:40:18.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f071bf94740 1 -- 172.21.15.86:0/1130087091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f070c03a040 con 0x5613862c9350 2024-04-16T05:40:18.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f06fa7fc640 1 -- 172.21.15.86:0/1130087091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06fc01cc00 con 0x5613862c9350 2024-04-16T05:40:18.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f071bf94740 10 client.17256 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:18.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f071bf94740 1 -- 172.21.15.86:0/1130087091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561386347ab0 con 0x5613862c9350 2024-04-16T05:40:18.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f06fa7fc640 1 -- 172.21.15.86:0/1130087091 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c03a040 con 0x5613862c9350 2024-04-16T05:40:18.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f06fa7fc640 1 -- 172.21.15.86:0/1130087091 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06fc004d20 con 0x5613862c9350 2024-04-16T05:40:18.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f071bf94740 10 client.17256 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:18.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f071bf94740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:18.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f071bf94740 20 client.17256 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:18.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f071bf94740 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561386348180 0x56138634a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:18.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f071bf94740 4 client.17256 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:40:18.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f071bf94740 1 -- 172.21.15.86:0/1130087091 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56138622ca10 con 0x561386348180 2024-04-16T05:40:18.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.873+0000 7f0712ffd640 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561386348180 0x56138634a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:18.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f06fa7fc640 10 client.17256 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:18.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f0712ffd640 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561386348180 0x56138634a5a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad2a79214a45a55a server_cookie=de1f8b15594da90c in_seq=0 out_seq=0 2024-04-16T05:40:18.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f06fa7fc640 1 -- 172.21.15.86:0/1130087091 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56138622ca10 con 0x561386348180 2024-04-16T05:40:18.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f06fa7fc640 10 client.17256 handle_command_reply: tid=0 2024-04-16T05:40:18.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 client.17256 shutdown 2024-04-16T05:40:18.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 -- 172.21.15.86:0/1130087091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 msgr2=0x5613862d0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862d0800 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0058c0 tx=0x7f06fc005740 comp rx=0 tx=0).stop 2024-04-16T05:40:18.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 -- 172.21.15.86:0/1130087091 shutdown_connections 2024-04-16T05:40:18.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561386348180 0x56138634a5a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613862c9350 0x5613862d0800 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f06fa7fc640 0 client.17256 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:18.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5613862c0770 0x56138633f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 --2- 172.21.15.86:0/1130087091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613862bfe50 0x56138633f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 -- 172.21.15.86:0/1130087091 >> 172.21.15.86:0/1130087091 conn(0x5613862ba580 msgr2=0x5613862bb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 -- 172.21.15.86:0/1130087091 shutdown_connections 2024-04-16T05:40:18.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 1 -- 172.21.15.86:0/1130087091 wait complete. 2024-04-16T05:40:18.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.874+0000 7f071bf94740 20 client.17256 trim_cache size 0 max 16384 2024-04-16T05:40:18.887 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:40:18.887 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:40:18.887 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:40:18.887 INFO:teuthology.orchestra.run.smithi086.stdout: "c3d0b082-dd52-4616-905a-80720da0424e": { 2024-04-16T05:40:18.887 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:40:18.888 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c3d0b082-dd52-4616-905a-80720da0424e", 2024-04-16T05:40:18.888 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:40:18.888 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:40:18.888 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:40:18.888 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:40:18.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 -- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f07000a3d20 msgr2=0x7f07000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f07000a3d20 0x7f07000a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f070409c410 tx=0x7f0704091910 comp rx=0 tx=0).stop 2024-04-16T05:40:18.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 -- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071410c5c0 msgr2=0x7f07141987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:18.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071410c5c0 0x7f07141987a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f070400c970 tx=0x7f070400ce30 comp rx=0 tx=0).stop 2024-04-16T05:40:18.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 -- 172.21.15.86:0/1369934323 shutdown_connections 2024-04-16T05:40:18.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071410c5c0 0x7f07141987a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0714103ba0 0x7f071419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f07000a3d20 0x7f07000a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 --2- 172.21.15.86:0/1369934323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0714103280 0x7f071419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 -- 172.21.15.86:0/1369934323 >> 172.21.15.86:0/1369934323 conn(0x7f0714078000 msgr2=0x7f07140ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 -- 172.21.15.86:0/1369934323 shutdown_connections 2024-04-16T05:40:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:18.875+0000 7f07193a8640 1 -- 172.21.15.86:0/1369934323 wait complete. 2024-04-16T05:40:18.891 INFO:tasks.cephfs.filesystem:scrub status for tag:c3d0b082-dd52-4616-905a-80720da0424e - {'path': '/', 'tag': 'c3d0b082-dd52-4616-905a-80720da0424e', 'options': 'recursive,force'} 2024-04-16T05:40:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:20 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:40:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:20 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:20 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:40:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:20 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:20 smithi194 ceph-mon[35865]: pgmap v154: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-16T05:40:20.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:20 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:40:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:20 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:40:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:20 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:20 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:40:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:20 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:20 smithi086 ceph-mon[32028]: pgmap v154: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-16T05:40:20.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:20 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:40:20.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:20 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:40:20.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:20 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:20.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:20 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:40:20.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:20 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:20.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:20 smithi196 ceph-mon[36373]: pgmap v154: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-16T05:40:20.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:20 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:40:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:22 smithi194 ceph-mon[35865]: pgmap v155: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:40:22.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:22 smithi086 ceph-mon[32028]: pgmap v155: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:40:22.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:22 smithi196 ceph-mon[36373]: pgmap v155: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:40:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:24 smithi194 ceph-mon[35865]: pgmap v156: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-16T05:40:24.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:24 smithi086 ceph-mon[32028]: pgmap v156: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-16T05:40:24.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:24 smithi196 ceph-mon[36373]: pgmap v156: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-16T05:40:26.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:26 smithi194 ceph-mon[35865]: pgmap v157: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:26.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:26 smithi086 ceph-mon[32028]: pgmap v157: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:26.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:26 smithi196 ceph-mon[36373]: pgmap v157: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:27 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:40:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:27 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:40:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:27 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:40:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:27 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:40:27.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:27 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:40:27.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:27 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:40:27.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:27 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:40:27.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:27 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:40:27.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:27 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:40:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:27 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:40:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:27 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:40:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:27 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:40:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:28 smithi194 ceph-mon[35865]: pgmap v158: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:28.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:28 smithi086 ceph-mon[32028]: pgmap v158: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:28 smithi196 ceph-mon[36373]: pgmap v158: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:30 smithi194 ceph-mon[35865]: pgmap v159: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:30.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:30 smithi086 ceph-mon[32028]: pgmap v159: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:30 smithi196 ceph-mon[36373]: pgmap v159: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:32.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:32 smithi194 ceph-mon[35865]: pgmap v160: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:32.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:32 smithi086 ceph-mon[32028]: pgmap v160: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:32 smithi196 ceph-mon[36373]: pgmap v160: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:34.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:34 smithi194 ceph-mon[35865]: pgmap v161: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:34.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:34 smithi086 ceph-mon[32028]: pgmap v161: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:34.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:34 smithi196 ceph-mon[36373]: pgmap v161: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:36.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:36 smithi194 ceph-mon[35865]: pgmap v162: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:36.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:36 smithi086 ceph-mon[32028]: pgmap v162: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:36 smithi196 ceph-mon[36373]: pgmap v162: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:38.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:38 smithi194 ceph-mon[35865]: pgmap v163: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:38.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:38 smithi086 ceph-mon[32028]: pgmap v163: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:38.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:38 smithi196 ceph-mon[36373]: pgmap v163: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:40.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:40 smithi194 ceph-mon[35865]: pgmap v164: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:40:40.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:40 smithi086 ceph-mon[32028]: pgmap v164: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:40:40.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:40 smithi196 ceph-mon[36373]: pgmap v164: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:40:42.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:42 smithi194 ceph-mon[35865]: pgmap v165: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:42.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:42 smithi086 ceph-mon[32028]: pgmap v165: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:42.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:42 smithi196 ceph-mon[36373]: pgmap v165: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:44.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:44 smithi194 ceph-mon[35865]: pgmap v166: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:40:44.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:44 smithi086 ceph-mon[32028]: pgmap v166: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:40:44.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:44 smithi196 ceph-mon[36373]: pgmap v166: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:40:46.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:46 smithi194 ceph-mon[35865]: pgmap v167: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:46.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:46 smithi086 ceph-mon[32028]: pgmap v167: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:46.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:46 smithi196 ceph-mon[36373]: pgmap v167: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:48.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:48 smithi194 ceph-mon[35865]: pgmap v168: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:48.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:48 smithi086 ceph-mon[32028]: pgmap v168: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:48.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:48 smithi196 ceph-mon[36373]: pgmap v168: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:48.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:40:48.897 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:40:49.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.014+0000 7fce973e2640 1 -- 172.21.15.86:0/707379569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 msgr2=0x7fce9010c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.014+0000 7fce973e2640 1 --2- 172.21.15.86:0/707379569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 0x7fce9010c090 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009900 tx=0x7fce8c02bd60 comp rx=0 tx=0).stop 2024-04-16T05:40:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.014+0000 7fce973e2640 1 -- 172.21.15.86:0/707379569 shutdown_connections 2024-04-16T05:40:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.014+0000 7fce973e2640 1 --2- 172.21.15.86:0/707379569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9010c5d0 0x7fce9010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.014+0000 7fce973e2640 1 --2- 172.21.15.86:0/707379569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 0x7fce9010c090 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.014+0000 7fce973e2640 1 --2- 172.21.15.86:0/707379569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90103290 0x7fce90103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.014+0000 7fce973e2640 1 -- 172.21.15.86:0/707379569 >> 172.21.15.86:0/707379569 conn(0x7fce90077280 msgr2=0x7fce90077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.014+0000 7fce973e2640 1 -- 172.21.15.86:0/707379569 shutdown_connections 2024-04-16T05:40:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.014+0000 7fce973e2640 1 -- 172.21.15.86:0/707379569 wait complete. 2024-04-16T05:40:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce973e2640 1 Processor -- start 2024-04-16T05:40:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce973e2640 1 -- start start 2024-04-16T05:40:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce973e2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90103290 0x7fce9019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce973e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 0x7fce9019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce973e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9010c5d0 0x7fce901987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce973e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce90112460 con 0x7fce90103bb0 2024-04-16T05:40:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce973e2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fce901122e0 con 0x7fce90103290 2024-04-16T05:40:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce973e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce901125e0 con 0x7fce9010c5d0 2024-04-16T05:40:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce9497d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 0x7fce9019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce9497d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 0x7fce9019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52490/0 (socket says 172.21.15.86:52490) 2024-04-16T05:40:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce9497d640 1 -- 172.21.15.86:0/1368415437 learned_addr learned my addr 172.21.15.86:0/1368415437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.015+0000 7fce9517e640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90103290 0x7fce9019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce9597f640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9010c5d0 0x7fce901987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce9497d640 1 -- 172.21.15.86:0/1368415437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9010c5d0 msgr2=0x7fce901987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce9497d640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9010c5d0 0x7fce901987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce9497d640 1 -- 172.21.15.86:0/1368415437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90103290 msgr2=0x7fce9019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce9497d640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90103290 0x7fce9019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce9497d640 1 -- 172.21.15.86:0/1368415437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce90199080 con 0x7fce90103bb0 2024-04-16T05:40:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce9597f640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9010c5d0 0x7fce901987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce9497d640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 0x7fce9019ebf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009a20 tx=0x7fce8c02df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce867fc640 1 -- 172.21.15.86:0/1368415437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c043020 con 0x7fce90103bb0 2024-04-16T05:40:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce867fc640 1 -- 172.21.15.86:0/1368415437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8c0054e0 con 0x7fce90103bb0 2024-04-16T05:40:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce973e2640 1 -- 172.21.15.86:0/1368415437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce90199310 con 0x7fce90103bb0 2024-04-16T05:40:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce867fc640 1 -- 172.21.15.86:0/1368415437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c003ce0 con 0x7fce90103bb0 2024-04-16T05:40:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.016+0000 7fce973e2640 1 -- 172.21.15.86:0/1368415437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901a5630 con 0x7fce90103bb0 2024-04-16T05:40:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.018+0000 7fce867fc640 1 -- 172.21.15.86:0/1368415437 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce8c046020 con 0x7fce90103bb0 2024-04-16T05:40:49.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce99fce740 1 Processor -- start 2024-04-16T05:40:49.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce99fce740 1 -- start start 2024-04-16T05:40:49.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce867fc640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fce7c0a3d20 0x7fce7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce99fce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c60e50 0x55be59c61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce99fce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c69e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce99fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c6a350 0x55be59c6c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce99fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55be59bc6180 con 0x55be59c6a350 2024-04-16T05:40:49.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce99fce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55be59b63360 con 0x55be59c61770 2024-04-16T05:40:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce99fce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55be59bc2140 con 0x55be59c60e50 2024-04-16T05:40:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.019+0000 7fce9597f640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fce7c0a3d20 0x7fce7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce867fc640 1 -- 172.21.15.86:0/1368415437 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fce8c0f2c30 con 0x7fce90103bb0 2024-04-16T05:40:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c6a350 0x55be59c6c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c6a350 0x55be59c6c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52502/0 (socket says 172.21.15.86:52502) 2024-04-16T05:40:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9517e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c60e50 0x55be59c61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 -- 172.21.15.86:0/2363719998 learned_addr learned my addr 172.21.15.86:0/2363719998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9497d640 1 --2- 172.21.15.86:0/2363719998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c69e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 -- 172.21.15.86:0/2363719998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c60e50 msgr2=0x55be59c61230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 --2- 172.21.15.86:0/2363719998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c60e50 0x55be59c61230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 -- 172.21.15.86:0/2363719998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 msgr2=0x55be59c69e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 --2- 172.21.15.86:0/2363719998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c69e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 -- 172.21.15.86:0/2363719998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be599a27d0 con 0x55be59c6a350 2024-04-16T05:40:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fce7c0a3d20 0x7fce7c0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce8800ec60 tx=0x7fce88009e30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce9597f640 1 --2- 172.21.15.86:0/2363719998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c6a350 0x55be59c6c740 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fce8800cfd0 tx=0x7fce88018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f76da14ea2bd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce7b7fe640 1 -- 172.21.15.86:0/2363719998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88020020 con 0x55be59c6a350 2024-04-16T05:40:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce7b7fe640 1 -- 172.21.15.86:0/2363719998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8800e510 con 0x55be59c6a350 2024-04-16T05:40:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.020+0000 7fce7b7fe640 1 -- 172.21.15.86:0/2363719998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8801a3f0 con 0x55be59c6a350 2024-04-16T05:40:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 -- 172.21.15.86:0/2363719998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c6a350 msgr2=0x55be59c6c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 --2- 172.21.15.86:0/2363719998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c6a350 0x55be59c6c740 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fce8800cfd0 tx=0x7fce88018000 comp rx=0 tx=0).stop 2024-04-16T05:40:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 -- 172.21.15.86:0/2363719998 shutdown_connections 2024-04-16T05:40:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 --2- 172.21.15.86:0/2363719998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c6a350 0x55be59c6c740 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 --2- 172.21.15.86:0/2363719998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c69e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 --2- 172.21.15.86:0/2363719998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c60e50 0x55be59c61230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 -- 172.21.15.86:0/2363719998 >> 172.21.15.86:0/2363719998 conn(0x55be59c5b580 msgr2=0x55be59c5b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 -- 172.21.15.86:0/2363719998 shutdown_connections 2024-04-16T05:40:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 -- 172.21.15.86:0/2363719998 wait complete. 2024-04-16T05:40:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 -- start start 2024-04-16T05:40:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.021+0000 7fce99fce740 1 Processor -- start 2024-04-16T05:40:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce99fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce99fce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c61770 0x55be59c678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce99fce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c6a350 0x55be59c67e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce99fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55be59c6d0e0 con 0x55be59c60e50 2024-04-16T05:40:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce99fce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55be59b7f2c0 con 0x55be59c6a350 2024-04-16T05:40:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce99fce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55be59c6d260 con 0x55be59c61770 2024-04-16T05:40:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce99fce740 10 client.0 _ll_register_callbacks cb 0x55be59b66090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9517e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9517e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52512/0 (socket says 172.21.15.86:52512) 2024-04-16T05:40:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9517e640 1 -- 172.21.15.86:0/2338950694 learned_addr learned my addr 172.21.15.86:0/2338950694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9597f640 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c6a350 0x55be59c67e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9497d640 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c61770 0x55be59c678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9517e640 1 -- 172.21.15.86:0/2338950694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c61770 msgr2=0x55be59c678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9517e640 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c61770 0x55be59c678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9517e640 1 -- 172.21.15.86:0/2338950694 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c6a350 msgr2=0x55be59c67e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9517e640 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c6a350 0x55be59c67e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9517e640 1 -- 172.21.15.86:0/2338950694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be598d83f0 con 0x55be59c60e50 2024-04-16T05:40:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce9517e640 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c673a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fce8000bab0 tx=0x7fce8000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79927ccb9a363beb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce78ff9640 1 -- 172.21.15.86:0/2338950694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80011440 con 0x55be59c60e50 2024-04-16T05:40:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce78ff9640 1 -- 172.21.15.86:0/2338950694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fce800115e0 con 0x55be59c60e50 2024-04-16T05:40:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.022+0000 7fce99fce740 1 -- 172.21.15.86:0/2338950694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce8800e510 con 0x55be59c60e50 2024-04-16T05:40:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce78ff9640 1 -- 172.21.15.86:0/2338950694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce800117e0 con 0x55be59c60e50 2024-04-16T05:40:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce99fce740 10 client.17268 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce99fce740 1 -- 172.21.15.86:0/2338950694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be598980f0 con 0x55be59c60e50 2024-04-16T05:40:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce78ff9640 1 -- 172.21.15.86:0/2338950694 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8800e510 con 0x55be59c60e50 2024-04-16T05:40:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce78ff9640 1 -- 172.21.15.86:0/2338950694 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce8001ba80 con 0x55be59c60e50 2024-04-16T05:40:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce99fce740 10 client.17268 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce99fce740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce99fce740 20 client.17268 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce99fce740 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55be59c7a7c0 0x55be59c7cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce99fce740 4 client.17268 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce99fce740 1 -- 172.21.15.86:0/2338950694 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be59bd3ba0 con 0x55be59c7a7c0 2024-04-16T05:40:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.023+0000 7fce9497d640 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55be59c7a7c0 0x55be59c7cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.024+0000 7fce9497d640 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55be59c7a7c0 0x55be59c7cc10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3f0c14e36a03bc server_cookie=41fa9cd550b39ed8 in_seq=0 out_seq=0 2024-04-16T05:40:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.024+0000 7fce78ff9640 10 client.17268 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.024+0000 7fce78ff9640 1 -- 172.21.15.86:0/2338950694 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be59bd3ba0 con 0x55be59c7a7c0 2024-04-16T05:40:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.024+0000 7fce78ff9640 10 client.17268 handle_command_reply: tid=0 2024-04-16T05:40:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.024+0000 7fce99fce740 1 client.17268 shutdown 2024-04-16T05:40:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.024+0000 7fce99fce740 1 -- 172.21.15.86:0/2338950694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 msgr2=0x55be59c673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c673a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fce8000bab0 tx=0x7fce8000bf70 comp rx=0 tx=0).stop 2024-04-16T05:40:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 1 -- 172.21.15.86:0/2338950694 shutdown_connections 2024-04-16T05:40:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55be59c7a7c0 0x55be59c7cc10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c6a350 0x55be59c67e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c61770 0x55be59c678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 1 --2- 172.21.15.86:0/2338950694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c673a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce78ff9640 0 client.17268 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 1 -- 172.21.15.86:0/2338950694 >> 172.21.15.86:0/2338950694 conn(0x55be59c5b580 msgr2=0x55be59c5d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 1 -- 172.21.15.86:0/2338950694 shutdown_connections 2024-04-16T05:40:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 1 -- 172.21.15.86:0/2338950694 wait complete. 2024-04-16T05:40:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.025+0000 7fce99fce740 20 client.17268 trim_cache size 0 max 16384 2024-04-16T05:40:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce99fce740 1 Processor -- start 2024-04-16T05:40:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce99fce740 1 -- start start 2024-04-16T05:40:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce99fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c76020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce99fce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c67c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce99fce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c6a350 0x55be59c5d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce99fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce901af4d0 con 0x55be59c60e50 2024-04-16T05:40:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce99fce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b1010 con 0x55be59c61770 2024-04-16T05:40:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce99fce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce901afba0 con 0x55be59c6a350 2024-04-16T05:40:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce9517e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c76020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce9517e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c76020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52516/0 (socket says 172.21.15.86:52516) 2024-04-16T05:40:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce9517e640 1 -- 172.21.15.86:0/867769666 learned_addr learned my addr 172.21.15.86:0/867769666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce9517e640 1 -- 172.21.15.86:0/867769666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c6a350 msgr2=0x55be59c5d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce9517e640 1 --2- 172.21.15.86:0/867769666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c6a350 0x55be59c5d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce9517e640 1 -- 172.21.15.86:0/867769666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 msgr2=0x55be59c67c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce9517e640 1 --2- 172.21.15.86:0/867769666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c67c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce9517e640 1 -- 172.21.15.86:0/867769666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be59c62f80 con 0x55be59c60e50 2024-04-16T05:40:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce9517e640 1 --2- 172.21.15.86:0/867769666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c76020 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fce8001bef0 tx=0x7fce80005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62ee843ea6d5bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce7affd640 1 -- 172.21.15.86:0/867769666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80011980 con 0x55be59c60e50 2024-04-16T05:40:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce7affd640 1 -- 172.21.15.86:0/867769666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8002e070 con 0x55be59c60e50 2024-04-16T05:40:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.043+0000 7fce7affd640 1 -- 172.21.15.86:0/867769666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8001b420 con 0x55be59c60e50 2024-04-16T05:40:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 -- 172.21.15.86:0/867769666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 msgr2=0x55be59c76020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 --2- 172.21.15.86:0/867769666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c76020 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fce8001bef0 tx=0x7fce80005850 comp rx=0 tx=0).stop 2024-04-16T05:40:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 -- 172.21.15.86:0/867769666 shutdown_connections 2024-04-16T05:40:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 --2- 172.21.15.86:0/867769666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c6a350 0x55be59c5d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 --2- 172.21.15.86:0/867769666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c67c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 --2- 172.21.15.86:0/867769666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c76020 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 -- 172.21.15.86:0/867769666 >> 172.21.15.86:0/867769666 conn(0x55be59c5b580 msgr2=0x55be59c5c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 -- 172.21.15.86:0/867769666 shutdown_connections 2024-04-16T05:40:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 -- 172.21.15.86:0/867769666 wait complete. 2024-04-16T05:40:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 -- start start 2024-04-16T05:40:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.044+0000 7fce99fce740 1 Processor -- start 2024-04-16T05:40:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce99fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c72760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce99fce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c6faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce99fce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c6a350 0x55be59c70030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce99fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b0940 con 0x55be59c60e50 2024-04-16T05:40:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce99fce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b0270 con 0x55be59c61770 2024-04-16T05:40:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce99fce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce901ad490 con 0x55be59c6a350 2024-04-16T05:40:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce99fce740 10 client.0 _ll_register_callbacks cb 0x7fce80004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9517e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9517e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c72760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52520/0 (socket says 172.21.15.86:52520) 2024-04-16T05:40:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9517e640 1 -- 172.21.15.86:0/1149707855 learned_addr learned my addr 172.21.15.86:0/1149707855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9597f640 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c6a350 0x55be59c70030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9497d640 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c6faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9517e640 1 -- 172.21.15.86:0/1149707855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c6a350 msgr2=0x55be59c70030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9517e640 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c6a350 0x55be59c70030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9517e640 1 -- 172.21.15.86:0/1149707855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 msgr2=0x55be59c6faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9517e640 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c6faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9517e640 1 -- 172.21.15.86:0/1149707855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be59c70670 con 0x55be59c60e50 2024-04-16T05:40:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce9517e640 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c72760 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fce8002b040 tx=0x7fce80012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630f5e1e7f9c7e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.045+0000 7fce78ff9640 1 -- 172.21.15.86:0/1149707855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80037070 con 0x55be59c60e50 2024-04-16T05:40:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce78ff9640 1 -- 172.21.15.86:0/1149707855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fce8001b420 con 0x55be59c60e50 2024-04-16T05:40:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce99fce740 1 -- 172.21.15.86:0/1149707855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce8002e070 con 0x55be59c60e50 2024-04-16T05:40:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce78ff9640 1 -- 172.21.15.86:0/1149707855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce8000b390 con 0x55be59c60e50 2024-04-16T05:40:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce99fce740 10 client.17274 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce99fce740 1 -- 172.21.15.86:0/1149707855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be59ce4ff0 con 0x55be59c60e50 2024-04-16T05:40:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce78ff9640 1 -- 172.21.15.86:0/1149707855 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8002e070 con 0x55be59c60e50 2024-04-16T05:40:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce78ff9640 1 -- 172.21.15.86:0/1149707855 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce80010af0 con 0x55be59c60e50 2024-04-16T05:40:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce99fce740 10 client.17274 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce99fce740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce99fce740 20 client.17274 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce99fce740 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55be59ce5500 0x55be59ce7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce99fce740 4 client.17274 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:40:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce99fce740 1 -- 172.21.15.86:0/1149707855 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be59bcda10 con 0x55be59ce5500 2024-04-16T05:40:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.046+0000 7fce9497d640 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55be59ce5500 0x55be59ce7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce78ff9640 10 client.17274 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce9497d640 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55be59ce5500 0x55be59ce7920 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e67dad8770243e3b server_cookie=2baf1a06baef897c in_seq=0 out_seq=0 2024-04-16T05:40:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce78ff9640 1 -- 172.21.15.86:0/1149707855 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55be59bcda10 con 0x55be59ce5500 2024-04-16T05:40:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce78ff9640 10 client.17274 handle_command_reply: tid=0 2024-04-16T05:40:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 client.17274 shutdown 2024-04-16T05:40:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 -- 172.21.15.86:0/1149707855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 msgr2=0x55be59c72760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c72760 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fce8002b040 tx=0x7fce80012c40 comp rx=0 tx=0).stop 2024-04-16T05:40:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 -- 172.21.15.86:0/1149707855 shutdown_connections 2024-04-16T05:40:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55be59ce5500 0x55be59ce7920 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be59c6a350 0x55be59c70030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be59c61770 0x55be59c6faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce78ff9640 0 client.17274 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 --2- 172.21.15.86:0/1149707855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be59c60e50 0x55be59c72760 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 -- 172.21.15.86:0/1149707855 >> 172.21.15.86:0/1149707855 conn(0x55be59c5b580 msgr2=0x55be59c5c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 -- 172.21.15.86:0/1149707855 shutdown_connections 2024-04-16T05:40:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 1 -- 172.21.15.86:0/1149707855 wait complete. 2024-04-16T05:40:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.047+0000 7fce99fce740 20 client.17274 trim_cache size 0 max 16384 2024-04-16T05:40:49.064 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:40:49.065 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:40:49.065 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:40:49.065 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:40:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 -- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fce7c0a3d20 msgr2=0x7fce7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fce7c0a3d20 0x7fce7c0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce8800ec60 tx=0x7fce88009e30 comp rx=0 tx=0).stop 2024-04-16T05:40:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 -- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 msgr2=0x7fce9019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 0x7fce9019ebf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009a20 tx=0x7fce8c02df60 comp rx=0 tx=0).stop 2024-04-16T05:40:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 -- 172.21.15.86:0/1368415437 shutdown_connections 2024-04-16T05:40:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9010c5d0 0x7fce901987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce90103bb0 0x7fce9019ebf0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fce7c0a3d20 0x7fce7c0a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 --2- 172.21.15.86:0/1368415437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90103290 0x7fce9019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 -- 172.21.15.86:0/1368415437 >> 172.21.15.86:0/1368415437 conn(0x7fce90077280 msgr2=0x7fce900fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 -- 172.21.15.86:0/1368415437 shutdown_connections 2024-04-16T05:40:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.048+0000 7fce973e2640 1 -- 172.21.15.86:0/1368415437 wait complete. 2024-04-16T05:40:49.069 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:40:49.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:40:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 -- 172.21.15.86:0/1972151437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510102420 msgr2=0x7f151010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 --2- 172.21.15.86:0/1972151437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510102420 0x7f151010ddb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1508009cd0 tx=0x7f150802c180 comp rx=0 tx=0).stop 2024-04-16T05:40:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 -- 172.21.15.86:0/1972151437 shutdown_connections 2024-04-16T05:40:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 --2- 172.21.15.86:0/1972151437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510102420 0x7f151010ddb0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 --2- 172.21.15.86:0/1972151437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510101a60 0x7f1510101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 --2- 172.21.15.86:0/1972151437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1510105f40 0x7f1510106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 -- 172.21.15.86:0/1972151437 >> 172.21.15.86:0/1972151437 conn(0x7f15100761e0 msgr2=0x7f15100765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 -- 172.21.15.86:0/1972151437 shutdown_connections 2024-04-16T05:40:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 -- 172.21.15.86:0/1972151437 wait complete. 2024-04-16T05:40:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.175+0000 7f1516477640 1 Processor -- start 2024-04-16T05:40:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f1516477640 1 -- start start 2024-04-16T05:40:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f1516477640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510101a60 0x7f151019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f1516477640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510102420 0x7f151019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f1516477640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1510105f40 0x7f1510198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f1516477640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1510112400 con 0x7f1510101a60 2024-04-16T05:40:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f1516477640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1510112280 con 0x7f1510105f40 2024-04-16T05:40:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f1516477640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1510112580 con 0x7f1510102420 2024-04-16T05:40:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f150ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510101a60 0x7f151019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f150ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510101a60 0x7f151019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52530/0 (socket says 172.21.15.86:52530) 2024-04-16T05:40:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f150ffff640 1 -- 172.21.15.86:0/3516940291 learned_addr learned my addr 172.21.15.86:0/3516940291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f1514a14640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1510105f40 0x7f1510198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f150ffff640 1 -- 172.21.15.86:0/3516940291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510102420 msgr2=0x7f151019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f150ffff640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510102420 0x7f151019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f150ffff640 1 -- 172.21.15.86:0/3516940291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1510105f40 msgr2=0x7f1510198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f150ffff640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1510105f40 0x7f1510198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f150ffff640 1 -- 172.21.15.86:0/3516940291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1510198da0 con 0x7f1510101a60 2024-04-16T05:40:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.176+0000 7f150ffff640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510101a60 0x7f151019e470 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f150000c970 tx=0x7f150000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.177+0000 7f150d7fa640 1 -- 172.21.15.86:0/3516940291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500004220 con 0x7f1510101a60 2024-04-16T05:40:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.177+0000 7f150d7fa640 1 -- 172.21.15.86:0/3516940291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15000043c0 con 0x7f1510101a60 2024-04-16T05:40:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.177+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1510199090 con 0x7f1510101a60 2024-04-16T05:40:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.177+0000 7f150d7fa640 1 -- 172.21.15.86:0/3516940291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150000f450 con 0x7f1510101a60 2024-04-16T05:40:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.177+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15101a5330 con 0x7f1510101a60 2024-04-16T05:40:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.178+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151006a270 con 0x7f1510101a60 2024-04-16T05:40:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.179+0000 7f150d7fa640 1 -- 172.21.15.86:0/3516940291 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f15000079b0 con 0x7f1510101a60 2024-04-16T05:40:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.179+0000 7f150d7fa640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f14fc0a3d20 0x7f14fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.179+0000 7f150f7fe640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f14fc0a3d20 0x7f14fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.179+0000 7f150d7fa640 1 -- 172.21.15.86:0/3516940291 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f15000d09e0 con 0x7f1510101a60 2024-04-16T05:40:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.180+0000 7f150f7fe640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f14fc0a3d20 0x7f14fc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14f8005fd0 tx=0x7f14f8005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.184+0000 7f150d7fa640 1 -- 172.21.15.86:0/3516940291 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1500014030 con 0x7f1510101a60 2024-04-16T05:40:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.403+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1510069270 con 0x7f1510101a60 2024-04-16T05:40:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.404+0000 7f150d7fa640 1 -- 172.21.15.86:0/3516940291 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f150009cd10 con 0x7f1510101a60 2024-04-16T05:40:49.403 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:40:49.403 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:40:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:40:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f14fc0a3d20 msgr2=0x7f14fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f14fc0a3d20 0x7f14fc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14f8005fd0 tx=0x7f14f8005850 comp rx=0 tx=0).stop 2024-04-16T05:40:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510101a60 msgr2=0x7f151019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510101a60 0x7f151019e470 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f150000c970 tx=0x7f150000ce30 comp rx=0 tx=0).stop 2024-04-16T05:40:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 shutdown_connections 2024-04-16T05:40:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f14fc0a3d20 0x7f14fc0a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1510105f40 0x7f1510198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510102420 0x7f151019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 --2- 172.21.15.86:0/3516940291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1510101a60 0x7f151019e470 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 >> 172.21.15.86:0/3516940291 conn(0x7f15100761e0 msgr2=0x7f15100fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 shutdown_connections 2024-04-16T05:40:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.406+0000 7f1516477640 1 -- 172.21.15.86:0/3516940291 wait complete. 2024-04-16T05:40:49.416 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:40:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.523+0000 7ff686350640 1 -- 172.21.15.86:0/2317893662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101c10 msgr2=0x7ff68010dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.523+0000 7ff686350640 1 --2- 172.21.15.86:0/2317893662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101c10 0x7ff68010dce0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff678009cd0 tx=0x7ff67802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.524+0000 7ff686350640 1 -- 172.21.15.86:0/2317893662 shutdown_connections 2024-04-16T05:40:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.524+0000 7ff686350640 1 --2- 172.21.15.86:0/2317893662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101c10 0x7ff68010dce0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.524+0000 7ff686350640 1 --2- 172.21.15.86:0/2317893662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680101250 0x7ff6801016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.524+0000 7ff686350640 1 --2- 172.21.15.86:0/2317893662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff680105ee0 0x7ff6801062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.524+0000 7ff686350640 1 -- 172.21.15.86:0/2317893662 >> 172.21.15.86:0/2317893662 conn(0x7ff680076d00 msgr2=0x7ff680077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.524+0000 7ff686350640 1 -- 172.21.15.86:0/2317893662 shutdown_connections 2024-04-16T05:40:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.524+0000 7ff686350640 1 -- 172.21.15.86:0/2317893662 wait complete. 2024-04-16T05:40:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.524+0000 7ff686350640 1 Processor -- start 2024-04-16T05:40:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.524+0000 7ff686350640 1 -- start start 2024-04-16T05:40:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff686350640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101250 0x7ff68019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff686350640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680101c10 0x7ff68019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff686350640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff680105ee0 0x7ff6801987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff686350640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff680111330 con 0x7ff680101250 2024-04-16T05:40:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff686350640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff6801111b0 con 0x7ff680101c10 2024-04-16T05:40:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff686350640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6801114b0 con 0x7ff680105ee0 2024-04-16T05:40:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101250 0x7ff68019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101250 0x7ff68019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52542/0 (socket says 172.21.15.86:52542) 2024-04-16T05:40:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67ffff640 1 -- 172.21.15.86:0/3818493935 learned_addr learned my addr 172.21.15.86:0/3818493935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67f7fe640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680101c10 0x7ff68019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff6848ed640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff680105ee0 0x7ff6801987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67ffff640 1 -- 172.21.15.86:0/3818493935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff680105ee0 msgr2=0x7ff6801987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67ffff640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff680105ee0 0x7ff6801987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67ffff640 1 -- 172.21.15.86:0/3818493935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680101c10 msgr2=0x7ff68019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67ffff640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680101c10 0x7ff68019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67ffff640 1 -- 172.21.15.86:0/3818493935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff680198f90 con 0x7ff680101250 2024-04-16T05:40:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67f7fe640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680101c10 0x7ff68019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67ffff640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101250 0x7ff68019e650 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff67000c890 tx=0x7ff67000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67d7fa640 1 -- 172.21.15.86:0/3818493935 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670004220 con 0x7ff680101250 2024-04-16T05:40:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67d7fa640 1 -- 172.21.15.86:0/3818493935 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6700043c0 con 0x7ff680101250 2024-04-16T05:40:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff686350640 1 -- 172.21.15.86:0/3818493935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff680199280 con 0x7ff680101250 2024-04-16T05:40:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff67d7fa640 1 -- 172.21.15.86:0/3818493935 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670004a00 con 0x7ff680101250 2024-04-16T05:40:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.525+0000 7ff686350640 1 -- 172.21.15.86:0/3818493935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6801a54a0 con 0x7ff680101250 2024-04-16T05:40:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.527+0000 7ff67d7fa640 1 -- 172.21.15.86:0/3818493935 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff6700079b0 con 0x7ff680101250 2024-04-16T05:40:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff689012740 1 Processor -- start 2024-04-16T05:40:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff689012740 1 -- start start 2024-04-16T05:40:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff689012740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff689012740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a6780 0x5654b73aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff689012740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff689012740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654b730b0e0 con 0x5654b73a5e60 2024-04-16T05:40:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff689012740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5654b72a8520 con 0x5654b73af360 2024-04-16T05:40:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff689012740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654b73070a0 con 0x5654b73a6780 2024-04-16T05:40:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff6848ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff6848ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52556/0 (socket says 172.21.15.86:52556) 2024-04-16T05:40:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff6848ed640 1 -- 172.21.15.86:0/924623212 learned_addr learned my addr 172.21.15.86:0/924623212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff67d7fa640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6680a3d20 0x7ff6680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff67ffff640 1 --2- 172.21.15.86:0/924623212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a6780 0x5654b73aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.528+0000 7ff67f7fe640 1 --2- 172.21.15.86:0/924623212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6848ed640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6680a3d20 0x7ff6680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6848ed640 1 -- 172.21.15.86:0/924623212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a6780 msgr2=0x5654b73aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6848ed640 1 --2- 172.21.15.86:0/924623212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a6780 0x5654b73aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6848ed640 1 -- 172.21.15.86:0/924623212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 msgr2=0x5654b73b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6848ed640 1 --2- 172.21.15.86:0/924623212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6848ed640 1 -- 172.21.15.86:0/924623212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b72d08d0 con 0x5654b73a5e60 2024-04-16T05:40:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff67d7fa640 1 -- 172.21.15.86:0/3818493935 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff6700d0900 con 0x7ff680101250 2024-04-16T05:40:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6848ed640 1 --2- 172.21.15.86:0/924623212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73a6240 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff678037040 tx=0x7ff678032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbba831bf53b8cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6848ed640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6680a3d20 0x7ff6680a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff678045000 tx=0x7ff678048000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6667fc640 1 -- 172.21.15.86:0/924623212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678039070 con 0x5654b73a5e60 2024-04-16T05:40:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6667fc640 1 -- 172.21.15.86:0/924623212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff678004520 con 0x5654b73a5e60 2024-04-16T05:40:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.529+0000 7ff6667fc640 1 -- 172.21.15.86:0/924623212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678034980 con 0x5654b73a5e60 2024-04-16T05:40:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 -- 172.21.15.86:0/924623212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 msgr2=0x5654b73a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 --2- 172.21.15.86:0/924623212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73a6240 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff678037040 tx=0x7ff678032590 comp rx=0 tx=0).stop 2024-04-16T05:40:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 -- 172.21.15.86:0/924623212 shutdown_connections 2024-04-16T05:40:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 --2- 172.21.15.86:0/924623212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 --2- 172.21.15.86:0/924623212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a6780 0x5654b73aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 --2- 172.21.15.86:0/924623212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73a6240 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 -- 172.21.15.86:0/924623212 >> 172.21.15.86:0/924623212 conn(0x5654b73a0590 msgr2=0x5654b73a0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 -- 172.21.15.86:0/924623212 shutdown_connections 2024-04-16T05:40:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 -- 172.21.15.86:0/924623212 wait complete. 2024-04-16T05:40:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 -- start start 2024-04-16T05:40:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.530+0000 7ff689012740 1 Processor -- start 2024-04-16T05:40:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff689012740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 0x5654b73ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff689012740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73a6780 0x5654b73ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff689012740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73af360 0x5654b73acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff689012740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654b73b1fa0 con 0x5654b73af360 2024-04-16T05:40:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff689012740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5654b72c44d0 con 0x5654b73a6780 2024-04-16T05:40:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff689012740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654b73b2120 con 0x5654b73a5e60 2024-04-16T05:40:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff689012740 10 client.0 _ll_register_callbacks cb 0x5654b72a92c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff6848ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73af360 0x5654b73acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff6848ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73af360 0x5654b73acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52564/0 (socket says 172.21.15.86:52564) 2024-04-16T05:40:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff6848ed640 1 -- 172.21.15.86:0/931561240 learned_addr learned my addr 172.21.15.86:0/931561240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff67ffff640 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 0x5654b73ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff67f7fe640 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73a6780 0x5654b73ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff6848ed640 1 -- 172.21.15.86:0/931561240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 msgr2=0x5654b73ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff6848ed640 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 0x5654b73ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff6848ed640 1 -- 172.21.15.86:0/931561240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73a6780 msgr2=0x5654b73ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff6848ed640 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73a6780 0x5654b73ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff6848ed640 1 -- 172.21.15.86:0/931561240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b701d3f0 con 0x5654b73af360 2024-04-16T05:40:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff67f7fe640 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73a6780 0x5654b73ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff6848ed640 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73af360 0x5654b73acf00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff678033000 tx=0x7ff6780344e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db14ee1b09e8161 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff65f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff65f7fe640 1 -- 172.21.15.86:0/931561240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678051030 con 0x5654b73af360 2024-04-16T05:40:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff65f7fe640 1 -- 172.21.15.86:0/931561240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff6780346a0 con 0x5654b73af360 2024-04-16T05:40:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.531+0000 7ff689012740 1 -- 172.21.15.86:0/931561240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff678004520 con 0x5654b73af360 2024-04-16T05:40:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff65f7fe640 1 -- 172.21.15.86:0/931561240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff678039040 con 0x5654b73af360 2024-04-16T05:40:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff689012740 10 client.17292 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff689012740 1 -- 172.21.15.86:0/931561240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654b70e77d0 con 0x5654b73af360 2024-04-16T05:40:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff65f7fe640 1 -- 172.21.15.86:0/931561240 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff678004520 con 0x5654b73af360 2024-04-16T05:40:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff65f7fe640 1 -- 172.21.15.86:0/931561240 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff678043840 con 0x5654b73af360 2024-04-16T05:40:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff689012740 10 client.17292 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff689012740 10 client.17292 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:40:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff689012740 20 client.17292 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff689012740 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5654b73bf260 0x5654b73c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff689012740 4 client.17292 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff689012740 1 -- 172.21.15.86:0/931561240 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654b7318b00 con 0x5654b73bf260 2024-04-16T05:40:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.532+0000 7ff67ffff640 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5654b73bf260 0x5654b73c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.533+0000 7ff67ffff640 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5654b73bf260 0x5654b73c1650 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6df884adf5bcdbc server_cookie=c85957182167d2ef in_seq=0 out_seq=0 2024-04-16T05:40:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.533+0000 7ff65f7fe640 10 client.17292 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:40:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.533+0000 7ff65f7fe640 1 -- 172.21.15.86:0/931561240 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654b7318b00 con 0x5654b73bf260 2024-04-16T05:40:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff65f7fe640 10 client.17292 handle_command_reply: tid=0 2024-04-16T05:40:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 client.17292 shutdown 2024-04-16T05:40:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 -- 172.21.15.86:0/931561240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73af360 msgr2=0x5654b73acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73af360 0x5654b73acf00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff678033000 tx=0x7ff6780344e0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 -- 172.21.15.86:0/931561240 shutdown_connections 2024-04-16T05:40:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5654b73bf260 0x5654b73c1650 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73af360 0x5654b73acf00 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73a6780 0x5654b73ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 --2- 172.21.15.86:0/931561240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 0x5654b73ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff65f7fe640 0 client.17292 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:40:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 -- 172.21.15.86:0/931561240 >> 172.21.15.86:0/931561240 conn(0x5654b73a0590 msgr2=0x5654b73a22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 -- 172.21.15.86:0/931561240 shutdown_connections 2024-04-16T05:40:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 1 -- 172.21.15.86:0/931561240 wait complete. 2024-04-16T05:40:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.534+0000 7ff689012740 20 client.17292 trim_cache size 0 max 16384 2024-04-16T05:40:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff689012740 1 Processor -- start 2024-04-16T05:40:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff689012740 1 -- start start 2024-04-16T05:40:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff689012740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff689012740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a6780 0x5654b73c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff689012740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff689012740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6801af4e0 con 0x5654b73a5e60 2024-04-16T05:40:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff689012740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff6801b1020 con 0x5654b73af360 2024-04-16T05:40:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff689012740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6801afbb0 con 0x5654b73a6780 2024-04-16T05:40:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52576/0 (socket says 172.21.15.86:52576) 2024-04-16T05:40:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.547+0000 7ff67ffff640 1 -- 172.21.15.86:0/2349683796 learned_addr learned my addr 172.21.15.86:0/2349683796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff67ffff640 1 -- 172.21.15.86:0/2349683796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a6780 msgr2=0x5654b73c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff67ffff640 1 --2- 172.21.15.86:0/2349683796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a6780 0x5654b73c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff67ffff640 1 -- 172.21.15.86:0/2349683796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 msgr2=0x5654b73c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff67ffff640 1 --2- 172.21.15.86:0/2349683796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff67ffff640 1 -- 172.21.15.86:0/2349683796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b739be60 con 0x5654b73a5e60 2024-04-16T05:40:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff67ffff640 1 --2- 172.21.15.86:0/2349683796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73acd20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff6700058c0 tx=0x7ff67006f020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d5c9e0381d5317 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff665ffb640 1 -- 172.21.15.86:0/2349683796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670081e60 con 0x5654b73a5e60 2024-04-16T05:40:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff665ffb640 1 -- 172.21.15.86:0/2349683796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff67009c530 con 0x5654b73a5e60 2024-04-16T05:40:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff665ffb640 1 -- 172.21.15.86:0/2349683796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67001d040 con 0x5654b73a5e60 2024-04-16T05:40:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff689012740 1 -- 172.21.15.86:0/2349683796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 msgr2=0x5654b73acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff689012740 1 --2- 172.21.15.86:0/2349683796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73acd20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff6700058c0 tx=0x7ff67006f020 comp rx=0 tx=0).stop 2024-04-16T05:40:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff689012740 1 -- 172.21.15.86:0/2349683796 shutdown_connections 2024-04-16T05:40:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff689012740 1 --2- 172.21.15.86:0/2349683796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff689012740 1 --2- 172.21.15.86:0/2349683796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a6780 0x5654b73c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff689012740 1 --2- 172.21.15.86:0/2349683796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a5e60 0x5654b73acd20 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff689012740 1 -- 172.21.15.86:0/2349683796 >> 172.21.15.86:0/2349683796 conn(0x5654b73a0590 msgr2=0x5654b73ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff689012740 1 -- 172.21.15.86:0/2349683796 shutdown_connections 2024-04-16T05:40:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.548+0000 7ff689012740 1 -- 172.21.15.86:0/2349683796 wait complete. 2024-04-16T05:40:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff689012740 1 -- start start 2024-04-16T05:40:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff689012740 1 Processor -- start 2024-04-16T05:40:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff689012740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 0x5654b73b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff689012740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a6780 0x5654b73a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff689012740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff689012740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6801b0950 con 0x5654b73a6780 2024-04-16T05:40:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff689012740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff6801b0280 con 0x5654b73af360 2024-04-16T05:40:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff689012740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6801ad4a0 con 0x5654b73a5e60 2024-04-16T05:40:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff689012740 10 client.0 _ll_register_callbacks cb 0x7ff678009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff67f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a6780 0x5654b73a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff67f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a6780 0x5654b73a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52580/0 (socket says 172.21.15.86:52580) 2024-04-16T05:40:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.549+0000 7ff67f7fe640 1 -- 172.21.15.86:0/2852540479 learned_addr learned my addr 172.21.15.86:0/2852540479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff67ffff640 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 0x5654b73b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff6848ed640 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff67f7fe640 1 -- 172.21.15.86:0/2852540479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 msgr2=0x5654b73b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff67f7fe640 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 0x5654b73b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff67f7fe640 1 -- 172.21.15.86:0/2852540479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 msgr2=0x5654b73a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff67f7fe640 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73a92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff67f7fe640 1 -- 172.21.15.86:0/2852540479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b73a9a10 con 0x5654b73a6780 2024-04-16T05:40:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff67f7fe640 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a6780 0x5654b73a8d90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x5654b73a1720 tx=0x7ff674011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241b36c25ce3c269 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff65f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff65f7fe640 1 -- 172.21.15.86:0/2852540479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674019070 con 0x5654b73a6780 2024-04-16T05:40:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff65f7fe640 1 -- 172.21.15.86:0/2852540479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff674015440 con 0x5654b73a6780 2024-04-16T05:40:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff689012740 1 -- 172.21.15.86:0/2852540479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff67009c530 con 0x5654b73a6780 2024-04-16T05:40:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff65f7fe640 1 -- 172.21.15.86:0/2852540479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff674025050 con 0x5654b73a6780 2024-04-16T05:40:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff689012740 10 client.17298 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff689012740 1 -- 172.21.15.86:0/2852540479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654b7425540 con 0x5654b73a6780 2024-04-16T05:40:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff65f7fe640 1 -- 172.21.15.86:0/2852540479 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff67009c530 con 0x5654b73a6780 2024-04-16T05:40:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff65f7fe640 1 -- 172.21.15.86:0/2852540479 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff674014850 con 0x5654b73a6780 2024-04-16T05:40:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff689012740 10 client.17298 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff689012740 10 client.17298 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:40:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff689012740 20 client.17298 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff689012740 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5654b7425cf0 0x5654b7428110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff689012740 4 client.17298 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.550+0000 7ff689012740 1 -- 172.21.15.86:0/2852540479 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654b7312970 con 0x5654b7425cf0 2024-04-16T05:40:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff67ffff640 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5654b7425cf0 0x5654b7428110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff67ffff640 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5654b7425cf0 0x5654b7428110 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10c8eb49aa7af29d server_cookie=575fd32f4ac5ba74 in_seq=0 out_seq=0 2024-04-16T05:40:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff65f7fe640 10 client.17298 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:40:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff65f7fe640 1 -- 172.21.15.86:0/2852540479 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654b7312970 con 0x5654b7425cf0 2024-04-16T05:40:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff65f7fe640 10 client.17298 handle_command_reply: tid=0 2024-04-16T05:40:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff689012740 1 client.17298 shutdown 2024-04-16T05:40:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff689012740 1 -- 172.21.15.86:0/2852540479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a6780 msgr2=0x5654b73a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff689012740 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a6780 0x5654b73a8d90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x5654b73a1720 tx=0x7ff674011630 comp rx=0 tx=0).stop 2024-04-16T05:40:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff689012740 1 -- 172.21.15.86:0/2852540479 shutdown_connections 2024-04-16T05:40:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff689012740 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5654b7425cf0 0x5654b7428110 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff689012740 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654b73af360 0x5654b73a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff689012740 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b73a6780 0x5654b73a8d90 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff689012740 1 --2- 172.21.15.86:0/2852540479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654b73a5e60 0x5654b73b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff65f7fe640 0 client.17298 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:40:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.551+0000 7ff689012740 1 -- 172.21.15.86:0/2852540479 >> 172.21.15.86:0/2852540479 conn(0x5654b73a0590 msgr2=0x5654b739eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff689012740 1 -- 172.21.15.86:0/2852540479 shutdown_connections 2024-04-16T05:40:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff689012740 1 -- 172.21.15.86:0/2852540479 wait complete. 2024-04-16T05:40:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff689012740 20 client.17298 trim_cache size 0 max 16384 2024-04-16T05:40:49.563 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 -- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6680a3d20 msgr2=0x7ff6680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6680a3d20 0x7ff6680a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff678045000 tx=0x7ff678048000 comp rx=0 tx=0).stop 2024-04-16T05:40:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 -- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101250 msgr2=0x7ff68019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101250 0x7ff68019e650 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff67000c890 tx=0x7ff67000cd50 comp rx=0 tx=0).stop 2024-04-16T05:40:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 -- 172.21.15.86:0/3818493935 shutdown_connections 2024-04-16T05:40:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff680105ee0 0x7ff6801987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff6680a3d20 0x7ff6680a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680101c10 0x7ff68019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 --2- 172.21.15.86:0/3818493935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff680101250 0x7ff68019e650 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 -- 172.21.15.86:0/3818493935 >> 172.21.15.86:0/3818493935 conn(0x7ff680076d00 msgr2=0x7ff68006a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 -- 172.21.15.86:0/3818493935 shutdown_connections 2024-04-16T05:40:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.552+0000 7ff686350640 1 -- 172.21.15.86:0/3818493935 wait complete. 2024-04-16T05:40:49.565 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:40:49.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.673+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/389136531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 msgr2=0x7f0dd010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.673+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/389136531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 0x7f0dd010ddd0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009ca0 tx=0x7f0dc802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/389136531 shutdown_connections 2024-04-16T05:40:49.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/389136531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 0x7f0dd010ddd0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/389136531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0dd0101ae0 0x7f0dd0105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/389136531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd0101130 0x7f0dd0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/389136531 >> 172.21.15.86:0/389136531 conn(0x7f0dd00780b0 msgr2=0x7f0dd00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/389136531 shutdown_connections 2024-04-16T05:40:49.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/389136531 wait complete. 2024-04-16T05:40:49.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 Processor -- start 2024-04-16T05:40:49.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 -- start start 2024-04-16T05:40:49.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0dd0101130 0x7f0dd019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 0x7f0dd019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd01986d0 0x7f0dd0198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0111330 con 0x7f0dd0106200 2024-04-16T05:40:49.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01111b0 con 0x7f0dd0101130 2024-04-16T05:40:49.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.674+0000 7f0dd6fbd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01114b0 con 0x7f0dd01986d0 2024-04-16T05:40:49.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 0x7f0dd019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 0x7f0dd019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52600/0 (socket says 172.21.15.86:52600) 2024-04-16T05:40:49.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcffff640 1 -- 172.21.15.86:0/656352670 learned_addr learned my addr 172.21.15.86:0/656352670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dd555a640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd01986d0 0x7f0dd0198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0dd0101130 0x7f0dd019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcffff640 1 -- 172.21.15.86:0/656352670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd01986d0 msgr2=0x7f0dd0198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcffff640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd01986d0 0x7f0dd0198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcffff640 1 -- 172.21.15.86:0/656352670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0dd0101130 msgr2=0x7f0dd019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcffff640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0dd0101130 0x7f0dd019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcffff640 1 -- 172.21.15.86:0/656352670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd01992d0 con 0x7f0dd0106200 2024-04-16T05:40:49.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dd555a640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd01986d0 0x7f0dd0198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:49.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0dd0101130 0x7f0dd019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:49.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcffff640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 0x7f0dd019eb50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400b450 tx=0x7f0dc400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcdffb640 1 -- 172.21.15.86:0/656352670 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4015020 con 0x7f0dd0106200 2024-04-16T05:40:49.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcdffb640 1 -- 172.21.15.86:0/656352670 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc4011d70 con 0x7f0dd0106200 2024-04-16T05:40:49.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/656352670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd01a53b0 con 0x7f0dd0106200 2024-04-16T05:40:49.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dcdffb640 1 -- 172.21.15.86:0/656352670 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400f340 con 0x7f0dd0106200 2024-04-16T05:40:49.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.675+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/656352670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd01a5810 con 0x7f0dd0106200 2024-04-16T05:40:49.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.677+0000 7f0dcdffb640 1 -- 172.21.15.86:0/656352670 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0dc4018020 con 0x7f0dd0106200 2024-04-16T05:40:49.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd9bae740 1 Processor -- start 2024-04-16T05:40:49.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd9bae740 1 -- start start 2024-04-16T05:40:49.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dcdffb640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0db80a3da0 0x7f0db80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde13240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde1be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde1c360 0x5643cde1e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643cdd780e0 con 0x5643cde12e60 2024-04-16T05:40:49.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5643cdd15520 con 0x5643cde13780 2024-04-16T05:40:49.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643cdd740a0 con 0x5643cde1c360 2024-04-16T05:40:49.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd555a640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0db80a3da0 0x7f0db80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd4d59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde13240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dcdffb640 1 -- 172.21.15.86:0/656352670 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0dc40d0540 con 0x7f0dd0106200 2024-04-16T05:40:49.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd4d59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde13240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52602/0 (socket says 172.21.15.86:52602) 2024-04-16T05:40:49.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.678+0000 7f0dd4d59640 1 -- 172.21.15.86:0/3650460290 learned_addr learned my addr 172.21.15.86:0/3650460290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0dcffff640 1 --2- 172.21.15.86:0/3650460290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde1be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0dd4d59640 1 -- 172.21.15.86:0/3650460290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde1c360 msgr2=0x5643cde1e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/3650460290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde1c360 0x5643cde1e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0dd4d59640 1 -- 172.21.15.86:0/3650460290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 msgr2=0x5643cde1be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/3650460290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde1be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0dd4d59640 1 -- 172.21.15.86:0/3650460290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cdd3d8d0 con 0x5643cde12e60 2024-04-16T05:40:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0dcffff640 1 --2- 172.21.15.86:0/3650460290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde1be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/3650460290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde13240 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009c30 tx=0x7f0dc0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b753f6307542c4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0dd555a640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0db80a3da0 0x7f0db80a6260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8032530 tx=0x7f0dc8032340 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0db6ffd640 1 -- 172.21.15.86:0/3650460290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0004020 con 0x5643cde12e60 2024-04-16T05:40:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0db6ffd640 1 -- 172.21.15.86:0/3650460290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc00041c0 con 0x5643cde12e60 2024-04-16T05:40:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.679+0000 7f0db6ffd640 1 -- 172.21.15.86:0/3650460290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00126b0 con 0x5643cde12e60 2024-04-16T05:40:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3650460290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 msgr2=0x5643cde13240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3650460290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde13240 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009c30 tx=0x7f0dc0010740 comp rx=0 tx=0).stop 2024-04-16T05:40:49.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3650460290 shutdown_connections 2024-04-16T05:40:49.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3650460290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde1c360 0x5643cde1e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3650460290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde1be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3650460290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde13240 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3650460290 >> 172.21.15.86:0/3650460290 conn(0x5643cde0d590 msgr2=0x5643cde0d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3650460290 shutdown_connections 2024-04-16T05:40:49.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3650460290 wait complete. 2024-04-16T05:40:49.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 -- start start 2024-04-16T05:40:49.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 Processor -- start 2024-04-16T05:40:49.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde12e60 0x5643cde193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde19910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde19e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643cde1f060 con 0x5643cde1c360 2024-04-16T05:40:49.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5643cdd314d0 con 0x5643cde13780 2024-04-16T05:40:49.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.680+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643cde1f1e0 con 0x5643cde12e60 2024-04-16T05:40:49.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd9bae740 10 client.0 _ll_register_callbacks cb 0x5643cdd162c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:49.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd555a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde19e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd555a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde19e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52610/0 (socket says 172.21.15.86:52610) 2024-04-16T05:40:49.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd555a640 1 -- 172.21.15.86:0/1065021494 learned_addr learned my addr 172.21.15.86:0/1065021494 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde12e60 0x5643cde193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dcffff640 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde19910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd555a640 1 -- 172.21.15.86:0/1065021494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde12e60 msgr2=0x5643cde193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd555a640 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde12e60 0x5643cde193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd555a640 1 -- 172.21.15.86:0/1065021494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 msgr2=0x5643cde19910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd555a640 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde19910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd555a640 1 -- 172.21.15.86:0/1065021494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cda8a3f0 con 0x5643cde1c360 2024-04-16T05:40:49.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dcffff640 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde19910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:49.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd555a640 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde19e50 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009ca0 tx=0x7f0dc80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99420b6d98f2bee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0daffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:49.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0daffff640 1 -- 172.21.15.86:0/1065021494 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8051060 con 0x5643cde1c360 2024-04-16T05:40:49.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd9bae740 1 -- 172.21.15.86:0/1065021494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc00041c0 con 0x5643cde1c360 2024-04-16T05:40:49.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0daffff640 1 -- 172.21.15.86:0/1065021494 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0dc8004040 con 0x5643cde1c360 2024-04-16T05:40:49.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0daffff640 1 -- 172.21.15.86:0/1065021494 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0dc8058050 con 0x5643cde1c360 2024-04-16T05:40:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd9bae740 10 client.17310 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.681+0000 7f0dd9bae740 1 -- 172.21.15.86:0/1065021494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643cdb547d0 con 0x5643cde1c360 2024-04-16T05:40:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.682+0000 7f0daffff640 1 -- 172.21.15.86:0/1065021494 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc00041c0 con 0x5643cde1c360 2024-04-16T05:40:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.682+0000 7f0daffff640 1 -- 172.21.15.86:0/1065021494 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0dc8040c40 con 0x5643cde1c360 2024-04-16T05:40:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.682+0000 7f0dd9bae740 10 client.17310 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.682+0000 7f0dd9bae740 10 client.17310 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:40:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.682+0000 7f0dd9bae740 20 client.17310 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.682+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5643cde2c2b0 0x5643cde2e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.682+0000 7f0dd9bae740 4 client.17310 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.682+0000 7f0dd9bae740 1 -- 172.21.15.86:0/1065021494 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643cdd85b00 con 0x5643cde2c2b0 2024-04-16T05:40:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.682+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5643cde2c2b0 0x5643cde2e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.683+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5643cde2c2b0 0x5643cde2e700 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64a03d25959d8ccd server_cookie=bb9fb1b8741d967a in_seq=0 out_seq=0 2024-04-16T05:40:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.683+0000 7f0daffff640 10 client.17310 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:40:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.683+0000 7f0daffff640 1 -- 172.21.15.86:0/1065021494 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643cdd85b00 con 0x5643cde2c2b0 2024-04-16T05:40:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.683+0000 7f0daffff640 10 client.17310 handle_command_reply: tid=0 2024-04-16T05:40:49.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.683+0000 7f0dd9bae740 1 client.17310 shutdown 2024-04-16T05:40:49.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.683+0000 7f0dd9bae740 1 -- 172.21.15.86:0/1065021494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 msgr2=0x5643cde19e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.683+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde19e50 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009ca0 tx=0x7f0dc80095c0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0dd9bae740 1 -- 172.21.15.86:0/1065021494 shutdown_connections 2024-04-16T05:40:49.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde19e50 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde19910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5643cde2c2b0 0x5643cde2e700 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/1065021494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde12e60 0x5643cde193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0dd9bae740 1 -- 172.21.15.86:0/1065021494 >> 172.21.15.86:0/1065021494 conn(0x5643cde0d590 msgr2=0x5643cde0f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0daffff640 0 client.17310 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:40:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0dd9bae740 1 -- 172.21.15.86:0/1065021494 shutdown_connections 2024-04-16T05:40:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0dd9bae740 1 -- 172.21.15.86:0/1065021494 wait complete. 2024-04-16T05:40:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.684+0000 7f0dd9bae740 20 client.17310 trim_cache size 0 max 16384 2024-04-16T05:40:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.697+0000 7f0dd9bae740 1 Processor -- start 2024-04-16T05:40:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.697+0000 7f0dd9bae740 1 -- start start 2024-04-16T05:40:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.697+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde12e60 0x5643cde1bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.697+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde28030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.697+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde0f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.697+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01af4a0 con 0x5643cde1c360 2024-04-16T05:40:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.697+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01b0fe0 con 0x5643cde13780 2024-04-16T05:40:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.697+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01afb70 con 0x5643cde12e60 2024-04-16T05:40:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0dd555a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde0f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0dd555a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde0f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52622/0 (socket says 172.21.15.86:52622) 2024-04-16T05:40:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0dd555a640 1 -- 172.21.15.86:0/2830507468 learned_addr learned my addr 172.21.15.86:0/2830507468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0dd555a640 1 -- 172.21.15.86:0/2830507468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde12e60 msgr2=0x5643cde1bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0dd555a640 1 --2- 172.21.15.86:0/2830507468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde12e60 0x5643cde1bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0dd555a640 1 -- 172.21.15.86:0/2830507468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 msgr2=0x5643cde28030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0dd555a640 1 --2- 172.21.15.86:0/2830507468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde28030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0dd555a640 1 -- 172.21.15.86:0/2830507468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cde17920 con 0x5643cde1c360 2024-04-16T05:40:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0dd555a640 1 --2- 172.21.15.86:0/2830507468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde0f0d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009a60 tx=0x7f0dc8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8bd1d1c2cae3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0db67fc640 1 -- 172.21.15.86:0/2830507468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8051040 con 0x5643cde1c360 2024-04-16T05:40:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0db67fc640 1 -- 172.21.15.86:0/2830507468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc8042760 con 0x5643cde1c360 2024-04-16T05:40:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.698+0000 7f0db67fc640 1 -- 172.21.15.86:0/2830507468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8039030 con 0x5643cde1c360 2024-04-16T05:40:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 -- 172.21.15.86:0/2830507468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 msgr2=0x5643cde0f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/2830507468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde0f0d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009a60 tx=0x7f0dc8009b20 comp rx=0 tx=0).stop 2024-04-16T05:40:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 -- 172.21.15.86:0/2830507468 shutdown_connections 2024-04-16T05:40:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/2830507468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde1c360 0x5643cde0f0d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/2830507468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde13780 0x5643cde28030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/2830507468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde12e60 0x5643cde1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 -- 172.21.15.86:0/2830507468 >> 172.21.15.86:0/2830507468 conn(0x5643cde0d590 msgr2=0x5643cde0e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 -- 172.21.15.86:0/2830507468 shutdown_connections 2024-04-16T05:40:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 -- 172.21.15.86:0/2830507468 wait complete. 2024-04-16T05:40:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 -- start start 2024-04-16T05:40:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.699+0000 7f0dd9bae740 1 Processor -- start 2024-04-16T05:40:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde8f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde13780 0x5643cde8f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde1c360 0x5643cde337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01b0910 con 0x5643cde12e60 2024-04-16T05:40:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01b0240 con 0x5643cde1c360 2024-04-16T05:40:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01ad460 con 0x5643cde13780 2024-04-16T05:40:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 10 client.0 _ll_register_callbacks cb 0x7f0dc8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd4d59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde8f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd4d59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde8f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52632/0 (socket says 172.21.15.86:52632) 2024-04-16T05:40:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd4d59640 1 -- 172.21.15.86:0/3817226239 learned_addr learned my addr 172.21.15.86:0/3817226239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dcffff640 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde13780 0x5643cde8f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd4d59640 1 -- 172.21.15.86:0/3817226239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde13780 msgr2=0x5643cde8f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde13780 0x5643cde8f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd4d59640 1 -- 172.21.15.86:0/3817226239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde1c360 msgr2=0x5643cde337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde1c360 0x5643cde337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd4d59640 1 -- 172.21.15.86:0/3817226239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cde33ee0 con 0x5643cde12e60 2024-04-16T05:40:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dcffff640 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde13780 0x5643cde8f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd4d59640 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde8f160 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009820 tx=0x7f0dc00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23acd487c0f2aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0daffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0daffff640 1 -- 172.21.15.86:0/3817226239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc002d030 con 0x5643cde12e60 2024-04-16T05:40:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0daffff640 1 -- 172.21.15.86:0/3817226239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0dc0003dd0 con 0x5643cde12e60 2024-04-16T05:40:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3817226239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc8042760 con 0x5643cde12e60 2024-04-16T05:40:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0daffff640 1 -- 172.21.15.86:0/3817226239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0dc0037050 con 0x5643cde12e60 2024-04-16T05:40:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 10 client.17316 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.700+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3817226239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643cde96960 con 0x5643cde12e60 2024-04-16T05:40:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0daffff640 1 -- 172.21.15.86:0/3817226239 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc8042760 con 0x5643cde12e60 2024-04-16T05:40:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0daffff640 1 -- 172.21.15.86:0/3817226239 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0dc000f040 con 0x5643cde12e60 2024-04-16T05:40:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0dd9bae740 10 client.17316 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0dd9bae740 10 client.17316 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:40:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0dd9bae740 20 client.17316 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5643cde96e00 0x5643cde99220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0dd9bae740 4 client.17316 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3817226239 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643cdd7f970 con 0x5643cde96e00 2024-04-16T05:40:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0dcffff640 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5643cde96e00 0x5643cde99220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0dcffff640 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5643cde96e00 0x5643cde99220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a15a6759e71e31d server_cookie=214ccfc2ee853b2d in_seq=0 out_seq=0 2024-04-16T05:40:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.701+0000 7f0daffff640 10 client.17316 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:40:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0daffff640 1 -- 172.21.15.86:0/3817226239 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643cdd7f970 con 0x5643cde96e00 2024-04-16T05:40:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0daffff640 10 client.17316 handle_command_reply: tid=0 2024-04-16T05:40:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 client.17316 shutdown 2024-04-16T05:40:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3817226239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 msgr2=0x5643cde8f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde8f160 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009820 tx=0x7f0dc00095c0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3817226239 shutdown_connections 2024-04-16T05:40:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643cde1c360 0x5643cde337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5643cde96e00 0x5643cde99220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cde13780 0x5643cde8f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 --2- 172.21.15.86:0/3817226239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643cde12e60 0x5643cde8f160 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0daffff640 0 client.17316 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:40:49.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3817226239 >> 172.21.15.86:0/3817226239 conn(0x5643cde0d590 msgr2=0x5643cde0f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3817226239 shutdown_connections 2024-04-16T05:40:49.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 1 -- 172.21.15.86:0/3817226239 wait complete. 2024-04-16T05:40:49.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd9bae740 20 client.17316 trim_cache size 0 max 16384 2024-04-16T05:40:49.712 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:49.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0db80a3da0 msgr2=0x7f0db80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0db80a3da0 0x7f0db80a6260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8032530 tx=0x7f0dc8032340 comp rx=0 tx=0).stop 2024-04-16T05:40:49.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 msgr2=0x7f0dd019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.702+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 0x7f0dd019eb50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400b450 tx=0x7f0dc400b910 comp rx=0 tx=0).stop 2024-04-16T05:40:49.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.703+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/656352670 shutdown_connections 2024-04-16T05:40:49.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.703+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd01986d0 0x7f0dd0198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.703+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd0106200 0x7f0dd019eb50 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.703+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0db80a3da0 0x7f0db80a6260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.703+0000 7f0dd6fbd640 1 --2- 172.21.15.86:0/656352670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0dd0101130 0x7f0dd019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.703+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/656352670 >> 172.21.15.86:0/656352670 conn(0x7f0dd00780b0 msgr2=0x7f0dd0104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.703+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/656352670 shutdown_connections 2024-04-16T05:40:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.703+0000 7f0dd6fbd640 1 -- 172.21.15.86:0/656352670 wait complete. 2024-04-16T05:40:49.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:40:49.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 -- 172.21.15.86:0/2065898386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023010c5b0 msgr2=0x7f023010ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 --2- 172.21.15.86:0/2065898386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023010c5b0 0x7f023010ea50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0228009ca0 tx=0x7f022802c230 comp rx=0 tx=0).stop 2024-04-16T05:40:49.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 -- 172.21.15.86:0/2065898386 shutdown_connections 2024-04-16T05:40:49.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 --2- 172.21.15.86:0/2065898386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023010c5b0 0x7f023010ea50 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 --2- 172.21.15.86:0/2065898386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230103b90 0x7f023010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 --2- 172.21.15.86:0/2065898386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0230103270 0x7f0230103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 -- 172.21.15.86:0/2065898386 >> 172.21.15.86:0/2065898386 conn(0x7f02300780b0 msgr2=0x7f02300784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 -- 172.21.15.86:0/2065898386 shutdown_connections 2024-04-16T05:40:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 -- 172.21.15.86:0/2065898386 wait complete. 2024-04-16T05:40:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.823+0000 7f023859d640 1 Processor -- start 2024-04-16T05:40:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f023859d640 1 -- start start 2024-04-16T05:40:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f023859d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230103270 0x7f023019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f023859d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230103b90 0x7f023019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f023859d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f023010c5b0 0x7f02301985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f023859d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0230112370 con 0x7f0230103270 2024-04-16T05:40:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f023859d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f02301121f0 con 0x7f023010c5b0 2024-04-16T05:40:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f023859d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02301124f0 con 0x7f0230103b90 2024-04-16T05:40:49.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230103270 0x7f023019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230103270 0x7f023019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52650/0 (socket says 172.21.15.86:52650) 2024-04-16T05:40:49.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236339640 1 -- 172.21.15.86:0/1193586105 learned_addr learned my addr 172.21.15.86:0/1193586105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0235b38640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230103b90 0x7f023019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236b3a640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f023010c5b0 0x7f02301985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236339640 1 -- 172.21.15.86:0/1193586105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230103b90 msgr2=0x7f023019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236339640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230103b90 0x7f023019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236339640 1 -- 172.21.15.86:0/1193586105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f023010c5b0 msgr2=0x7f02301985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236339640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f023010c5b0 0x7f02301985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236339640 1 -- 172.21.15.86:0/1193586105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0230198d50 con 0x7f0230103270 2024-04-16T05:40:49.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f0236339640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230103270 0x7f023019e420 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f022c00d950 tx=0x7f022c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f02277fe640 1 -- 172.21.15.86:0/1193586105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c017070 con 0x7f0230103270 2024-04-16T05:40:49.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f02277fe640 1 -- 172.21.15.86:0/1193586105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f022c009c30 con 0x7f0230103270 2024-04-16T05:40:49.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f023859d640 1 -- 172.21.15.86:0/1193586105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0230199040 con 0x7f0230103270 2024-04-16T05:40:49.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f02277fe640 1 -- 172.21.15.86:0/1193586105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c00f6b0 con 0x7f0230103270 2024-04-16T05:40:49.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.824+0000 7f023859d640 1 -- 172.21.15.86:0/1193586105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02301a52e0 con 0x7f0230103270 2024-04-16T05:40:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.827+0000 7f02277fe640 1 -- 172.21.15.86:0/1193586105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f022c0109d0 con 0x7f0230103270 2024-04-16T05:40:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f023b186740 1 Processor -- start 2024-04-16T05:40:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f023b186740 1 -- start start 2024-04-16T05:40:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f023b186740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 0x55e21d79ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f023b186740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79f4b0 0x55e21d7ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f023b186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f023b186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e21d7040e0 con 0x55e21d7ac770 2024-04-16T05:40:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f023b186740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e21d6a1520 con 0x55e21d79f4b0 2024-04-16T05:40:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f023b186740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e21d7000a0 con 0x55e21d79eb90 2024-04-16T05:40:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f0235b38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f0235b38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52656/0 (socket says 172.21.15.86:52656) 2024-04-16T05:40:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f0235b38640 1 -- 172.21.15.86:0/413525475 learned_addr learned my addr 172.21.15.86:0/413525475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f02277fe640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f021c0a3d20 0x7f021c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f0236339640 1 --2- 172.21.15.86:0/413525475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79f4b0 0x55e21d7ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f0236b3a640 1 --2- 172.21.15.86:0/413525475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 0x55e21d79ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f0236b3a640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f021c0a3d20 0x7f021c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f0235b38640 1 -- 172.21.15.86:0/413525475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 msgr2=0x55e21d79ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f0235b38640 1 --2- 172.21.15.86:0/413525475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 0x55e21d79ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.828+0000 7f0235b38640 1 -- 172.21.15.86:0/413525475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79f4b0 msgr2=0x55e21d7ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.829+0000 7f0235b38640 1 --2- 172.21.15.86:0/413525475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79f4b0 0x55e21d7ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.829+0000 7f0235b38640 1 -- 172.21.15.86:0/413525475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21d6c98d0 con 0x55e21d7ac770 2024-04-16T05:40:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.829+0000 7f02277fe640 1 -- 172.21.15.86:0/1193586105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f022c0d0850 con 0x7f0230103270 2024-04-16T05:40:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.829+0000 7f0235b38640 1 --2- 172.21.15.86:0/413525475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7aeb60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f022000cb90 tx=0x7f022000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e92335e668dfee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.829+0000 7f0236b3a640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f021c0a3d20 0x7f021c0a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0228009a60 tx=0x7f02280095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.829+0000 7f0224bfe640 1 -- 172.21.15.86:0/413525475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220005220 con 0x55e21d7ac770 2024-04-16T05:40:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.829+0000 7f0224bfe640 1 -- 172.21.15.86:0/413525475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0220004c80 con 0x55e21d7ac770 2024-04-16T05:40:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.829+0000 7f0224bfe640 1 -- 172.21.15.86:0/413525475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200107b0 con 0x55e21d7ac770 2024-04-16T05:40:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 -- 172.21.15.86:0/413525475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 msgr2=0x55e21d7aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 --2- 172.21.15.86:0/413525475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7aeb60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f022000cb90 tx=0x7f022000e720 comp rx=0 tx=0).stop 2024-04-16T05:40:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 -- 172.21.15.86:0/413525475 shutdown_connections 2024-04-16T05:40:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 --2- 172.21.15.86:0/413525475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7aeb60 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 --2- 172.21.15.86:0/413525475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79f4b0 0x55e21d7ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 --2- 172.21.15.86:0/413525475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 0x55e21d79ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 -- 172.21.15.86:0/413525475 >> 172.21.15.86:0/413525475 conn(0x55e21d799590 msgr2=0x55e21d799970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 -- 172.21.15.86:0/413525475 shutdown_connections 2024-04-16T05:40:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 -- 172.21.15.86:0/413525475 wait complete. 2024-04-16T05:40:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 -- start start 2024-04-16T05:40:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 Processor -- start 2024-04-16T05:40:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 0x55e21d7a9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d7ac770 0x55e21d7aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e21d7af530 con 0x55e21d79f4b0 2024-04-16T05:40:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e21d6bd4d0 con 0x55e21d7ac770 2024-04-16T05:40:49.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e21d7af6b0 con 0x55e21d79eb90 2024-04-16T05:40:49.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.830+0000 7f023b186740 10 client.0 _ll_register_callbacks cb 0x55e21d6a22c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:49.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0235b38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0235b38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52658/0 (socket says 172.21.15.86:52658) 2024-04-16T05:40:49.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0235b38640 1 -- 172.21.15.86:0/540025280 learned_addr learned my addr 172.21.15.86:0/540025280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0236339640 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 0x55e21d7a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0236b3a640 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d7ac770 0x55e21d7aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0235b38640 1 -- 172.21.15.86:0/540025280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 msgr2=0x55e21d7a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0235b38640 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 0x55e21d7a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0235b38640 1 -- 172.21.15.86:0/540025280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d7ac770 msgr2=0x55e21d7aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0235b38640 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d7ac770 0x55e21d7aa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0235b38640 1 -- 172.21.15.86:0/540025280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21d4163f0 con 0x55e21d79f4b0 2024-04-16T05:40:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0235b38640 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a9c70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f02200054e0 tx=0x7f022000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df580322993019c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0219ffb640 1 -- 172.21.15.86:0/540025280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022000ce90 con 0x55e21d79f4b0 2024-04-16T05:40:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0219ffb640 1 -- 172.21.15.86:0/540025280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f02200048d0 con 0x55e21d79f4b0 2024-04-16T05:40:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f023b186740 1 -- 172.21.15.86:0/540025280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0220004c80 con 0x55e21d79f4b0 2024-04-16T05:40:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0219ffb640 1 -- 172.21.15.86:0/540025280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0220004ad0 con 0x55e21d79f4b0 2024-04-16T05:40:49.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f023b186740 10 client.17328 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:49.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f023b186740 1 -- 172.21.15.86:0/540025280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e21d4e07d0 con 0x55e21d79f4b0 2024-04-16T05:40:49.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.831+0000 7f0219ffb640 1 -- 172.21.15.86:0/540025280 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0220004c80 con 0x55e21d79f4b0 2024-04-16T05:40:49.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f0219ffb640 1 -- 172.21.15.86:0/540025280 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0220010510 con 0x55e21d79f4b0 2024-04-16T05:40:49.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f023b186740 10 client.17328 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:49.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f023b186740 10 client.17328 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:40:49.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f023b186740 20 client.17328 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:49.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f023b186740 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e21d7bc710 0x55e21d7beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f023b186740 4 client.17328 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:49.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f023b186740 1 -- 172.21.15.86:0/540025280 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e21d711b00 con 0x55e21d7bc710 2024-04-16T05:40:49.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f0236339640 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e21d7bc710 0x55e21d7beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f0219ffb640 10 client.17328 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:40:49.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.832+0000 7f0236339640 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e21d7bc710 0x55e21d7beb00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4a00d641d20d15f server_cookie=55ae40286f34a3b1 in_seq=0 out_seq=0 2024-04-16T05:40:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f0219ffb640 1 -- 172.21.15.86:0/540025280 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e21d711b00 con 0x55e21d7bc710 2024-04-16T05:40:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f0219ffb640 10 client.17328 handle_command_reply: tid=0 2024-04-16T05:40:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 client.17328 shutdown 2024-04-16T05:40:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 -- 172.21.15.86:0/540025280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 msgr2=0x55e21d7a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a9c70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f02200054e0 tx=0x7f022000c8d0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 -- 172.21.15.86:0/540025280 shutdown_connections 2024-04-16T05:40:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d7ac770 0x55e21d7aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e21d7bc710 0x55e21d7beb00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a9c70 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f0219ffb640 0 client.17328 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:40:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 --2- 172.21.15.86:0/540025280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79eb90 0x55e21d7a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 -- 172.21.15.86:0/540025280 >> 172.21.15.86:0/540025280 conn(0x55e21d799590 msgr2=0x55e21d79b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 -- 172.21.15.86:0/540025280 shutdown_connections 2024-04-16T05:40:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 1 -- 172.21.15.86:0/540025280 wait complete. 2024-04-16T05:40:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.833+0000 7f023b186740 20 client.17328 trim_cache size 0 max 16384 2024-04-16T05:40:49.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.846+0000 7f023b186740 1 Processor -- start 2024-04-16T05:40:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.846+0000 7f023b186740 1 -- start start 2024-04-16T05:40:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f023b186740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f023b186740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79f4b0 0x55e21d7c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f023b186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f023b186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ad380 con 0x55e21d7ac770 2024-04-16T05:40:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f023b186740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f02301aeec0 con 0x55e21d79eb90 2024-04-16T05:40:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f023b186740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ada50 con 0x55e21d79f4b0 2024-04-16T05:40:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236b3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236b3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52670/0 (socket says 172.21.15.86:52670) 2024-04-16T05:40:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236b3a640 1 -- 172.21.15.86:0/2382393724 learned_addr learned my addr 172.21.15.86:0/2382393724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236339640 1 --2- 172.21.15.86:0/2382393724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236b3a640 1 -- 172.21.15.86:0/2382393724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79f4b0 msgr2=0x55e21d7c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236b3a640 1 --2- 172.21.15.86:0/2382393724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79f4b0 0x55e21d7c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236b3a640 1 -- 172.21.15.86:0/2382393724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 msgr2=0x55e21d7b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236b3a640 1 --2- 172.21.15.86:0/2382393724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236b3a640 1 -- 172.21.15.86:0/2382393724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21d7bf130 con 0x55e21d7ac770 2024-04-16T05:40:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236339640 1 --2- 172.21.15.86:0/2382393724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f0236b3a640 1 --2- 172.21.15.86:0/2382393724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7c48a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0228009ca0 tx=0x7f022803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4c4a9ff63eca30 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f021bfff640 1 -- 172.21.15.86:0/2382393724 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228039040 con 0x55e21d7ac770 2024-04-16T05:40:49.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f021bfff640 1 -- 172.21.15.86:0/2382393724 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0228053070 con 0x55e21d7ac770 2024-04-16T05:40:49.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.847+0000 7f021bfff640 1 -- 172.21.15.86:0/2382393724 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228034730 con 0x55e21d7ac770 2024-04-16T05:40:49.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 -- 172.21.15.86:0/2382393724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 msgr2=0x55e21d7c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 --2- 172.21.15.86:0/2382393724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7c48a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0228009ca0 tx=0x7f022803f000 comp rx=0 tx=0).stop 2024-04-16T05:40:49.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 -- 172.21.15.86:0/2382393724 shutdown_connections 2024-04-16T05:40:49.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 --2- 172.21.15.86:0/2382393724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d7ac770 0x55e21d7c48a0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 --2- 172.21.15.86:0/2382393724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d79f4b0 0x55e21d7c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 --2- 172.21.15.86:0/2382393724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 -- 172.21.15.86:0/2382393724 >> 172.21.15.86:0/2382393724 conn(0x55e21d799590 msgr2=0x55e21d799970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 -- 172.21.15.86:0/2382393724 shutdown_connections 2024-04-16T05:40:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 -- 172.21.15.86:0/2382393724 wait complete. 2024-04-16T05:40:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 -- start start 2024-04-16T05:40:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.848+0000 7f023b186740 1 Processor -- start 2024-04-16T05:40:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d7ac770 0x55e21d7a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ae7f0 con 0x55e21d79f4b0 2024-04-16T05:40:49.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ae120 con 0x55e21d79eb90 2024-04-16T05:40:49.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ab340 con 0x55e21d7ac770 2024-04-16T05:40:49.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 10 client.0 _ll_register_callbacks cb 0x7f0220014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:49.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0235b38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0235b38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52674/0 (socket says 172.21.15.86:52674) 2024-04-16T05:40:49.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0235b38640 1 -- 172.21.15.86:0/654614509 learned_addr learned my addr 172.21.15.86:0/654614509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0236339640 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0236b3a640 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d7ac770 0x55e21d7a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0235b38640 1 -- 172.21.15.86:0/654614509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d7ac770 msgr2=0x55e21d7a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0235b38640 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d7ac770 0x55e21d7a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0235b38640 1 -- 172.21.15.86:0/654614509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 msgr2=0x55e21d7b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0235b38640 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0235b38640 1 -- 172.21.15.86:0/654614509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21d7a5f00 con 0x55e21d79f4b0 2024-04-16T05:40:49.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0236339640 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:49.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0235b38640 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a6eb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0220010b20 tx=0x7f02200058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15006f8b4b5f066 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:49.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0219ffb640 1 -- 172.21.15.86:0/654614509 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220010640 con 0x55e21d79f4b0 2024-04-16T05:40:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 1 -- 172.21.15.86:0/654614509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0228053070 con 0x55e21d79f4b0 2024-04-16T05:40:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0219ffb640 1 -- 172.21.15.86:0/654614509 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f02200048d0 con 0x55e21d79f4b0 2024-04-16T05:40:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f0219ffb640 1 -- 172.21.15.86:0/654614509 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0220004ad0 con 0x55e21d79f4b0 2024-04-16T05:40:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 10 client.17334 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.849+0000 7f023b186740 1 -- 172.21.15.86:0/654614509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e21d822160 con 0x55e21d79f4b0 2024-04-16T05:40:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.850+0000 7f0219ffb640 1 -- 172.21.15.86:0/654614509 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0228053070 con 0x55e21d79f4b0 2024-04-16T05:40:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.850+0000 7f0219ffb640 1 -- 172.21.15.86:0/654614509 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0220031020 con 0x55e21d79f4b0 2024-04-16T05:40:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.850+0000 7f023b186740 10 client.17334 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.850+0000 7f023b186740 10 client.17334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:40:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.850+0000 7f023b186740 20 client.17334 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.850+0000 7f023b186740 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e21d822820 0x55e21d824c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.850+0000 7f023b186740 4 client.17334 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.850+0000 7f023b186740 1 -- 172.21.15.86:0/654614509 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e21d70b970 con 0x55e21d822820 2024-04-16T05:40:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.850+0000 7f0236339640 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e21d822820 0x55e21d824c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f0219ffb640 10 client.17334 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:40:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f0236339640 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e21d822820 0x55e21d824c40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=828077eef684497e server_cookie=922350b1d082cae6 in_seq=0 out_seq=0 2024-04-16T05:40:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f0219ffb640 1 -- 172.21.15.86:0/654614509 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e21d70b970 con 0x55e21d822820 2024-04-16T05:40:49.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f0219ffb640 10 client.17334 handle_command_reply: tid=0 2024-04-16T05:40:49.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 client.17334 shutdown 2024-04-16T05:40:49.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 -- 172.21.15.86:0/654614509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 msgr2=0x55e21d7a6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a6eb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0220010b20 tx=0x7f02200058c0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 -- 172.21.15.86:0/654614509 shutdown_connections 2024-04-16T05:40:49.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e21d7ac770 0x55e21d7a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e21d822820 0x55e21d824c40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e21d79f4b0 0x55e21d7a6eb0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 --2- 172.21.15.86:0/654614509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e21d79eb90 0x55e21d7b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f0219ffb640 0 client.17334 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:40:49.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 -- 172.21.15.86:0/654614509 >> 172.21.15.86:0/654614509 conn(0x55e21d799590 msgr2=0x55e21d798730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 -- 172.21.15.86:0/654614509 shutdown_connections 2024-04-16T05:40:49.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 1 -- 172.21.15.86:0/654614509 wait complete. 2024-04-16T05:40:49.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023b186740 20 client.17334 trim_cache size 0 max 16384 2024-04-16T05:40:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:49.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023859d640 1 -- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f021c0a3d20 msgr2=0x7f021c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.851+0000 7f023859d640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f021c0a3d20 0x7f021c0a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0228009a60 tx=0x7f02280095c0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 -- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230103270 msgr2=0x7f023019e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230103270 0x7f023019e420 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f022c00d950 tx=0x7f022c00de10 comp rx=0 tx=0).stop 2024-04-16T05:40:49.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 -- 172.21.15.86:0/1193586105 shutdown_connections 2024-04-16T05:40:49.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f021c0a3d20 0x7f021c0a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f023010c5b0 0x7f02301985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230103b90 0x7f023019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 --2- 172.21.15.86:0/1193586105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230103270 0x7f023019e420 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 -- 172.21.15.86:0/1193586105 >> 172.21.15.86:0/1193586105 conn(0x7f02300780b0 msgr2=0x7f02300fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 -- 172.21.15.86:0/1193586105 shutdown_connections 2024-04-16T05:40:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.852+0000 7f023859d640 1 -- 172.21.15.86:0/1193586105 wait complete. 2024-04-16T05:40:49.863 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:40:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 -- 172.21.15.86:0/3272193121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc10c5b0 msgr2=0x7f38fc10ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 --2- 172.21.15.86:0/3272193121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc10c5b0 0x7f38fc10ea50 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009ca0 tx=0x7f38f402c230 comp rx=0 tx=0).stop 2024-04-16T05:40:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 -- 172.21.15.86:0/3272193121 shutdown_connections 2024-04-16T05:40:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 --2- 172.21.15.86:0/3272193121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc10c5b0 0x7f38fc10ea50 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 --2- 172.21.15.86:0/3272193121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38fc103b90 0x7f38fc10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 --2- 172.21.15.86:0/3272193121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38fc103270 0x7f38fc103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 -- 172.21.15.86:0/3272193121 >> 172.21.15.86:0/3272193121 conn(0x7f38fc0780b0 msgr2=0x7f38fc0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 -- 172.21.15.86:0/3272193121 shutdown_connections 2024-04-16T05:40:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 -- 172.21.15.86:0/3272193121 wait complete. 2024-04-16T05:40:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 Processor -- start 2024-04-16T05:40:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.972+0000 7f39026d3640 1 -- start start 2024-04-16T05:40:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f39026d3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38fc103270 0x7f38fc100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f39026d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc103b90 0x7f38fc101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f39026d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38fc10c5b0 0x7f38fc101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f39026d3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc112370 con 0x7f38fc103b90 2024-04-16T05:40:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f39026d3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1121f0 con 0x7f38fc103270 2024-04-16T05:40:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f39026d3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1124f0 con 0x7f38fc10c5b0 2024-04-16T05:40:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc103b90 0x7f38fc101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc103b90 0x7f38fc101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52682/0 (socket says 172.21.15.86:52682) 2024-04-16T05:40:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fb7fe640 1 -- 172.21.15.86:0/4078290140 learned_addr learned my addr 172.21.15.86:0/4078290140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fbfff640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38fc103270 0x7f38fc100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f3900c70640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38fc10c5b0 0x7f38fc101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fb7fe640 1 -- 172.21.15.86:0/4078290140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38fc10c5b0 msgr2=0x7f38fc101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38fc10c5b0 0x7f38fc101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fb7fe640 1 -- 172.21.15.86:0/4078290140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38fc103270 msgr2=0x7f38fc100ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38fc103270 0x7f38fc100ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fb7fe640 1 -- 172.21.15.86:0/4078290140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc1a6d10 con 0x7f38fc103b90 2024-04-16T05:40:49.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fbfff640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38fc103270 0x7f38fc100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:49.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc103b90 0x7f38fc101400 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f38f000d950 tx=0x7f38f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38f97fa640 1 -- 172.21.15.86:0/4078290140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0017070 con 0x7f38fc103b90 2024-04-16T05:40:49.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38f97fa640 1 -- 172.21.15.86:0/4078290140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f0009c30 con 0x7f38fc103b90 2024-04-16T05:40:49.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f38f97fa640 1 -- 172.21.15.86:0/4078290140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f000f6b0 con 0x7f38fc103b90 2024-04-16T05:40:49.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f39026d3640 1 -- 172.21.15.86:0/4078290140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc1a6fa0 con 0x7f38fc103b90 2024-04-16T05:40:49.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.973+0000 7f39026d3640 1 -- 172.21.15.86:0/4078290140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc1a74c0 con 0x7f38fc103b90 2024-04-16T05:40:49.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.976+0000 7f3905397740 1 Processor -- start 2024-04-16T05:40:49.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.976+0000 7f38f97fa640 1 -- 172.21.15.86:0/4078290140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f38f00109d0 con 0x7f38fc103b90 2024-04-16T05:40:49.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.976+0000 7f3905397740 1 -- start start 2024-04-16T05:40:49.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f3905397740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7669240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f3905397740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7669780 0x55c6c7671e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f3905397740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7672360 0x55c6c7674750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f3905397740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6c75ce0e0 con 0x55c6c7672360 2024-04-16T05:40:49.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f3905397740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c6c756b520 con 0x55c6c7668e60 2024-04-16T05:40:49.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f3905397740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6c75ca0a0 con 0x55c6c7669780 2024-04-16T05:40:49.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7672360 0x55c6c7674750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7672360 0x55c6c7674750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52688/0 (socket says 172.21.15.86:52688) 2024-04-16T05:40:49.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fb7fe640 1 -- 172.21.15.86:0/3422070157 learned_addr learned my addr 172.21.15.86:0/3422070157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f3900c70640 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7669240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fbfff640 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7669780 0x55c6c7671e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fb7fe640 1 -- 172.21.15.86:0/3422070157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7669780 msgr2=0x55c6c7671e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7669780 0x55c6c7671e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fb7fe640 1 -- 172.21.15.86:0/3422070157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 msgr2=0x55c6c7669240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7669240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fb7fe640 1 -- 172.21.15.86:0/3422070157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c75938d0 con 0x55c6c7672360 2024-04-16T05:40:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f3900c70640 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7669240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38f97fa640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38e40a3d20 0x7f38e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f3900c70640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38e40a3d20 0x7f38e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7672360 0x55c6c7674750 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f38f00cdad0 tx=0x7f38f00d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3961b8a5996016 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38f97fa640 1 -- 172.21.15.86:0/4078290140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f38f00d07b0 con 0x7f38fc103b90 2024-04-16T05:40:49.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38e27fc640 1 -- 172.21.15.86:0/3422070157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0020070 con 0x55c6c7672360 2024-04-16T05:40:49.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38e27fc640 1 -- 172.21.15.86:0/3422070157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f009cae0 con 0x55c6c7672360 2024-04-16T05:40:49.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.977+0000 7f38e27fc640 1 -- 172.21.15.86:0/3422070157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f009c1b0 con 0x55c6c7672360 2024-04-16T05:40:49.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 -- 172.21.15.86:0/3422070157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7672360 msgr2=0x55c6c7674750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7672360 0x55c6c7674750 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f38f00cdad0 tx=0x7f38f00d8670 comp rx=0 tx=0).stop 2024-04-16T05:40:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3900c70640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38e40a3d20 0x7f38e40a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f38f4006010 tx=0x7f38f402e040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 -- 172.21.15.86:0/3422070157 shutdown_connections 2024-04-16T05:40:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7672360 0x55c6c7674750 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7669780 0x55c6c7671e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 --2- 172.21.15.86:0/3422070157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7669240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 -- 172.21.15.86:0/3422070157 >> 172.21.15.86:0/3422070157 conn(0x55c6c7663590 msgr2=0x55c6c7663970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 -- 172.21.15.86:0/3422070157 shutdown_connections 2024-04-16T05:40:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 -- 172.21.15.86:0/3422070157 wait complete. 2024-04-16T05:40:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.978+0000 7f3905397740 1 -- start start 2024-04-16T05:40:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3905397740 1 Processor -- start 2024-04-16T05:40:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3905397740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c766f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3905397740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 0x55c6c766f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3905397740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c766fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3905397740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6c7674fa0 con 0x55c6c7668e60 2024-04-16T05:40:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3905397740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c6c75874d0 con 0x55c6c7669780 2024-04-16T05:40:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3905397740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6c7675120 con 0x55c6c7672360 2024-04-16T05:40:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3905397740 10 client.0 _ll_register_callbacks cb 0x55c6c756c2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c766f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c766f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52694/0 (socket says 172.21.15.86:52694) 2024-04-16T05:40:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f38fbfff640 1 -- 172.21.15.86:0/1045242738 learned_addr learned my addr 172.21.15.86:0/1045242738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3900c70640 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c766fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 0x55c6c766f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f38fbfff640 1 -- 172.21.15.86:0/1045242738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 msgr2=0x55c6c766fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f38fbfff640 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c766fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f38fbfff640 1 -- 172.21.15.86:0/1045242738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 msgr2=0x55c6c766f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f38fbfff640 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 0x55c6c766f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f38fbfff640 1 -- 172.21.15.86:0/1045242738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c72e03f0 con 0x55c6c7668e60 2024-04-16T05:40:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.979+0000 7f3900c70640 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c766fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f38fbfff640 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c766f390 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00f410 tx=0x7f38ec00f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a00e0a7176db2d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f38db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f38db7fe640 1 -- 172.21.15.86:0/1045242738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec005220 con 0x55c6c7668e60 2024-04-16T05:40:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f38db7fe640 1 -- 172.21.15.86:0/1045242738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f38ec0053c0 con 0x55c6c7668e60 2024-04-16T05:40:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f3905397740 1 -- 172.21.15.86:0/1045242738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f009cae0 con 0x55c6c7668e60 2024-04-16T05:40:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f38db7fe640 1 -- 172.21.15.86:0/1045242738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38ec0055c0 con 0x55c6c7668e60 2024-04-16T05:40:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f3905397740 10 client.17346 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f3905397740 1 -- 172.21.15.86:0/1045242738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6c73aa7d0 con 0x55c6c7668e60 2024-04-16T05:40:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f38db7fe640 1 -- 172.21.15.86:0/1045242738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f009cae0 con 0x55c6c7668e60 2024-04-16T05:40:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f38db7fe640 1 -- 172.21.15.86:0/1045242738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38ec019430 con 0x55c6c7668e60 2024-04-16T05:40:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f3905397740 10 client.17346 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f3905397740 10 client.17346 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:40:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f3905397740 20 client.17346 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f3905397740 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55c6c7682360 0x55c6c7684820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f3905397740 4 client.17346 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.980+0000 7f3905397740 1 -- 172.21.15.86:0/1045242738 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6c75dbb00 con 0x55c6c7682360 2024-04-16T05:40:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.981+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55c6c7682360 0x55c6c7684820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.981+0000 7f38db7fe640 10 client.17346 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:40:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.981+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55c6c7682360 0x55c6c7684820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58c0d0fb52ae7cd6 server_cookie=671b39bbbc6b54b4 in_seq=0 out_seq=0 2024-04-16T05:40:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f38db7fe640 1 -- 172.21.15.86:0/1045242738 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6c75dbb00 con 0x55c6c7682360 2024-04-16T05:40:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f38db7fe640 10 client.17346 handle_command_reply: tid=0 2024-04-16T05:40:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 client.17346 shutdown 2024-04-16T05:40:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 -- 172.21.15.86:0/1045242738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 msgr2=0x55c6c766f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c766f390 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00f410 tx=0x7f38ec00f8d0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 -- 172.21.15.86:0/1045242738 shutdown_connections 2024-04-16T05:40:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55c6c7682360 0x55c6c7684820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c766fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 0x55c6c766f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 --2- 172.21.15.86:0/1045242738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c766f390 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:49.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f38db7fe640 0 client.17346 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:40:49.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 -- 172.21.15.86:0/1045242738 >> 172.21.15.86:0/1045242738 conn(0x55c6c7663590 msgr2=0x55c6c76652e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:49.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 -- 172.21.15.86:0/1045242738 shutdown_connections 2024-04-16T05:40:49.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 1 -- 172.21.15.86:0/1045242738 wait complete. 2024-04-16T05:40:49.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.982+0000 7f3905397740 20 client.17346 trim_cache size 0 max 16384 2024-04-16T05:40:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.995+0000 7f3905397740 1 Processor -- start 2024-04-16T05:40:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.995+0000 7f3905397740 1 -- start start 2024-04-16T05:40:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f3905397740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7671ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f3905397740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7669780 0x55c6c767e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f3905397740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c76625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f3905397740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1b15e0 con 0x55c6c7669780 2024-04-16T05:40:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f3905397740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1b3120 con 0x55c6c7668e60 2024-04-16T05:40:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f3905397740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1b1cb0 con 0x55c6c7672360 2024-04-16T05:40:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7669780 0x55c6c767e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7669780 0x55c6c767e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52710/0 (socket says 172.21.15.86:52710) 2024-04-16T05:40:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fb7fe640 1 -- 172.21.15.86:0/2024428328 learned_addr learned my addr 172.21.15.86:0/2024428328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f3900c70640 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c76625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fbfff640 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7671ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fb7fe640 1 -- 172.21.15.86:0/2024428328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 msgr2=0x55c6c76625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c76625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fb7fe640 1 -- 172.21.15.86:0/2024428328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 msgr2=0x55c6c7671ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7671ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fb7fe640 1 -- 172.21.15.86:0/2024428328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c7684ea0 con 0x55c6c7669780 2024-04-16T05:40:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fbfff640 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7671ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7669780 0x55c6c767e000 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f38f00058c0 tx=0x7f38f0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfb1378d18e6095 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38e1ffb640 1 -- 172.21.15.86:0/2024428328 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0020040 con 0x55c6c7669780 2024-04-16T05:40:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38e1ffb640 1 -- 172.21.15.86:0/2024428328 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f0009de0 con 0x55c6c7669780 2024-04-16T05:40:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.996+0000 7f38e1ffb640 1 -- 172.21.15.86:0/2024428328 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f009bd40 con 0x55c6c7669780 2024-04-16T05:40:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 -- 172.21.15.86:0/2024428328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7669780 msgr2=0x55c6c767e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7669780 0x55c6c767e000 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f38f00058c0 tx=0x7f38f0010d70 comp rx=0 tx=0).stop 2024-04-16T05:40:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 -- 172.21.15.86:0/2024428328 shutdown_connections 2024-04-16T05:40:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c76625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7669780 0x55c6c767e000 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 --2- 172.21.15.86:0/2024428328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7668e60 0x55c6c7671ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 -- 172.21.15.86:0/2024428328 >> 172.21.15.86:0/2024428328 conn(0x55c6c7663590 msgr2=0x55c6c7661ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 -- 172.21.15.86:0/2024428328 shutdown_connections 2024-04-16T05:40:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 -- 172.21.15.86:0/2024428328 wait complete. 2024-04-16T05:40:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 -- start start 2024-04-16T05:40:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.997+0000 7f3905397740 1 Processor -- start 2024-04-16T05:40:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c767aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 0x55c6c7675d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c7676380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1b2a50 con 0x55c6c7668e60 2024-04-16T05:40:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1b2380 con 0x55c6c7669780 2024-04-16T05:40:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1af5a0 con 0x55c6c7672360 2024-04-16T05:40:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 10 client.0 _ll_register_callbacks cb 0x7f38ec00a3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c767aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c767aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52724/0 (socket says 172.21.15.86:52724) 2024-04-16T05:40:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fbfff640 1 -- 172.21.15.86:0/3136867797 learned_addr learned my addr 172.21.15.86:0/3136867797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 0x55c6c7675d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3900c70640 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c7676380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fbfff640 1 -- 172.21.15.86:0/3136867797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 msgr2=0x55c6c7676380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fbfff640 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c7676380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fbfff640 1 -- 172.21.15.86:0/3136867797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 msgr2=0x55c6c7675d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fbfff640 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 0x55c6c7675d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fbfff640 1 -- 172.21.15.86:0/3136867797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c7676bb0 con 0x55c6c7668e60 2024-04-16T05:40:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 0x55c6c7675d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38fbfff640 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c767aa00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f38ec019740 tx=0x7f38ec019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5fb080defecef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38db7fe640 1 -- 172.21.15.86:0/3136867797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec011710 con 0x55c6c7668e60 2024-04-16T05:40:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 1 -- 172.21.15.86:0/3136867797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f0009de0 con 0x55c6c7668e60 2024-04-16T05:40:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38db7fe640 1 -- 172.21.15.86:0/3136867797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f38ec0118b0 con 0x55c6c7668e60 2024-04-16T05:40:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f38db7fe640 1 -- 172.21.15.86:0/3136867797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38ec038050 con 0x55c6c7668e60 2024-04-16T05:40:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 10 client.17352 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.998+0000 7f3905397740 1 -- 172.21.15.86:0/3136867797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6c76e8a00 con 0x55c6c7668e60 2024-04-16T05:40:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f38db7fe640 1 -- 172.21.15.86:0/3136867797 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f0009de0 con 0x55c6c7668e60 2024-04-16T05:40:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f38db7fe640 1 -- 172.21.15.86:0/3136867797 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38ec003c60 con 0x55c6c7668e60 2024-04-16T05:40:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f3905397740 10 client.17352 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f3905397740 10 client.17352 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:40:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f3905397740 20 client.17352 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f3905397740 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55c6c76e9180 0x55c6c76eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f3905397740 4 client.17352 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f3905397740 1 -- 172.21.15.86:0/3136867797 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6c75d5970 con 0x55c6c76e9180 2024-04-16T05:40:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55c6c76e9180 0x55c6c76eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f38db7fe640 10 client.17352 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:40:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:49.999+0000 7f38fb7fe640 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55c6c76e9180 0x55c6c76eb5a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6bfe80de73bfce0 server_cookie=c599f97ee931353e in_seq=0 out_seq=0 2024-04-16T05:40:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f38db7fe640 1 -- 172.21.15.86:0/3136867797 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6c75d5970 con 0x55c6c76e9180 2024-04-16T05:40:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f38db7fe640 10 client.17352 handle_command_reply: tid=0 2024-04-16T05:40:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 client.17352 shutdown 2024-04-16T05:40:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 -- 172.21.15.86:0/3136867797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 msgr2=0x55c6c767aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c767aa00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f38ec019740 tx=0x7f38ec019770 comp rx=0 tx=0).stop 2024-04-16T05:40:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 -- 172.21.15.86:0/3136867797 shutdown_connections 2024-04-16T05:40:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55c6c76e9180 0x55c6c76eb5a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c7672360 0x55c6c7676380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6c7669780 0x55c6c7675d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f38db7fe640 0 client.17352 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:40:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 --2- 172.21.15.86:0/3136867797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6c7668e60 0x55c6c767aa00 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 -- 172.21.15.86:0/3136867797 >> 172.21.15.86:0/3136867797 conn(0x55c6c7663590 msgr2=0x55c6c766fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 -- 172.21.15.86:0/3136867797 shutdown_connections 2024-04-16T05:40:50.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 1 -- 172.21.15.86:0/3136867797 wait complete. 2024-04-16T05:40:50.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f3905397740 20 client.17352 trim_cache size 0 max 16384 2024-04-16T05:40:50.013 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:50.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f39026d3640 1 -- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38e40a3d20 msgr2=0x7f38e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f39026d3640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38e40a3d20 0x7f38e40a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f38f4006010 tx=0x7f38f402e040 comp rx=0 tx=0).stop 2024-04-16T05:40:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f39026d3640 1 -- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc103b90 msgr2=0x7f38fc101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f39026d3640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc103b90 0x7f38fc101400 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f38f000d950 tx=0x7f38f000de10 comp rx=0 tx=0).stop 2024-04-16T05:40:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f39026d3640 1 -- 172.21.15.86:0/4078290140 shutdown_connections 2024-04-16T05:40:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f39026d3640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38fc10c5b0 0x7f38fc101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f39026d3640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc103b90 0x7f38fc101400 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.000+0000 7f39026d3640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f38e40a3d20 0x7f38e40a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.001+0000 7f39026d3640 1 --2- 172.21.15.86:0/4078290140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38fc103270 0x7f38fc100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.001+0000 7f39026d3640 1 -- 172.21.15.86:0/4078290140 >> 172.21.15.86:0/4078290140 conn(0x7f38fc0780b0 msgr2=0x7f38fc0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:50.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.001+0000 7f39026d3640 1 -- 172.21.15.86:0/4078290140 shutdown_connections 2024-04-16T05:40:50.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.001+0000 7f39026d3640 1 -- 172.21.15.86:0/4078290140 wait complete. 2024-04-16T05:40:50.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:40:50.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.124+0000 7fc642f1c640 1 -- 172.21.15.86:0/3090864871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 msgr2=0x7fc63c1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.124+0000 7fc642f1c640 1 --2- 172.21.15.86:0/3090864871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 0x7fc63c1073c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009900 tx=0x7fc62c02bc80 comp rx=0 tx=0).stop 2024-04-16T05:40:50.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.124+0000 7fc642f1c640 1 -- 172.21.15.86:0/3090864871 shutdown_connections 2024-04-16T05:40:50.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.124+0000 7fc642f1c640 1 --2- 172.21.15.86:0/3090864871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63c101b10 0x7fc63c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.124+0000 7fc642f1c640 1 --2- 172.21.15.86:0/3090864871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc63c101150 0x7fc63c1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.124+0000 7fc642f1c640 1 --2- 172.21.15.86:0/3090864871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 0x7fc63c1073c0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.124+0000 7fc642f1c640 1 -- 172.21.15.86:0/3090864871 >> 172.21.15.86:0/3090864871 conn(0x7fc63c077d50 msgr2=0x7fc63c078160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:50.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.124+0000 7fc642f1c640 1 -- 172.21.15.86:0/3090864871 shutdown_connections 2024-04-16T05:40:50.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.124+0000 7fc642f1c640 1 -- 172.21.15.86:0/3090864871 wait complete. 2024-04-16T05:40:50.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc642f1c640 1 Processor -- start 2024-04-16T05:40:50.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc642f1c640 1 -- start start 2024-04-16T05:40:50.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc642f1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63c101150 0x7fc63c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc642f1c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc63c101b10 0x7fc63c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc642f1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 0x7fc63c1a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc642f1c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1112c0 con 0x7fc63c106fe0 2024-04-16T05:40:50.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc642f1c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c111140 con 0x7fc63c101b10 2024-04-16T05:40:50.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc642f1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c111440 con 0x7fc63c101150 2024-04-16T05:40:50.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc6414b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 0x7fc63c1a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc6414b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 0x7fc63c1a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52752/0 (socket says 172.21.15.86:52752) 2024-04-16T05:40:50.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc6414b9640 1 -- 172.21.15.86:0/1339127127 learned_addr learned my addr 172.21.15.86:0/1339127127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63c101150 0x7fc63c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc63c101b10 0x7fc63c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc6414b9640 1 -- 172.21.15.86:0/1339127127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63c101150 msgr2=0x7fc63c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc6414b9640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63c101150 0x7fc63c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc6414b9640 1 -- 172.21.15.86:0/1339127127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc63c101b10 msgr2=0x7fc63c199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc6414b9640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc63c101b10 0x7fc63c199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc6414b9640 1 -- 172.21.15.86:0/1339127127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c1a4c80 con 0x7fc63c106fe0 2024-04-16T05:40:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc63c101b10 0x7fc63c199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.125+0000 7fc6414b9640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 0x7fc63c1a4740 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc634009fd0 tx=0x7fc63400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:50.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.126+0000 7fc639ffb640 1 -- 172.21.15.86:0/1339127127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634017070 con 0x7fc63c106fe0 2024-04-16T05:40:50.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.126+0000 7fc639ffb640 1 -- 172.21.15.86:0/1339127127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc634003e50 con 0x7fc63c106fe0 2024-04-16T05:40:50.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.126+0000 7fc642f1c640 1 -- 172.21.15.86:0/1339127127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63c1a4e50 con 0x7fc63c106fe0 2024-04-16T05:40:50.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.126+0000 7fc639ffb640 1 -- 172.21.15.86:0/1339127127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6340106c0 con 0x7fc63c106fe0 2024-04-16T05:40:50.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.126+0000 7fc642f1c640 1 -- 172.21.15.86:0/1339127127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63c1a5370 con 0x7fc63c106fe0 2024-04-16T05:40:50.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.128+0000 7fc639ffb640 1 -- 172.21.15.86:0/1339127127 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc634003970 con 0x7fc63c106fe0 2024-04-16T05:40:50.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc645be8740 1 Processor -- start 2024-04-16T05:40:50.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc645be8740 1 -- start start 2024-04-16T05:40:50.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc645be8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc645be8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 0x56551c9dce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc645be8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9df750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc645be8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56551c9390e0 con 0x56551c9d3e60 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc645be8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56551c8d6520 con 0x56551c9dd360 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc645be8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56551c9350a0 con 0x56551c9d4780 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc639ffb640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6280a3da0 0x7fc6280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9d4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52754/0 (socket says 172.21.15.86:52754) 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc63bfff640 1 -- 172.21.15.86:0/1986079342 learned_addr learned my addr 172.21.15.86:0/1986079342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc6414b9640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6280a3da0 0x7fc6280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1986079342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 0x56551c9dce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc639ffb640 1 -- 172.21.15.86:0/1339127127 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc6340d0890 con 0x7fc63c106fe0 2024-04-16T05:40:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc63bfff640 1 -- 172.21.15.86:0/1986079342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 msgr2=0x56551c9dce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1986079342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 0x56551c9dce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc63bfff640 1 -- 172.21.15.86:0/1986079342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 msgr2=0x56551c9df750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1986079342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9df750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.129+0000 7fc63bfff640 1 -- 172.21.15.86:0/1986079342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56551c8fe8d0 con 0x56551c9d3e60 2024-04-16T05:40:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.130+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1986079342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9d4240 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc624009ca0 tx=0x7fc624012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b42c7a02d804ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.130+0000 7fc6414b9640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6280a3da0 0x7fc6280a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc63406ef80 tx=0x7fc6340c75d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.130+0000 7fc632ffd640 1 -- 172.21.15.86:0/1986079342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624004030 con 0x56551c9d3e60 2024-04-16T05:40:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.130+0000 7fc632ffd640 1 -- 172.21.15.86:0/1986079342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6240041d0 con 0x56551c9d3e60 2024-04-16T05:40:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.130+0000 7fc632ffd640 1 -- 172.21.15.86:0/1986079342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6240136f0 con 0x56551c9d3e60 2024-04-16T05:40:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.130+0000 7fc645be8740 1 -- 172.21.15.86:0/1986079342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 msgr2=0x56551c9d4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 --2- 172.21.15.86:0/1986079342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9d4240 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc624009ca0 tx=0x7fc624012760 comp rx=0 tx=0).stop 2024-04-16T05:40:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 -- 172.21.15.86:0/1986079342 shutdown_connections 2024-04-16T05:40:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 --2- 172.21.15.86:0/1986079342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9df750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 --2- 172.21.15.86:0/1986079342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 0x56551c9dce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 --2- 172.21.15.86:0/1986079342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9d4240 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 -- 172.21.15.86:0/1986079342 >> 172.21.15.86:0/1986079342 conn(0x56551c9ce590 msgr2=0x56551c9ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 -- 172.21.15.86:0/1986079342 shutdown_connections 2024-04-16T05:40:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 -- 172.21.15.86:0/1986079342 wait complete. 2024-04-16T05:40:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 -- start start 2024-04-16T05:40:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 Processor -- start 2024-04-16T05:40:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9d4780 0x56551c9da8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9dd360 0x56551c9dae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56551c9e0060 con 0x56551c9d3e60 2024-04-16T05:40:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56551c8f24d0 con 0x56551c9d4780 2024-04-16T05:40:50.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56551c9e01e0 con 0x56551c9dd360 2024-04-16T05:40:50.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc645be8740 10 client.0 _ll_register_callbacks cb 0x56551c8d72c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:50.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.131+0000 7fc63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9da3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52764/0 (socket says 172.21.15.86:52764) 2024-04-16T05:40:50.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc63bfff640 1 -- 172.21.15.86:0/1252364490 learned_addr learned my addr 172.21.15.86:0/1252364490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:50.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9dd360 0x56551c9dae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9d4780 0x56551c9da8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc63bfff640 1 -- 172.21.15.86:0/1252364490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9dd360 msgr2=0x56551c9dae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9dd360 0x56551c9dae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc63bfff640 1 -- 172.21.15.86:0/1252364490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9d4780 msgr2=0x56551c9da8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9d4780 0x56551c9da8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc63bfff640 1 -- 172.21.15.86:0/1252364490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56551c64b3f0 con 0x56551c9d3e60 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9d4780 0x56551c9da8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9da3a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc624003f70 tx=0x7fc6240056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7e6584b47ed48a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc61bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc61bfff640 1 -- 172.21.15.86:0/1252364490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624021070 con 0x56551c9d3e60 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc61bfff640 1 -- 172.21.15.86:0/1252364490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc624013490 con 0x56551c9d3e60 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc645be8740 1 -- 172.21.15.86:0/1252364490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6240041d0 con 0x56551c9d3e60 2024-04-16T05:40:50.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc61bfff640 1 -- 172.21.15.86:0/1252364490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc624034050 con 0x56551c9d3e60 2024-04-16T05:40:50.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc645be8740 10 client.17364 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:50.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc645be8740 1 -- 172.21.15.86:0/1252364490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56551c7157d0 con 0x56551c9d3e60 2024-04-16T05:40:50.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.132+0000 7fc61bfff640 1 -- 172.21.15.86:0/1252364490 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6240041d0 con 0x56551c9d3e60 2024-04-16T05:40:50.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc61bfff640 1 -- 172.21.15.86:0/1252364490 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc62400f040 con 0x56551c9d3e60 2024-04-16T05:40:50.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc645be8740 10 client.17364 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:50.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc645be8740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:50.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc645be8740 20 client.17364 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:50.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc645be8740 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56551c9ed250 0x56551c9ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc645be8740 4 client.17364 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:50.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc645be8740 1 -- 172.21.15.86:0/1252364490 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56551c946b00 con 0x56551c9ed250 2024-04-16T05:40:50.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56551c9ed250 0x56551c9ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc61bfff640 10 client.17364 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:50.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.133+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56551c9ed250 0x56551c9ef640 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0de458f9ddbf0b7 server_cookie=39489d3437700cbc in_seq=0 out_seq=0 2024-04-16T05:40:50.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc61bfff640 1 -- 172.21.15.86:0/1252364490 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56551c946b00 con 0x56551c9ed250 2024-04-16T05:40:50.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc61bfff640 10 client.17364 handle_command_reply: tid=0 2024-04-16T05:40:50.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 client.17364 shutdown 2024-04-16T05:40:50.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 -- 172.21.15.86:0/1252364490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 msgr2=0x56551c9da3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9da3a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc624003f70 tx=0x7fc6240056d0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 -- 172.21.15.86:0/1252364490 shutdown_connections 2024-04-16T05:40:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56551c9ed250 0x56551c9ef640 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9dd360 0x56551c9dae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9d4780 0x56551c9da8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 --2- 172.21.15.86:0/1252364490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9da3a0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc61bfff640 0 client.17364 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 -- 172.21.15.86:0/1252364490 >> 172.21.15.86:0/1252364490 conn(0x56551c9ce590 msgr2=0x56551c9d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 -- 172.21.15.86:0/1252364490 shutdown_connections 2024-04-16T05:40:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 1 -- 172.21.15.86:0/1252364490 wait complete. 2024-04-16T05:40:50.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.134+0000 7fc645be8740 20 client.17364 trim_cache size 0 max 16384 2024-04-16T05:40:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.147+0000 7fc645be8740 1 Processor -- start 2024-04-16T05:40:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.147+0000 7fc645be8740 1 -- start start 2024-04-16T05:40:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.147+0000 7fc645be8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d3e60 0x56551c9e9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc645be8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d4780 0x56551c9d4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc645be8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc645be8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1af430 con 0x56551c9d4780 2024-04-16T05:40:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc645be8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1b0f70 con 0x56551c9dd360 2024-04-16T05:40:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc645be8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1afb00 con 0x56551c9d3e60 2024-04-16T05:40:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc640cb8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d4780 0x56551c9d4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc640cb8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d4780 0x56551c9d4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52770/0 (socket says 172.21.15.86:52770) 2024-04-16T05:40:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc640cb8640 1 -- 172.21.15.86:0/2992720241 learned_addr learned my addr 172.21.15.86:0/2992720241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc63bfff640 1 --2- 172.21.15.86:0/2992720241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d3e60 0x56551c9e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc63bfff640 1 --2- 172.21.15.86:0/2992720241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc640cb8640 1 -- 172.21.15.86:0/2992720241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d3e60 msgr2=0x56551c9e9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc640cb8640 1 --2- 172.21.15.86:0/2992720241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d3e60 0x56551c9e9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc640cb8640 1 -- 172.21.15.86:0/2992720241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 msgr2=0x56551c9c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc640cb8640 1 --2- 172.21.15.86:0/2992720241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc640cb8640 1 -- 172.21.15.86:0/2992720241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56551c9d8930 con 0x56551c9d4780 2024-04-16T05:40:50.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc640cb8640 1 --2- 172.21.15.86:0/2992720241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d4780 0x56551c9d4b60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02dfd0 tx=0x7fc62c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21d9fa2c543a413 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:50.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc6327fc640 1 -- 172.21.15.86:0/2992720241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c02cc30 con 0x56551c9d4780 2024-04-16T05:40:50.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc6327fc640 1 -- 172.21.15.86:0/2992720241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c005050 con 0x56551c9d4780 2024-04-16T05:40:50.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.148+0000 7fc6327fc640 1 -- 172.21.15.86:0/2992720241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0049b0 con 0x56551c9d4780 2024-04-16T05:40:50.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 -- 172.21.15.86:0/2992720241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d4780 msgr2=0x56551c9d4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 --2- 172.21.15.86:0/2992720241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d4780 0x56551c9d4b60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02dfd0 tx=0x7fc62c02c650 comp rx=0 tx=0).stop 2024-04-16T05:40:50.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 -- 172.21.15.86:0/2992720241 shutdown_connections 2024-04-16T05:40:50.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 --2- 172.21.15.86:0/2992720241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 --2- 172.21.15.86:0/2992720241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d4780 0x56551c9d4b60 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 --2- 172.21.15.86:0/2992720241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d3e60 0x56551c9e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 -- 172.21.15.86:0/2992720241 >> 172.21.15.86:0/2992720241 conn(0x56551c9ce590 msgr2=0x56551c9cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:50.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 -- 172.21.15.86:0/2992720241 shutdown_connections 2024-04-16T05:40:50.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 -- 172.21.15.86:0/2992720241 wait complete. 2024-04-16T05:40:50.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 -- start start 2024-04-16T05:40:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.149+0000 7fc645be8740 1 Processor -- start 2024-04-16T05:40:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 0x56551c9e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1b08a0 con 0x56551c9d3e60 2024-04-16T05:40:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1b01d0 con 0x56551c9dd360 2024-04-16T05:40:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1ad3f0 con 0x56551c9d4780 2024-04-16T05:40:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 10 client.0 _ll_register_callbacks cb 0x7fc624009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9e5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52774/0 (socket says 172.21.15.86:52774) 2024-04-16T05:40:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc63bfff640 1 -- 172.21.15.86:0/1398366692 learned_addr learned my addr 172.21.15.86:0/1398366692 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 0x56551c9e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc63bfff640 1 -- 172.21.15.86:0/1398366692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 msgr2=0x56551c9e5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 0x56551c9e5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc63bfff640 1 -- 172.21.15.86:0/1398366692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 msgr2=0x56551c9d6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc63bfff640 1 -- 172.21.15.86:0/1398366692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56551c9d73b0 con 0x56551c9d3e60 2024-04-16T05:40:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 0x56551c9e5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:50.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc63bfff640 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9e5540 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc624013010 tx=0x7fc62401bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee4a4f36e82b76a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:50.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc61bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:50.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc61bfff640 1 -- 172.21.15.86:0/1398366692 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624021040 con 0x56551c9d3e60 2024-04-16T05:40:50.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc61bfff640 1 -- 172.21.15.86:0/1398366692 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc6240050f0 con 0x56551c9d3e60 2024-04-16T05:40:50.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 1 -- 172.21.15.86:0/1398366692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc62c005050 con 0x56551c9d3e60 2024-04-16T05:40:50.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc61bfff640 1 -- 172.21.15.86:0/1398366692 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc624024040 con 0x56551c9d3e60 2024-04-16T05:40:50.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 10 client.17370 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:50.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc645be8740 1 -- 172.21.15.86:0/1398366692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56551ca53110 con 0x56551c9d3e60 2024-04-16T05:40:50.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.150+0000 7fc61bfff640 1 -- 172.21.15.86:0/1398366692 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c005050 con 0x56551c9d3e60 2024-04-16T05:40:50.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc61bfff640 1 -- 172.21.15.86:0/1398366692 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc624013940 con 0x56551c9d3e60 2024-04-16T05:40:50.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc645be8740 10 client.17370 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:50.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc645be8740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:50.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc645be8740 20 client.17370 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:50.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc645be8740 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56551ca537d0 0x56551ca55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:50.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc645be8740 4 client.17370 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:50.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc645be8740 1 -- 172.21.15.86:0/1398366692 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56551c940970 con 0x56551ca537d0 2024-04-16T05:40:50.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56551ca537d0 0x56551ca55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:50.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc61bfff640 10 client.17370 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:50.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc640cb8640 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56551ca537d0 0x56551ca55bc0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a671fbe474b3ee server_cookie=f4b45d6607f1fea0 in_seq=0 out_seq=0 2024-04-16T05:40:50.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc61bfff640 1 -- 172.21.15.86:0/1398366692 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56551c940970 con 0x56551ca537d0 2024-04-16T05:40:50.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc61bfff640 10 client.17370 handle_command_reply: tid=0 2024-04-16T05:40:50.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.151+0000 7fc645be8740 1 client.17370 shutdown 2024-04-16T05:40:50.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 -- 172.21.15.86:0/1398366692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 msgr2=0x56551c9e5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9e5540 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc624013010 tx=0x7fc62401bb30 comp rx=0 tx=0).stop 2024-04-16T05:40:50.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 -- 172.21.15.86:0/1398366692 shutdown_connections 2024-04-16T05:40:50.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56551ca537d0 0x56551ca55bc0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56551c9dd360 0x56551c9d6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc61bfff640 0 client.17370 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:50.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56551c9d4780 0x56551c9e5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 --2- 172.21.15.86:0/1398366692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56551c9d3e60 0x56551c9e5540 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 -- 172.21.15.86:0/1398366692 >> 172.21.15.86:0/1398366692 conn(0x56551c9ce590 msgr2=0x56551c9cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:50.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 -- 172.21.15.86:0/1398366692 shutdown_connections 2024-04-16T05:40:50.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 1 -- 172.21.15.86:0/1398366692 wait complete. 2024-04-16T05:40:50.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc645be8740 20 client.17370 trim_cache size 0 max 16384 2024-04-16T05:40:50.162 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:40:50.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 -- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6280a3da0 msgr2=0x7fc6280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6280a3da0 0x7fc6280a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc63406ef80 tx=0x7fc6340c75d0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 -- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 msgr2=0x7fc63c1a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:50.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 0x7fc63c1a4740 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc634009fd0 tx=0x7fc63400b260 comp rx=0 tx=0).stop 2024-04-16T05:40:50.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 -- 172.21.15.86:0/1339127127 shutdown_connections 2024-04-16T05:40:50.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc63c106fe0 0x7fc63c1a4740 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc6280a3da0 0x7fc6280a6260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc63c101b10 0x7fc63c199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 --2- 172.21.15.86:0/1339127127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63c101150 0x7fc63c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:50.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 -- 172.21.15.86:0/1339127127 >> 172.21.15.86:0/1339127127 conn(0x7fc63c077d50 msgr2=0x7fc63c0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:50.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 -- 172.21.15.86:0/1339127127 shutdown_connections 2024-04-16T05:40:50.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:50.152+0000 7fc642f1c640 1 -- 172.21.15.86:0/1339127127 wait complete. 2024-04-16T05:40:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:50 smithi194 ceph-mon[35865]: pgmap v169: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:40:50.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:50 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3516940291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:50 smithi086 ceph-mon[32028]: pgmap v169: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:40:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:50 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3516940291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:50.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:50 smithi196 ceph-mon[36373]: pgmap v169: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:40:50.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:50 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3516940291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:51.166 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:40:51.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:40:51.167 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:40:51.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 -- 172.21.15.86:0/2822061011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9410c5c0 msgr2=0x7fca9410ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 --2- 172.21.15.86:0/2822061011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9410c5c0 0x7fca9410ea60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009ca0 tx=0x7fca8c02c230 comp rx=0 tx=0).stop 2024-04-16T05:40:51.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 -- 172.21.15.86:0/2822061011 shutdown_connections 2024-04-16T05:40:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 --2- 172.21.15.86:0/2822061011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9410c5c0 0x7fca9410ea60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 --2- 172.21.15.86:0/2822061011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca94103ba0 0x7fca9410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 --2- 172.21.15.86:0/2822061011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca94103280 0x7fca94103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 -- 172.21.15.86:0/2822061011 >> 172.21.15.86:0/2822061011 conn(0x7fca94078000 msgr2=0x7fca94078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 -- 172.21.15.86:0/2822061011 shutdown_connections 2024-04-16T05:40:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 -- 172.21.15.86:0/2822061011 wait complete. 2024-04-16T05:40:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.290+0000 7fca9a21b640 1 Processor -- start 2024-04-16T05:40:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca9a21b640 1 -- start start 2024-04-16T05:40:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca9a21b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca94103280 0x7fca9419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca9a21b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca94103ba0 0x7fca9419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca9a21b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9410c5c0 0x7fca94198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca9a21b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca941123a0 con 0x7fca94103ba0 2024-04-16T05:40:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca9a21b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fca94112220 con 0x7fca94103280 2024-04-16T05:40:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca9a21b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca94112520 con 0x7fca9410c5c0 2024-04-16T05:40:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca94103ba0 0x7fca9419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca94103ba0 0x7fca9419eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43744/0 (socket says 172.21.15.86:43744) 2024-04-16T05:40:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca8bfff640 1 -- 172.21.15.86:0/4101033714 learned_addr learned my addr 172.21.15.86:0/4101033714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca937fe640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca94103280 0x7fca9419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca93fff640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9410c5c0 0x7fca94198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca8bfff640 1 -- 172.21.15.86:0/4101033714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9410c5c0 msgr2=0x7fca94198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca8bfff640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9410c5c0 0x7fca94198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca8bfff640 1 -- 172.21.15.86:0/4101033714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca94103280 msgr2=0x7fca9419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca8bfff640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca94103280 0x7fca9419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca8bfff640 1 -- 172.21.15.86:0/4101033714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca94198fc0 con 0x7fca94103ba0 2024-04-16T05:40:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca937fe640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca94103280 0x7fca9419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.291+0000 7fca8bfff640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca94103ba0 0x7fca9419eb30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d770 tx=0x7fca7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.292+0000 7fca917fa640 1 -- 172.21.15.86:0/4101033714 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c017070 con 0x7fca94103ba0 2024-04-16T05:40:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.292+0000 7fca917fa640 1 -- 172.21.15.86:0/4101033714 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca7c005500 con 0x7fca94103ba0 2024-04-16T05:40:51.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.292+0000 7fca9a21b640 1 -- 172.21.15.86:0/4101033714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca941992b0 con 0x7fca94103ba0 2024-04-16T05:40:51.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.292+0000 7fca917fa640 1 -- 172.21.15.86:0/4101033714 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c004e00 con 0x7fca94103ba0 2024-04-16T05:40:51.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.292+0000 7fca9a21b640 1 -- 172.21.15.86:0/4101033714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca941a5570 con 0x7fca94103ba0 2024-04-16T05:40:51.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca917fa640 1 -- 172.21.15.86:0/4101033714 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fca7c004fa0 con 0x7fca94103ba0 2024-04-16T05:40:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca9ce06740 1 Processor -- start 2024-04-16T05:40:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca9ce06740 1 -- start start 2024-04-16T05:40:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fbce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fc83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca9ce06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5e0f21e90 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca9ce06740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e5e0ebf4c0 con 0x55e5e0fbcab0 2024-04-16T05:40:51.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca9ce06740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5e0f1de50 con 0x55e5e0fc5fb0 2024-04-16T05:40:51.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca917fa640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fca800a3d20 0x7fca800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43748/0 (socket says 172.21.15.86:43748) 2024-04-16T05:40:51.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.294+0000 7fca937fe640 1 -- 172.21.15.86:0/1054801611 learned_addr learned my addr 172.21.15.86:0/1054801611 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca93fff640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fca800a3d20 0x7fca800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca8bfff640 1 --2- 172.21.15.86:0/1054801611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fc83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca93fff640 1 --2- 172.21.15.86:0/1054801611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fbce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca917fa640 1 -- 172.21.15.86:0/4101033714 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fca7c0d05d0 con 0x7fca94103ba0 2024-04-16T05:40:51.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca937fe640 1 -- 172.21.15.86:0/1054801611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fc5fb0 msgr2=0x55e5e0fc83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca937fe640 1 --2- 172.21.15.86:0/1054801611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fc83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca937fe640 1 -- 172.21.15.86:0/1054801611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 msgr2=0x55e5e0fbce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca937fe640 1 --2- 172.21.15.86:0/1054801611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fbce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca937fe640 1 -- 172.21.15.86:0/1054801611 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e0cfc7e0 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca937fe640 1 --2- 172.21.15.86:0/1054801611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc5a70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fca8400cca0 tx=0x7fca8400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775b2d1ea873aaa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca93fff640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fca800a3d20 0x7fca800a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0326c0 tx=0x7fca8c005d80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca89ffb640 1 -- 172.21.15.86:0/1054801611 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84005220 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca89ffb640 1 -- 172.21.15.86:0/1054801611 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca84004c80 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.295+0000 7fca89ffb640 1 -- 172.21.15.86:0/1054801611 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84010890 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 -- 172.21.15.86:0/1054801611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 msgr2=0x55e5e0fc5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1054801611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc5a70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fca8400cca0 tx=0x7fca8400e720 comp rx=0 tx=0).stop 2024-04-16T05:40:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 -- 172.21.15.86:0/1054801611 shutdown_connections 2024-04-16T05:40:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1054801611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fc83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1054801611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc5a70 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1054801611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fbce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 -- 172.21.15.86:0/1054801611 >> 172.21.15.86:0/1054801611 conn(0x55e5e0fb71e0 msgr2=0x55e5e0fb75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 -- 172.21.15.86:0/1054801611 shutdown_connections 2024-04-16T05:40:51.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 -- 172.21.15.86:0/1054801611 wait complete. 2024-04-16T05:40:51.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.296+0000 7fca9ce06740 1 -- start start 2024-04-16T05:40:51.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca9ce06740 1 Processor -- start 2024-04-16T05:40:51.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fc2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fc39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca9ce06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5e0fc8da0 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca9ce06740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e5e0eaa410 con 0x55e5e0fc5fb0 2024-04-16T05:40:51.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca9ce06740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5e0fc8f20 con 0x55e5e0fbcab0 2024-04-16T05:40:51.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca9ce06740 10 client.0 _ll_register_callbacks cb 0x55e5e0ea5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:51.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43760/0 (socket says 172.21.15.86:43760) 2024-04-16T05:40:51.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca8bfff640 1 -- 172.21.15.86:0/1283901394 learned_addr learned my addr 172.21.15.86:0/1283901394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca93fff640 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fc39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca937fe640 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fc2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca8bfff640 1 -- 172.21.15.86:0/1283901394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbcab0 msgr2=0x55e5e0fc2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca8bfff640 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fc2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca8bfff640 1 -- 172.21.15.86:0/1283901394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fc5fb0 msgr2=0x55e5e0fc39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca8bfff640 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fc39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca8bfff640 1 -- 172.21.15.86:0/1283901394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e0d5cf70 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca8bfff640 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc3480 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fca7c098bd0 tx=0x7fca7c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8768f87fde4ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca72ffd640 1 -- 172.21.15.86:0/1283901394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c09c900 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.297+0000 7fca72ffd640 1 -- 172.21.15.86:0/1283901394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fca7c0510b0 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca9ce06740 1 -- 172.21.15.86:0/1283901394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca84004c80 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca72ffd640 1 -- 172.21.15.86:0/1283901394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca7c09c200 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca9ce06740 10 client.17382 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:51.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca9ce06740 1 -- 172.21.15.86:0/1283901394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5e0bf4170 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca72ffd640 1 -- 172.21.15.86:0/1283901394 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca84004c80 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca72ffd640 1 -- 172.21.15.86:0/1283901394 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fca7c0d0240 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca9ce06740 10 client.17382 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:51.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca9ce06740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:51.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca9ce06740 20 client.17382 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0fd6320 0x55e5e0fd8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca9ce06740 4 client.17382 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca9ce06740 1 -- 172.21.15.86:0/1283901394 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5e0f2f8b0 con 0x55e5e0fd6320 2024-04-16T05:40:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.298+0000 7fca937fe640 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0fd6320 0x55e5e0fd8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.299+0000 7fca937fe640 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0fd6320 0x55e5e0fd8710 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbbdeabfb4098d64 server_cookie=ec66547932d21a99 in_seq=0 out_seq=0 2024-04-16T05:40:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.299+0000 7fca72ffd640 10 client.17382 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca72ffd640 1 -- 172.21.15.86:0/1283901394 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5e0f2f8b0 con 0x55e5e0fd6320 2024-04-16T05:40:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca72ffd640 10 client.17382 handle_command_reply: tid=0 2024-04-16T05:40:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 client.17382 shutdown 2024-04-16T05:40:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 -- 172.21.15.86:0/1283901394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 msgr2=0x55e5e0fc3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc3480 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fca7c098bd0 tx=0x7fca7c010f90 comp rx=0 tx=0).stop 2024-04-16T05:40:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 -- 172.21.15.86:0/1283901394 shutdown_connections 2024-04-16T05:40:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0fd6320 0x55e5e0fd8710 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fc39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc3480 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca72ffd640 0 client.17382 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:51.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 --2- 172.21.15.86:0/1283901394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fc2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 -- 172.21.15.86:0/1283901394 >> 172.21.15.86:0/1283901394 conn(0x55e5e0fb71e0 msgr2=0x55e5e0fb8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 -- 172.21.15.86:0/1283901394 shutdown_connections 2024-04-16T05:40:51.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 1 -- 172.21.15.86:0/1283901394 wait complete. 2024-04-16T05:40:51.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.300+0000 7fca9ce06740 20 client.17382 trim_cache size 0 max 16384 2024-04-16T05:40:51.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.317+0000 7fca9ce06740 1 Processor -- start 2024-04-16T05:40:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.317+0000 7fca9ce06740 1 -- start start 2024-04-16T05:40:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.317+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fc5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca9ce06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca941af460 con 0x55e5e0fc5fb0 2024-04-16T05:40:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca9ce06740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fca941b0fa0 con 0x55e5e0fbcab0 2024-04-16T05:40:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca9ce06740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca941afb30 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca93fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca93fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fddfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43774/0 (socket says 172.21.15.86:43774) 2024-04-16T05:40:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca93fff640 1 -- 172.21.15.86:0/361558414 learned_addr learned my addr 172.21.15.86:0/361558414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca8bfff640 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca937fe640 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fc5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca93fff640 1 -- 172.21.15.86:0/361558414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbd3d0 msgr2=0x55e5e0fc37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca93fff640 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca93fff640 1 -- 172.21.15.86:0/361558414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 msgr2=0x55e5e0fc5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca93fff640 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fc5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca93fff640 1 -- 172.21.15.86:0/361558414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e0fbeb40 con 0x55e5e0fc5fb0 2024-04-16T05:40:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca8bfff640 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca93fff640 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fddfc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0325e0 tx=0x7fca8c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72379d09799b0d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca897fa640 1 -- 172.21.15.86:0/361558414 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c048040 con 0x55e5e0fc5fb0 2024-04-16T05:40:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca897fa640 1 -- 172.21.15.86:0/361558414 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca8c03a040 con 0x55e5e0fc5fb0 2024-04-16T05:40:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.318+0000 7fca897fa640 1 -- 172.21.15.86:0/361558414 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c039030 con 0x55e5e0fc5fb0 2024-04-16T05:40:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 -- 172.21.15.86:0/361558414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fc5fb0 msgr2=0x55e5e0fddfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fddfc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0325e0 tx=0x7fca8c03d000 comp rx=0 tx=0).stop 2024-04-16T05:40:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 -- 172.21.15.86:0/361558414 shutdown_connections 2024-04-16T05:40:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fddfc0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fc37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 --2- 172.21.15.86:0/361558414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fc5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 -- 172.21.15.86:0/361558414 >> 172.21.15.86:0/361558414 conn(0x55e5e0fb71e0 msgr2=0x55e5e0fb81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 -- 172.21.15.86:0/361558414 shutdown_connections 2024-04-16T05:40:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 -- 172.21.15.86:0/361558414 wait complete. 2024-04-16T05:40:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 -- start start 2024-04-16T05:40:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.319+0000 7fca9ce06740 1 Processor -- start 2024-04-16T05:40:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fcb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fcb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fcbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca941b08d0 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fca941b0200 con 0x55e5e0fbcab0 2024-04-16T05:40:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca941ad420 con 0x55e5e0fc5fb0 2024-04-16T05:40:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 10 client.0 _ll_register_callbacks cb 0x7fca7c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fcb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fcb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43780/0 (socket says 172.21.15.86:43780) 2024-04-16T05:40:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca8bfff640 1 -- 172.21.15.86:0/2463571543 learned_addr learned my addr 172.21.15.86:0/2463571543 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca8bfff640 1 -- 172.21.15.86:0/2463571543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fc5fb0 msgr2=0x55e5e0fcbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca937fe640 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fcb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca8bfff640 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fcbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca8bfff640 1 -- 172.21.15.86:0/2463571543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 msgr2=0x55e5e0fcb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca8bfff640 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fcb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca8bfff640 1 -- 172.21.15.86:0/2463571543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e103c970 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca8bfff640 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fcb6b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0cdfb0 tx=0x7fca7c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2bb41975cc72a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:51.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca72ffd640 1 -- 172.21.15.86:0/2463571543 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b9020 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca72ffd640 1 -- 172.21.15.86:0/2463571543 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fca7c0d0240 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 1 -- 172.21.15.86:0/2463571543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca8c03a040 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca72ffd640 1 -- 172.21.15.86:0/2463571543 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca7c07f050 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 10 client.17388 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:51.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.320+0000 7fca9ce06740 1 -- 172.21.15.86:0/2463571543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5e103dc00 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca72ffd640 1 -- 172.21.15.86:0/2463571543 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca8c03a040 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca72ffd640 1 -- 172.21.15.86:0/2463571543 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fca7c0a0120 con 0x55e5e0fbd3d0 2024-04-16T05:40:51.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca9ce06740 10 client.17388 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:51.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca9ce06740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:51.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca9ce06740 20 client.17388 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:51.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca9ce06740 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e103e270 0x55e5e1040690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca9ce06740 4 client.17388 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:40:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca9ce06740 1 -- 172.21.15.86:0/2463571543 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e5e0f29720 con 0x55e5e103e270 2024-04-16T05:40:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca937fe640 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e103e270 0x55e5e1040690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca937fe640 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e103e270 0x55e5e1040690 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76951f501622bf01 server_cookie=ac90161c9cf9b7ef in_seq=0 out_seq=0 2024-04-16T05:40:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.321+0000 7fca72ffd640 10 client.17388 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.322+0000 7fca72ffd640 1 -- 172.21.15.86:0/2463571543 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e5e0f29720 con 0x55e5e103e270 2024-04-16T05:40:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.322+0000 7fca72ffd640 10 client.17388 handle_command_reply: tid=0 2024-04-16T05:40:51.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.322+0000 7fca9ce06740 1 client.17388 shutdown 2024-04-16T05:40:51.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 -- 172.21.15.86:0/2463571543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 msgr2=0x55e5e0fcb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fcb6b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0cdfb0 tx=0x7fca7c0057b0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 -- 172.21.15.86:0/2463571543 shutdown_connections 2024-04-16T05:40:51.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e103e270 0x55e5e1040690 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0fc5fb0 0x55e5e0fcbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca72ffd640 0 client.17388 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:51.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0fbd3d0 0x55e5e0fcb6b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 --2- 172.21.15.86:0/2463571543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e0fbcab0 0x55e5e0fcb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 -- 172.21.15.86:0/2463571543 >> 172.21.15.86:0/2463571543 conn(0x55e5e0fb71e0 msgr2=0x55e5e0fb8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 -- 172.21.15.86:0/2463571543 shutdown_connections 2024-04-16T05:40:51.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 1 -- 172.21.15.86:0/2463571543 wait complete. 2024-04-16T05:40:51.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9ce06740 20 client.17388 trim_cache size 0 max 16384 2024-04-16T05:40:51.332 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:40:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:40:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "8e133513-755f-4c9c-afb7-7ab9d1943ef2", 2024-04-16T05:40:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:40:51.333 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:40:51.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 -- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fca800a3d20 msgr2=0x7fca800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fca800a3d20 0x7fca800a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0326c0 tx=0x7fca8c005d80 comp rx=0 tx=0).stop 2024-04-16T05:40:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 -- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca94103ba0 msgr2=0x7fca9419eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca94103ba0 0x7fca9419eb30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d770 tx=0x7fca7c00dc30 comp rx=0 tx=0).stop 2024-04-16T05:40:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 -- 172.21.15.86:0/4101033714 shutdown_connections 2024-04-16T05:40:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9410c5c0 0x7fca94198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca94103ba0 0x7fca9419eb30 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fca800a3d20 0x7fca800a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 --2- 172.21.15.86:0/4101033714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca94103280 0x7fca9419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.323+0000 7fca9a21b640 1 -- 172.21.15.86:0/4101033714 >> 172.21.15.86:0/4101033714 conn(0x7fca94078000 msgr2=0x7fca940fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.324+0000 7fca9a21b640 1 -- 172.21.15.86:0/4101033714 shutdown_connections 2024-04-16T05:40:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.324+0000 7fca9a21b640 1 -- 172.21.15.86:0/4101033714 wait complete. 2024-04-16T05:40:51.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:40:51.336 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:40:51.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.444+0000 7f021459d640 1 -- 172.21.15.86:0/906190245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 msgr2=0x7f020c103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.444+0000 7f021459d640 1 --2- 172.21.15.86:0/906190245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 0x7f020c103660 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0208009900 tx=0x7f020802bd60 comp rx=0 tx=0).stop 2024-04-16T05:40:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.444+0000 7f021459d640 1 -- 172.21.15.86:0/906190245 shutdown_connections 2024-04-16T05:40:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.444+0000 7f021459d640 1 --2- 172.21.15.86:0/906190245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c10c5c0 0x7f020c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.444+0000 7f021459d640 1 --2- 172.21.15.86:0/906190245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f020c103ba0 0x7f020c10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.444+0000 7f021459d640 1 --2- 172.21.15.86:0/906190245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 0x7f020c103660 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.444+0000 7f021459d640 1 -- 172.21.15.86:0/906190245 >> 172.21.15.86:0/906190245 conn(0x7f020c078000 msgr2=0x7f020c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.444+0000 7f021459d640 1 -- 172.21.15.86:0/906190245 shutdown_connections 2024-04-16T05:40:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.444+0000 7f021459d640 1 -- 172.21.15.86:0/906190245 wait complete. 2024-04-16T05:40:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f021459d640 1 Processor -- start 2024-04-16T05:40:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f021459d640 1 -- start start 2024-04-16T05:40:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f021459d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 0x7f020c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f021459d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103ba0 0x7f020c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f021459d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f020c10c5c0 0x7f020c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f021459d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f020c1123f0 con 0x7f020c103280 2024-04-16T05:40:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f021459d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f020c112270 con 0x7f020c10c5c0 2024-04-16T05:40:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f021459d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f020c112570 con 0x7f020c103ba0 2024-04-16T05:40:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0212339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 0x7f020c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0212339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 0x7f020c19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43812/0 (socket says 172.21.15.86:43812) 2024-04-16T05:40:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0212339640 1 -- 172.21.15.86:0/3814800978 learned_addr learned my addr 172.21.15.86:0/3814800978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0211b38640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103ba0 0x7f020c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0212b3a640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f020c10c5c0 0x7f020c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0212339640 1 -- 172.21.15.86:0/3814800978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103ba0 msgr2=0x7f020c19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0212339640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103ba0 0x7f020c19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0212339640 1 -- 172.21.15.86:0/3814800978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f020c10c5c0 msgr2=0x7f020c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0212339640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f020c10c5c0 0x7f020c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.445+0000 7f0212339640 1 -- 172.21.15.86:0/3814800978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c198ef0 con 0x7f020c103280 2024-04-16T05:40:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.446+0000 7f0212339640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 0x7f020c19e570 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0208009a20 tx=0x7f020802df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.446+0000 7f02037fe640 1 -- 172.21.15.86:0/3814800978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208043020 con 0x7f020c103280 2024-04-16T05:40:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.446+0000 7f02037fe640 1 -- 172.21.15.86:0/3814800978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02080054e0 con 0x7f020c103280 2024-04-16T05:40:51.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.446+0000 7f021459d640 1 -- 172.21.15.86:0/3814800978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c199180 con 0x7f020c103280 2024-04-16T05:40:51.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.446+0000 7f02037fe640 1 -- 172.21.15.86:0/3814800978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208003ce0 con 0x7f020c103280 2024-04-16T05:40:51.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.446+0000 7f021459d640 1 -- 172.21.15.86:0/3814800978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c1a54f0 con 0x7f020c103280 2024-04-16T05:40:51.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.448+0000 7f02037fe640 1 -- 172.21.15.86:0/3814800978 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0208046020 con 0x7f020c103280 2024-04-16T05:40:51.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.448+0000 7f0217188740 1 Processor -- start 2024-04-16T05:40:51.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0217188740 1 -- start start 2024-04-16T05:40:51.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0217188740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32739e50 0x55aa3273a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0217188740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa3273a770 0x55aa32742e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0217188740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa32743350 0x55aa32745740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0217188740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa3269f180 con 0x55aa3273a770 2024-04-16T05:40:51.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0217188740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55aa3263c360 con 0x55aa32743350 2024-04-16T05:40:51.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0217188740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aa3269b140 con 0x55aa32739e50 2024-04-16T05:40:51.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f02037fe640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f01f80a3d20 0x7f01f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa3273a770 0x55aa32742e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212b3a640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f01f80a3d20 0x7f01f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa3273a770 0x55aa32742e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43828/0 (socket says 172.21.15.86:43828) 2024-04-16T05:40:51.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212339640 1 -- 172.21.15.86:0/275301824 learned_addr learned my addr 172.21.15.86:0/275301824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f02037fe640 1 -- 172.21.15.86:0/3814800978 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f02080f2cd0 con 0x7f020c103280 2024-04-16T05:40:51.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212b3a640 1 --2- 172.21.15.86:0/275301824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32739e50 0x55aa3273a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0211b38640 1 --2- 172.21.15.86:0/275301824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa32743350 0x55aa32745740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212339640 1 -- 172.21.15.86:0/275301824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32739e50 msgr2=0x55aa3273a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212339640 1 --2- 172.21.15.86:0/275301824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32739e50 0x55aa3273a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212339640 1 -- 172.21.15.86:0/275301824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa32743350 msgr2=0x55aa32745740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212339640 1 --2- 172.21.15.86:0/275301824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa32743350 0x55aa32745740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212339640 1 -- 172.21.15.86:0/275301824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa3247b7d0 con 0x55aa3273a770 2024-04-16T05:40:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.449+0000 7f0212b3a640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f01f80a3d20 0x7f01f80a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0204005c40 tx=0x7f02040129b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0212339640 1 --2- 172.21.15.86:0/275301824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa3273a770 0x55aa32742e10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f02080c7150 tx=0x7f0208091590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8dc3d902340c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0200bfe640 1 -- 172.21.15.86:0/275301824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080be400 con 0x55aa3273a770 2024-04-16T05:40:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0200bfe640 1 -- 172.21.15.86:0/275301824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02080044a0 con 0x55aa3273a770 2024-04-16T05:40:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0200bfe640 1 -- 172.21.15.86:0/275301824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080beaa0 con 0x55aa3273a770 2024-04-16T05:40:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0217188740 1 -- 172.21.15.86:0/275301824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa3273a770 msgr2=0x55aa32742e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0217188740 1 --2- 172.21.15.86:0/275301824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa3273a770 0x55aa32742e10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f02080c7150 tx=0x7f0208091590 comp rx=0 tx=0).stop 2024-04-16T05:40:51.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0217188740 1 -- 172.21.15.86:0/275301824 shutdown_connections 2024-04-16T05:40:51.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0217188740 1 --2- 172.21.15.86:0/275301824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa32743350 0x55aa32745740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0217188740 1 --2- 172.21.15.86:0/275301824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa3273a770 0x55aa32742e10 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0217188740 1 --2- 172.21.15.86:0/275301824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32739e50 0x55aa3273a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.450+0000 7f0217188740 1 -- 172.21.15.86:0/275301824 >> 172.21.15.86:0/275301824 conn(0x55aa32734580 msgr2=0x55aa32734960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 -- 172.21.15.86:0/275301824 shutdown_connections 2024-04-16T05:40:51.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 -- 172.21.15.86:0/275301824 wait complete. 2024-04-16T05:40:51.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 -- start start 2024-04-16T05:40:51.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 Processor -- start 2024-04-16T05:40:51.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa32740370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 0x55aa327408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 0x55aa32740df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa32745f00 con 0x55aa32739e50 2024-04-16T05:40:51.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55aa326582c0 con 0x55aa3273a770 2024-04-16T05:40:51.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aa32746080 con 0x55aa32743350 2024-04-16T05:40:51.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0217188740 10 client.0 _ll_register_callbacks cb 0x55aa3263f090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:51.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0212339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa32740370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0212339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa32740370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43844/0 (socket says 172.21.15.86:43844) 2024-04-16T05:40:51.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0212339640 1 -- 172.21.15.86:0/4012244879 learned_addr learned my addr 172.21.15.86:0/4012244879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0212b3a640 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 0x55aa32740df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.451+0000 7f0211b38640 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 0x55aa327408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0212339640 1 -- 172.21.15.86:0/4012244879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 msgr2=0x55aa32740df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0212339640 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 0x55aa32740df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0212339640 1 -- 172.21.15.86:0/4012244879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 msgr2=0x55aa327408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0212339640 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 0x55aa327408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0212339640 1 -- 172.21.15.86:0/4012244879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa323b13f0 con 0x55aa32739e50 2024-04-16T05:40:51.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0211b38640 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 0x55aa327408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:51.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0212339640 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa32740370 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f02080bb420 tx=0x7f0208005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb76b0abccf071f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f01f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:51.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f01f5ffb640 1 -- 172.21.15.86:0/4012244879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080f0530 con 0x55aa32739e50 2024-04-16T05:40:51.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f01f5ffb640 1 -- 172.21.15.86:0/4012244879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f02080bed00 con 0x55aa32739e50 2024-04-16T05:40:51.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0217188740 1 -- 172.21.15.86:0/4012244879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa326acba0 con 0x55aa32739e50 2024-04-16T05:40:51.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f01f5ffb640 1 -- 172.21.15.86:0/4012244879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0208005690 con 0x55aa32739e50 2024-04-16T05:40:51.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0217188740 10 client.17400 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:51.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0217188740 1 -- 172.21.15.86:0/4012244879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa323710f0 con 0x55aa32739e50 2024-04-16T05:40:51.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f01f5ffb640 1 -- 172.21.15.86:0/4012244879 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55aa326acba0 con 0x55aa32739e50 2024-04-16T05:40:51.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f01f5ffb640 1 -- 172.21.15.86:0/4012244879 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02080ddbd0 con 0x55aa32739e50 2024-04-16T05:40:51.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0217188740 10 client.17400 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:51.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0217188740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:51.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0217188740 20 client.17400 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:51.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0217188740 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55aa32753270 0x55aa32755660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0217188740 4 client.17400 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:51.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.452+0000 7f0217188740 1 -- 172.21.15.86:0/4012244879 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa326a6a10 con 0x55aa32753270 2024-04-16T05:40:51.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.453+0000 7f0211b38640 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55aa32753270 0x55aa32755660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.453+0000 7f01f5ffb640 10 client.17400 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:51.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.453+0000 7f0211b38640 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55aa32753270 0x55aa32755660 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79912fb9ef65beb5 server_cookie=9b248a4c0b5748a6 in_seq=0 out_seq=0 2024-04-16T05:40:51.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f01f5ffb640 1 -- 172.21.15.86:0/4012244879 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa326a6a10 con 0x55aa32753270 2024-04-16T05:40:51.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f01f5ffb640 10 client.17400 handle_command_reply: tid=0 2024-04-16T05:40:51.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 client.17400 shutdown 2024-04-16T05:40:51.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 -- 172.21.15.86:0/4012244879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 msgr2=0x55aa32740370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa32740370 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f02080bb420 tx=0x7f0208005c00 comp rx=0 tx=0).stop 2024-04-16T05:40:51.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 -- 172.21.15.86:0/4012244879 shutdown_connections 2024-04-16T05:40:51.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55aa32753270 0x55aa32755660 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 0x55aa32740df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 0x55aa327408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f01f5ffb640 0 client.17400 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:51.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 --2- 172.21.15.86:0/4012244879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa32740370 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 -- 172.21.15.86:0/4012244879 >> 172.21.15.86:0/4012244879 conn(0x55aa32734580 msgr2=0x55aa327362d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 -- 172.21.15.86:0/4012244879 shutdown_connections 2024-04-16T05:40:51.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 1 -- 172.21.15.86:0/4012244879 wait complete. 2024-04-16T05:40:51.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.454+0000 7f0217188740 20 client.17400 trim_cache size 0 max 16384 2024-04-16T05:40:51.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.471+0000 7f0217188740 1 Processor -- start 2024-04-16T05:40:51.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.471+0000 7f0217188740 1 -- start start 2024-04-16T05:40:51.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0217188740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa32739e50 0x55aa3274ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0217188740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa3273a770 0x55aa32740c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0217188740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32743350 0x55aa327335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0217188740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f020c1af430 con 0x55aa32743350 2024-04-16T05:40:51.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0217188740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f020c1b0f70 con 0x55aa32739e50 2024-04-16T05:40:51.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0217188740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f020c1afb00 con 0x55aa3273a770 2024-04-16T05:40:51.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0212b3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32743350 0x55aa327335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0212b3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32743350 0x55aa327335e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43850/0 (socket says 172.21.15.86:43850) 2024-04-16T05:40:51.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0212b3a640 1 -- 172.21.15.86:0/2551397999 learned_addr learned my addr 172.21.15.86:0/2551397999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0212b3a640 1 -- 172.21.15.86:0/2551397999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa3273a770 msgr2=0x55aa32740c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0212b3a640 1 --2- 172.21.15.86:0/2551397999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa3273a770 0x55aa32740c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0212b3a640 1 -- 172.21.15.86:0/2551397999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa32739e50 msgr2=0x55aa3274ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0212b3a640 1 --2- 172.21.15.86:0/2551397999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa32739e50 0x55aa3274ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0212b3a640 1 -- 172.21.15.86:0/2551397999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa32745d10 con 0x55aa32743350 2024-04-16T05:40:51.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f0212b3a640 1 --2- 172.21.15.86:0/2551397999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32743350 0x55aa327335e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0204010860 tx=0x7f020401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fed6852573dcd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f01f7fff640 1 -- 172.21.15.86:0/2551397999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020401c8a0 con 0x55aa32743350 2024-04-16T05:40:51.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f01f7fff640 1 -- 172.21.15.86:0/2551397999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020401f3f0 con 0x55aa32743350 2024-04-16T05:40:51.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.472+0000 7f01f7fff640 1 -- 172.21.15.86:0/2551397999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02040285e0 con 0x55aa32743350 2024-04-16T05:40:51.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 -- 172.21.15.86:0/2551397999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32743350 msgr2=0x55aa327335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 --2- 172.21.15.86:0/2551397999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32743350 0x55aa327335e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0204010860 tx=0x7f020401d000 comp rx=0 tx=0).stop 2024-04-16T05:40:51.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 -- 172.21.15.86:0/2551397999 shutdown_connections 2024-04-16T05:40:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 --2- 172.21.15.86:0/2551397999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32743350 0x55aa327335e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 --2- 172.21.15.86:0/2551397999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa3273a770 0x55aa32740c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 --2- 172.21.15.86:0/2551397999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa32739e50 0x55aa3274ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 -- 172.21.15.86:0/2551397999 >> 172.21.15.86:0/2551397999 conn(0x55aa32734580 msgr2=0x55aa32732ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 -- 172.21.15.86:0/2551397999 shutdown_connections 2024-04-16T05:40:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 -- 172.21.15.86:0/2551397999 wait complete. 2024-04-16T05:40:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 -- start start 2024-04-16T05:40:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 Processor -- start 2024-04-16T05:40:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa327484c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.473+0000 7f0217188740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 0x55aa32748a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0217188740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 0x55aa327b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0217188740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f020c1b08a0 con 0x55aa32739e50 2024-04-16T05:40:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0217188740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f020c1b01d0 con 0x55aa3273a770 2024-04-16T05:40:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0217188740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f020c1ad3f0 con 0x55aa32743350 2024-04-16T05:40:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0217188740 10 client.0 _ll_register_callbacks cb 0x7f0208004c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa327484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa327484c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43854/0 (socket says 172.21.15.86:43854) 2024-04-16T05:40:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212339640 1 -- 172.21.15.86:0/1654635120 learned_addr learned my addr 172.21.15.86:0/1654635120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:40:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212b3a640 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 0x55aa327b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212339640 1 -- 172.21.15.86:0/1654635120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 msgr2=0x55aa327b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212339640 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 0x55aa327b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212339640 1 -- 172.21.15.86:0/1654635120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 msgr2=0x55aa32748a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212339640 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 0x55aa32748a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212339640 1 -- 172.21.15.86:0/1654635120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa327343b0 con 0x55aa32739e50 2024-04-16T05:40:51.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0212339640 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa327484c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0208006010 tx=0x7f02080c7420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39b512a4571afbe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:51.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f01f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:40:51.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f01f5ffb640 1 -- 172.21.15.86:0/1654635120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080be400 con 0x55aa32739e50 2024-04-16T05:40:51.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f01f5ffb640 1 -- 172.21.15.86:0/1654635120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f02080f04a0 con 0x55aa32739e50 2024-04-16T05:40:51.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0217188740 1 -- 172.21.15.86:0/1654635120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f020401f3f0 con 0x55aa32739e50 2024-04-16T05:40:51.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f01f5ffb640 1 -- 172.21.15.86:0/1654635120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02080bf1a0 con 0x55aa32739e50 2024-04-16T05:40:51.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0217188740 10 client.17406 fetch_fsmap learned FSMap version 29 2024-04-16T05:40:51.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f0217188740 1 -- 172.21.15.86:0/1654635120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa327343b0 con 0x55aa32739e50 2024-04-16T05:40:51.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.474+0000 7f01f5ffb640 1 -- 172.21.15.86:0/1654635120 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020401f3f0 con 0x55aa32739e50 2024-04-16T05:40:51.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f01f5ffb640 1 -- 172.21.15.86:0/1654635120 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02080bdef0 con 0x55aa32739e50 2024-04-16T05:40:51.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f0217188740 10 client.17406 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:40:51.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f0217188740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:51.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f0217188740 20 client.17406 populate_metadata read hostname 'smithi086' 2024-04-16T05:40:51.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f0217188740 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55aa327b91c0 0x55aa327bb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:51.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f0217188740 4 client.17406 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:40:51.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f0217188740 1 -- 172.21.15.86:0/1654635120 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa32622560 con 0x55aa327b91c0 2024-04-16T05:40:51.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f0211b38640 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55aa327b91c0 0x55aa327bb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:51.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f01f5ffb640 10 client.17406 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:51.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.475+0000 7f0211b38640 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55aa327b91c0 0x55aa327bb5e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43bef24fcb320591 server_cookie=8366bf1b0f806122 in_seq=0 out_seq=0 2024-04-16T05:40:51.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f01f5ffb640 1 -- 172.21.15.86:0/1654635120 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55aa32622560 con 0x55aa327b91c0 2024-04-16T05:40:51.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f01f5ffb640 10 client.17406 handle_command_reply: tid=0 2024-04-16T05:40:51.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 client.17406 shutdown 2024-04-16T05:40:51.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 -- 172.21.15.86:0/1654635120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 msgr2=0x55aa327484c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa327484c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0208006010 tx=0x7f02080c7420 comp rx=0 tx=0).stop 2024-04-16T05:40:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 -- 172.21.15.86:0/1654635120 shutdown_connections 2024-04-16T05:40:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55aa327b91c0 0x55aa327bb5e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa32743350 0x55aa327b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f01f5ffb640 0 client.17406 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:40:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aa3273a770 0x55aa32748a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 --2- 172.21.15.86:0/1654635120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa32739e50 0x55aa327484c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 -- 172.21.15.86:0/1654635120 >> 172.21.15.86:0/1654635120 conn(0x55aa32734580 msgr2=0x55aa32733760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 -- 172.21.15.86:0/1654635120 shutdown_connections 2024-04-16T05:40:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 1 -- 172.21.15.86:0/1654635120 wait complete. 2024-04-16T05:40:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f0217188740 20 client.17406 trim_cache size 0 max 16384 2024-04-16T05:40:51.486 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:40:51.487 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:40:51.487 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:40:51.487 INFO:teuthology.orchestra.run.smithi086.stdout: "8e133513-755f-4c9c-afb7-7ab9d1943ef2": { 2024-04-16T05:40:51.487 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:40:51.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "8e133513-755f-4c9c-afb7-7ab9d1943ef2", 2024-04-16T05:40:51.487 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:40:51.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:40:51.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:40:51.488 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:40:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 -- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f01f80a3d20 msgr2=0x7f01f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f01f80a3d20 0x7f01f80a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0204005c40 tx=0x7f02040129b0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 -- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 msgr2=0x7f020c19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 0x7f020c19e570 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0208009a20 tx=0x7f020802df60 comp rx=0 tx=0).stop 2024-04-16T05:40:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 -- 172.21.15.86:0/3814800978 shutdown_connections 2024-04-16T05:40:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f01f80a3d20 0x7f01f80a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f020c10c5c0 0x7f020c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103ba0 0x7f020c19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 --2- 172.21.15.86:0/3814800978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103280 0x7f020c19e570 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.476+0000 7f021459d640 1 -- 172.21.15.86:0/3814800978 >> 172.21.15.86:0/3814800978 conn(0x7f020c078000 msgr2=0x7f020c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.477+0000 7f021459d640 1 -- 172.21.15.86:0/3814800978 shutdown_connections 2024-04-16T05:40:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:40:51.477+0000 7f021459d640 1 -- 172.21.15.86:0/3814800978 wait complete. 2024-04-16T05:40:51.490 INFO:tasks.cephfs.filesystem:scrub status for tag:8e133513-755f-4c9c-afb7-7ab9d1943ef2 - {'path': '/', 'tag': '8e133513-755f-4c9c-afb7-7ab9d1943ef2', 'options': 'recursive,force'} 2024-04-16T05:40:52.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:52 smithi194 ceph-mon[35865]: pgmap v170: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:52.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:52 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:40:52.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:52 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:52.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:52 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:40:52.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:52 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:52 smithi086 ceph-mon[32028]: pgmap v170: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:52 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:40:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:52 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:52 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:40:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:52 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:52 smithi196 ceph-mon[36373]: pgmap v170: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:40:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:52 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:40:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:52 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:52 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:40:52.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:52 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:54.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:54 smithi194 ceph-mon[35865]: pgmap v171: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:54.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:54 smithi086 ceph-mon[32028]: pgmap v171: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:54.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:54 smithi196 ceph-mon[36373]: pgmap v171: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:40:55.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:55 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:40:55.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:55 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:40:55.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:55 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:40:56.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:56 smithi194 ceph-mon[35865]: pgmap v172: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:56.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:56 smithi086 ceph-mon[32028]: pgmap v172: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:56 smithi196 ceph-mon[36373]: pgmap v172: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:58.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:40:58 smithi194 ceph-mon[35865]: pgmap v173: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:58.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:40:58 smithi086 ceph-mon[32028]: pgmap v173: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:40:58.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:40:58 smithi196 ceph-mon[36373]: pgmap v173: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:00.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:00 smithi194 ceph-mon[35865]: pgmap v174: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:00.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:00 smithi086 ceph-mon[32028]: pgmap v174: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:00.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:00 smithi196 ceph-mon[36373]: pgmap v174: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:02.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:02 smithi194 ceph-mon[35865]: pgmap v175: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:02.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:02 smithi086 ceph-mon[32028]: pgmap v175: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:02 smithi196 ceph-mon[36373]: pgmap v175: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:04.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:04 smithi194 ceph-mon[35865]: pgmap v176: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-16T05:41:04.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:04 smithi086 ceph-mon[32028]: pgmap v176: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-16T05:41:04.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:04 smithi196 ceph-mon[36373]: pgmap v176: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-16T05:41:06.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:06 smithi194 ceph-mon[35865]: pgmap v177: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:06.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:06 smithi086 ceph-mon[32028]: pgmap v177: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:06 smithi196 ceph-mon[36373]: pgmap v177: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:08.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:08 smithi194 ceph-mon[35865]: pgmap v178: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:08.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:08 smithi086 ceph-mon[32028]: pgmap v178: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:08.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:08 smithi196 ceph-mon[36373]: pgmap v178: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:10.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:10 smithi194 ceph-mon[35865]: pgmap v179: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:41:10.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:10 smithi086 ceph-mon[32028]: pgmap v179: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:41:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:10 smithi196 ceph-mon[36373]: pgmap v179: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:41:12.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:12 smithi194 ceph-mon[35865]: pgmap v180: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:12.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:12 smithi086 ceph-mon[32028]: pgmap v180: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:12.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:12 smithi196 ceph-mon[36373]: pgmap v180: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:14.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:14 smithi194 ceph-mon[35865]: pgmap v181: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:41:14.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:14 smithi086 ceph-mon[32028]: pgmap v181: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:41:14.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:14 smithi196 ceph-mon[36373]: pgmap v181: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-16T05:41:16.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:16 smithi194 ceph-mon[35865]: pgmap v182: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:41:16.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:16 smithi086 ceph-mon[32028]: pgmap v182: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:41:16.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:16 smithi196 ceph-mon[36373]: pgmap v182: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:41:18.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:18 smithi194 ceph-mon[35865]: pgmap v183: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:41:18.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:18 smithi086 ceph-mon[32028]: pgmap v183: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:41:18.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:18 smithi196 ceph-mon[36373]: pgmap v183: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:41:20.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:20 smithi194 ceph-mon[35865]: pgmap v184: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-16T05:41:20.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:20 smithi086 ceph-mon[32028]: pgmap v184: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-16T05:41:20.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:20 smithi196 ceph-mon[36373]: pgmap v184: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-16T05:41:21.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:41:21.492 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:41:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.612+0000 7f0add87b640 1 -- 172.21.15.86:0/4108941309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 msgr2=0x7f0ad8107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.612+0000 7f0add87b640 1 --2- 172.21.15.86:0/4108941309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 0x7f0ad8107420 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009930 tx=0x7f0ac802bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:21.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.612+0000 7f0add87b640 1 -- 172.21.15.86:0/4108941309 shutdown_connections 2024-04-16T05:41:21.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.612+0000 7f0add87b640 1 --2- 172.21.15.86:0/4108941309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad806afe0 0x7f0ad810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.612+0000 7f0add87b640 1 --2- 172.21.15.86:0/4108941309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad806a620 0x7f0ad806aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.612+0000 7f0add87b640 1 --2- 172.21.15.86:0/4108941309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 0x7f0ad8107420 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.612+0000 7f0add87b640 1 -- 172.21.15.86:0/4108941309 >> 172.21.15.86:0/4108941309 conn(0x7f0ad8077ee0 msgr2=0x7f0ad80782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:21.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.612+0000 7f0add87b640 1 -- 172.21.15.86:0/4108941309 shutdown_connections 2024-04-16T05:41:21.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.612+0000 7f0add87b640 1 -- 172.21.15.86:0/4108941309 wait complete. 2024-04-16T05:41:21.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.613+0000 7f0add87b640 1 Processor -- start 2024-04-16T05:41:21.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.613+0000 7f0add87b640 1 -- start start 2024-04-16T05:41:21.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.613+0000 7f0add87b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad806a620 0x7f0ad819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.613+0000 7f0add87b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad806afe0 0x7f0ad819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.613+0000 7f0add87b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 0x7f0ad81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.613+0000 7f0add87b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81124e0 con 0x7f0ad8107040 2024-04-16T05:41:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.613+0000 7f0add87b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8112360 con 0x7f0ad806a620 2024-04-16T05:41:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.613+0000 7f0add87b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8112660 con 0x7f0ad806afe0 2024-04-16T05:41:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.613+0000 7f0ad77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 0x7f0ad81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 0x7f0ad81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56580/0 (socket says 172.21.15.86:56580) 2024-04-16T05:41:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad77fe640 1 -- 172.21.15.86:0/3436323271 learned_addr learned my addr 172.21.15.86:0/3436323271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad806afe0 0x7f0ad819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad806a620 0x7f0ad819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad77fe640 1 -- 172.21.15.86:0/3436323271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad806afe0 msgr2=0x7f0ad819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad77fe640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad806afe0 0x7f0ad819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad77fe640 1 -- 172.21.15.86:0/3436323271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad806a620 msgr2=0x7f0ad819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad77fe640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad806a620 0x7f0ad819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad77fe640 1 -- 172.21.15.86:0/3436323271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad8198fb0 con 0x7f0ad8107040 2024-04-16T05:41:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad806a620 0x7f0ad819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0ad77fe640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 0x7f0ad81987e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b970 tx=0x7f0ad000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0adc879640 1 -- 172.21.15.86:0/3436323271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0019070 con 0x7f0ad8107040 2024-04-16T05:41:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0adc879640 1 -- 172.21.15.86:0/3436323271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad000cb40 con 0x7f0ad8107040 2024-04-16T05:41:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0add87b640 1 -- 172.21.15.86:0/3436323271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad81992a0 con 0x7f0ad8107040 2024-04-16T05:41:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0adc879640 1 -- 172.21.15.86:0/3436323271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0007650 con 0x7f0ad8107040 2024-04-16T05:41:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.614+0000 7f0add87b640 1 -- 172.21.15.86:0/3436323271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad81a54c0 con 0x7f0ad8107040 2024-04-16T05:41:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.616+0000 7f0adc879640 1 -- 172.21.15.86:0/3436323271 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ad001c070 con 0x7f0ad8107040 2024-04-16T05:41:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ae046c740 1 Processor -- start 2024-04-16T05:41:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ae046c740 1 -- start start 2024-04-16T05:41:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0adc879640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0ac40a3d20 0x7f0ac40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdce50 0x556d01cdd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 0x556d01ce8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ae046c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d01c42180 con 0x556d01cdd770 2024-04-16T05:41:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ae046c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556d01bdf360 con 0x556d01cdce50 2024-04-16T05:41:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ae046c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d01c3e140 con 0x556d01ce6350 2024-04-16T05:41:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0ac40a3d20 0x7f0ac40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0adc879640 1 -- 172.21.15.86:0/3436323271 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0ad00d0040 con 0x7f0ad8107040 2024-04-16T05:41:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ace7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ace7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56584/0 (socket says 172.21.15.86:56584) 2024-04-16T05:41:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ace7fc640 1 -- 172.21.15.86:0/97148210 learned_addr learned my addr 172.21.15.86:0/97148210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ad77fe640 1 --2- 172.21.15.86:0/97148210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 0x556d01ce8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.617+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/97148210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdce50 0x556d01cdd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0ace7fc640 1 -- 172.21.15.86:0/97148210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 msgr2=0x556d01ce8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/97148210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 0x556d01ce8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0ace7fc640 1 -- 172.21.15.86:0/97148210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdce50 msgr2=0x556d01cdd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/97148210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdce50 0x556d01cdd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0ace7fc640 1 -- 172.21.15.86:0/97148210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d01a1e7d0 con 0x556d01cdd770 2024-04-16T05:41:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0ac40a3d20 0x7f0ac40a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000afd0 tx=0x7f0ac000a910 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/97148210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce5e10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0006aa0 tx=0x7f0ac0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f6ead2418bf897 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0acdffb640 1 -- 172.21.15.86:0/97148210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0025070 con 0x556d01cdd770 2024-04-16T05:41:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0acdffb640 1 -- 172.21.15.86:0/97148210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ac001c070 con 0x556d01cdd770 2024-04-16T05:41:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.618+0000 7f0acdffb640 1 -- 172.21.15.86:0/97148210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0020680 con 0x556d01cdd770 2024-04-16T05:41:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 -- 172.21.15.86:0/97148210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 msgr2=0x556d01ce5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 --2- 172.21.15.86:0/97148210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce5e10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0006aa0 tx=0x7f0ac0019000 comp rx=0 tx=0).stop 2024-04-16T05:41:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 -- 172.21.15.86:0/97148210 shutdown_connections 2024-04-16T05:41:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 --2- 172.21.15.86:0/97148210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 0x556d01ce8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 --2- 172.21.15.86:0/97148210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce5e10 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 --2- 172.21.15.86:0/97148210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdce50 0x556d01cdd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 -- 172.21.15.86:0/97148210 >> 172.21.15.86:0/97148210 conn(0x556d01cd7580 msgr2=0x556d01cd7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 -- 172.21.15.86:0/97148210 shutdown_connections 2024-04-16T05:41:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 -- 172.21.15.86:0/97148210 wait complete. 2024-04-16T05:41:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 -- start start 2024-04-16T05:41:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.619+0000 7f0ae046c740 1 Processor -- start 2024-04-16T05:41:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdce50 0x556d01ce33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdd770 0x556d01ce3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 0x556d01ce3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ae046c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d01ce90e0 con 0x556d01cdce50 2024-04-16T05:41:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ae046c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556d01bfb2c0 con 0x556d01cdd770 2024-04-16T05:41:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ae046c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d01ce9260 con 0x556d01ce6350 2024-04-16T05:41:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdce50 0x556d01ce33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ae046c740 10 client.0 _ll_register_callbacks cb 0x556d01be2090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdce50 0x556d01ce33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56588/0 (socket says 172.21.15.86:56588) 2024-04-16T05:41:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad6ffd640 1 -- 172.21.15.86:0/2017015948 learned_addr learned my addr 172.21.15.86:0/2017015948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdd770 0x556d01ce3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad77fe640 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 0x556d01ce3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad6ffd640 1 -- 172.21.15.86:0/2017015948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 msgr2=0x556d01ce3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 0x556d01ce3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad6ffd640 1 -- 172.21.15.86:0/2017015948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdd770 msgr2=0x556d01ce3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:21.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdd770 0x556d01ce3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad6ffd640 1 -- 172.21.15.86:0/2017015948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d019543f0 con 0x556d01cdce50 2024-04-16T05:41:21.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdce50 0x556d01ce33c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009900 tx=0x7f0ac8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c93cb3b03ea20 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:21.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ab6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:21.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/2017015948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8043030 con 0x556d01cdce50 2024-04-16T05:41:21.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/2017015948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0ac8005000 con 0x556d01cdce50 2024-04-16T05:41:21.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.620+0000 7f0ae046c740 1 -- 172.21.15.86:0/2017015948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac001c070 con 0x556d01cdce50 2024-04-16T05:41:21.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/2017015948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ac8050050 con 0x556d01cdce50 2024-04-16T05:41:21.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ae046c740 10 client.17418 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:21.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ae046c740 1 -- 172.21.15.86:0/2017015948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d019140f0 con 0x556d01cdce50 2024-04-16T05:41:21.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/2017015948 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ac001c070 con 0x556d01cdce50 2024-04-16T05:41:21.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/2017015948 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ac80035f0 con 0x556d01cdce50 2024-04-16T05:41:21.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ae046c740 10 client.17418 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:21.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ae046c740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:21.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ae046c740 20 client.17418 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ae046c740 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556d01cf66c0 0x556d01cf8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ae046c740 4 client.17418 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ae046c740 1 -- 172.21.15.86:0/2017015948 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d01c4fba0 con 0x556d01cf66c0 2024-04-16T05:41:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.621+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556d01cf66c0 0x556d01cf8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.622+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556d01cf66c0 0x556d01cf8b60 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db15d126fe58db7d server_cookie=a1a3fa382273af2e in_seq=0 out_seq=0 2024-04-16T05:41:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.622+0000 7f0ab6ffd640 10 client.17418 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.622+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/2017015948 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d01c4fba0 con 0x556d01cf66c0 2024-04-16T05:41:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.622+0000 7f0ab6ffd640 10 client.17418 handle_command_reply: tid=0 2024-04-16T05:41:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.622+0000 7f0ae046c740 1 client.17418 shutdown 2024-04-16T05:41:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 -- 172.21.15.86:0/2017015948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdce50 msgr2=0x556d01ce33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdce50 0x556d01ce33c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009900 tx=0x7f0ac8004270 comp rx=0 tx=0).stop 2024-04-16T05:41:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 -- 172.21.15.86:0/2017015948 shutdown_connections 2024-04-16T05:41:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556d01cf66c0 0x556d01cf8b60 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01ce6350 0x556d01ce3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01cdd770 0x556d01ce3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 --2- 172.21.15.86:0/2017015948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdce50 0x556d01ce33c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ab6ffd640 0 client.17418 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 -- 172.21.15.86:0/2017015948 >> 172.21.15.86:0/2017015948 conn(0x556d01cd7580 msgr2=0x556d01cd92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 -- 172.21.15.86:0/2017015948 shutdown_connections 2024-04-16T05:41:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 1 -- 172.21.15.86:0/2017015948 wait complete. 2024-04-16T05:41:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.623+0000 7f0ae046c740 20 client.17418 trim_cache size 0 max 16384 2024-04-16T05:41:21.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ae046c740 1 Processor -- start 2024-04-16T05:41:21.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ae046c740 1 -- start start 2024-04-16T05:41:21.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cf1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01ce6350 0x556d01cfe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ae046c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81af410 con 0x556d01cdd770 2024-04-16T05:41:21.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ae046c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81b0f50 con 0x556d01ce6350 2024-04-16T05:41:21.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ae046c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81afae0 con 0x556d01cdce50 2024-04-16T05:41:21.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ace7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ace7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56604/0 (socket says 172.21.15.86:56604) 2024-04-16T05:41:21.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.640+0000 7f0ace7fc640 1 -- 172.21.15.86:0/4186308587 learned_addr learned my addr 172.21.15.86:0/4186308587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:21.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cf1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0ad77fe640 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01ce6350 0x556d01cfe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0ace7fc640 1 -- 172.21.15.86:0/4186308587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 msgr2=0x556d01cf1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:21.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cf1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0ace7fc640 1 -- 172.21.15.86:0/4186308587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01ce6350 msgr2=0x556d01cfe480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:21.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01ce6350 0x556d01cfe480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0ace7fc640 1 -- 172.21.15.86:0/4186308587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d01cd8150 con 0x556d01cdd770 2024-04-16T05:41:21.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cf1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce3c60 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0006a70 tx=0x7f0ac0018d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242b639e28854e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0acd7fa640 1 -- 172.21.15.86:0/4186308587 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0034070 con 0x556d01cdd770 2024-04-16T05:41:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0acd7fa640 1 -- 172.21.15.86:0/4186308587 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ac001c050 con 0x556d01cdd770 2024-04-16T05:41:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.641+0000 7f0acd7fa640 1 -- 172.21.15.86:0/4186308587 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0025040 con 0x556d01cdd770 2024-04-16T05:41:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 -- 172.21.15.86:0/4186308587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 msgr2=0x556d01ce3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce3c60 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0006a70 tx=0x7f0ac0018d90 comp rx=0 tx=0).stop 2024-04-16T05:41:21.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 -- 172.21.15.86:0/4186308587 shutdown_connections 2024-04-16T05:41:21.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01ce6350 0x556d01cfe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01ce3c60 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 --2- 172.21.15.86:0/4186308587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cf1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 -- 172.21.15.86:0/4186308587 >> 172.21.15.86:0/4186308587 conn(0x556d01cd7580 msgr2=0x556d01cd9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:21.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 -- 172.21.15.86:0/4186308587 shutdown_connections 2024-04-16T05:41:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 -- 172.21.15.86:0/4186308587 wait complete. 2024-04-16T05:41:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 -- start start 2024-04-16T05:41:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 Processor -- start 2024-04-16T05:41:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01cec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.642+0000 7f0ae046c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01ce6350 0x556d01cec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ae046c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81b0880 con 0x556d01cdd770 2024-04-16T05:41:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ae046c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81b01b0 con 0x556d01ce6350 2024-04-16T05:41:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ae046c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81ad3d0 con 0x556d01cdce50 2024-04-16T05:41:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ae046c740 10 client.0 _ll_register_callbacks cb 0x7f0ac802c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ace7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01cec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ace7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01cec4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56612/0 (socket says 172.21.15.86:56612) 2024-04-16T05:41:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ace7fc640 1 -- 172.21.15.86:0/808121183 learned_addr learned my addr 172.21.15.86:0/808121183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ace7fc640 1 -- 172.21.15.86:0/808121183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 msgr2=0x556d01cebf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cebf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ace7fc640 1 -- 172.21.15.86:0/808121183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01ce6350 msgr2=0x556d01cec9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01ce6350 0x556d01cec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ace7fc640 1 -- 172.21.15.86:0/808121183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d01d5bd80 con 0x556d01cdd770 2024-04-16T05:41:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ace7fc640 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01cec4b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0023040 tx=0x7f0ac0006bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3660a7dc2dd71651 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ab6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/808121183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0034070 con 0x556d01cdd770 2024-04-16T05:41:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/808121183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0ac0018e00 con 0x556d01cdd770 2024-04-16T05:41:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ae046c740 1 -- 172.21.15.86:0/808121183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac001c050 con 0x556d01cdd770 2024-04-16T05:41:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/808121183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ac0025040 con 0x556d01cdd770 2024-04-16T05:41:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ae046c740 10 client.17424 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ae046c740 1 -- 172.21.15.86:0/808121183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d01d5cfe0 con 0x556d01cdd770 2024-04-16T05:41:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.643+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/808121183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ac001c050 con 0x556d01cdd770 2024-04-16T05:41:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/808121183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ac0015040 con 0x556d01cdd770 2024-04-16T05:41:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ae046c740 10 client.17424 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ae046c740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ae046c740 20 client.17424 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ae046c740 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556d01d5d440 0x556d01d5f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ae046c740 4 client.17424 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:41:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ae046c740 1 -- 172.21.15.86:0/808121183 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d01c49a10 con 0x556d01d5d440 2024-04-16T05:41:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556d01d5d440 0x556d01d5f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ab6ffd640 10 client.17424 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ad6ffd640 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556d01d5d440 0x556d01d5f860 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506c4e972b8755bd server_cookie=4b265b3d59c82bf6 in_seq=0 out_seq=0 2024-04-16T05:41:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ab6ffd640 1 -- 172.21.15.86:0/808121183 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556d01c49a10 con 0x556d01d5d440 2024-04-16T05:41:21.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ab6ffd640 10 client.17424 handle_command_reply: tid=0 2024-04-16T05:41:21.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.644+0000 7f0ae046c740 1 client.17424 shutdown 2024-04-16T05:41:21.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 -- 172.21.15.86:0/808121183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 msgr2=0x556d01cec4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01cec4b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0023040 tx=0x7f0ac0006bc0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 -- 172.21.15.86:0/808121183 shutdown_connections 2024-04-16T05:41:21.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556d01d5d440 0x556d01d5f860 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d01ce6350 0x556d01cec9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ab6ffd640 0 client.17424 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d01cdd770 0x556d01cec4b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 --2- 172.21.15.86:0/808121183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d01cdce50 0x556d01cebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 -- 172.21.15.86:0/808121183 >> 172.21.15.86:0/808121183 conn(0x556d01cd7580 msgr2=0x556d01cd5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 -- 172.21.15.86:0/808121183 shutdown_connections 2024-04-16T05:41:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 1 -- 172.21.15.86:0/808121183 wait complete. 2024-04-16T05:41:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0ae046c740 20 client.17424 trim_cache size 0 max 16384 2024-04-16T05:41:21.667 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:41:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:41:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:41:21.668 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:41:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 -- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0ac40a3d20 msgr2=0x7f0ac40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0ac40a3d20 0x7f0ac40a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000afd0 tx=0x7f0ac000a910 comp rx=0 tx=0).stop 2024-04-16T05:41:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 -- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 msgr2=0x7f0ad81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 0x7f0ad81987e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b970 tx=0x7f0ad000be30 comp rx=0 tx=0).stop 2024-04-16T05:41:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 -- 172.21.15.86:0/3436323271 shutdown_connections 2024-04-16T05:41:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad8107040 0x7f0ad81987e0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad806afe0 0x7f0ad819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0ac40a3d20 0x7f0ac40a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 --2- 172.21.15.86:0/3436323271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad806a620 0x7f0ad819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 -- 172.21.15.86:0/3436323271 >> 172.21.15.86:0/3436323271 conn(0x7f0ad8077ee0 msgr2=0x7f0ad80fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 -- 172.21.15.86:0/3436323271 shutdown_connections 2024-04-16T05:41:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.645+0000 7f0add87b640 1 -- 172.21.15.86:0/3436323271 wait complete. 2024-04-16T05:41:21.671 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:41:21.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:41:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 -- 172.21.15.86:0/938040800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c103b80 msgr2=0x7f354c10c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 --2- 172.21.15.86:0/938040800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c103b80 0x7f354c10c060 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f3548009900 tx=0x7f354802bcd0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 -- 172.21.15.86:0/938040800 shutdown_connections 2024-04-16T05:41:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 --2- 172.21.15.86:0/938040800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f354c10c630 0x7f354c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 --2- 172.21.15.86:0/938040800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c103b80 0x7f354c10c060 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 --2- 172.21.15.86:0/938040800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c103260 0x7f354c103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 -- 172.21.15.86:0/938040800 >> 172.21.15.86:0/938040800 conn(0x7f354c077280 msgr2=0x7f354c077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 -- 172.21.15.86:0/938040800 shutdown_connections 2024-04-16T05:41:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 -- 172.21.15.86:0/938040800 wait complete. 2024-04-16T05:41:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.777+0000 7f35547ba640 1 Processor -- start 2024-04-16T05:41:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f35547ba640 1 -- start start 2024-04-16T05:41:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f35547ba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f354c103260 0x7f354c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f35547ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c103b80 0x7f354c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f35547ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c10c630 0x7f354c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f35547ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f354c112420 con 0x7f354c10c630 2024-04-16T05:41:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f35547ba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1122a0 con 0x7f354c103260 2024-04-16T05:41:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f35547ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1125a0 con 0x7f354c103b80 2024-04-16T05:41:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552d57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c10c630 0x7f354c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552d57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c10c630 0x7f354c198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56632/0 (socket says 172.21.15.86:56632) 2024-04-16T05:41:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552d57640 1 -- 172.21.15.86:0/3397333939 learned_addr learned my addr 172.21.15.86:0/3397333939 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3551d55640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c103b80 0x7f354c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552556640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f354c103260 0x7f354c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552d57640 1 -- 172.21.15.86:0/3397333939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c103b80 msgr2=0x7f354c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552d57640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c103b80 0x7f354c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552d57640 1 -- 172.21.15.86:0/3397333939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f354c103260 msgr2=0x7f354c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552d57640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f354c103260 0x7f354c19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552d57640 1 -- 172.21.15.86:0/3397333939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c198f50 con 0x7f354c10c630 2024-04-16T05:41:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3551d55640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c103b80 0x7f354c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f3552d57640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c10c630 0x7f354c198780 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f354400e9d0 tx=0x7f354400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f353f7fe640 1 -- 172.21.15.86:0/3397333939 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354400cd80 con 0x7f354c10c630 2024-04-16T05:41:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f353f7fe640 1 -- 172.21.15.86:0/3397333939 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3544005020 con 0x7f354c10c630 2024-04-16T05:41:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c199240 con 0x7f354c10c630 2024-04-16T05:41:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f353f7fe640 1 -- 172.21.15.86:0/3397333939 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35440116d0 con 0x7f354c10c630 2024-04-16T05:41:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.778+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c1a5460 con 0x7f354c10c630 2024-04-16T05:41:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.779+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354c06aec0 con 0x7f354c10c630 2024-04-16T05:41:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.781+0000 7f353f7fe640 1 -- 172.21.15.86:0/3397333939 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f35440051d0 con 0x7f354c10c630 2024-04-16T05:41:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.782+0000 7f353f7fe640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f35380a3da0 0x7f35380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.782+0000 7f3552556640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f35380a3da0 0x7f35380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.782+0000 7f353f7fe640 1 -- 172.21.15.86:0/3397333939 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f35440d1140 con 0x7f354c10c630 2024-04-16T05:41:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.782+0000 7f3552556640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f35380a3da0 0x7f35380a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f354000a810 tx=0x7f3540008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:21.785+0000 7f353f7fe640 1 -- 172.21.15.86:0/3397333939 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3544016030 con 0x7f354c10c630 2024-04-16T05:41:22.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.001+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f354c02d620 con 0x7f354c10c630 2024-04-16T05:41:22.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.002+0000 7f353f7fe640 1 -- 172.21.15.86:0/3397333939 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f354409d4f0 con 0x7f354c10c630 2024-04-16T05:41:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:41:22.001 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:41:22.001 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:41:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f35380a3da0 msgr2=0x7f35380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f35380a3da0 0x7f35380a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f354000a810 tx=0x7f3540008040 comp rx=0 tx=0).stop 2024-04-16T05:41:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c10c630 msgr2=0x7f354c198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c10c630 0x7f354c198780 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f354400e9d0 tx=0x7f354400ee90 comp rx=0 tx=0).stop 2024-04-16T05:41:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 shutdown_connections 2024-04-16T05:41:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354c10c630 0x7f354c198780 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c103b80 0x7f354c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f35380a3da0 0x7f35380a6260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 --2- 172.21.15.86:0/3397333939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f354c103260 0x7f354c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 >> 172.21.15.86:0/3397333939 conn(0x7f354c077280 msgr2=0x7f354c0fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 shutdown_connections 2024-04-16T05:41:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.004+0000 7f35547ba640 1 -- 172.21.15.86:0/3397333939 wait complete. 2024-04-16T05:41:22.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:41:22.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 -- 172.21.15.86:0/466701892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc106200 msgr2=0x7fa0dc10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 --2- 172.21.15.86:0/466701892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc106200 0x7fa0dc10ddd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009ca0 tx=0x7fa0d402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 -- 172.21.15.86:0/466701892 shutdown_connections 2024-04-16T05:41:22.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 --2- 172.21.15.86:0/466701892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc106200 0x7fa0dc10ddd0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 --2- 172.21.15.86:0/466701892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc101ae0 0x7fa0dc105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 --2- 172.21.15.86:0/466701892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0dc101130 0x7fa0dc101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 -- 172.21.15.86:0/466701892 >> 172.21.15.86:0/466701892 conn(0x7fa0dc0780b0 msgr2=0x7fa0dc0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 -- 172.21.15.86:0/466701892 shutdown_connections 2024-04-16T05:41:22.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 -- 172.21.15.86:0/466701892 wait complete. 2024-04-16T05:41:22.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 Processor -- start 2024-04-16T05:41:22.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.122+0000 7fa0e287b640 1 -- start start 2024-04-16T05:41:22.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0e287b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc101130 0x7fa0dc19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0e287b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0dc106200 0x7fa0dc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0e287b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc198670 0x7fa0dc198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0e287b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1123d0 con 0x7fa0dc101130 2024-04-16T05:41:22.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0e287b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc112250 con 0x7fa0dc106200 2024-04-16T05:41:22.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0e287b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc112550 con 0x7fa0dc198670 2024-04-16T05:41:22.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0dbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc101130 0x7fa0dc19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0dbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc101130 0x7fa0dc19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56660/0 (socket says 172.21.15.86:56660) 2024-04-16T05:41:22.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0dbfff640 1 -- 172.21.15.86:0/1542078542 learned_addr learned my addr 172.21.15.86:0/1542078542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0dc106200 0x7fa0dc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0dbfff640 1 -- 172.21.15.86:0/1542078542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc198670 msgr2=0x7fa0dc198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0dbfff640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc198670 0x7fa0dc198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0dbfff640 1 -- 172.21.15.86:0/1542078542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0dc106200 msgr2=0x7fa0dc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0dbfff640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0dc106200 0x7fa0dc19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0dbfff640 1 -- 172.21.15.86:0/1542078542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc199270 con 0x7fa0dc101130 2024-04-16T05:41:22.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0dbfff640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc101130 0x7fa0dc19e5b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00d950 tx=0x7fa0cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.123+0000 7fa0d97fa640 1 -- 172.21.15.86:0/1542078542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc017070 con 0x7fa0dc101130 2024-04-16T05:41:22.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.124+0000 7fa0d97fa640 1 -- 172.21.15.86:0/1542078542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0cc009c30 con 0x7fa0dc101130 2024-04-16T05:41:22.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.124+0000 7fa0e287b640 1 -- 172.21.15.86:0/1542078542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc1a5350 con 0x7fa0dc101130 2024-04-16T05:41:22.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.124+0000 7fa0d97fa640 1 -- 172.21.15.86:0/1542078542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00f6b0 con 0x7fa0dc101130 2024-04-16T05:41:22.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.124+0000 7fa0e287b640 1 -- 172.21.15.86:0/1542078542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc1a5750 con 0x7fa0dc101130 2024-04-16T05:41:22.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.126+0000 7fa0e546c740 1 Processor -- start 2024-04-16T05:41:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0d97fa640 1 -- 172.21.15.86:0/1542078542 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0cc0109d0 con 0x7fa0dc101130 2024-04-16T05:41:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e546c740 1 -- start start 2024-04-16T05:41:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 0x56428ad33e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad36750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e546c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56428ac900e0 con 0x56428ad2ae60 2024-04-16T05:41:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e546c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56428ac2d520 con 0x56428ad34360 2024-04-16T05:41:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e546c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56428ac8c0a0 con 0x56428ad2b780 2024-04-16T05:41:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e0e18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e0e18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56676/0 (socket says 172.21.15.86:56676) 2024-04-16T05:41:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e0e18640 1 -- 172.21.15.86:0/3308761128 learned_addr learned my addr 172.21.15.86:0/3308761128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad36750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0dbfff640 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 0x56428ad33e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e0e18640 1 -- 172.21.15.86:0/3308761128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 msgr2=0x56428ad33e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 0x56428ad33e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e0e18640 1 -- 172.21.15.86:0/3308761128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 msgr2=0x56428ad36750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad36750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e0e18640 1 -- 172.21.15.86:0/3308761128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56428ac558d0 con 0x56428ad2ae60 2024-04-16T05:41:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0d97fa640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa0c40a3d20 0x7fa0c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0dbfff640 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 0x56428ad33e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2b240 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa0d402c700 tx=0x7fa0d4032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc56c486d7a9b8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0dbfff640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa0c40a3d20 0x7fa0c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.127+0000 7fa0d97fa640 1 -- 172.21.15.86:0/1542078542 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa0cc0d0850 con 0x7fa0dc101130 2024-04-16T05:41:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0c27fc640 1 -- 172.21.15.86:0/3308761128 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4041020 con 0x56428ad2ae60 2024-04-16T05:41:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0c27fc640 1 -- 172.21.15.86:0/3308761128 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d4032c90 con 0x56428ad2ae60 2024-04-16T05:41:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0c27fc640 1 -- 172.21.15.86:0/3308761128 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4039040 con 0x56428ad2ae60 2024-04-16T05:41:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0e546c740 1 -- 172.21.15.86:0/3308761128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 msgr2=0x56428ad2b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0dbfff640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa0c40a3d20 0x7fa0c40a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x56428ad2a930 tx=0x7fa0cc00bf60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2b240 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa0d402c700 tx=0x7fa0d4032340 comp rx=0 tx=0).stop 2024-04-16T05:41:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0e546c740 1 -- 172.21.15.86:0/3308761128 shutdown_connections 2024-04-16T05:41:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad36750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 0x56428ad33e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3308761128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2b240 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0e546c740 1 -- 172.21.15.86:0/3308761128 >> 172.21.15.86:0/3308761128 conn(0x56428ad25590 msgr2=0x56428ad25970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0e546c740 1 -- 172.21.15.86:0/3308761128 shutdown_connections 2024-04-16T05:41:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.128+0000 7fa0e546c740 1 -- 172.21.15.86:0/3308761128 wait complete. 2024-04-16T05:41:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e546c740 1 -- start start 2024-04-16T05:41:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e546c740 1 Processor -- start 2024-04-16T05:41:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 0x56428ad31480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad31f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e546c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56428ad36fa0 con 0x56428ad2b780 2024-04-16T05:41:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e546c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56428ac494d0 con 0x56428ad34360 2024-04-16T05:41:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e546c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56428ad37120 con 0x56428ad2ae60 2024-04-16T05:41:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e546c740 10 client.0 _ll_register_callbacks cb 0x56428ac2e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad319c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56678/0 (socket says 172.21.15.86:56678) 2024-04-16T05:41:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0db7fe640 1 -- 172.21.15.86:0/3910946236 learned_addr learned my addr 172.21.15.86:0/3910946236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad31f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 0x56428ad31480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.129+0000 7fa0db7fe640 1 -- 172.21.15.86:0/3910946236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 msgr2=0x56428ad31480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 0x56428ad31480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0db7fe640 1 -- 172.21.15.86:0/3910946236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 msgr2=0x56428ad31f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad31f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0db7fe640 1 -- 172.21.15.86:0/3910946236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56428a9a23f0 con 0x56428ad2b780 2024-04-16T05:41:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 0x56428ad31480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad319c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000efd0 tx=0x7fa0d0010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b340f7d48f9c25fc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3910946236 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0004220 con 0x56428ad2b780 2024-04-16T05:41:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3910946236 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa0d00043c0 con 0x56428ad2b780 2024-04-16T05:41:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e546c740 1 -- 172.21.15.86:0/3910946236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d4032c90 con 0x56428ad2b780 2024-04-16T05:41:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3910946236 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0d000c910 con 0x56428ad2b780 2024-04-16T05:41:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e546c740 10 client.17442 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e546c740 1 -- 172.21.15.86:0/3910946236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56428aa6c7d0 con 0x56428ad2b780 2024-04-16T05:41:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3910946236 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d4032c90 con 0x56428ad2b780 2024-04-16T05:41:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3910946236 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa0d0011440 con 0x56428ad2b780 2024-04-16T05:41:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e546c740 10 client.17442 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e546c740 10 client.17442 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:41:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e546c740 20 client.17442 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56428ad443f0 0x56428ad46850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e546c740 4 client.17442 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.130+0000 7fa0e546c740 1 -- 172.21.15.86:0/3910946236 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56428ac9db00 con 0x56428ad443f0 2024-04-16T05:41:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.131+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56428ad443f0 0x56428ad46850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.131+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56428ad443f0 0x56428ad46850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0b5ad49ec346c60 server_cookie=bc8706521746d9ec in_seq=0 out_seq=0 2024-04-16T05:41:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.131+0000 7fa0bb7fe640 10 client.17442 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:41:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3910946236 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56428ac9db00 con 0x56428ad443f0 2024-04-16T05:41:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0bb7fe640 10 client.17442 handle_command_reply: tid=0 2024-04-16T05:41:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 client.17442 shutdown 2024-04-16T05:41:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 -- 172.21.15.86:0/3910946236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 msgr2=0x56428ad319c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad319c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000efd0 tx=0x7fa0d0010b60 comp rx=0 tx=0).stop 2024-04-16T05:41:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 -- 172.21.15.86:0/3910946236 shutdown_connections 2024-04-16T05:41:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56428ad443f0 0x56428ad46850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad31f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad319c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 --2- 172.21.15.86:0/3910946236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 0x56428ad31480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0bb7fe640 0 client.17442 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:41:22.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 -- 172.21.15.86:0/3910946236 >> 172.21.15.86:0/3910946236 conn(0x56428ad25590 msgr2=0x56428ad272e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 -- 172.21.15.86:0/3910946236 shutdown_connections 2024-04-16T05:41:22.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 1 -- 172.21.15.86:0/3910946236 wait complete. 2024-04-16T05:41:22.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.132+0000 7fa0e546c740 20 client.17442 trim_cache size 0 max 16384 2024-04-16T05:41:22.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.145+0000 7fa0e546c740 1 Processor -- start 2024-04-16T05:41:22.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.145+0000 7fa0e546c740 1 -- start start 2024-04-16T05:41:22.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.145+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 0x56428ad4c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.145+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.145+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad23ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.145+0000 7fa0e546c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1af490 con 0x56428ad2b780 2024-04-16T05:41:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.145+0000 7fa0e546c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b0fd0 con 0x56428ad34360 2024-04-16T05:41:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.145+0000 7fa0e546c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1afb60 con 0x56428ad2ae60 2024-04-16T05:41:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad4c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56686/0 (socket says 172.21.15.86:56686) 2024-04-16T05:41:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0db7fe640 1 -- 172.21.15.86:0/2543005556 learned_addr learned my addr 172.21.15.86:0/2543005556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/2543005556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 0x56428ad4c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0db7fe640 1 -- 172.21.15.86:0/2543005556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 msgr2=0x56428ad4c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/2543005556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 0x56428ad4c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0db7fe640 1 -- 172.21.15.86:0/2543005556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 msgr2=0x56428ad23ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/2543005556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad23ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0db7fe640 1 -- 172.21.15.86:0/2543005556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56428ad2b5a0 con 0x56428ad2b780 2024-04-16T05:41:22.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/2543005556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad4c6a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0001eb0 tx=0x7fa0d0011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222cdc9ed5ddc96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0c1ffb640 1 -- 172.21.15.86:0/2543005556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d002d070 con 0x56428ad2b780 2024-04-16T05:41:22.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0c1ffb640 1 -- 172.21.15.86:0/2543005556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d0011640 con 0x56428ad2b780 2024-04-16T05:41:22.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.146+0000 7fa0c1ffb640 1 -- 172.21.15.86:0/2543005556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d00044a0 con 0x56428ad2b780 2024-04-16T05:41:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 -- 172.21.15.86:0/2543005556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 msgr2=0x56428ad4c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 --2- 172.21.15.86:0/2543005556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad4c6a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0001eb0 tx=0x7fa0d0011ea0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 -- 172.21.15.86:0/2543005556 shutdown_connections 2024-04-16T05:41:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 --2- 172.21.15.86:0/2543005556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad23ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 --2- 172.21.15.86:0/2543005556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2b780 0x56428ad4c6a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 --2- 172.21.15.86:0/2543005556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2ae60 0x56428ad4c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 -- 172.21.15.86:0/2543005556 >> 172.21.15.86:0/2543005556 conn(0x56428ad25590 msgr2=0x56428ad2fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 -- 172.21.15.86:0/2543005556 shutdown_connections 2024-04-16T05:41:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 -- 172.21.15.86:0/2543005556 wait complete. 2024-04-16T05:41:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 -- start start 2024-04-16T05:41:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 Processor -- start 2024-04-16T05:41:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.147+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 0x56428ad2e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e546c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad2ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e546c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b0900 con 0x56428ad2ae60 2024-04-16T05:41:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e546c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b0230 con 0x56428ad34360 2024-04-16T05:41:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e546c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1ad450 con 0x56428ad2b780 2024-04-16T05:41:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e546c740 10 client.0 _ll_register_callbacks cb 0x7fa0d000e350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e0e18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e0e18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56692/0 (socket says 172.21.15.86:56692) 2024-04-16T05:41:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e0e18640 1 -- 172.21.15.86:0/897381346 learned_addr learned my addr 172.21.15.86:0/897381346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e0e18640 1 -- 172.21.15.86:0/897381346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 msgr2=0x56428ad2e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 0x56428ad2e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e0e18640 1 -- 172.21.15.86:0/897381346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 msgr2=0x56428ad2ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad2ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e0e18640 1 -- 172.21.15.86:0/897381346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56428adadde0 con 0x56428ad2ae60 2024-04-16T05:41:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e0e18640 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2dfc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009ca0 tx=0x7fa0d4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89272352ac8dc6c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/897381346 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4057060 con 0x56428ad2ae60 2024-04-16T05:41:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/897381346 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa0d4032590 con 0x56428ad2ae60 2024-04-16T05:41:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e546c740 1 -- 172.21.15.86:0/897381346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d0011640 con 0x56428ad2ae60 2024-04-16T05:41:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/897381346 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0d405e050 con 0x56428ad2ae60 2024-04-16T05:41:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e546c740 10 client.17448 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0e546c740 1 -- 172.21.15.86:0/897381346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56428adaf020 con 0x56428ad2ae60 2024-04-16T05:41:22.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.148+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/897381346 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d0011640 con 0x56428ad2ae60 2024-04-16T05:41:22.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/897381346 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa0d403c030 con 0x56428ad2ae60 2024-04-16T05:41:22.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0e546c740 10 client.17448 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0e546c740 10 client.17448 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:41:22.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0e546c740 20 client.17448 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0e546c740 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56428adaf310 0x56428adb1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0e546c740 4 client.17448 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0e546c740 1 -- 172.21.15.86:0/897381346 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56428ac97970 con 0x56428adaf310 2024-04-16T05:41:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56428adaf310 0x56428adb1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0bb7fe640 10 client.17448 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:41:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0db7fe640 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56428adaf310 0x56428adb1700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec753354532bec7 server_cookie=8cffc9b02627b90c in_seq=0 out_seq=0 2024-04-16T05:41:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/897381346 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56428ac97970 con 0x56428adaf310 2024-04-16T05:41:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0bb7fe640 10 client.17448 handle_command_reply: tid=0 2024-04-16T05:41:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.149+0000 7fa0e546c740 1 client.17448 shutdown 2024-04-16T05:41:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 -- 172.21.15.86:0/897381346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 msgr2=0x56428ad2dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2dfc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009ca0 tx=0x7fa0d4003d40 comp rx=0 tx=0).stop 2024-04-16T05:41:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 -- 172.21.15.86:0/897381346 shutdown_connections 2024-04-16T05:41:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56428adaf310 0x56428adb1700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56428ad34360 0x56428ad2ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56428ad2b780 0x56428ad2e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 --2- 172.21.15.86:0/897381346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56428ad2ae60 0x56428ad2dfc0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0bb7fe640 0 client.17448 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:41:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 -- 172.21.15.86:0/897381346 >> 172.21.15.86:0/897381346 conn(0x56428ad25590 msgr2=0x56428ad31420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 -- 172.21.15.86:0/897381346 shutdown_connections 2024-04-16T05:41:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 1 -- 172.21.15.86:0/897381346 wait complete. 2024-04-16T05:41:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e546c740 20 client.17448 trim_cache size 0 max 16384 2024-04-16T05:41:22.160 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:22.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 -- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa0c40a3d20 msgr2=0x7fa0c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa0c40a3d20 0x7fa0c40a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x56428ad2a930 tx=0x7fa0cc00bf60 comp rx=0 tx=0).stop 2024-04-16T05:41:22.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 -- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc101130 msgr2=0x7fa0dc19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc101130 0x7fa0dc19e5b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00d950 tx=0x7fa0cc00de10 comp rx=0 tx=0).stop 2024-04-16T05:41:22.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 -- 172.21.15.86:0/1542078542 shutdown_connections 2024-04-16T05:41:22.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc198670 0x7fa0dc198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa0c40a3d20 0x7fa0c40a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0dc106200 0x7fa0dc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 --2- 172.21.15.86:0/1542078542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc101130 0x7fa0dc19e5b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 -- 172.21.15.86:0/1542078542 >> 172.21.15.86:0/1542078542 conn(0x7fa0dc0780b0 msgr2=0x7fa0dc0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 -- 172.21.15.86:0/1542078542 shutdown_connections 2024-04-16T05:41:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.150+0000 7fa0e287b640 1 -- 172.21.15.86:0/1542078542 wait complete. 2024-04-16T05:41:22.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:41:22.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.269+0000 7f75a7c4a640 1 -- 172.21.15.86:0/1964808530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0102250 msgr2=0x7f75a0102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.269+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/1964808530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0102250 0x7f75a0102630 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7590009900 tx=0x7f759002bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 -- 172.21.15.86:0/1964808530 shutdown_connections 2024-04-16T05:41:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/1964808530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75a0107320 0x7f75a010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/1964808530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a0102c00 0x7f75a0106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/1964808530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0102250 0x7f75a0102630 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 -- 172.21.15.86:0/1964808530 >> 172.21.15.86:0/1964808530 conn(0x7f75a0079490 msgr2=0x7f75a00798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 -- 172.21.15.86:0/1964808530 shutdown_connections 2024-04-16T05:41:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 -- 172.21.15.86:0/1964808530 wait complete. 2024-04-16T05:41:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 Processor -- start 2024-04-16T05:41:22.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 -- start start 2024-04-16T05:41:22.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75a0102250 0x7f75a019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0107320 0x7f75a019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a01985d0 0x7f75a0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01123b0 con 0x7f75a0107320 2024-04-16T05:41:22.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0112230 con 0x7f75a0102250 2024-04-16T05:41:22.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a7c4a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0112530 con 0x7f75a01985d0 2024-04-16T05:41:22.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.270+0000 7f75a51e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0107320 0x7f75a019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a51e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0107320 0x7f75a019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56714/0 (socket says 172.21.15.86:56714) 2024-04-16T05:41:22.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a51e5640 1 -- 172.21.15.86:0/3956941672 learned_addr learned my addr 172.21.15.86:0/3956941672 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a61e7640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a01985d0 0x7f75a0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a59e6640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75a0102250 0x7f75a019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a51e5640 1 -- 172.21.15.86:0/3956941672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a01985d0 msgr2=0x7f75a0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a51e5640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a01985d0 0x7f75a0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a51e5640 1 -- 172.21.15.86:0/3956941672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75a0102250 msgr2=0x7f75a019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a51e5640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75a0102250 0x7f75a019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a51e5640 1 -- 172.21.15.86:0/3956941672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a01990f0 con 0x7f75a0107320 2024-04-16T05:41:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a59e6640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75a0102250 0x7f75a019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a51e5640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0107320 0x7f75a019e960 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f759400c970 tx=0x7f759400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f759effd640 1 -- 172.21.15.86:0/3956941672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594014070 con 0x7f75a0107320 2024-04-16T05:41:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f759effd640 1 -- 172.21.15.86:0/3956941672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7594004440 con 0x7f75a0107320 2024-04-16T05:41:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a7c4a640 1 -- 172.21.15.86:0/3956941672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a01993e0 con 0x7f75a0107320 2024-04-16T05:41:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f759effd640 1 -- 172.21.15.86:0/3956941672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759400f450 con 0x7f75a0107320 2024-04-16T05:41:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.271+0000 7f75a7c4a640 1 -- 172.21.15.86:0/3956941672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a01a5630 con 0x7f75a0107320 2024-04-16T05:41:22.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.273+0000 7f759effd640 1 -- 172.21.15.86:0/3956941672 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7594017070 con 0x7f75a0107320 2024-04-16T05:41:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75aa834740 1 Processor -- start 2024-04-16T05:41:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75aa834740 1 -- start start 2024-04-16T05:41:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f759effd640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f75880a3d20 0x7f75880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75aa834740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd2cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75aa834740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 0x55783dd3a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75aa834740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd3cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75aa834740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55783dc920e0 con 0x55783dd3a770 2024-04-16T05:41:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75aa834740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55783dc2f520 con 0x55783dd2d4b0 2024-04-16T05:41:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75aa834740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55783dc8e0a0 con 0x55783dd2cb90 2024-04-16T05:41:22.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f759effd640 1 -- 172.21.15.86:0/3956941672 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f75940d0a70 con 0x7f75a0107320 2024-04-16T05:41:22.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75a61e7640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f75880a3d20 0x7f75880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.274+0000 7f75a61e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd3cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a61e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd3cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56726/0 (socket says 172.21.15.86:56726) 2024-04-16T05:41:22.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a61e7640 1 -- 172.21.15.86:0/951191587 learned_addr learned my addr 172.21.15.86:0/951191587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a59e6640 1 --2- 172.21.15.86:0/951191587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd2cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a51e5640 1 --2- 172.21.15.86:0/951191587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 0x55783dd3a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a61e7640 1 -- 172.21.15.86:0/951191587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 msgr2=0x55783dd2cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a61e7640 1 --2- 172.21.15.86:0/951191587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd2cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a61e7640 1 -- 172.21.15.86:0/951191587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 msgr2=0x55783dd3a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a61e7640 1 --2- 172.21.15.86:0/951191587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 0x55783dd3a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a61e7640 1 -- 172.21.15.86:0/951191587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783dc578d0 con 0x55783dd3a770 2024-04-16T05:41:22.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a61e7640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f75880a3d20 0x7f75880a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f759800afd0 tx=0x7f759800ac60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f75a61e7640 1 --2- 172.21.15.86:0/951191587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd3cb60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f759800fbb0 tx=0x7f7598019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4525c6f251f416c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f7587fff640 1 -- 172.21.15.86:0/951191587 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759800dde0 con 0x55783dd3a770 2024-04-16T05:41:22.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f7587fff640 1 -- 172.21.15.86:0/951191587 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7598011470 con 0x55783dd3a770 2024-04-16T05:41:22.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.275+0000 7f7587fff640 1 -- 172.21.15.86:0/951191587 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759801e640 con 0x55783dd3a770 2024-04-16T05:41:22.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 -- 172.21.15.86:0/951191587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 msgr2=0x55783dd3cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 --2- 172.21.15.86:0/951191587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd3cb60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f759800fbb0 tx=0x7f7598019000 comp rx=0 tx=0).stop 2024-04-16T05:41:22.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 -- 172.21.15.86:0/951191587 shutdown_connections 2024-04-16T05:41:22.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 --2- 172.21.15.86:0/951191587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd3cb60 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 --2- 172.21.15.86:0/951191587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 0x55783dd3a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 --2- 172.21.15.86:0/951191587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd2cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 -- 172.21.15.86:0/951191587 >> 172.21.15.86:0/951191587 conn(0x55783dd27590 msgr2=0x55783dd27970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 -- 172.21.15.86:0/951191587 shutdown_connections 2024-04-16T05:41:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 -- 172.21.15.86:0/951191587 wait complete. 2024-04-16T05:41:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 -- start start 2024-04-16T05:41:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.276+0000 7f75aa834740 1 Processor -- start 2024-04-16T05:41:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75aa834740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75aa834740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 0x55783dd37d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75aa834740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd38260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75aa834740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55783dd3d4d0 con 0x55783dd3a770 2024-04-16T05:41:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75aa834740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55783dc4b4d0 con 0x55783dd2d4b0 2024-04-16T05:41:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75aa834740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55783dd3d650 con 0x55783dd2cb90 2024-04-16T05:41:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75aa834740 10 client.0 _ll_register_callbacks cb 0x55783dc302c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a61e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd38260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a61e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd38260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56736/0 (socket says 172.21.15.86:56736) 2024-04-16T05:41:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a61e7640 1 -- 172.21.15.86:0/3787929399 learned_addr learned my addr 172.21.15.86:0/3787929399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a59e6640 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a51e5640 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 0x55783dd37d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a61e7640 1 -- 172.21.15.86:0/3787929399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 msgr2=0x55783dd377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a61e7640 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd377e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a61e7640 1 -- 172.21.15.86:0/3787929399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 msgr2=0x55783dd37d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a61e7640 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 0x55783dd37d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a61e7640 1 -- 172.21.15.86:0/3787929399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783d9a43f0 con 0x55783dd3a770 2024-04-16T05:41:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a59e6640 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75a61e7640 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd38260 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f759800a2d0 tx=0x7f759800a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddccae545a6315ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.277+0000 7f75857fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75857fa640 1 -- 172.21.15.86:0/3787929399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598034060 con 0x55783dd3a770 2024-04-16T05:41:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75857fa640 1 -- 172.21.15.86:0/3787929399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f75980121b0 con 0x55783dd3a770 2024-04-16T05:41:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75aa834740 1 -- 172.21.15.86:0/3787929399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7598011470 con 0x55783dd3a770 2024-04-16T05:41:22.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75857fa640 1 -- 172.21.15.86:0/3787929399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f759803b050 con 0x55783dd3a770 2024-04-16T05:41:22.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75aa834740 10 client.17460 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75aa834740 1 -- 172.21.15.86:0/3787929399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55783da6e7d0 con 0x55783dd3a770 2024-04-16T05:41:22.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75857fa640 1 -- 172.21.15.86:0/3787929399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7598011470 con 0x55783dd3a770 2024-04-16T05:41:22.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75857fa640 1 -- 172.21.15.86:0/3787929399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f75980118f0 con 0x55783dd3a770 2024-04-16T05:41:22.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75aa834740 10 client.17460 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75aa834740 10 client.17460 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:41:22.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75aa834740 20 client.17460 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75aa834740 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55783dd4a6a0 0x55783dd4ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75aa834740 4 client.17460 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:22.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75aa834740 1 -- 172.21.15.86:0/3787929399 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55783dc9fb00 con 0x55783dd4a6a0 2024-04-16T05:41:22.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.278+0000 7f75a59e6640 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55783dd4a6a0 0x55783dd4ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.279+0000 7f75857fa640 10 client.17460 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:41:22.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.279+0000 7f75a59e6640 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55783dd4a6a0 0x55783dd4ca90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1537d449b3f9a18 server_cookie=c39d5b4c9137e2c1 in_seq=0 out_seq=0 2024-04-16T05:41:22.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75857fa640 1 -- 172.21.15.86:0/3787929399 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55783dc9fb00 con 0x55783dd4a6a0 2024-04-16T05:41:22.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75857fa640 10 client.17460 handle_command_reply: tid=0 2024-04-16T05:41:22.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 client.17460 shutdown 2024-04-16T05:41:22.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 -- 172.21.15.86:0/3787929399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 msgr2=0x55783dd38260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd38260 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f759800a2d0 tx=0x7f759800a630 comp rx=0 tx=0).stop 2024-04-16T05:41:22.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 -- 172.21.15.86:0/3787929399 shutdown_connections 2024-04-16T05:41:22.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd38260 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2d4b0 0x55783dd37d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55783dd4a6a0 0x55783dd4ca90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 --2- 172.21.15.86:0/3787929399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75857fa640 0 client.17460 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:41:22.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 -- 172.21.15.86:0/3787929399 >> 172.21.15.86:0/3787929399 conn(0x55783dd27590 msgr2=0x55783dd292e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 -- 172.21.15.86:0/3787929399 shutdown_connections 2024-04-16T05:41:22.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 1 -- 172.21.15.86:0/3787929399 wait complete. 2024-04-16T05:41:22.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.280+0000 7f75aa834740 20 client.17460 trim_cache size 0 max 16384 2024-04-16T05:41:22.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75aa834740 1 Processor -- start 2024-04-16T05:41:22.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75aa834740 1 -- start start 2024-04-16T05:41:22.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75aa834740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2cb90 0x55783dd46420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75aa834740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2d4b0 0x55783dd29270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75aa834740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd36fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75aa834740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01ad240 con 0x55783dd3a770 2024-04-16T05:41:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75aa834740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01aed80 con 0x55783dd2cb90 2024-04-16T05:41:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75aa834740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01ad910 con 0x55783dd2d4b0 2024-04-16T05:41:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75a61e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd36fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75a61e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd36fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56742/0 (socket says 172.21.15.86:56742) 2024-04-16T05:41:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.293+0000 7f75a61e7640 1 -- 172.21.15.86:0/4211241828 learned_addr learned my addr 172.21.15.86:0/4211241828 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75a51e5640 1 --2- 172.21.15.86:0/4211241828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2d4b0 0x55783dd29270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75a61e7640 1 -- 172.21.15.86:0/4211241828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2d4b0 msgr2=0x55783dd29270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75a61e7640 1 --2- 172.21.15.86:0/4211241828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2d4b0 0x55783dd29270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75a61e7640 1 -- 172.21.15.86:0/4211241828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2cb90 msgr2=0x55783dd46420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75a61e7640 1 --2- 172.21.15.86:0/4211241828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2cb90 0x55783dd46420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75a61e7640 1 -- 172.21.15.86:0/4211241828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783dd22c30 con 0x55783dd3a770 2024-04-16T05:41:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75a51e5640 1 --2- 172.21.15.86:0/4211241828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2d4b0 0x55783dd29270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75a61e7640 1 --2- 172.21.15.86:0/4211241828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd36fd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7598012810 tx=0x7f7598012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650a13c05d06a562 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75877fe640 1 -- 172.21.15.86:0/4211241828 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598034050 con 0x55783dd3a770 2024-04-16T05:41:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75877fe640 1 -- 172.21.15.86:0/4211241828 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759800de60 con 0x55783dd3a770 2024-04-16T05:41:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75877fe640 1 -- 172.21.15.86:0/4211241828 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598012410 con 0x55783dd3a770 2024-04-16T05:41:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75aa834740 1 -- 172.21.15.86:0/4211241828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 msgr2=0x55783dd36fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75aa834740 1 --2- 172.21.15.86:0/4211241828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd36fd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7598012810 tx=0x7f7598012f50 comp rx=0 tx=0).stop 2024-04-16T05:41:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75aa834740 1 -- 172.21.15.86:0/4211241828 shutdown_connections 2024-04-16T05:41:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75aa834740 1 --2- 172.21.15.86:0/4211241828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd3a770 0x55783dd36fd0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75aa834740 1 --2- 172.21.15.86:0/4211241828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2d4b0 0x55783dd29270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75aa834740 1 --2- 172.21.15.86:0/4211241828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd2cb90 0x55783dd46420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.294+0000 7f75aa834740 1 -- 172.21.15.86:0/4211241828 >> 172.21.15.86:0/4211241828 conn(0x55783dd27590 msgr2=0x55783dd38f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 -- 172.21.15.86:0/4211241828 shutdown_connections 2024-04-16T05:41:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 -- 172.21.15.86:0/4211241828 wait complete. 2024-04-16T05:41:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 -- start start 2024-04-16T05:41:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 Processor -- start 2024-04-16T05:41:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd42d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd2d4b0 0x55783dd3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd3a770 0x55783dd3e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01ae6b0 con 0x55783dd2d4b0 2024-04-16T05:41:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01adfe0 con 0x55783dd3a770 2024-04-16T05:41:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01ab200 con 0x55783dd2cb90 2024-04-16T05:41:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75aa834740 10 client.0 _ll_register_callbacks cb 0x7f7590009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75a51e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd2d4b0 0x55783dd3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75a51e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd2d4b0 0x55783dd3e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56744/0 (socket says 172.21.15.86:56744) 2024-04-16T05:41:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.295+0000 7f75a51e5640 1 -- 172.21.15.86:0/2692638916 learned_addr learned my addr 172.21.15.86:0/2692638916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75a61e7640 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd3a770 0x55783dd3e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75a59e6640 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd42d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75a51e5640 1 -- 172.21.15.86:0/2692638916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 msgr2=0x55783dd42d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75a51e5640 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd42d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75a51e5640 1 -- 172.21.15.86:0/2692638916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd3a770 msgr2=0x55783dd3e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75a51e5640 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd3a770 0x55783dd3e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75a51e5640 1 -- 172.21.15.86:0/2692638916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783dd3ee70 con 0x55783dd2d4b0 2024-04-16T05:41:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75a59e6640 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd42d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75a51e5640 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd2d4b0 0x55783dd3e0e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f759400c940 tx=0x7f759409cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb8296c887039b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75857fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75857fa640 1 -- 172.21.15.86:0/2692638916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75940a5310 con 0x55783dd2d4b0 2024-04-16T05:41:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75857fa640 1 -- 172.21.15.86:0/2692638916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f75940a4e80 con 0x55783dd2d4b0 2024-04-16T05:41:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75aa834740 1 -- 172.21.15.86:0/2692638916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f759800de60 con 0x55783dd2d4b0 2024-04-16T05:41:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75857fa640 1 -- 172.21.15.86:0/2692638916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f75940a5080 con 0x55783dd2d4b0 2024-04-16T05:41:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75aa834740 10 client.17466 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.296+0000 7f75aa834740 1 -- 172.21.15.86:0/2692638916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55783ddb0960 con 0x55783dd2d4b0 2024-04-16T05:41:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75857fa640 1 -- 172.21.15.86:0/2692638916 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759800de60 con 0x55783dd2d4b0 2024-04-16T05:41:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75857fa640 1 -- 172.21.15.86:0/2692638916 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f75940b9030 con 0x55783dd2d4b0 2024-04-16T05:41:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75aa834740 10 client.17466 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75aa834740 10 client.17466 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:41:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75aa834740 20 client.17466 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75aa834740 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55783ddb10b0 0x55783ddb34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75aa834740 4 client.17466 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75aa834740 1 -- 172.21.15.86:0/2692638916 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55783dc99970 con 0x55783ddb10b0 2024-04-16T05:41:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75a59e6640 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55783ddb10b0 0x55783ddb34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75857fa640 10 client.17466 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:41:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.297+0000 7f75a59e6640 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55783ddb10b0 0x55783ddb34d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69911dafc62517c2 server_cookie=74ce61afdfa70cef in_seq=0 out_seq=0 2024-04-16T05:41:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75857fa640 1 -- 172.21.15.86:0/2692638916 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55783dc99970 con 0x55783ddb10b0 2024-04-16T05:41:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75857fa640 10 client.17466 handle_command_reply: tid=0 2024-04-16T05:41:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 client.17466 shutdown 2024-04-16T05:41:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 -- 172.21.15.86:0/2692638916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd2d4b0 msgr2=0x55783dd3e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd2d4b0 0x55783dd3e0e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f759400c940 tx=0x7f759409cee0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 -- 172.21.15.86:0/2692638916 shutdown_connections 2024-04-16T05:41:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55783dd3a770 0x55783dd3e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55783dd2d4b0 0x55783dd3e0e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55783ddb10b0 0x55783ddb34d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 --2- 172.21.15.86:0/2692638916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783dd2cb90 0x55783dd42d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75857fa640 0 client.17466 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:41:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 -- 172.21.15.86:0/2692638916 >> 172.21.15.86:0/2692638916 conn(0x55783dd27590 msgr2=0x55783dd37fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 -- 172.21.15.86:0/2692638916 shutdown_connections 2024-04-16T05:41:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 1 -- 172.21.15.86:0/2692638916 wait complete. 2024-04-16T05:41:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75aa834740 20 client.17466 trim_cache size 0 max 16384 2024-04-16T05:41:22.310 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 -- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f75880a3d20 msgr2=0x7f75880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f75880a3d20 0x7f75880a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f759800afd0 tx=0x7f759800ac60 comp rx=0 tx=0).stop 2024-04-16T05:41:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 -- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0107320 msgr2=0x7f75a019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0107320 0x7f75a019e960 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f759400c970 tx=0x7f759400ce30 comp rx=0 tx=0).stop 2024-04-16T05:41:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 -- 172.21.15.86:0/3956941672 shutdown_connections 2024-04-16T05:41:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a01985d0 0x7f75a0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0107320 0x7f75a019e960 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f75880a3d20 0x7f75880a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 --2- 172.21.15.86:0/3956941672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75a0102250 0x7f75a019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.298+0000 7f75a7c4a640 1 -- 172.21.15.86:0/3956941672 >> 172.21.15.86:0/3956941672 conn(0x7f75a0079490 msgr2=0x7f75a00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.299+0000 7f75a7c4a640 1 -- 172.21.15.86:0/3956941672 shutdown_connections 2024-04-16T05:41:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.299+0000 7f75a7c4a640 1 -- 172.21.15.86:0/3956941672 wait complete. 2024-04-16T05:41:22.312 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:41:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.422+0000 7fc1323d4640 1 -- 172.21.15.86:0/1981207878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c107040 msgr2=0x7fc12c107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.422+0000 7fc1323d4640 1 --2- 172.21.15.86:0/1981207878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c107040 0x7fc12c107420 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009900 tx=0x7fc11c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.422+0000 7fc1323d4640 1 -- 172.21.15.86:0/1981207878 shutdown_connections 2024-04-16T05:41:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.422+0000 7fc1323d4640 1 --2- 172.21.15.86:0/1981207878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc12c101bf0 0x7fc12c10dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.422+0000 7fc1323d4640 1 --2- 172.21.15.86:0/1981207878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c101250 0x7fc12c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.422+0000 7fc1323d4640 1 --2- 172.21.15.86:0/1981207878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c107040 0x7fc12c107420 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.422+0000 7fc1323d4640 1 -- 172.21.15.86:0/1981207878 >> 172.21.15.86:0/1981207878 conn(0x7fc12c076d00 msgr2=0x7fc12c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.422+0000 7fc1323d4640 1 -- 172.21.15.86:0/1981207878 shutdown_connections 2024-04-16T05:41:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.422+0000 7fc1323d4640 1 -- 172.21.15.86:0/1981207878 wait complete. 2024-04-16T05:41:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc1323d4640 1 Processor -- start 2024-04-16T05:41:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc1323d4640 1 -- start start 2024-04-16T05:41:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc1323d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc12c101250 0x7fc12c102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc1323d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c101bf0 0x7fc12c103120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc1323d4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c107040 0x7fc12c103660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc1323d4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1112f0 con 0x7fc12c101bf0 2024-04-16T05:41:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc1323d4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c111170 con 0x7fc12c107040 2024-04-16T05:41:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc1323d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c111470 con 0x7fc12c101250 2024-04-16T05:41:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc123fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c101bf0 0x7fc12c103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc123fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c101bf0 0x7fc12c103120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56758/0 (socket says 172.21.15.86:56758) 2024-04-16T05:41:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc123fff640 1 -- 172.21.15.86:0/3655641728 learned_addr learned my addr 172.21.15.86:0/3655641728 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc130971640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c107040 0x7fc12c103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc123fff640 1 -- 172.21.15.86:0/3655641728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc12c101250 msgr2=0x7fc12c102be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.423+0000 7fc12bfff640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc12c101250 0x7fc12c102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc123fff640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc12c101250 0x7fc12c102be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc123fff640 1 -- 172.21.15.86:0/3655641728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c107040 msgr2=0x7fc12c103660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc123fff640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c107040 0x7fc12c103660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc123fff640 1 -- 172.21.15.86:0/3655641728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c1a6de0 con 0x7fc12c101bf0 2024-04-16T05:41:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc130971640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c107040 0x7fc12c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc12bfff640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc12c101250 0x7fc12c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc123fff640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c101bf0 0x7fc12c103120 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c890 tx=0x7fc11400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc129ffb640 1 -- 172.21.15.86:0/3655641728 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114004220 con 0x7fc12c101bf0 2024-04-16T05:41:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc129ffb640 1 -- 172.21.15.86:0/3655641728 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1140043c0 con 0x7fc12c101bf0 2024-04-16T05:41:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc1323d4640 1 -- 172.21.15.86:0/3655641728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c1a70d0 con 0x7fc12c101bf0 2024-04-16T05:41:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc129ffb640 1 -- 172.21.15.86:0/3655641728 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140049e0 con 0x7fc12c101bf0 2024-04-16T05:41:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.424+0000 7fc1323d4640 1 -- 172.21.15.86:0/3655641728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c1a7620 con 0x7fc12c101bf0 2024-04-16T05:41:22.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.426+0000 7fc129ffb640 1 -- 172.21.15.86:0/3655641728 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc1140079b0 con 0x7fc12c101bf0 2024-04-16T05:41:22.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.426+0000 7fc134fc3740 1 Processor -- start 2024-04-16T05:41:22.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.426+0000 7fc134fc3740 1 -- start start 2024-04-16T05:41:22.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c87e60 0x559de7c88240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c88780 0x559de7c90e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c91360 0x559de7c93750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc134fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559de7bed0e0 con 0x559de7c88780 2024-04-16T05:41:22.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc134fc3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559de7b8a520 con 0x559de7c87e60 2024-04-16T05:41:22.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc134fc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559de7be90a0 con 0x559de7c91360 2024-04-16T05:41:22.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc12bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c88780 0x559de7c90e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc129ffb640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1180a3d20 0x7fc1180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc12bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c88780 0x559de7c90e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56774/0 (socket says 172.21.15.86:56774) 2024-04-16T05:41:22.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc12bfff640 1 -- 172.21.15.86:0/1385924476 learned_addr learned my addr 172.21.15.86:0/1385924476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc130971640 1 --2- 172.21.15.86:0/1385924476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c87e60 0x559de7c88240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc130971640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1180a3d20 0x7fc1180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc123fff640 1 --2- 172.21.15.86:0/1385924476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c91360 0x559de7c93750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc129ffb640 1 -- 172.21.15.86:0/3655641728 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc1140d0cc0 con 0x7fc12c101bf0 2024-04-16T05:41:22.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc12bfff640 1 -- 172.21.15.86:0/1385924476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c91360 msgr2=0x559de7c93750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc12bfff640 1 --2- 172.21.15.86:0/1385924476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c91360 0x559de7c93750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc12bfff640 1 -- 172.21.15.86:0/1385924476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c87e60 msgr2=0x559de7c88240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc12bfff640 1 --2- 172.21.15.86:0/1385924476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c87e60 0x559de7c88240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc12bfff640 1 -- 172.21.15.86:0/1385924476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de7bb28d0 con 0x559de7c88780 2024-04-16T05:41:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc12bfff640 1 --2- 172.21.15.86:0/1385924476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c88780 0x559de7c90e20 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc11c036040 tx=0x7fc11c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b4cdc1238e9732 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc130971640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1180a3d20 0x7fc1180a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc124009fd0 tx=0x7fc1240093c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.427+0000 7fc1227fc640 1 -- 172.21.15.86:0/1385924476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c02ca80 con 0x559de7c88780 2024-04-16T05:41:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc1227fc640 1 -- 172.21.15.86:0/1385924476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc11c02cc20 con 0x559de7c88780 2024-04-16T05:41:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc1227fc640 1 -- 172.21.15.86:0/1385924476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c004920 con 0x559de7c88780 2024-04-16T05:41:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 -- 172.21.15.86:0/1385924476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c88780 msgr2=0x559de7c90e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 --2- 172.21.15.86:0/1385924476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c88780 0x559de7c90e20 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc11c036040 tx=0x7fc11c02c680 comp rx=0 tx=0).stop 2024-04-16T05:41:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 -- 172.21.15.86:0/1385924476 shutdown_connections 2024-04-16T05:41:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 --2- 172.21.15.86:0/1385924476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c91360 0x559de7c93750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 --2- 172.21.15.86:0/1385924476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c88780 0x559de7c90e20 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 --2- 172.21.15.86:0/1385924476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c87e60 0x559de7c88240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 -- 172.21.15.86:0/1385924476 >> 172.21.15.86:0/1385924476 conn(0x559de7c82590 msgr2=0x559de7c82970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 -- 172.21.15.86:0/1385924476 shutdown_connections 2024-04-16T05:41:22.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 -- 172.21.15.86:0/1385924476 wait complete. 2024-04-16T05:41:22.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.428+0000 7fc134fc3740 1 -- start start 2024-04-16T05:41:22.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc134fc3740 1 Processor -- start 2024-04-16T05:41:22.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c87e60 0x559de7c8e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c88780 0x559de7c8e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c91360 0x559de7c8edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc134fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559de7c94000 con 0x559de7c87e60 2024-04-16T05:41:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc134fc3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559de7ba64d0 con 0x559de7c91360 2024-04-16T05:41:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc134fc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559de7c94180 con 0x559de7c88780 2024-04-16T05:41:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc134fc3740 10 client.0 _ll_register_callbacks cb 0x559de7b8b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc12bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c87e60 0x559de7c8e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc12bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c87e60 0x559de7c8e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56782/0 (socket says 172.21.15.86:56782) 2024-04-16T05:41:22.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc12bfff640 1 -- 172.21.15.86:0/2896782926 learned_addr learned my addr 172.21.15.86:0/2896782926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc130971640 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c91360 0x559de7c8edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc12bfff640 1 -- 172.21.15.86:0/2896782926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c88780 msgr2=0x559de7c8e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc12bfff640 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c88780 0x559de7c8e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc12bfff640 1 -- 172.21.15.86:0/2896782926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c91360 msgr2=0x559de7c8edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc12bfff640 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c91360 0x559de7c8edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc12bfff640 1 -- 172.21.15.86:0/2896782926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de78ff3f0 con 0x559de7c87e60 2024-04-16T05:41:22.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.429+0000 7fc12bfff640 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c87e60 0x559de7c8e370 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc11c02bc70 tx=0x7fc11c035680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44eebb7996f2aaed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc10b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc10b7fe640 1 -- 172.21.15.86:0/2896782926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c04d060 con 0x559de7c87e60 2024-04-16T05:41:22.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc10b7fe640 1 -- 172.21.15.86:0/2896782926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc11c004440 con 0x559de7c87e60 2024-04-16T05:41:22.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc134fc3740 1 -- 172.21.15.86:0/2896782926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc11c02cc20 con 0x559de7c87e60 2024-04-16T05:41:22.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc10b7fe640 1 -- 172.21.15.86:0/2896782926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc11c054050 con 0x559de7c87e60 2024-04-16T05:41:22.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc134fc3740 10 client.17478 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc134fc3740 1 -- 172.21.15.86:0/2896782926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559de79c97d0 con 0x559de7c87e60 2024-04-16T05:41:22.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc10b7fe640 1 -- 172.21.15.86:0/2896782926 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc11c02cc20 con 0x559de7c87e60 2024-04-16T05:41:22.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc10b7fe640 1 -- 172.21.15.86:0/2896782926 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc11c003c90 con 0x559de7c87e60 2024-04-16T05:41:22.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc134fc3740 10 client.17478 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc134fc3740 10 client.17478 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:41:22.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc134fc3740 20 client.17478 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc134fc3740 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x559de7ca11f0 0x559de7ca35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc134fc3740 4 client.17478 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:22.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.430+0000 7fc134fc3740 1 -- 172.21.15.86:0/2896782926 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559de7bfab00 con 0x559de7ca11f0 2024-04-16T05:41:22.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.431+0000 7fc123fff640 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x559de7ca11f0 0x559de7ca35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.431+0000 7fc123fff640 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x559de7ca11f0 0x559de7ca35e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47b1be88ade23d18 server_cookie=6b39c0908c3d5180 in_seq=0 out_seq=0 2024-04-16T05:41:22.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.431+0000 7fc10b7fe640 10 client.17478 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:41:22.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc10b7fe640 1 -- 172.21.15.86:0/2896782926 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559de7bfab00 con 0x559de7ca11f0 2024-04-16T05:41:22.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc10b7fe640 10 client.17478 handle_command_reply: tid=0 2024-04-16T05:41:22.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 client.17478 shutdown 2024-04-16T05:41:22.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 -- 172.21.15.86:0/2896782926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c87e60 msgr2=0x559de7c8e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c87e60 0x559de7c8e370 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc11c02bc70 tx=0x7fc11c035680 comp rx=0 tx=0).stop 2024-04-16T05:41:22.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 -- 172.21.15.86:0/2896782926 shutdown_connections 2024-04-16T05:41:22.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c91360 0x559de7c8edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c88780 0x559de7c8e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x559de7ca11f0 0x559de7ca35e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 --2- 172.21.15.86:0/2896782926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c87e60 0x559de7c8e370 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 -- 172.21.15.86:0/2896782926 >> 172.21.15.86:0/2896782926 conn(0x559de7c82590 msgr2=0x559de7c842e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc10b7fe640 0 client.17478 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:41:22.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 -- 172.21.15.86:0/2896782926 shutdown_connections 2024-04-16T05:41:22.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 1 -- 172.21.15.86:0/2896782926 wait complete. 2024-04-16T05:41:22.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.432+0000 7fc134fc3740 20 client.17478 trim_cache size 0 max 16384 2024-04-16T05:41:22.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.445+0000 7fc134fc3740 1 Processor -- start 2024-04-16T05:41:22.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.445+0000 7fc134fc3740 1 -- start start 2024-04-16T05:41:22.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.445+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7c90aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.445+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 0x559de7c9cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.445+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.445+0000 7fc134fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1b1680 con 0x559de7c91360 2024-04-16T05:41:22.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.445+0000 7fc134fc3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1b31c0 con 0x559de7c88780 2024-04-16T05:41:22.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.445+0000 7fc134fc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1b1d50 con 0x559de7c87e60 2024-04-16T05:41:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc130971640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc130971640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56790/0 (socket says 172.21.15.86:56790) 2024-04-16T05:41:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc130971640 1 -- 172.21.15.86:0/1755874021 learned_addr learned my addr 172.21.15.86:0/1755874021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc12bfff640 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7c90aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc130971640 1 -- 172.21.15.86:0/1755874021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 msgr2=0x559de7c90aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc130971640 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7c90aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc130971640 1 -- 172.21.15.86:0/1755874021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 msgr2=0x559de7c9cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc123fff640 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 0x559de7c9cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc130971640 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 0x559de7c9cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc130971640 1 -- 172.21.15.86:0/1755874021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de7c820c0 con 0x559de7c91360 2024-04-16T05:41:22.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc12bfff640 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7c90aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc123fff640 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 0x559de7c9cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc130971640 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca6220 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc124009dc0 tx=0x7fc1240097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e042ea589146bf41 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc121ffb640 1 -- 172.21.15.86:0/1755874021 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124004b20 con 0x559de7c91360 2024-04-16T05:41:22.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc121ffb640 1 -- 172.21.15.86:0/1755874021 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc124004cc0 con 0x559de7c91360 2024-04-16T05:41:22.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.446+0000 7fc121ffb640 1 -- 172.21.15.86:0/1755874021 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124005950 con 0x559de7c91360 2024-04-16T05:41:22.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 -- 172.21.15.86:0/1755874021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 msgr2=0x559de7ca6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca6220 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc124009dc0 tx=0x7fc1240097a0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 -- 172.21.15.86:0/1755874021 shutdown_connections 2024-04-16T05:41:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca6220 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 0x559de7c9cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 --2- 172.21.15.86:0/1755874021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7c90aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 -- 172.21.15.86:0/1755874021 >> 172.21.15.86:0/1755874021 conn(0x559de7c82590 msgr2=0x559de7c9c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 -- 172.21.15.86:0/1755874021 shutdown_connections 2024-04-16T05:41:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 -- 172.21.15.86:0/1755874021 wait complete. 2024-04-16T05:41:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 -- start start 2024-04-16T05:41:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.447+0000 7fc134fc3740 1 Processor -- start 2024-04-16T05:41:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7ca5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 0x559de7ca64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc134fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc134fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1b2af0 con 0x559de7c91360 2024-04-16T05:41:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc134fc3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1b2420 con 0x559de7c88780 2024-04-16T05:41:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc134fc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1af640 con 0x559de7c87e60 2024-04-16T05:41:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc134fc3740 10 client.0 _ll_register_callbacks cb 0x7fc12c07a330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc130971640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc130971640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56800/0 (socket says 172.21.15.86:56800) 2024-04-16T05:41:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc130971640 1 -- 172.21.15.86:0/80560071 learned_addr learned my addr 172.21.15.86:0/80560071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc123fff640 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 0x559de7ca64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc12bfff640 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7ca5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc130971640 1 -- 172.21.15.86:0/80560071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 msgr2=0x559de7ca5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc130971640 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7ca5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc130971640 1 -- 172.21.15.86:0/80560071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 msgr2=0x559de7ca64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc130971640 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 0x559de7ca64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc130971640 1 -- 172.21.15.86:0/80560071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de7ca3b80 con 0x559de7c91360 2024-04-16T05:41:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc12bfff640 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7ca5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc130971640 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca3440 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc124014570 tx=0x7fc124009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2729cb27350a59be server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc10b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc10b7fe640 1 -- 172.21.15.86:0/80560071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124038020 con 0x559de7c91360 2024-04-16T05:41:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc10b7fe640 1 -- 172.21.15.86:0/80560071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc124011e00 con 0x559de7c91360 2024-04-16T05:41:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.448+0000 7fc134fc3740 1 -- 172.21.15.86:0/80560071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc124004cc0 con 0x559de7c91360 2024-04-16T05:41:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc10b7fe640 1 -- 172.21.15.86:0/80560071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc124043050 con 0x559de7c91360 2024-04-16T05:41:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc134fc3740 10 client.17484 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc134fc3740 1 -- 172.21.15.86:0/80560071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559de7d0f690 con 0x559de7c91360 2024-04-16T05:41:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc10b7fe640 1 -- 172.21.15.86:0/80560071 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc124004cc0 con 0x559de7c91360 2024-04-16T05:41:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc10b7fe640 1 -- 172.21.15.86:0/80560071 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc124033050 con 0x559de7c91360 2024-04-16T05:41:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc134fc3740 10 client.17484 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc134fc3740 10 client.17484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:41:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc134fc3740 20 client.17484 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc134fc3740 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x559de7d0fda0 0x559de7d121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc134fc3740 4 client.17484 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc134fc3740 1 -- 172.21.15.86:0/80560071 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559de7bf4970 con 0x559de7d0fda0 2024-04-16T05:41:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.449+0000 7fc12bfff640 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x559de7d0fda0 0x559de7d121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc10b7fe640 10 client.17484 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:41:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc12bfff640 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x559de7d0fda0 0x559de7d121c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe9d97003dd6bef4 server_cookie=bb7662f3c4de4403 in_seq=0 out_seq=0 2024-04-16T05:41:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc10b7fe640 1 -- 172.21.15.86:0/80560071 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559de7bf4970 con 0x559de7d0fda0 2024-04-16T05:41:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc10b7fe640 10 client.17484 handle_command_reply: tid=0 2024-04-16T05:41:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 client.17484 shutdown 2024-04-16T05:41:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 -- 172.21.15.86:0/80560071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 msgr2=0x559de7ca3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca3440 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc124014570 tx=0x7fc124009a30 comp rx=0 tx=0).stop 2024-04-16T05:41:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 -- 172.21.15.86:0/80560071 shutdown_connections 2024-04-16T05:41:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x559de7d0fda0 0x559de7d121c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de7c91360 0x559de7ca3440 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559de7c88780 0x559de7ca64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc10b7fe640 0 client.17484 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:41:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 --2- 172.21.15.86:0/80560071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559de7c87e60 0x559de7ca5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 -- 172.21.15.86:0/80560071 >> 172.21.15.86:0/80560071 conn(0x559de7c82590 msgr2=0x559de7c9cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 -- 172.21.15.86:0/80560071 shutdown_connections 2024-04-16T05:41:22.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 1 -- 172.21.15.86:0/80560071 wait complete. 2024-04-16T05:41:22.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.450+0000 7fc134fc3740 20 client.17484 trim_cache size 0 max 16384 2024-04-16T05:41:22.461 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 -- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1180a3d20 msgr2=0x7fc1180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1180a3d20 0x7fc1180a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc124009fd0 tx=0x7fc1240093c0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 -- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c101bf0 msgr2=0x7fc12c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c101bf0 0x7fc12c103120 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c890 tx=0x7fc11400cd50 comp rx=0 tx=0).stop 2024-04-16T05:41:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 -- 172.21.15.86:0/3655641728 shutdown_connections 2024-04-16T05:41:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1180a3d20 0x7fc1180a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c107040 0x7fc12c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc12c101bf0 0x7fc12c103120 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 --2- 172.21.15.86:0/3655641728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc12c101250 0x7fc12c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 -- 172.21.15.86:0/3655641728 >> 172.21.15.86:0/3655641728 conn(0x7fc12c076d00 msgr2=0x7fc12c06a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 -- 172.21.15.86:0/3655641728 shutdown_connections 2024-04-16T05:41:22.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.451+0000 7fc1323d4640 1 -- 172.21.15.86:0/3655641728 wait complete. 2024-04-16T05:41:22.464 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:41:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.572+0000 7f830cca5640 1 -- 172.21.15.86:0/2003854091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 msgr2=0x7f830810e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.572+0000 7f830cca5640 1 --2- 172.21.15.86:0/2003854091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 0x7f830810e450 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009900 tx=0x7f82fc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.572+0000 7f830cca5640 1 -- 172.21.15.86:0/2003854091 shutdown_connections 2024-04-16T05:41:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.572+0000 7f830cca5640 1 --2- 172.21.15.86:0/2003854091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f830810e990 0x7f8308110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.572+0000 7f830cca5640 1 --2- 172.21.15.86:0/2003854091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 0x7f830810e450 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.572+0000 7f830cca5640 1 --2- 172.21.15.86:0/2003854091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308100de0 0x7f83081011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.572+0000 7f830cca5640 1 -- 172.21.15.86:0/2003854091 >> 172.21.15.86:0/2003854091 conn(0x7f8308077280 msgr2=0x7f8308077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.572+0000 7f830cca5640 1 -- 172.21.15.86:0/2003854091 shutdown_connections 2024-04-16T05:41:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.572+0000 7f830cca5640 1 -- 172.21.15.86:0/2003854091 wait complete. 2024-04-16T05:41:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.573+0000 7f830cca5640 1 Processor -- start 2024-04-16T05:41:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.573+0000 7f830cca5640 1 -- start start 2024-04-16T05:41:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.573+0000 7f830cca5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8308100de0 0x7f83081a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.573+0000 7f830cca5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 0x7f83081a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.573+0000 7f830cca5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f830810e990 0x7f830819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.573+0000 7f830cca5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83081145e0 con 0x7f8308101700 2024-04-16T05:41:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.573+0000 7f830cca5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8308114460 con 0x7f8308100de0 2024-04-16T05:41:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.573+0000 7f830cca5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8308114760 con 0x7f830810e990 2024-04-16T05:41:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8305d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 0x7f83081a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8305d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 0x7f83081a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56810/0 (socket says 172.21.15.86:56810) 2024-04-16T05:41:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8305d9b640 1 -- 172.21.15.86:0/3758594374 learned_addr learned my addr 172.21.15.86:0/3758594374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f830659c640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8308100de0 0x7f83081a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8306d9d640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f830810e990 0x7f830819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8305d9b640 1 -- 172.21.15.86:0/3758594374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f830810e990 msgr2=0x7f830819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8305d9b640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f830810e990 0x7f830819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8305d9b640 1 -- 172.21.15.86:0/3758594374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8308100de0 msgr2=0x7f83081a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8305d9b640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8308100de0 0x7f83081a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8305d9b640 1 -- 172.21.15.86:0/3758594374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830819b080 con 0x7f8308101700 2024-04-16T05:41:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8306d9d640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f830810e990 0x7f830819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:22.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f8305d9b640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 0x7f83081a0c90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f82fc006010 tx=0x7f82fc02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f82f77fe640 1 -- 172.21.15.86:0/3758594374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc041070 con 0x7f8308101700 2024-04-16T05:41:22.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f82f77fe640 1 -- 172.21.15.86:0/3758594374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc004450 con 0x7f8308101700 2024-04-16T05:41:22.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f830cca5640 1 -- 172.21.15.86:0/3758594374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830819b310 con 0x7f8308101700 2024-04-16T05:41:22.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f82f77fe640 1 -- 172.21.15.86:0/3758594374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc034440 con 0x7f8308101700 2024-04-16T05:41:22.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.574+0000 7f830cca5640 1 -- 172.21.15.86:0/3758594374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83081a7610 con 0x7f8308101700 2024-04-16T05:41:22.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.577+0000 7f82f77fe640 1 -- 172.21.15.86:0/3758594374 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f82fc044070 con 0x7f8308101700 2024-04-16T05:41:22.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.577+0000 7f830f96d740 1 Processor -- start 2024-04-16T05:41:22.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.577+0000 7f830f96d740 1 -- start start 2024-04-16T05:41:22.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.577+0000 7f830f96d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f830f96d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f830f96d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970bd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f830f96d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a970170e0 con 0x557a970bb360 2024-04-16T05:41:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f830f96d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557a96fb4520 con 0x557a970b2780 2024-04-16T05:41:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f830f96d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a970130a0 con 0x557a970b1e60 2024-04-16T05:41:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8305d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8305d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970bd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56826/0 (socket says 172.21.15.86:56826) 2024-04-16T05:41:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8305d9b640 1 -- 172.21.15.86:0/3920261921 learned_addr learned my addr 172.21.15.86:0/3920261921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8306d9d640 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f830659c640 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8305d9b640 1 -- 172.21.15.86:0/3920261921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 msgr2=0x557a970b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8305d9b640 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8305d9b640 1 -- 172.21.15.86:0/3920261921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 msgr2=0x557a970bae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8305d9b640 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970bae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8305d9b640 1 -- 172.21.15.86:0/3920261921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a96fdc8d0 con 0x557a970bb360 2024-04-16T05:41:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8306d9d640 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f82f77fe640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82f00a3da0 0x7f82f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8306d9d640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82f00a3da0 0x7f82f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f8305d9b640 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970bd750 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0efa50 tx=0x7f82fc0f8b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd34880c9307aad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f82f77fe640 1 -- 172.21.15.86:0/3758594374 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f82fc0f2620 con 0x7f8308101700 2024-04-16T05:41:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f82f4ff9640 1 -- 172.21.15.86:0/3920261921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0be130 con 0x557a970bb360 2024-04-16T05:41:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f82f4ff9640 1 -- 172.21.15.86:0/3920261921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc004c80 con 0x557a970bb360 2024-04-16T05:41:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.578+0000 7f82f4ff9640 1 -- 172.21.15.86:0/3920261921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0bdcc0 con 0x557a970bb360 2024-04-16T05:41:22.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f830f96d740 1 -- 172.21.15.86:0/3920261921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 msgr2=0x557a970bd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f8306d9d640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82f00a3da0 0x7f82f00a6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f830806a470 tx=0x7f8300009990 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f830f96d740 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970bd750 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0efa50 tx=0x7f82fc0f8b20 comp rx=0 tx=0).stop 2024-04-16T05:41:22.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f830f96d740 1 -- 172.21.15.86:0/3920261921 shutdown_connections 2024-04-16T05:41:22.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f830f96d740 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970bd750 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f830f96d740 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970bae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f830f96d740 1 --2- 172.21.15.86:0/3920261921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f830f96d740 1 -- 172.21.15.86:0/3920261921 >> 172.21.15.86:0/3920261921 conn(0x557a970ac590 msgr2=0x557a970ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f830f96d740 1 -- 172.21.15.86:0/3920261921 shutdown_connections 2024-04-16T05:41:22.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.579+0000 7f830f96d740 1 -- 172.21.15.86:0/3920261921 wait complete. 2024-04-16T05:41:22.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830f96d740 1 -- start start 2024-04-16T05:41:22.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830f96d740 1 Processor -- start 2024-04-16T05:41:22.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830f96d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830f96d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830f96d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830f96d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a970bdf10 con 0x557a970bb360 2024-04-16T05:41:22.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830f96d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557a96fd04d0 con 0x557a970b2780 2024-04-16T05:41:22.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830f96d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a970be090 con 0x557a970b1e60 2024-04-16T05:41:22.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830f96d740 10 client.0 _ll_register_callbacks cb 0x557a96fb52c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f8306d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f8306d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970b8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56834/0 (socket says 172.21.15.86:56834) 2024-04-16T05:41:22.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f8306d9d640 1 -- 172.21.15.86:0/1753291003 learned_addr learned my addr 172.21.15.86:0/1753291003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f8305d9b640 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f830659c640 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f8306d9d640 1 -- 172.21.15.86:0/1753291003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 msgr2=0x557a970b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f8306d9d640 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970b83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f8306d9d640 1 -- 172.21.15.86:0/1753291003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 msgr2=0x557a970b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f8306d9d640 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970b88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.580+0000 7f8306d9d640 1 -- 172.21.15.86:0/1753291003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a96d293f0 con 0x557a970bb360 2024-04-16T05:41:22.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f8306d9d640 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970b8e30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f83000108c0 tx=0x7f8300004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b57729da7205ae server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f82edffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f82edffb640 1 -- 172.21.15.86:0/1753291003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000faf0 con 0x557a970bb360 2024-04-16T05:41:22.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f82edffb640 1 -- 172.21.15.86:0/1753291003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f830000fc90 con 0x557a970bb360 2024-04-16T05:41:22.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f830f96d740 1 -- 172.21.15.86:0/1753291003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82fc004c80 con 0x557a970bb360 2024-04-16T05:41:22.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f82edffb640 1 -- 172.21.15.86:0/1753291003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f830000fe90 con 0x557a970bb360 2024-04-16T05:41:22.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f830f96d740 10 client.17496 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f830f96d740 1 -- 172.21.15.86:0/1753291003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a96df37d0 con 0x557a970bb360 2024-04-16T05:41:22.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f82edffb640 1 -- 172.21.15.86:0/1753291003 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc004c80 con 0x557a970bb360 2024-04-16T05:41:22.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f82edffb640 1 -- 172.21.15.86:0/1753291003 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f830000f040 con 0x557a970bb360 2024-04-16T05:41:22.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f830f96d740 10 client.17496 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f830f96d740 10 client.17496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:41:22.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f830f96d740 20 client.17496 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f830f96d740 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x557a970cb0d0 0x557a970cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f830f96d740 4 client.17496 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:22.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.581+0000 7f830f96d740 1 -- 172.21.15.86:0/1753291003 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a97024b00 con 0x557a970cb0d0 2024-04-16T05:41:22.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.582+0000 7f830659c640 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x557a970cb0d0 0x557a970cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.582+0000 7f82edffb640 10 client.17496 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:41:22.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.582+0000 7f830659c640 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x557a970cb0d0 0x557a970cd500 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c989e285d0c92a4e server_cookie=757862908f3e3002 in_seq=0 out_seq=0 2024-04-16T05:41:22.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f82edffb640 1 -- 172.21.15.86:0/1753291003 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a97024b00 con 0x557a970cb0d0 2024-04-16T05:41:22.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f82edffb640 10 client.17496 handle_command_reply: tid=0 2024-04-16T05:41:22.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 client.17496 shutdown 2024-04-16T05:41:22.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 -- 172.21.15.86:0/1753291003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 msgr2=0x557a970b8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970b8e30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f83000108c0 tx=0x7f8300004a40 comp rx=0 tx=0).stop 2024-04-16T05:41:22.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 -- 172.21.15.86:0/1753291003 shutdown_connections 2024-04-16T05:41:22.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x557a970cb0d0 0x557a970cd500 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970b8e30 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f82edffb640 0 client.17496 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:41:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970b88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 --2- 172.21.15.86:0/1753291003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 -- 172.21.15.86:0/1753291003 >> 172.21.15.86:0/1753291003 conn(0x557a970ac590 msgr2=0x557a970ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 -- 172.21.15.86:0/1753291003 shutdown_connections 2024-04-16T05:41:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 1 -- 172.21.15.86:0/1753291003 wait complete. 2024-04-16T05:41:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.583+0000 7f830f96d740 20 client.17496 trim_cache size 0 max 16384 2024-04-16T05:41:22.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f830f96d740 1 Processor -- start 2024-04-16T05:41:22.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f830f96d740 1 -- start start 2024-04-16T05:41:22.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f830f96d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f830f96d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f830f96d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f830f96d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83081b1730 con 0x557a970bb360 2024-04-16T05:41:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f830f96d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f83081b3270 con 0x557a970b2780 2024-04-16T05:41:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f830f96d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83081b1e00 con 0x557a970b1e60 2024-04-16T05:41:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f8306d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f8306d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970d2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56836/0 (socket says 172.21.15.86:56836) 2024-04-16T05:41:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.596+0000 7f8306d9d640 1 -- 172.21.15.86:0/1958095835 learned_addr learned my addr 172.21.15.86:0/1958095835 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f8305d9b640 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f830659c640 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f8306d9d640 1 -- 172.21.15.86:0/1958095835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 msgr2=0x557a970c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f8306d9d640 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f8306d9d640 1 -- 172.21.15.86:0/1958095835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 msgr2=0x557a970ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f8306d9d640 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f8306d9d640 1 -- 172.21.15.86:0/1958095835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a970cdb80 con 0x557a970bb360 2024-04-16T05:41:22.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f8305d9b640 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f8306d9d640 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970d2e10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f830001f6d0 tx=0x7f8300009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c3c6f8776c6e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f82effff640 1 -- 172.21.15.86:0/1958095835 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000f730 con 0x557a970bb360 2024-04-16T05:41:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f82effff640 1 -- 172.21.15.86:0/1958095835 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830000f040 con 0x557a970bb360 2024-04-16T05:41:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.597+0000 7f82effff640 1 -- 172.21.15.86:0/1958095835 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300034640 con 0x557a970bb360 2024-04-16T05:41:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 -- 172.21.15.86:0/1958095835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 msgr2=0x557a970d2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970d2e10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f830001f6d0 tx=0x7f8300009d10 comp rx=0 tx=0).stop 2024-04-16T05:41:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 -- 172.21.15.86:0/1958095835 shutdown_connections 2024-04-16T05:41:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970bb360 0x557a970d2e10 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 --2- 172.21.15.86:0/1958095835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970b1e60 0x557a970c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 -- 172.21.15.86:0/1958095835 >> 172.21.15.86:0/1958095835 conn(0x557a970ac590 msgr2=0x557a970aaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 -- 172.21.15.86:0/1958095835 shutdown_connections 2024-04-16T05:41:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 -- 172.21.15.86:0/1958095835 wait complete. 2024-04-16T05:41:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 -- start start 2024-04-16T05:41:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 Processor -- start 2024-04-16T05:41:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970b1e60 0x557a970b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970bb360 0x557a97130680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83081b2ba0 con 0x557a970b1e60 2024-04-16T05:41:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f83081b24d0 con 0x557a970b2780 2024-04-16T05:41:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83081af6f0 con 0x557a970bb360 2024-04-16T05:41:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830f96d740 10 client.0 _ll_register_callbacks cb 0x7f8300009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970b1e60 0x557a970b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970b1e60 0x557a970b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56848/0 (socket says 172.21.15.86:56848) 2024-04-16T05:41:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830659c640 1 -- 172.21.15.86:0/10072490 learned_addr learned my addr 172.21.15.86:0/10072490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f8305d9b640 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f8306d9d640 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970bb360 0x557a97130680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.598+0000 7f830659c640 1 -- 172.21.15.86:0/10072490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970bb360 msgr2=0x557a97130680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830659c640 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970bb360 0x557a97130680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830659c640 1 -- 172.21.15.86:0/10072490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 msgr2=0x557a970b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830659c640 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830659c640 1 -- 172.21.15.86:0/10072490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a97130bc0 con 0x557a970b1e60 2024-04-16T05:41:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f8305d9b640 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f8306d9d640 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970bb360 0x557a97130680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830659c640 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970b1e60 0x557a970b5740 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f82f800bc40 tx=0x7f82f8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49dbdb00dad3f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f82edffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f82edffb640 1 -- 172.21.15.86:0/10072490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8021030 con 0x557a970b1e60 2024-04-16T05:41:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f82edffb640 1 -- 172.21.15.86:0/10072490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f82f8003d30 con 0x557a970b1e60 2024-04-16T05:41:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830f96d740 1 -- 172.21.15.86:0/10072490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830000f040 con 0x557a970b1e60 2024-04-16T05:41:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f82edffb640 1 -- 172.21.15.86:0/10072490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82f802b050 con 0x557a970b1e60 2024-04-16T05:41:22.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830f96d740 10 client.17502 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830f96d740 1 -- 172.21.15.86:0/10072490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a97131d90 con 0x557a970b1e60 2024-04-16T05:41:22.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f82edffb640 1 -- 172.21.15.86:0/10072490 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830000f040 con 0x557a970b1e60 2024-04-16T05:41:22.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f82edffb640 1 -- 172.21.15.86:0/10072490 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f82f80047a0 con 0x557a970b1e60 2024-04-16T05:41:22.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830f96d740 10 client.17502 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830f96d740 10 client.17502 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:41:22.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830f96d740 20 client.17502 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830f96d740 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x557a97132510 0x557a97134930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830f96d740 4 client.17502 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:22.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.599+0000 7f830f96d740 1 -- 172.21.15.86:0/10072490 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a9701e970 con 0x557a97132510 2024-04-16T05:41:22.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f8305d9b640 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x557a97132510 0x557a97134930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f82edffb640 10 client.17502 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:41:22.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f8305d9b640 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x557a97132510 0x557a97134930 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d249a16c76ebf9b server_cookie=251e2d46abe4d340 in_seq=0 out_seq=0 2024-04-16T05:41:22.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f82edffb640 1 -- 172.21.15.86:0/10072490 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a9701e970 con 0x557a97132510 2024-04-16T05:41:22.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f82edffb640 10 client.17502 handle_command_reply: tid=0 2024-04-16T05:41:22.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 client.17502 shutdown 2024-04-16T05:41:22.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 -- 172.21.15.86:0/10072490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970b1e60 msgr2=0x557a970b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970b1e60 0x557a970b5740 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f82f800bc40 tx=0x7f82f8004410 comp rx=0 tx=0).stop 2024-04-16T05:41:22.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 -- 172.21.15.86:0/10072490 shutdown_connections 2024-04-16T05:41:22.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x557a97132510 0x557a97134930 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a970bb360 0x557a97130680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557a970b2780 0x557a970b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f82edffb640 0 client.17502 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:41:22.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 --2- 172.21.15.86:0/10072490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a970b1e60 0x557a970b5740 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 -- 172.21.15.86:0/10072490 >> 172.21.15.86:0/10072490 conn(0x557a970ac590 msgr2=0x557a970ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 -- 172.21.15.86:0/10072490 shutdown_connections 2024-04-16T05:41:22.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 1 -- 172.21.15.86:0/10072490 wait complete. 2024-04-16T05:41:22.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.600+0000 7f830f96d740 20 client.17502 trim_cache size 0 max 16384 2024-04-16T05:41:22.620 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:22.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 -- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82f00a3da0 msgr2=0x7f82f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82f00a3da0 0x7f82f00a6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f830806a470 tx=0x7f8300009990 comp rx=0 tx=0).stop 2024-04-16T05:41:22.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 -- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 msgr2=0x7f83081a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 0x7f83081a0c90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f82fc006010 tx=0x7f82fc02cc70 comp rx=0 tx=0).stop 2024-04-16T05:41:22.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 -- 172.21.15.86:0/3758594374 shutdown_connections 2024-04-16T05:41:22.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f830810e990 0x7f830819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101700 0x7f83081a0c90 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82f00a3da0 0x7f82f00a6260 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 --2- 172.21.15.86:0/3758594374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8308100de0 0x7f83081a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 -- 172.21.15.86:0/3758594374 >> 172.21.15.86:0/3758594374 conn(0x7f8308077280 msgr2=0x7f830810ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 -- 172.21.15.86:0/3758594374 shutdown_connections 2024-04-16T05:41:22.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.601+0000 7f830cca5640 1 -- 172.21.15.86:0/3758594374 wait complete. 2024-04-16T05:41:22.623 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:22 smithi086 ceph-mon[32028]: pgmap v185: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:41:22.623 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:22 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3397333939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:41:22.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:41:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:22 smithi194 ceph-mon[35865]: pgmap v185: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:41:22.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:22 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3397333939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:41:22.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 -- 172.21.15.86:0/1156079053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 msgr2=0x7fcbf0104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/1156079053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 0x7fcbf0104e90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009ca0 tx=0x7fcbe802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 -- 172.21.15.86:0/1156079053 shutdown_connections 2024-04-16T05:41:22.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/1156079053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 0x7fcbf0104e90 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/1156079053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbf006abe0 0x7fcbf006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/1156079053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf0106f80 0x7fcbf006a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 -- 172.21.15.86:0/1156079053 >> 172.21.15.86:0/1156079053 conn(0x7fcbf0077cf0 msgr2=0x7fcbf0078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 -- 172.21.15.86:0/1156079053 shutdown_connections 2024-04-16T05:41:22.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 -- 172.21.15.86:0/1156079053 wait complete. 2024-04-16T05:41:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 Processor -- start 2024-04-16T05:41:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.732+0000 7fcbf78c7640 1 -- start start 2024-04-16T05:41:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf78c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf006abe0 0x7fcbf01a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf78c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 0x7fcbf01a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf78c7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbf0106f80 0x7fcbf019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf78c7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf0118920 con 0x7fcbf0104a10 2024-04-16T05:41:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf78c7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01187a0 con 0x7fcbf0106f80 2024-04-16T05:41:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf78c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf0118aa0 con 0x7fcbf006abe0 2024-04-16T05:41:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf4e62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 0x7fcbf01a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf4e62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 0x7fcbf01a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56866/0 (socket says 172.21.15.86:56866) 2024-04-16T05:41:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf4e62640 1 -- 172.21.15.86:0/2944012566 learned_addr learned my addr 172.21.15.86:0/2944012566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf5e64640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbf0106f80 0x7fcbf019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf5663640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf006abe0 0x7fcbf01a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf4e62640 1 -- 172.21.15.86:0/2944012566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf006abe0 msgr2=0x7fcbf01a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf006abe0 0x7fcbf01a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf4e62640 1 -- 172.21.15.86:0/2944012566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbf0106f80 msgr2=0x7fcbf019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbf0106f80 0x7fcbf019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf4e62640 1 -- 172.21.15.86:0/2944012566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf019f3e0 con 0x7fcbf0104a10 2024-04-16T05:41:22.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 0x7fcbf01a4fa0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000d950 tx=0x7fcbe000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbe67fc640 1 -- 172.21.15.86:0/2944012566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0017070 con 0x7fcbf0104a10 2024-04-16T05:41:22.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbe67fc640 1 -- 172.21.15.86:0/2944012566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe0009c30 con 0x7fcbf0104a10 2024-04-16T05:41:22.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf78c7640 1 -- 172.21.15.86:0/2944012566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf019f6d0 con 0x7fcbf0104a10 2024-04-16T05:41:22.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbe67fc640 1 -- 172.21.15.86:0/2944012566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000f6b0 con 0x7fcbf0104a10 2024-04-16T05:41:22.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.733+0000 7fcbf78c7640 1 -- 172.21.15.86:0/2944012566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf01ab9e0 con 0x7fcbf0104a10 2024-04-16T05:41:22.739 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:22 smithi196 ceph-mon[36373]: pgmap v185: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:41:22.740 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:22 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3397333939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:41:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbe67fc640 1 -- 172.21.15.86:0/2944012566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcbe00109d0 con 0x7fcbf0104a10 2024-04-16T05:41:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbfa582740 1 Processor -- start 2024-04-16T05:41:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbfa582740 1 -- start start 2024-04-16T05:41:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e03e60 0x55c0b3e04240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e0ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbfa582740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0b3d690e0 con 0x55c0b3e04780 2024-04-16T05:41:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbfa582740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c0b3d06520 con 0x55c0b3e03e60 2024-04-16T05:41:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbfa582740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0b3d650a0 con 0x55c0b3e0d360 2024-04-16T05:41:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.736+0000 7fcbe67fc640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcbdc0a3da0 0x7fcbdc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5663640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e0ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5663640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e0ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56874/0 (socket says 172.21.15.86:56874) 2024-04-16T05:41:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5663640 1 -- 172.21.15.86:0/299746092 learned_addr learned my addr 172.21.15.86:0/299746092 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/299746092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5e64640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcbdc0a3da0 0x7fcbdc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbe67fc640 1 -- 172.21.15.86:0/2944012566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcbe00d0710 con 0x7fcbf0104a10 2024-04-16T05:41:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5e64640 1 --2- 172.21.15.86:0/299746092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e03e60 0x55c0b3e04240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5663640 1 -- 172.21.15.86:0/299746092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e0d360 msgr2=0x55c0b3e0f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5663640 1 --2- 172.21.15.86:0/299746092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5663640 1 -- 172.21.15.86:0/299746092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e03e60 msgr2=0x55c0b3e04240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5663640 1 --2- 172.21.15.86:0/299746092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e03e60 0x55c0b3e04240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5663640 1 -- 172.21.15.86:0/299746092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b3d2e8d0 con 0x55c0b3e04780 2024-04-16T05:41:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5663640 1 --2- 172.21.15.86:0/299746092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e0ce20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00b820 tx=0x7fcbec00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8a40bb0b1715fd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbf5e64640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcbdc0a3da0 0x7fcbdc0a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8004af0 tx=0x7fcbe802e040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbdb7fe640 1 -- 172.21.15.86:0/299746092 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00d950 con 0x55c0b3e04780 2024-04-16T05:41:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbdb7fe640 1 -- 172.21.15.86:0/299746092 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbec019070 con 0x55c0b3e04780 2024-04-16T05:41:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.737+0000 7fcbdb7fe640 1 -- 172.21.15.86:0/299746092 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0123e0 con 0x55c0b3e04780 2024-04-16T05:41:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 -- 172.21.15.86:0/299746092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 msgr2=0x55c0b3e0ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 --2- 172.21.15.86:0/299746092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e0ce20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00b820 tx=0x7fcbec00bce0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 -- 172.21.15.86:0/299746092 shutdown_connections 2024-04-16T05:41:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 --2- 172.21.15.86:0/299746092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 --2- 172.21.15.86:0/299746092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e0ce20 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 --2- 172.21.15.86:0/299746092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e03e60 0x55c0b3e04240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 -- 172.21.15.86:0/299746092 >> 172.21.15.86:0/299746092 conn(0x55c0b3dfe590 msgr2=0x55c0b3dfe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 -- 172.21.15.86:0/299746092 shutdown_connections 2024-04-16T05:41:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 -- 172.21.15.86:0/299746092 wait complete. 2024-04-16T05:41:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 -- start start 2024-04-16T05:41:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.738+0000 7fcbfa582740 1 Processor -- start 2024-04-16T05:41:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e0a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 0x55c0b3e0a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbfa582740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0b3e100c0 con 0x55c0b3e03e60 2024-04-16T05:41:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbfa582740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c0b3d224d0 con 0x55c0b3e0d360 2024-04-16T05:41:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbfa582740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0b3e10240 con 0x55c0b3e04780 2024-04-16T05:41:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbfa582740 10 client.0 _ll_register_callbacks cb 0x55c0b3d072c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5663640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e0a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5663640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e0a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56878/0 (socket says 172.21.15.86:56878) 2024-04-16T05:41:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5663640 1 -- 172.21.15.86:0/4007867742 learned_addr learned my addr 172.21.15.86:0/4007867742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5e64640 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 0x55c0b3e0a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5663640 1 -- 172.21.15.86:0/4007867742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 msgr2=0x55c0b3e0a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5663640 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 0x55c0b3e0a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5663640 1 -- 172.21.15.86:0/4007867742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 msgr2=0x55c0b3e0ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5663640 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5663640 1 -- 172.21.15.86:0/4007867742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b3a7b3f0 con 0x55c0b3e03e60 2024-04-16T05:41:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 0x55c0b3e0a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5e64640 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbf5663640 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e0a310 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcbec004700 tx=0x7fcbec014660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54db30373109b42f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.739+0000 7fcbd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/4007867742 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec023070 con 0x55c0b3e03e60 2024-04-16T05:41:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/4007867742 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcbec00d950 con 0x55c0b3e03e60 2024-04-16T05:41:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbfa582740 1 -- 172.21.15.86:0/4007867742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbec019070 con 0x55c0b3e03e60 2024-04-16T05:41:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/4007867742 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcbec02f050 con 0x55c0b3e03e60 2024-04-16T05:41:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbfa582740 10 client.17514 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbfa582740 1 -- 172.21.15.86:0/4007867742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0b3b457d0 con 0x55c0b3e03e60 2024-04-16T05:41:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/4007867742 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbec019070 con 0x55c0b3e03e60 2024-04-16T05:41:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/4007867742 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcbec013070 con 0x55c0b3e03e60 2024-04-16T05:41:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbfa582740 10 client.17514 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbfa582740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbfa582740 20 client.17514 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbfa582740 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c0b3e1d1f0 0x55c0b3e1f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbfa582740 4 client.17514 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbfa582740 1 -- 172.21.15.86:0/4007867742 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0b3d76b00 con 0x55c0b3e1d1f0 2024-04-16T05:41:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.740+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c0b3e1d1f0 0x55c0b3e1f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.741+0000 7fcbd8ff9640 10 client.17514 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.741+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c0b3e1d1f0 0x55c0b3e1f5e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11faf0279b6df603 server_cookie=d1e5cb9e43a25f1c in_seq=0 out_seq=0 2024-04-16T05:41:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.741+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/4007867742 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0b3d76b00 con 0x55c0b3e1d1f0 2024-04-16T05:41:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.741+0000 7fcbd8ff9640 10 client.17514 handle_command_reply: tid=0 2024-04-16T05:41:22.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 client.17514 shutdown 2024-04-16T05:41:22.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 -- 172.21.15.86:0/4007867742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 msgr2=0x55c0b3e0a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e0a310 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcbec004700 tx=0x7fcbec014660 comp rx=0 tx=0).stop 2024-04-16T05:41:22.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 -- 172.21.15.86:0/4007867742 shutdown_connections 2024-04-16T05:41:22.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c0b3e1d1f0 0x55c0b3e1f5e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 0x55c0b3e0a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 --2- 172.21.15.86:0/4007867742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e0a310 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbd8ff9640 0 client.17514 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:22.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 -- 172.21.15.86:0/4007867742 >> 172.21.15.86:0/4007867742 conn(0x55c0b3dfe590 msgr2=0x55c0b3e002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 -- 172.21.15.86:0/4007867742 shutdown_connections 2024-04-16T05:41:22.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 1 -- 172.21.15.86:0/4007867742 wait complete. 2024-04-16T05:41:22.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.742+0000 7fcbfa582740 20 client.17514 trim_cache size 0 max 16384 2024-04-16T05:41:22.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbfa582740 1 Processor -- start 2024-04-16T05:41:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbfa582740 1 -- start start 2024-04-16T05:41:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e03e60 0x55c0b3e190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e09bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbfa582740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01b5980 con 0x55c0b3e04780 2024-04-16T05:41:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbfa582740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01b74c0 con 0x55c0b3e0d360 2024-04-16T05:41:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbfa582740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01b6050 con 0x55c0b3e03e60 2024-04-16T05:41:22.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbf4e62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e09bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbf4e62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e09bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56882/0 (socket says 172.21.15.86:56882) 2024-04-16T05:41:22.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbf4e62640 1 -- 172.21.15.86:0/2619399727 learned_addr learned my addr 172.21.15.86:0/2619399727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.755+0000 7fcbf5663640 1 --2- 172.21.15.86:0/2619399727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e03e60 0x55c0b3e190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbf4e62640 1 -- 172.21.15.86:0/2619399727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e03e60 msgr2=0x55c0b3e190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/2619399727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e03e60 0x55c0b3e190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbf4e62640 1 -- 172.21.15.86:0/2619399727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 msgr2=0x55c0b3e0c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/2619399727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbf4e62640 1 -- 172.21.15.86:0/2619399727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b3dff160 con 0x55c0b3e04780 2024-04-16T05:41:22.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/2619399727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e09bf0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0010fd0 tx=0x7fcbe000dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816d8a43776c90ab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbdaffd640 1 -- 172.21.15.86:0/2619399727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0020040 con 0x55c0b3e04780 2024-04-16T05:41:22.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbdaffd640 1 -- 172.21.15.86:0/2619399727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe00a4fb0 con 0x55c0b3e04780 2024-04-16T05:41:22.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbdaffd640 1 -- 172.21.15.86:0/2619399727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00a4b20 con 0x55c0b3e04780 2024-04-16T05:41:22.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbfa582740 1 -- 172.21.15.86:0/2619399727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 msgr2=0x55c0b3e09bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbfa582740 1 --2- 172.21.15.86:0/2619399727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e09bf0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0010fd0 tx=0x7fcbe000dfa0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbfa582740 1 -- 172.21.15.86:0/2619399727 shutdown_connections 2024-04-16T05:41:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbfa582740 1 --2- 172.21.15.86:0/2619399727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e0c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbfa582740 1 --2- 172.21.15.86:0/2619399727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e04780 0x55c0b3e09bf0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbfa582740 1 --2- 172.21.15.86:0/2619399727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e03e60 0x55c0b3e190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbfa582740 1 -- 172.21.15.86:0/2619399727 >> 172.21.15.86:0/2619399727 conn(0x55c0b3dfe590 msgr2=0x55c0b3df95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.756+0000 7fcbfa582740 1 -- 172.21.15.86:0/2619399727 shutdown_connections 2024-04-16T05:41:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 1 -- 172.21.15.86:0/2619399727 wait complete. 2024-04-16T05:41:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 1 -- start start 2024-04-16T05:41:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 1 Processor -- start 2024-04-16T05:41:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e15a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 0x55c0b3e07900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e07f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01b6df0 con 0x55c0b3e03e60 2024-04-16T05:41:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01b6720 con 0x55c0b3e0d360 2024-04-16T05:41:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01b3940 con 0x55c0b3e04780 2024-04-16T05:41:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbfa582740 10 client.0 _ll_register_callbacks cb 0x7fcbe00985e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbf5663640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e15a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbf5663640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e15a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56890/0 (socket says 172.21.15.86:56890) 2024-04-16T05:41:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbf5663640 1 -- 172.21.15.86:0/356925608 learned_addr learned my addr 172.21.15.86:0/356925608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.757+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 0x55c0b3e07900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbf5e64640 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e07f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbf5663640 1 -- 172.21.15.86:0/356925608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 msgr2=0x55c0b3e07900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbf5663640 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 0x55c0b3e07900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbf5663640 1 -- 172.21.15.86:0/356925608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 msgr2=0x55c0b3e07f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbf5663640 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e07f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbf5663640 1 -- 172.21.15.86:0/356925608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b3e06730 con 0x55c0b3e03e60 2024-04-16T05:41:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbf5663640 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e15a80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0046d0 tx=0x7fcbec006bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2154e9d01631ab9d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/356925608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec023070 con 0x55c0b3e03e60 2024-04-16T05:41:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/356925608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcbec00d950 con 0x55c0b3e03e60 2024-04-16T05:41:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbfa582740 1 -- 172.21.15.86:0/356925608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe00a4fb0 con 0x55c0b3e03e60 2024-04-16T05:41:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/356925608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcbec00b140 con 0x55c0b3e03e60 2024-04-16T05:41:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbfa582740 10 client.17520 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbfa582740 1 -- 172.21.15.86:0/356925608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0b3e82930 con 0x55c0b3e03e60 2024-04-16T05:41:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/356925608 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe00a4fb0 con 0x55c0b3e03e60 2024-04-16T05:41:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/356925608 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcbec01d920 con 0x55c0b3e03e60 2024-04-16T05:41:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbfa582740 10 client.17520 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbfa582740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbfa582740 20 client.17520 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbfa582740 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c0b3e830b0 0x55c0b3e854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbfa582740 4 client.17520 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.758+0000 7fcbfa582740 1 -- 172.21.15.86:0/356925608 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0b3d70970 con 0x55c0b3e830b0 2024-04-16T05:41:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.759+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c0b3e830b0 0x55c0b3e854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.759+0000 7fcbd8ff9640 10 client.17520 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.759+0000 7fcbf4e62640 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c0b3e830b0 0x55c0b3e854d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c795d6b15e4cb80 server_cookie=b3626494b3ec1744 in_seq=0 out_seq=0 2024-04-16T05:41:22.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.759+0000 7fcbd8ff9640 1 -- 172.21.15.86:0/356925608 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0b3d70970 con 0x55c0b3e830b0 2024-04-16T05:41:22.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.759+0000 7fcbd8ff9640 10 client.17520 handle_command_reply: tid=0 2024-04-16T05:41:22.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.759+0000 7fcbfa582740 1 client.17520 shutdown 2024-04-16T05:41:22.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 -- 172.21.15.86:0/356925608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 msgr2=0x55c0b3e15a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e15a80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0046d0 tx=0x7fcbec006bc0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 -- 172.21.15.86:0/356925608 shutdown_connections 2024-04-16T05:41:22.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c0b3e830b0 0x55c0b3e854d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0b3e0d360 0x55c0b3e07f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0b3e04780 0x55c0b3e07900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbd8ff9640 0 client.17520 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 --2- 172.21.15.86:0/356925608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0b3e03e60 0x55c0b3e15a80 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 -- 172.21.15.86:0/356925608 >> 172.21.15.86:0/356925608 conn(0x55c0b3dfe590 msgr2=0x55c0b3dfd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 -- 172.21.15.86:0/356925608 shutdown_connections 2024-04-16T05:41:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 1 -- 172.21.15.86:0/356925608 wait complete. 2024-04-16T05:41:22.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbfa582740 20 client.17520 trim_cache size 0 max 16384 2024-04-16T05:41:22.778 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:22.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 -- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcbdc0a3da0 msgr2=0x7fcbdc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcbdc0a3da0 0x7fcbdc0a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8004af0 tx=0x7fcbe802e040 comp rx=0 tx=0).stop 2024-04-16T05:41:22.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 -- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 msgr2=0x7fcbf01a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:22.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 0x7fcbf01a4fa0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000d950 tx=0x7fcbe000de10 comp rx=0 tx=0).stop 2024-04-16T05:41:22.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 -- 172.21.15.86:0/2944012566 shutdown_connections 2024-04-16T05:41:22.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcbdc0a3da0 0x7fcbdc0a6260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbf0106f80 0x7fcbf019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbf0104a10 0x7fcbf01a4fa0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 --2- 172.21.15.86:0/2944012566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf006abe0 0x7fcbf01a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:22.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 -- 172.21.15.86:0/2944012566 >> 172.21.15.86:0/2944012566 conn(0x7fcbf0077cf0 msgr2=0x7fcbf0103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:22.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 -- 172.21.15.86:0/2944012566 shutdown_connections 2024-04-16T05:41:22.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:22.760+0000 7fcbf78c7640 1 -- 172.21.15.86:0/2944012566 wait complete. 2024-04-16T05:41:23.782 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:41:23.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:41:23.782 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:41:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.904+0000 7fa64a9f4640 1 -- 172.21.15.86:0/3769303214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6441070e0 msgr2=0x7fa6441074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.904+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/3769303214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6441070e0 0x7fa6441074c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa634009930 tx=0x7fa63402bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.904+0000 7fa64a9f4640 1 -- 172.21.15.86:0/3769303214 shutdown_connections 2024-04-16T05:41:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.904+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/3769303214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa644101bf0 0x7fa64410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.904+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/3769303214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa644101250 0x7fa6441016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.904+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/3769303214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6441070e0 0x7fa6441074c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.904+0000 7fa64a9f4640 1 -- 172.21.15.86:0/3769303214 >> 172.21.15.86:0/3769303214 conn(0x7fa644076d00 msgr2=0x7fa644077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.904+0000 7fa64a9f4640 1 -- 172.21.15.86:0/3769303214 shutdown_connections 2024-04-16T05:41:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.904+0000 7fa64a9f4640 1 -- 172.21.15.86:0/3769303214 wait complete. 2024-04-16T05:41:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa64a9f4640 1 Processor -- start 2024-04-16T05:41:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa64a9f4640 1 -- start start 2024-04-16T05:41:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa64a9f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa644101250 0x7fa64419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa64a9f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101bf0 0x7fa64419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa64a9f4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6441070e0 0x7fa644198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa64a9f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa644112460 con 0x7fa644101bf0 2024-04-16T05:41:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa64a9f4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441122e0 con 0x7fa6441070e0 2024-04-16T05:41:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa64a9f4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441125e0 con 0x7fa644101250 2024-04-16T05:41:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101bf0 0x7fa64419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101bf0 0x7fa64419e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56910/0 (socket says 172.21.15.86:56910) 2024-04-16T05:41:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa63bfff640 1 -- 172.21.15.86:0/806507078 learned_addr learned my addr 172.21.15.86:0/806507078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa648f91640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6441070e0 0x7fa644198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.905+0000 7fa63bfff640 1 -- 172.21.15.86:0/806507078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa644101250 msgr2=0x7fa64419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa63bfff640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa644101250 0x7fa64419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa63bfff640 1 -- 172.21.15.86:0/806507078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6441070e0 msgr2=0x7fa644198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa63bfff640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6441070e0 0x7fa644198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa63bfff640 1 -- 172.21.15.86:0/806507078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa644198de0 con 0x7fa644101bf0 2024-04-16T05:41:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa648f91640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6441070e0 0x7fa644198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa63bfff640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101bf0 0x7fa64419e9a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c920 tx=0x7fa62c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa641ffb640 1 -- 172.21.15.86:0/806507078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c007c70 con 0x7fa644101bf0 2024-04-16T05:41:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa641ffb640 1 -- 172.21.15.86:0/806507078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa62c007e10 con 0x7fa644101bf0 2024-04-16T05:41:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa64a9f4640 1 -- 172.21.15.86:0/806507078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6441990d0 con 0x7fa644101bf0 2024-04-16T05:41:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa641ffb640 1 -- 172.21.15.86:0/806507078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c018890 con 0x7fa644101bf0 2024-04-16T05:41:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.906+0000 7fa64a9f4640 1 -- 172.21.15.86:0/806507078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6441a5320 con 0x7fa644101bf0 2024-04-16T05:41:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.908+0000 7fa641ffb640 1 -- 172.21.15.86:0/806507078 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa62c018a70 con 0x7fa644101bf0 2024-04-16T05:41:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa64d5e6740 1 Processor -- start 2024-04-16T05:41:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa64d5e6740 1 -- start start 2024-04-16T05:41:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 0x5564fe7b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564fe71ee90 con 0x5564fe7ba080 2024-04-16T05:41:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5564fe6bc4c0 con 0x5564fe7c73c0 2024-04-16T05:41:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5564fe71ae50 con 0x5564fe7b9760 2024-04-16T05:41:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa641ffb640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6300a3d20 0x7fa6300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa643fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa643fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56912/0 (socket says 172.21.15.86:56912) 2024-04-16T05:41:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa643fff640 1 -- 172.21.15.86:0/1863583043 learned_addr learned my addr 172.21.15.86:0/1863583043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa648f91640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6300a3d20 0x7fa6300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa648f91640 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 0x5564fe7b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa643fff640 1 -- 172.21.15.86:0/1863583043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 msgr2=0x5564fe7b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa63bfff640 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa641ffb640 1 -- 172.21.15.86:0/806507078 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa62c0d0da0 con 0x7fa644101bf0 2024-04-16T05:41:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa643fff640 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 0x5564fe7b9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa643fff640 1 -- 172.21.15.86:0/1863583043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 msgr2=0x5564fe7c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa643fff640 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa643fff640 1 -- 172.21.15.86:0/1863583043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564fe4f97e0 con 0x5564fe7ba080 2024-04-16T05:41:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa648f91640 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 0x5564fe7b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.909+0000 7fa63bfff640 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa643fff640 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7c6e80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa63402cfd0 tx=0x7fa634005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef1a05616e4fde6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa648f91640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6300a3d20 0x7fa6300a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa63c012690 tx=0x7fa63c0074c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa63a7fc640 1 -- 172.21.15.86:0/1863583043 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634041020 con 0x5564fe7ba080 2024-04-16T05:41:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa63a7fc640 1 -- 172.21.15.86:0/1863583043 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa63402ca80 con 0x5564fe7ba080 2024-04-16T05:41:23.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa63a7fc640 1 -- 172.21.15.86:0/1863583043 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63403d440 con 0x5564fe7ba080 2024-04-16T05:41:23.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa64d5e6740 1 -- 172.21.15.86:0/1863583043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 msgr2=0x5564fe7c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:23.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7c6e80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa63402cfd0 tx=0x7fa634005e80 comp rx=0 tx=0).stop 2024-04-16T05:41:23.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa64d5e6740 1 -- 172.21.15.86:0/1863583043 shutdown_connections 2024-04-16T05:41:23.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7c6e80 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/1863583043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 0x5564fe7b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa64d5e6740 1 -- 172.21.15.86:0/1863583043 >> 172.21.15.86:0/1863583043 conn(0x5564fe7b41e0 msgr2=0x5564fe7b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa64d5e6740 1 -- 172.21.15.86:0/1863583043 shutdown_connections 2024-04-16T05:41:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.910+0000 7fa64d5e6740 1 -- 172.21.15.86:0/1863583043 wait complete. 2024-04-16T05:41:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa64d5e6740 1 -- start start 2024-04-16T05:41:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa64d5e6740 1 Processor -- start 2024-04-16T05:41:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7b9760 0x5564fe7c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 0x5564fe7c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564fe7ca1a0 con 0x5564fe7b9760 2024-04-16T05:41:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5564fe6a7410 con 0x5564fe7c73c0 2024-04-16T05:41:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5564fe7ca320 con 0x5564fe7ba080 2024-04-16T05:41:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa64d5e6740 10 client.0 _ll_register_callbacks cb 0x5564fe6a2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa643fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7b9760 0x5564fe7c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa643fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7b9760 0x5564fe7c42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56922/0 (socket says 172.21.15.86:56922) 2024-04-16T05:41:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.911+0000 7fa643fff640 1 -- 172.21.15.86:0/2382812484 learned_addr learned my addr 172.21.15.86:0/2382812484 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa63bfff640 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 0x5564fe7c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa643fff640 1 -- 172.21.15.86:0/2382812484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 msgr2=0x5564fe7c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa643fff640 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 0x5564fe7c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa643fff640 1 -- 172.21.15.86:0/2382812484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 msgr2=0x5564fe7c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa643fff640 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa643fff640 1 -- 172.21.15.86:0/2382812484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564fe559f70 con 0x5564fe7b9760 2024-04-16T05:41:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa643fff640 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7b9760 0x5564fe7c42c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa6340096c0 tx=0x7fa63402c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1fdd3eaf6ef8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa6237fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa6237fe640 1 -- 172.21.15.86:0/2382812484 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634041030 con 0x5564fe7b9760 2024-04-16T05:41:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa6237fe640 1 -- 172.21.15.86:0/2382812484 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa6340040a0 con 0x5564fe7b9760 2024-04-16T05:41:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2382812484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa63402ca80 con 0x5564fe7b9760 2024-04-16T05:41:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa6237fe640 1 -- 172.21.15.86:0/2382812484 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa634055050 con 0x5564fe7b9760 2024-04-16T05:41:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa64d5e6740 10 client.17532 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2382812484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564fe3f1170 con 0x5564fe7b9760 2024-04-16T05:41:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa6237fe640 1 -- 172.21.15.86:0/2382812484 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa63402ca80 con 0x5564fe7b9760 2024-04-16T05:41:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.912+0000 7fa6237fe640 1 -- 172.21.15.86:0/2382812484 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa634034440 con 0x5564fe7b9760 2024-04-16T05:41:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.913+0000 7fa64d5e6740 10 client.17532 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.913+0000 7fa64d5e6740 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.913+0000 7fa64d5e6740 20 client.17532 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.913+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5564fe7d7470 0x5564fe7d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.913+0000 7fa64d5e6740 4 client.17532 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.913+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2382812484 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564fe72c8b0 con 0x5564fe7d7470 2024-04-16T05:41:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.913+0000 7fa63bfff640 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5564fe7d7470 0x5564fe7d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.913+0000 7fa6237fe640 10 client.17532 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.913+0000 7fa63bfff640 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5564fe7d7470 0x5564fe7d9860 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d3535d4ebee50a7 server_cookie=fae8460562fce198 in_seq=0 out_seq=0 2024-04-16T05:41:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa6237fe640 1 -- 172.21.15.86:0/2382812484 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564fe72c8b0 con 0x5564fe7d7470 2024-04-16T05:41:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa6237fe640 10 client.17532 handle_command_reply: tid=0 2024-04-16T05:41:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 client.17532 shutdown 2024-04-16T05:41:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2382812484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7b9760 msgr2=0x5564fe7c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7b9760 0x5564fe7c42c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa6340096c0 tx=0x7fa63402c880 comp rx=0 tx=0).stop 2024-04-16T05:41:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2382812484 shutdown_connections 2024-04-16T05:41:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5564fe7d7470 0x5564fe7d9860 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa6237fe640 0 client.17532 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 0x5564fe7c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2382812484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7b9760 0x5564fe7c42c0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2382812484 >> 172.21.15.86:0/2382812484 conn(0x5564fe7b41e0 msgr2=0x5564fe7b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2382812484 shutdown_connections 2024-04-16T05:41:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2382812484 wait complete. 2024-04-16T05:41:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.914+0000 7fa64d5e6740 20 client.17532 trim_cache size 0 max 16384 2024-04-16T05:41:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa64d5e6740 1 Processor -- start 2024-04-16T05:41:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa64d5e6740 1 -- start start 2024-04-16T05:41:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 0x5564fe7c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441ad020 con 0x5564fe7ba080 2024-04-16T05:41:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441aeb60 con 0x5564fe7c73c0 2024-04-16T05:41:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441ad6f0 con 0x5564fe7b9760 2024-04-16T05:41:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa63bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7d30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56934/0 (socket says 172.21.15.86:56934) 2024-04-16T05:41:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa63bfff640 1 -- 172.21.15.86:0/937057691 learned_addr learned my addr 172.21.15.86:0/937057691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa63bfff640 1 -- 172.21.15.86:0/937057691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 msgr2=0x5564fe7c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa63bfff640 1 --2- 172.21.15.86:0/937057691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 0x5564fe7c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa63bfff640 1 -- 172.21.15.86:0/937057691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 msgr2=0x5564fe7c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa63bfff640 1 --2- 172.21.15.86:0/937057691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa63bfff640 1 -- 172.21.15.86:0/937057691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564fe7b4010 con 0x5564fe7ba080 2024-04-16T05:41:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa63bfff640 1 --2- 172.21.15.86:0/937057691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7d30d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0bd1c0 tx=0x7fa62c098c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d5360f124db6ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa639ffb640 1 -- 172.21.15.86:0/937057691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c09acd0 con 0x5564fe7ba080 2024-04-16T05:41:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.932+0000 7fa639ffb640 1 -- 172.21.15.86:0/937057691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa62c018d60 con 0x5564fe7ba080 2024-04-16T05:41:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa639ffb640 1 -- 172.21.15.86:0/937057691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0991d0 con 0x5564fe7ba080 2024-04-16T05:41:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 -- 172.21.15.86:0/937057691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 msgr2=0x5564fe7d30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/937057691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7d30d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0bd1c0 tx=0x7fa62c098c70 comp rx=0 tx=0).stop 2024-04-16T05:41:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 -- 172.21.15.86:0/937057691 shutdown_connections 2024-04-16T05:41:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/937057691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7c73c0 0x5564fe7c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/937057691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7ba080 0x5564fe7d30d0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/937057691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7b9760 0x5564fe7c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 -- 172.21.15.86:0/937057691 >> 172.21.15.86:0/937057691 conn(0x5564fe7b41e0 msgr2=0x5564fe7b5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 -- 172.21.15.86:0/937057691 shutdown_connections 2024-04-16T05:41:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 -- 172.21.15.86:0/937057691 wait complete. 2024-04-16T05:41:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.933+0000 7fa64d5e6740 1 -- start start 2024-04-16T05:41:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa64d5e6740 1 Processor -- start 2024-04-16T05:41:23.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7b9760 0x5564fe7cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 0x5564fe7cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa64d5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7c73c0 0x5564fe7cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441ae490 con 0x5564fe7c73c0 2024-04-16T05:41:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441addc0 con 0x5564fe7b9760 2024-04-16T05:41:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa64d5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441aafe0 con 0x5564fe7ba080 2024-04-16T05:41:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa64d5e6740 10 client.0 _ll_register_callbacks cb 0x7fa634005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa648f91640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7c73c0 0x5564fe7cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa648f91640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7c73c0 0x5564fe7cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56936/0 (socket says 172.21.15.86:56936) 2024-04-16T05:41:23.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa648f91640 1 -- 172.21.15.86:0/2715195936 learned_addr learned my addr 172.21.15.86:0/2715195936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:23.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa643fff640 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7b9760 0x5564fe7cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa648f91640 1 -- 172.21.15.86:0/2715195936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 msgr2=0x5564fe7cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:23.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa63bfff640 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 0x5564fe7cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa648f91640 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 0x5564fe7cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa648f91640 1 -- 172.21.15.86:0/2715195936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7b9760 msgr2=0x5564fe7cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:23.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa648f91640 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7b9760 0x5564fe7cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa648f91640 1 -- 172.21.15.86:0/2715195936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564fe7c9b70 con 0x5564fe7c73c0 2024-04-16T05:41:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa63bfff640 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 0x5564fe7cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa643fff640 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7b9760 0x5564fe7cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa648f91640 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7c73c0 0x5564fe7cbb70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00ef30 tx=0x7fa63c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f585f60488fac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.934+0000 7fa6237fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa6237fe640 1 -- 172.21.15.86:0/2715195936 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c022070 con 0x5564fe7c73c0 2024-04-16T05:41:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa6237fe640 1 -- 172.21.15.86:0/2715195936 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa63c004d20 con 0x5564fe7c73c0 2024-04-16T05:41:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2715195936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa62c018d60 con 0x5564fe7c73c0 2024-04-16T05:41:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa6237fe640 1 -- 172.21.15.86:0/2715195936 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa63c02d050 con 0x5564fe7c73c0 2024-04-16T05:41:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa64d5e6740 10 client.17538 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2715195936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564fe83e280 con 0x5564fe7c73c0 2024-04-16T05:41:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa6237fe640 1 -- 172.21.15.86:0/2715195936 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa62c018d60 con 0x5564fe7c73c0 2024-04-16T05:41:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa6237fe640 1 -- 172.21.15.86:0/2715195936 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa63c01a040 con 0x5564fe7c73c0 2024-04-16T05:41:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa64d5e6740 10 client.17538 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa64d5e6740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa64d5e6740 20 client.17538 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5564fe83ea30 0x5564fe840e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa64d5e6740 4 client.17538 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:41:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2715195936 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5564fe7ba7d0 con 0x5564fe83ea30 2024-04-16T05:41:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.935+0000 7fa643fff640 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5564fe83ea30 0x5564fe840e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.936+0000 7fa643fff640 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5564fe83ea30 0x5564fe840e50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f747119b5891793b server_cookie=ed6602bad2e640b4 in_seq=0 out_seq=0 2024-04-16T05:41:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.936+0000 7fa6237fe640 10 client.17538 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa6237fe640 1 -- 172.21.15.86:0/2715195936 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5564fe7ba7d0 con 0x5564fe83ea30 2024-04-16T05:41:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa6237fe640 10 client.17538 handle_command_reply: tid=0 2024-04-16T05:41:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 client.17538 shutdown 2024-04-16T05:41:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2715195936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7c73c0 msgr2=0x5564fe7cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7c73c0 0x5564fe7cbb70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00ef30 tx=0x7fa63c010800 comp rx=0 tx=0).stop 2024-04-16T05:41:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2715195936 shutdown_connections 2024-04-16T05:41:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5564fe83ea30 0x5564fe840e50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe7c73c0 0x5564fe7cbb70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa6237fe640 0 client.17538 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564fe7ba080 0x5564fe7cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 --2- 172.21.15.86:0/2715195936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5564fe7b9760 0x5564fe7cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2715195936 >> 172.21.15.86:0/2715195936 conn(0x5564fe7b41e0 msgr2=0x5564fe7b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2715195936 shutdown_connections 2024-04-16T05:41:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.937+0000 7fa64d5e6740 1 -- 172.21.15.86:0/2715195936 wait complete. 2024-04-16T05:41:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64d5e6740 20 client.17538 trim_cache size 0 max 16384 2024-04-16T05:41:23.946 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:41:23.946 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:41:23.946 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "f31e0ea5-6c90-4479-897e-774306d43a57", 2024-04-16T05:41:23.946 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:41:23.947 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:41:23.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 -- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6300a3d20 msgr2=0x7fa6300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:23.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6300a3d20 0x7fa6300a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa63c012690 tx=0x7fa63c0074c0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 -- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101bf0 msgr2=0x7fa64419e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:23.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101bf0 0x7fa64419e9a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c920 tx=0x7fa62c00cde0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 -- 172.21.15.86:0/806507078 shutdown_connections 2024-04-16T05:41:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa6300a3d20 0x7fa6300a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6441070e0 0x7fa644198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101bf0 0x7fa64419e9a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 --2- 172.21.15.86:0/806507078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa644101250 0x7fa64419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 -- 172.21.15.86:0/806507078 >> 172.21.15.86:0/806507078 conn(0x7fa644076d00 msgr2=0x7fa6440ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 -- 172.21.15.86:0/806507078 shutdown_connections 2024-04-16T05:41:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:23.938+0000 7fa64a9f4640 1 -- 172.21.15.86:0/806507078 wait complete. 2024-04-16T05:41:23.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:41:23.950 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:41:24.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 -- 172.21.15.86:0/1692952924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c101af0 msgr2=0x7fcd3c105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:24.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/1692952924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c101af0 0x7fcd3c105b30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009900 tx=0x7fcd2802bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:24.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 -- 172.21.15.86:0/1692952924 shutdown_connections 2024-04-16T05:41:24.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/1692952924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd3c106210 0x7fcd3c10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/1692952924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c101af0 0x7fcd3c105b30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/1692952924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3c101140 0x7fcd3c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 -- 172.21.15.86:0/1692952924 >> 172.21.15.86:0/1692952924 conn(0x7fcd3c078000 msgr2=0x7fcd3c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:24.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 -- 172.21.15.86:0/1692952924 shutdown_connections 2024-04-16T05:41:24.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 -- 172.21.15.86:0/1692952924 wait complete. 2024-04-16T05:41:24.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 Processor -- start 2024-04-16T05:41:24.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 -- start start 2024-04-16T05:41:24.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.059+0000 7fcd42fa8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd3c101140 0x7fcd3c102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd42fa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c106210 0x7fcd3c103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd42fa8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3c103650 0x7fcd3c1a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd42fa8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1112a0 con 0x7fcd3c106210 2024-04-16T05:41:24.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd42fa8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c111120 con 0x7fcd3c101140 2024-04-16T05:41:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd42fa8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c111420 con 0x7fcd3c103650 2024-04-16T05:41:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd3bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c106210 0x7fcd3c103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd3bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c106210 0x7fcd3c103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56954/0 (socket says 172.21.15.86:56954) 2024-04-16T05:41:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd3bfff640 1 -- 172.21.15.86:0/2277753668 learned_addr learned my addr 172.21.15.86:0/2277753668 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd40d44640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd3c101140 0x7fcd3c102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd41545640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3c103650 0x7fcd3c1a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd3bfff640 1 -- 172.21.15.86:0/2277753668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3c103650 msgr2=0x7fcd3c1a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3c103650 0x7fcd3c1a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd3bfff640 1 -- 172.21.15.86:0/2277753668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd3c101140 msgr2=0x7fcd3c102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd3c101140 0x7fcd3c102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd3bfff640 1 -- 172.21.15.86:0/2277753668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c1a7200 con 0x7fcd3c106210 2024-04-16T05:41:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd41545640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3c103650 0x7fcd3c1a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd40d44640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd3c101140 0x7fcd3c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c106210 0x7fcd3c103110 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fcd28036040 tx=0x7fcd2802ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd39ffb640 1 -- 172.21.15.86:0/2277753668 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28041070 con 0x7fcd3c106210 2024-04-16T05:41:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd39ffb640 1 -- 172.21.15.86:0/2277753668 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd280054e0 con 0x7fcd3c106210 2024-04-16T05:41:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd42fa8640 1 -- 172.21.15.86:0/2277753668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1a7490 con 0x7fcd3c106210 2024-04-16T05:41:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.060+0000 7fcd39ffb640 1 -- 172.21.15.86:0/2277753668 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28034440 con 0x7fcd3c106210 2024-04-16T05:41:24.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.061+0000 7fcd42fa8640 1 -- 172.21.15.86:0/2277753668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c1a7980 con 0x7fcd3c106210 2024-04-16T05:41:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd39ffb640 1 -- 172.21.15.86:0/2277753668 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd28044070 con 0x7fcd3c106210 2024-04-16T05:41:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd45b98740 1 Processor -- start 2024-04-16T05:41:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd45b98740 1 -- start start 2024-04-16T05:41:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266941e50 0x561266942230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x56126694ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x56126694d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd45b98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612668a7180 con 0x56126694b350 2024-04-16T05:41:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd45b98740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561266844360 con 0x561266942770 2024-04-16T05:41:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd45b98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612668a3140 con 0x561266941e50 2024-04-16T05:41:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd3bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x56126694d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd3bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x56126694d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56964/0 (socket says 172.21.15.86:56964) 2024-04-16T05:41:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd3bfff640 1 -- 172.21.15.86:0/3605515622 learned_addr learned my addr 172.21.15.86:0/3605515622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.063+0000 7fcd39ffb640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd2c0a3d50 0x7fcd2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd41545640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd2c0a3d50 0x7fcd2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd40d44640 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x56126694ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd3bfff640 1 -- 172.21.15.86:0/3605515622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266941e50 msgr2=0x561266942230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd41545640 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266941e50 0x561266942230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266941e50 0x561266942230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd3bfff640 1 -- 172.21.15.86:0/3605515622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 msgr2=0x56126694ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x56126694ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd39ffb640 1 -- 172.21.15.86:0/2277753668 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcd280f2750 con 0x7fcd3c106210 2024-04-16T05:41:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd3bfff640 1 -- 172.21.15.86:0/3605515622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612666837d0 con 0x56126694b350 2024-04-16T05:41:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd40d44640 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x56126694ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd41545640 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266941e50 0x561266942230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x56126694d740 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fcd280be220 tx=0x7fcd2802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0364009f2ade7c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd26ffd640 1 -- 172.21.15.86:0/3605515622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2804b070 con 0x56126694b350 2024-04-16T05:41:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd26ffd640 1 -- 172.21.15.86:0/3605515622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd28005690 con 0x56126694b350 2024-04-16T05:41:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd26ffd640 1 -- 172.21.15.86:0/3605515622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd280035f0 con 0x56126694b350 2024-04-16T05:41:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.064+0000 7fcd41545640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd2c0a3d50 0x7fcd2c0a6210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd34005c40 tx=0x7fcd34009490 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 -- 172.21.15.86:0/3605515622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 msgr2=0x56126694d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x56126694d740 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fcd280be220 tx=0x7fcd2802bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 -- 172.21.15.86:0/3605515622 shutdown_connections 2024-04-16T05:41:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x56126694d740 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x56126694ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 --2- 172.21.15.86:0/3605515622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266941e50 0x561266942230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 -- 172.21.15.86:0/3605515622 >> 172.21.15.86:0/3605515622 conn(0x56126693c580 msgr2=0x56126693c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 -- 172.21.15.86:0/3605515622 shutdown_connections 2024-04-16T05:41:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 -- 172.21.15.86:0/3605515622 wait complete. 2024-04-16T05:41:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 -- start start 2024-04-16T05:41:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 Processor -- start 2024-04-16T05:41:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.065+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x5612669482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x561266948810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56126694b350 0x561266948d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd45b98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612668a7180 con 0x561266941e50 2024-04-16T05:41:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd45b98740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612668602c0 con 0x561266942770 2024-04-16T05:41:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd45b98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56126694df00 con 0x56126694b350 2024-04-16T05:41:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd45b98740 10 client.0 _ll_register_callbacks cb 0x561266847090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd40d44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x5612669482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd40d44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x5612669482d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56968/0 (socket says 172.21.15.86:56968) 2024-04-16T05:41:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd40d44640 1 -- 172.21.15.86:0/2680682434 learned_addr learned my addr 172.21.15.86:0/2680682434 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd41545640 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56126694b350 0x561266948d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x561266948810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd40d44640 1 -- 172.21.15.86:0/2680682434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56126694b350 msgr2=0x561266948d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd40d44640 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56126694b350 0x561266948d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd40d44640 1 -- 172.21.15.86:0/2680682434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 msgr2=0x561266948810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd40d44640 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x561266948810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd40d44640 1 -- 172.21.15.86:0/2680682434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612665b93f0 con 0x561266941e50 2024-04-16T05:41:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x561266948810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd40d44640 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x5612669482d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fcd300106a0 tx=0x7fcd30010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3d8a80bbad80be server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd1bfff640 1 -- 172.21.15.86:0/2680682434 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000e980 con 0x561266941e50 2024-04-16T05:41:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd1bfff640 1 -- 172.21.15.86:0/2680682434 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcd3000eb20 con 0x561266941e50 2024-04-16T05:41:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd45b98740 1 -- 172.21.15.86:0/2680682434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612668b4ba0 con 0x561266941e50 2024-04-16T05:41:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd1bfff640 1 -- 172.21.15.86:0/2680682434 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd30025050 con 0x561266941e50 2024-04-16T05:41:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd45b98740 10 client.17550 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.066+0000 7fcd45b98740 1 -- 172.21.15.86:0/2680682434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612665790f0 con 0x561266941e50 2024-04-16T05:41:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd1bfff640 1 -- 172.21.15.86:0/2680682434 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5612668b4ba0 con 0x561266941e50 2024-04-16T05:41:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd1bfff640 1 -- 172.21.15.86:0/2680682434 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd3000f040 con 0x561266941e50 2024-04-16T05:41:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd45b98740 10 client.17550 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd45b98740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd45b98740 20 client.17550 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd45b98740 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56126695b2c0 0x56126695d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd45b98740 4 client.17550 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd45b98740 1 -- 172.21.15.86:0/2680682434 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612668aea10 con 0x56126695b2c0 2024-04-16T05:41:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56126695b2c0 0x56126695d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56126695b2c0 0x56126695d750 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65d8161b03fa6a5 server_cookie=604e55a6ec7aa0a8 in_seq=0 out_seq=0 2024-04-16T05:41:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.067+0000 7fcd1bfff640 10 client.17550 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd1bfff640 1 -- 172.21.15.86:0/2680682434 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612668aea10 con 0x56126695b2c0 2024-04-16T05:41:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd1bfff640 10 client.17550 handle_command_reply: tid=0 2024-04-16T05:41:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 client.17550 shutdown 2024-04-16T05:41:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 -- 172.21.15.86:0/2680682434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 msgr2=0x5612669482d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x5612669482d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fcd300106a0 tx=0x7fcd30010aa0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 -- 172.21.15.86:0/2680682434 shutdown_connections 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56126695b2c0 0x56126695d750 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56126694b350 0x561266948d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266942770 0x561266948810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 --2- 172.21.15.86:0/2680682434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x5612669482d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd1bfff640 0 client.17550 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 -- 172.21.15.86:0/2680682434 >> 172.21.15.86:0/2680682434 conn(0x56126693c580 msgr2=0x56126693e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 -- 172.21.15.86:0/2680682434 shutdown_connections 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.068+0000 7fcd45b98740 1 -- 172.21.15.86:0/2680682434 wait complete. 2024-04-16T05:41:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.069+0000 7fcd45b98740 20 client.17550 trim_cache size 0 max 16384 2024-04-16T05:41:24.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd45b98740 1 Processor -- start 2024-04-16T05:41:24.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd45b98740 1 -- start start 2024-04-16T05:41:24.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x561266956c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266942770 0x561266948b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56126694b350 0x561266963020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd45b98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b16e0 con 0x561266941e50 2024-04-16T05:41:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd45b98740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b3220 con 0x56126694b350 2024-04-16T05:41:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd45b98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b1db0 con 0x561266942770 2024-04-16T05:41:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd40d44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x561266956c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd40d44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x561266956c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56978/0 (socket says 172.21.15.86:56978) 2024-04-16T05:41:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd40d44640 1 -- 172.21.15.86:0/381977280 learned_addr learned my addr 172.21.15.86:0/381977280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd41545640 1 --2- 172.21.15.86:0/381977280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56126694b350 0x561266963020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd40d44640 1 -- 172.21.15.86:0/381977280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266942770 msgr2=0x561266948b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd40d44640 1 --2- 172.21.15.86:0/381977280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266942770 0x561266948b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd40d44640 1 -- 172.21.15.86:0/381977280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56126694b350 msgr2=0x561266963020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd40d44640 1 --2- 172.21.15.86:0/381977280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56126694b350 0x561266963020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd40d44640 1 -- 172.21.15.86:0/381977280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56126693c3b0 con 0x561266941e50 2024-04-16T05:41:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd41545640 1 --2- 172.21.15.86:0/381977280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56126694b350 0x561266963020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.086+0000 7fcd40d44640 1 --2- 172.21.15.86:0/381977280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x561266956c40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fcd30011fd0 tx=0x7fcd3000b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd7b1b5144cb510 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd267fc640 1 -- 172.21.15.86:0/381977280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3002d070 con 0x561266941e50 2024-04-16T05:41:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd267fc640 1 -- 172.21.15.86:0/381977280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd3000ec60 con 0x561266941e50 2024-04-16T05:41:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd267fc640 1 -- 172.21.15.86:0/381977280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30012450 con 0x561266941e50 2024-04-16T05:41:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 -- 172.21.15.86:0/381977280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 msgr2=0x561266956c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 --2- 172.21.15.86:0/381977280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x561266956c40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fcd30011fd0 tx=0x7fcd3000b320 comp rx=0 tx=0).stop 2024-04-16T05:41:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 -- 172.21.15.86:0/381977280 shutdown_connections 2024-04-16T05:41:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 --2- 172.21.15.86:0/381977280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56126694b350 0x561266963020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 --2- 172.21.15.86:0/381977280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266942770 0x561266948b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 --2- 172.21.15.86:0/381977280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561266941e50 0x561266956c40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 -- 172.21.15.86:0/381977280 >> 172.21.15.86:0/381977280 conn(0x56126693c580 msgr2=0x56126693b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 -- 172.21.15.86:0/381977280 shutdown_connections 2024-04-16T05:41:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 -- 172.21.15.86:0/381977280 wait complete. 2024-04-16T05:41:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.087+0000 7fcd45b98740 1 -- start start 2024-04-16T05:41:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd45b98740 1 Processor -- start 2024-04-16T05:41:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266941e50 0x561266953300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266942770 0x561266950720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd45b98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x561266950c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd45b98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b2b50 con 0x56126694b350 2024-04-16T05:41:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd45b98740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b2480 con 0x561266941e50 2024-04-16T05:41:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd45b98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1af6a0 con 0x561266942770 2024-04-16T05:41:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd45b98740 10 client.0 _ll_register_callbacks cb 0x7fcd3000b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd41545640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x561266950c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd41545640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x561266950c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56988/0 (socket says 172.21.15.86:56988) 2024-04-16T05:41:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd41545640 1 -- 172.21.15.86:0/4134194915 learned_addr learned my addr 172.21.15.86:0/4134194915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd40d44640 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266941e50 0x561266953300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd41545640 1 -- 172.21.15.86:0/4134194915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266942770 msgr2=0x561266950720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd3bfff640 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266942770 0x561266950720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd41545640 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266942770 0x561266950720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd41545640 1 -- 172.21.15.86:0/4134194915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266941e50 msgr2=0x561266953300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd41545640 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266941e50 0x561266953300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd41545640 1 -- 172.21.15.86:0/4134194915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561266951250 con 0x56126694b350 2024-04-16T05:41:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.088+0000 7fcd40d44640 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266941e50 0x561266953300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd41545640 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x561266950c60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fcd34020c70 tx=0x7fcd34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382f227adc45a57c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd1bfff640 1 -- 172.21.15.86:0/4134194915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34004d20 con 0x56126694b350 2024-04-16T05:41:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd45b98740 1 -- 172.21.15.86:0/4134194915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd3000ec60 con 0x56126694b350 2024-04-16T05:41:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd1bfff640 1 -- 172.21.15.86:0/4134194915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcd3401e660 con 0x56126694b350 2024-04-16T05:41:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd1bfff640 1 -- 172.21.15.86:0/4134194915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd34031050 con 0x56126694b350 2024-04-16T05:41:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd45b98740 10 client.17556 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd45b98740 1 -- 172.21.15.86:0/4134194915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612669c1660 con 0x56126694b350 2024-04-16T05:41:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd1bfff640 1 -- 172.21.15.86:0/4134194915 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd3000ec60 con 0x56126694b350 2024-04-16T05:41:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd1bfff640 1 -- 172.21.15.86:0/4134194915 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd34027030 con 0x56126694b350 2024-04-16T05:41:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd45b98740 10 client.17556 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:24.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd45b98740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:24.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd45b98740 20 client.17556 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:24.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd45b98740 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5612669c1e10 0x5612669c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:24.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd45b98740 4 client.17556 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:41:24.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd45b98740 1 -- 172.21.15.86:0/4134194915 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56126682a560 con 0x5612669c1e10 2024-04-16T05:41:24.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.089+0000 7fcd40d44640 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5612669c1e10 0x5612669c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:24.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd1bfff640 10 client.17556 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd40d44640 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5612669c1e10 0x5612669c4230 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb36f0291afc2bc server_cookie=a8881a865cecf6cb in_seq=0 out_seq=0 2024-04-16T05:41:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd1bfff640 1 -- 172.21.15.86:0/4134194915 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56126682a560 con 0x5612669c1e10 2024-04-16T05:41:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd1bfff640 10 client.17556 handle_command_reply: tid=0 2024-04-16T05:41:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 client.17556 shutdown 2024-04-16T05:41:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 -- 172.21.15.86:0/4134194915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 msgr2=0x561266950c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x561266950c60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fcd34020c70 tx=0x7fcd34004ab0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 -- 172.21.15.86:0/4134194915 shutdown_connections 2024-04-16T05:41:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5612669c1e10 0x5612669c4230 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56126694b350 0x561266950c60 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd1bfff640 0 client.17556 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561266942770 0x561266950720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 --2- 172.21.15.86:0/4134194915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561266941e50 0x561266953300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 -- 172.21.15.86:0/4134194915 >> 172.21.15.86:0/4134194915 conn(0x56126693c580 msgr2=0x56126693d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 -- 172.21.15.86:0/4134194915 shutdown_connections 2024-04-16T05:41:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.090+0000 7fcd45b98740 1 -- 172.21.15.86:0/4134194915 wait complete. 2024-04-16T05:41:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd45b98740 20 client.17556 trim_cache size 0 max 16384 2024-04-16T05:41:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:41:24.101 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:41:24.101 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:41:24.101 INFO:teuthology.orchestra.run.smithi086.stdout: "f31e0ea5-6c90-4479-897e-774306d43a57": { 2024-04-16T05:41:24.101 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:41:24.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "f31e0ea5-6c90-4479-897e-774306d43a57", 2024-04-16T05:41:24.101 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:41:24.101 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:41:24.102 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:41:24.102 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:41:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 -- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd2c0a3d50 msgr2=0x7fcd2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd2c0a3d50 0x7fcd2c0a6210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd34005c40 tx=0x7fcd34009490 comp rx=0 tx=0).stop 2024-04-16T05:41:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 -- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c106210 msgr2=0x7fcd3c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c106210 0x7fcd3c103110 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fcd28036040 tx=0x7fcd2802ce20 comp rx=0 tx=0).stop 2024-04-16T05:41:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 -- 172.21.15.86:0/2277753668 shutdown_connections 2024-04-16T05:41:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3c103650 0x7fcd3c1a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd3c106210 0x7fcd3c103110 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fcd2c0a3d50 0x7fcd2c0a6210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 --2- 172.21.15.86:0/2277753668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd3c101140 0x7fcd3c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 -- 172.21.15.86:0/2277753668 >> 172.21.15.86:0/2277753668 conn(0x7fcd3c078000 msgr2=0x7fcd3c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 -- 172.21.15.86:0/2277753668 shutdown_connections 2024-04-16T05:41:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:24.091+0000 7fcd42fa8640 1 -- 172.21.15.86:0/2277753668 wait complete. 2024-04-16T05:41:24.105 INFO:tasks.cephfs.filesystem:scrub status for tag:f31e0ea5-6c90-4479-897e-774306d43a57 - {'path': '/', 'tag': 'f31e0ea5-6c90-4479-897e-774306d43a57', 'options': 'recursive,force'} 2024-04-16T05:41:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:24 smithi194 ceph-mon[35865]: pgmap v186: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:41:24.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:24 smithi086 ceph-mon[32028]: pgmap v186: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:41:24.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:24 smithi196 ceph-mon[36373]: pgmap v186: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:41:25.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:25 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:41:25.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:25 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:25.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:25 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:41:25.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:25 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:25.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:25 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:41:25.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:25 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:41:25.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:25 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:25.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:25 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:41:25.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:25 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:25.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:25 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:41:25.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:25 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:41:25.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:25 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:25.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:25 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:41:25.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:25 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:25.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:25 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:41:26.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:26 smithi086 ceph-mon[32028]: pgmap v187: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:26.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:26 smithi196 ceph-mon[36373]: pgmap v187: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:26.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:26 smithi194 ceph-mon[35865]: pgmap v187: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:27.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:27 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:41:27.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:27 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:41:27.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:27 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:41:28.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:28 smithi086 ceph-mon[32028]: pgmap v188: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:28.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:41:28.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:41:28.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:41:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:28 smithi196 ceph-mon[36373]: pgmap v188: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:41:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:41:28.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:41:28.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:28 smithi194 ceph-mon[35865]: pgmap v188: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:28.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:41:28.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:41:28.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:41:30.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:30 smithi086 ceph-mon[32028]: pgmap v189: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:30.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:30 smithi196 ceph-mon[36373]: pgmap v189: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:30.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:30 smithi194 ceph-mon[35865]: pgmap v189: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:32.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:32 smithi086 ceph-mon[32028]: pgmap v190: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:32 smithi196 ceph-mon[36373]: pgmap v190: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:32 smithi194 ceph-mon[35865]: pgmap v190: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:34.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:34 smithi086 ceph-mon[32028]: pgmap v191: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:34.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:34 smithi196 ceph-mon[36373]: pgmap v191: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:34.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:34 smithi194 ceph-mon[35865]: pgmap v191: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:36.708 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:36 smithi194 ceph-mon[35865]: pgmap v192: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:36.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:36 smithi086 ceph-mon[32028]: pgmap v192: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:36 smithi196 ceph-mon[36373]: pgmap v192: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:38.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:38 smithi086 ceph-mon[32028]: pgmap v193: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:38.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:38 smithi196 ceph-mon[36373]: pgmap v193: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:38.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:38 smithi194 ceph-mon[35865]: pgmap v193: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:41:40.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:40 smithi086 ceph-mon[32028]: pgmap v194: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:40.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:40 smithi196 ceph-mon[36373]: pgmap v194: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:40.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:40 smithi194 ceph-mon[35865]: pgmap v194: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:41:42.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:42 smithi086 ceph-mon[32028]: pgmap v195: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:42.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:42 smithi196 ceph-mon[36373]: pgmap v195: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:42.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:42 smithi194 ceph-mon[35865]: pgmap v195: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:44.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:44 smithi086 ceph-mon[32028]: pgmap v196: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:41:44.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:44 smithi196 ceph-mon[36373]: pgmap v196: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:41:44.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:44 smithi194 ceph-mon[35865]: pgmap v196: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:41:46.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:46 smithi086 ceph-mon[32028]: pgmap v197: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:46.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:46 smithi196 ceph-mon[36373]: pgmap v197: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:46.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:46 smithi194 ceph-mon[35865]: pgmap v197: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:48.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:48 smithi086 ceph-mon[32028]: pgmap v198: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:48.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:48 smithi196 ceph-mon[36373]: pgmap v198: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:48.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:48 smithi194 ceph-mon[35865]: pgmap v198: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:50.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:50 smithi086 ceph-mon[32028]: pgmap v199: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:41:50.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:50 smithi196 ceph-mon[36373]: pgmap v199: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:41:50.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:50 smithi194 ceph-mon[35865]: pgmap v199: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:41:52.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:52 smithi086 ceph-mon[32028]: pgmap v200: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:52.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:52 smithi196 ceph-mon[36373]: pgmap v200: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:52.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:52 smithi194 ceph-mon[35865]: pgmap v200: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:54.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:41:54.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:41:54.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.226+0000 7fa99510e640 1 -- 172.21.15.86:0/922398406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa990107330 msgr2=0x7fa99010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.226+0000 7fa99510e640 1 --2- 172.21.15.86:0/922398406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa990107330 0x7fa99010eee0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa988009ca0 tx=0x7fa98802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.227+0000 7fa99510e640 1 -- 172.21.15.86:0/922398406 shutdown_connections 2024-04-16T05:41:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.227+0000 7fa99510e640 1 --2- 172.21.15.86:0/922398406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa990107330 0x7fa99010eee0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.227+0000 7fa99510e640 1 --2- 172.21.15.86:0/922398406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa99006b220 0x7fa990106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.227+0000 7fa99510e640 1 --2- 172.21.15.86:0/922398406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa99006a870 0x7fa99006ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.227+0000 7fa99510e640 1 -- 172.21.15.86:0/922398406 >> 172.21.15.86:0/922398406 conn(0x7fa990076d00 msgr2=0x7fa990077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.227+0000 7fa99510e640 1 -- 172.21.15.86:0/922398406 shutdown_connections 2024-04-16T05:41:54.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.227+0000 7fa99510e640 1 -- 172.21.15.86:0/922398406 wait complete. 2024-04-16T05:41:54.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.227+0000 7fa99510e640 1 Processor -- start 2024-04-16T05:41:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa99510e640 1 -- start start 2024-04-16T05:41:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa99510e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99006a870 0x7fa99019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa99510e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa990107330 0x7fa99019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa99510e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990198680 0x7fa990198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa99510e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901123e0 con 0x7fa99006a870 2024-04-16T05:41:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa99510e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa990112260 con 0x7fa990198680 2024-04-16T05:41:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa99510e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa990112560 con 0x7fa990107330 2024-04-16T05:41:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99006a870 0x7fa99019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99006a870 0x7fa99019e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41708/0 (socket says 172.21.15.86:41708) 2024-04-16T05:41:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98ed9d640 1 -- 172.21.15.86:0/2532614580 learned_addr learned my addr 172.21.15.86:0/2532614580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98f59e640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990198680 0x7fa990198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98e59c640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa990107330 0x7fa99019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98ed9d640 1 -- 172.21.15.86:0/2532614580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa990107330 msgr2=0x7fa99019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa990107330 0x7fa99019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98ed9d640 1 -- 172.21.15.86:0/2532614580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990198680 msgr2=0x7fa990198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990198680 0x7fa990198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98ed9d640 1 -- 172.21.15.86:0/2532614580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa990199280 con 0x7fa99006a870 2024-04-16T05:41:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.228+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99006a870 0x7fa99019e570 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa98000d950 tx=0x7fa98000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.229+0000 7fa977fff640 1 -- 172.21.15.86:0/2532614580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980017030 con 0x7fa99006a870 2024-04-16T05:41:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.229+0000 7fa977fff640 1 -- 172.21.15.86:0/2532614580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa980004020 con 0x7fa99006a870 2024-04-16T05:41:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.229+0000 7fa99510e640 1 -- 172.21.15.86:0/2532614580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9901a5310 con 0x7fa99006a870 2024-04-16T05:41:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.229+0000 7fa977fff640 1 -- 172.21.15.86:0/2532614580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9800047a0 con 0x7fa99006a870 2024-04-16T05:41:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.229+0000 7fa99510e640 1 -- 172.21.15.86:0/2532614580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9901a5710 con 0x7fa99006a870 2024-04-16T05:41:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.231+0000 7fa977fff640 1 -- 172.21.15.86:0/2532614580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9800041d0 con 0x7fa99006a870 2024-04-16T05:41:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.231+0000 7fa997dd1740 1 Processor -- start 2024-04-16T05:41:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.231+0000 7fa997dd1740 1 -- start start 2024-04-16T05:41:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x56158999f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 0x5615899a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615899a8350 0x5615899aa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa997dd1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561589904180 con 0x56158999ee50 2024-04-16T05:41:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa997dd1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5615898a1360 con 0x5615899a8350 2024-04-16T05:41:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa977fff640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa9780a3d20 0x7fa9780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa997dd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561589900140 con 0x56158999f770 2024-04-16T05:41:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x56158999f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x56158999f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41714/0 (socket says 172.21.15.86:41714) 2024-04-16T05:41:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98f59e640 1 -- 172.21.15.86:0/613038161 learned_addr learned my addr 172.21.15.86:0/613038161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98f59e640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa9780a3d20 0x7fa9780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa977fff640 1 -- 172.21.15.86:0/2532614580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa9800d0610 con 0x7fa99006a870 2024-04-16T05:41:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/613038161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 0x5615899a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98e59c640 1 --2- 172.21.15.86:0/613038161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615899a8350 0x5615899aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98ed9d640 1 -- 172.21.15.86:0/613038161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615899a8350 msgr2=0x5615899aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/613038161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615899a8350 0x5615899aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98ed9d640 1 -- 172.21.15.86:0/613038161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 msgr2=0x56158999f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/613038161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x56158999f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98ed9d640 1 -- 172.21.15.86:0/613038161 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615896e07d0 con 0x56158999f770 2024-04-16T05:41:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98f59e640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa9780a3d20 0x7fa9780a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa988009a60 tx=0x7fa988005ef0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.232+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/613038161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 0x5615899a7e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9800cd9f0 tx=0x7fa980098600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5470dcc029976cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa9757fa640 1 -- 172.21.15.86:0/613038161 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9800b5070 con 0x56158999f770 2024-04-16T05:41:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa9757fa640 1 -- 172.21.15.86:0/613038161 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa98009c940 con 0x56158999f770 2024-04-16T05:41:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa9757fa640 1 -- 172.21.15.86:0/613038161 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9800988b0 con 0x56158999f770 2024-04-16T05:41:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa997dd1740 1 -- 172.21.15.86:0/613038161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 msgr2=0x5615899a7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa997dd1740 1 --2- 172.21.15.86:0/613038161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 0x5615899a7e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9800cd9f0 tx=0x7fa980098600 comp rx=0 tx=0).stop 2024-04-16T05:41:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa997dd1740 1 -- 172.21.15.86:0/613038161 shutdown_connections 2024-04-16T05:41:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa997dd1740 1 --2- 172.21.15.86:0/613038161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615899a8350 0x5615899aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa997dd1740 1 --2- 172.21.15.86:0/613038161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 0x5615899a7e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa997dd1740 1 --2- 172.21.15.86:0/613038161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x56158999f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa997dd1740 1 -- 172.21.15.86:0/613038161 >> 172.21.15.86:0/613038161 conn(0x561589999580 msgr2=0x561589999960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa997dd1740 1 -- 172.21.15.86:0/613038161 shutdown_connections 2024-04-16T05:41:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.233+0000 7fa997dd1740 1 -- 172.21.15.86:0/613038161 wait complete. 2024-04-16T05:41:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa997dd1740 1 -- start start 2024-04-16T05:41:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa997dd1740 1 Processor -- start 2024-04-16T05:41:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999ee50 0x5615899a5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999f770 0x5615899a5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa997dd1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615899ab140 con 0x5615899a8350 2024-04-16T05:41:54.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa997dd1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5615898bd2c0 con 0x56158999f770 2024-04-16T05:41:54.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa997dd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615899ab2c0 con 0x56158999ee50 2024-04-16T05:41:54.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa997dd1740 10 client.0 _ll_register_callbacks cb 0x5615898a4090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:54.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41726/0 (socket says 172.21.15.86:41726) 2024-04-16T05:41:54.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98f59e640 1 -- 172.21.15.86:0/1156705987 learned_addr learned my addr 172.21.15.86:0/1156705987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999ee50 0x5615899a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98e59c640 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999f770 0x5615899a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98f59e640 1 -- 172.21.15.86:0/1156705987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999ee50 msgr2=0x5615899a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98f59e640 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999ee50 0x5615899a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98f59e640 1 -- 172.21.15.86:0/1156705987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999f770 msgr2=0x5615899a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98f59e640 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999f770 0x5615899a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98f59e640 1 -- 172.21.15.86:0/1156705987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615896163f0 con 0x5615899a8350 2024-04-16T05:41:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.234+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999ee50 0x5615899a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa98f59e640 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899a5ed0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa988004af0 tx=0x7fa98802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90218330cffe2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa972ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa972ff5640 1 -- 172.21.15.86:0/1156705987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988039040 con 0x5615899a8350 2024-04-16T05:41:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa972ff5640 1 -- 172.21.15.86:0/1156705987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa988003e00 con 0x5615899a8350 2024-04-16T05:41:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa997dd1740 1 -- 172.21.15.86:0/1156705987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa98009c940 con 0x5615899a8350 2024-04-16T05:41:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa972ff5640 1 -- 172.21.15.86:0/1156705987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa988004b40 con 0x5615899a8350 2024-04-16T05:41:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa997dd1740 10 client.17568 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa997dd1740 1 -- 172.21.15.86:0/1156705987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615895d60f0 con 0x5615899a8350 2024-04-16T05:41:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa972ff5640 1 -- 172.21.15.86:0/1156705987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa98009c940 con 0x5615899a8350 2024-04-16T05:41:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa972ff5640 1 -- 172.21.15.86:0/1156705987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa98804e8f0 con 0x5615899a8350 2024-04-16T05:41:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa997dd1740 10 client.17568 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:54.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa997dd1740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:54.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa997dd1740 20 client.17568 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:54.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa997dd1740 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5615899b8850 0x5615899bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa997dd1740 4 client.17568 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:54.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.235+0000 7fa997dd1740 1 -- 172.21.15.86:0/1156705987 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561589911ba0 con 0x5615899b8850 2024-04-16T05:41:54.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.236+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5615899b8850 0x5615899bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.236+0000 7fa972ff5640 10 client.17568 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:54.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.236+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5615899b8850 0x5615899bacc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36794964cf9ea15f server_cookie=85312108a41d3a1f in_seq=0 out_seq=0 2024-04-16T05:41:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa972ff5640 1 -- 172.21.15.86:0/1156705987 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561589911ba0 con 0x5615899b8850 2024-04-16T05:41:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa972ff5640 10 client.17568 handle_command_reply: tid=0 2024-04-16T05:41:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 client.17568 shutdown 2024-04-16T05:41:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 -- 172.21.15.86:0/1156705987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 msgr2=0x5615899a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899a5ed0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa988004af0 tx=0x7fa98802e040 comp rx=0 tx=0).stop 2024-04-16T05:41:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 -- 172.21.15.86:0/1156705987 shutdown_connections 2024-04-16T05:41:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5615899b8850 0x5615899bacc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899a5ed0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999f770 0x5615899a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 --2- 172.21.15.86:0/1156705987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999ee50 0x5615899a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa972ff5640 0 client.17568 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 -- 172.21.15.86:0/1156705987 >> 172.21.15.86:0/1156705987 conn(0x561589999580 msgr2=0x56158999b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 -- 172.21.15.86:0/1156705987 shutdown_connections 2024-04-16T05:41:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 1 -- 172.21.15.86:0/1156705987 wait complete. 2024-04-16T05:41:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.237+0000 7fa997dd1740 20 client.17568 trim_cache size 0 max 16384 2024-04-16T05:41:54.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.254+0000 7fa997dd1740 1 Processor -- start 2024-04-16T05:41:54.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.254+0000 7fa997dd1740 1 -- start start 2024-04-16T05:41:54.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x5615899a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999f770 0x5615899b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615899a8350 0x5615899a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa997dd1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901af350 con 0x56158999ee50 2024-04-16T05:41:54.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa997dd1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901b0e90 con 0x56158999f770 2024-04-16T05:41:54.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa997dd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901afa20 con 0x5615899a8350 2024-04-16T05:41:54.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa98ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x5615899a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa98ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x5615899a7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41728/0 (socket says 172.21.15.86:41728) 2024-04-16T05:41:54.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa98ed9d640 1 -- 172.21.15.86:0/712249276 learned_addr learned my addr 172.21.15.86:0/712249276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa98ed9d640 1 -- 172.21.15.86:0/712249276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615899a8350 msgr2=0x5615899a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/712249276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615899a8350 0x5615899a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa98ed9d640 1 -- 172.21.15.86:0/712249276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999f770 msgr2=0x5615899b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/712249276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999f770 0x5615899b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa98ed9d640 1 -- 172.21.15.86:0/712249276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615899aaa70 con 0x56158999ee50 2024-04-16T05:41:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/712249276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x5615899a7b80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa98009bf40 tx=0x7fa9800a5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8393a68516e683f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa974ff9640 1 -- 172.21.15.86:0/712249276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9800b5040 con 0x56158999ee50 2024-04-16T05:41:54.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa974ff9640 1 -- 172.21.15.86:0/712249276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa98009c940 con 0x56158999ee50 2024-04-16T05:41:54.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.255+0000 7fa974ff9640 1 -- 172.21.15.86:0/712249276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9800988b0 con 0x56158999ee50 2024-04-16T05:41:54.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 -- 172.21.15.86:0/712249276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 msgr2=0x5615899a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 --2- 172.21.15.86:0/712249276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x5615899a7b80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa98009bf40 tx=0x7fa9800a5000 comp rx=0 tx=0).stop 2024-04-16T05:41:54.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 -- 172.21.15.86:0/712249276 shutdown_connections 2024-04-16T05:41:54.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 --2- 172.21.15.86:0/712249276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615899a8350 0x5615899a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 --2- 172.21.15.86:0/712249276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999f770 0x5615899b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 --2- 172.21.15.86:0/712249276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56158999ee50 0x5615899a7b80 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 -- 172.21.15.86:0/712249276 >> 172.21.15.86:0/712249276 conn(0x561589999580 msgr2=0x561589997ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 -- 172.21.15.86:0/712249276 shutdown_connections 2024-04-16T05:41:54.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 -- 172.21.15.86:0/712249276 wait complete. 2024-04-16T05:41:54.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 -- start start 2024-04-16T05:41:54.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 Processor -- start 2024-04-16T05:41:54.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999ee50 0x5615899b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 0x5615899adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901b07c0 con 0x5615899a8350 2024-04-16T05:41:54.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901b00f0 con 0x56158999ee50 2024-04-16T05:41:54.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901ad310 con 0x56158999f770 2024-04-16T05:41:54.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.256+0000 7fa997dd1740 10 client.0 _ll_register_callbacks cb 0x7fa988009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:54.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41730/0 (socket says 172.21.15.86:41730) 2024-04-16T05:41:54.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98f59e640 1 -- 172.21.15.86:0/3134255520 learned_addr learned my addr 172.21.15.86:0/3134255520 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98e59c640 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 0x5615899adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999ee50 0x5615899b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98f59e640 1 -- 172.21.15.86:0/3134255520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 msgr2=0x5615899adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98f59e640 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 0x5615899adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98f59e640 1 -- 172.21.15.86:0/3134255520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999ee50 msgr2=0x5615899b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98f59e640 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999ee50 0x5615899b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98f59e640 1 -- 172.21.15.86:0/3134255520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615899ae620 con 0x5615899a8350 2024-04-16T05:41:54.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa98f59e640 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899adfe0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa988009ca0 tx=0x7fa98804ffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500e1e935dbba9da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa972ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:54.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa972ff5640 1 -- 172.21.15.86:0/3134255520 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988039040 con 0x5615899a8350 2024-04-16T05:41:54.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa972ff5640 1 -- 172.21.15.86:0/3134255520 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa98804a050 con 0x5615899a8350 2024-04-16T05:41:54.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa997dd1740 1 -- 172.21.15.86:0/3134255520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa98009c940 con 0x5615899a8350 2024-04-16T05:41:54.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa972ff5640 1 -- 172.21.15.86:0/3134255520 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa988004250 con 0x5615899a8350 2024-04-16T05:41:54.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa997dd1740 10 client.17574 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa997dd1740 1 -- 172.21.15.86:0/3134255520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561589a1ec90 con 0x5615899a8350 2024-04-16T05:41:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.257+0000 7fa972ff5640 1 -- 172.21.15.86:0/3134255520 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa98009c940 con 0x5615899a8350 2024-04-16T05:41:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa972ff5640 1 -- 172.21.15.86:0/3134255520 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa988060480 con 0x5615899a8350 2024-04-16T05:41:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa997dd1740 10 client.17574 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa997dd1740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa997dd1740 20 client.17574 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa997dd1740 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561589a1f3d0 0x561589a217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa997dd1740 4 client.17574 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:41:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa997dd1740 1 -- 172.21.15.86:0/3134255520 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56158990ba10 con 0x561589a1f3d0 2024-04-16T05:41:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561589a1f3d0 0x561589a217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa98ed9d640 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561589a1f3d0 0x561589a217f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b0aeb5c679ac8d1 server_cookie=5548a03850fc97e9 in_seq=0 out_seq=0 2024-04-16T05:41:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.258+0000 7fa972ff5640 10 client.17574 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa972ff5640 1 -- 172.21.15.86:0/3134255520 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56158990ba10 con 0x561589a1f3d0 2024-04-16T05:41:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa972ff5640 10 client.17574 handle_command_reply: tid=0 2024-04-16T05:41:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 client.17574 shutdown 2024-04-16T05:41:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 -- 172.21.15.86:0/3134255520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 msgr2=0x5615899adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899adfe0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa988009ca0 tx=0x7fa98804ffe0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 -- 172.21.15.86:0/3134255520 shutdown_connections 2024-04-16T05:41:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561589a1f3d0 0x561589a217f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615899a8350 0x5615899adfe0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56158999f770 0x5615899adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa972ff5640 0 client.17574 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 --2- 172.21.15.86:0/3134255520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56158999ee50 0x5615899b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 -- 172.21.15.86:0/3134255520 >> 172.21.15.86:0/3134255520 conn(0x561589999580 msgr2=0x561589998760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 -- 172.21.15.86:0/3134255520 shutdown_connections 2024-04-16T05:41:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 1 -- 172.21.15.86:0/3134255520 wait complete. 2024-04-16T05:41:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa997dd1740 20 client.17574 trim_cache size 0 max 16384 2024-04-16T05:41:54.272 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:41:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:41:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:41:54.272 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:41:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 -- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa9780a3d20 msgr2=0x7fa9780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa9780a3d20 0x7fa9780a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa988009a60 tx=0x7fa988005ef0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 -- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99006a870 msgr2=0x7fa99019e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99006a870 0x7fa99019e570 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa98000d950 tx=0x7fa98000de10 comp rx=0 tx=0).stop 2024-04-16T05:41:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 -- 172.21.15.86:0/2532614580 shutdown_connections 2024-04-16T05:41:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fa9780a3d20 0x7fa9780a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990198680 0x7fa990198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa990107330 0x7fa99019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 --2- 172.21.15.86:0/2532614580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99006a870 0x7fa99019e570 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 -- 172.21.15.86:0/2532614580 >> 172.21.15.86:0/2532614580 conn(0x7fa990076d00 msgr2=0x7fa9900ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 -- 172.21.15.86:0/2532614580 shutdown_connections 2024-04-16T05:41:54.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.259+0000 7fa99510e640 1 -- 172.21.15.86:0/2532614580 wait complete. 2024-04-16T05:41:54.275 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:41:54.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:41:54.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.381+0000 7f48eaf7c640 1 -- 172.21.15.86:0/2471340379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4100bc0 msgr2=0x7f48e4100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.381+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/2471340379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4100bc0 0x7f48e4100fa0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009900 tx=0x7f48d802bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:54.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.381+0000 7f48eaf7c640 1 -- 172.21.15.86:0/2471340379 shutdown_connections 2024-04-16T05:41:54.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.381+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/2471340379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48e4107380 0x7f48e410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.381+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/2471340379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4101570 0x7f48e4106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.381+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/2471340379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4100bc0 0x7f48e4100fa0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.381+0000 7f48eaf7c640 1 -- 172.21.15.86:0/2471340379 >> 172.21.15.86:0/2471340379 conn(0x7f48e40761e0 msgr2=0x7f48e40765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.381+0000 7f48eaf7c640 1 -- 172.21.15.86:0/2471340379 shutdown_connections 2024-04-16T05:41:54.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.381+0000 7f48eaf7c640 1 -- 172.21.15.86:0/2471340379 wait complete. 2024-04-16T05:41:54.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48eaf7c640 1 Processor -- start 2024-04-16T05:41:54.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48eaf7c640 1 -- start start 2024-04-16T05:41:54.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48eaf7c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4100bc0 0x7f48e419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48eaf7c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4107380 0x7f48e419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48eaf7c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48e4198710 0x7f48e4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48eaf7c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4112430 con 0x7f48e4107380 2024-04-16T05:41:54.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48eaf7c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f48e41122b0 con 0x7f48e4198710 2024-04-16T05:41:54.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48eaf7c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48e41125b0 con 0x7f48e4100bc0 2024-04-16T05:41:54.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4107380 0x7f48e419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4107380 0x7f48e419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41758/0 (socket says 172.21.15.86:41758) 2024-04-16T05:41:54.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e3fff640 1 -- 172.21.15.86:0/1753806696 learned_addr learned my addr 172.21.15.86:0/1753806696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e8d18640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4100bc0 0x7f48e419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e9519640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48e4198710 0x7f48e4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e3fff640 1 -- 172.21.15.86:0/1753806696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4100bc0 msgr2=0x7f48e419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e3fff640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4100bc0 0x7f48e419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e3fff640 1 -- 172.21.15.86:0/1753806696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48e4198710 msgr2=0x7f48e4198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e3fff640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48e4198710 0x7f48e4198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e3fff640 1 -- 172.21.15.86:0/1753806696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e4199310 con 0x7f48e4107380 2024-04-16T05:41:54.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.382+0000 7f48e9519640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48e4198710 0x7f48e4198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:54.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.383+0000 7f48e3fff640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4107380 0x7f48e419eb90 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f48d000d770 tx=0x7f48d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.383+0000 7f48e1ffb640 1 -- 172.21.15.86:0/1753806696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0017070 con 0x7f48e4107380 2024-04-16T05:41:54.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.383+0000 7f48e1ffb640 1 -- 172.21.15.86:0/1753806696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d0009c30 con 0x7f48e4107380 2024-04-16T05:41:54.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.383+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e41a53f0 con 0x7f48e4107380 2024-04-16T05:41:54.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.383+0000 7f48e1ffb640 1 -- 172.21.15.86:0/1753806696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d000f620 con 0x7f48e4107380 2024-04-16T05:41:54.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.383+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e41a57f0 con 0x7f48e4107380 2024-04-16T05:41:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.384+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e406a190 con 0x7f48e4107380 2024-04-16T05:41:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.385+0000 7f48e1ffb640 1 -- 172.21.15.86:0/1753806696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f48d001a070 con 0x7f48e4107380 2024-04-16T05:41:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.385+0000 7f48e1ffb640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48d40a3d20 0x7f48d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.386+0000 7f48e8d18640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48d40a3d20 0x7f48d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.386+0000 7f48e1ffb640 1 -- 172.21.15.86:0/1753806696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f48d00d0690 con 0x7f48e4107380 2024-04-16T05:41:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.386+0000 7f48e8d18640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48d40a3d20 0x7f48d40a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006010 tx=0x7f48d8005dd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.390+0000 7f48e1ffb640 1 -- 172.21.15.86:0/1753806696 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48d0009de0 con 0x7f48e4107380 2024-04-16T05:41:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.608+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48e4069270 con 0x7f48e4107380 2024-04-16T05:41:54.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.609+0000 7f48e1ffb640 1 -- 172.21.15.86:0/1753806696 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f48d009c9c0 con 0x7f48e4107380 2024-04-16T05:41:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:41:54.609 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:41:54.609 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:41:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48d40a3d20 msgr2=0x7f48d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48d40a3d20 0x7f48d40a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006010 tx=0x7f48d8005dd0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4107380 msgr2=0x7f48e419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4107380 0x7f48e419eb90 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f48d000d770 tx=0x7f48d000dc30 comp rx=0 tx=0).stop 2024-04-16T05:41:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 shutdown_connections 2024-04-16T05:41:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f48d40a3d20 0x7f48d40a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48e4198710 0x7f48e4198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48e4107380 0x7f48e419eb90 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 --2- 172.21.15.86:0/1753806696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4100bc0 0x7f48e419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 >> 172.21.15.86:0/1753806696 conn(0x7f48e40761e0 msgr2=0x7f48e40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 shutdown_connections 2024-04-16T05:41:54.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.611+0000 7f48eaf7c640 1 -- 172.21.15.86:0/1753806696 wait complete. 2024-04-16T05:41:54.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:41:54.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:54 smithi086 ceph-mon[32028]: pgmap v201: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:41:54.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.728+0000 7f55fc1b5640 1 -- 172.21.15.86:0/1149781036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102c00 msgr2=0x7f55f4106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.728+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/1149781036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102c00 0x7f55f4106c40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009900 tx=0x7f55e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:54.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.728+0000 7f55fc1b5640 1 -- 172.21.15.86:0/1149781036 shutdown_connections 2024-04-16T05:41:54.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.728+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/1149781036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55f4107320 0x7f55f410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.728+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/1149781036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102c00 0x7f55f4106c40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.728+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/1149781036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f4102250 0x7f55f4102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.728+0000 7f55fc1b5640 1 -- 172.21.15.86:0/1149781036 >> 172.21.15.86:0/1149781036 conn(0x7f55f4079490 msgr2=0x7f55f40798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.728+0000 7f55fc1b5640 1 -- 172.21.15.86:0/1149781036 shutdown_connections 2024-04-16T05:41:54.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.728+0000 7f55fc1b5640 1 -- 172.21.15.86:0/1149781036 wait complete. 2024-04-16T05:41:54.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55fc1b5640 1 Processor -- start 2024-04-16T05:41:54.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55fc1b5640 1 -- start start 2024-04-16T05:41:54.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55fc1b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102250 0x7f55f419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55fc1b5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55f4107320 0x7f55f419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55fc1b5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f41987f0 0x7f55f4198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55fc1b5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55f4112490 con 0x7f55f4102250 2024-04-16T05:41:54.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55fc1b5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f55f4112310 con 0x7f55f4107320 2024-04-16T05:41:54.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55fc1b5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55f4112610 con 0x7f55f41987f0 2024-04-16T05:41:54.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55f9f51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102250 0x7f55f419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55f9f51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102250 0x7f55f419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41780/0 (socket says 172.21.15.86:41780) 2024-04-16T05:41:54.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55f9f51640 1 -- 172.21.15.86:0/2662883905 learned_addr learned my addr 172.21.15.86:0/2662883905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55f9750640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55f4107320 0x7f55f419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55f9f51640 1 -- 172.21.15.86:0/2662883905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f41987f0 msgr2=0x7f55f4198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55f9f51640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f41987f0 0x7f55f4198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55f9f51640 1 -- 172.21.15.86:0/2662883905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55f4107320 msgr2=0x7f55f419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55f9f51640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55f4107320 0x7f55f419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.729+0000 7f55f9f51640 1 -- 172.21.15.86:0/2662883905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f4199380 con 0x7f55f4102250 2024-04-16T05:41:54.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.730+0000 7f55f9f51640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102250 0x7f55f419e630 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f55f000ba50 tx=0x7f55f000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.730+0000 7f55eaffd640 1 -- 172.21.15.86:0/2662883905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0019070 con 0x7f55f4102250 2024-04-16T05:41:54.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.730+0000 7f55eaffd640 1 -- 172.21.15.86:0/2662883905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f000cbe0 con 0x7f55f4102250 2024-04-16T05:41:54.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.730+0000 7f55fc1b5640 1 -- 172.21.15.86:0/2662883905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f4199670 con 0x7f55f4102250 2024-04-16T05:41:54.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.730+0000 7f55eaffd640 1 -- 172.21.15.86:0/2662883905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00076e0 con 0x7f55f4102250 2024-04-16T05:41:54.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.730+0000 7f55fc1b5640 1 -- 172.21.15.86:0/2662883905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f41a57d0 con 0x7f55f4102250 2024-04-16T05:41:54.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.732+0000 7f55eaffd640 1 -- 172.21.15.86:0/2662883905 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f55f001c070 con 0x7f55f4102250 2024-04-16T05:41:54.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.732+0000 7f55fed9b740 1 Processor -- start 2024-04-16T05:41:54.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fed9b740 1 -- start start 2024-04-16T05:41:54.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 0x56299f6a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 0x56299f6ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fed9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56299f6060e0 con 0x56299f6ae770 2024-04-16T05:41:54.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fed9b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56299f5a3520 con 0x56299f6a0b10 2024-04-16T05:41:54.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fed9b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56299f6020a0 con 0x56299f6a1430 2024-04-16T05:41:54.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55eaffd640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55e00a3da0 0x7f55e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fa752640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55e00a3da0 0x7f55e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9750640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9750640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41794/0 (socket says 172.21.15.86:41794) 2024-04-16T05:41:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fa752640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 0x56299f6a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9750640 1 -- 172.21.15.86:0/2803611410 learned_addr learned my addr 172.21.15.86:0/2803611410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55eaffd640 1 -- 172.21.15.86:0/2662883905 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f55f00d0090 con 0x7f55f4102250 2024-04-16T05:41:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9f51640 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 0x56299f6ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9750640 1 -- 172.21.15.86:0/2803611410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 msgr2=0x56299f6ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9750640 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 0x56299f6ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9750640 1 -- 172.21.15.86:0/2803611410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 msgr2=0x56299f6a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9750640 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 0x56299f6a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9750640 1 -- 172.21.15.86:0/2803611410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56299f5cb8d0 con 0x56299f6ae770 2024-04-16T05:41:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55f9f51640 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 0x56299f6ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fa752640 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 0x56299f6a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:41:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.733+0000 7f55fa752640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55e00a3da0 0x7f55e00a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009fd0 tx=0x7f55ec001530 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55f9750640 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6b0b60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f55e4000c00 tx=0x7f55e4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a463f7de2cbef5fc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55dffff640 1 -- 172.21.15.86:0/2803611410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4042070 con 0x56299f6ae770 2024-04-16T05:41:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55dffff640 1 -- 172.21.15.86:0/2803611410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e402ca80 con 0x56299f6ae770 2024-04-16T05:41:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55dffff640 1 -- 172.21.15.86:0/2803611410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4005000 con 0x56299f6ae770 2024-04-16T05:41:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55fed9b740 1 -- 172.21.15.86:0/2803611410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 msgr2=0x56299f6b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55fed9b740 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6b0b60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f55e4000c00 tx=0x7f55e4005e70 comp rx=0 tx=0).stop 2024-04-16T05:41:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55fed9b740 1 -- 172.21.15.86:0/2803611410 shutdown_connections 2024-04-16T05:41:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55fed9b740 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6b0b60 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55fed9b740 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 0x56299f6ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.734+0000 7f55fed9b740 1 --2- 172.21.15.86:0/2803611410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 0x56299f6a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 -- 172.21.15.86:0/2803611410 >> 172.21.15.86:0/2803611410 conn(0x56299f69b590 msgr2=0x56299f69b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 -- 172.21.15.86:0/2803611410 shutdown_connections 2024-04-16T05:41:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 -- 172.21.15.86:0/2803611410 wait complete. 2024-04-16T05:41:54.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 -- start start 2024-04-16T05:41:54.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 Processor -- start 2024-04-16T05:41:54.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a0b10 0x56299f6ab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56299f6b1550 con 0x56299f6ae770 2024-04-16T05:41:54.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56299f5bf4d0 con 0x56299f6a1430 2024-04-16T05:41:54.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56299f6b16d0 con 0x56299f6a0b10 2024-04-16T05:41:54.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fed9b740 10 client.0 _ll_register_callbacks cb 0x56299f5a42c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:54.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.735+0000 7f55fa752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fa752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6ac1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41806/0 (socket says 172.21.15.86:41806) 2024-04-16T05:41:54.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fa752640 1 -- 172.21.15.86:0/968650123 learned_addr learned my addr 172.21.15.86:0/968650123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55f9f51640 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a0b10 0x56299f6ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55f9750640 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fa752640 1 -- 172.21.15.86:0/968650123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a0b10 msgr2=0x56299f6ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fa752640 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a0b10 0x56299f6ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fa752640 1 -- 172.21.15.86:0/968650123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 msgr2=0x56299f6abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fa752640 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fa752640 1 -- 172.21.15.86:0/968650123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56299f3183f0 con 0x56299f6ae770 2024-04-16T05:41:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55f9750640 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fa752640 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6ac1d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f55ec007650 tx=0x7f55ec001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c59e570d28ac3e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55dd7fa640 1 -- 172.21.15.86:0/968650123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec012040 con 0x56299f6ae770 2024-04-16T05:41:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55dd7fa640 1 -- 172.21.15.86:0/968650123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f55ec008b90 con 0x56299f6ae770 2024-04-16T05:41:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fed9b740 1 -- 172.21.15.86:0/968650123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55e402ca80 con 0x56299f6ae770 2024-04-16T05:41:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55dd7fa640 1 -- 172.21.15.86:0/968650123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55ec02d050 con 0x56299f6ae770 2024-04-16T05:41:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fed9b740 10 client.17592 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55fed9b740 1 -- 172.21.15.86:0/968650123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56299f3e27d0 con 0x56299f6ae770 2024-04-16T05:41:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55dd7fa640 1 -- 172.21.15.86:0/968650123 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e402ca80 con 0x56299f6ae770 2024-04-16T05:41:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.736+0000 7f55dd7fa640 1 -- 172.21.15.86:0/968650123 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f55ec022410 con 0x56299f6ae770 2024-04-16T05:41:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.737+0000 7f55fed9b740 10 client.17592 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.737+0000 7f55fed9b740 10 client.17592 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:41:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.737+0000 7f55fed9b740 20 client.17592 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.737+0000 7f55fed9b740 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56299f6be810 0x56299f6c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.737+0000 7f55fed9b740 4 client.17592 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.737+0000 7f55fed9b740 1 -- 172.21.15.86:0/968650123 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56299f613b00 con 0x56299f6be810 2024-04-16T05:41:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.737+0000 7f55f9f51640 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56299f6be810 0x56299f6c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.737+0000 7f55dd7fa640 10 client.17592 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:41:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.737+0000 7f55f9f51640 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56299f6be810 0x56299f6c0c50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b37231e34e8c9b85 server_cookie=4a313e85a53bb5ca in_seq=0 out_seq=0 2024-04-16T05:41:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55dd7fa640 1 -- 172.21.15.86:0/968650123 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56299f613b00 con 0x56299f6be810 2024-04-16T05:41:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55dd7fa640 10 client.17592 handle_command_reply: tid=0 2024-04-16T05:41:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 client.17592 shutdown 2024-04-16T05:41:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 -- 172.21.15.86:0/968650123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 msgr2=0x56299f6ac1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6ac1d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f55ec007650 tx=0x7f55ec001040 comp rx=0 tx=0).stop 2024-04-16T05:41:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 -- 172.21.15.86:0/968650123 shutdown_connections 2024-04-16T05:41:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56299f6be810 0x56299f6c0c50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f6ac1d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 --2- 172.21.15.86:0/968650123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a0b10 0x56299f6ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55dd7fa640 0 client.17592 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:41:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 -- 172.21.15.86:0/968650123 >> 172.21.15.86:0/968650123 conn(0x56299f69b590 msgr2=0x56299f69d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 -- 172.21.15.86:0/968650123 shutdown_connections 2024-04-16T05:41:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 1 -- 172.21.15.86:0/968650123 wait complete. 2024-04-16T05:41:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.738+0000 7f55fed9b740 20 client.17592 trim_cache size 0 max 16384 2024-04-16T05:41:54.752 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:54 smithi196 ceph-mon[36373]: pgmap v201: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:41:54.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fed9b740 1 Processor -- start 2024-04-16T05:41:54.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fed9b740 1 -- start start 2024-04-16T05:41:54.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 0x56299f6ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 0x56299f6ba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f69a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fed9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55f41ad250 con 0x56299f6ae770 2024-04-16T05:41:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fed9b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f55f41aed90 con 0x56299f6a0b10 2024-04-16T05:41:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fed9b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55f41ad920 con 0x56299f6a1430 2024-04-16T05:41:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fa752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f69a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fa752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f69a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41820/0 (socket says 172.21.15.86:41820) 2024-04-16T05:41:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fa752640 1 -- 172.21.15.86:0/1514403444 learned_addr learned my addr 172.21.15.86:0/1514403444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fa752640 1 -- 172.21.15.86:0/1514403444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 msgr2=0x56299f6ba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fa752640 1 --2- 172.21.15.86:0/1514403444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 0x56299f6ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55f9f51640 1 --2- 172.21.15.86:0/1514403444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 0x56299f6ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fa752640 1 -- 172.21.15.86:0/1514403444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 msgr2=0x56299f6ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fa752640 1 --2- 172.21.15.86:0/1514403444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 0x56299f6ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fa752640 1 -- 172.21.15.86:0/1514403444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56299f695b40 con 0x56299f6ae770 2024-04-16T05:41:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55fa752640 1 --2- 172.21.15.86:0/1514403444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f69a5b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f55ec004fb0 tx=0x7f55ec001340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43c33ac87db8f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55df7fe640 1 -- 172.21.15.86:0/1514403444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec036070 con 0x56299f6ae770 2024-04-16T05:41:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55df7fe640 1 -- 172.21.15.86:0/1514403444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec01e070 con 0x56299f6ae770 2024-04-16T05:41:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.752+0000 7f55df7fe640 1 -- 172.21.15.86:0/1514403444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0317f0 con 0x56299f6ae770 2024-04-16T05:41:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 -- 172.21.15.86:0/1514403444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 msgr2=0x56299f69a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1514403444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f69a5b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f55ec004fb0 tx=0x7f55ec001340 comp rx=0 tx=0).stop 2024-04-16T05:41:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 -- 172.21.15.86:0/1514403444 shutdown_connections 2024-04-16T05:41:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1514403444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6ae770 0x56299f69a5b0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1514403444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6a1430 0x56299f6ba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1514403444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a0b10 0x56299f6ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 -- 172.21.15.86:0/1514403444 >> 172.21.15.86:0/1514403444 conn(0x56299f69b590 msgr2=0x56299f699ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 -- 172.21.15.86:0/1514403444 shutdown_connections 2024-04-16T05:41:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 -- 172.21.15.86:0/1514403444 wait complete. 2024-04-16T05:41:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 -- start start 2024-04-16T05:41:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.753+0000 7f55fed9b740 1 Processor -- start 2024-04-16T05:41:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6a0b10 0x56299f6b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6ae770 0x56299f6a8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55f41ae6c0 con 0x56299f6a0b10 2024-04-16T05:41:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f55f41adff0 con 0x56299f6a1430 2024-04-16T05:41:54.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55f41ab210 con 0x56299f6ae770 2024-04-16T05:41:54.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 10 client.0 _ll_register_callbacks cb 0x7f55ec009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:54.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9f51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6a0b10 0x56299f6b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9f51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6a0b10 0x56299f6b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41828/0 (socket says 172.21.15.86:41828) 2024-04-16T05:41:54.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9f51640 1 -- 172.21.15.86:0/1510230820 learned_addr learned my addr 172.21.15.86:0/1510230820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9750640 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9f51640 1 -- 172.21.15.86:0/1510230820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6ae770 msgr2=0x56299f6a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9f51640 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6ae770 0x56299f6a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9f51640 1 -- 172.21.15.86:0/1510230820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 msgr2=0x56299f6a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9f51640 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9f51640 1 -- 172.21.15.86:0/1510230820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56299f6a8f40 con 0x56299f6a0b10 2024-04-16T05:41:54.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9750640 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:54.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55f9f51640 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6a0b10 0x56299f6b6e50 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f55f009bad0 tx=0x7f55f000e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46db6d18c477772 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:54.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55dd7fa640 1 -- 172.21.15.86:0/1510230820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f009f040 con 0x56299f6a0b10 2024-04-16T05:41:54.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55dd7fa640 1 -- 172.21.15.86:0/1510230820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f55f0098440 con 0x56299f6a0b10 2024-04-16T05:41:54.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 1 -- 172.21.15.86:0/1510230820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55ec01e070 con 0x56299f6a0b10 2024-04-16T05:41:54.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55dd7fa640 1 -- 172.21.15.86:0/1510230820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55f00a4a40 con 0x56299f6a0b10 2024-04-16T05:41:54.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 10 client.17598 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:54.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.754+0000 7f55fed9b740 1 -- 172.21.15.86:0/1510230820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56299f724e60 con 0x56299f6a0b10 2024-04-16T05:41:54.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55dd7fa640 1 -- 172.21.15.86:0/1510230820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec01e070 con 0x56299f6a0b10 2024-04-16T05:41:54.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55dd7fa640 1 -- 172.21.15.86:0/1510230820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f55f009c030 con 0x56299f6a0b10 2024-04-16T05:41:54.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55fed9b740 10 client.17598 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:54.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55fed9b740 10 client.17598 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:41:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55fed9b740 20 client.17598 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56299f725580 0x56299f727970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55fed9b740 4 client.17598 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55fed9b740 1 -- 172.21.15.86:0/1510230820 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56299f60d970 con 0x56299f725580 2024-04-16T05:41:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55f9750640 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56299f725580 0x56299f727970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55dd7fa640 10 client.17598 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:41:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.755+0000 7f55f9750640 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56299f725580 0x56299f727970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=720dd5628c32d335 server_cookie=34acf05738faf920 in_seq=0 out_seq=0 2024-04-16T05:41:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55dd7fa640 1 -- 172.21.15.86:0/1510230820 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56299f60d970 con 0x56299f725580 2024-04-16T05:41:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55dd7fa640 10 client.17598 handle_command_reply: tid=0 2024-04-16T05:41:54.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 client.17598 shutdown 2024-04-16T05:41:54.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 -- 172.21.15.86:0/1510230820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6a0b10 msgr2=0x56299f6b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6a0b10 0x56299f6b6e50 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f55f009bad0 tx=0x7f55f000e040 comp rx=0 tx=0).stop 2024-04-16T05:41:54.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 -- 172.21.15.86:0/1510230820 shutdown_connections 2024-04-16T05:41:54.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x56299f725580 0x56299f727970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56299f6ae770 0x56299f6a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55dd7fa640 0 client.17598 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:41:54.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56299f6a1430 0x56299f6a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 --2- 172.21.15.86:0/1510230820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56299f6a0b10 0x56299f6b6e50 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 -- 172.21.15.86:0/1510230820 >> 172.21.15.86:0/1510230820 conn(0x56299f69b590 msgr2=0x56299f69a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 -- 172.21.15.86:0/1510230820 shutdown_connections 2024-04-16T05:41:54.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 1 -- 172.21.15.86:0/1510230820 wait complete. 2024-04-16T05:41:54.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fed9b740 20 client.17598 trim_cache size 0 max 16384 2024-04-16T05:41:54.767 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:54.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 -- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55e00a3da0 msgr2=0x7f55e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55e00a3da0 0x7f55e00a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009fd0 tx=0x7f55ec001530 comp rx=0 tx=0).stop 2024-04-16T05:41:54.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 -- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102250 msgr2=0x7f55f419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102250 0x7f55f419e630 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f55f000ba50 tx=0x7f55f000bf10 comp rx=0 tx=0).stop 2024-04-16T05:41:54.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 -- 172.21.15.86:0/2662883905 shutdown_connections 2024-04-16T05:41:54.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f41987f0 0x7f55f4198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f55e00a3da0 0x7f55e00a6260 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55f4107320 0x7f55f419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 --2- 172.21.15.86:0/2662883905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55f4102250 0x7f55f419e630 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.756+0000 7f55fc1b5640 1 -- 172.21.15.86:0/2662883905 >> 172.21.15.86:0/2662883905 conn(0x7f55f4079490 msgr2=0x7f55f40ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.757+0000 7f55fc1b5640 1 -- 172.21.15.86:0/2662883905 shutdown_connections 2024-04-16T05:41:54.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.757+0000 7f55fc1b5640 1 -- 172.21.15.86:0/2662883905 wait complete. 2024-04-16T05:41:54.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:41:54.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.878+0000 7f82d4d11640 1 -- 172.21.15.86:0/2915874582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 msgr2=0x7f82d0103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.878+0000 7f82d4d11640 1 --2- 172.21.15.86:0/2915874582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 0x7f82d0103650 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c402bd60 comp rx=0 tx=0).stop 2024-04-16T05:41:54.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.878+0000 7f82d4d11640 1 -- 172.21.15.86:0/2915874582 shutdown_connections 2024-04-16T05:41:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.878+0000 7f82d4d11640 1 --2- 172.21.15.86:0/2915874582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82d010c5b0 0x7f82d010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.878+0000 7f82d4d11640 1 --2- 172.21.15.86:0/2915874582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82d0103b90 0x7f82d010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.878+0000 7f82d4d11640 1 --2- 172.21.15.86:0/2915874582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 0x7f82d0103650 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.878+0000 7f82d4d11640 1 -- 172.21.15.86:0/2915874582 >> 172.21.15.86:0/2915874582 conn(0x7f82d00780b0 msgr2=0x7f82d00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.878+0000 7f82d4d11640 1 -- 172.21.15.86:0/2915874582 shutdown_connections 2024-04-16T05:41:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.878+0000 7f82d4d11640 1 -- 172.21.15.86:0/2915874582 wait complete. 2024-04-16T05:41:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82d4d11640 1 Processor -- start 2024-04-16T05:41:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82d4d11640 1 -- start start 2024-04-16T05:41:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82d4d11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 0x7f82d019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82d4d11640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82d0103b90 0x7f82d019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82d4d11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82d010c5b0 0x7f82d0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82d4d11640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0112460 con 0x7f82d0103270 2024-04-16T05:41:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82d4d11640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01122e0 con 0x7f82d0103b90 2024-04-16T05:41:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82d4d11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01125e0 con 0x7f82d010c5b0 2024-04-16T05:41:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 0x7f82d019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 0x7f82d019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41844/0 (socket says 172.21.15.86:41844) 2024-04-16T05:41:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.879+0000 7f82ce59c640 1 -- 172.21.15.86:0/1786844008 learned_addr learned my addr 172.21.15.86:0/1786844008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82ced9d640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82d010c5b0 0x7f82d0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82ce59c640 1 -- 172.21.15.86:0/1786844008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82d010c5b0 msgr2=0x7f82d0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82ce59c640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82d010c5b0 0x7f82d0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82ce59c640 1 -- 172.21.15.86:0/1786844008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82d0103b90 msgr2=0x7f82d019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82cdd9b640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82d0103b90 0x7f82d019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82ce59c640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82d0103b90 0x7f82d019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82ce59c640 1 -- 172.21.15.86:0/1786844008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d0199000 con 0x7f82d0103270 2024-04-16T05:41:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82ced9d640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82d010c5b0 0x7f82d0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:54.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82ce59c640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 0x7f82d019e680 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f82c4004f50 tx=0x7f82c402c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82bf7fe640 1 -- 172.21.15.86:0/1786844008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4043020 con 0x7f82d0103270 2024-04-16T05:41:54.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82d4d11640 1 -- 172.21.15.86:0/1786844008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d0199290 con 0x7f82d0103270 2024-04-16T05:41:54.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82bf7fe640 1 -- 172.21.15.86:0/1786844008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c40054e0 con 0x7f82d0103270 2024-04-16T05:41:54.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82bf7fe640 1 -- 172.21.15.86:0/1786844008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4003de0 con 0x7f82d0103270 2024-04-16T05:41:54.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.880+0000 7f82d4d11640 1 -- 172.21.15.86:0/1786844008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d01a5600 con 0x7f82d0103270 2024-04-16T05:41:54.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.883+0000 7f82d79cf740 1 Processor -- start 2024-04-16T05:41:54.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.883+0000 7f82bf7fe640 1 -- 172.21.15.86:0/1786844008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f82c4046020 con 0x7f82d0103270 2024-04-16T05:41:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.883+0000 7f82d79cf740 1 -- start start 2024-04-16T05:41:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.883+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabcb10 0x55cd5dabcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.883+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabd430 0x55cd5daca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.883+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 0x55cd5daccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.883+0000 7f82d79cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd5da220e0 con 0x55cd5dabcb10 2024-04-16T05:41:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.883+0000 7f82d79cf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cd5d9bf520 con 0x55cd5daca770 2024-04-16T05:41:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.883+0000 7f82d79cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cd5da1e0a0 con 0x55cd5dabd430 2024-04-16T05:41:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabcb10 0x55cd5dabcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabcb10 0x55cd5dabcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41860/0 (socket says 172.21.15.86:41860) 2024-04-16T05:41:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ced9d640 1 -- 172.21.15.86:0/1694155357 learned_addr learned my addr 172.21.15.86:0/1694155357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ce59c640 1 --2- 172.21.15.86:0/1694155357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabd430 0x55cd5daca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ced9d640 1 -- 172.21.15.86:0/1694155357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabd430 msgr2=0x55cd5daca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ced9d640 1 --2- 172.21.15.86:0/1694155357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabd430 0x55cd5daca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ced9d640 1 -- 172.21.15.86:0/1694155357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 msgr2=0x55cd5daccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82cdd9b640 1 --2- 172.21.15.86:0/1694155357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 0x55cd5daccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ced9d640 1 --2- 172.21.15.86:0/1694155357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 0x55cd5daccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ced9d640 1 -- 172.21.15.86:0/1694155357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd5d9e78d0 con 0x55cd5dabcb10 2024-04-16T05:41:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82bf7fe640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82b80a3d20 0x7f82b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ced9d640 1 --2- 172.21.15.86:0/1694155357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabcb10 0x55cd5dabcef0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f82c8008dd0 tx=0x7f82c800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f15b500f7fb2da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82ce59c640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82b80a3d20 0x7f82b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82bf7fe640 1 -- 172.21.15.86:0/1786844008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f82c40f2d40 con 0x7f82d0103270 2024-04-16T05:41:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82bcff9640 1 -- 172.21.15.86:0/1694155357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c800f830 con 0x55cd5dabcb10 2024-04-16T05:41:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82bcff9640 1 -- 172.21.15.86:0/1694155357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c800de20 con 0x55cd5dabcb10 2024-04-16T05:41:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.884+0000 7f82bcff9640 1 -- 172.21.15.86:0/1694155357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c800e3b0 con 0x55cd5dabcb10 2024-04-16T05:41:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 -- 172.21.15.86:0/1694155357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabcb10 msgr2=0x55cd5dabcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 --2- 172.21.15.86:0/1694155357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabcb10 0x55cd5dabcef0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f82c8008dd0 tx=0x7f82c800d710 comp rx=0 tx=0).stop 2024-04-16T05:41:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82ce59c640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82b80a3d20 0x7f82b80a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f82c40baee0 tx=0x7f82c40a9040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 -- 172.21.15.86:0/1694155357 shutdown_connections 2024-04-16T05:41:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 --2- 172.21.15.86:0/1694155357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 0x55cd5daccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 --2- 172.21.15.86:0/1694155357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabd430 0x55cd5daca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 --2- 172.21.15.86:0/1694155357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabcb10 0x55cd5dabcef0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 -- 172.21.15.86:0/1694155357 >> 172.21.15.86:0/1694155357 conn(0x55cd5dab7590 msgr2=0x55cd5dab7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 -- 172.21.15.86:0/1694155357 shutdown_connections 2024-04-16T05:41:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 -- 172.21.15.86:0/1694155357 wait complete. 2024-04-16T05:41:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.885+0000 7f82d79cf740 1 -- start start 2024-04-16T05:41:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82d79cf740 1 Processor -- start 2024-04-16T05:41:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5dabcb10 0x55cd5dac76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5daca770 0x55cd5dac8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82d79cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd5dacd5b0 con 0x55cd5dabd430 2024-04-16T05:41:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82d79cf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cd5d9db4d0 con 0x55cd5dabcb10 2024-04-16T05:41:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82d79cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cd5dacd730 con 0x55cd5daca770 2024-04-16T05:41:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82d79cf740 10 client.0 _ll_register_callbacks cb 0x55cd5d9c02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82cdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82cdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41868/0 (socket says 172.21.15.86:41868) 2024-04-16T05:41:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82cdd9b640 1 -- 172.21.15.86:0/2642227011 learned_addr learned my addr 172.21.15.86:0/2642227011 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82ced9d640 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5dabcb10 0x55cd5dac76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82ced9d640 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5daca770 0x55cd5dac8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82cdd9b640 1 -- 172.21.15.86:0/2642227011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5daca770 msgr2=0x55cd5dac8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82cdd9b640 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5daca770 0x55cd5dac8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82cdd9b640 1 -- 172.21.15.86:0/2642227011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5dabcb10 msgr2=0x55cd5dac76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82cdd9b640 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5dabcb10 0x55cd5dac76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82cdd9b640 1 -- 172.21.15.86:0/2642227011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd5d7343f0 con 0x55cd5dabd430 2024-04-16T05:41:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82ced9d640 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5daca770 0x55cd5dac8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82cdd9b640 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac7c30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f82c000bf50 tx=0x7f82c000d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffe8ea0a292106c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2642227011 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0016070 con 0x55cd5dabd430 2024-04-16T05:41:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2642227011 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f82c0012670 con 0x55cd5dabd430 2024-04-16T05:41:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.886+0000 7f82d79cf740 1 -- 172.21.15.86:0/2642227011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c800de20 con 0x55cd5dabd430 2024-04-16T05:41:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2642227011 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82c0022050 con 0x55cd5dabd430 2024-04-16T05:41:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82d79cf740 10 client.17610 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82d79cf740 1 -- 172.21.15.86:0/2642227011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd5d7fe7d0 con 0x55cd5dabd430 2024-04-16T05:41:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2642227011 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c800de20 con 0x55cd5dabd430 2024-04-16T05:41:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2642227011 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f82c0011a30 con 0x55cd5dabd430 2024-04-16T05:41:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82d79cf740 10 client.17610 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82d79cf740 10 client.17610 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:41:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82d79cf740 20 client.17610 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:54.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55cd5dada810 0x55cd5dadcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82d79cf740 4 client.17610 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:54.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82d79cf740 1 -- 172.21.15.86:0/2642227011 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd5da2fb00 con 0x55cd5dada810 2024-04-16T05:41:54.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.887+0000 7f82ced9d640 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55cd5dada810 0x55cd5dadcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.888+0000 7f82b5ffb640 10 client.17610 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:41:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.888+0000 7f82ced9d640 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55cd5dada810 0x55cd5dadcc40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=92d884853e690143 server_cookie=8b70a45b72acaf86 in_seq=0 out_seq=0 2024-04-16T05:41:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.888+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2642227011 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd5da2fb00 con 0x55cd5dada810 2024-04-16T05:41:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.888+0000 7f82b5ffb640 10 client.17610 handle_command_reply: tid=0 2024-04-16T05:41:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.888+0000 7f82d79cf740 1 client.17610 shutdown 2024-04-16T05:41:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 -- 172.21.15.86:0/2642227011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 msgr2=0x55cd5dac7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac7c30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f82c000bf50 tx=0x7f82c000d650 comp rx=0 tx=0).stop 2024-04-16T05:41:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 -- 172.21.15.86:0/2642227011 shutdown_connections 2024-04-16T05:41:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55cd5dada810 0x55cd5dadcc40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5daca770 0x55cd5dac8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82b5ffb640 0 client.17610 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:41:54.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac7c30 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2642227011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5dabcb10 0x55cd5dac76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 -- 172.21.15.86:0/2642227011 >> 172.21.15.86:0/2642227011 conn(0x55cd5dab7590 msgr2=0x55cd5dab92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 -- 172.21.15.86:0/2642227011 shutdown_connections 2024-04-16T05:41:54.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 1 -- 172.21.15.86:0/2642227011 wait complete. 2024-04-16T05:41:54.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.889+0000 7f82d79cf740 20 client.17610 trim_cache size 0 max 16384 2024-04-16T05:41:54.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:54 smithi194 ceph-mon[35865]: pgmap v201: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:41:54.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82d79cf740 1 Processor -- start 2024-04-16T05:41:54.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82d79cf740 1 -- start start 2024-04-16T05:41:54.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5dabcb10 0x55cd5dac9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabd430 0x55cd5dad6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5daca770 0x55cd5dab9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82d79cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ad2a0 con 0x55cd5daca770 2024-04-16T05:41:54.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82d79cf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01aede0 con 0x55cd5dabcb10 2024-04-16T05:41:54.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82d79cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ad970 con 0x55cd5dabd430 2024-04-16T05:41:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5daca770 0x55cd5dab9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5daca770 0x55cd5dab9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41876/0 (socket says 172.21.15.86:41876) 2024-04-16T05:41:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82ced9d640 1 -- 172.21.15.86:0/3012965483 learned_addr learned my addr 172.21.15.86:0/3012965483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82ced9d640 1 -- 172.21.15.86:0/3012965483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabd430 msgr2=0x55cd5dad6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82ced9d640 1 --2- 172.21.15.86:0/3012965483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabd430 0x55cd5dad6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82ced9d640 1 -- 172.21.15.86:0/3012965483 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5dabcb10 msgr2=0x55cd5dac9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82ced9d640 1 --2- 172.21.15.86:0/3012965483 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5dabcb10 0x55cd5dac9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82ced9d640 1 -- 172.21.15.86:0/3012965483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd5dac66a0 con 0x55cd5daca770 2024-04-16T05:41:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.902+0000 7f82ced9d640 1 --2- 172.21.15.86:0/3012965483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5daca770 0x55cd5dab9270 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f82c8018c20 tx=0x7f82c8008a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3776bef3cfd5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82b7fff640 1 -- 172.21.15.86:0/3012965483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8021040 con 0x55cd5daca770 2024-04-16T05:41:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82b7fff640 1 -- 172.21.15.86:0/3012965483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c8018600 con 0x55cd5daca770 2024-04-16T05:41:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82b7fff640 1 -- 172.21.15.86:0/3012965483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8029370 con 0x55cd5daca770 2024-04-16T05:41:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82d79cf740 1 -- 172.21.15.86:0/3012965483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5daca770 msgr2=0x55cd5dab9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82d79cf740 1 --2- 172.21.15.86:0/3012965483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5daca770 0x55cd5dab9270 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f82c8018c20 tx=0x7f82c8008a80 comp rx=0 tx=0).stop 2024-04-16T05:41:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82d79cf740 1 -- 172.21.15.86:0/3012965483 shutdown_connections 2024-04-16T05:41:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82d79cf740 1 --2- 172.21.15.86:0/3012965483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5daca770 0x55cd5dab9270 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82d79cf740 1 --2- 172.21.15.86:0/3012965483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabd430 0x55cd5dad6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82d79cf740 1 --2- 172.21.15.86:0/3012965483 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5dabcb10 0x55cd5dac9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82d79cf740 1 -- 172.21.15.86:0/3012965483 >> 172.21.15.86:0/3012965483 conn(0x55cd5dab7590 msgr2=0x55cd5dab7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82d79cf740 1 -- 172.21.15.86:0/3012965483 shutdown_connections 2024-04-16T05:41:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.903+0000 7f82d79cf740 1 -- 172.21.15.86:0/3012965483 wait complete. 2024-04-16T05:41:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82d79cf740 1 -- start start 2024-04-16T05:41:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82d79cf740 1 Processor -- start 2024-04-16T05:41:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabcb10 0x55cd5dac4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82d79cf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 0x55cd5db3fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82d79cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ae710 con 0x55cd5dabd430 2024-04-16T05:41:54.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82d79cf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ae040 con 0x55cd5daca770 2024-04-16T05:41:54.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82d79cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ab260 con 0x55cd5dabcb10 2024-04-16T05:41:54.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82d79cf740 10 client.0 _ll_register_callbacks cb 0x7f82c0006e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:54.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82cdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82cdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41878/0 (socket says 172.21.15.86:41878) 2024-04-16T05:41:54.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82cdd9b640 1 -- 172.21.15.86:0/2647998443 learned_addr learned my addr 172.21.15.86:0/2647998443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82ced9d640 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabcb10 0x55cd5dac4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82ced9d640 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 0x55cd5db3fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82cdd9b640 1 -- 172.21.15.86:0/2647998443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabcb10 msgr2=0x55cd5dac4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82cdd9b640 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabcb10 0x55cd5dac4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82cdd9b640 1 -- 172.21.15.86:0/2647998443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 msgr2=0x55cd5db3fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82cdd9b640 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 0x55cd5db3fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82cdd9b640 1 -- 172.21.15.86:0/2647998443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd5db40300 con 0x55cd5dabd430 2024-04-16T05:41:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82cdd9b640 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac4c70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f82c0001eb0 tx=0x7f82c0006c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a19abb7ae4f83b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.904+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2647998443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0016040 con 0x55cd5dabd430 2024-04-16T05:41:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2647998443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f82c001aad0 con 0x55cd5dabd430 2024-04-16T05:41:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82d79cf740 1 -- 172.21.15.86:0/2647998443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c8018600 con 0x55cd5dabd430 2024-04-16T05:41:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2647998443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82c0034050 con 0x55cd5dabd430 2024-04-16T05:41:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82d79cf740 10 client.17616 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82d79cf740 1 -- 172.21.15.86:0/2647998443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd5db41440 con 0x55cd5dabd430 2024-04-16T05:41:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2647998443 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c8018600 con 0x55cd5dabd430 2024-04-16T05:41:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2647998443 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f82c0012c70 con 0x55cd5dabd430 2024-04-16T05:41:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82d79cf740 10 client.17616 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82d79cf740 10 client.17616 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:41:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82d79cf740 20 client.17616 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55cd5db41be0 0x55cd5db44000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82d79cf740 4 client.17616 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82d79cf740 1 -- 172.21.15.86:0/2647998443 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd5da29970 con 0x55cd5db41be0 2024-04-16T05:41:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.905+0000 7f82ced9d640 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55cd5db41be0 0x55cd5db44000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82ced9d640 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55cd5db41be0 0x55cd5db44000 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61853142d8b6f0ff server_cookie=912c61942cc5513c in_seq=0 out_seq=0 2024-04-16T05:41:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82b5ffb640 10 client.17616 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:41:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82b5ffb640 1 -- 172.21.15.86:0/2647998443 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd5da29970 con 0x55cd5db41be0 2024-04-16T05:41:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82b5ffb640 10 client.17616 handle_command_reply: tid=0 2024-04-16T05:41:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 client.17616 shutdown 2024-04-16T05:41:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 -- 172.21.15.86:0/2647998443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 msgr2=0x55cd5dac4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac4c70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f82c0001eb0 tx=0x7f82c0006c10 comp rx=0 tx=0).stop 2024-04-16T05:41:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 -- 172.21.15.86:0/2647998443 shutdown_connections 2024-04-16T05:41:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd5daca770 0x55cd5db3fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd5dabd430 0x55cd5dac4c70 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55cd5db41be0 0x55cd5db44000 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 --2- 172.21.15.86:0/2647998443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd5dabcb10 0x55cd5dac4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 -- 172.21.15.86:0/2647998443 >> 172.21.15.86:0/2647998443 conn(0x55cd5dab7590 msgr2=0x55cd5dab6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82b5ffb640 0 client.17616 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:41:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 -- 172.21.15.86:0/2647998443 shutdown_connections 2024-04-16T05:41:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 1 -- 172.21.15.86:0/2647998443 wait complete. 2024-04-16T05:41:54.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.906+0000 7f82d79cf740 20 client.17616 trim_cache size 0 max 16384 2024-04-16T05:41:54.918 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 -- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82b80a3d20 msgr2=0x7f82b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82b80a3d20 0x7f82b80a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f82c40baee0 tx=0x7f82c40a9040 comp rx=0 tx=0).stop 2024-04-16T05:41:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 -- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 msgr2=0x7f82d019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 0x7f82d019e680 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f82c4004f50 tx=0x7f82c402c860 comp rx=0 tx=0).stop 2024-04-16T05:41:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 -- 172.21.15.86:0/1786844008 shutdown_connections 2024-04-16T05:41:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82d010c5b0 0x7f82d0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82b80a3d20 0x7f82b80a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82d0103b90 0x7f82d019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 --2- 172.21.15.86:0/1786844008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d0103270 0x7f82d019e680 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 -- 172.21.15.86:0/1786844008 >> 172.21.15.86:0/1786844008 conn(0x7f82d00780b0 msgr2=0x7f82d00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 -- 172.21.15.86:0/1786844008 shutdown_connections 2024-04-16T05:41:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:54.907+0000 7f82d4d11640 1 -- 172.21.15.86:0/1786844008 wait complete. 2024-04-16T05:41:54.921 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:41:55.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.028+0000 7f67ab1db640 1 -- 172.21.15.86:0/3917554570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4102250 msgr2=0x7f67a4102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.028+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3917554570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4102250 0x7f67a4102630 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6798009900 tx=0x7f679802bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:55.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.028+0000 7f67ab1db640 1 -- 172.21.15.86:0/3917554570 shutdown_connections 2024-04-16T05:41:55.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.028+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3917554570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a4107320 0x7f67a410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.028+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3917554570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67a4102c00 0x7f67a4106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.028+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3917554570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4102250 0x7f67a4102630 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.028+0000 7f67ab1db640 1 -- 172.21.15.86:0/3917554570 >> 172.21.15.86:0/3917554570 conn(0x7f67a4079490 msgr2=0x7f67a40798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.028+0000 7f67ab1db640 1 -- 172.21.15.86:0/3917554570 shutdown_connections 2024-04-16T05:41:55.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.028+0000 7f67ab1db640 1 -- 172.21.15.86:0/3917554570 wait complete. 2024-04-16T05:41:55.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67ab1db640 1 Processor -- start 2024-04-16T05:41:55.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67ab1db640 1 -- start start 2024-04-16T05:41:55.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67ab1db640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a4102250 0x7f67a419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67ab1db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4107320 0x7f67a419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67ab1db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67a4198750 0x7f67a4198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67ab1db640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41123d0 con 0x7f67a4107320 2024-04-16T05:41:55.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67ab1db640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4112250 con 0x7f67a4102250 2024-04-16T05:41:55.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67ab1db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4112550 con 0x7f67a4198750 2024-04-16T05:41:55.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4107320 0x7f67a419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4107320 0x7f67a419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41886/0 (socket says 172.21.15.86:41886) 2024-04-16T05:41:55.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a3fff640 1 -- 172.21.15.86:0/3446247462 learned_addr learned my addr 172.21.15.86:0/3446247462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a8f77640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a4102250 0x7f67a419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a3fff640 1 -- 172.21.15.86:0/3446247462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67a4198750 msgr2=0x7f67a4198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a3fff640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67a4198750 0x7f67a4198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a3fff640 1 -- 172.21.15.86:0/3446247462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a4102250 msgr2=0x7f67a419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a3fff640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a4102250 0x7f67a419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a3fff640 1 -- 172.21.15.86:0/3446247462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a4199350 con 0x7f67a4107320 2024-04-16T05:41:55.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a8f77640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a4102250 0x7f67a419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:55.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.029+0000 7f67a3fff640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4107320 0x7f67a419ebd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f679000d740 tx=0x7f679000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.030+0000 7f67a1ffb640 1 -- 172.21.15.86:0/3446247462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790017070 con 0x7f67a4107320 2024-04-16T05:41:55.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.030+0000 7f67a1ffb640 1 -- 172.21.15.86:0/3446247462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679000de50 con 0x7f67a4107320 2024-04-16T05:41:55.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.030+0000 7f67ab1db640 1 -- 172.21.15.86:0/3446247462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a41a5430 con 0x7f67a4107320 2024-04-16T05:41:55.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.030+0000 7f67a1ffb640 1 -- 172.21.15.86:0/3446247462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790004e00 con 0x7f67a4107320 2024-04-16T05:41:55.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.030+0000 7f67ab1db640 1 -- 172.21.15.86:0/3446247462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a41a5830 con 0x7f67a4107320 2024-04-16T05:41:55.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67a1ffb640 1 -- 172.21.15.86:0/3446247462 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6790004fa0 con 0x7f67a4107320 2024-04-16T05:41:55.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67addc8740 1 Processor -- start 2024-04-16T05:41:55.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67addc8740 1 -- start start 2024-04-16T05:41:55.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67addc8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 0x55cb524c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67addc8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 0x55cb524d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67addc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524d2360 0x55cb524d4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67addc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb5242e0e0 con 0x55cb524d2360 2024-04-16T05:41:55.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67addc8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cb523cb520 con 0x55cb524c9780 2024-04-16T05:41:55.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67addc8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb5242a0a0 con 0x55cb524c8e60 2024-04-16T05:41:55.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67a1ffb640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f67940a3da0 0x7f67940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524d2360 0x55cb524d4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524d2360 0x55cb524d4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41892/0 (socket says 172.21.15.86:41892) 2024-04-16T05:41:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67a3fff640 1 -- 172.21.15.86:0/4099746151 learned_addr learned my addr 172.21.15.86:0/4099746151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67a9778640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f67940a3da0 0x7f67940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67a8f77640 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 0x55cb524d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67a9778640 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 0x55cb524c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.033+0000 7f67a1ffb640 1 -- 172.21.15.86:0/3446247462 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f67900d0560 con 0x7f67a4107320 2024-04-16T05:41:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f67a3fff640 1 -- 172.21.15.86:0/4099746151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 msgr2=0x55cb524c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f67a3fff640 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 0x55cb524c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f67a3fff640 1 -- 172.21.15.86:0/4099746151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 msgr2=0x55cb524d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f67a3fff640 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 0x55cb524d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f67a3fff640 1 -- 172.21.15.86:0/4099746151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb523f38d0 con 0x55cb524d2360 2024-04-16T05:41:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f67a8f77640 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 0x55cb524d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f67a3fff640 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524d2360 0x55cb524d4750 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f67900cd8e0 tx=0x7f6790009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bbecd2fc843007 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f67a9778640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f67940a3da0 0x7f67940a6260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f679c009370 tx=0x7f679c009240 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f678effd640 1 -- 172.21.15.86:0/4099746151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790020070 con 0x55cb524d2360 2024-04-16T05:41:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f678effd640 1 -- 172.21.15.86:0/4099746151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67900a0540 con 0x55cb524d2360 2024-04-16T05:41:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.034+0000 7f678effd640 1 -- 172.21.15.86:0/4099746151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67900cddc0 con 0x55cb524d2360 2024-04-16T05:41:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 -- 172.21.15.86:0/4099746151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524d2360 msgr2=0x55cb524d4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524d2360 0x55cb524d4750 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f67900cd8e0 tx=0x7f6790009c30 comp rx=0 tx=0).stop 2024-04-16T05:41:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 -- 172.21.15.86:0/4099746151 shutdown_connections 2024-04-16T05:41:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524d2360 0x55cb524d4750 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 0x55cb524d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 --2- 172.21.15.86:0/4099746151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 0x55cb524c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 -- 172.21.15.86:0/4099746151 >> 172.21.15.86:0/4099746151 conn(0x55cb524c3590 msgr2=0x55cb524c3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 -- 172.21.15.86:0/4099746151 shutdown_connections 2024-04-16T05:41:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 -- 172.21.15.86:0/4099746151 wait complete. 2024-04-16T05:41:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 -- start start 2024-04-16T05:41:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 Processor -- start 2024-04-16T05:41:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 0x55cb524cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c9780 0x55cb524cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 0x55cb524cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb524d4fa0 con 0x55cb524c9780 2024-04-16T05:41:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cb523e74d0 con 0x55cb524d2360 2024-04-16T05:41:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.035+0000 7f67addc8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb524d5120 con 0x55cb524c8e60 2024-04-16T05:41:55.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c9780 0x55cb524cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67addc8740 10 client.0 _ll_register_callbacks cb 0x55cb523cc2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:55.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c9780 0x55cb524cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41896/0 (socket says 172.21.15.86:41896) 2024-04-16T05:41:55.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a3fff640 1 -- 172.21.15.86:0/134074322 learned_addr learned my addr 172.21.15.86:0/134074322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a8f77640 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 0x55cb524cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a9778640 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 0x55cb524cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a3fff640 1 -- 172.21.15.86:0/134074322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 msgr2=0x55cb524cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a3fff640 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 0x55cb524cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a3fff640 1 -- 172.21.15.86:0/134074322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 msgr2=0x55cb524cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a3fff640 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 0x55cb524cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a3fff640 1 -- 172.21.15.86:0/134074322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb521403f0 con 0x55cb524c9780 2024-04-16T05:41:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a8f77640 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 0x55cb524cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a9778640 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 0x55cb524cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67a3fff640 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c9780 0x55cb524cf8d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f6790098b60 tx=0x7f6790009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5eb5ed1e7a347e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f6783fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f6783fff640 1 -- 172.21.15.86:0/134074322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67900b4020 con 0x55cb524c9780 2024-04-16T05:41:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f6783fff640 1 -- 172.21.15.86:0/134074322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f67900d0180 con 0x55cb524c9780 2024-04-16T05:41:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67addc8740 1 -- 172.21.15.86:0/134074322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67900a0540 con 0x55cb524c9780 2024-04-16T05:41:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f6783fff640 1 -- 172.21.15.86:0/134074322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6790020070 con 0x55cb524c9780 2024-04-16T05:41:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67addc8740 10 client.17628 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f67addc8740 1 -- 172.21.15.86:0/134074322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb5220a7d0 con 0x55cb524c9780 2024-04-16T05:41:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.036+0000 7f6783fff640 1 -- 172.21.15.86:0/134074322 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67900a0540 con 0x55cb524c9780 2024-04-16T05:41:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f6783fff640 1 -- 172.21.15.86:0/134074322 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f67900a4970 con 0x55cb524c9780 2024-04-16T05:41:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f67addc8740 10 client.17628 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f67addc8740 10 client.17628 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:41:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f67addc8740 20 client.17628 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f67addc8740 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55cb524e20e0 0x55cb524e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f67addc8740 4 client.17628 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f67addc8740 1 -- 172.21.15.86:0/134074322 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb5243bb00 con 0x55cb524e20e0 2024-04-16T05:41:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f67a8f77640 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55cb524e20e0 0x55cb524e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f6783fff640 10 client.17628 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:41:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.037+0000 7f67a8f77640 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55cb524e20e0 0x55cb524e44d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=195303e24cfc629a server_cookie=bec64ed39fac1629 in_seq=0 out_seq=0 2024-04-16T05:41:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f6783fff640 1 -- 172.21.15.86:0/134074322 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb5243bb00 con 0x55cb524e20e0 2024-04-16T05:41:55.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f6783fff640 10 client.17628 handle_command_reply: tid=0 2024-04-16T05:41:55.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 client.17628 shutdown 2024-04-16T05:41:55.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 -- 172.21.15.86:0/134074322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c9780 msgr2=0x55cb524cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c9780 0x55cb524cf8d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f6790098b60 tx=0x7f6790009d40 comp rx=0 tx=0).stop 2024-04-16T05:41:55.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 -- 172.21.15.86:0/134074322 shutdown_connections 2024-04-16T05:41:55.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 0x55cb524cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55cb524e20e0 0x55cb524e44d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c9780 0x55cb524cf8d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 --2- 172.21.15.86:0/134074322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c8e60 0x55cb524cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f6783fff640 0 client.17628 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:41:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 -- 172.21.15.86:0/134074322 >> 172.21.15.86:0/134074322 conn(0x55cb524c3590 msgr2=0x55cb524c52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 -- 172.21.15.86:0/134074322 shutdown_connections 2024-04-16T05:41:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.038+0000 7f67addc8740 1 -- 172.21.15.86:0/134074322 wait complete. 2024-04-16T05:41:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.039+0000 7f67addc8740 20 client.17628 trim_cache size 0 max 16384 2024-04-16T05:41:55.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.051+0000 7f67addc8740 1 Processor -- start 2024-04-16T05:41:55.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67addc8740 1 -- start start 2024-04-16T05:41:55.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67addc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb524e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67addc8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 0x55cb524c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67addc8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524d2360 0x55cb524c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67addc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41af4e0 con 0x55cb524c8e60 2024-04-16T05:41:55.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67addc8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41b1020 con 0x55cb524c9780 2024-04-16T05:41:55.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67addc8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41afbb0 con 0x55cb524d2360 2024-04-16T05:41:55.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67a8f77640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb524e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67a8f77640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb524e78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41904/0 (socket says 172.21.15.86:41904) 2024-04-16T05:41:55.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67a8f77640 1 -- 172.21.15.86:0/1635695320 learned_addr learned my addr 172.21.15.86:0/1635695320 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67a8f77640 1 -- 172.21.15.86:0/1635695320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524d2360 msgr2=0x55cb524c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67a8f77640 1 --2- 172.21.15.86:0/1635695320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524d2360 0x55cb524c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67a8f77640 1 -- 172.21.15.86:0/1635695320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 msgr2=0x55cb524c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67a8f77640 1 --2- 172.21.15.86:0/1635695320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 0x55cb524c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67a8f77640 1 -- 172.21.15.86:0/1635695320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb524bee60 con 0x55cb524c8e60 2024-04-16T05:41:55.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f67a8f77640 1 --2- 172.21.15.86:0/1635695320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb524e78e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f6798009900 tx=0x7f6798009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b07db5d7586f0f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f678e7fc640 1 -- 172.21.15.86:0/1635695320 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679802cc80 con 0x55cb524c8e60 2024-04-16T05:41:55.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f678e7fc640 1 -- 172.21.15.86:0/1635695320 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6798005040 con 0x55cb524c8e60 2024-04-16T05:41:55.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.052+0000 7f678e7fc640 1 -- 172.21.15.86:0/1635695320 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67980049f0 con 0x55cb524c8e60 2024-04-16T05:41:55.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 -- 172.21.15.86:0/1635695320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 msgr2=0x55cb524e78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 --2- 172.21.15.86:0/1635695320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb524e78e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f6798009900 tx=0x7f6798009260 comp rx=0 tx=0).stop 2024-04-16T05:41:55.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 -- 172.21.15.86:0/1635695320 shutdown_connections 2024-04-16T05:41:55.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 --2- 172.21.15.86:0/1635695320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524d2360 0x55cb524c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 --2- 172.21.15.86:0/1635695320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524c9780 0x55cb524c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 --2- 172.21.15.86:0/1635695320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb524e78e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 -- 172.21.15.86:0/1635695320 >> 172.21.15.86:0/1635695320 conn(0x55cb524c3590 msgr2=0x55cb524be5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 -- 172.21.15.86:0/1635695320 shutdown_connections 2024-04-16T05:41:55.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 -- 172.21.15.86:0/1635695320 wait complete. 2024-04-16T05:41:55.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 -- start start 2024-04-16T05:41:55.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.053+0000 7f67addc8740 1 Processor -- start 2024-04-16T05:41:55.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67addc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb52544680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67addc8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c9780 0x55cb52544bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67addc8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 0x55cb52540270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67addc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41b0950 con 0x55cb524c8e60 2024-04-16T05:41:55.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67addc8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41b0280 con 0x55cb524d2360 2024-04-16T05:41:55.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67addc8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41ad4a0 con 0x55cb524c9780 2024-04-16T05:41:55.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67addc8740 10 client.0 _ll_register_callbacks cb 0x7f67900046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:55.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a8f77640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb52544680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a8f77640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb52544680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41918/0 (socket says 172.21.15.86:41918) 2024-04-16T05:41:55.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a8f77640 1 -- 172.21.15.86:0/2179034509 learned_addr learned my addr 172.21.15.86:0/2179034509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a3fff640 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c9780 0x55cb52544bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a9778640 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 0x55cb52540270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a8f77640 1 -- 172.21.15.86:0/2179034509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c9780 msgr2=0x55cb52544bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a8f77640 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c9780 0x55cb52544bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a8f77640 1 -- 172.21.15.86:0/2179034509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 msgr2=0x55cb52540270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a8f77640 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 0x55cb52540270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a8f77640 1 -- 172.21.15.86:0/2179034509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb52540a70 con 0x55cb524c8e60 2024-04-16T05:41:55.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a3fff640 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c9780 0x55cb52544bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:55.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67a8f77640 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb52544680 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6798005ec0 tx=0x7f679802cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9294e0b70e08fbcc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f6783fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:55.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f6783fff640 1 -- 172.21.15.86:0/2179034509 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798005600 con 0x55cb524c8e60 2024-04-16T05:41:55.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f6783fff640 1 -- 172.21.15.86:0/2179034509 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6798004040 con 0x55cb524c8e60 2024-04-16T05:41:55.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.054+0000 7f67addc8740 1 -- 172.21.15.86:0/2179034509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6798005040 con 0x55cb524c8e60 2024-04-16T05:41:55.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f6783fff640 1 -- 172.21.15.86:0/2179034509 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6798004240 con 0x55cb524c8e60 2024-04-16T05:41:55.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f67addc8740 10 client.17634 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:55.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f67addc8740 1 -- 172.21.15.86:0/2179034509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb5254ce10 con 0x55cb524c8e60 2024-04-16T05:41:55.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f6783fff640 1 -- 172.21.15.86:0/2179034509 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6798005040 con 0x55cb524c8e60 2024-04-16T05:41:55.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f6783fff640 1 -- 172.21.15.86:0/2179034509 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6798059030 con 0x55cb524c8e60 2024-04-16T05:41:55.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f67addc8740 10 client.17634 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:55.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f67addc8740 10 client.17634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:41:55.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f67addc8740 20 client.17634 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:55.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f67addc8740 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55cb5254d4c0 0x55cb5254f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f67addc8740 4 client.17634 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:55.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.055+0000 7f67addc8740 1 -- 172.21.15.86:0/2179034509 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb52435970 con 0x55cb5254d4c0 2024-04-16T05:41:55.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67a3fff640 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55cb5254d4c0 0x55cb5254f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f6783fff640 10 client.17634 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:41:55.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67a3fff640 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55cb5254d4c0 0x55cb5254f8b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3995494589a67cf4 server_cookie=1fa0715cbe63bdfa in_seq=0 out_seq=0 2024-04-16T05:41:55.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f6783fff640 1 -- 172.21.15.86:0/2179034509 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb52435970 con 0x55cb5254d4c0 2024-04-16T05:41:55.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f6783fff640 10 client.17634 handle_command_reply: tid=0 2024-04-16T05:41:55.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67addc8740 1 client.17634 shutdown 2024-04-16T05:41:55.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67addc8740 1 -- 172.21.15.86:0/2179034509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 msgr2=0x55cb52544680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67addc8740 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb52544680 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6798005ec0 tx=0x7f679802cbe0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67addc8740 1 -- 172.21.15.86:0/2179034509 shutdown_connections 2024-04-16T05:41:55.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67addc8740 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb524d2360 0x55cb52540270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67addc8740 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb524c9780 0x55cb52544bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67addc8740 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55cb5254d4c0 0x55cb5254f8b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67addc8740 1 --2- 172.21.15.86:0/2179034509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb524c8e60 0x55cb52544680 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f6783fff640 0 client.17634 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:41:55.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.056+0000 7f67addc8740 1 -- 172.21.15.86:0/2179034509 >> 172.21.15.86:0/2179034509 conn(0x55cb524c3590 msgr2=0x55cb524cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67addc8740 1 -- 172.21.15.86:0/2179034509 shutdown_connections 2024-04-16T05:41:55.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67addc8740 1 -- 172.21.15.86:0/2179034509 wait complete. 2024-04-16T05:41:55.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67addc8740 20 client.17634 trim_cache size 0 max 16384 2024-04-16T05:41:55.068 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:55.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 -- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f67940a3da0 msgr2=0x7f67940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f67940a3da0 0x7f67940a6260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f679c009370 tx=0x7f679c009240 comp rx=0 tx=0).stop 2024-04-16T05:41:55.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 -- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4107320 msgr2=0x7f67a419ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4107320 0x7f67a419ebd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f679000d740 tx=0x7f679000dc00 comp rx=0 tx=0).stop 2024-04-16T05:41:55.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 -- 172.21.15.86:0/3446247462 shutdown_connections 2024-04-16T05:41:55.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67a4198750 0x7f67a4198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67a4107320 0x7f67a419ebd0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f67940a3da0 0x7f67940a6260 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 --2- 172.21.15.86:0/3446247462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a4102250 0x7f67a419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 -- 172.21.15.86:0/3446247462 >> 172.21.15.86:0/3446247462 conn(0x7f67a4079490 msgr2=0x7f67a40ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 -- 172.21.15.86:0/3446247462 shutdown_connections 2024-04-16T05:41:55.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.057+0000 7f67ab1db640 1 -- 172.21.15.86:0/3446247462 wait complete. 2024-04-16T05:41:55.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:41:55.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.177+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2045171748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101150 msgr2=0x7f9c881015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.177+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2045171748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101150 0x7f9c881015b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009900 tx=0x7f9c7c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.177+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2045171748 shutdown_connections 2024-04-16T05:41:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.177+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2045171748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c88101af0 0x7f9c8810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.177+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2045171748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101150 0x7f9c881015b0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.177+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2045171748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88105e80 0x7f9c88106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.177+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2045171748 >> 172.21.15.86:0/2045171748 conn(0x7f9c88077cf0 msgr2=0x7f9c88078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.177+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2045171748 shutdown_connections 2024-04-16T05:41:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.177+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2045171748 wait complete. 2024-04-16T05:41:55.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.178+0000 7f9c8d24a640 1 Processor -- start 2024-04-16T05:41:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.178+0000 7f9c8d24a640 1 -- start start 2024-04-16T05:41:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8d24a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88101150 0x7f9c8819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8d24a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101af0 0x7f9c8819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8d24a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c88105e80 0x7f9c88197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8d24a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88111370 con 0x7f9c88101af0 2024-04-16T05:41:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8d24a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881111f0 con 0x7f9c88105e80 2024-04-16T05:41:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8d24a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881114f0 con 0x7f9c88101150 2024-04-16T05:41:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101af0 0x7f9c8819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101af0 0x7f9c8819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41938/0 (socket says 172.21.15.86:41938) 2024-04-16T05:41:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8659c640 1 -- 172.21.15.86:0/2700760204 learned_addr learned my addr 172.21.15.86:0/2700760204 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88101150 0x7f9c8819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8759e640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c88105e80 0x7f9c88197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8659c640 1 -- 172.21.15.86:0/2700760204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88101150 msgr2=0x7f9c8819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8659c640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88101150 0x7f9c8819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8659c640 1 -- 172.21.15.86:0/2700760204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c88105e80 msgr2=0x7f9c88197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8659c640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c88105e80 0x7f9c88197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8659c640 1 -- 172.21.15.86:0/2700760204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c88197ef0 con 0x7f9c88101af0 2024-04-16T05:41:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8659c640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101af0 0x7f9c8819dab0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c006010 tx=0x7f9c7c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c6ffff640 1 -- 172.21.15.86:0/2700760204 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c041070 con 0x7f9c88101af0 2024-04-16T05:41:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2700760204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c88198180 con 0x7f9c88101af0 2024-04-16T05:41:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c6ffff640 1 -- 172.21.15.86:0/2700760204 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7c004450 con 0x7f9c88101af0 2024-04-16T05:41:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c6ffff640 1 -- 172.21.15.86:0/2700760204 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c034440 con 0x7f9c88101af0 2024-04-16T05:41:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.179+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2700760204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c881a44f0 con 0x7f9c88101af0 2024-04-16T05:41:55.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c6ffff640 1 -- 172.21.15.86:0/2700760204 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9c7c044070 con 0x7f9c88101af0 2024-04-16T05:41:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c8fe34740 1 Processor -- start 2024-04-16T05:41:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c8fe34740 1 -- start start 2024-04-16T05:41:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 0x5598b25f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b2600030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2602960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5598b255a0e0 con 0x5598b2600570 2024-04-16T05:41:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5598b24f7520 con 0x5598b25f7990 2024-04-16T05:41:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598b25560a0 con 0x5598b25f7070 2024-04-16T05:41:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c6ffff640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9c700a3d20 0x7f9c700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.182+0000 7f9c8659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2602960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2602960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41942/0 (socket says 172.21.15.86:41942) 2024-04-16T05:41:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8659c640 1 -- 172.21.15.86:0/933201576 learned_addr learned my addr 172.21.15.86:0/933201576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8759e640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9c700a3d20 0x7f9c700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/933201576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b2600030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8759e640 1 --2- 172.21.15.86:0/933201576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 0x5598b25f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c6ffff640 1 -- 172.21.15.86:0/2700760204 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9c7c0f2670 con 0x7f9c88101af0 2024-04-16T05:41:55.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8659c640 1 -- 172.21.15.86:0/933201576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 msgr2=0x5598b25f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8659c640 1 --2- 172.21.15.86:0/933201576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 0x5598b25f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8659c640 1 -- 172.21.15.86:0/933201576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 msgr2=0x5598b2600030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8659c640 1 --2- 172.21.15.86:0/933201576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b2600030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8659c640 1 -- 172.21.15.86:0/933201576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b251f8d0 con 0x5598b2600570 2024-04-16T05:41:55.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8759e640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9c700a3d20 0x7f9c700a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009fd0 tx=0x7f9c80009d00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c8659c640 1 --2- 172.21.15.86:0/933201576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2602960 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0c6af0 tx=0x7f9c7c003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de2fbe86a88d29 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c6d7fa640 1 -- 172.21.15.86:0/933201576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c04b070 con 0x5598b2600570 2024-04-16T05:41:55.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c6d7fa640 1 -- 172.21.15.86:0/933201576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7c0beab0 con 0x5598b2600570 2024-04-16T05:41:55.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.183+0000 7f9c6d7fa640 1 -- 172.21.15.86:0/933201576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0c9650 con 0x5598b2600570 2024-04-16T05:41:55.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 -- 172.21.15.86:0/933201576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 msgr2=0x5598b2602960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/933201576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2602960 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0c6af0 tx=0x7f9c7c003990 comp rx=0 tx=0).stop 2024-04-16T05:41:55.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 -- 172.21.15.86:0/933201576 shutdown_connections 2024-04-16T05:41:55.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/933201576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2602960 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/933201576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b2600030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/933201576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 0x5598b25f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 -- 172.21.15.86:0/933201576 >> 172.21.15.86:0/933201576 conn(0x5598b25f17a0 msgr2=0x5598b25f1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 -- 172.21.15.86:0/933201576 shutdown_connections 2024-04-16T05:41:55.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 -- 172.21.15.86:0/933201576 wait complete. 2024-04-16T05:41:55.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 -- start start 2024-04-16T05:41:55.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.184+0000 7f9c8fe34740 1 Processor -- start 2024-04-16T05:41:55.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25fd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b25fdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b2600570 0x5598b25fdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5598b255a0e0 con 0x5598b25f7070 2024-04-16T05:41:55.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5598b25134d0 con 0x5598b25f7990 2024-04-16T05:41:55.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598b2603120 con 0x5598b2600570 2024-04-16T05:41:55.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c86d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c86d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25fd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41944/0 (socket says 172.21.15.86:41944) 2024-04-16T05:41:55.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c8fe34740 10 client.0 _ll_register_callbacks cb 0x5598b24f82c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:55.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c86d9d640 1 -- 172.21.15.86:0/4284621027 learned_addr learned my addr 172.21.15.86:0/4284621027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c86d9d640 1 -- 172.21.15.86:0/4284621027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b2600570 msgr2=0x5598b25fdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b2600570 0x5598b25fdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c86d9d640 1 -- 172.21.15.86:0/4284621027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 msgr2=0x5598b25fdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c8659c640 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b25fdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b25fdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c86d9d640 1 -- 172.21.15.86:0/4284621027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b226c3f0 con 0x5598b25f7070 2024-04-16T05:41:55.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25fd570 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9c78010820 tx=0x7f9c78010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a62d6b2f4716e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.185+0000 7f9c6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:55.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c6aff5640 1 -- 172.21.15.86:0/4284621027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7801e060 con 0x5598b25f7070 2024-04-16T05:41:55.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c6aff5640 1 -- 172.21.15.86:0/4284621027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9c7800fb90 con 0x5598b25f7070 2024-04-16T05:41:55.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c8fe34740 1 -- 172.21.15.86:0/4284621027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c7c0beab0 con 0x5598b25f7070 2024-04-16T05:41:55.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c6aff5640 1 -- 172.21.15.86:0/4284621027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c78025050 con 0x5598b25f7070 2024-04-16T05:41:55.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c8fe34740 10 client.17646 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:55.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c8fe34740 1 -- 172.21.15.86:0/4284621027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598b23367d0 con 0x5598b25f7070 2024-04-16T05:41:55.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c6aff5640 1 -- 172.21.15.86:0/4284621027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7c0beab0 con 0x5598b25f7070 2024-04-16T05:41:55.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c6aff5640 1 -- 172.21.15.86:0/4284621027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c7800e3b0 con 0x5598b25f7070 2024-04-16T05:41:55.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c8fe34740 10 client.17646 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:55.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c8fe34740 10 client.17646 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:41:55.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c8fe34740 20 client.17646 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:55.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5598b25f0190 0x5598b25f0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c8fe34740 4 client.17646 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:55.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.186+0000 7f9c8fe34740 1 -- 172.21.15.86:0/4284621027 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598b2567b00 con 0x5598b25f0190 2024-04-16T05:41:55.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.187+0000 7f9c8659c640 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5598b25f0190 0x5598b25f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.187+0000 7f9c8659c640 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5598b25f0190 0x5598b25f0620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba1e8e386f9ed1ef server_cookie=8728d0d0f4a9a599 in_seq=0 out_seq=0 2024-04-16T05:41:55.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.187+0000 7f9c6aff5640 10 client.17646 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:41:55.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c6aff5640 1 -- 172.21.15.86:0/4284621027 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598b2567b00 con 0x5598b25f0190 2024-04-16T05:41:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c6aff5640 10 client.17646 handle_command_reply: tid=0 2024-04-16T05:41:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 client.17646 shutdown 2024-04-16T05:41:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 -- 172.21.15.86:0/4284621027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 msgr2=0x5598b25fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25fd570 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9c78010820 tx=0x7f9c78010ce0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 -- 172.21.15.86:0/4284621027 shutdown_connections 2024-04-16T05:41:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5598b25f0190 0x5598b25f0620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b2600570 0x5598b25fdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b25fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c6aff5640 0 client.17646 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:41:55.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/4284621027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25fd570 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 -- 172.21.15.86:0/4284621027 >> 172.21.15.86:0/4284621027 conn(0x5598b25f17a0 msgr2=0x5598b25ee730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 -- 172.21.15.86:0/4284621027 shutdown_connections 2024-04-16T05:41:55.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 1 -- 172.21.15.86:0/4284621027 wait complete. 2024-04-16T05:41:55.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.188+0000 7f9c8fe34740 20 client.17646 trim_cache size 0 max 16384 2024-04-16T05:41:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c8fe34740 1 Processor -- start 2024-04-16T05:41:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c8fe34740 1 -- start start 2024-04-16T05:41:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7990 0x5598b25fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b2600570 0x5598b25fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881ad190 con 0x5598b25f7070 2024-04-16T05:41:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881aecd0 con 0x5598b2600570 2024-04-16T05:41:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881ad860 con 0x5598b25f7990 2024-04-16T05:41:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c86d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c86d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41946/0 (socket says 172.21.15.86:41946) 2024-04-16T05:41:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c86d9d640 1 -- 172.21.15.86:0/693923251 learned_addr learned my addr 172.21.15.86:0/693923251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c86d9d640 1 -- 172.21.15.86:0/693923251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7990 msgr2=0x5598b25fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/693923251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7990 0x5598b25fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c86d9d640 1 -- 172.21.15.86:0/693923251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b2600570 msgr2=0x5598b25fad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/693923251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b2600570 0x5598b25fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.201+0000 7f9c86d9d640 1 -- 172.21.15.86:0/693923251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b25f77c0 con 0x5598b25f7070 2024-04-16T05:41:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/693923251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25ffca0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9c780058c0 tx=0x7f9c7800a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440e5036ff30a14e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c6cff9640 1 -- 172.21.15.86:0/693923251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7802e070 con 0x5598b25f7070 2024-04-16T05:41:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c6cff9640 1 -- 172.21.15.86:0/693923251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7800e3b0 con 0x5598b25f7070 2024-04-16T05:41:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c6cff9640 1 -- 172.21.15.86:0/693923251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78005490 con 0x5598b25f7070 2024-04-16T05:41:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c8fe34740 1 -- 172.21.15.86:0/693923251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 msgr2=0x5598b25ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/693923251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25ffca0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9c780058c0 tx=0x7f9c7800a450 comp rx=0 tx=0).stop 2024-04-16T05:41:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c8fe34740 1 -- 172.21.15.86:0/693923251 shutdown_connections 2024-04-16T05:41:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/693923251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b2600570 0x5598b25fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/693923251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7990 0x5598b25fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/693923251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b25f7070 0x5598b25ffca0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c8fe34740 1 -- 172.21.15.86:0/693923251 >> 172.21.15.86:0/693923251 conn(0x5598b25f17a0 msgr2=0x5598b25ea5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c8fe34740 1 -- 172.21.15.86:0/693923251 shutdown_connections 2024-04-16T05:41:55.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.202+0000 7f9c8fe34740 1 -- 172.21.15.86:0/693923251 wait complete. 2024-04-16T05:41:55.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8fe34740 1 -- start start 2024-04-16T05:41:55.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8fe34740 1 Processor -- start 2024-04-16T05:41:55.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 0x5598b2615c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b2616150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8fe34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2616690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881ae600 con 0x5598b2600570 2024-04-16T05:41:55.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881adf30 con 0x5598b25f7990 2024-04-16T05:41:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8fe34740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881ab150 con 0x5598b25f7070 2024-04-16T05:41:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8fe34740 10 client.0 _ll_register_callbacks cb 0x7f9c7800a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2616690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2616690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41954/0 (socket says 172.21.15.86:41954) 2024-04-16T05:41:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8759e640 1 -- 172.21.15.86:0/2446562126 learned_addr learned my addr 172.21.15.86:0/2446562126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 0x5598b2615c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8759e640 1 -- 172.21.15.86:0/2446562126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 msgr2=0x5598b2615c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8759e640 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 0x5598b2615c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8759e640 1 -- 172.21.15.86:0/2446562126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 msgr2=0x5598b2616150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8759e640 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b2616150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.203+0000 7f9c8759e640 1 -- 172.21.15.86:0/2446562126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b2677b40 con 0x5598b2600570 2024-04-16T05:41:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8759e640 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2616690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9c80010cf0 tx=0x7f9c80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04c02b0db5d22c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c6aff5640 1 -- 172.21.15.86:0/2446562126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80024070 con 0x5598b2600570 2024-04-16T05:41:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c6aff5640 1 -- 172.21.15.86:0/2446562126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9c8000cca0 con 0x5598b2600570 2024-04-16T05:41:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8fe34740 1 -- 172.21.15.86:0/2446562126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c7800e3b0 con 0x5598b2600570 2024-04-16T05:41:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c6aff5640 1 -- 172.21.15.86:0/2446562126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c80033050 con 0x5598b2600570 2024-04-16T05:41:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8fe34740 10 client.17652 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8fe34740 1 -- 172.21.15.86:0/2446562126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598b2678d80 con 0x5598b2600570 2024-04-16T05:41:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c6aff5640 1 -- 172.21.15.86:0/2446562126 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7800e3b0 con 0x5598b2600570 2024-04-16T05:41:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c6aff5640 1 -- 172.21.15.86:0/2446562126 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c8000c070 con 0x5598b2600570 2024-04-16T05:41:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8fe34740 10 client.17652 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8fe34740 10 client.17652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:41:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8fe34740 20 client.17652 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:55.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5598b2679550 0x5598b267b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8fe34740 4 client.17652 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:55.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c8fe34740 1 -- 172.21.15.86:0/2446562126 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598b2561970 con 0x5598b2679550 2024-04-16T05:41:55.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.204+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5598b2679550 0x5598b267b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c6aff5640 10 client.17652 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:41:55.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c86d9d640 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5598b2679550 0x5598b267b970 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dc885995855e3ba server_cookie=dbefd9984fb161ac in_seq=0 out_seq=0 2024-04-16T05:41:55.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c6aff5640 1 -- 172.21.15.86:0/2446562126 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598b2561970 con 0x5598b2679550 2024-04-16T05:41:55.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c6aff5640 10 client.17652 handle_command_reply: tid=0 2024-04-16T05:41:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 client.17652 shutdown 2024-04-16T05:41:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 -- 172.21.15.86:0/2446562126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 msgr2=0x5598b2616690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2616690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9c80010cf0 tx=0x7f9c80004ab0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 -- 172.21.15.86:0/2446562126 shutdown_connections 2024-04-16T05:41:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x5598b2679550 0x5598b267b970 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b2600570 0x5598b2616690 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b25f7990 0x5598b2616150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c6aff5640 0 client.17652 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:41:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 --2- 172.21.15.86:0/2446562126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598b25f7070 0x5598b2615c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 -- 172.21.15.86:0/2446562126 >> 172.21.15.86:0/2446562126 conn(0x5598b25f17a0 msgr2=0x5598b260b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 -- 172.21.15.86:0/2446562126 shutdown_connections 2024-04-16T05:41:55.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 1 -- 172.21.15.86:0/2446562126 wait complete. 2024-04-16T05:41:55.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.205+0000 7f9c8fe34740 20 client.17652 trim_cache size 0 max 16384 2024-04-16T05:41:55.216 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9c700a3d20 msgr2=0x7f9c700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9c700a3d20 0x7f9c700a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009fd0 tx=0x7f9c80009d00 comp rx=0 tx=0).stop 2024-04-16T05:41:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101af0 msgr2=0x7f9c8819dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101af0 0x7f9c8819dab0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c006010 tx=0x7f9c7c02cc70 comp rx=0 tx=0).stop 2024-04-16T05:41:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2700760204 shutdown_connections 2024-04-16T05:41:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f9c700a3d20 0x7f9c700a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c88105e80 0x7f9c88197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c88101af0 0x7f9c8819dab0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 --2- 172.21.15.86:0/2700760204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88101150 0x7f9c8819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2700760204 >> 172.21.15.86:0/2700760204 conn(0x7f9c88077cf0 msgr2=0x7f9c880fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2700760204 shutdown_connections 2024-04-16T05:41:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.206+0000 7f9c8d24a640 1 -- 172.21.15.86:0/2700760204 wait complete. 2024-04-16T05:41:55.219 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:41:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.326+0000 7f84bc331640 1 -- 172.21.15.86:0/2922849799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 msgr2=0x7f84b410ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.326+0000 7f84bc331640 1 --2- 172.21.15.86:0/2922849799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 0x7f84b410ee90 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009ca0 tx=0x7f84ac02c180 comp rx=0 tx=0).stop 2024-04-16T05:41:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.326+0000 7f84bc331640 1 -- 172.21.15.86:0/2922849799 shutdown_connections 2024-04-16T05:41:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.326+0000 7f84bc331640 1 --2- 172.21.15.86:0/2922849799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 0x7f84b410ee90 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.326+0000 7f84bc331640 1 --2- 172.21.15.86:0/2922849799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b41089d0 0x7f84b406a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.326+0000 7f84bc331640 1 --2- 172.21.15.86:0/2922849799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b41080b0 0x7f84b4108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.326+0000 7f84bc331640 1 -- 172.21.15.86:0/2922849799 >> 172.21.15.86:0/2922849799 conn(0x7f84b4076d00 msgr2=0x7f84b4077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.326+0000 7f84bc331640 1 -- 172.21.15.86:0/2922849799 shutdown_connections 2024-04-16T05:41:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.326+0000 7f84bc331640 1 -- 172.21.15.86:0/2922849799 wait complete. 2024-04-16T05:41:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84bc331640 1 Processor -- start 2024-04-16T05:41:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84bc331640 1 -- start start 2024-04-16T05:41:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84bc331640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 0x7f84b419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84bc331640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b41080b0 0x7f84b419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84bc331640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b41089d0 0x7f84b41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84bc331640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4112390 con 0x7f84b406ad90 2024-04-16T05:41:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84bc331640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4112210 con 0x7f84b41080b0 2024-04-16T05:41:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84bc331640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4112510 con 0x7f84b41089d0 2024-04-16T05:41:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84ba0cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 0x7f84b419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84ba0cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 0x7f84b419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41958/0 (socket says 172.21.15.86:41958) 2024-04-16T05:41:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84ba0cd640 1 -- 172.21.15.86:0/398663965 learned_addr learned my addr 172.21.15.86:0/398663965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.327+0000 7f84ba8ce640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b41089d0 0x7f84b41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84b98cc640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b41080b0 0x7f84b419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84ba0cd640 1 -- 172.21.15.86:0/398663965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b41089d0 msgr2=0x7f84b41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b41089d0 0x7f84b41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84ba0cd640 1 -- 172.21.15.86:0/398663965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b41080b0 msgr2=0x7f84b419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b41080b0 0x7f84b419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84ba0cd640 1 -- 172.21.15.86:0/398663965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b4198fb0 con 0x7f84b406ad90 2024-04-16T05:41:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84b98cc640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b41080b0 0x7f84b419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 0x7f84b419e5e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c970 tx=0x7f84b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84ab7fe640 1 -- 172.21.15.86:0/398663965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0007bb0 con 0x7f84b406ad90 2024-04-16T05:41:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84ab7fe640 1 -- 172.21.15.86:0/398663965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b0007d50 con 0x7f84b406ad90 2024-04-16T05:41:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84bc331640 1 -- 172.21.15.86:0/398663965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b41992a0 con 0x7f84b406ad90 2024-04-16T05:41:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84ab7fe640 1 -- 172.21.15.86:0/398663965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b000f450 con 0x7f84b406ad90 2024-04-16T05:41:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.328+0000 7f84bc331640 1 -- 172.21.15.86:0/398663965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a5560 con 0x7f84b406ad90 2024-04-16T05:41:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.330+0000 7f84ab7fe640 1 -- 172.21.15.86:0/398663965 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84b0005020 con 0x7f84b406ad90 2024-04-16T05:41:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84beff0740 1 Processor -- start 2024-04-16T05:41:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84beff0740 1 -- start start 2024-04-16T05:41:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84beff0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e72240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84beff0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e7ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84beff0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43e7d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84beff0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba43dd70e0 con 0x55ba43e71e60 2024-04-16T05:41:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84beff0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ba43d74520 con 0x55ba43e72780 2024-04-16T05:41:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84beff0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba43dd30a0 con 0x55ba43e7b360 2024-04-16T05:41:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84ba8ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e72240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84ab7fe640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84a00a3d20 0x7f84a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84ba8ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e72240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41968/0 (socket says 172.21.15.86:41968) 2024-04-16T05:41:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84ba8ce640 1 -- 172.21.15.86:0/936644770 learned_addr learned my addr 172.21.15.86:0/936644770 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84ba8ce640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84a00a3d20 0x7f84a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.331+0000 7f84b98cc640 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43e7d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84ab7fe640 1 -- 172.21.15.86:0/398663965 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f84b00d0a70 con 0x7f84b406ad90 2024-04-16T05:41:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e7ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84b98cc640 1 -- 172.21.15.86:0/936644770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 msgr2=0x55ba43e7ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84b98cc640 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e7ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84b98cc640 1 -- 172.21.15.86:0/936644770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 msgr2=0x55ba43e72240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84b98cc640 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e72240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84b98cc640 1 -- 172.21.15.86:0/936644770 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba43d9c8d0 con 0x55ba43e7b360 2024-04-16T05:41:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84ba8ce640 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e72240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:41:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84ba8ce640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84a00a3d20 0x7f84a00a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009ca0 tx=0x7f84ac02c180 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84b98cc640 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43e7d750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84a400bab0 tx=0x7f84a400bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc7d437295dd7d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84a8bfe640 1 -- 172.21.15.86:0/936644770 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4017420 con 0x55ba43e7b360 2024-04-16T05:41:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84a8bfe640 1 -- 172.21.15.86:0/936644770 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a4017aa0 con 0x55ba43e7b360 2024-04-16T05:41:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.332+0000 7f84a8bfe640 1 -- 172.21.15.86:0/936644770 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a400fad0 con 0x55ba43e7b360 2024-04-16T05:41:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.333+0000 7f84beff0740 1 -- 172.21.15.86:0/936644770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 msgr2=0x55ba43e7d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.333+0000 7f84beff0740 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43e7d750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84a400bab0 tx=0x7f84a400bf70 comp rx=0 tx=0).stop 2024-04-16T05:41:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.333+0000 7f84beff0740 1 -- 172.21.15.86:0/936644770 shutdown_connections 2024-04-16T05:41:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.333+0000 7f84beff0740 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43e7d750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.333+0000 7f84beff0740 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e7ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.333+0000 7f84beff0740 1 --2- 172.21.15.86:0/936644770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e72240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.333+0000 7f84beff0740 1 -- 172.21.15.86:0/936644770 >> 172.21.15.86:0/936644770 conn(0x55ba43e6c590 msgr2=0x55ba43e6c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.333+0000 7f84beff0740 1 -- 172.21.15.86:0/936644770 shutdown_connections 2024-04-16T05:41:55.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.333+0000 7f84beff0740 1 -- 172.21.15.86:0/936644770 wait complete. 2024-04-16T05:41:55.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84beff0740 1 -- start start 2024-04-16T05:41:55.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84beff0740 1 Processor -- start 2024-04-16T05:41:55.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84beff0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e71e60 0x55ba43e783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84beff0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e72780 0x55ba43e78900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84beff0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e7b360 0x55ba43e78e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84beff0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba43e7e0c0 con 0x55ba43e72780 2024-04-16T05:41:55.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84beff0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ba43d904d0 con 0x55ba43e7b360 2024-04-16T05:41:55.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84beff0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba43e7e240 con 0x55ba43e71e60 2024-04-16T05:41:55.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84beff0740 10 client.0 _ll_register_callbacks cb 0x55ba43d752c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84b98cc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e72780 0x55ba43e78900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84b98cc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e72780 0x55ba43e78900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41982/0 (socket says 172.21.15.86:41982) 2024-04-16T05:41:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84b98cc640 1 -- 172.21.15.86:0/3140775315 learned_addr learned my addr 172.21.15.86:0/3140775315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e71e60 0x55ba43e783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84b98cc640 1 -- 172.21.15.86:0/3140775315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e71e60 msgr2=0x55ba43e783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84ba8ce640 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e7b360 0x55ba43e78e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84b98cc640 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e71e60 0x55ba43e783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84b98cc640 1 -- 172.21.15.86:0/3140775315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e7b360 msgr2=0x55ba43e78e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84b98cc640 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e7b360 0x55ba43e78e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84b98cc640 1 -- 172.21.15.86:0/3140775315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba43ae93f0 con 0x55ba43e72780 2024-04-16T05:41:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e71e60 0x55ba43e783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84ba8ce640 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e7b360 0x55ba43e78e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f84b98cc640 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e72780 0x55ba43e78900 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f84a4019040 tx=0x7f84a400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd1d168b2cfa291 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.334+0000 7f849dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f849dffb640 1 -- 172.21.15.86:0/3140775315 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4025070 con 0x55ba43e72780 2024-04-16T05:41:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f849dffb640 1 -- 172.21.15.86:0/3140775315 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f84a4005040 con 0x55ba43e72780 2024-04-16T05:41:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f84beff0740 1 -- 172.21.15.86:0/3140775315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a4017aa0 con 0x55ba43e72780 2024-04-16T05:41:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f849dffb640 1 -- 172.21.15.86:0/3140775315 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84a4031050 con 0x55ba43e72780 2024-04-16T05:41:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f84beff0740 10 client.17664 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f84beff0740 1 -- 172.21.15.86:0/3140775315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba43bb37d0 con 0x55ba43e72780 2024-04-16T05:41:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f849dffb640 1 -- 172.21.15.86:0/3140775315 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a4017aa0 con 0x55ba43e72780 2024-04-16T05:41:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f849dffb640 1 -- 172.21.15.86:0/3140775315 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f84a400f630 con 0x55ba43e72780 2024-04-16T05:41:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f84beff0740 10 client.17664 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f84beff0740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f84beff0740 20 client.17664 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f84beff0740 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ba43e8b2d0 0x55ba43e8d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f84beff0740 4 client.17664 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.335+0000 7f84beff0740 1 -- 172.21.15.86:0/3140775315 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba43de4b00 con 0x55ba43e8b2d0 2024-04-16T05:41:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.336+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ba43e8b2d0 0x55ba43e8d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.336+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ba43e8b2d0 0x55ba43e8d6c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a44c499c87e0c4d server_cookie=907978bd6be83f04 in_seq=0 out_seq=0 2024-04-16T05:41:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.336+0000 7f849dffb640 10 client.17664 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f849dffb640 1 -- 172.21.15.86:0/3140775315 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba43de4b00 con 0x55ba43e8b2d0 2024-04-16T05:41:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f849dffb640 10 client.17664 handle_command_reply: tid=0 2024-04-16T05:41:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 client.17664 shutdown 2024-04-16T05:41:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 -- 172.21.15.86:0/3140775315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e72780 msgr2=0x55ba43e78900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e72780 0x55ba43e78900 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f84a4019040 tx=0x7f84a400f490 comp rx=0 tx=0).stop 2024-04-16T05:41:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 -- 172.21.15.86:0/3140775315 shutdown_connections 2024-04-16T05:41:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ba43e8b2d0 0x55ba43e8d6c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e7b360 0x55ba43e78e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e72780 0x55ba43e78900 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f849dffb640 0 client.17664 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 --2- 172.21.15.86:0/3140775315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e71e60 0x55ba43e783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 -- 172.21.15.86:0/3140775315 >> 172.21.15.86:0/3140775315 conn(0x55ba43e6c590 msgr2=0x55ba43e6e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 -- 172.21.15.86:0/3140775315 shutdown_connections 2024-04-16T05:41:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 1 -- 172.21.15.86:0/3140775315 wait complete. 2024-04-16T05:41:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.337+0000 7f84beff0740 20 client.17664 trim_cache size 0 max 16384 2024-04-16T05:41:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.350+0000 7f84beff0740 1 Processor -- start 2024-04-16T05:41:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.350+0000 7f84beff0740 1 -- start start 2024-04-16T05:41:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.350+0000 7f84beff0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e7aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.350+0000 7f84beff0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e87010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.350+0000 7f84beff0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43e675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.350+0000 7f84beff0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41af3e0 con 0x55ba43e71e60 2024-04-16T05:41:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.350+0000 7f84beff0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41b0f20 con 0x55ba43e72780 2024-04-16T05:41:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.350+0000 7f84beff0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41afab0 con 0x55ba43e7b360 2024-04-16T05:41:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f84ba0cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e7aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f84ba0cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e7aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41992/0 (socket says 172.21.15.86:41992) 2024-04-16T05:41:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f84ba0cd640 1 -- 172.21.15.86:0/3218438351 learned_addr learned my addr 172.21.15.86:0/3218438351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f84ba0cd640 1 -- 172.21.15.86:0/3218438351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 msgr2=0x55ba43e675c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/3218438351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43e675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f84ba0cd640 1 -- 172.21.15.86:0/3218438351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 msgr2=0x55ba43e87010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/3218438351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e87010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f84ba0cd640 1 -- 172.21.15.86:0/3218438351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba43e77370 con 0x55ba43e71e60 2024-04-16T05:41:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/3218438351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e7aaf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f84b0098ac0 tx=0x7f84b00d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca007f58a553c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f849ffff640 1 -- 172.21.15.86:0/3218438351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b001d040 con 0x55ba43e71e60 2024-04-16T05:41:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f849ffff640 1 -- 172.21.15.86:0/3218438351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b00a0a10 con 0x55ba43e71e60 2024-04-16T05:41:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.351+0000 7f849ffff640 1 -- 172.21.15.86:0/3218438351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b00d0e60 con 0x55ba43e71e60 2024-04-16T05:41:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 -- 172.21.15.86:0/3218438351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 msgr2=0x55ba43e7aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 --2- 172.21.15.86:0/3218438351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e7aaf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f84b0098ac0 tx=0x7f84b00d2040 comp rx=0 tx=0).stop 2024-04-16T05:41:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 -- 172.21.15.86:0/3218438351 shutdown_connections 2024-04-16T05:41:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 --2- 172.21.15.86:0/3218438351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43e675c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 --2- 172.21.15.86:0/3218438351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e87010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 --2- 172.21.15.86:0/3218438351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e7aaf0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 -- 172.21.15.86:0/3218438351 >> 172.21.15.86:0/3218438351 conn(0x55ba43e6c590 msgr2=0x55ba43e6e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 -- 172.21.15.86:0/3218438351 shutdown_connections 2024-04-16T05:41:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 -- 172.21.15.86:0/3218438351 wait complete. 2024-04-16T05:41:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 -- start start 2024-04-16T05:41:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 Processor -- start 2024-04-16T05:41:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e75950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43ef0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41b0850 con 0x55ba43e71e60 2024-04-16T05:41:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41b0180 con 0x55ba43e72780 2024-04-16T05:41:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41ad3a0 con 0x55ba43e7b360 2024-04-16T05:41:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.352+0000 7f84beff0740 10 client.0 _ll_register_callbacks cb 0x7f84b407a330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba0cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba0cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e75410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42004/0 (socket says 172.21.15.86:42004) 2024-04-16T05:41:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba0cd640 1 -- 172.21.15.86:0/1678278917 learned_addr learned my addr 172.21.15.86:0/1678278917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba8ce640 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43ef0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba0cd640 1 -- 172.21.15.86:0/1678278917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 msgr2=0x55ba43ef0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43ef0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba0cd640 1 -- 172.21.15.86:0/1678278917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 msgr2=0x55ba43e75950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e75950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba0cd640 1 -- 172.21.15.86:0/1678278917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba43ef0b90 con 0x55ba43e71e60 2024-04-16T05:41:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84ba0cd640 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e75410 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f84b009c360 tx=0x7f84b009ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72afa8b48a1cdf8f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f849dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f849dffb640 1 -- 172.21.15.86:0/1678278917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0066060 con 0x55ba43e71e60 2024-04-16T05:41:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f849dffb640 1 -- 172.21.15.86:0/1678278917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f84b0098cd0 con 0x55ba43e71e60 2024-04-16T05:41:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84beff0740 1 -- 172.21.15.86:0/1678278917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b00a0a10 con 0x55ba43e71e60 2024-04-16T05:41:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f849dffb640 1 -- 172.21.15.86:0/1678278917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84b001d070 con 0x55ba43e71e60 2024-04-16T05:41:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84beff0740 10 client.17670 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84beff0740 1 -- 172.21.15.86:0/1678278917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba43ef1cd0 con 0x55ba43e71e60 2024-04-16T05:41:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f849dffb640 1 -- 172.21.15.86:0/1678278917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b00a0a10 con 0x55ba43e71e60 2024-04-16T05:41:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f849dffb640 1 -- 172.21.15.86:0/1678278917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f84b00ce120 con 0x55ba43e71e60 2024-04-16T05:41:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84beff0740 10 client.17670 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84beff0740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84beff0740 20 client.17670 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.353+0000 7f84beff0740 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ba43ef2420 0x55ba43ef4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 4 client.17670 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 -- 172.21.15.86:0/1678278917 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba43dde970 con 0x55ba43ef2420 2024-04-16T05:41:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84b98cc640 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ba43ef2420 0x55ba43ef4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f849dffb640 10 client.17670 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84b98cc640 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ba43ef2420 0x55ba43ef4840 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebe9c5d2d1ed5c8 server_cookie=b33a72e7b7b4b419 in_seq=0 out_seq=0 2024-04-16T05:41:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f849dffb640 1 -- 172.21.15.86:0/1678278917 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba43dde970 con 0x55ba43ef2420 2024-04-16T05:41:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f849dffb640 10 client.17670 handle_command_reply: tid=0 2024-04-16T05:41:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 client.17670 shutdown 2024-04-16T05:41:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 -- 172.21.15.86:0/1678278917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 msgr2=0x55ba43e75410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e75410 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f84b009c360 tx=0x7f84b009ab00 comp rx=0 tx=0).stop 2024-04-16T05:41:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 -- 172.21.15.86:0/1678278917 shutdown_connections 2024-04-16T05:41:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ba43ef2420 0x55ba43ef4840 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba43e7b360 0x55ba43ef0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba43e72780 0x55ba43e75950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f849dffb640 0 client.17670 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 --2- 172.21.15.86:0/1678278917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba43e71e60 0x55ba43e75410 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.354+0000 7f84beff0740 1 -- 172.21.15.86:0/1678278917 >> 172.21.15.86:0/1678278917 conn(0x55ba43e6c590 msgr2=0x55ba43e6aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84beff0740 1 -- 172.21.15.86:0/1678278917 shutdown_connections 2024-04-16T05:41:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84beff0740 1 -- 172.21.15.86:0/1678278917 wait complete. 2024-04-16T05:41:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84beff0740 20 client.17670 trim_cache size 0 max 16384 2024-04-16T05:41:55.369 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:41:55.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 -- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84a00a3d20 msgr2=0x7f84a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84a00a3d20 0x7f84a00a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009ca0 tx=0x7f84ac02c180 comp rx=0 tx=0).stop 2024-04-16T05:41:55.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 -- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 msgr2=0x7f84b419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:55.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 0x7f84b419e5e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c970 tx=0x7f84b000ce30 comp rx=0 tx=0).stop 2024-04-16T05:41:55.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 -- 172.21.15.86:0/398663965 shutdown_connections 2024-04-16T05:41:55.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84b41089d0 0x7f84b41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84a00a3d20 0x7f84a00a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b41080b0 0x7f84b419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 --2- 172.21.15.86:0/398663965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b406ad90 0x7f84b419e5e0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 -- 172.21.15.86:0/398663965 >> 172.21.15.86:0/398663965 conn(0x7f84b4076d00 msgr2=0x7f84b40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 -- 172.21.15.86:0/398663965 shutdown_connections 2024-04-16T05:41:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:55.355+0000 7f84bc331640 1 -- 172.21.15.86:0/398663965 wait complete. 2024-04-16T05:41:55.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:55 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1753806696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:41:55.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:55 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1753806696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:41:55.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:55 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1753806696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:41:56.373 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:41:56.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:41:56.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:41:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.497+0000 7f82a8da8640 1 -- 172.21.15.86:0/1668393800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4106210 msgr2=0x7f82a410dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.497+0000 7f82a8da8640 1 --2- 172.21.15.86:0/1668393800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4106210 0x7f82a410dde0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f829c009ca0 tx=0x7f829c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.497+0000 7f82a8da8640 1 -- 172.21.15.86:0/1668393800 shutdown_connections 2024-04-16T05:41:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.497+0000 7f82a8da8640 1 --2- 172.21.15.86:0/1668393800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4106210 0x7f82a410dde0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.497+0000 7f82a8da8640 1 --2- 172.21.15.86:0/1668393800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82a4101af0 0x7f82a4105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.497+0000 7f82a8da8640 1 --2- 172.21.15.86:0/1668393800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a4101140 0x7f82a4101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.497+0000 7f82a8da8640 1 -- 172.21.15.86:0/1668393800 >> 172.21.15.86:0/1668393800 conn(0x7f82a4078000 msgr2=0x7f82a4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.497+0000 7f82a8da8640 1 -- 172.21.15.86:0/1668393800 shutdown_connections 2024-04-16T05:41:56.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.497+0000 7f82a8da8640 1 -- 172.21.15.86:0/1668393800 wait complete. 2024-04-16T05:41:56.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a8da8640 1 Processor -- start 2024-04-16T05:41:56.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a8da8640 1 -- start start 2024-04-16T05:41:56.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a8da8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4101140 0x7f82a4197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a8da8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82a4106210 0x7f82a4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a8da8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a419bf30 0x7f82a419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a8da8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41112b0 con 0x7f82a4101140 2024-04-16T05:41:56.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a8da8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4111130 con 0x7f82a4106210 2024-04-16T05:41:56.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a8da8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4111430 con 0x7f82a419bf30 2024-04-16T05:41:56.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4101140 0x7f82a4197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4101140 0x7f82a4197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42022/0 (socket says 172.21.15.86:42022) 2024-04-16T05:41:56.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a259c640 1 -- 172.21.15.86:0/2713482802 learned_addr learned my addr 172.21.15.86:0/2713482802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f829bfff640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82a4106210 0x7f82a4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a259c640 1 -- 172.21.15.86:0/2713482802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a419bf30 msgr2=0x7f82a419c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a259c640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a419bf30 0x7f82a419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a259c640 1 -- 172.21.15.86:0/2713482802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82a4106210 msgr2=0x7f82a4197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a259c640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82a4106210 0x7f82a4197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.498+0000 7f82a259c640 1 -- 172.21.15.86:0/2713482802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a419ca20 con 0x7f82a4101140 2024-04-16T05:41:56.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.499+0000 7f82a259c640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4101140 0x7f82a4197670 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f829400c920 tx=0x7f829400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.499+0000 7f829b7fe640 1 -- 172.21.15.86:0/2713482802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294007c70 con 0x7f82a4101140 2024-04-16T05:41:56.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.499+0000 7f829b7fe640 1 -- 172.21.15.86:0/2713482802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8294007e10 con 0x7f82a4101140 2024-04-16T05:41:56.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.499+0000 7f82a8da8640 1 -- 172.21.15.86:0/2713482802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a419cd10 con 0x7f82a4101140 2024-04-16T05:41:56.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.499+0000 7f829b7fe640 1 -- 172.21.15.86:0/2713482802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294018890 con 0x7f82a4101140 2024-04-16T05:41:56.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.499+0000 7f82a8da8640 1 -- 172.21.15.86:0/2713482802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a41a4500 con 0x7f82a4101140 2024-04-16T05:41:56.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.501+0000 7f829b7fe640 1 -- 172.21.15.86:0/2713482802 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8294018a70 con 0x7f82a4101140 2024-04-16T05:41:56.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.501+0000 7f82ab98c740 1 Processor -- start 2024-04-16T05:41:56.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82ab98c740 1 -- start start 2024-04-16T05:41:56.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede3415ab0 0x55ede3415e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede34163d0 0x55ede341ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede34213a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82ab98c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ede337ae90 con 0x55ede341efb0 2024-04-16T05:41:56.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82ab98c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ede33184c0 con 0x55ede34163d0 2024-04-16T05:41:56.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82ab98c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ede3376e50 con 0x55ede3415ab0 2024-04-16T05:41:56.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829b7fe640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82900a3d20 0x7f82900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede34213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede34213a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42032/0 (socket says 172.21.15.86:42032) 2024-04-16T05:41:56.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829bfff640 1 -- 172.21.15.86:0/213538392 learned_addr learned my addr 172.21.15.86:0/213538392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82a2d9d640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82900a3d20 0x7f82900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82a2d9d640 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede3415ab0 0x55ede3415e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829b7fe640 1 -- 172.21.15.86:0/2713482802 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f82940d0db0 con 0x7f82a4101140 2024-04-16T05:41:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82a259c640 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede34163d0 0x55ede341ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829bfff640 1 -- 172.21.15.86:0/213538392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede3415ab0 msgr2=0x55ede3415e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829bfff640 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede3415ab0 0x55ede3415e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829bfff640 1 -- 172.21.15.86:0/213538392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede34163d0 msgr2=0x55ede341ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829bfff640 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede34163d0 0x55ede341ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f829bfff640 1 -- 172.21.15.86:0/213538392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede31557e0 con 0x55ede341efb0 2024-04-16T05:41:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.502+0000 7f82a259c640 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede34163d0 0x55ede341ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.503+0000 7f82a2d9d640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82900a3d20 0x7f82900a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f829c02c630 tx=0x7f829c005d80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.503+0000 7f829bfff640 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede34213a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f828c00cb90 tx=0x7f828c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4508ea6b85f67a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.503+0000 7f8298ff9640 1 -- 172.21.15.86:0/213538392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c005220 con 0x55ede341efb0 2024-04-16T05:41:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.503+0000 7f8298ff9640 1 -- 172.21.15.86:0/213538392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f828c004c80 con 0x55ede341efb0 2024-04-16T05:41:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.503+0000 7f8298ff9640 1 -- 172.21.15.86:0/213538392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c010820 con 0x55ede341efb0 2024-04-16T05:41:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.503+0000 7f82ab98c740 1 -- 172.21.15.86:0/213538392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 msgr2=0x55ede34213a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede34213a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f828c00cb90 tx=0x7f828c00e720 comp rx=0 tx=0).stop 2024-04-16T05:41:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 -- 172.21.15.86:0/213538392 shutdown_connections 2024-04-16T05:41:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede34213a0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede34163d0 0x55ede341ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 --2- 172.21.15.86:0/213538392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede3415ab0 0x55ede3415e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 -- 172.21.15.86:0/213538392 >> 172.21.15.86:0/213538392 conn(0x55ede34101e0 msgr2=0x55ede34105c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 -- 172.21.15.86:0/213538392 shutdown_connections 2024-04-16T05:41:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 -- 172.21.15.86:0/213538392 wait complete. 2024-04-16T05:41:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 -- start start 2024-04-16T05:41:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 Processor -- start 2024-04-16T05:41:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede341c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34163d0 0x55ede341c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede341caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ede3421da0 con 0x55ede341efb0 2024-04-16T05:41:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ede3303410 con 0x55ede3415ab0 2024-04-16T05:41:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ede3421f20 con 0x55ede34163d0 2024-04-16T05:41:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82ab98c740 10 client.0 _ll_register_callbacks cb 0x55ede32feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82a2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede341caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82a2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede341caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42034/0 (socket says 172.21.15.86:42034) 2024-04-16T05:41:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.504+0000 7f82a2d9d640 1 -- 172.21.15.86:0/4149187698 learned_addr learned my addr 172.21.15.86:0/4149187698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82a259c640 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede341c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f829bfff640 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34163d0 0x55ede341c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82a2d9d640 1 -- 172.21.15.86:0/4149187698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34163d0 msgr2=0x55ede341c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82a2d9d640 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34163d0 0x55ede341c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82a2d9d640 1 -- 172.21.15.86:0/4149187698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 msgr2=0x55ede341c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82a2d9d640 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede341c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82a2d9d640 1 -- 172.21.15.86:0/4149187698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede31b5f70 con 0x55ede341efb0 2024-04-16T05:41:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82a2d9d640 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede341caa0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f829c036000 tx=0x7f829c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5f04c8a6ed0d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f8289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f8289ffb640 1 -- 172.21.15.86:0/4149187698 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c039040 con 0x55ede341efb0 2024-04-16T05:41:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f8289ffb640 1 -- 172.21.15.86:0/4149187698 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f829c042670 con 0x55ede341efb0 2024-04-16T05:41:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82ab98c740 1 -- 172.21.15.86:0/4149187698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f828c004c80 con 0x55ede341efb0 2024-04-16T05:41:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f8289ffb640 1 -- 172.21.15.86:0/4149187698 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f829c05b050 con 0x55ede341efb0 2024-04-16T05:41:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82ab98c740 10 client.17682 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f82ab98c740 1 -- 172.21.15.86:0/4149187698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ede304d170 con 0x55ede341efb0 2024-04-16T05:41:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.505+0000 7f8289ffb640 1 -- 172.21.15.86:0/4149187698 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f828c004c80 con 0x55ede341efb0 2024-04-16T05:41:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f8289ffb640 1 -- 172.21.15.86:0/4149187698 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f829c04c070 con 0x55ede341efb0 2024-04-16T05:41:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f82ab98c740 10 client.17682 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f82ab98c740 10 client.17682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f82ab98c740 20 client.17682 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f82ab98c740 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ede342f420 0x55ede34318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f82ab98c740 4 client.17682 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f82ab98c740 1 -- 172.21.15.86:0/4149187698 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ede33888b0 con 0x55ede342f420 2024-04-16T05:41:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f82a259c640 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ede342f420 0x55ede34318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f82a259c640 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ede342f420 0x55ede34318a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d490bb1f824298 server_cookie=9fe2ffcfe087d1ac in_seq=0 out_seq=0 2024-04-16T05:41:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.506+0000 7f8289ffb640 10 client.17682 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f8289ffb640 1 -- 172.21.15.86:0/4149187698 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ede33888b0 con 0x55ede342f420 2024-04-16T05:41:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f8289ffb640 10 client.17682 handle_command_reply: tid=0 2024-04-16T05:41:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 client.17682 shutdown 2024-04-16T05:41:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 -- 172.21.15.86:0/4149187698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 msgr2=0x55ede341caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede341caa0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f829c036000 tx=0x7f829c0095c0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 -- 172.21.15.86:0/4149187698 shutdown_connections 2024-04-16T05:41:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ede342f420 0x55ede34318a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede341efb0 0x55ede341caa0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34163d0 0x55ede341c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f8289ffb640 0 client.17682 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 --2- 172.21.15.86:0/4149187698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede341c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 -- 172.21.15.86:0/4149187698 >> 172.21.15.86:0/4149187698 conn(0x55ede34101e0 msgr2=0x55ede3411f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 -- 172.21.15.86:0/4149187698 shutdown_connections 2024-04-16T05:41:56.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.507+0000 7f82ab98c740 1 -- 172.21.15.86:0/4149187698 wait complete. 2024-04-16T05:41:56.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.508+0000 7f82ab98c740 20 client.17682 trim_cache size 0 max 16384 2024-04-16T05:41:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82ab98c740 1 Processor -- start 2024-04-16T05:41:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82ab98c740 1 -- start start 2024-04-16T05:41:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34101e0 0x55ede342ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede3437170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34163d0 0x55ede34376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82ab98c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41ae210 con 0x55ede34101e0 2024-04-16T05:41:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82ab98c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41afd50 con 0x55ede3415ab0 2024-04-16T05:41:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82ab98c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41ae8e0 con 0x55ede34163d0 2024-04-16T05:41:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82a259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34101e0 0x55ede342ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82a259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34101e0 0x55ede342ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42040/0 (socket says 172.21.15.86:42040) 2024-04-16T05:41:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82a259c640 1 -- 172.21.15.86:0/2947500633 learned_addr learned my addr 172.21.15.86:0/2947500633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.525+0000 7f82a259c640 1 -- 172.21.15.86:0/2947500633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34163d0 msgr2=0x55ede34376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82a259c640 1 --2- 172.21.15.86:0/2947500633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34163d0 0x55ede34376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82a259c640 1 -- 172.21.15.86:0/2947500633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 msgr2=0x55ede3437170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f829bfff640 1 --2- 172.21.15.86:0/2947500633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede3437170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82a259c640 1 --2- 172.21.15.86:0/2947500633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede3437170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82a259c640 1 -- 172.21.15.86:0/2947500633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede340e660 con 0x55ede34101e0 2024-04-16T05:41:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82a259c640 1 --2- 172.21.15.86:0/2947500633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34101e0 0x55ede342ac60 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8294098e00 tx=0x7f8294020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ba9a36ecdb14f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f828bfff640 1 -- 172.21.15.86:0/2947500633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940a0d50 con 0x55ede34101e0 2024-04-16T05:41:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f828bfff640 1 -- 172.21.15.86:0/2947500633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82940a5610 con 0x55ede34101e0 2024-04-16T05:41:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f828bfff640 1 -- 172.21.15.86:0/2947500633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940ce3d0 con 0x55ede34101e0 2024-04-16T05:41:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82ab98c740 1 -- 172.21.15.86:0/2947500633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34101e0 msgr2=0x55ede342ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82ab98c740 1 --2- 172.21.15.86:0/2947500633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34101e0 0x55ede342ac60 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8294098e00 tx=0x7f8294020000 comp rx=0 tx=0).stop 2024-04-16T05:41:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82ab98c740 1 -- 172.21.15.86:0/2947500633 shutdown_connections 2024-04-16T05:41:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82ab98c740 1 --2- 172.21.15.86:0/2947500633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34163d0 0x55ede34376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82ab98c740 1 --2- 172.21.15.86:0/2947500633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede3437170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82ab98c740 1 --2- 172.21.15.86:0/2947500633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34101e0 0x55ede342ac60 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.526+0000 7f82ab98c740 1 -- 172.21.15.86:0/2947500633 >> 172.21.15.86:0/2947500633 conn(0x55ede341ac60 msgr2=0x55ede34114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 -- 172.21.15.86:0/2947500633 shutdown_connections 2024-04-16T05:41:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 -- 172.21.15.86:0/2947500633 wait complete. 2024-04-16T05:41:56.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 -- start start 2024-04-16T05:41:56.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 Processor -- start 2024-04-16T05:41:56.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34101e0 0x55ede34276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede3423a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34163d0 0x55ede3424170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41af680 con 0x55ede34163d0 2024-04-16T05:41:56.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41aefb0 con 0x55ede3415ab0 2024-04-16T05:41:56.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41ac1d0 con 0x55ede34101e0 2024-04-16T05:41:56.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82ab98c740 10 client.0 _ll_register_callbacks cb 0x7f829c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:56.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82a2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34163d0 0x55ede3424170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82a2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34163d0 0x55ede3424170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42042/0 (socket says 172.21.15.86:42042) 2024-04-16T05:41:56.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82a2d9d640 1 -- 172.21.15.86:0/860637408 learned_addr learned my addr 172.21.15.86:0/860637408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f829bfff640 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede3423a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82a259c640 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34101e0 0x55ede34276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82a2d9d640 1 -- 172.21.15.86:0/860637408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34101e0 msgr2=0x55ede34276b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82a2d9d640 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34101e0 0x55ede34276b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82a2d9d640 1 -- 172.21.15.86:0/860637408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 msgr2=0x55ede3423a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82a2d9d640 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede3423a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f82a2d9d640 1 -- 172.21.15.86:0/860637408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede3499e00 con 0x55ede34163d0 2024-04-16T05:41:56.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.527+0000 7f829bfff640 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede3423a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:56.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82a2d9d640 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34163d0 0x55ede3424170 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f829c042040 tx=0x7f829c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cf4d07ac59c28d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f8289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:56.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f8289ffb640 1 -- 172.21.15.86:0/860637408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c063020 con 0x55ede34163d0 2024-04-16T05:41:56.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82ab98c740 1 -- 172.21.15.86:0/860637408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82940a5610 con 0x55ede34163d0 2024-04-16T05:41:56.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f8289ffb640 1 -- 172.21.15.86:0/860637408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f829c034640 con 0x55ede34163d0 2024-04-16T05:41:56.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f8289ffb640 1 -- 172.21.15.86:0/860637408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f829c06e050 con 0x55ede34163d0 2024-04-16T05:41:56.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82ab98c740 10 client.17688 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:56.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82ab98c740 1 -- 172.21.15.86:0/860637408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ede349afe0 con 0x55ede34163d0 2024-04-16T05:41:56.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f8289ffb640 1 -- 172.21.15.86:0/860637408 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82940a5610 con 0x55ede34163d0 2024-04-16T05:41:56.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f8289ffb640 1 -- 172.21.15.86:0/860637408 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f829c04c040 con 0x55ede34163d0 2024-04-16T05:41:56.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82ab98c740 10 client.17688 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:56.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82ab98c740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:56.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82ab98c740 20 client.17688 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:56.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82ab98c740 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ede341efb0 0x55ede349d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82ab98c740 4 client.17688 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:41:56.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.528+0000 7f82ab98c740 1 -- 172.21.15.86:0/860637408 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ede3382720 con 0x55ede341efb0 2024-04-16T05:41:56.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.529+0000 7f82a259c640 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ede341efb0 0x55ede349d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.529+0000 7f8289ffb640 10 client.17688 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:56.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.529+0000 7f82a259c640 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ede341efb0 0x55ede349d730 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe76237802729b2 server_cookie=b44d229397003f45 in_seq=0 out_seq=0 2024-04-16T05:41:56.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f8289ffb640 1 -- 172.21.15.86:0/860637408 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ede3382720 con 0x55ede341efb0 2024-04-16T05:41:56.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f8289ffb640 10 client.17688 handle_command_reply: tid=0 2024-04-16T05:41:56.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 client.17688 shutdown 2024-04-16T05:41:56.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 -- 172.21.15.86:0/860637408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34163d0 msgr2=0x55ede3424170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34163d0 0x55ede3424170 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f829c042040 tx=0x7f829c009b20 comp rx=0 tx=0).stop 2024-04-16T05:41:56.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 -- 172.21.15.86:0/860637408 shutdown_connections 2024-04-16T05:41:56.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ede341efb0 0x55ede349d730 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ede34163d0 0x55ede3424170 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ede3415ab0 0x55ede3423a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 --2- 172.21.15.86:0/860637408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ede34101e0 0x55ede34276b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f8289ffb640 0 client.17688 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:56.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 -- 172.21.15.86:0/860637408 >> 172.21.15.86:0/860637408 conn(0x55ede341ac60 msgr2=0x55ede340f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 -- 172.21.15.86:0/860637408 shutdown_connections 2024-04-16T05:41:56.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.530+0000 7f82ab98c740 1 -- 172.21.15.86:0/860637408 wait complete. 2024-04-16T05:41:56.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82ab98c740 20 client.17688 trim_cache size 0 max 16384 2024-04-16T05:41:56.541 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:41:56.541 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:41:56.541 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "5c32a05e-36fa-4a60-ba2b-02a6aad127fc", 2024-04-16T05:41:56.541 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:41:56.541 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:41:56.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 -- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82900a3d20 msgr2=0x7f82900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82900a3d20 0x7f82900a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f829c02c630 tx=0x7f829c005d80 comp rx=0 tx=0).stop 2024-04-16T05:41:56.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 -- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4101140 msgr2=0x7f82a4197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4101140 0x7f82a4197670 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f829400c920 tx=0x7f829400cde0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 -- 172.21.15.86:0/2713482802 shutdown_connections 2024-04-16T05:41:56.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a419bf30 0x7f82a419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f82900a3d20 0x7f82900a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82a4106210 0x7f82a4197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 --2- 172.21.15.86:0/2713482802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a4101140 0x7f82a4197670 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 -- 172.21.15.86:0/2713482802 >> 172.21.15.86:0/2713482802 conn(0x7f82a4078000 msgr2=0x7f82a40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 -- 172.21.15.86:0/2713482802 shutdown_connections 2024-04-16T05:41:56.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.531+0000 7f82a8da8640 1 -- 172.21.15.86:0/2713482802 wait complete. 2024-04-16T05:41:56.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:41:56.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:41:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 -- 172.21.15.86:0/4107343560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101af0 msgr2=0x7f62c8105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/4107343560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101af0 0x7f62c8105b30 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009900 tx=0x7f62bc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 -- 172.21.15.86:0/4107343560 shutdown_connections 2024-04-16T05:41:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/4107343560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c8106210 0x7f62c810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/4107343560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101af0 0x7f62c8105b30 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/4107343560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c8101140 0x7f62c8101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 -- 172.21.15.86:0/4107343560 >> 172.21.15.86:0/4107343560 conn(0x7f62c8078000 msgr2=0x7f62c8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 -- 172.21.15.86:0/4107343560 shutdown_connections 2024-04-16T05:41:56.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 -- 172.21.15.86:0/4107343560 wait complete. 2024-04-16T05:41:56.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 Processor -- start 2024-04-16T05:41:56.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.652+0000 7f62cd6d9640 1 -- start start 2024-04-16T05:41:56.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62cd6d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101140 0x7f62c8199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62cd6d9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c8106210 0x7f62c8199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62cd6d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c819a2c0 0x7f62c81a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62cd6d9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8111320 con 0x7f62c8101140 2024-04-16T05:41:56.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62cd6d9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81111a0 con 0x7f62c8106210 2024-04-16T05:41:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62cd6d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81114a0 con 0x7f62c819a2c0 2024-04-16T05:41:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101140 0x7f62c8199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101140 0x7f62c8199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42064/0 (socket says 172.21.15.86:42064) 2024-04-16T05:41:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c6ffd640 1 -- 172.21.15.86:0/3619401066 learned_addr learned my addr 172.21.15.86:0/3619401066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c77fe640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c819a2c0 0x7f62c81a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c67fc640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c8106210 0x7f62c8199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c6ffd640 1 -- 172.21.15.86:0/3619401066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c819a2c0 msgr2=0x7f62c81a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c819a2c0 0x7f62c81a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c6ffd640 1 -- 172.21.15.86:0/3619401066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c8106210 msgr2=0x7f62c8199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c8106210 0x7f62c8199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c6ffd640 1 -- 172.21.15.86:0/3619401066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c81a5040 con 0x7f62c8101140 2024-04-16T05:41:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c67fc640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c8106210 0x7f62c8199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101140 0x7f62c8199840 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f62b800d770 tx=0x7f62b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62affff640 1 -- 172.21.15.86:0/3619401066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8017070 con 0x7f62c8101140 2024-04-16T05:41:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62affff640 1 -- 172.21.15.86:0/3619401066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b8005500 con 0x7f62c8101140 2024-04-16T05:41:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62cd6d9640 1 -- 172.21.15.86:0/3619401066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c81a5330 con 0x7f62c8101140 2024-04-16T05:41:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62affff640 1 -- 172.21.15.86:0/3619401066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8004e00 con 0x7f62c8101140 2024-04-16T05:41:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.653+0000 7f62cd6d9640 1 -- 172.21.15.86:0/3619401066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c81a5830 con 0x7f62c8101140 2024-04-16T05:41:56.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.656+0000 7f62affff640 1 -- 172.21.15.86:0/3619401066 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62b8004fa0 con 0x7f62c8101140 2024-04-16T05:41:56.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.656+0000 7f62d03a3740 1 Processor -- start 2024-04-16T05:41:56.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.656+0000 7f62d03a3740 1 -- start start 2024-04-16T05:41:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d598230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d5a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5a3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62d03a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56456d4fd180 con 0x56456d597e50 2024-04-16T05:41:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62d03a3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56456d49a360 con 0x56456d598770 2024-04-16T05:41:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62d03a3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56456d4f9140 con 0x56456d5a1350 2024-04-16T05:41:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d598230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d598230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42080/0 (socket says 172.21.15.86:42080) 2024-04-16T05:41:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 -- 172.21.15.86:0/3385354760 learned_addr learned my addr 172.21.15.86:0/3385354760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/3385354760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d5a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c67fc640 1 --2- 172.21.15.86:0/3385354760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62affff640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62b00a3d20 0x7f62b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 -- 172.21.15.86:0/3385354760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 msgr2=0x56456d5a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 --2- 172.21.15.86:0/3385354760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 -- 172.21.15.86:0/3385354760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 msgr2=0x56456d5a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 --2- 172.21.15.86:0/3385354760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d5a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 -- 172.21.15.86:0/3385354760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456d2d97d0 con 0x56456d597e50 2024-04-16T05:41:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62affff640 1 -- 172.21.15.86:0/3619401066 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f62b80d0670 con 0x7f62c8101140 2024-04-16T05:41:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62b00a3d20 0x7f62b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62c77fe640 1 --2- 172.21.15.86:0/3385354760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d598230 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c4a0 tx=0x7f62c000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843c600d9e9cff96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.657+0000 7f62ad7fa640 1 -- 172.21.15.86:0/3385354760 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0018640 con 0x56456d597e50 2024-04-16T05:41:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62ad7fa640 1 -- 172.21.15.86:0/3385354760 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c001e070 con 0x56456d597e50 2024-04-16T05:41:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62ad7fa640 1 -- 172.21.15.86:0/3385354760 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c000a910 con 0x56456d597e50 2024-04-16T05:41:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62c77fe640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62b00a3d20 0x7f62b00a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62c001d000 tx=0x7f62c0017000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62d03a3740 1 -- 172.21.15.86:0/3385354760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 msgr2=0x56456d598230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62d03a3740 1 --2- 172.21.15.86:0/3385354760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d598230 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c4a0 tx=0x7f62c000c960 comp rx=0 tx=0).stop 2024-04-16T05:41:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62d03a3740 1 -- 172.21.15.86:0/3385354760 shutdown_connections 2024-04-16T05:41:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62d03a3740 1 --2- 172.21.15.86:0/3385354760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62d03a3740 1 --2- 172.21.15.86:0/3385354760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d5a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62d03a3740 1 --2- 172.21.15.86:0/3385354760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d598230 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62d03a3740 1 -- 172.21.15.86:0/3385354760 >> 172.21.15.86:0/3385354760 conn(0x56456d592580 msgr2=0x56456d592960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62d03a3740 1 -- 172.21.15.86:0/3385354760 shutdown_connections 2024-04-16T05:41:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.658+0000 7f62d03a3740 1 -- 172.21.15.86:0/3385354760 wait complete. 2024-04-16T05:41:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62d03a3740 1 -- start start 2024-04-16T05:41:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62d03a3740 1 Processor -- start 2024-04-16T05:41:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d59e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d59e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d59ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62d03a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56456d5a4140 con 0x56456d597e50 2024-04-16T05:41:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62d03a3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56456d4b62c0 con 0x56456d598770 2024-04-16T05:41:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62d03a3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56456d5a42c0 con 0x56456d5a1350 2024-04-16T05:41:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d59e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62d03a3740 10 client.0 _ll_register_callbacks cb 0x56456d49d090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d59e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42084/0 (socket says 172.21.15.86:42084) 2024-04-16T05:41:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c6ffd640 1 -- 172.21.15.86:0/4048118636 learned_addr learned my addr 172.21.15.86:0/4048118636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c67fc640 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d59e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c77fe640 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d59ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c6ffd640 1 -- 172.21.15.86:0/4048118636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 msgr2=0x56456d59ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d59ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c6ffd640 1 -- 172.21.15.86:0/4048118636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 msgr2=0x56456d59e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d59e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c6ffd640 1 -- 172.21.15.86:0/4048118636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456d20f3f0 con 0x56456d597e50 2024-04-16T05:41:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c67fc640 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d59e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d59e3f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f62b8098c70 tx=0x7f62b8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557c2ca8c0c0c223 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.659+0000 7f62aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62aaff5640 1 -- 172.21.15.86:0/4048118636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8020040 con 0x56456d597e50 2024-04-16T05:41:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62aaff5640 1 -- 172.21.15.86:0/4048118636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f62b809a5a0 con 0x56456d597e50 2024-04-16T05:41:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62d03a3740 1 -- 172.21.15.86:0/4048118636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c001e070 con 0x56456d597e50 2024-04-16T05:41:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62aaff5640 1 -- 172.21.15.86:0/4048118636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62b80056b0 con 0x56456d597e50 2024-04-16T05:41:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62d03a3740 10 client.17700 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62d03a3740 1 -- 172.21.15.86:0/4048118636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56456d1cf0f0 con 0x56456d597e50 2024-04-16T05:41:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62aaff5640 1 -- 172.21.15.86:0/4048118636 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c001e070 con 0x56456d597e50 2024-04-16T05:41:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62aaff5640 1 -- 172.21.15.86:0/4048118636 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f62b8098910 con 0x56456d597e50 2024-04-16T05:41:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62d03a3740 10 client.17700 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62d03a3740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62d03a3740 20 client.17700 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62d03a3740 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56456d5b17e0 0x56456d5b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62d03a3740 4 client.17700 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62d03a3740 1 -- 172.21.15.86:0/4048118636 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56456d50aba0 con 0x56456d5b17e0 2024-04-16T05:41:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.660+0000 7f62c67fc640 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56456d5b17e0 0x56456d5b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.661+0000 7f62aaff5640 10 client.17700 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.661+0000 7f62c67fc640 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56456d5b17e0 0x56456d5b3bd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c7d611564ed762f server_cookie=fb2930172d87c10f in_seq=0 out_seq=0 2024-04-16T05:41:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.661+0000 7f62aaff5640 1 -- 172.21.15.86:0/4048118636 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56456d50aba0 con 0x56456d5b17e0 2024-04-16T05:41:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.661+0000 7f62aaff5640 10 client.17700 handle_command_reply: tid=0 2024-04-16T05:41:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.661+0000 7f62d03a3740 1 client.17700 shutdown 2024-04-16T05:41:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.661+0000 7f62d03a3740 1 -- 172.21.15.86:0/4048118636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 msgr2=0x56456d59e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.661+0000 7f62d03a3740 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d59e3f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f62b8098c70 tx=0x7f62b8005bb0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62d03a3740 1 -- 172.21.15.86:0/4048118636 shutdown_connections 2024-04-16T05:41:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62d03a3740 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56456d5b17e0 0x56456d5b3bd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62d03a3740 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d59ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62aaff5640 0 client.17700 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62d03a3740 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d59e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62d03a3740 1 --2- 172.21.15.86:0/4048118636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d59e3f0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62d03a3740 1 -- 172.21.15.86:0/4048118636 >> 172.21.15.86:0/4048118636 conn(0x56456d592580 msgr2=0x56456d5942d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62d03a3740 1 -- 172.21.15.86:0/4048118636 shutdown_connections 2024-04-16T05:41:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62d03a3740 1 -- 172.21.15.86:0/4048118636 wait complete. 2024-04-16T05:41:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.662+0000 7f62d03a3740 20 client.17700 trim_cache size 0 max 16384 2024-04-16T05:41:56.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62d03a3740 1 Processor -- start 2024-04-16T05:41:56.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62d03a3740 1 -- start start 2024-04-16T05:41:56.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d597e50 0x56456d5ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d598770 0x56456d59ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5b9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62d03a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81af500 con 0x56456d598770 2024-04-16T05:41:56.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62d03a3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81b1040 con 0x56456d597e50 2024-04-16T05:41:56.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62d03a3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81afbd0 con 0x56456d5a1350 2024-04-16T05:41:56.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d598770 0x56456d59ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d598770 0x56456d59ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42090/0 (socket says 172.21.15.86:42090) 2024-04-16T05:41:56.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c67fc640 1 -- 172.21.15.86:0/2467057745 learned_addr learned my addr 172.21.15.86:0/2467057745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c67fc640 1 -- 172.21.15.86:0/2467057745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 msgr2=0x56456d5b9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c67fc640 1 --2- 172.21.15.86:0/2467057745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c67fc640 1 -- 172.21.15.86:0/2467057745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d597e50 msgr2=0x56456d5ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/2467057745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d597e50 0x56456d5ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c67fc640 1 --2- 172.21.15.86:0/2467057745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d597e50 0x56456d5ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c67fc640 1 -- 172.21.15.86:0/2467057745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456d5985a0 con 0x56456d598770 2024-04-16T05:41:56.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/2467057745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d597e50 0x56456d5ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:56.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.679+0000 7f62c67fc640 1 --2- 172.21.15.86:0/2467057745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d598770 0x56456d59ec90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f62c806b790 tx=0x7f62bc0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36f3e8a4b2bb8c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62acff9640 1 -- 172.21.15.86:0/2467057745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc045070 con 0x56456d598770 2024-04-16T05:41:56.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62acff9640 1 -- 172.21.15.86:0/2467057745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62bc005040 con 0x56456d598770 2024-04-16T05:41:56.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62acff9640 1 -- 172.21.15.86:0/2467057745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc047690 con 0x56456d598770 2024-04-16T05:41:56.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 -- 172.21.15.86:0/2467057745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d598770 msgr2=0x56456d59ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2467057745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d598770 0x56456d59ec90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f62c806b790 tx=0x7f62bc0095f0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 -- 172.21.15.86:0/2467057745 shutdown_connections 2024-04-16T05:41:56.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2467057745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5b9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2467057745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d598770 0x56456d59ec90 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2467057745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d597e50 0x56456d5ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 -- 172.21.15.86:0/2467057745 >> 172.21.15.86:0/2467057745 conn(0x56456d592580 msgr2=0x56456d591760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 -- 172.21.15.86:0/2467057745 shutdown_connections 2024-04-16T05:41:56.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 -- 172.21.15.86:0/2467057745 wait complete. 2024-04-16T05:41:56.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.680+0000 7f62d03a3740 1 -- start start 2024-04-16T05:41:56.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62d03a3740 1 Processor -- start 2024-04-16T05:41:56.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d5a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d5a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62d03a3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62d03a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81b0970 con 0x56456d597e50 2024-04-16T05:41:56.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62d03a3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81b02a0 con 0x56456d598770 2024-04-16T05:41:56.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62d03a3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81ad4c0 con 0x56456d5a1350 2024-04-16T05:41:56.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62d03a3740 10 client.0 _ll_register_callbacks cb 0x7f62b800de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:56.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62c6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d5a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62c6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d5a9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42098/0 (socket says 172.21.15.86:42098) 2024-04-16T05:41:56.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62c6ffd640 1 -- 172.21.15.86:0/2565796220 learned_addr learned my addr 172.21.15.86:0/2565796220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:41:56.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62c6ffd640 1 -- 172.21.15.86:0/2565796220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 msgr2=0x56456d5a74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62c6ffd640 1 -- 172.21.15.86:0/2565796220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 msgr2=0x56456d5a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:56.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d5a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62c6ffd640 1 -- 172.21.15.86:0/2565796220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456d616cf0 con 0x56456d597e50 2024-04-16T05:41:56.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62c6ffd640 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d5a9a40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009610 tx=0x7f62b8009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c64149e6b92013 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:41:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62aaff5640 1 -- 172.21.15.86:0/2565796220 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8020050 con 0x56456d597e50 2024-04-16T05:41:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62aaff5640 1 -- 172.21.15.86:0/2565796220 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f62b80d02a0 con 0x56456d597e50 2024-04-16T05:41:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62d03a3740 1 -- 172.21.15.86:0/2565796220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62bc005040 con 0x56456d597e50 2024-04-16T05:41:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.681+0000 7f62aaff5640 1 -- 172.21.15.86:0/2565796220 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62b809c070 con 0x56456d597e50 2024-04-16T05:41:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62d03a3740 10 client.17706 fetch_fsmap learned FSMap version 29 2024-04-16T05:41:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62d03a3740 1 -- 172.21.15.86:0/2565796220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56456d617e20 con 0x56456d597e50 2024-04-16T05:41:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62aaff5640 1 -- 172.21.15.86:0/2565796220 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62bc005040 con 0x56456d597e50 2024-04-16T05:41:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62aaff5640 1 -- 172.21.15.86:0/2565796220 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f62b8064080 con 0x56456d597e50 2024-04-16T05:41:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62d03a3740 10 client.17706 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:41:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62d03a3740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62d03a3740 20 client.17706 populate_metadata read hostname 'smithi086' 2024-04-16T05:41:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56456d618410 0x56456d61a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62d03a3740 4 client.17706 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:41:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62d03a3740 1 -- 172.21.15.86:0/2565796220 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56456d504a10 con 0x56456d618410 2024-04-16T05:41:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62c67fc640 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56456d618410 0x56456d61a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62aaff5640 10 client.17706 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.682+0000 7f62c67fc640 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56456d618410 0x56456d61a8b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c62f095f5e8ffb server_cookie=d4bf90b1866e1286 in_seq=0 out_seq=0 2024-04-16T05:41:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62aaff5640 1 -- 172.21.15.86:0/2565796220 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56456d504a10 con 0x56456d618410 2024-04-16T05:41:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62aaff5640 10 client.17706 handle_command_reply: tid=0 2024-04-16T05:41:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 client.17706 shutdown 2024-04-16T05:41:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 -- 172.21.15.86:0/2565796220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 msgr2=0x56456d5a9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d5a9a40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009610 tx=0x7f62b8009d90 comp rx=0 tx=0).stop 2024-04-16T05:41:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 -- 172.21.15.86:0/2565796220 shutdown_connections 2024-04-16T05:41:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x56456d618410 0x56456d61a8b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56456d5a1350 0x56456d5a74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56456d598770 0x56456d5a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62aaff5640 0 client.17706 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:41:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 --2- 172.21.15.86:0/2565796220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456d597e50 0x56456d5a9a40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 -- 172.21.15.86:0/2565796220 >> 172.21.15.86:0/2565796220 conn(0x56456d592580 msgr2=0x56456d593880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 -- 172.21.15.86:0/2565796220 shutdown_connections 2024-04-16T05:41:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 1 -- 172.21.15.86:0/2565796220 wait complete. 2024-04-16T05:41:56.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62d03a3740 20 client.17706 trim_cache size 0 max 16384 2024-04-16T05:41:56.694 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:41:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:41:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:41:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "5c32a05e-36fa-4a60-ba2b-02a6aad127fc": { 2024-04-16T05:41:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:41:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "5c32a05e-36fa-4a60-ba2b-02a6aad127fc", 2024-04-16T05:41:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:41:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:41:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:41:56.695 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:41:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 -- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62b00a3d20 msgr2=0x7f62b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62b00a3d20 0x7f62b00a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62c001d000 tx=0x7f62c0017000 comp rx=0 tx=0).stop 2024-04-16T05:41:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 -- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101140 msgr2=0x7f62c8199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101140 0x7f62c8199840 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f62b800d770 tx=0x7f62b800dc30 comp rx=0 tx=0).stop 2024-04-16T05:41:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 -- 172.21.15.86:0/3619401066 shutdown_connections 2024-04-16T05:41:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c819a2c0 0x7f62c81a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f62b00a3d20 0x7f62b00a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c8106210 0x7f62c8199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 --2- 172.21.15.86:0/3619401066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8101140 0x7f62c8199840 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.683+0000 7f62cd6d9640 1 -- 172.21.15.86:0/3619401066 >> 172.21.15.86:0/3619401066 conn(0x7f62c8078000 msgr2=0x7f62c80ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.684+0000 7f62cd6d9640 1 -- 172.21.15.86:0/3619401066 shutdown_connections 2024-04-16T05:41:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:41:56.684+0000 7f62cd6d9640 1 -- 172.21.15.86:0/3619401066 wait complete. 2024-04-16T05:41:56.698 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:56 smithi086 ceph-mon[32028]: pgmap v202: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:56.698 INFO:tasks.cephfs.filesystem:scrub status for tag:5c32a05e-36fa-4a60-ba2b-02a6aad127fc - {'path': '/', 'tag': '5c32a05e-36fa-4a60-ba2b-02a6aad127fc', 'options': 'recursive,force'} 2024-04-16T05:41:56.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:56 smithi196 ceph-mon[36373]: pgmap v202: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:56.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:56 smithi194 ceph-mon[35865]: pgmap v202: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:57 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:41:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:57 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:57 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:41:57.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:57 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:57.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:57 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:41:57.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:57 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:57.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:57 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:41:57.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:57 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:57.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:57 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:41:57.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:57 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:57 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:41:57.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:57 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:58.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:41:58 smithi086 ceph-mon[32028]: pgmap v203: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:58.756 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:41:58 smithi196 ceph-mon[36373]: pgmap v203: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:41:58.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:41:58 smithi194 ceph-mon[35865]: pgmap v203: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:00.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:00 smithi196 ceph-mon[36373]: pgmap v204: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:00.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:00 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:42:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:00 smithi194 ceph-mon[35865]: pgmap v204: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:00.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:00 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:42:00.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:00 smithi086 ceph-mon[32028]: pgmap v204: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:00.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:00 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:42:02.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:02 smithi196 ceph-mon[36373]: pgmap v205: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:02.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:02 smithi194 ceph-mon[35865]: pgmap v205: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:02.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:02 smithi086 ceph-mon[32028]: pgmap v205: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:04.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:04 smithi196 ceph-mon[36373]: pgmap v206: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:42:04.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:04 smithi194 ceph-mon[35865]: pgmap v206: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:42:04.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:04 smithi086 ceph-mon[32028]: pgmap v206: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:42:06.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:06 smithi196 ceph-mon[36373]: pgmap v207: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:42:06.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:06 smithi194 ceph-mon[35865]: pgmap v207: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:42:06.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:06 smithi086 ceph-mon[32028]: pgmap v207: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:42:08.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:08 smithi196 ceph-mon[36373]: pgmap v208: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:42:08.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:08 smithi194 ceph-mon[35865]: pgmap v208: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:42:08.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:08 smithi086 ceph-mon[32028]: pgmap v208: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:42:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:10 smithi196 ceph-mon[36373]: pgmap v209: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:42:10.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:10 smithi194 ceph-mon[35865]: pgmap v209: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:42:10.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:10 smithi086 ceph-mon[32028]: pgmap v209: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:42:11.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:11 smithi194 ceph-mon[35865]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T05:42:11.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:11 smithi086 ceph-mon[32028]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T05:42:11.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:11 smithi196 ceph-mon[36373]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T05:42:12.869 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:12 smithi086 ceph-mon[32028]: pgmap v211: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-16T05:42:12.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:12 smithi194 ceph-mon[35865]: pgmap v211: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-16T05:42:12.984 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:12 smithi196 ceph-mon[36373]: pgmap v211: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-16T05:42:14.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:14 smithi194 ceph-mon[35865]: pgmap v212: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:42:14.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:14 smithi086 ceph-mon[32028]: pgmap v212: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:42:14.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:14 smithi196 ceph-mon[36373]: pgmap v212: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:42:16.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:15 smithi086 ceph-mon[32028]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T05:42:16.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:15 smithi086 ceph-mon[32028]: pgmap v214: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:42:16.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:15 smithi196 ceph-mon[36373]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T05:42:16.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:15 smithi196 ceph-mon[36373]: pgmap v214: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:42:16.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:15 smithi194 ceph-mon[35865]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T05:42:16.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:15 smithi194 ceph-mon[35865]: pgmap v214: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:42:18.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:18 smithi194 ceph-mon[35865]: pgmap v215: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:42:18.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:18 smithi086 ceph-mon[32028]: pgmap v215: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:42:18.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:18 smithi196 ceph-mon[36373]: pgmap v215: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:42:20.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:20 smithi194 ceph-mon[35865]: pgmap v216: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 238 B/s wr, 12 op/s 2024-04-16T05:42:20.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:20 smithi086 ceph-mon[32028]: pgmap v216: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 238 B/s wr, 12 op/s 2024-04-16T05:42:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:20 smithi196 ceph-mon[36373]: pgmap v216: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 238 B/s wr, 12 op/s 2024-04-16T05:42:22.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:22 smithi194 ceph-mon[35865]: pgmap v217: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:42:22.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:22 smithi086 ceph-mon[32028]: pgmap v217: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:42:22.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:22 smithi196 ceph-mon[36373]: pgmap v217: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:42:24.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:24 smithi194 ceph-mon[35865]: pgmap v218: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:42:24.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:24 smithi086 ceph-mon[32028]: pgmap v218: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:42:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:24 smithi196 ceph-mon[36373]: pgmap v218: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:42:26.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:26 smithi194 ceph-mon[35865]: pgmap v219: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:42:26.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:26 smithi086 ceph-mon[32028]: pgmap v219: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:42:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:26 smithi196 ceph-mon[36373]: pgmap v219: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:42:26.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:42:26.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:42:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.820+0000 7ff8fe1db640 1 -- 172.21.15.86:0/4155637046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 msgr2=0x7ff8f810ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.820+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/4155637046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 0x7ff8f810ee90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009cd0 tx=0x7ff8f002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.820+0000 7ff8fe1db640 1 -- 172.21.15.86:0/4155637046 shutdown_connections 2024-04-16T05:42:26.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.820+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/4155637046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 0x7ff8f810ee90 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.820+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/4155637046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f8100500 0x7ff8f8100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.820+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/4155637046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8f8106fe0 0x7ff8f81073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.820+0000 7ff8fe1db640 1 -- 172.21.15.86:0/4155637046 >> 172.21.15.86:0/4155637046 conn(0x7ff8f80774c0 msgr2=0x7ff8f80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:26.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.820+0000 7ff8fe1db640 1 -- 172.21.15.86:0/4155637046 shutdown_connections 2024-04-16T05:42:26.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8fe1db640 1 -- 172.21.15.86:0/4155637046 wait complete. 2024-04-16T05:42:26.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8fe1db640 1 Processor -- start 2024-04-16T05:42:26.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8fe1db640 1 -- start start 2024-04-16T05:42:26.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8fe1db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f8100500 0x7ff8f819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8fe1db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 0x7ff8f819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8fe1db640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8f8106fe0 0x7ff8f81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8fe1db640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81124f0 con 0x7ff8f8100ec0 2024-04-16T05:42:26.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8fe1db640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8112370 con 0x7ff8f8106fe0 2024-04-16T05:42:26.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8fe1db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8112670 con 0x7ff8f8100500 2024-04-16T05:42:26.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 0x7ff8f819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 0x7ff8f819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53972/0 (socket says 172.21.15.86:53972) 2024-04-16T05:42:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.821+0000 7ff8effff640 1 -- 172.21.15.86:0/3868971292 learned_addr learned my addr 172.21.15.86:0/3868971292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8f7fff640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8f8106fe0 0x7ff8f81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f8100500 0x7ff8f819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8effff640 1 -- 172.21.15.86:0/3868971292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f8100500 msgr2=0x7ff8f819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8effff640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f8100500 0x7ff8f819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8effff640 1 -- 172.21.15.86:0/3868971292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8f8106fe0 msgr2=0x7ff8f81987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8effff640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8f8106fe0 0x7ff8f81987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8effff640 1 -- 172.21.15.86:0/3868971292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f8198f80 con 0x7ff8f8100ec0 2024-04-16T05:42:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f8100500 0x7ff8f819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:26.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8effff640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 0x7ff8f819eb80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000b4a0 tx=0x7ff8e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:26.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8f57fa640 1 -- 172.21.15.86:0/3868971292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0014070 con 0x7ff8f8100ec0 2024-04-16T05:42:26.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8fe1db640 1 -- 172.21.15.86:0/3868971292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f8199270 con 0x7ff8f8100ec0 2024-04-16T05:42:26.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8f57fa640 1 -- 172.21.15.86:0/3868971292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e0010af0 con 0x7ff8f8100ec0 2024-04-16T05:42:26.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8f57fa640 1 -- 172.21.15.86:0/3868971292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0017d10 con 0x7ff8f8100ec0 2024-04-16T05:42:26.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.822+0000 7ff8fe1db640 1 -- 172.21.15.86:0/3868971292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f81a5490 con 0x7ff8f8100ec0 2024-04-16T05:42:26.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.824+0000 7ff8f57fa640 1 -- 172.21.15.86:0/3868971292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff8e0017070 con 0x7ff8f8100ec0 2024-04-16T05:42:26.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff900dc7740 1 Processor -- start 2024-04-16T05:42:26.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff900dc7740 1 -- start start 2024-04-16T05:42:26.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f57fa640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff8e40a3d20 0x7ff8e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd329f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd337220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 0x55c5bd339b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff900dc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5bd28f180 con 0x55c5bd329b80 2024-04-16T05:42:26.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff900dc7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c5bd22c360 con 0x55c5bd337760 2024-04-16T05:42:26.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff900dc7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c5bd28b140 con 0x55c5bd32a4a0 2024-04-16T05:42:26.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f7fff640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff8e40a3d20 0x7ff8e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd329f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd329f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53978/0 (socket says 172.21.15.86:53978) 2024-04-16T05:42:26.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f77fe640 1 -- 172.21.15.86:0/3120633081 learned_addr learned my addr 172.21.15.86:0/3120633081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:26.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f57fa640 1 -- 172.21.15.86:0/3868971292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7ff8e00cfeb0 con 0x7ff8f8100ec0 2024-04-16T05:42:26.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f77fe640 1 -- 172.21.15.86:0/3120633081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 msgr2=0x55c5bd337220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:26.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/3120633081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd337220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f77fe640 1 -- 172.21.15.86:0/3120633081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 msgr2=0x55c5bd339b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:26.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/3120633081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 0x55c5bd339b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.825+0000 7ff8f77fe640 1 -- 172.21.15.86:0/3120633081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5bd06b7d0 con 0x55c5bd329b80 2024-04-16T05:42:26.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.826+0000 7ff8f7fff640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff8e40a3d20 0x7ff8e40a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002c630 tx=0x7ff8f0005f10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:26.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.826+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/3120633081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd329f60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800bd40 tx=0x7ff8e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558e52ee0f952127 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:26.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.826+0000 7ff8edffb640 1 -- 172.21.15.86:0/3120633081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8015070 con 0x55c5bd329b80 2024-04-16T05:42:26.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.826+0000 7ff8edffb640 1 -- 172.21.15.86:0/3120633081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e800be30 con 0x55c5bd329b80 2024-04-16T05:42:26.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.826+0000 7ff8edffb640 1 -- 172.21.15.86:0/3120633081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80119a0 con 0x55c5bd329b80 2024-04-16T05:42:26.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.826+0000 7ff900dc7740 1 -- 172.21.15.86:0/3120633081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 msgr2=0x55c5bd329f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.826+0000 7ff900dc7740 1 --2- 172.21.15.86:0/3120633081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd329f60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800bd40 tx=0x7ff8e800f730 comp rx=0 tx=0).stop 2024-04-16T05:42:26.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 -- 172.21.15.86:0/3120633081 shutdown_connections 2024-04-16T05:42:26.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 --2- 172.21.15.86:0/3120633081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 0x55c5bd339b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 --2- 172.21.15.86:0/3120633081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd337220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 --2- 172.21.15.86:0/3120633081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd329f60 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 -- 172.21.15.86:0/3120633081 >> 172.21.15.86:0/3120633081 conn(0x55c5bd324580 msgr2=0x55c5bd324960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 -- 172.21.15.86:0/3120633081 shutdown_connections 2024-04-16T05:42:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 -- 172.21.15.86:0/3120633081 wait complete. 2024-04-16T05:42:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 -- start start 2024-04-16T05:42:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 Processor -- start 2024-04-16T05:42:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd329b80 0x55c5bd3348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd334df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd337760 0x55c5bd335330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5bd33a520 con 0x55c5bd337760 2024-04-16T05:42:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c5bd2482c0 con 0x55c5bd329b80 2024-04-16T05:42:26.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c5bd33a6a0 con 0x55c5bd32a4a0 2024-04-16T05:42:26.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.827+0000 7ff900dc7740 10 client.0 _ll_register_callbacks cb 0x55c5bd22f090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:26.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd337760 0x55c5bd335330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd337760 0x55c5bd335330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53994/0 (socket says 172.21.15.86:53994) 2024-04-16T05:42:26.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f7fff640 1 -- 172.21.15.86:0/2764679881 learned_addr learned my addr 172.21.15.86:0/2764679881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:26.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8effff640 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd334df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd329b80 0x55c5bd3348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f7fff640 1 -- 172.21.15.86:0/2764679881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 msgr2=0x55c5bd334df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:26.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f7fff640 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd334df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f7fff640 1 -- 172.21.15.86:0/2764679881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd329b80 msgr2=0x55c5bd3348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:26.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f7fff640 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd329b80 0x55c5bd3348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f7fff640 1 -- 172.21.15.86:0/2764679881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5bcfa13f0 con 0x55c5bd337760 2024-04-16T05:42:26.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd329b80 0x55c5bd3348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:26.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8f7fff640 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd337760 0x55c5bd335330 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0005e30 tx=0x7ff8f002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a6cc98dcec36bf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:26.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:26.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/2764679881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0051060 con 0x55c5bd337760 2024-04-16T05:42:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/2764679881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7ff8f0003c30 con 0x55c5bd337760 2024-04-16T05:42:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff900dc7740 1 -- 172.21.15.86:0/2764679881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e800be30 con 0x55c5bd337760 2024-04-16T05:42:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/2764679881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff8f0058050 con 0x55c5bd337760 2024-04-16T05:42:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff900dc7740 10 client.17718 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff900dc7740 1 -- 172.21.15.86:0/2764679881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5bcf610f0 con 0x55c5bd337760 2024-04-16T05:42:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.828+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/2764679881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e800be30 con 0x55c5bd337760 2024-04-16T05:42:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/2764679881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff8f003f040 con 0x55c5bd337760 2024-04-16T05:42:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff900dc7740 10 client.17718 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff900dc7740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff900dc7740 20 client.17718 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c5bd347c20 0x55c5bd34a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff900dc7740 4 client.17718 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff900dc7740 1 -- 172.21.15.86:0/2764679881 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5bd29cba0 con 0x55c5bd347c20 2024-04-16T05:42:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c5bd347c20 0x55c5bd34a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff8d6ffd640 10 client.17718 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.829+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c5bd347c20 0x55c5bd34a050 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef9310a002087621 server_cookie=ced2be61205a0fad in_seq=0 out_seq=0 2024-04-16T05:42:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/2764679881 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5bd29cba0 con 0x55c5bd347c20 2024-04-16T05:42:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff8d6ffd640 10 client.17718 handle_command_reply: tid=0 2024-04-16T05:42:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 client.17718 shutdown 2024-04-16T05:42:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 -- 172.21.15.86:0/2764679881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd337760 msgr2=0x55c5bd335330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd337760 0x55c5bd335330 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0005e30 tx=0x7ff8f002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 -- 172.21.15.86:0/2764679881 shutdown_connections 2024-04-16T05:42:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c5bd347c20 0x55c5bd34a050 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd337760 0x55c5bd335330 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd334df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff8d6ffd640 0 client.17718 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2764679881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd329b80 0x55c5bd3348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 -- 172.21.15.86:0/2764679881 >> 172.21.15.86:0/2764679881 conn(0x55c5bd324580 msgr2=0x55c5bd3262d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 -- 172.21.15.86:0/2764679881 shutdown_connections 2024-04-16T05:42:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 1 -- 172.21.15.86:0/2764679881 wait complete. 2024-04-16T05:42:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.830+0000 7ff900dc7740 20 client.17718 trim_cache size 0 max 16384 2024-04-16T05:42:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff900dc7740 1 Processor -- start 2024-04-16T05:42:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff900dc7740 1 -- start start 2024-04-16T05:42:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd336fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd32a4a0 0x55c5bd343420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd337760 0x55c5bd3235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff900dc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81ad330 con 0x55c5bd329b80 2024-04-16T05:42:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff900dc7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81aee70 con 0x55c5bd32a4a0 2024-04-16T05:42:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff900dc7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81ada00 con 0x55c5bd337760 2024-04-16T05:42:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd336fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd336fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54006/0 (socket says 172.21.15.86:54006) 2024-04-16T05:42:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f77fe640 1 -- 172.21.15.86:0/2061356383 learned_addr learned my addr 172.21.15.86:0/2061356383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8effff640 1 --2- 172.21.15.86:0/2061356383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd32a4a0 0x55c5bd343420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f7fff640 1 --2- 172.21.15.86:0/2061356383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd337760 0x55c5bd3235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f77fe640 1 -- 172.21.15.86:0/2061356383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd337760 msgr2=0x55c5bd3235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/2061356383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd337760 0x55c5bd3235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f77fe640 1 -- 172.21.15.86:0/2061356383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd32a4a0 msgr2=0x55c5bd343420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/2061356383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd32a4a0 0x55c5bd343420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f77fe640 1 -- 172.21.15.86:0/2061356383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5bd31fbe0 con 0x55c5bd329b80 2024-04-16T05:42:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.848+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/2061356383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd336fe0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80114b0 tx=0x7ff8e8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b55acec7e41c206 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff8ed7fa640 1 -- 172.21.15.86:0/2061356383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e802d070 con 0x55c5bd329b80 2024-04-16T05:42:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff8ed7fa640 1 -- 172.21.15.86:0/2061356383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8028440 con 0x55c5bd329b80 2024-04-16T05:42:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff8ed7fa640 1 -- 172.21.15.86:0/2061356383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8015070 con 0x55c5bd329b80 2024-04-16T05:42:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff900dc7740 1 -- 172.21.15.86:0/2061356383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 msgr2=0x55c5bd336fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2061356383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd336fe0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80114b0 tx=0x7ff8e8011440 comp rx=0 tx=0).stop 2024-04-16T05:42:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff900dc7740 1 -- 172.21.15.86:0/2061356383 shutdown_connections 2024-04-16T05:42:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2061356383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd337760 0x55c5bd3235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2061356383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd32a4a0 0x55c5bd343420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff900dc7740 1 --2- 172.21.15.86:0/2061356383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd336fe0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff900dc7740 1 -- 172.21.15.86:0/2061356383 >> 172.21.15.86:0/2061356383 conn(0x55c5bd324580 msgr2=0x55c5bd322ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff900dc7740 1 -- 172.21.15.86:0/2061356383 shutdown_connections 2024-04-16T05:42:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.849+0000 7ff900dc7740 1 -- 172.21.15.86:0/2061356383 wait complete. 2024-04-16T05:42:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff900dc7740 1 -- start start 2024-04-16T05:42:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff900dc7740 1 Processor -- start 2024-04-16T05:42:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd33fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd33cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff900dc7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 0x55c5bd33d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff900dc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81ae7a0 con 0x55c5bd329b80 2024-04-16T05:42:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff900dc7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81ae0d0 con 0x55c5bd337760 2024-04-16T05:42:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff900dc7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81ab330 con 0x55c5bd32a4a0 2024-04-16T05:42:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff900dc7740 10 client.0 _ll_register_callbacks cb 0x7ff8f0045040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd33fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd33fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54022/0 (socket says 172.21.15.86:54022) 2024-04-16T05:42:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f77fe640 1 -- 172.21.15.86:0/1435995154 learned_addr learned my addr 172.21.15.86:0/1435995154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f7fff640 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 0x55c5bd33d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f77fe640 1 -- 172.21.15.86:0/1435995154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 msgr2=0x55c5bd33cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd33cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f77fe640 1 -- 172.21.15.86:0/1435995154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 msgr2=0x55c5bd33d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 0x55c5bd33d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f77fe640 1 -- 172.21.15.86:0/1435995154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5bd33dc60 con 0x55c5bd329b80 2024-04-16T05:42:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.850+0000 7ff8f7fff640 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 0x55c5bd33d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff8f77fe640 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd33fc50 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff8e802b040 tx=0x7ff8e800b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c043c782466af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff8d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/1435995154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8015050 con 0x55c5bd329b80 2024-04-16T05:42:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/1435995154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7ff8e800be30 con 0x55c5bd329b80 2024-04-16T05:42:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff900dc7740 1 -- 172.21.15.86:0/1435995154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e8028440 con 0x55c5bd329b80 2024-04-16T05:42:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/1435995154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff8e803e050 con 0x55c5bd329b80 2024-04-16T05:42:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff900dc7740 10 client.17724 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff900dc7740 1 -- 172.21.15.86:0/1435995154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5bd3ae0e0 con 0x55c5bd329b80 2024-04-16T05:42:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/1435995154 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8028440 con 0x55c5bd329b80 2024-04-16T05:42:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/1435995154 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff8e80286f0 con 0x55c5bd329b80 2024-04-16T05:42:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff900dc7740 10 client.17724 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff900dc7740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff900dc7740 20 client.17724 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff900dc7740 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c5bd3ae650 0x55c5bd3b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff900dc7740 4 client.17724 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:42:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.851+0000 7ff900dc7740 1 -- 172.21.15.86:0/1435995154 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5bd296a10 con 0x55c5bd3ae650 2024-04-16T05:42:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff8effff640 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c5bd3ae650 0x55c5bd3b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff8d6ffd640 10 client.17724 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff8effff640 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c5bd3ae650 0x55c5bd3b0a40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef67e6855f1ce2f server_cookie=5fc0c3d6d1f109f7 in_seq=0 out_seq=0 2024-04-16T05:42:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff8d6ffd640 1 -- 172.21.15.86:0/1435995154 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c5bd296a10 con 0x55c5bd3ae650 2024-04-16T05:42:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff8d6ffd640 10 client.17724 handle_command_reply: tid=0 2024-04-16T05:42:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 client.17724 shutdown 2024-04-16T05:42:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 -- 172.21.15.86:0/1435995154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 msgr2=0x55c5bd33fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd33fc50 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff8e802b040 tx=0x7ff8e800b710 comp rx=0 tx=0).stop 2024-04-16T05:42:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 -- 172.21.15.86:0/1435995154 shutdown_connections 2024-04-16T05:42:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55c5bd3ae650 0x55c5bd3b0a40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5bd337760 0x55c5bd33d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff8d6ffd640 0 client.17724 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5bd32a4a0 0x55c5bd33cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 --2- 172.21.15.86:0/1435995154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bd329b80 0x55c5bd33fc50 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 -- 172.21.15.86:0/1435995154 >> 172.21.15.86:0/1435995154 conn(0x55c5bd324580 msgr2=0x55c5bd335100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 -- 172.21.15.86:0/1435995154 shutdown_connections 2024-04-16T05:42:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 1 -- 172.21.15.86:0/1435995154 wait complete. 2024-04-16T05:42:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.852+0000 7ff900dc7740 20 client.17724 trim_cache size 0 max 16384 2024-04-16T05:42:26.862 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:42:26.862 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:42:26.862 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:42:26.862 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:42:26.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 -- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff8e40a3d20 msgr2=0x7ff8e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff8e40a3d20 0x7ff8e40a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002c630 tx=0x7ff8f0005f10 comp rx=0 tx=0).stop 2024-04-16T05:42:26.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 -- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 msgr2=0x7ff8f819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 0x7ff8f819eb80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000b4a0 tx=0x7ff8e000b960 comp rx=0 tx=0).stop 2024-04-16T05:42:26.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 -- 172.21.15.86:0/3868971292 shutdown_connections 2024-04-16T05:42:26.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff8e40a3d20 0x7ff8e40a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8f8106fe0 0x7ff8f81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8100ec0 0x7ff8f819eb80 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 --2- 172.21.15.86:0/3868971292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f8100500 0x7ff8f819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 -- 172.21.15.86:0/3868971292 >> 172.21.15.86:0/3868971292 conn(0x7ff8f80774c0 msgr2=0x7ff8f8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 -- 172.21.15.86:0/3868971292 shutdown_connections 2024-04-16T05:42:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.853+0000 7ff8fe1db640 1 -- 172.21.15.86:0/3868971292 wait complete. 2024-04-16T05:42:26.865 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:42:26.866 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:42:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 -- 172.21.15.86:0/4040224789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08101570 msgr2=0x7f1e08106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 --2- 172.21.15.86:0/4040224789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08101570 0x7f1e08106ca0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009900 tx=0x7f1df002bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 -- 172.21.15.86:0/4040224789 shutdown_connections 2024-04-16T05:42:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 --2- 172.21.15.86:0/4040224789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e08107380 0x7f1e0810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 --2- 172.21.15.86:0/4040224789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08101570 0x7f1e08106ca0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 --2- 172.21.15.86:0/4040224789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e08100bc0 0x7f1e08100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 -- 172.21.15.86:0/4040224789 >> 172.21.15.86:0/4040224789 conn(0x7f1e080761e0 msgr2=0x7f1e080765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:26.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 -- 172.21.15.86:0/4040224789 shutdown_connections 2024-04-16T05:42:26.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 -- 172.21.15.86:0/4040224789 wait complete. 2024-04-16T05:42:26.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 Processor -- start 2024-04-16T05:42:26.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.972+0000 7f1e0e329640 1 -- start start 2024-04-16T05:42:26.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e0e329640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e08100bc0 0x7f1e0819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e0e329640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08107380 0x7f1e0819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e0e329640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e08198730 0x7f1e08198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e0e329640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08112410 con 0x7f1e08107380 2024-04-16T05:42:26.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e0e329640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08112290 con 0x7f1e08198730 2024-04-16T05:42:26.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e0e329640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08112590 con 0x7f1e08100bc0 2024-04-16T05:42:26.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1dfffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08107380 0x7f1e0819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1dfffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08107380 0x7f1e0819ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54054/0 (socket says 172.21.15.86:54054) 2024-04-16T05:42:26.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1dfffff640 1 -- 172.21.15.86:0/2359279883 learned_addr learned my addr 172.21.15.86:0/2359279883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:26.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e0c8c6640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e08198730 0x7f1e08198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e07fff640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e08100bc0 0x7f1e0819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1dfffff640 1 -- 172.21.15.86:0/2359279883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e08100bc0 msgr2=0x7f1e0819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1dfffff640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e08100bc0 0x7f1e0819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1dfffff640 1 -- 172.21.15.86:0/2359279883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e08198730 msgr2=0x7f1e08198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1dfffff640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e08198730 0x7f1e08198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1dfffff640 1 -- 172.21.15.86:0/2359279883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e08199330 con 0x7f1e08107380 2024-04-16T05:42:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1dfffff640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08107380 0x7f1e0819ebb0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1df0036040 tx=0x7f1df002cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e05ffb640 1 -- 172.21.15.86:0/2359279883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0041070 con 0x7f1e08107380 2024-04-16T05:42:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e081995c0 con 0x7f1e08107380 2024-04-16T05:42:26.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e05ffb640 1 -- 172.21.15.86:0/2359279883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df00054e0 con 0x7f1e08107380 2024-04-16T05:42:26.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e05ffb640 1 -- 172.21.15.86:0/2359279883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0034440 con 0x7f1e08107380 2024-04-16T05:42:26.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.973+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e081a5700 con 0x7f1e08107380 2024-04-16T05:42:26.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.974+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e0806a190 con 0x7f1e08107380 2024-04-16T05:42:26.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.975+0000 7f1e05ffb640 1 -- 172.21.15.86:0/2359279883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1df0044070 con 0x7f1e08107380 2024-04-16T05:42:26.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.976+0000 7f1e05ffb640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1df40a3d20 0x7f1df40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:26.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.976+0000 7f1e07fff640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1df40a3d20 0x7f1df40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:26.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.976+0000 7f1e05ffb640 1 -- 172.21.15.86:0/2359279883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f1df00f2970 con 0x7f1e08107380 2024-04-16T05:42:26.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.977+0000 7f1e07fff640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1df40a3d20 0x7f1df40a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009b60 tx=0x7f1df8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:26.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:26.981+0000 7f1e05ffb640 1 -- 172.21.15.86:0/2359279883 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1df00bed50 con 0x7f1e08107380 2024-04-16T05:42:27.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.199+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e08069270 con 0x7f1e08107380 2024-04-16T05:42:27.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.200+0000 7f1e05ffb640 1 -- 172.21.15.86:0/2359279883 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f1df00be650 con 0x7f1e08107380 2024-04-16T05:42:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:42:27.199 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:42:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:42:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1df40a3d20 msgr2=0x7f1df40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1df40a3d20 0x7f1df40a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009b60 tx=0x7f1df8005f90 comp rx=0 tx=0).stop 2024-04-16T05:42:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08107380 msgr2=0x7f1e0819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08107380 0x7f1e0819ebb0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1df0036040 tx=0x7f1df002cbf0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 shutdown_connections 2024-04-16T05:42:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1df40a3d20 0x7f1df40a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e08198730 0x7f1e08198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e08107380 0x7f1e0819ebb0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 --2- 172.21.15.86:0/2359279883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e08100bc0 0x7f1e0819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 >> 172.21.15.86:0/2359279883 conn(0x7f1e080761e0 msgr2=0x7f1e080fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 shutdown_connections 2024-04-16T05:42:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.202+0000 7f1e0e329640 1 -- 172.21.15.86:0/2359279883 wait complete. 2024-04-16T05:42:27.212 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:42:27.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.319+0000 7fea48738640 1 -- 172.21.15.86:0/4261251261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40107060 msgr2=0x7fea40107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.319+0000 7fea48738640 1 --2- 172.21.15.86:0/4261251261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40107060 0x7fea40107440 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009ca0 tx=0x7fea3c02c180 comp rx=0 tx=0).stop 2024-04-16T05:42:27.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.319+0000 7fea48738640 1 -- 172.21.15.86:0/4261251261 shutdown_connections 2024-04-16T05:42:27.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.319+0000 7fea48738640 1 --2- 172.21.15.86:0/4261251261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea40100a70 0x7fea4010eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.319+0000 7fea48738640 1 --2- 172.21.15.86:0/4261251261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea401000d0 0x7fea40100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.319+0000 7fea48738640 1 --2- 172.21.15.86:0/4261251261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40107060 0x7fea40107440 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.319+0000 7fea48738640 1 -- 172.21.15.86:0/4261251261 >> 172.21.15.86:0/4261251261 conn(0x7fea40077050 msgr2=0x7fea400fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 -- 172.21.15.86:0/4261251261 shutdown_connections 2024-04-16T05:42:27.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 -- 172.21.15.86:0/4261251261 wait complete. 2024-04-16T05:42:27.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 Processor -- start 2024-04-16T05:42:27.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 -- start start 2024-04-16T05:42:27.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea401000d0 0x7fea401a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea40100a70 0x7fea401a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea40107060 0x7fea4019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea40112320 con 0x7fea401000d0 2024-04-16T05:42:27.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea401121a0 con 0x7fea40100a70 2024-04-16T05:42:27.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea48738640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea401124a0 con 0x7fea40107060 2024-04-16T05:42:27.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea464d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea401000d0 0x7fea401a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea464d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea401000d0 0x7fea401a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54072/0 (socket says 172.21.15.86:54072) 2024-04-16T05:42:27.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea464d4640 1 -- 172.21.15.86:0/2098207600 learned_addr learned my addr 172.21.15.86:0/2098207600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.320+0000 7fea45cd3640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea40100a70 0x7fea401a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea46cd5640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea40107060 0x7fea4019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea464d4640 1 -- 172.21.15.86:0/2098207600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea40107060 msgr2=0x7fea4019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea464d4640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea40107060 0x7fea4019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea464d4640 1 -- 172.21.15.86:0/2098207600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea40100a70 msgr2=0x7fea401a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea464d4640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea40100a70 0x7fea401a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea464d4640 1 -- 172.21.15.86:0/2098207600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4019afc0 con 0x7fea401000d0 2024-04-16T05:42:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea464d4640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea401000d0 0x7fea401a0680 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fea3c005e30 tx=0x7fea3c035ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea377fe640 1 -- 172.21.15.86:0/2098207600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c039040 con 0x7fea401000d0 2024-04-16T05:42:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea48738640 1 -- 172.21.15.86:0/2098207600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4019b250 con 0x7fea401000d0 2024-04-16T05:42:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea377fe640 1 -- 172.21.15.86:0/2098207600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c044070 con 0x7fea401000d0 2024-04-16T05:42:27.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea377fe640 1 -- 172.21.15.86:0/2098207600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0329f0 con 0x7fea401000d0 2024-04-16T05:42:27.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.321+0000 7fea48738640 1 -- 172.21.15.86:0/2098207600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea401a74d0 con 0x7fea401000d0 2024-04-16T05:42:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.323+0000 7fea377fe640 1 -- 172.21.15.86:0/2098207600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea3c032b90 con 0x7fea401000d0 2024-04-16T05:42:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.323+0000 7fea4b3f5740 1 Processor -- start 2024-04-16T05:42:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea4b3f5740 1 -- start start 2024-04-16T05:42:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8be60 0x5650d9f8c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f94e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f95360 0x5650d9f97750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5650d9ef10e0 con 0x5650d9f8c780 2024-04-16T05:42:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5650d9e8e520 con 0x5650d9f95360 2024-04-16T05:42:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5650d9eed0a0 con 0x5650d9f8be60 2024-04-16T05:42:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea464d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f94e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea464d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f94e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54074/0 (socket says 172.21.15.86:54074) 2024-04-16T05:42:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea464d4640 1 -- 172.21.15.86:0/1666776988 learned_addr learned my addr 172.21.15.86:0/1666776988 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea46cd5640 1 --2- 172.21.15.86:0/1666776988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8be60 0x5650d9f8c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea377fe640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea45cd3640 1 --2- 172.21.15.86:0/1666776988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f95360 0x5650d9f97750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea46cd5640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea377fe640 1 -- 172.21.15.86:0/2098207600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fea3c0f30d0 con 0x7fea401000d0 2024-04-16T05:42:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea464d4640 1 -- 172.21.15.86:0/1666776988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8be60 msgr2=0x5650d9f8c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea464d4640 1 --2- 172.21.15.86:0/1666776988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8be60 0x5650d9f8c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea464d4640 1 -- 172.21.15.86:0/1666776988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f95360 msgr2=0x5650d9f97750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea464d4640 1 --2- 172.21.15.86:0/1666776988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f95360 0x5650d9f97750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.324+0000 7fea464d4640 1 -- 172.21.15.86:0/1666776988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650d9eb68d0 con 0x5650d9f8c780 2024-04-16T05:42:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.325+0000 7fea464d4640 1 --2- 172.21.15.86:0/1666776988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f94e20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0bed40 tx=0x7fea3c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b97d6428cc3811 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.325+0000 7fea46cd5640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea38005850 tx=0x7fea38004730 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.325+0000 7fea34ff9640 1 -- 172.21.15.86:0/1666776988 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0be710 con 0x5650d9f8c780 2024-04-16T05:42:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.325+0000 7fea34ff9640 1 -- 172.21.15.86:0/1666776988 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c0bf4b0 con 0x5650d9f8c780 2024-04-16T05:42:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.325+0000 7fea34ff9640 1 -- 172.21.15.86:0/1666776988 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0f06d0 con 0x5650d9f8c780 2024-04-16T05:42:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.325+0000 7fea4b3f5740 1 -- 172.21.15.86:0/1666776988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 msgr2=0x5650d9f94e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.325+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/1666776988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f94e20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0bed40 tx=0x7fea3c003f50 comp rx=0 tx=0).stop 2024-04-16T05:42:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 -- 172.21.15.86:0/1666776988 shutdown_connections 2024-04-16T05:42:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/1666776988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f95360 0x5650d9f97750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/1666776988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f94e20 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/1666776988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8be60 0x5650d9f8c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 -- 172.21.15.86:0/1666776988 >> 172.21.15.86:0/1666776988 conn(0x5650d9f86590 msgr2=0x5650d9f86970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 -- 172.21.15.86:0/1666776988 shutdown_connections 2024-04-16T05:42:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 -- 172.21.15.86:0/1666776988 wait complete. 2024-04-16T05:42:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 -- start start 2024-04-16T05:42:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 Processor -- start 2024-04-16T05:42:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8be60 0x5650d9f924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f929f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f95360 0x5650d9f92f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5650d9f97f10 con 0x5650d9f8c780 2024-04-16T05:42:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5650d9eaa4d0 con 0x5650d9f8be60 2024-04-16T05:42:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5650d9f98090 con 0x5650d9f95360 2024-04-16T05:42:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea4b3f5740 10 client.0 _ll_register_callbacks cb 0x5650d9e8f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea45cd3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f929f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea45cd3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f929f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54082/0 (socket says 172.21.15.86:54082) 2024-04-16T05:42:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.326+0000 7fea45cd3640 1 -- 172.21.15.86:0/3149354613 learned_addr learned my addr 172.21.15.86:0/3149354613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea464d4640 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8be60 0x5650d9f924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea45cd3640 1 -- 172.21.15.86:0/3149354613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f95360 msgr2=0x5650d9f92f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea45cd3640 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f95360 0x5650d9f92f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea45cd3640 1 -- 172.21.15.86:0/3149354613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8be60 msgr2=0x5650d9f924b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea45cd3640 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8be60 0x5650d9f924b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea45cd3640 1 -- 172.21.15.86:0/3149354613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650d9c033f0 con 0x5650d9f8c780 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea464d4640 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8be60 0x5650d9f924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea45cd3640 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f929f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fea3000b990 tx=0x7fea3000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43586d9837a5c19d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea29ffb640 1 -- 172.21.15.86:0/3149354613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30005020 con 0x5650d9f8c780 2024-04-16T05:42:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea29ffb640 1 -- 172.21.15.86:0/3149354613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fea300051c0 con 0x5650d9f8c780 2024-04-16T05:42:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3149354613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3c0bf4b0 con 0x5650d9f8c780 2024-04-16T05:42:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea29ffb640 1 -- 172.21.15.86:0/3149354613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea300053c0 con 0x5650d9f8c780 2024-04-16T05:42:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea4b3f5740 10 client.17742 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3149354613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650d9ccd7d0 con 0x5650d9f8c780 2024-04-16T05:42:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea29ffb640 1 -- 172.21.15.86:0/3149354613 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c0bf4b0 con 0x5650d9f8c780 2024-04-16T05:42:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea29ffb640 1 -- 172.21.15.86:0/3149354613 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fea3001b570 con 0x5650d9f8c780 2024-04-16T05:42:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea4b3f5740 10 client.17742 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.327+0000 7fea4b3f5740 10 client.17742 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:42:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.328+0000 7fea4b3f5740 20 client.17742 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.328+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5650d9fa5230 0x5650d9fa76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.328+0000 7fea4b3f5740 4 client.17742 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.328+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3149354613 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650d9efeb00 con 0x5650d9fa5230 2024-04-16T05:42:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.328+0000 7fea464d4640 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5650d9fa5230 0x5650d9fa76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.328+0000 7fea29ffb640 10 client.17742 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:42:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.328+0000 7fea464d4640 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5650d9fa5230 0x5650d9fa76d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a86b328a89dc357b server_cookie=7ea225d5143cfa5b in_seq=0 out_seq=0 2024-04-16T05:42:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea29ffb640 1 -- 172.21.15.86:0/3149354613 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650d9efeb00 con 0x5650d9fa5230 2024-04-16T05:42:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea29ffb640 10 client.17742 handle_command_reply: tid=0 2024-04-16T05:42:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 client.17742 shutdown 2024-04-16T05:42:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3149354613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 msgr2=0x5650d9f929f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f929f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fea3000b990 tx=0x7fea3000be50 comp rx=0 tx=0).stop 2024-04-16T05:42:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3149354613 shutdown_connections 2024-04-16T05:42:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5650d9fa5230 0x5650d9fa76d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f95360 0x5650d9f92f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea29ffb640 0 client.17742 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:42:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8c780 0x5650d9f929f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3149354613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8be60 0x5650d9f924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3149354613 >> 172.21.15.86:0/3149354613 conn(0x5650d9f86590 msgr2=0x5650d9f882e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3149354613 shutdown_connections 2024-04-16T05:42:27.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3149354613 wait complete. 2024-04-16T05:42:27.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.329+0000 7fea4b3f5740 20 client.17742 trim_cache size 0 max 16384 2024-04-16T05:42:27.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.342+0000 7fea4b3f5740 1 Processor -- start 2024-04-16T05:42:27.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.342+0000 7fea4b3f5740 1 -- start start 2024-04-16T05:42:27.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.342+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9fa0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.342+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8c780 0x5650d9f92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.342+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f95360 0x5650d9facfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.342+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b1760 con 0x5650d9f8be60 2024-04-16T05:42:27.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.342+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b32a0 con 0x5650d9f95360 2024-04-16T05:42:27.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.342+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b1e30 con 0x5650d9f8c780 2024-04-16T05:42:27.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.342+0000 7fea464d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9fa0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea464d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9fa0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54096/0 (socket says 172.21.15.86:54096) 2024-04-16T05:42:27.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea464d4640 1 -- 172.21.15.86:0/37444828 learned_addr learned my addr 172.21.15.86:0/37444828 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea45cd3640 1 --2- 172.21.15.86:0/37444828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8c780 0x5650d9f92b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea464d4640 1 -- 172.21.15.86:0/37444828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8c780 msgr2=0x5650d9f92b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea464d4640 1 --2- 172.21.15.86:0/37444828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8c780 0x5650d9f92b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea464d4640 1 -- 172.21.15.86:0/37444828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f95360 msgr2=0x5650d9facfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea464d4640 1 --2- 172.21.15.86:0/37444828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f95360 0x5650d9facfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea464d4640 1 -- 172.21.15.86:0/37444828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650d9f8cbc0 con 0x5650d9f8be60 2024-04-16T05:42:27.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea464d4640 1 --2- 172.21.15.86:0/37444828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9fa0c10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0beab0 tx=0x7fea3c0df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70450473c53d25f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea2bfff640 1 -- 172.21.15.86:0/37444828 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0be710 con 0x5650d9f8be60 2024-04-16T05:42:27.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea2bfff640 1 -- 172.21.15.86:0/37444828 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c0c2070 con 0x5650d9f8be60 2024-04-16T05:42:27.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea2bfff640 1 -- 172.21.15.86:0/37444828 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0be8f0 con 0x5650d9f8be60 2024-04-16T05:42:27.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea4b3f5740 1 -- 172.21.15.86:0/37444828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 msgr2=0x5650d9fa0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.343+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/37444828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9fa0c10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0beab0 tx=0x7fea3c0df040 comp rx=0 tx=0).stop 2024-04-16T05:42:27.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 -- 172.21.15.86:0/37444828 shutdown_connections 2024-04-16T05:42:27.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/37444828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f95360 0x5650d9facfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/37444828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f8c780 0x5650d9f92b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/37444828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9fa0c10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 -- 172.21.15.86:0/37444828 >> 172.21.15.86:0/37444828 conn(0x5650d9f86590 msgr2=0x5650d9f92470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 -- 172.21.15.86:0/37444828 shutdown_connections 2024-04-16T05:42:27.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 -- 172.21.15.86:0/37444828 wait complete. 2024-04-16T05:42:27.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 -- start start 2024-04-16T05:42:27.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 Processor -- start 2024-04-16T05:42:27.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9f99200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8c780 0x5650d9f99740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f95360 0x5650da00a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b2bd0 con 0x5650d9f8be60 2024-04-16T05:42:27.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b2500 con 0x5650d9f8c780 2024-04-16T05:42:27.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea401af760 con 0x5650d9f95360 2024-04-16T05:42:27.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea464d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9f99200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea4b3f5740 10 client.0 _ll_register_callbacks cb 0x7fea300042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea464d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9f99200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54112/0 (socket says 172.21.15.86:54112) 2024-04-16T05:42:27.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea464d4640 1 -- 172.21.15.86:0/3799809931 learned_addr learned my addr 172.21.15.86:0/3799809931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea46cd5640 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f95360 0x5650da00a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea464d4640 1 -- 172.21.15.86:0/3799809931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f95360 msgr2=0x5650da00a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea464d4640 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f95360 0x5650da00a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea464d4640 1 -- 172.21.15.86:0/3799809931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8c780 msgr2=0x5650d9f99740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea464d4640 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8c780 0x5650d9f99740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.344+0000 7fea464d4640 1 -- 172.21.15.86:0/3799809931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650da00ad50 con 0x5650d9f8be60 2024-04-16T05:42:27.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea464d4640 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9f99200 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fea3c003f90 tx=0x7fea3c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf1ec9caa031bba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea29ffb640 1 -- 172.21.15.86:0/3799809931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0be8f0 con 0x5650d9f8be60 2024-04-16T05:42:27.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea29ffb640 1 -- 172.21.15.86:0/3799809931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fea3c009890 con 0x5650d9f8be60 2024-04-16T05:42:27.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3799809931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3c0c2070 con 0x5650d9f8be60 2024-04-16T05:42:27.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea29ffb640 1 -- 172.21.15.86:0/3799809931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea3c0be710 con 0x5650d9f8be60 2024-04-16T05:42:27.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea4b3f5740 10 client.17748 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3799809931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650da00bff0 con 0x5650d9f8be60 2024-04-16T05:42:27.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea29ffb640 1 -- 172.21.15.86:0/3799809931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c0c2070 con 0x5650d9f8be60 2024-04-16T05:42:27.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea29ffb640 1 -- 172.21.15.86:0/3799809931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fea3c0a9070 con 0x5650d9f8be60 2024-04-16T05:42:27.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea4b3f5740 10 client.17748 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea4b3f5740 10 client.17748 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:42:27.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea4b3f5740 20 client.17748 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5650da00c4a0 0x5650da00e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea4b3f5740 4 client.17748 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:27.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3799809931 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650d9ef8970 con 0x5650da00c4a0 2024-04-16T05:42:27.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.345+0000 7fea45cd3640 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5650da00c4a0 0x5650da00e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea29ffb640 10 client.17748 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:42:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea45cd3640 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5650da00c4a0 0x5650da00e8e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef9be0cd871bebd1 server_cookie=f5f68745c07fbaab in_seq=0 out_seq=0 2024-04-16T05:42:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea29ffb640 1 -- 172.21.15.86:0/3799809931 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650d9ef8970 con 0x5650da00c4a0 2024-04-16T05:42:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea29ffb640 10 client.17748 handle_command_reply: tid=0 2024-04-16T05:42:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 client.17748 shutdown 2024-04-16T05:42:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3799809931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 msgr2=0x5650d9f99200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9f99200 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fea3c003f90 tx=0x7fea3c005c30 comp rx=0 tx=0).stop 2024-04-16T05:42:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3799809931 shutdown_connections 2024-04-16T05:42:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5650da00c4a0 0x5650da00e8e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650d9f95360 0x5650da00a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea29ffb640 0 client.17748 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:42:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5650d9f8c780 0x5650d9f99740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 --2- 172.21.15.86:0/3799809931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650d9f8be60 0x5650d9f99200 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3799809931 >> 172.21.15.86:0/3799809931 conn(0x5650d9f86590 msgr2=0x5650d9f92ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3799809931 shutdown_connections 2024-04-16T05:42:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 1 -- 172.21.15.86:0/3799809931 wait complete. 2024-04-16T05:42:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.346+0000 7fea4b3f5740 20 client.17748 trim_cache size 0 max 16384 2024-04-16T05:42:27.357 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:42:27.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 -- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea2c0a3d20 msgr2=0x7fea2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea38005850 tx=0x7fea38004730 comp rx=0 tx=0).stop 2024-04-16T05:42:27.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 -- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea401000d0 msgr2=0x7fea401a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea401000d0 0x7fea401a0680 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fea3c005e30 tx=0x7fea3c035ed0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 -- 172.21.15.86:0/2098207600 shutdown_connections 2024-04-16T05:42:27.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea40107060 0x7fea4019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea40100a70 0x7fea401a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 --2- 172.21.15.86:0/2098207600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea401000d0 0x7fea401a0680 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 -- 172.21.15.86:0/2098207600 >> 172.21.15.86:0/2098207600 conn(0x7fea40077050 msgr2=0x7fea40105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 -- 172.21.15.86:0/2098207600 shutdown_connections 2024-04-16T05:42:27.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.347+0000 7fea48738640 1 -- 172.21.15.86:0/2098207600 wait complete. 2024-04-16T05:42:27.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:42:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.467+0000 7efeed115640 1 -- 172.21.15.86:0/3089085352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106210 msgr2=0x7efee810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.467+0000 7efeed115640 1 --2- 172.21.15.86:0/3089085352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106210 0x7efee810dde0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7efee0009ca0 tx=0x7efee002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.467+0000 7efeed115640 1 -- 172.21.15.86:0/3089085352 shutdown_connections 2024-04-16T05:42:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.467+0000 7efeed115640 1 --2- 172.21.15.86:0/3089085352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106210 0x7efee810dde0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.467+0000 7efeed115640 1 --2- 172.21.15.86:0/3089085352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee8101af0 0x7efee8105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.467+0000 7efeed115640 1 --2- 172.21.15.86:0/3089085352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee8101140 0x7efee8101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.467+0000 7efeed115640 1 -- 172.21.15.86:0/3089085352 >> 172.21.15.86:0/3089085352 conn(0x7efee8077280 msgr2=0x7efee8077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.467+0000 7efeed115640 1 -- 172.21.15.86:0/3089085352 shutdown_connections 2024-04-16T05:42:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.467+0000 7efeed115640 1 -- 172.21.15.86:0/3089085352 wait complete. 2024-04-16T05:42:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efeed115640 1 Processor -- start 2024-04-16T05:42:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efeed115640 1 -- start start 2024-04-16T05:42:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efeed115640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101140 0x7efee819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efeed115640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee8106210 0x7efee819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efeed115640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee8198620 0x7efee8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efeed115640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efee8112420 con 0x7efee8101140 2024-04-16T05:42:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efeed115640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efee81122a0 con 0x7efee8198620 2024-04-16T05:42:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efeed115640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efee81125a0 con 0x7efee8106210 2024-04-16T05:42:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efee6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101140 0x7efee819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efee6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101140 0x7efee819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54144/0 (socket says 172.21.15.86:54144) 2024-04-16T05:42:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.468+0000 7efee6d9d640 1 -- 172.21.15.86:0/3476032489 learned_addr learned my addr 172.21.15.86:0/3476032489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efede59c640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee8106210 0x7efee819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee6d9d640 1 -- 172.21.15.86:0/3476032489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee8106210 msgr2=0x7efee819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee6d9d640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee8106210 0x7efee819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee759e640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee8198620 0x7efee8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee6d9d640 1 -- 172.21.15.86:0/3476032489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee8198620 msgr2=0x7efee8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee6d9d640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee8198620 0x7efee8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee6d9d640 1 -- 172.21.15.86:0/3476032489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee8199140 con 0x7efee8101140 2024-04-16T05:42:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efede59c640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee8106210 0x7efee819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee759e640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee8198620 0x7efee8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee6d9d640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101140 0x7efee819e470 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7efed800c920 tx=0x7efed800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee4d99640 1 -- 172.21.15.86:0/3476032489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8007c70 con 0x7efee8101140 2024-04-16T05:42:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efeed115640 1 -- 172.21.15.86:0/3476032489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee8199430 con 0x7efee8101140 2024-04-16T05:42:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee4d99640 1 -- 172.21.15.86:0/3476032489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed8007e10 con 0x7efee8101140 2024-04-16T05:42:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efee4d99640 1 -- 172.21.15.86:0/3476032489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8018890 con 0x7efee8101140 2024-04-16T05:42:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.469+0000 7efeed115640 1 -- 172.21.15.86:0/3476032489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee81a5680 con 0x7efee8101140 2024-04-16T05:42:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.471+0000 7efee4d99640 1 -- 172.21.15.86:0/3476032489 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efed8018a70 con 0x7efee8101140 2024-04-16T05:42:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efeefddd740 1 Processor -- start 2024-04-16T05:42:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efeefddd740 1 -- start start 2024-04-16T05:42:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efeefddd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22abff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efeefddd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 0x55d22ac07e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efeefddd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22ac08360 0x55d22ac0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efeefddd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d22ab640e0 con 0x55d22abfee60 2024-04-16T05:42:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efeefddd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d22ab01520 con 0x55d22ac08360 2024-04-16T05:42:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efeefddd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d22ab600a0 con 0x55d22abff780 2024-04-16T05:42:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efee759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22abff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efee759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22abff240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54146/0 (socket says 172.21.15.86:54146) 2024-04-16T05:42:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efee759e640 1 -- 172.21.15.86:0/2002375936 learned_addr learned my addr 172.21.15.86:0/2002375936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efee4d99640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7efed40a3d20 0x7efed40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efee6d9d640 1 --2- 172.21.15.86:0/2002375936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 0x55d22ac07e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.472+0000 7efede59c640 1 --2- 172.21.15.86:0/2002375936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22ac08360 0x55d22ac0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efee759e640 1 -- 172.21.15.86:0/2002375936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 msgr2=0x55d22ac07e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efee759e640 1 --2- 172.21.15.86:0/2002375936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 0x55d22ac07e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efee759e640 1 -- 172.21.15.86:0/2002375936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22ac08360 msgr2=0x55d22ac0a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efee759e640 1 --2- 172.21.15.86:0/2002375936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22ac08360 0x55d22ac0a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efee759e640 1 -- 172.21.15.86:0/2002375936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d22ab298d0 con 0x55d22abfee60 2024-04-16T05:42:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efee759e640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7efed40a3d20 0x7efed40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efee4d99640 1 -- 172.21.15.86:0/3476032489 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7efed80d0e50 con 0x7efee8101140 2024-04-16T05:42:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efee759e640 1 --2- 172.21.15.86:0/2002375936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22abff240 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7efee0009950 tx=0x7efee0009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6337ce303e3188 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efedd59a640 1 -- 172.21.15.86:0/2002375936 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0039050 con 0x55d22abfee60 2024-04-16T05:42:27.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efedd59a640 1 -- 172.21.15.86:0/2002375936 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee0046030 con 0x55d22abfee60 2024-04-16T05:42:27.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.473+0000 7efedd59a640 1 -- 172.21.15.86:0/2002375936 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0034640 con 0x55d22abfee60 2024-04-16T05:42:27.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 -- 172.21.15.86:0/2002375936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 msgr2=0x55d22abff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efee759e640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7efed40a3d20 0x7efed40a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efee004a000 tx=0x7efee004b000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 --2- 172.21.15.86:0/2002375936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22abff240 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7efee0009950 tx=0x7efee0009980 comp rx=0 tx=0).stop 2024-04-16T05:42:27.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 -- 172.21.15.86:0/2002375936 shutdown_connections 2024-04-16T05:42:27.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 --2- 172.21.15.86:0/2002375936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22ac08360 0x55d22ac0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 --2- 172.21.15.86:0/2002375936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 0x55d22ac07e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 --2- 172.21.15.86:0/2002375936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22abff240 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 -- 172.21.15.86:0/2002375936 >> 172.21.15.86:0/2002375936 conn(0x55d22abf9590 msgr2=0x55d22abf9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 -- 172.21.15.86:0/2002375936 shutdown_connections 2024-04-16T05:42:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 -- 172.21.15.86:0/2002375936 wait complete. 2024-04-16T05:42:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 -- start start 2024-04-16T05:42:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 Processor -- start 2024-04-16T05:42:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.474+0000 7efeefddd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efeefddd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 0x55d22ac058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efeefddd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22ac08360 0x55d22ac05e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efeefddd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d22ac0b000 con 0x55d22abfee60 2024-04-16T05:42:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efeefddd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d22ab1d4d0 con 0x55d22ac08360 2024-04-16T05:42:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efeefddd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d22ac0b180 con 0x55d22abff780 2024-04-16T05:42:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efeefddd740 10 client.0 _ll_register_callbacks cb 0x55d22ab022c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efee6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efee6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac053a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54160/0 (socket says 172.21.15.86:54160) 2024-04-16T05:42:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efee6d9d640 1 -- 172.21.15.86:0/3647257724 learned_addr learned my addr 172.21.15.86:0/3647257724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efede59c640 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 0x55d22ac058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efee6d9d640 1 -- 172.21.15.86:0/3647257724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 msgr2=0x55d22ac058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efee6d9d640 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 0x55d22ac058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efee6d9d640 1 -- 172.21.15.86:0/3647257724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22ac08360 msgr2=0x55d22ac05e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efee6d9d640 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22ac08360 0x55d22ac05e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efee6d9d640 1 -- 172.21.15.86:0/3647257724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d22a8763f0 con 0x55d22abfee60 2024-04-16T05:42:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efee6d9d640 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac053a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7efed8098e80 tx=0x7efed809c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75f40703f84fb4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efece7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efece7fc640 1 -- 172.21.15.86:0/3647257724 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed809d120 con 0x55d22abfee60 2024-04-16T05:42:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efece7fc640 1 -- 172.21.15.86:0/3647257724 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7efed800fb30 con 0x55d22abfee60 2024-04-16T05:42:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efeefddd740 1 -- 172.21.15.86:0/3647257724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee0046030 con 0x55d22abfee60 2024-04-16T05:42:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efece7fc640 1 -- 172.21.15.86:0/3647257724 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efed80a0dd0 con 0x55d22abfee60 2024-04-16T05:42:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efeefddd740 10 client.17760 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efeefddd740 1 -- 172.21.15.86:0/3647257724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d22a9407d0 con 0x55d22abfee60 2024-04-16T05:42:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.475+0000 7efece7fc640 1 -- 172.21.15.86:0/3647257724 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee0046030 con 0x55d22abfee60 2024-04-16T05:42:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efece7fc640 1 -- 172.21.15.86:0/3647257724 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efed809cc80 con 0x55d22abfee60 2024-04-16T05:42:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efeefddd740 10 client.17760 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efeefddd740 10 client.17760 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:42:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efeefddd740 20 client.17760 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efeefddd740 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55d22ac18480 0x55d22ac1a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efeefddd740 4 client.17760 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efeefddd740 1 -- 172.21.15.86:0/3647257724 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d22ab71b00 con 0x55d22ac18480 2024-04-16T05:42:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efede59c640 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55d22ac18480 0x55d22ac1a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efede59c640 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55d22ac18480 0x55d22ac1a870 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ad6c7392a0a6d7e server_cookie=612e9e187261a6dd in_seq=0 out_seq=0 2024-04-16T05:42:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.476+0000 7efece7fc640 10 client.17760 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:42:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efece7fc640 1 -- 172.21.15.86:0/3647257724 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d22ab71b00 con 0x55d22ac18480 2024-04-16T05:42:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efece7fc640 10 client.17760 handle_command_reply: tid=0 2024-04-16T05:42:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 client.17760 shutdown 2024-04-16T05:42:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 -- 172.21.15.86:0/3647257724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 msgr2=0x55d22ac053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac053a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7efed8098e80 tx=0x7efed809c440 comp rx=0 tx=0).stop 2024-04-16T05:42:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 -- 172.21.15.86:0/3647257724 shutdown_connections 2024-04-16T05:42:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22ac08360 0x55d22ac05e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55d22ac18480 0x55d22ac1a870 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efece7fc640 0 client.17760 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:42:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22abff780 0x55d22ac058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 --2- 172.21.15.86:0/3647257724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac053a0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 -- 172.21.15.86:0/3647257724 >> 172.21.15.86:0/3647257724 conn(0x55d22abf9590 msgr2=0x55d22abfb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 -- 172.21.15.86:0/3647257724 shutdown_connections 2024-04-16T05:42:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.477+0000 7efeefddd740 1 -- 172.21.15.86:0/3647257724 wait complete. 2024-04-16T05:42:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.478+0000 7efeefddd740 20 client.17760 trim_cache size 0 max 16384 2024-04-16T05:42:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efeefddd740 1 Processor -- start 2024-04-16T05:42:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efeefddd740 1 -- start start 2024-04-16T05:42:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efeefddd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abfee60 0x55d22ac07ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efeefddd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abff780 0x55d22ac20040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efeefddd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22ac08360 0x55d22ac20580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efeefddd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efee81af4b0 con 0x55d22abff780 2024-04-16T05:42:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efeefddd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efee81b0ff0 con 0x55d22abfee60 2024-04-16T05:42:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efeefddd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efee81afb80 con 0x55d22ac08360 2024-04-16T05:42:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efede59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abff780 0x55d22ac20040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efede59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abff780 0x55d22ac20040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54168/0 (socket says 172.21.15.86:54168) 2024-04-16T05:42:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efede59c640 1 -- 172.21.15.86:0/1297251299 learned_addr learned my addr 172.21.15.86:0/1297251299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efee759e640 1 --2- 172.21.15.86:0/1297251299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22ac08360 0x55d22ac20580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efede59c640 1 -- 172.21.15.86:0/1297251299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22ac08360 msgr2=0x55d22ac20580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efede59c640 1 --2- 172.21.15.86:0/1297251299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22ac08360 0x55d22ac20580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efede59c640 1 -- 172.21.15.86:0/1297251299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abfee60 msgr2=0x55d22ac07ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efede59c640 1 --2- 172.21.15.86:0/1297251299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abfee60 0x55d22ac07ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efede59c640 1 -- 172.21.15.86:0/1297251299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d22ac1ae10 con 0x55d22abff780 2024-04-16T05:42:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.491+0000 7efede59c640 1 --2- 172.21.15.86:0/1297251299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abff780 0x55d22ac20040 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7efee8100680 tx=0x7efed000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cadff3e73ff828 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efedcd99640 1 -- 172.21.15.86:0/1297251299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0015930 con 0x55d22abff780 2024-04-16T05:42:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efedcd99640 1 -- 172.21.15.86:0/1297251299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed0023430 con 0x55d22abff780 2024-04-16T05:42:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efedcd99640 1 -- 172.21.15.86:0/1297251299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0013600 con 0x55d22abff780 2024-04-16T05:42:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efeefddd740 1 -- 172.21.15.86:0/1297251299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abff780 msgr2=0x55d22ac20040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efeefddd740 1 --2- 172.21.15.86:0/1297251299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abff780 0x55d22ac20040 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7efee8100680 tx=0x7efed000cd60 comp rx=0 tx=0).stop 2024-04-16T05:42:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efeefddd740 1 -- 172.21.15.86:0/1297251299 shutdown_connections 2024-04-16T05:42:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efeefddd740 1 --2- 172.21.15.86:0/1297251299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22ac08360 0x55d22ac20580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efeefddd740 1 --2- 172.21.15.86:0/1297251299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abff780 0x55d22ac20040 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efeefddd740 1 --2- 172.21.15.86:0/1297251299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abfee60 0x55d22ac07ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efeefddd740 1 -- 172.21.15.86:0/1297251299 >> 172.21.15.86:0/1297251299 conn(0x55d22abf9590 msgr2=0x55d22abfa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efeefddd740 1 -- 172.21.15.86:0/1297251299 shutdown_connections 2024-04-16T05:42:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.492+0000 7efeefddd740 1 -- 172.21.15.86:0/1297251299 wait complete. 2024-04-16T05:42:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efeefddd740 1 -- start start 2024-04-16T05:42:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efeefddd740 1 Processor -- start 2024-04-16T05:42:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efeefddd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac10510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efeefddd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abff780 0x55d22ac10a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efeefddd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22ac08360 0x55d22ac0d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efeefddd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efee81b0920 con 0x55d22abfee60 2024-04-16T05:42:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efeefddd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efee81b0250 con 0x55d22abff780 2024-04-16T05:42:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efeefddd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efee81ad4b0 con 0x55d22ac08360 2024-04-16T05:42:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efeefddd740 10 client.0 _ll_register_callbacks cb 0x7efed800cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efee6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac10510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efee6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac10510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54180/0 (socket says 172.21.15.86:54180) 2024-04-16T05:42:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efee6d9d640 1 -- 172.21.15.86:0/2036614410 learned_addr learned my addr 172.21.15.86:0/2036614410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efede59c640 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abff780 0x55d22ac10a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efee6d9d640 1 -- 172.21.15.86:0/2036614410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22ac08360 msgr2=0x55d22ac0d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efee6d9d640 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22ac08360 0x55d22ac0d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efee6d9d640 1 -- 172.21.15.86:0/2036614410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abff780 msgr2=0x55d22ac10a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efee6d9d640 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abff780 0x55d22ac10a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efee6d9d640 1 -- 172.21.15.86:0/2036614410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d22ac0dbd0 con 0x55d22abfee60 2024-04-16T05:42:27.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efede59c640 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abff780 0x55d22ac10a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:27.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efee6d9d640 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac10510 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7efed809c9b0 tx=0x7efed800a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5178aa1d4bef9f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efece7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.493+0000 7efece7fc640 1 -- 172.21.15.86:0/2036614410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed809c7f0 con 0x55d22abfee60 2024-04-16T05:42:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efece7fc640 1 -- 172.21.15.86:0/2036614410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7efed80a5200 con 0x55d22abfee60 2024-04-16T05:42:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efeefddd740 1 -- 172.21.15.86:0/2036614410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed0023430 con 0x55d22abfee60 2024-04-16T05:42:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efece7fc640 1 -- 172.21.15.86:0/2036614410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efed809d120 con 0x55d22abfee60 2024-04-16T05:42:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efeefddd740 10 client.17766 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efeefddd740 1 -- 172.21.15.86:0/2036614410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d22ac7f300 con 0x55d22abfee60 2024-04-16T05:42:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efece7fc640 1 -- 172.21.15.86:0/2036614410 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed0023430 con 0x55d22abfee60 2024-04-16T05:42:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efece7fc640 1 -- 172.21.15.86:0/2036614410 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efed8098770 con 0x55d22abfee60 2024-04-16T05:42:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efeefddd740 10 client.17766 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efeefddd740 10 client.17766 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:42:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efeefddd740 20 client.17766 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efeefddd740 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55d22ac18480 0x55d22ac81920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efeefddd740 4 client.17766 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efeefddd740 1 -- 172.21.15.86:0/2036614410 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d22ab6b970 con 0x55d22ac18480 2024-04-16T05:42:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.494+0000 7efede59c640 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55d22ac18480 0x55d22ac81920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efece7fc640 10 client.17766 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:42:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efede59c640 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55d22ac18480 0x55d22ac81920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7adfd2bb8d8aa2fb server_cookie=f11dbce4ea4592a8 in_seq=0 out_seq=0 2024-04-16T05:42:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efece7fc640 1 -- 172.21.15.86:0/2036614410 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d22ab6b970 con 0x55d22ac18480 2024-04-16T05:42:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efece7fc640 10 client.17766 handle_command_reply: tid=0 2024-04-16T05:42:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 client.17766 shutdown 2024-04-16T05:42:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 -- 172.21.15.86:0/2036614410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 msgr2=0x55d22ac10510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac10510 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7efed809c9b0 tx=0x7efed800a5d0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 -- 172.21.15.86:0/2036614410 shutdown_connections 2024-04-16T05:42:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55d22ac18480 0x55d22ac81920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d22ac08360 0x55d22ac0d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d22abff780 0x55d22ac10a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efece7fc640 0 client.17766 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:42:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 --2- 172.21.15.86:0/2036614410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d22abfee60 0x55d22ac10510 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 -- 172.21.15.86:0/2036614410 >> 172.21.15.86:0/2036614410 conn(0x55d22abf9590 msgr2=0x55d22abfb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 -- 172.21.15.86:0/2036614410 shutdown_connections 2024-04-16T05:42:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 1 -- 172.21.15.86:0/2036614410 wait complete. 2024-04-16T05:42:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.495+0000 7efeefddd740 20 client.17766 trim_cache size 0 max 16384 2024-04-16T05:42:27.506 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:42:27.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 -- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7efed40a3d20 msgr2=0x7efed40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7efed40a3d20 0x7efed40a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efee004a000 tx=0x7efee004b000 comp rx=0 tx=0).stop 2024-04-16T05:42:27.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 -- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101140 msgr2=0x7efee819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101140 0x7efee819e470 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7efed800c920 tx=0x7efed800cde0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 -- 172.21.15.86:0/3476032489 shutdown_connections 2024-04-16T05:42:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7efed40a3d20 0x7efed40a61e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee8198620 0x7efee8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee8106210 0x7efee819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 --2- 172.21.15.86:0/3476032489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101140 0x7efee819e470 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 -- 172.21.15.86:0/3476032489 >> 172.21.15.86:0/3476032489 conn(0x7efee8077280 msgr2=0x7efee80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 -- 172.21.15.86:0/3476032489 shutdown_connections 2024-04-16T05:42:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.496+0000 7efeed115640 1 -- 172.21.15.86:0/3476032489 wait complete. 2024-04-16T05:42:27.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:42:27.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/41104974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c94101130 msgr2=0x7f2c94101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/41104974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c94101130 0x7f2c94101510 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:27.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/41104974 shutdown_connections 2024-04-16T05:42:27.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/41104974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c94106200 0x7f2c9410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/41104974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c94101ae0 0x7f2c94105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/41104974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c94101130 0x7f2c94101510 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/41104974 >> 172.21.15.86:0/41104974 conn(0x7f2c940780b0 msgr2=0x7f2c940784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/41104974 shutdown_connections 2024-04-16T05:42:27.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/41104974 wait complete. 2024-04-16T05:42:27.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 Processor -- start 2024-04-16T05:42:27.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.618+0000 7f2c9a9a8640 1 -- start start 2024-04-16T05:42:27.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c9a9a8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c94101130 0x7f2c94199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c9a9a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c94106200 0x7f2c94199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c9a9a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9419a1e0 0x7f2c941a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c9a9a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94111290 con 0x7f2c9419a1e0 2024-04-16T05:42:27.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c9a9a8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94111110 con 0x7f2c94101130 2024-04-16T05:42:27.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c9a9a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94111410 con 0x7f2c94106200 2024-04-16T05:42:27.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c98f45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9419a1e0 0x7f2c941a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c98f45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9419a1e0 0x7f2c941a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54196/0 (socket says 172.21.15.86:54196) 2024-04-16T05:42:27.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c98f45640 1 -- 172.21.15.86:0/3365182571 learned_addr learned my addr 172.21.15.86:0/3365182571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c93fff640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c94101130 0x7f2c94199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c98f45640 1 -- 172.21.15.86:0/3365182571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c94106200 msgr2=0x7f2c94199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c98f45640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c94106200 0x7f2c94199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c98f45640 1 -- 172.21.15.86:0/3365182571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c94101130 msgr2=0x7f2c94199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c98f45640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c94101130 0x7f2c94199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c98f45640 1 -- 172.21.15.86:0/3365182571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c941a4f60 con 0x7f2c9419a1e0 2024-04-16T05:42:27.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c98f45640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9419a1e0 0x7f2c941a4790 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00cad0 tx=0x7f2c8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c91ffb640 1 -- 172.21.15.86:0/3365182571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00ae20 con 0x7f2c9419a1e0 2024-04-16T05:42:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/3365182571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c941a5250 con 0x7f2c9419a1e0 2024-04-16T05:42:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c91ffb640 1 -- 172.21.15.86:0/3365182571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8c005020 con 0x7f2c9419a1e0 2024-04-16T05:42:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c91ffb640 1 -- 172.21.15.86:0/3365182571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00f6f0 con 0x7f2c9419a1e0 2024-04-16T05:42:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.619+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/3365182571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c941a5750 con 0x7f2c9419a1e0 2024-04-16T05:42:27.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.621+0000 7f2c91ffb640 1 -- 172.21.15.86:0/3365182571 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c8c0051d0 con 0x7f2c9419a1e0 2024-04-16T05:42:27.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c9d591740 1 Processor -- start 2024-04-16T05:42:27.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c9d591740 1 -- start start 2024-04-16T05:42:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 0x564438f15240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f15780 0x564438f1de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f1e360 0x564438f20750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c9d591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564438e7a0e0 con 0x564438f1e360 2024-04-16T05:42:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c9d591740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564438e17520 con 0x564438f15780 2024-04-16T05:42:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c9d591740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564438e760a0 con 0x564438f14e60 2024-04-16T05:42:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f1e360 0x564438f20750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c91ffb640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c800a3d20 0x7f2c800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f1e360 0x564438f20750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54206/0 (socket says 172.21.15.86:54206) 2024-04-16T05:42:27.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c8bfff640 1 -- 172.21.15.86:0/3441888901 learned_addr learned my addr 172.21.15.86:0/3441888901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c93fff640 1 --2- 172.21.15.86:0/3441888901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f15780 0x564438f1de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/3441888901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 0x564438f15240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c98f45640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c800a3d20 0x7f2c800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c8bfff640 1 -- 172.21.15.86:0/3441888901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 msgr2=0x564438f15240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/3441888901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 0x564438f15240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c8bfff640 1 -- 172.21.15.86:0/3441888901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f15780 msgr2=0x564438f1de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/3441888901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f15780 0x564438f1de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.622+0000 7f2c8bfff640 1 -- 172.21.15.86:0/3441888901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564438e3f8d0 con 0x564438f1e360 2024-04-16T05:42:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.623+0000 7f2c91ffb640 1 -- 172.21.15.86:0/3365182571 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f2c8c0d0cc0 con 0x7f2c9419a1e0 2024-04-16T05:42:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.623+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/3441888901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f1e360 0x564438f20750 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c005fd0 tx=0x7f2c7c00d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aefeb0d6ea43476 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.623+0000 7f2c98f45640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c800a3d20 0x7f2c800a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0ce640 tx=0x7f2c8c06e2d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.623+0000 7f2c8a7fc640 1 -- 172.21.15.86:0/3441888901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00a690 con 0x564438f1e360 2024-04-16T05:42:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.623+0000 7f2c8a7fc640 1 -- 172.21.15.86:0/3441888901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c00ad10 con 0x564438f1e360 2024-04-16T05:42:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.623+0000 7f2c8a7fc640 1 -- 172.21.15.86:0/3441888901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c010e50 con 0x564438f1e360 2024-04-16T05:42:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.623+0000 7f2c9d591740 1 -- 172.21.15.86:0/3441888901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f1e360 msgr2=0x564438f20750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.623+0000 7f2c9d591740 1 --2- 172.21.15.86:0/3441888901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f1e360 0x564438f20750 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c005fd0 tx=0x7f2c7c00d3c0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 -- 172.21.15.86:0/3441888901 shutdown_connections 2024-04-16T05:42:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 --2- 172.21.15.86:0/3441888901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f1e360 0x564438f20750 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 --2- 172.21.15.86:0/3441888901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f15780 0x564438f1de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 --2- 172.21.15.86:0/3441888901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 0x564438f15240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 -- 172.21.15.86:0/3441888901 >> 172.21.15.86:0/3441888901 conn(0x564438f0f590 msgr2=0x564438f0f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 -- 172.21.15.86:0/3441888901 shutdown_connections 2024-04-16T05:42:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 -- 172.21.15.86:0/3441888901 wait complete. 2024-04-16T05:42:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 -- start start 2024-04-16T05:42:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 Processor -- start 2024-04-16T05:42:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 0x564438f1b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f1e360 0x564438f1bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564438f21000 con 0x564438f15780 2024-04-16T05:42:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564438e334d0 con 0x564438f1e360 2024-04-16T05:42:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564438f21180 con 0x564438f14e60 2024-04-16T05:42:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c9d591740 10 client.0 _ll_register_callbacks cb 0x564438e182c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54212/0 (socket says 172.21.15.86:54212) 2024-04-16T05:42:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.624+0000 7f2c93fff640 1 -- 172.21.15.86:0/4197839288 learned_addr learned my addr 172.21.15.86:0/4197839288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 0x564438f1b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c93fff640 1 -- 172.21.15.86:0/4197839288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 msgr2=0x564438f1b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c93fff640 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 0x564438f1b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c93fff640 1 -- 172.21.15.86:0/4197839288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f1e360 msgr2=0x564438f1bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c93fff640 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f1e360 0x564438f1bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c93fff640 1 -- 172.21.15.86:0/4197839288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564438b8c3f0 con 0x564438f15780 2024-04-16T05:42:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c93fff640 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1b9d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d57c930275852e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c737fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c737fe640 1 -- 172.21.15.86:0/4197839288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84005030 con 0x564438f15780 2024-04-16T05:42:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c737fe640 1 -- 172.21.15.86:0/4197839288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f2c840051d0 con 0x564438f15780 2024-04-16T05:42:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c9d591740 1 -- 172.21.15.86:0/4197839288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7c00ad10 con 0x564438f15780 2024-04-16T05:42:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c737fe640 1 -- 172.21.15.86:0/4197839288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c8404e050 con 0x564438f15780 2024-04-16T05:42:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c9d591740 10 client.17778 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c9d591740 1 -- 172.21.15.86:0/4197839288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564438c567d0 con 0x564438f15780 2024-04-16T05:42:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.625+0000 7f2c737fe640 1 -- 172.21.15.86:0/4197839288 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c00ad10 con 0x564438f15780 2024-04-16T05:42:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c737fe640 1 -- 172.21.15.86:0/4197839288 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c840035f0 con 0x564438f15780 2024-04-16T05:42:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c9d591740 10 client.17778 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c9d591740 10 client.17778 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:42:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c9d591740 20 client.17778 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c9d591740 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564438f32830 0x564438f34c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c9d591740 4 client.17778 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c9d591740 1 -- 172.21.15.86:0/4197839288 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564438e87b00 con 0x564438f32830 2024-04-16T05:42:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564438f32830 0x564438f34c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564438f32830 0x564438f34c70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96ffc56087f3e31 server_cookie=ce7215e80e34fb4 in_seq=0 out_seq=0 2024-04-16T05:42:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.626+0000 7f2c737fe640 10 client.17778 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:42:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c737fe640 1 -- 172.21.15.86:0/4197839288 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564438e87b00 con 0x564438f32830 2024-04-16T05:42:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c737fe640 10 client.17778 handle_command_reply: tid=0 2024-04-16T05:42:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 client.17778 shutdown 2024-04-16T05:42:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 -- 172.21.15.86:0/4197839288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 msgr2=0x564438f1b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1b9d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402ce30 comp rx=0 tx=0).stop 2024-04-16T05:42:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 -- 172.21.15.86:0/4197839288 shutdown_connections 2024-04-16T05:42:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f1e360 0x564438f1bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564438f32830 0x564438f34c70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1b9d0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 --2- 172.21.15.86:0/4197839288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f14e60 0x564438f1b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c737fe640 0 client.17778 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:42:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 -- 172.21.15.86:0/4197839288 >> 172.21.15.86:0/4197839288 conn(0x564438f0f590 msgr2=0x564438f112e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 -- 172.21.15.86:0/4197839288 shutdown_connections 2024-04-16T05:42:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 1 -- 172.21.15.86:0/4197839288 wait complete. 2024-04-16T05:42:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.627+0000 7f2c9d591740 20 client.17778 trim_cache size 0 max 16384 2024-04-16T05:42:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.640+0000 7f2c9d591740 1 Processor -- start 2024-04-16T05:42:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.640+0000 7f2c9d591740 1 -- start start 2024-04-16T05:42:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f14e60 0x564438f1dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f1e360 0x564438f0e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c9d591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941ad390 con 0x564438f15780 2024-04-16T05:42:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c9d591740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941aeed0 con 0x564438f14e60 2024-04-16T05:42:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c9d591740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941ada60 con 0x564438f1e360 2024-04-16T05:42:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54214/0 (socket says 172.21.15.86:54214) 2024-04-16T05:42:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c93fff640 1 -- 172.21.15.86:0/951274330 learned_addr learned my addr 172.21.15.86:0/951274330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/951274330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f14e60 0x564438f1dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c93fff640 1 -- 172.21.15.86:0/951274330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f1e360 msgr2=0x564438f0e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c93fff640 1 --2- 172.21.15.86:0/951274330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f1e360 0x564438f0e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c93fff640 1 -- 172.21.15.86:0/951274330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f14e60 msgr2=0x564438f1dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c93fff640 1 --2- 172.21.15.86:0/951274330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f14e60 0x564438f1dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c93fff640 1 -- 172.21.15.86:0/951274330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564438f352f0 con 0x564438f15780 2024-04-16T05:42:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c93fff640 1 --2- 172.21.15.86:0/951274330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1a5f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402c820 tx=0x7f2c840095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0b0cdc6ec95943 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c89ffb640 1 -- 172.21.15.86:0/951274330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84005d50 con 0x564438f15780 2024-04-16T05:42:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c89ffb640 1 -- 172.21.15.86:0/951274330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c84004310 con 0x564438f15780 2024-04-16T05:42:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.641+0000 7f2c89ffb640 1 -- 172.21.15.86:0/951274330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84034b80 con 0x564438f15780 2024-04-16T05:42:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 -- 172.21.15.86:0/951274330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 msgr2=0x564438f1a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 --2- 172.21.15.86:0/951274330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1a5f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402c820 tx=0x7f2c840095a0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 -- 172.21.15.86:0/951274330 shutdown_connections 2024-04-16T05:42:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 --2- 172.21.15.86:0/951274330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f1e360 0x564438f0e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 --2- 172.21.15.86:0/951274330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f15780 0x564438f1a5f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 --2- 172.21.15.86:0/951274330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f14e60 0x564438f1dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 -- 172.21.15.86:0/951274330 >> 172.21.15.86:0/951274330 conn(0x564438f0f590 msgr2=0x564438f0dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 -- 172.21.15.86:0/951274330 shutdown_connections 2024-04-16T05:42:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 -- 172.21.15.86:0/951274330 wait complete. 2024-04-16T05:42:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.642+0000 7f2c9d591740 1 -- start start 2024-04-16T05:42:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c9d591740 1 Processor -- start 2024-04-16T05:42:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f14e60 0x564438f2aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f15780 0x564438f25f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c9d591740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f1e360 0x564438f26470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c9d591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941ae800 con 0x564438f14e60 2024-04-16T05:42:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c9d591740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941ae130 con 0x564438f1e360 2024-04-16T05:42:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c9d591740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941ab390 con 0x564438f15780 2024-04-16T05:42:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c9d591740 10 client.0 _ll_register_callbacks cb 0x7f2c84009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f14e60 0x564438f2aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f14e60 0x564438f2aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54230/0 (socket says 172.21.15.86:54230) 2024-04-16T05:42:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 -- 172.21.15.86:0/2800009062 learned_addr learned my addr 172.21.15.86:0/2800009062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c93fff640 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f15780 0x564438f25f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f1e360 0x564438f26470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 -- 172.21.15.86:0/2800009062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f15780 msgr2=0x564438f25f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f15780 0x564438f25f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 -- 172.21.15.86:0/2800009062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f1e360 msgr2=0x564438f26470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f1e360 0x564438f26470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 -- 172.21.15.86:0/2800009062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564438f269b0 con 0x564438f14e60 2024-04-16T05:42:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c8bfff640 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f14e60 0x564438f2aa80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c019000 tx=0x7f2c7c00d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e76cd231952e4da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c737fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c737fe640 1 -- 172.21.15.86:0/2800009062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c0105d0 con 0x564438f14e60 2024-04-16T05:42:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c737fe640 1 -- 172.21.15.86:0/2800009062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f2c7c012050 con 0x564438f14e60 2024-04-16T05:42:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.643+0000 7f2c9d591740 1 -- 172.21.15.86:0/2800009062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c84004310 con 0x564438f14e60 2024-04-16T05:42:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c737fe640 1 -- 172.21.15.86:0/2800009062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c034050 con 0x564438f14e60 2024-04-16T05:42:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c9d591740 10 client.17784 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c9d591740 1 -- 172.21.15.86:0/2800009062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564438f98b00 con 0x564438f14e60 2024-04-16T05:42:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c737fe640 1 -- 172.21.15.86:0/2800009062 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c84004310 con 0x564438f14e60 2024-04-16T05:42:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c737fe640 1 -- 172.21.15.86:0/2800009062 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c7c0042b0 con 0x564438f14e60 2024-04-16T05:42:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c9d591740 10 client.17784 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c9d591740 10 client.17784 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:42:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c9d591740 20 client.17784 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c9d591740 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564438f98ef0 0x564438f9b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c9d591740 4 client.17784 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c9d591740 1 -- 172.21.15.86:0/2800009062 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564438e81970 con 0x564438f98ef0 2024-04-16T05:42:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.644+0000 7f2c93fff640 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564438f98ef0 0x564438f9b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c737fe640 10 client.17784 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:42:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c93fff640 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564438f98ef0 0x564438f9b2e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85989db336d28352 server_cookie=befffa66b323e7e2 in_seq=0 out_seq=0 2024-04-16T05:42:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c737fe640 1 -- 172.21.15.86:0/2800009062 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564438e81970 con 0x564438f98ef0 2024-04-16T05:42:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c737fe640 10 client.17784 handle_command_reply: tid=0 2024-04-16T05:42:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 client.17784 shutdown 2024-04-16T05:42:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 -- 172.21.15.86:0/2800009062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f14e60 msgr2=0x564438f2aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f14e60 0x564438f2aa80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c019000 tx=0x7f2c7c00d940 comp rx=0 tx=0).stop 2024-04-16T05:42:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 -- 172.21.15.86:0/2800009062 shutdown_connections 2024-04-16T05:42:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564438f1e360 0x564438f26470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564438f15780 0x564438f25f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564438f98ef0 0x564438f9b2e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 --2- 172.21.15.86:0/2800009062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564438f14e60 0x564438f2aa80 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c737fe640 0 client.17784 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:42:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 -- 172.21.15.86:0/2800009062 >> 172.21.15.86:0/2800009062 conn(0x564438f0f590 msgr2=0x564438f1b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 -- 172.21.15.86:0/2800009062 shutdown_connections 2024-04-16T05:42:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 1 -- 172.21.15.86:0/2800009062 wait complete. 2024-04-16T05:42:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.645+0000 7f2c9d591740 20 client.17784 trim_cache size 0 max 16384 2024-04-16T05:42:27.660 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:42:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c800a3d20 msgr2=0x7f2c800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c800a3d20 0x7f2c800a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0ce640 tx=0x7f2c8c06e2d0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9419a1e0 msgr2=0x7f2c941a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9419a1e0 0x7f2c941a4790 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00cad0 tx=0x7f2c8c00cf90 comp rx=0 tx=0).stop 2024-04-16T05:42:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/3365182571 shutdown_connections 2024-04-16T05:42:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9419a1e0 0x7f2c941a4790 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c94106200 0x7f2c94199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c800a3d20 0x7f2c800a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 --2- 172.21.15.86:0/3365182571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c94101130 0x7f2c94199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/3365182571 >> 172.21.15.86:0/3365182571 conn(0x7f2c940780b0 msgr2=0x7f2c940ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/3365182571 shutdown_connections 2024-04-16T05:42:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.646+0000 7f2c9a9a8640 1 -- 172.21.15.86:0/3365182571 wait complete. 2024-04-16T05:42:27.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:42:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 -- 172.21.15.86:0/532801649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14810c5b0 msgr2=0x7fc14810ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 --2- 172.21.15.86:0/532801649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14810c5b0 0x7fc14810ea50 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc140009ca0 tx=0x7fc14002c230 comp rx=0 tx=0).stop 2024-04-16T05:42:27.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 -- 172.21.15.86:0/532801649 shutdown_connections 2024-04-16T05:42:27.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 --2- 172.21.15.86:0/532801649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14810c5b0 0x7fc14810ea50 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 --2- 172.21.15.86:0/532801649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc148103b90 0x7fc14810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 --2- 172.21.15.86:0/532801649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc148103270 0x7fc148103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 -- 172.21.15.86:0/532801649 >> 172.21.15.86:0/532801649 conn(0x7fc1480780b0 msgr2=0x7fc1480784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 -- 172.21.15.86:0/532801649 shutdown_connections 2024-04-16T05:42:27.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 -- 172.21.15.86:0/532801649 wait complete. 2024-04-16T05:42:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 Processor -- start 2024-04-16T05:42:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 -- start start 2024-04-16T05:42:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.770+0000 7fc14e97c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc148103270 0x7fc148100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc14e97c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc148103b90 0x7fc148101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc14e97c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14810c5b0 0x7fc148101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc14e97c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1481123f0 con 0x7fc148103b90 2024-04-16T05:42:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc14e97c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc148112270 con 0x7fc14810c5b0 2024-04-16T05:42:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc14e97c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc148112570 con 0x7fc148103270 2024-04-16T05:42:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc148103b90 0x7fc148101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc148103b90 0x7fc148101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54252/0 (socket says 172.21.15.86:54252) 2024-04-16T05:42:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc13ffff640 1 -- 172.21.15.86:0/4086843242 learned_addr learned my addr 172.21.15.86:0/4086843242 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc147fff640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc148103270 0x7fc148100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc13ffff640 1 -- 172.21.15.86:0/4086843242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc148103270 msgr2=0x7fc148100ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc14cf19640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14810c5b0 0x7fc148101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc13ffff640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc148103270 0x7fc148100ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc13ffff640 1 -- 172.21.15.86:0/4086843242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14810c5b0 msgr2=0x7fc148101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc13ffff640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14810c5b0 0x7fc148101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc13ffff640 1 -- 172.21.15.86:0/4086843242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1481a6d70 con 0x7fc148103b90 2024-04-16T05:42:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc14cf19640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14810c5b0 0x7fc148101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc13ffff640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc148103b90 0x7fc148101400 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c890 tx=0x7fc13000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc145ffb640 1 -- 172.21.15.86:0/4086843242 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130004220 con 0x7fc148103b90 2024-04-16T05:42:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc145ffb640 1 -- 172.21.15.86:0/4086843242 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1300043c0 con 0x7fc148103b90 2024-04-16T05:42:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc14e97c640 1 -- 172.21.15.86:0/4086843242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1481a7060 con 0x7fc148103b90 2024-04-16T05:42:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc145ffb640 1 -- 172.21.15.86:0/4086843242 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130004a00 con 0x7fc148103b90 2024-04-16T05:42:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.771+0000 7fc14e97c640 1 -- 172.21.15.86:0/4086843242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1481a75b0 con 0x7fc148103b90 2024-04-16T05:42:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.773+0000 7fc145ffb640 1 -- 172.21.15.86:0/4086843242 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc1300079b0 con 0x7fc148103b90 2024-04-16T05:42:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc151563740 1 Processor -- start 2024-04-16T05:42:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc151563740 1 -- start start 2024-04-16T05:42:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc145ffb640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1340a3d50 0x7fc1340a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc151563740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a5e60 0x56040f6a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc151563740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a6780 0x56040f6aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc151563740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc151563740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56040f60b0e0 con 0x56040f6af360 2024-04-16T05:42:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc151563740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56040f5a8520 con 0x56040f6a5e60 2024-04-16T05:42:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc151563740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56040f6070a0 con 0x56040f6a6780 2024-04-16T05:42:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc14cf19640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1340a3d50 0x7fc1340a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc145ffb640 1 -- 172.21.15.86:0/4086843242 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fc1300d0850 con 0x7fc148103b90 2024-04-16T05:42:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc14cf19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc14cf19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54260/0 (socket says 172.21.15.86:54260) 2024-04-16T05:42:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc14cf19640 1 -- 172.21.15.86:0/3375709086 learned_addr learned my addr 172.21.15.86:0/3375709086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc147fff640 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a5e60 0x56040f6a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.774+0000 7fc13ffff640 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a6780 0x56040f6aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc14cf19640 1 -- 172.21.15.86:0/3375709086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a6780 msgr2=0x56040f6aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc14cf19640 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a6780 0x56040f6aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc14cf19640 1 -- 172.21.15.86:0/3375709086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a5e60 msgr2=0x56040f6a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc14cf19640 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a5e60 0x56040f6a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc14cf19640 1 -- 172.21.15.86:0/3375709086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56040f5d08d0 con 0x56040f6af360 2024-04-16T05:42:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc147fff640 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a5e60 0x56040f6a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:42:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc14cf19640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1340a3d50 0x7fc1340a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc1400326c0 tx=0x7fc140032450 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc14cf19640 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6b1750 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc140036000 tx=0x7fc140045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3077af3d417cf7c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc13e7fc640 1 -- 172.21.15.86:0/3375709086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140050070 con 0x56040f6af360 2024-04-16T05:42:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc13e7fc640 1 -- 172.21.15.86:0/3375709086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc14003a040 con 0x56040f6af360 2024-04-16T05:42:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.775+0000 7fc13e7fc640 1 -- 172.21.15.86:0/3375709086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140039070 con 0x56040f6af360 2024-04-16T05:42:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 -- 172.21.15.86:0/3375709086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 msgr2=0x56040f6b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6b1750 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc140036000 tx=0x7fc140045000 comp rx=0 tx=0).stop 2024-04-16T05:42:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 -- 172.21.15.86:0/3375709086 shutdown_connections 2024-04-16T05:42:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6b1750 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a6780 0x56040f6aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 --2- 172.21.15.86:0/3375709086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a5e60 0x56040f6a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 -- 172.21.15.86:0/3375709086 >> 172.21.15.86:0/3375709086 conn(0x56040f6a0590 msgr2=0x56040f6a0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 -- 172.21.15.86:0/3375709086 shutdown_connections 2024-04-16T05:42:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 -- 172.21.15.86:0/3375709086 wait complete. 2024-04-16T05:42:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 -- start start 2024-04-16T05:42:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 Processor -- start 2024-04-16T05:42:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.776+0000 7fc151563740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc151563740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a6780 0x56040f6ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc151563740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc151563740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56040f6b1fa0 con 0x56040f6af360 2024-04-16T05:42:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc151563740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56040f5c44d0 con 0x56040f6a6780 2024-04-16T05:42:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc151563740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56040f6b2120 con 0x56040f6a5e60 2024-04-16T05:42:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc151563740 10 client.0 _ll_register_callbacks cb 0x56040f5a92c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc14cf19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc14cf19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54270/0 (socket says 172.21.15.86:54270) 2024-04-16T05:42:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc14cf19640 1 -- 172.21.15.86:0/2566112305 learned_addr learned my addr 172.21.15.86:0/2566112305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc147fff640 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc14cf19640 1 -- 172.21.15.86:0/2566112305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 msgr2=0x56040f6ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc13ffff640 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a6780 0x56040f6ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc14cf19640 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc14cf19640 1 -- 172.21.15.86:0/2566112305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a6780 msgr2=0x56040f6ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc14cf19640 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a6780 0x56040f6ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc14cf19640 1 -- 172.21.15.86:0/2566112305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56040f31d3f0 con 0x56040f6af360 2024-04-16T05:42:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc14cf19640 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6ace40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc140003ea0 tx=0x7fc140004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889b0c76d434956b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc1277fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc1277fe640 1 -- 172.21.15.86:0/2566112305 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140059020 con 0x56040f6af360 2024-04-16T05:42:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc1277fe640 1 -- 172.21.15.86:0/2566112305 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fc140004040 con 0x56040f6af360 2024-04-16T05:42:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc151563740 1 -- 172.21.15.86:0/2566112305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc14003a040 con 0x56040f6af360 2024-04-16T05:42:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc1277fe640 1 -- 172.21.15.86:0/2566112305 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc140064050 con 0x56040f6af360 2024-04-16T05:42:27.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc151563740 10 client.17796 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.777+0000 7fc151563740 1 -- 172.21.15.86:0/2566112305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56040f3e77d0 con 0x56040f6af360 2024-04-16T05:42:27.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc1277fe640 1 -- 172.21.15.86:0/2566112305 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc14003a040 con 0x56040f6af360 2024-04-16T05:42:27.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc1277fe640 1 -- 172.21.15.86:0/2566112305 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc140005840 con 0x56040f6af360 2024-04-16T05:42:27.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc151563740 10 client.17796 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc151563740 10 client.17796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:42:27.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc151563740 20 client.17796 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc151563740 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56040f6bf170 0x56040f6c1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc151563740 4 client.17796 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:27.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc151563740 1 -- 172.21.15.86:0/2566112305 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56040f618b00 con 0x56040f6bf170 2024-04-16T05:42:27.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc147fff640 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56040f6bf170 0x56040f6c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc1277fe640 10 client.17796 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:42:27.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.778+0000 7fc147fff640 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56040f6bf170 0x56040f6c1560 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38ee25bed22863e8 server_cookie=1a4b45a01e639e85 in_seq=0 out_seq=0 2024-04-16T05:42:27.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc1277fe640 1 -- 172.21.15.86:0/2566112305 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56040f618b00 con 0x56040f6bf170 2024-04-16T05:42:27.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc1277fe640 10 client.17796 handle_command_reply: tid=0 2024-04-16T05:42:27.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 client.17796 shutdown 2024-04-16T05:42:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 -- 172.21.15.86:0/2566112305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 msgr2=0x56040f6ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6ace40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc140003ea0 tx=0x7fc140004a60 comp rx=0 tx=0).stop 2024-04-16T05:42:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 -- 172.21.15.86:0/2566112305 shutdown_connections 2024-04-16T05:42:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56040f6bf170 0x56040f6c1560 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6af360 0x56040f6ace40 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6a6780 0x56040f6ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc1277fe640 0 client.17796 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:42:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 --2- 172.21.15.86:0/2566112305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 -- 172.21.15.86:0/2566112305 >> 172.21.15.86:0/2566112305 conn(0x56040f6a0590 msgr2=0x56040f6a22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 -- 172.21.15.86:0/2566112305 shutdown_connections 2024-04-16T05:42:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.779+0000 7fc151563740 1 -- 172.21.15.86:0/2566112305 wait complete. 2024-04-16T05:42:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.780+0000 7fc151563740 20 client.17796 trim_cache size 0 max 16384 2024-04-16T05:42:27.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc151563740 1 Processor -- start 2024-04-16T05:42:27.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc151563740 1 -- start start 2024-04-16T05:42:27.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc151563740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc151563740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6acc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc151563740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 0x56040f6a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc151563740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1481b1790 con 0x56040f6a6780 2024-04-16T05:42:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc151563740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc1481b32d0 con 0x56040f6af360 2024-04-16T05:42:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc151563740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1481b1e60 con 0x56040f6a5e60 2024-04-16T05:42:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6acc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54284/0 (socket says 172.21.15.86:54284) 2024-04-16T05:42:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13ffff640 1 -- 172.21.15.86:0/922751961 learned_addr learned my addr 172.21.15.86:0/922751961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc14cf19640 1 --2- 172.21.15.86:0/922751961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 0x56040f6a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13ffff640 1 -- 172.21.15.86:0/922751961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 msgr2=0x56040f6aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13ffff640 1 --2- 172.21.15.86:0/922751961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13ffff640 1 -- 172.21.15.86:0/922751961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 msgr2=0x56040f6a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13ffff640 1 --2- 172.21.15.86:0/922751961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 0x56040f6a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13ffff640 1 -- 172.21.15.86:0/922751961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56040f6b1750 con 0x56040f6a6780 2024-04-16T05:42:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13ffff640 1 --2- 172.21.15.86:0/922751961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6acc60 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc130000c00 tx=0x7fc13000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5d2418c668be31 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13dffb640 1 -- 172.21.15.86:0/922751961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13001d040 con 0x56040f6a6780 2024-04-16T05:42:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13dffb640 1 -- 172.21.15.86:0/922751961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13009cb20 con 0x56040f6a6780 2024-04-16T05:42:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.793+0000 7fc13dffb640 1 -- 172.21.15.86:0/922751961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130088150 con 0x56040f6a6780 2024-04-16T05:42:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 -- 172.21.15.86:0/922751961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 msgr2=0x56040f6acc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 --2- 172.21.15.86:0/922751961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6acc60 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc130000c00 tx=0x7fc13000a3f0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 -- 172.21.15.86:0/922751961 shutdown_connections 2024-04-16T05:42:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 --2- 172.21.15.86:0/922751961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 0x56040f6a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 --2- 172.21.15.86:0/922751961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6acc60 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 --2- 172.21.15.86:0/922751961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 -- 172.21.15.86:0/922751961 >> 172.21.15.86:0/922751961 conn(0x56040f6a0590 msgr2=0x56040f6a19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 -- 172.21.15.86:0/922751961 shutdown_connections 2024-04-16T05:42:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 -- 172.21.15.86:0/922751961 wait complete. 2024-04-16T05:42:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.794+0000 7fc151563740 1 -- start start 2024-04-16T05:42:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc151563740 1 Processor -- start 2024-04-16T05:42:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc151563740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc151563740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc151563740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 0x56040f6c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc151563740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1481b2c00 con 0x56040f6a6780 2024-04-16T05:42:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc151563740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc1481b2530 con 0x56040f6af360 2024-04-16T05:42:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc151563740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1481af790 con 0x56040f6a5e60 2024-04-16T05:42:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc151563740 10 client.0 _ll_register_callbacks cb 0x7fc14004e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54288/0 (socket says 172.21.15.86:54288) 2024-04-16T05:42:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc13ffff640 1 -- 172.21.15.86:0/2714144028 learned_addr learned my addr 172.21.15.86:0/2714144028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc13ffff640 1 -- 172.21.15.86:0/2714144028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 msgr2=0x56040f6c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc13ffff640 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc14cf19640 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 0x56040f6c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc13ffff640 1 -- 172.21.15.86:0/2714144028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 msgr2=0x56040f6c1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc13ffff640 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 0x56040f6c1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc13ffff640 1 -- 172.21.15.86:0/2714144028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56040f6c2360 con 0x56040f6a6780 2024-04-16T05:42:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc14cf19640 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 0x56040f6c1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc13ffff640 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6c4ac0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc130007590 tx=0x7fc130094d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca8332f3ad23468 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc1277fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc1277fe640 1 -- 172.21.15.86:0/2714144028 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1300b4020 con 0x56040f6a6780 2024-04-16T05:42:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc1277fe640 1 -- 172.21.15.86:0/2714144028 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fc130098a90 con 0x56040f6a6780 2024-04-16T05:42:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.795+0000 7fc151563740 1 -- 172.21.15.86:0/2714144028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13009cb20 con 0x56040f6a6780 2024-04-16T05:42:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc1277fe640 1 -- 172.21.15.86:0/2714144028 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc13001d070 con 0x56040f6a6780 2024-04-16T05:42:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc151563740 10 client.17802 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc151563740 1 -- 172.21.15.86:0/2714144028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56040f724f80 con 0x56040f6a6780 2024-04-16T05:42:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc1277fe640 1 -- 172.21.15.86:0/2714144028 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13009cb20 con 0x56040f6a6780 2024-04-16T05:42:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc1277fe640 1 -- 172.21.15.86:0/2714144028 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc1300b5ba0 con 0x56040f6a6780 2024-04-16T05:42:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc151563740 10 client.17802 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc151563740 10 client.17802 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:42:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc151563740 20 client.17802 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc151563740 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56040f725620 0x56040f727a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc151563740 4 client.17802 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc151563740 1 -- 172.21.15.86:0/2714144028 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56040f612970 con 0x56040f725620 2024-04-16T05:42:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.796+0000 7fc147fff640 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56040f725620 0x56040f727a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc1277fe640 10 client.17802 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:42:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc147fff640 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56040f725620 0x56040f727a40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa7f4d656d6a0beb server_cookie=568b60ce56d50690 in_seq=0 out_seq=0 2024-04-16T05:42:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc1277fe640 1 -- 172.21.15.86:0/2714144028 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56040f612970 con 0x56040f725620 2024-04-16T05:42:27.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc1277fe640 10 client.17802 handle_command_reply: tid=0 2024-04-16T05:42:27.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 client.17802 shutdown 2024-04-16T05:42:27.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 -- 172.21.15.86:0/2714144028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 msgr2=0x56040f6c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6c4ac0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc130007590 tx=0x7fc130094d40 comp rx=0 tx=0).stop 2024-04-16T05:42:27.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 -- 172.21.15.86:0/2714144028 shutdown_connections 2024-04-16T05:42:27.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x56040f725620 0x56040f727a40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56040f6af360 0x56040f6c1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040f6a6780 0x56040f6c4ac0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc1277fe640 0 client.17802 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:42:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 --2- 172.21.15.86:0/2714144028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56040f6a5e60 0x56040f6c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 -- 172.21.15.86:0/2714144028 >> 172.21.15.86:0/2714144028 conn(0x56040f6a0590 msgr2=0x56040f6ba500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 -- 172.21.15.86:0/2714144028 shutdown_connections 2024-04-16T05:42:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 1 -- 172.21.15.86:0/2714144028 wait complete. 2024-04-16T05:42:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.797+0000 7fc151563740 20 client.17802 trim_cache size 0 max 16384 2024-04-16T05:42:27.808 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:42:27.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 -- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1340a3d50 msgr2=0x7fc1340a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1340a3d50 0x7fc1340a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc1400326c0 tx=0x7fc140032450 comp rx=0 tx=0).stop 2024-04-16T05:42:27.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 -- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc148103b90 msgr2=0x7fc148101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc148103b90 0x7fc148101400 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c890 tx=0x7fc13000cc90 comp rx=0 tx=0).stop 2024-04-16T05:42:27.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 -- 172.21.15.86:0/4086843242 shutdown_connections 2024-04-16T05:42:27.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1340a3d50 0x7fc1340a6210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14810c5b0 0x7fc148101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc148103b90 0x7fc148101400 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 --2- 172.21.15.86:0/4086843242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc148103270 0x7fc148100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 -- 172.21.15.86:0/4086843242 >> 172.21.15.86:0/4086843242 conn(0x7fc1480780b0 msgr2=0x7fc1480fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 -- 172.21.15.86:0/4086843242 shutdown_connections 2024-04-16T05:42:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.798+0000 7fc14e97c640 1 -- 172.21.15.86:0/4086843242 wait complete. 2024-04-16T05:42:27.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:42:27.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 -- 172.21.15.86:0/3453779625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 msgr2=0x7fd21010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 --2- 172.21.15.86:0/3453779625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 0x7fd21010ef00 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd208009ca0 tx=0x7fd20802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 -- 172.21.15.86:0/3453779625 shutdown_connections 2024-04-16T05:42:27.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 --2- 172.21.15.86:0/3453779625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 0x7fd21010ef00 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 --2- 172.21.15.86:0/3453779625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd21006b2c0 0x7fd210106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 --2- 172.21.15.86:0/3453779625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd21006a910 0x7fd21006acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 -- 172.21.15.86:0/3453779625 >> 172.21.15.86:0/3453779625 conn(0x7fd210077cf0 msgr2=0x7fd210078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 -- 172.21.15.86:0/3453779625 shutdown_connections 2024-04-16T05:42:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 -- 172.21.15.86:0/3453779625 wait complete. 2024-04-16T05:42:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 Processor -- start 2024-04-16T05:42:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.920+0000 7fd215847640 1 -- start start 2024-04-16T05:42:27.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd215847640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd21006a910 0x7fd21019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd215847640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 0x7fd21019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd215847640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd210198680 0x7fd210198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd215847640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd210112400 con 0x7fd2101073c0 2024-04-16T05:42:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd215847640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd210112280 con 0x7fd210198680 2024-04-16T05:42:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd215847640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd210112580 con 0x7fd21006a910 2024-04-16T05:42:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 0x7fd21019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 0x7fd21019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54308/0 (socket says 172.21.15.86:54308) 2024-04-16T05:42:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20e7fc640 1 -- 172.21.15.86:0/1729012321 learned_addr learned my addr 172.21.15.86:0/1729012321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20f7fe640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd210198680 0x7fd210198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20effd640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd21006a910 0x7fd21019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20e7fc640 1 -- 172.21.15.86:0/1729012321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd21006a910 msgr2=0x7fd21019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd21006a910 0x7fd21019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20e7fc640 1 -- 172.21.15.86:0/1729012321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd210198680 msgr2=0x7fd210198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd210198680 0x7fd210198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20e7fc640 1 -- 172.21.15.86:0/1729012321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd210199280 con 0x7fd2101073c0 2024-04-16T05:42:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20effd640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd21006a910 0x7fd21019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20f7fe640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd210198680 0x7fd210198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 0x7fd21019eab0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c940 tx=0x7fd20400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd214845640 1 -- 172.21.15.86:0/1729012321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204004220 con 0x7fd2101073c0 2024-04-16T05:42:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd214845640 1 -- 172.21.15.86:0/1729012321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2040043c0 con 0x7fd2101073c0 2024-04-16T05:42:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd215847640 1 -- 172.21.15.86:0/1729012321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2101a5310 con 0x7fd2101073c0 2024-04-16T05:42:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd214845640 1 -- 172.21.15.86:0/1729012321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204004a00 con 0x7fd2101073c0 2024-04-16T05:42:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.921+0000 7fd215847640 1 -- 172.21.15.86:0/1729012321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2101a5710 con 0x7fd2101073c0 2024-04-16T05:42:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd214845640 1 -- 172.21.15.86:0/1729012321 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd2040079b0 con 0x7fd2101073c0 2024-04-16T05:42:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd21842f740 1 Processor -- start 2024-04-16T05:42:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd21842f740 1 -- start start 2024-04-16T05:42:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd21842f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f086840 0x55e24f086c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd21842f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f094230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd21842f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 0x55e24f096b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd21842f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e24efec0e0 con 0x55e24f0871f0 2024-04-16T05:42:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd21842f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e24ef89520 con 0x55e24f086840 2024-04-16T05:42:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd21842f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e24efe80a0 con 0x55e24f094770 2024-04-16T05:42:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd20effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f094230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd20effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f094230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54312/0 (socket says 172.21.15.86:54312) 2024-04-16T05:42:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd20effd640 1 -- 172.21.15.86:0/616348793 learned_addr learned my addr 172.21.15.86:0/616348793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd214845640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1f80a3da0 0x7fd1f80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd20f7fe640 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f086840 0x55e24f086c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 0x55e24f096b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd20effd640 1 -- 172.21.15.86:0/616348793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 msgr2=0x55e24f096b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.924+0000 7fd20f7fe640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1f80a3da0 0x7fd1f80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd20effd640 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 0x55e24f096b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd214845640 1 -- 172.21.15.86:0/1729012321 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fd2040d0890 con 0x7fd2101073c0 2024-04-16T05:42:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd20effd640 1 -- 172.21.15.86:0/616348793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f086840 msgr2=0x55e24f086c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd20effd640 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f086840 0x55e24f086c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd20effd640 1 -- 172.21.15.86:0/616348793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24efb18d0 con 0x55e24f0871f0 2024-04-16T05:42:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 0x55e24f096b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd20effd640 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f094230 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd20000bb70 tx=0x7fd20000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a0b2dfa7fdd1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd20f7fe640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1f80a3da0 0x7fd1f80a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd208032530 tx=0x7fd2080095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd1f5ffb640 1 -- 172.21.15.86:0/616348793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200017620 con 0x55e24f0871f0 2024-04-16T05:42:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd1f5ffb640 1 -- 172.21.15.86:0/616348793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd200017ca0 con 0x55e24f0871f0 2024-04-16T05:42:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.925+0000 7fd1f5ffb640 1 -- 172.21.15.86:0/616348793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20000fc40 con 0x55e24f0871f0 2024-04-16T05:42:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 -- 172.21.15.86:0/616348793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 msgr2=0x55e24f094230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f094230 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd20000bb70 tx=0x7fd20000d710 comp rx=0 tx=0).stop 2024-04-16T05:42:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 -- 172.21.15.86:0/616348793 shutdown_connections 2024-04-16T05:42:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 0x55e24f096b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f094230 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 --2- 172.21.15.86:0/616348793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f086840 0x55e24f086c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 -- 172.21.15.86:0/616348793 >> 172.21.15.86:0/616348793 conn(0x55e24f081590 msgr2=0x55e24f081970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 -- 172.21.15.86:0/616348793 shutdown_connections 2024-04-16T05:42:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 -- 172.21.15.86:0/616348793 wait complete. 2024-04-16T05:42:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 -- start start 2024-04-16T05:42:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 Processor -- start 2024-04-16T05:42:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f086840 0x55e24f0916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f091bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.926+0000 7fd21842f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f094770 0x55e24f092130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd21842f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e24f097530 con 0x55e24f0871f0 2024-04-16T05:42:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd21842f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e24efa54d0 con 0x55e24f094770 2024-04-16T05:42:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd21842f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e24f0976b0 con 0x55e24f086840 2024-04-16T05:42:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f091bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd21842f740 10 client.0 _ll_register_callbacks cb 0x55e24ef8a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f091bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54314/0 (socket says 172.21.15.86:54314) 2024-04-16T05:42:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20e7fc640 1 -- 172.21.15.86:0/3882809770 learned_addr learned my addr 172.21.15.86:0/3882809770 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20effd640 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f086840 0x55e24f0916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20e7fc640 1 -- 172.21.15.86:0/3882809770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f086840 msgr2=0x55e24f0916b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f086840 0x55e24f0916b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20e7fc640 1 -- 172.21.15.86:0/3882809770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f094770 msgr2=0x55e24f092130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f094770 0x55e24f092130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20e7fc640 1 -- 172.21.15.86:0/3882809770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e24ecfe3f0 con 0x55e24f0871f0 2024-04-16T05:42:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f091bf0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd2040ce210 tx=0x7fd204008ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3093c36aa7b6b169 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd1eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3882809770 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20409bdc0 con 0x55e24f0871f0 2024-04-16T05:42:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3882809770 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fd20401ecc0 con 0x55e24f0871f0 2024-04-16T05:42:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd21842f740 1 -- 172.21.15.86:0/3882809770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd200017ca0 con 0x55e24f0871f0 2024-04-16T05:42:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3882809770 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd20401eec0 con 0x55e24f0871f0 2024-04-16T05:42:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd21842f740 10 client.17814 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd21842f740 1 -- 172.21.15.86:0/3882809770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e24edc87d0 con 0x55e24f0871f0 2024-04-16T05:42:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.927+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3882809770 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd200017ca0 con 0x55e24f0871f0 2024-04-16T05:42:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3882809770 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd20401cbb0 con 0x55e24f0871f0 2024-04-16T05:42:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd21842f740 10 client.17814 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd21842f740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd21842f740 20 client.17814 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd21842f740 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e24f0a46b0 0x55e24f0a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd21842f740 4 client.17814 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd21842f740 1 -- 172.21.15.86:0/3882809770 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e24eff9b00 con 0x55e24f0a46b0 2024-04-16T05:42:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd20effd640 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e24f0a46b0 0x55e24f0a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd1eeffd640 10 client.17814 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.928+0000 7fd20effd640 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e24f0a46b0 0x55e24f0a6b00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a56ad1177a0656a server_cookie=9674c8b4c0790932 in_seq=0 out_seq=0 2024-04-16T05:42:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3882809770 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e24eff9b00 con 0x55e24f0a46b0 2024-04-16T05:42:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd1eeffd640 10 client.17814 handle_command_reply: tid=0 2024-04-16T05:42:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 client.17814 shutdown 2024-04-16T05:42:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 -- 172.21.15.86:0/3882809770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 msgr2=0x55e24f091bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f091bf0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd2040ce210 tx=0x7fd204008ed0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 -- 172.21.15.86:0/3882809770 shutdown_connections 2024-04-16T05:42:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e24f0a46b0 0x55e24f0a6b00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f094770 0x55e24f092130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f091bf0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 --2- 172.21.15.86:0/3882809770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f086840 0x55e24f0916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd1eeffd640 0 client.17814 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 -- 172.21.15.86:0/3882809770 >> 172.21.15.86:0/3882809770 conn(0x55e24f081590 msgr2=0x55e24f0832e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 -- 172.21.15.86:0/3882809770 shutdown_connections 2024-04-16T05:42:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 1 -- 172.21.15.86:0/3882809770 wait complete. 2024-04-16T05:42:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.929+0000 7fd21842f740 20 client.17814 trim_cache size 0 max 16384 2024-04-16T05:42:27.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.942+0000 7fd21842f740 1 Processor -- start 2024-04-16T05:42:27.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.942+0000 7fd21842f740 1 -- start start 2024-04-16T05:42:27.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd21842f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f086840 0x55e24f093de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd21842f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f0871f0 0x55e24f0a0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd21842f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 0x55e24f090810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd21842f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101ad2e0 con 0x55e24f086840 2024-04-16T05:42:27.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd21842f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101aee20 con 0x55e24f0871f0 2024-04-16T05:42:27.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd21842f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101ad9b0 con 0x55e24f094770 2024-04-16T05:42:27.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd20effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f086840 0x55e24f093de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd20effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f086840 0x55e24f093de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54322/0 (socket says 172.21.15.86:54322) 2024-04-16T05:42:27.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd20effd640 1 -- 172.21.15.86:0/1657131994 learned_addr learned my addr 172.21.15.86:0/1657131994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd20effd640 1 -- 172.21.15.86:0/1657131994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 msgr2=0x55e24f090810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd20effd640 1 --2- 172.21.15.86:0/1657131994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 0x55e24f090810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd20effd640 1 -- 172.21.15.86:0/1657131994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f0871f0 msgr2=0x55e24f0a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd20effd640 1 --2- 172.21.15.86:0/1657131994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f0871f0 0x55e24f0a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd20effd640 1 -- 172.21.15.86:0/1657131994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24f085f50 con 0x55e24f086840 2024-04-16T05:42:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd20effd640 1 --2- 172.21.15.86:0/1657131994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f086840 0x55e24f093de0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fd20000ba90 tx=0x7fd20000f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f64541cea0d966c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd1f57fa640 1 -- 172.21.15.86:0/1657131994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20000fbb0 con 0x55e24f086840 2024-04-16T05:42:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd1f57fa640 1 -- 172.21.15.86:0/1657131994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20000f610 con 0x55e24f086840 2024-04-16T05:42:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.943+0000 7fd1f57fa640 1 -- 172.21.15.86:0/1657131994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200017e30 con 0x55e24f086840 2024-04-16T05:42:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 -- 172.21.15.86:0/1657131994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f086840 msgr2=0x55e24f093de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 --2- 172.21.15.86:0/1657131994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f086840 0x55e24f093de0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fd20000ba90 tx=0x7fd20000f880 comp rx=0 tx=0).stop 2024-04-16T05:42:27.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 -- 172.21.15.86:0/1657131994 shutdown_connections 2024-04-16T05:42:27.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 --2- 172.21.15.86:0/1657131994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f094770 0x55e24f090810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 --2- 172.21.15.86:0/1657131994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f0871f0 0x55e24f0a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 --2- 172.21.15.86:0/1657131994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f086840 0x55e24f093de0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 -- 172.21.15.86:0/1657131994 >> 172.21.15.86:0/1657131994 conn(0x55e24f081590 msgr2=0x55e24f07fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 -- 172.21.15.86:0/1657131994 shutdown_connections 2024-04-16T05:42:27.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 -- 172.21.15.86:0/1657131994 wait complete. 2024-04-16T05:42:27.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 -- start start 2024-04-16T05:42:27.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.944+0000 7fd21842f740 1 Processor -- start 2024-04-16T05:42:27.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f086840 0x55e24f09cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f08e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f094770 0x55e24f08e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101ae750 con 0x55e24f0871f0 2024-04-16T05:42:27.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101ae080 con 0x55e24f094770 2024-04-16T05:42:27.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101ab2e0 con 0x55e24f086840 2024-04-16T05:42:27.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 10 client.0 _ll_register_callbacks cb 0x7fd204005330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:27.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd20e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f08e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd20e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f08e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54324/0 (socket says 172.21.15.86:54324) 2024-04-16T05:42:27.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd20e7fc640 1 -- 172.21.15.86:0/3152594600 learned_addr learned my addr 172.21.15.86:0/3152594600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:27.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd20e7fc640 1 -- 172.21.15.86:0/3152594600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f086840 msgr2=0x55e24f09cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f086840 0x55e24f09cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd20e7fc640 1 -- 172.21.15.86:0/3152594600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f094770 msgr2=0x55e24f08e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:27.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f094770 0x55e24f08e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd20e7fc640 1 -- 172.21.15.86:0/3152594600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e24f0813c0 con 0x55e24f0871f0 2024-04-16T05:42:27.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd20e7fc640 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f08e110 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd204007590 tx=0x7fd204007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90223aaf2e18afb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:27.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd1eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:27.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3152594600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20409c230 con 0x55e24f0871f0 2024-04-16T05:42:27.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3152594600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fd20409b320 con 0x55e24f0871f0 2024-04-16T05:42:27.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 1 -- 172.21.15.86:0/3152594600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd20000f610 con 0x55e24f0871f0 2024-04-16T05:42:27.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3152594600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd20409c460 con 0x55e24f0871f0 2024-04-16T05:42:27.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 10 client.17820 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:27.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.945+0000 7fd21842f740 1 -- 172.21.15.86:0/3152594600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e24f10aad0 con 0x55e24f0871f0 2024-04-16T05:42:27.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3152594600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20000f610 con 0x55e24f0871f0 2024-04-16T05:42:27.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3152594600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd2040a0170 con 0x55e24f0871f0 2024-04-16T05:42:27.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd21842f740 10 client.17820 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:27.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd21842f740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:27.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd21842f740 20 client.17820 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:27.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd21842f740 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e24f10b270 0x55e24f10d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:27.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd21842f740 4 client.17820 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:27.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd21842f740 1 -- 172.21.15.86:0/3152594600 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e24eff3970 con 0x55e24f10b270 2024-04-16T05:42:27.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd20effd640 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e24f10b270 0x55e24f10d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:27.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd1eeffd640 10 client.17820 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:27.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.946+0000 7fd20effd640 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e24f10b270 0x55e24f10d690 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927322243fc2edd1 server_cookie=a042cc3701d1de7a in_seq=0 out_seq=0 2024-04-16T05:42:27.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd1eeffd640 1 -- 172.21.15.86:0/3152594600 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e24eff3970 con 0x55e24f10b270 2024-04-16T05:42:27.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd1eeffd640 10 client.17820 handle_command_reply: tid=0 2024-04-16T05:42:27.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 client.17820 shutdown 2024-04-16T05:42:27.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 -- 172.21.15.86:0/3152594600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 msgr2=0x55e24f08e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f08e110 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd204007590 tx=0x7fd204007eb0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 -- 172.21.15.86:0/3152594600 shutdown_connections 2024-04-16T05:42:27.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e24f10b270 0x55e24f10d690 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e24f094770 0x55e24f08e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e24f0871f0 0x55e24f08e110 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd1eeffd640 0 client.17820 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:27.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 --2- 172.21.15.86:0/3152594600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e24f086840 0x55e24f09cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 -- 172.21.15.86:0/3152594600 >> 172.21.15.86:0/3152594600 conn(0x55e24f081590 msgr2=0x55e24f080730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 -- 172.21.15.86:0/3152594600 shutdown_connections 2024-04-16T05:42:27.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 1 -- 172.21.15.86:0/3152594600 wait complete. 2024-04-16T05:42:27.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd21842f740 20 client.17820 trim_cache size 0 max 16384 2024-04-16T05:42:27.957 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:42:27.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 -- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1f80a3da0 msgr2=0x7fd1f80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1f80a3da0 0x7fd1f80a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd208032530 tx=0x7fd2080095c0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 -- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 msgr2=0x7fd21019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:27.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 0x7fd21019eab0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c940 tx=0x7fd20400ce00 comp rx=0 tx=0).stop 2024-04-16T05:42:27.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 -- 172.21.15.86:0/1729012321 shutdown_connections 2024-04-16T05:42:27.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1f80a3da0 0x7fd1f80a6260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd210198680 0x7fd210198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2101073c0 0x7fd21019eab0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 --2- 172.21.15.86:0/1729012321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd21006a910 0x7fd21019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:27.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.947+0000 7fd215847640 1 -- 172.21.15.86:0/1729012321 >> 172.21.15.86:0/1729012321 conn(0x7fd210077cf0 msgr2=0x7fd2100ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.948+0000 7fd215847640 1 -- 172.21.15.86:0/1729012321 shutdown_connections 2024-04-16T05:42:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:27.948+0000 7fd215847640 1 -- 172.21.15.86:0/1729012321 wait complete. 2024-04-16T05:42:28.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:28 smithi194 ceph-mon[35865]: pgmap v220: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:42:28.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:28 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2359279883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:42:28.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:42:28.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:28 smithi086 ceph-mon[32028]: pgmap v220: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:42:28.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:28 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2359279883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:42:28.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:42:28.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:28 smithi196 ceph-mon[36373]: pgmap v220: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:42:28.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:28 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2359279883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:42:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:42:28.961 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:42:28.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:42:28.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:42:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.074+0000 7f5ec0637640 1 -- 172.21.15.86:0/1076473939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8102240 msgr2=0x7f5eb8102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.074+0000 7f5ec0637640 1 --2- 172.21.15.86:0/1076473939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8102240 0x7f5eb8102620 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009900 tx=0x7f5eb402bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.074+0000 7f5ec0637640 1 -- 172.21.15.86:0/1076473939 shutdown_connections 2024-04-16T05:42:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.074+0000 7f5ec0637640 1 --2- 172.21.15.86:0/1076473939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5eb8107310 0x7f5eb810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.074+0000 7f5ec0637640 1 --2- 172.21.15.86:0/1076473939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb8102bf0 0x7f5eb8106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.074+0000 7f5ec0637640 1 --2- 172.21.15.86:0/1076473939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8102240 0x7f5eb8102620 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.074+0000 7f5ec0637640 1 -- 172.21.15.86:0/1076473939 >> 172.21.15.86:0/1076473939 conn(0x7f5eb8079450 msgr2=0x7f5eb8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.074+0000 7f5ec0637640 1 -- 172.21.15.86:0/1076473939 shutdown_connections 2024-04-16T05:42:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.074+0000 7f5ec0637640 1 -- 172.21.15.86:0/1076473939 wait complete. 2024-04-16T05:42:29.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ec0637640 1 Processor -- start 2024-04-16T05:42:29.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ec0637640 1 -- start start 2024-04-16T05:42:29.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ec0637640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb8102240 0x7f5eb819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ec0637640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5eb8107310 0x7f5eb819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ec0637640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8198700 0x7f5eb8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ec0637640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81123c0 con 0x7f5eb8198700 2024-04-16T05:42:29.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ec0637640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb8112240 con 0x7f5eb8107310 2024-04-16T05:42:29.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ec0637640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb8112540 con 0x7f5eb8102240 2024-04-16T05:42:29.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebebd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8198700 0x7f5eb8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebebd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8198700 0x7f5eb8198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54342/0 (socket says 172.21.15.86:54342) 2024-04-16T05:42:29.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebebd4640 1 -- 172.21.15.86:0/2150577959 learned_addr learned my addr 172.21.15.86:0/2150577959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb8102240 0x7f5eb819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5eb8107310 0x7f5eb819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebebd4640 1 -- 172.21.15.86:0/2150577959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb8102240 msgr2=0x7f5eb819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebebd4640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb8102240 0x7f5eb819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebebd4640 1 -- 172.21.15.86:0/2150577959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5eb8107310 msgr2=0x7f5eb819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebebd4640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5eb8107310 0x7f5eb819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.075+0000 7f5ebebd4640 1 -- 172.21.15.86:0/2150577959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb8199300 con 0x7f5eb8198700 2024-04-16T05:42:29.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.076+0000 7f5ebebd4640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8198700 0x7f5eb8198b80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000eaa0 tx=0x7f5eb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.076+0000 7f5eaf7fe640 1 -- 172.21.15.86:0/2150577959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000cdb0 con 0x7f5eb8198700 2024-04-16T05:42:29.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.076+0000 7f5eaf7fe640 1 -- 172.21.15.86:0/2150577959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb0005020 con 0x7f5eb8198700 2024-04-16T05:42:29.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.076+0000 7f5ec0637640 1 -- 172.21.15.86:0/2150577959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb81a53e0 con 0x7f5eb8198700 2024-04-16T05:42:29.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.076+0000 7f5eaf7fe640 1 -- 172.21.15.86:0/2150577959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00116b0 con 0x7f5eb8198700 2024-04-16T05:42:29.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.076+0000 7f5ec0637640 1 -- 172.21.15.86:0/2150577959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb81a5790 con 0x7f5eb8198700 2024-04-16T05:42:29.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.078+0000 7f5eaf7fe640 1 -- 172.21.15.86:0/2150577959 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5eb0005560 con 0x7f5eb8198700 2024-04-16T05:42:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ec321b740 1 Processor -- start 2024-04-16T05:42:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ec321b740 1 -- start start 2024-04-16T05:42:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c5760 0x55bb827c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c6080 0x55bb827d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827d33c0 0x55bb827d57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ec321b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb8272ae90 con 0x55bb827d33c0 2024-04-16T05:42:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ec321b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bb826c84c0 con 0x55bb827c5760 2024-04-16T05:42:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ec321b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb82726e50 con 0x55bb827c6080 2024-04-16T05:42:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5eaf7fe640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5ea40a3d20 0x7f5ea40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ebdbd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827d33c0 0x55bb827d57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ebdbd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827d33c0 0x55bb827d57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54356/0 (socket says 172.21.15.86:54356) 2024-04-16T05:42:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/3802659087 learned_addr learned my addr 172.21.15.86:0/3802659087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ebebd4640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5ea40a3d20 0x7f5ea40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/3802659087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c5760 0x55bb827c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.079+0000 7f5eaf7fe640 1 -- 172.21.15.86:0/2150577959 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f5eb00d0d20 con 0x7f5eb8198700 2024-04-16T05:42:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/3802659087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c6080 0x55bb827d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/3802659087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c6080 msgr2=0x55bb827d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/3802659087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c6080 0x55bb827d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/3802659087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c5760 msgr2=0x55bb827c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/3802659087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c5760 0x55bb827c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/3802659087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb825057e0 con 0x55bb827d33c0 2024-04-16T05:42:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/3802659087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827d33c0 0x55bb827d57b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800bde0 tx=0x7f5ea8004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c381a6d5dbe0ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5ebebd4640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5ea40a3d20 0x7f5ea40a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00ce150 tx=0x7f5eb0087040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5eacbfe640 1 -- 172.21.15.86:0/3802659087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8003e80 con 0x55bb827d33c0 2024-04-16T05:42:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5eacbfe640 1 -- 172.21.15.86:0/3802659087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea8005500 con 0x55bb827d33c0 2024-04-16T05:42:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.080+0000 7f5eacbfe640 1 -- 172.21.15.86:0/3802659087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea80126b0 con 0x55bb827d33c0 2024-04-16T05:42:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 -- 172.21.15.86:0/3802659087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827d33c0 msgr2=0x55bb827d57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 --2- 172.21.15.86:0/3802659087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827d33c0 0x55bb827d57b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800bde0 tx=0x7f5ea8004120 comp rx=0 tx=0).stop 2024-04-16T05:42:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 -- 172.21.15.86:0/3802659087 shutdown_connections 2024-04-16T05:42:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 --2- 172.21.15.86:0/3802659087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827d33c0 0x55bb827d57b0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 --2- 172.21.15.86:0/3802659087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c6080 0x55bb827d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 --2- 172.21.15.86:0/3802659087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c5760 0x55bb827c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 -- 172.21.15.86:0/3802659087 >> 172.21.15.86:0/3802659087 conn(0x55bb827c01e0 msgr2=0x55bb827c05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 -- 172.21.15.86:0/3802659087 shutdown_connections 2024-04-16T05:42:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 -- 172.21.15.86:0/3802659087 wait complete. 2024-04-16T05:42:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 -- start start 2024-04-16T05:42:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 Processor -- start 2024-04-16T05:42:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 0x55bb827d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 0x55bb827d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb827d6200 con 0x55bb827c5760 2024-04-16T05:42:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bb826b3410 con 0x55bb827c6080 2024-04-16T05:42:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb827d6380 con 0x55bb827d33c0 2024-04-16T05:42:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.081+0000 7f5ec321b740 10 client.0 _ll_register_callbacks cb 0x55bb826aeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54362/0 (socket says 172.21.15.86:54362) 2024-04-16T05:42:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/2784053755 learned_addr learned my addr 172.21.15.86:0/2784053755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 0x55bb827d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 0x55bb827d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/2784053755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 msgr2=0x55bb827d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 0x55bb827d0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/2784053755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 msgr2=0x55bb827d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 0x55bb827d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/2784053755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb82565f70 con 0x55bb827c5760 2024-04-16T05:42:29.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827d0400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8001eb0 tx=0x7f5ea8004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f181a931291ed28 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:29.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2784053755 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8003e50 con 0x55bb827c5760 2024-04-16T05:42:29.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2784053755 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f5ea8005230 con 0x55bb827c5760 2024-04-16T05:42:29.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ec321b740 1 -- 172.21.15.86:0/2784053755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ea8005500 con 0x55bb827c5760 2024-04-16T05:42:29.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2784053755 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ea8032050 con 0x55bb827c5760 2024-04-16T05:42:29.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ec321b740 10 client.17832 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:29.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.082+0000 7f5ec321b740 1 -- 172.21.15.86:0/2784053755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb823fd170 con 0x55bb827c5760 2024-04-16T05:42:29.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2784053755 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea8005500 con 0x55bb827c5760 2024-04-16T05:42:29.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2784053755 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5ea8012450 con 0x55bb827c5760 2024-04-16T05:42:29.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ec321b740 10 client.17832 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ec321b740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ec321b740 20 client.17832 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb827e3790 0x55bb827e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ec321b740 4 client.17832 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ec321b740 1 -- 172.21.15.86:0/2784053755 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb827388b0 con 0x55bb827e3790 2024-04-16T05:42:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb827e3790 0x55bb827e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ea1ffb640 10 client.17832 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.083+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb827e3790 0x55bb827e5b80 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=632b80b8ed734ccc server_cookie=fd8a3a4f05d0e9a in_seq=0 out_seq=0 2024-04-16T05:42:29.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2784053755 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb827388b0 con 0x55bb827e3790 2024-04-16T05:42:29.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ea1ffb640 10 client.17832 handle_command_reply: tid=0 2024-04-16T05:42:29.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ec321b740 1 client.17832 shutdown 2024-04-16T05:42:29.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ec321b740 1 -- 172.21.15.86:0/2784053755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 msgr2=0x55bb827d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827d0400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8001eb0 tx=0x7f5ea8004c90 comp rx=0 tx=0).stop 2024-04-16T05:42:29.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ec321b740 1 -- 172.21.15.86:0/2784053755 shutdown_connections 2024-04-16T05:42:29.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb827e3790 0x55bb827e5b80 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 0x55bb827d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 0x55bb827d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.085+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2784053755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827d0400 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.084+0000 7f5ea1ffb640 0 client.17832 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:29.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.085+0000 7f5ec321b740 1 -- 172.21.15.86:0/2784053755 >> 172.21.15.86:0/2784053755 conn(0x55bb827c01e0 msgr2=0x55bb827c1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.085+0000 7f5ec321b740 1 -- 172.21.15.86:0/2784053755 shutdown_connections 2024-04-16T05:42:29.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.085+0000 7f5ec321b740 1 -- 172.21.15.86:0/2784053755 wait complete. 2024-04-16T05:42:29.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.085+0000 7f5ec321b740 20 client.17832 trim_cache size 0 max 16384 2024-04-16T05:42:29.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ec321b740 1 Processor -- start 2024-04-16T05:42:29.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ec321b740 1 -- start start 2024-04-16T05:42:29.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c5760 0x55bb827cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c6080 0x55bb827df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827d33c0 0x55bb827d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ec321b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81ad370 con 0x55bb827c6080 2024-04-16T05:42:29.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ec321b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81aeeb0 con 0x55bb827d33c0 2024-04-16T05:42:29.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ec321b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81ada40 con 0x55bb827c5760 2024-04-16T05:42:29.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebe3d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c6080 0x55bb827df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebe3d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c6080 0x55bb827df160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54372/0 (socket says 172.21.15.86:54372) 2024-04-16T05:42:29.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebe3d3640 1 -- 172.21.15.86:0/619014594 learned_addr learned my addr 172.21.15.86:0/619014594 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/619014594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c5760 0x55bb827cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebe3d3640 1 -- 172.21.15.86:0/619014594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c5760 msgr2=0x55bb827cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/619014594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c5760 0x55bb827cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebe3d3640 1 -- 172.21.15.86:0/619014594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827d33c0 msgr2=0x55bb827d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/619014594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827d33c0 0x55bb827d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebe3d3640 1 -- 172.21.15.86:0/619014594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb827d5b70 con 0x55bb827c6080 2024-04-16T05:42:29.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/619014594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c5760 0x55bb827cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:29.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/619014594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c6080 0x55bb827df160 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5eb402c170 tx=0x7f5eb4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d95129bd5afca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.102+0000 7f5ea3fff640 1 -- 172.21.15.86:0/619014594 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb404c070 con 0x55bb827c6080 2024-04-16T05:42:29.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ea3fff640 1 -- 172.21.15.86:0/619014594 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb402cba0 con 0x55bb827c6080 2024-04-16T05:42:29.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ea3fff640 1 -- 172.21.15.86:0/619014594 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4004920 con 0x55bb827c6080 2024-04-16T05:42:29.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 -- 172.21.15.86:0/619014594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c6080 msgr2=0x55bb827df160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 --2- 172.21.15.86:0/619014594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c6080 0x55bb827df160 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5eb402c170 tx=0x7f5eb4005c00 comp rx=0 tx=0).stop 2024-04-16T05:42:29.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 -- 172.21.15.86:0/619014594 shutdown_connections 2024-04-16T05:42:29.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 --2- 172.21.15.86:0/619014594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827d33c0 0x55bb827d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 --2- 172.21.15.86:0/619014594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c6080 0x55bb827df160 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 --2- 172.21.15.86:0/619014594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827c5760 0x55bb827cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 -- 172.21.15.86:0/619014594 >> 172.21.15.86:0/619014594 conn(0x55bb827c01e0 msgr2=0x55bb827d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 -- 172.21.15.86:0/619014594 shutdown_connections 2024-04-16T05:42:29.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 -- 172.21.15.86:0/619014594 wait complete. 2024-04-16T05:42:29.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 -- start start 2024-04-16T05:42:29.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.103+0000 7f5ec321b740 1 Processor -- start 2024-04-16T05:42:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 0x55bb827cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 0x55bb827cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81ae7e0 con 0x55bb827c5760 2024-04-16T05:42:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81ae110 con 0x55bb827c6080 2024-04-16T05:42:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81ab370 con 0x55bb827d33c0 2024-04-16T05:42:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 10 client.0 _ll_register_callbacks cb 0x7f5ea800b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827db8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54378/0 (socket says 172.21.15.86:54378) 2024-04-16T05:42:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/2644813038 learned_addr learned my addr 172.21.15.86:0/2644813038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 0x55bb827cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 0x55bb827cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/2644813038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 msgr2=0x55bb827cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 0x55bb827cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/2644813038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 msgr2=0x55bb827cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 0x55bb827cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 -- 172.21.15.86:0/2644813038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb827cd860 con 0x55bb827c5760 2024-04-16T05:42:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ebdbd2640 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827db8d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8006010 tx=0x7f5ea800bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc2dfc5c8dad6a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2644813038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8003e50 con 0x55bb827c5760 2024-04-16T05:42:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 1 -- 172.21.15.86:0/2644813038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb402cba0 con 0x55bb827c5760 2024-04-16T05:42:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2644813038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f5ea8004da0 con 0x55bb827c5760 2024-04-16T05:42:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2644813038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ea8012a30 con 0x55bb827c5760 2024-04-16T05:42:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 10 client.17838 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.104+0000 7f5ec321b740 1 -- 172.21.15.86:0/2644813038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb828499a0 con 0x55bb827c5760 2024-04-16T05:42:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2644813038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb402cba0 con 0x55bb827c5760 2024-04-16T05:42:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2644813038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5ea801f600 con 0x55bb827c5760 2024-04-16T05:42:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ec321b740 10 client.17838 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ec321b740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ec321b740 20 client.17838 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb8284a010 0x55bb8284c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ec321b740 4 client.17838 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:42:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ec321b740 1 -- 172.21.15.86:0/2644813038 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bb82732720 con 0x55bb8284a010 2024-04-16T05:42:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb8284a010 0x55bb8284c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ea1ffb640 10 client.17838 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.105+0000 7f5ebe3d3640 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb8284a010 0x55bb8284c430 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=385685790f503d10 server_cookie=d3dda11f2786d9c5 in_seq=0 out_seq=0 2024-04-16T05:42:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.106+0000 7f5ea1ffb640 1 -- 172.21.15.86:0/2644813038 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bb82732720 con 0x55bb8284a010 2024-04-16T05:42:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.106+0000 7f5ea1ffb640 10 client.17838 handle_command_reply: tid=0 2024-04-16T05:42:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.106+0000 7f5ec321b740 1 client.17838 shutdown 2024-04-16T05:42:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.106+0000 7f5ec321b740 1 -- 172.21.15.86:0/2644813038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 msgr2=0x55bb827db8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.106+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827db8d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8006010 tx=0x7f5ea800bb80 comp rx=0 tx=0).stop 2024-04-16T05:42:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec321b740 1 -- 172.21.15.86:0/2644813038 shutdown_connections 2024-04-16T05:42:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55bb8284a010 0x55bb8284c430 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb827d33c0 0x55bb827cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb827c6080 0x55bb827cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ea1ffb640 0 client.17838 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec321b740 1 --2- 172.21.15.86:0/2644813038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb827c5760 0x55bb827db8d0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec321b740 1 -- 172.21.15.86:0/2644813038 >> 172.21.15.86:0/2644813038 conn(0x55bb827c01e0 msgr2=0x55bb827d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec321b740 1 -- 172.21.15.86:0/2644813038 shutdown_connections 2024-04-16T05:42:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec321b740 1 -- 172.21.15.86:0/2644813038 wait complete. 2024-04-16T05:42:29.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec321b740 20 client.17838 trim_cache size 0 max 16384 2024-04-16T05:42:29.116 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:42:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:42:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "c6a076ae-3450-4250-bcfd-8bc71c3864be", 2024-04-16T05:42:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:42:29.117 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:42:29.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 -- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5ea40a3d20 msgr2=0x7f5ea40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5ea40a3d20 0x7f5ea40a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00ce150 tx=0x7f5eb0087040 comp rx=0 tx=0).stop 2024-04-16T05:42:29.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 -- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8198700 msgr2=0x7f5eb8198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8198700 0x7f5eb8198b80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000eaa0 tx=0x7f5eb000ef60 comp rx=0 tx=0).stop 2024-04-16T05:42:29.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 -- 172.21.15.86:0/2150577959 shutdown_connections 2024-04-16T05:42:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb8198700 0x7f5eb8198b80 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f5ea40a3d20 0x7f5ea40a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5eb8107310 0x7f5eb819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 --2- 172.21.15.86:0/2150577959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb8102240 0x7f5eb819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 -- 172.21.15.86:0/2150577959 >> 172.21.15.86:0/2150577959 conn(0x7f5eb8079450 msgr2=0x7f5eb80fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 -- 172.21.15.86:0/2150577959 shutdown_connections 2024-04-16T05:42:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.107+0000 7f5ec0637640 1 -- 172.21.15.86:0/2150577959 wait complete. 2024-04-16T05:42:29.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:42:29.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:42:29.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.228+0000 7fe777b00640 1 -- 172.21.15.86:0/3969040021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 msgr2=0x7fe77010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.228+0000 7fe777b00640 1 --2- 172.21.15.86:0/3969040021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 0x7fe77010eee0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe760009900 tx=0x7fe76002bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:29.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.228+0000 7fe777b00640 1 -- 172.21.15.86:0/3969040021 shutdown_connections 2024-04-16T05:42:29.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.228+0000 7fe777b00640 1 --2- 172.21.15.86:0/3969040021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 0x7fe77010eee0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.228+0000 7fe777b00640 1 --2- 172.21.15.86:0/3969040021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe770102bf0 0x7fe770106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.228+0000 7fe777b00640 1 --2- 172.21.15.86:0/3969040021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe770102240 0x7fe770102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.228+0000 7fe777b00640 1 -- 172.21.15.86:0/3969040021 >> 172.21.15.86:0/3969040021 conn(0x7fe770079450 msgr2=0x7fe770079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.228+0000 7fe777b00640 1 -- 172.21.15.86:0/3969040021 shutdown_connections 2024-04-16T05:42:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.228+0000 7fe777b00640 1 -- 172.21.15.86:0/3969040021 wait complete. 2024-04-16T05:42:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe777b00640 1 Processor -- start 2024-04-16T05:42:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe777b00640 1 -- start start 2024-04-16T05:42:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe777b00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe770102240 0x7fe77019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe777b00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 0x7fe77019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe777b00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7701986f0 0x7fe770198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe777b00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7701123d0 con 0x7fe770107310 2024-04-16T05:42:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe777b00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe770112250 con 0x7fe770102240 2024-04-16T05:42:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe777b00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe770112550 con 0x7fe7701986f0 2024-04-16T05:42:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77509b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 0x7fe77019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77509b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 0x7fe77019eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54390/0 (socket says 172.21.15.86:54390) 2024-04-16T05:42:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77509b640 1 -- 172.21.15.86:0/3948869674 learned_addr learned my addr 172.21.15.86:0/3948869674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77609d640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7701986f0 0x7fe770198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77589c640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe770102240 0x7fe77019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77509b640 1 -- 172.21.15.86:0/3948869674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7701986f0 msgr2=0x7fe770198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77509b640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7701986f0 0x7fe770198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77509b640 1 -- 172.21.15.86:0/3948869674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe770102240 msgr2=0x7fe77019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77509b640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe770102240 0x7fe77019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.229+0000 7fe77509b640 1 -- 172.21.15.86:0/3948869674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7701992f0 con 0x7fe770107310 2024-04-16T05:42:29.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.230+0000 7fe77509b640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 0x7fe77019eb70 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9f0 tx=0x7fe76400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.230+0000 7fe76effd640 1 -- 172.21.15.86:0/3948869674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76400ccc0 con 0x7fe770107310 2024-04-16T05:42:29.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.230+0000 7fe76effd640 1 -- 172.21.15.86:0/3948869674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe764005500 con 0x7fe770107310 2024-04-16T05:42:29.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.230+0000 7fe777b00640 1 -- 172.21.15.86:0/3948869674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7701a53d0 con 0x7fe770107310 2024-04-16T05:42:29.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.230+0000 7fe76effd640 1 -- 172.21.15.86:0/3948869674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764010690 con 0x7fe770107310 2024-04-16T05:42:29.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.230+0000 7fe777b00640 1 -- 172.21.15.86:0/3948869674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7701a5780 con 0x7fe770107310 2024-04-16T05:42:29.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe76effd640 1 -- 172.21.15.86:0/3948869674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7640046c0 con 0x7fe770107310 2024-04-16T05:42:29.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77a7c1740 1 Processor -- start 2024-04-16T05:42:29.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77a7c1740 1 -- start start 2024-04-16T05:42:29.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f28230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f28770 0x55e610f30e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 0x55e610f33740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e610e8d180 con 0x55e610f28770 2024-04-16T05:42:29.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e610e2a360 con 0x55e610f31350 2024-04-16T05:42:29.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e610e89140 con 0x55e610f27e50 2024-04-16T05:42:29.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77589c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f28770 0x55e610f30e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77589c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f28770 0x55e610f30e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54398/0 (socket says 172.21.15.86:54398) 2024-04-16T05:42:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77589c640 1 -- 172.21.15.86:0/3759243986 learned_addr learned my addr 172.21.15.86:0/3759243986 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.233+0000 7fe77609d640 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f28230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe76effd640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe7580a3d20 0x7fe7580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77509b640 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 0x55e610f33740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77609d640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe7580a3d20 0x7fe7580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77589c640 1 -- 172.21.15.86:0/3759243986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 msgr2=0x55e610f28230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77589c640 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f28230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77589c640 1 -- 172.21.15.86:0/3759243986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 msgr2=0x55e610f33740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77589c640 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 0x55e610f33740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77589c640 1 -- 172.21.15.86:0/3759243986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e610c697d0 con 0x55e610f28770 2024-04-16T05:42:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77509b640 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 0x55e610f33740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe76effd640 1 -- 172.21.15.86:0/3948869674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fe7640d0f40 con 0x7fe770107310 2024-04-16T05:42:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77589c640 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f28770 0x55e610f30e10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fe768009f60 tx=0x7fe768009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84496f85514d9eae server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe77609d640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe7580a3d20 0x7fe7580a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe760005ab0 tx=0x7fe76002bc30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe757fff640 1 -- 172.21.15.86:0/3759243986 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768016070 con 0x55e610f28770 2024-04-16T05:42:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe757fff640 1 -- 172.21.15.86:0/3759243986 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76801bb40 con 0x55e610f28770 2024-04-16T05:42:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.234+0000 7fe757fff640 1 -- 172.21.15.86:0/3759243986 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768012cd0 con 0x55e610f28770 2024-04-16T05:42:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3759243986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f28770 msgr2=0x55e610f30e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f28770 0x55e610f30e10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fe768009f60 tx=0x7fe768009f90 comp rx=0 tx=0).stop 2024-04-16T05:42:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3759243986 shutdown_connections 2024-04-16T05:42:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 0x55e610f33740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f28770 0x55e610f30e10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/3759243986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f28230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3759243986 >> 172.21.15.86:0/3759243986 conn(0x55e610f22580 msgr2=0x55e610f22960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3759243986 shutdown_connections 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3759243986 wait complete. 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 -- start start 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.235+0000 7fe77a7c1740 1 Processor -- start 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f2e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 0x55e610f2e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610f2edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e610f34140 con 0x55e610f31350 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e610e462c0 con 0x55e610f28770 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e610f342c0 con 0x55e610f27e50 2024-04-16T05:42:29.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77a7c1740 10 client.0 _ll_register_callbacks cb 0x55e610e2d090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77609d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610f2edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77609d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610f2edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54400/0 (socket says 172.21.15.86:54400) 2024-04-16T05:42:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77609d640 1 -- 172.21.15.86:0/491450971 learned_addr learned my addr 172.21.15.86:0/491450971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77509b640 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 0x55e610f2e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77589c640 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f2e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77609d640 1 -- 172.21.15.86:0/491450971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 msgr2=0x55e610f2e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77609d640 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f2e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77609d640 1 -- 172.21.15.86:0/491450971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 msgr2=0x55e610f2e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77609d640 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 0x55e610f2e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77609d640 1 -- 172.21.15.86:0/491450971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e610b9f3f0 con 0x55e610f31350 2024-04-16T05:42:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77509b640 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 0x55e610f2e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe77609d640 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610f2edd0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fe760009900 tx=0x7fe760009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657c1c0bd83cb4d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.236+0000 7fe7557fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe7557fa640 1 -- 172.21.15.86:0/491450971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760052060 con 0x55e610f31350 2024-04-16T05:42:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe7557fa640 1 -- 172.21.15.86:0/491450971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fe760005340 con 0x55e610f31350 2024-04-16T05:42:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77a7c1740 1 -- 172.21.15.86:0/491450971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe76801bb40 con 0x55e610f31350 2024-04-16T05:42:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe7557fa640 1 -- 172.21.15.86:0/491450971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe760059050 con 0x55e610f31350 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77a7c1740 10 client.17850 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77a7c1740 1 -- 172.21.15.86:0/491450971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e610b5f0f0 con 0x55e610f31350 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe7557fa640 1 -- 172.21.15.86:0/491450971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76801bb40 con 0x55e610f31350 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe7557fa640 1 -- 172.21.15.86:0/491450971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe760042440 con 0x55e610f31350 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77a7c1740 10 client.17850 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77a7c1740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77a7c1740 20 client.17850 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e610f416c0 0x55e610f43b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77a7c1740 4 client.17850 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77a7c1740 1 -- 172.21.15.86:0/491450971 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e610e9aba0 con 0x55e610f416c0 2024-04-16T05:42:29.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.237+0000 7fe77589c640 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e610f416c0 0x55e610f43b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.238+0000 7fe77589c640 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e610f416c0 0x55e610f43b40 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8385945760ed75 server_cookie=9d809874d5325dec in_seq=0 out_seq=0 2024-04-16T05:42:29.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.238+0000 7fe7557fa640 10 client.17850 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:29.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.238+0000 7fe7557fa640 1 -- 172.21.15.86:0/491450971 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e610e9aba0 con 0x55e610f416c0 2024-04-16T05:42:29.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.238+0000 7fe7557fa640 10 client.17850 handle_command_reply: tid=0 2024-04-16T05:42:29.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.238+0000 7fe77a7c1740 1 client.17850 shutdown 2024-04-16T05:42:29.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 -- 172.21.15.86:0/491450971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 msgr2=0x55e610f2edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610f2edd0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fe760009900 tx=0x7fe760009260 comp rx=0 tx=0).stop 2024-04-16T05:42:29.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 -- 172.21.15.86:0/491450971 shutdown_connections 2024-04-16T05:42:29.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e610f416c0 0x55e610f43b40 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610f2edd0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 0x55e610f2e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/491450971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f2e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe7557fa640 0 client.17850 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:29.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 -- 172.21.15.86:0/491450971 >> 172.21.15.86:0/491450971 conn(0x55e610f22580 msgr2=0x55e610f242d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 -- 172.21.15.86:0/491450971 shutdown_connections 2024-04-16T05:42:29.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 1 -- 172.21.15.86:0/491450971 wait complete. 2024-04-16T05:42:29.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.239+0000 7fe77a7c1740 20 client.17850 trim_cache size 0 max 16384 2024-04-16T05:42:29.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.256+0000 7fe77a7c1740 1 Processor -- start 2024-04-16T05:42:29.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.256+0000 7fe77a7c1740 1 -- start start 2024-04-16T05:42:29.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.256+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f27e50 0x55e610f30a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.256+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f28770 0x55e610f3d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.256+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 0x55e610f1d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.256+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7701af620 con 0x55e610f27e50 2024-04-16T05:42:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.256+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe7701b1160 con 0x55e610f31350 2024-04-16T05:42:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.256+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7701afcf0 con 0x55e610f28770 2024-04-16T05:42:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77589c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f27e50 0x55e610f30a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77589c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f27e50 0x55e610f30a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54412/0 (socket says 172.21.15.86:54412) 2024-04-16T05:42:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77509b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f28770 0x55e610f3d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77589c640 1 -- 172.21.15.86:0/3237310813 learned_addr learned my addr 172.21.15.86:0/3237310813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77609d640 1 --2- 172.21.15.86:0/3237310813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 0x55e610f1d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77589c640 1 -- 172.21.15.86:0/3237310813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f28770 msgr2=0x55e610f3d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77589c640 1 --2- 172.21.15.86:0/3237310813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f28770 0x55e610f3d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77589c640 1 -- 172.21.15.86:0/3237310813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 msgr2=0x55e610f1d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77589c640 1 --2- 172.21.15.86:0/3237310813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 0x55e610f1d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77589c640 1 -- 172.21.15.86:0/3237310813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e610f2d340 con 0x55e610f27e50 2024-04-16T05:42:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe77589c640 1 --2- 172.21.15.86:0/3237310813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f27e50 0x55e610f30a80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe768009f60 tx=0x7fe768012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5a4950f99f0bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe7577fe640 1 -- 172.21.15.86:0/3237310813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76802c070 con 0x55e610f27e50 2024-04-16T05:42:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe7577fe640 1 -- 172.21.15.86:0/3237310813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe768003e50 con 0x55e610f27e50 2024-04-16T05:42:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.257+0000 7fe7577fe640 1 -- 172.21.15.86:0/3237310813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768016040 con 0x55e610f27e50 2024-04-16T05:42:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3237310813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f27e50 msgr2=0x55e610f30a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/3237310813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f27e50 0x55e610f30a80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe768009f60 tx=0x7fe768012950 comp rx=0 tx=0).stop 2024-04-16T05:42:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3237310813 shutdown_connections 2024-04-16T05:42:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/3237310813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f31350 0x55e610f1d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/3237310813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f28770 0x55e610f3d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/3237310813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f27e50 0x55e610f30a80 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3237310813 >> 172.21.15.86:0/3237310813 conn(0x55e610f22580 msgr2=0x55e610f21760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3237310813 shutdown_connections 2024-04-16T05:42:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 -- 172.21.15.86:0/3237310813 wait complete. 2024-04-16T05:42:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 -- start start 2024-04-16T05:42:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 Processor -- start 2024-04-16T05:42:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f9fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 0x55e610fa0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.258+0000 7fe77a7c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610fa06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7701b0a90 con 0x55e610f31350 2024-04-16T05:42:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe7701b03c0 con 0x55e610f28770 2024-04-16T05:42:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77a7c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7701ad620 con 0x55e610f27e50 2024-04-16T05:42:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77a7c1740 10 client.0 _ll_register_callbacks cb 0x7fe760004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77609d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610fa06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77609d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610fa06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54424/0 (socket says 172.21.15.86:54424) 2024-04-16T05:42:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77589c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f9fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77609d640 1 -- 172.21.15.86:0/1907506310 learned_addr learned my addr 172.21.15.86:0/1907506310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77509b640 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 0x55e610fa0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77609d640 1 -- 172.21.15.86:0/1907506310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 msgr2=0x55e610f9fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77609d640 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f9fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77609d640 1 -- 172.21.15.86:0/1907506310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 msgr2=0x55e610fa0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77609d640 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 0x55e610fa0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77609d640 1 -- 172.21.15.86:0/1907506310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e610faad90 con 0x55e610f31350 2024-04-16T05:42:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77609d640 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610fa06c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe76002cfd0 tx=0x7fe7600426f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0c279f9fcb6528 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe7557fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe7557fa640 1 -- 172.21.15.86:0/1907506310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760052070 con 0x55e610f31350 2024-04-16T05:42:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe7557fa640 1 -- 172.21.15.86:0/1907506310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fe760005140 con 0x55e610f31350 2024-04-16T05:42:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77a7c1740 1 -- 172.21.15.86:0/1907506310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe768003e50 con 0x55e610f31350 2024-04-16T05:42:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe7557fa640 1 -- 172.21.15.86:0/1907506310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe760004cd0 con 0x55e610f31350 2024-04-16T05:42:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77a7c1740 10 client.17856 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.259+0000 7fe77a7c1740 1 -- 172.21.15.86:0/1907506310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e610fac0a0 con 0x55e610f31350 2024-04-16T05:42:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe7557fa640 1 -- 172.21.15.86:0/1907506310 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe768003e50 con 0x55e610f31350 2024-04-16T05:42:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe7557fa640 1 -- 172.21.15.86:0/1907506310 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe760003b80 con 0x55e610f31350 2024-04-16T05:42:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe77a7c1740 10 client.17856 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe77a7c1740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe77a7c1740 20 client.17856 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e610fac780 0x55e610faeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe77a7c1740 4 client.17856 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:42:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe77a7c1740 1 -- 172.21.15.86:0/1907506310 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e610e94a10 con 0x55e610fac780 2024-04-16T05:42:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe77589c640 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e610fac780 0x55e610faeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe7557fa640 10 client.17856 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe77589c640 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e610fac780 0x55e610faeba0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43752980741c0c70 server_cookie=2d458eb8ae245c91 in_seq=0 out_seq=0 2024-04-16T05:42:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe7557fa640 1 -- 172.21.15.86:0/1907506310 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e610e94a10 con 0x55e610fac780 2024-04-16T05:42:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe7557fa640 10 client.17856 handle_command_reply: tid=0 2024-04-16T05:42:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.260+0000 7fe77a7c1740 1 client.17856 shutdown 2024-04-16T05:42:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 -- 172.21.15.86:0/1907506310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 msgr2=0x55e610fa06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610fa06c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe76002cfd0 tx=0x7fe7600426f0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 -- 172.21.15.86:0/1907506310 shutdown_connections 2024-04-16T05:42:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e610fac780 0x55e610faeba0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e610f31350 0x55e610fa06c0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e610f28770 0x55e610fa0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 --2- 172.21.15.86:0/1907506310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e610f27e50 0x55e610f9fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe7557fa640 0 client.17856 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 -- 172.21.15.86:0/1907506310 >> 172.21.15.86:0/1907506310 conn(0x55e610f22580 msgr2=0x55e610f20ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 -- 172.21.15.86:0/1907506310 shutdown_connections 2024-04-16T05:42:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 1 -- 172.21.15.86:0/1907506310 wait complete. 2024-04-16T05:42:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe77a7c1740 20 client.17856 trim_cache size 0 max 16384 2024-04-16T05:42:29.269 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:42:29.270 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:42:29.270 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:42:29.270 INFO:teuthology.orchestra.run.smithi086.stdout: "c6a076ae-3450-4250-bcfd-8bc71c3864be": { 2024-04-16T05:42:29.270 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:42:29.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c6a076ae-3450-4250-bcfd-8bc71c3864be", 2024-04-16T05:42:29.270 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:42:29.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:42:29.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:42:29.270 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:42:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 -- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe7580a3d20 msgr2=0x7fe7580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe7580a3d20 0x7fe7580a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe760005ab0 tx=0x7fe76002bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 -- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 msgr2=0x7fe77019eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 0x7fe77019eb70 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9f0 tx=0x7fe76400eeb0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 -- 172.21.15.86:0/3948869674 shutdown_connections 2024-04-16T05:42:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7701986f0 0x7fe770198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107310 0x7fe77019eb70 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe7580a3d20 0x7fe7580a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 --2- 172.21.15.86:0/3948869674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe770102240 0x7fe77019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 -- 172.21.15.86:0/3948869674 >> 172.21.15.86:0/3948869674 conn(0x7fe770079450 msgr2=0x7fe7700ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 -- 172.21.15.86:0/3948869674 shutdown_connections 2024-04-16T05:42:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:29.261+0000 7fe777b00640 1 -- 172.21.15.86:0/3948869674 wait complete. 2024-04-16T05:42:29.273 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:29 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:42:29.273 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:29 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:42:29.273 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:29 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:42:29.274 INFO:tasks.cephfs.filesystem:scrub status for tag:c6a076ae-3450-4250-bcfd-8bc71c3864be - {'path': '/', 'tag': 'c6a076ae-3450-4250-bcfd-8bc71c3864be', 'options': 'recursive,force'} 2024-04-16T05:42:29.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:29 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:42:29.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:29 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:42:29.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:29 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:42:29.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:29 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:42:29.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:29 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:42:29.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:29 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:42:30.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:30 smithi194 ceph-mon[35865]: pgmap v221: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:42:30.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:30 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:42:30.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:30 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:30.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:30 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:42:30.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:30 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:30.407 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:30 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:42:30.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:30 smithi086 ceph-mon[32028]: pgmap v221: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:42:30.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:30 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:42:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:30 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:30 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:42:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:30 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:30 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:42:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:30 smithi196 ceph-mon[36373]: pgmap v221: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:42:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:30 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:42:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:30 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:30 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:42:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:30 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:30 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:42:32.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:32 smithi194 ceph-mon[35865]: pgmap v222: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:32.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:32 smithi086 ceph-mon[32028]: pgmap v222: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:32.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:32 smithi196 ceph-mon[36373]: pgmap v222: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:34.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:34 smithi194 ceph-mon[35865]: pgmap v223: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:34.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:34 smithi086 ceph-mon[32028]: pgmap v223: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:34.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:34 smithi196 ceph-mon[36373]: pgmap v223: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:36.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:36 smithi194 ceph-mon[35865]: pgmap v224: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:36.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:36 smithi086 ceph-mon[32028]: pgmap v224: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:36.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:36 smithi196 ceph-mon[36373]: pgmap v224: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:38.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:38 smithi194 ceph-mon[35865]: pgmap v225: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:38.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:38 smithi086 ceph-mon[32028]: pgmap v225: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:38.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:38 smithi196 ceph-mon[36373]: pgmap v225: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:40.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:40 smithi194 ceph-mon[35865]: pgmap v226: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:40.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:40 smithi086 ceph-mon[32028]: pgmap v226: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:40.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:40 smithi196 ceph-mon[36373]: pgmap v226: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:42.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:42 smithi194 ceph-mon[35865]: pgmap v227: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:42.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:42 smithi086 ceph-mon[32028]: pgmap v227: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:42.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:42 smithi196 ceph-mon[36373]: pgmap v227: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:42:44.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:44 smithi086 ceph-mon[32028]: pgmap v228: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:44.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:44 smithi196 ceph-mon[36373]: pgmap v228: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:44.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:44 smithi194 ceph-mon[35865]: pgmap v228: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:42:46.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:46 smithi086 ceph-mon[32028]: pgmap v229: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:46.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:46 smithi196 ceph-mon[36373]: pgmap v229: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:46.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:46 smithi194 ceph-mon[35865]: pgmap v229: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:48.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:48 smithi086 ceph-mon[32028]: pgmap v230: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:48.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:48 smithi196 ceph-mon[36373]: pgmap v230: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:48.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:48 smithi194 ceph-mon[35865]: pgmap v230: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:50.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:50 smithi086 ceph-mon[32028]: pgmap v231: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:42:50.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:50 smithi196 ceph-mon[36373]: pgmap v231: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:42:50.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:50 smithi194 ceph-mon[35865]: pgmap v231: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:42:52.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:52 smithi086 ceph-mon[32028]: pgmap v232: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:52.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:52 smithi196 ceph-mon[36373]: pgmap v232: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:52.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:52 smithi194 ceph-mon[35865]: pgmap v232: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:54.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:54 smithi086 ceph-mon[32028]: pgmap v233: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:42:54.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:54 smithi196 ceph-mon[36373]: pgmap v233: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:42:54.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:54 smithi194 ceph-mon[35865]: pgmap v233: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:42:56.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:56 smithi086 ceph-mon[32028]: pgmap v234: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:56.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:56 smithi196 ceph-mon[36373]: pgmap v234: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:56.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:56 smithi194 ceph-mon[35865]: pgmap v234: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:58.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:42:58 smithi086 ceph-mon[32028]: pgmap v235: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:58.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:42:58 smithi196 ceph-mon[36373]: pgmap v235: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:58.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:42:58 smithi194 ceph-mon[35865]: pgmap v235: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:42:59.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:42:59.275 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:42:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.395+0000 7f3c65940640 1 -- 172.21.15.86:0/4223937937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c60107040 msgr2=0x7f3c60107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.395+0000 7f3c65940640 1 --2- 172.21.15.86:0/4223937937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c60107040 0x7f3c60107420 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009930 tx=0x7f3c5002bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.395+0000 7f3c65940640 1 -- 172.21.15.86:0/4223937937 shutdown_connections 2024-04-16T05:42:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.395+0000 7f3c65940640 1 --2- 172.21.15.86:0/4223937937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6006afe0 0x7f3c6010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.395+0000 7f3c65940640 1 --2- 172.21.15.86:0/4223937937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c6006a620 0x7f3c6006aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.395+0000 7f3c65940640 1 --2- 172.21.15.86:0/4223937937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c60107040 0x7f3c60107420 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.395+0000 7f3c65940640 1 -- 172.21.15.86:0/4223937937 >> 172.21.15.86:0/4223937937 conn(0x7f3c60077ee0 msgr2=0x7f3c600782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.395+0000 7f3c65940640 1 -- 172.21.15.86:0/4223937937 shutdown_connections 2024-04-16T05:42:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.395+0000 7f3c65940640 1 -- 172.21.15.86:0/4223937937 wait complete. 2024-04-16T05:42:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c65940640 1 Processor -- start 2024-04-16T05:42:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c65940640 1 -- start start 2024-04-16T05:42:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c65940640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c6006a620 0x7f3c6019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c65940640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6006afe0 0x7f3c6019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c65940640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60107040 0x7f3c60198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c65940640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601123e0 con 0x7f3c6006a620 2024-04-16T05:42:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c65940640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60112260 con 0x7f3c6006afe0 2024-04-16T05:42:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c65940640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60112560 con 0x7f3c60107040 2024-04-16T05:42:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c5effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c6006a620 0x7f3c6019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c5effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c6006a620 0x7f3c6019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41742/0 (socket says 172.21.15.86:41742) 2024-04-16T05:42:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.396+0000 7f3c5effd640 1 -- 172.21.15.86:0/975610487 learned_addr learned my addr 172.21.15.86:0/975610487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c5effd640 1 -- 172.21.15.86:0/975610487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60107040 msgr2=0x7f3c60198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60107040 0x7f3c60198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c567fc640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6006afe0 0x7f3c6019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c5effd640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60107040 0x7f3c60198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c5effd640 1 -- 172.21.15.86:0/975610487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6006afe0 msgr2=0x7f3c6019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c5effd640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6006afe0 0x7f3c6019eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c5effd640 1 -- 172.21.15.86:0/975610487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c60198f20 con 0x7f3c6006a620 2024-04-16T05:42:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60107040 0x7f3c60198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c567fc640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6006afe0 0x7f3c6019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c5effd640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c6006a620 0x7f3c6019e5a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3c50004f50 tx=0x7f3c5002ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c6493e640 1 -- 172.21.15.86:0/975610487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50041070 con 0x7f3c6006a620 2024-04-16T05:42:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c6493e640 1 -- 172.21.15.86:0/975610487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c500044d0 con 0x7f3c6006a620 2024-04-16T05:42:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c6493e640 1 -- 172.21.15.86:0/975610487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50034440 con 0x7f3c6006a620 2024-04-16T05:42:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c65940640 1 -- 172.21.15.86:0/975610487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c601991b0 con 0x7f3c6006a620 2024-04-16T05:42:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.397+0000 7f3c65940640 1 -- 172.21.15.86:0/975610487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c601a5340 con 0x7f3c6006a620 2024-04-16T05:42:59.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.399+0000 7f3c6493e640 1 -- 172.21.15.86:0/975610487 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3c50044070 con 0x7f3c6006a620 2024-04-16T05:42:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c68606740 1 Processor -- start 2024-04-16T05:42:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c68606740 1 -- start start 2024-04-16T05:42:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c68606740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924cfd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c68606740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 0x557924d05e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c68606740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d08740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c68606740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557924c62180 con 0x557924d06350 2024-04-16T05:42:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c68606740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557924bff360 con 0x557924cfce50 2024-04-16T05:42:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c68606740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557924c5e140 con 0x557924cfd770 2024-04-16T05:42:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d08740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c6493e640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3c4c0a3d20 0x7f3c4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d08740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41752/0 (socket says 172.21.15.86:41752) 2024-04-16T05:42:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c567fc640 1 -- 172.21.15.86:0/1305395186 learned_addr learned my addr 172.21.15.86:0/1305395186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924cfd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3c4c0a3d20 0x7f3c4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c567fc640 1 -- 172.21.15.86:0/1305395186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 msgr2=0x557924d05e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c6493e640 1 -- 172.21.15.86:0/975610487 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3c500f2770 con 0x7f3c6006a620 2024-04-16T05:42:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c5effd640 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 0x557924d05e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c567fc640 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 0x557924d05e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c567fc640 1 -- 172.21.15.86:0/1305395186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 msgr2=0x557924cfd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c567fc640 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924cfd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c567fc640 1 -- 172.21.15.86:0/1305395186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557924a3e7d0 con 0x557924d06350 2024-04-16T05:42:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924cfd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.400+0000 7f3c5effd640 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 0x557924d05e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c567fc640 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d08740 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800bb40 tx=0x7f3c4800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd57e491b5629e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3c4c0a3d20 0x7f3c4c0a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800afd0 tx=0x7f3c58005f40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c55ffb640 1 -- 172.21.15.86:0/1305395186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48017420 con 0x557924d06350 2024-04-16T05:42:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c55ffb640 1 -- 172.21.15.86:0/1305395186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c48017aa0 con 0x557924d06350 2024-04-16T05:42:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c55ffb640 1 -- 172.21.15.86:0/1305395186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800fb30 con 0x557924d06350 2024-04-16T05:42:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c68606740 1 -- 172.21.15.86:0/1305395186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 msgr2=0x557924d08740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c68606740 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d08740 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800bb40 tx=0x7f3c4800d340 comp rx=0 tx=0).stop 2024-04-16T05:42:59.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c68606740 1 -- 172.21.15.86:0/1305395186 shutdown_connections 2024-04-16T05:42:59.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c68606740 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d08740 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c68606740 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 0x557924d05e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c68606740 1 --2- 172.21.15.86:0/1305395186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924cfd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c68606740 1 -- 172.21.15.86:0/1305395186 >> 172.21.15.86:0/1305395186 conn(0x557924cf7580 msgr2=0x557924cf7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c68606740 1 -- 172.21.15.86:0/1305395186 shutdown_connections 2024-04-16T05:42:59.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.401+0000 7f3c68606740 1 -- 172.21.15.86:0/1305395186 wait complete. 2024-04-16T05:42:59.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c68606740 1 -- start start 2024-04-16T05:42:59.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c68606740 1 Processor -- start 2024-04-16T05:42:59.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c68606740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924d03310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c68606740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfd770 0x557924d03850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c68606740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924d06350 0x557924d03d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c68606740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557924d09140 con 0x557924cfd770 2024-04-16T05:42:59.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c68606740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557924c1b2c0 con 0x557924cfce50 2024-04-16T05:42:59.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c68606740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557924d092c0 con 0x557924d06350 2024-04-16T05:42:59.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c68606740 10 client.0 _ll_register_callbacks cb 0x557924c02090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:59.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfd770 0x557924d03850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfd770 0x557924d03850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41768/0 (socket says 172.21.15.86:41768) 2024-04-16T05:42:59.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.402+0000 7f3c567fc640 1 -- 172.21.15.86:0/3432032533 learned_addr learned my addr 172.21.15.86:0/3432032533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924d06350 0x557924d03d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c5effd640 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924d03310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c567fc640 1 -- 172.21.15.86:0/3432032533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924d06350 msgr2=0x557924d03d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c567fc640 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924d06350 0x557924d03d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c567fc640 1 -- 172.21.15.86:0/3432032533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 msgr2=0x557924d03310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c567fc640 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924d03310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c567fc640 1 -- 172.21.15.86:0/3432032533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579249743f0 con 0x557924cfd770 2024-04-16T05:42:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c5effd640 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924d03310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c567fc640 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfd770 0x557924d03850 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3c48019040 tx=0x7f3c48017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5964807c457d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c3effd640 1 -- 172.21.15.86:0/3432032533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48026070 con 0x557924cfd770 2024-04-16T05:42:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c3effd640 1 -- 172.21.15.86:0/3432032533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3c4800f910 con 0x557924cfd770 2024-04-16T05:42:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c68606740 1 -- 172.21.15.86:0/3432032533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c48017aa0 con 0x557924cfd770 2024-04-16T05:42:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c3effd640 1 -- 172.21.15.86:0/3432032533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3c48032050 con 0x557924cfd770 2024-04-16T05:42:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c68606740 10 client.17868 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c68606740 1 -- 172.21.15.86:0/3432032533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5579249340f0 con 0x557924cfd770 2024-04-16T05:42:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c3effd640 1 -- 172.21.15.86:0/3432032533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c48017aa0 con 0x557924cfd770 2024-04-16T05:42:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.403+0000 7f3c3effd640 1 -- 172.21.15.86:0/3432032533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3c48022aa0 con 0x557924cfd770 2024-04-16T05:42:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.404+0000 7f3c68606740 10 client.17868 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.404+0000 7f3c68606740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.404+0000 7f3c68606740 20 client.17868 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.404+0000 7f3c68606740 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557924d16500 0x557924d188f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.404+0000 7f3c68606740 4 client.17868 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.404+0000 7f3c68606740 1 -- 172.21.15.86:0/3432032533 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557924c6fba0 con 0x557924d16500 2024-04-16T05:42:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.404+0000 7f3c5effd640 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557924d16500 0x557924d188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.404+0000 7f3c5effd640 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557924d16500 0x557924d188f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452e68ccbf0c3e2c server_cookie=54aa3eda9761581a in_seq=0 out_seq=0 2024-04-16T05:42:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.404+0000 7f3c3effd640 10 client.17868 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c3effd640 1 -- 172.21.15.86:0/3432032533 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557924c6fba0 con 0x557924d16500 2024-04-16T05:42:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c3effd640 10 client.17868 handle_command_reply: tid=0 2024-04-16T05:42:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 client.17868 shutdown 2024-04-16T05:42:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 -- 172.21.15.86:0/3432032533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfd770 msgr2=0x557924d03850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfd770 0x557924d03850 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3c48019040 tx=0x7f3c48017650 comp rx=0 tx=0).stop 2024-04-16T05:42:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 -- 172.21.15.86:0/3432032533 shutdown_connections 2024-04-16T05:42:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557924d16500 0x557924d188f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924d06350 0x557924d03d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfd770 0x557924d03850 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 --2- 172.21.15.86:0/3432032533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfce50 0x557924d03310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c3effd640 0 client.17868 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 -- 172.21.15.86:0/3432032533 >> 172.21.15.86:0/3432032533 conn(0x557924cf7580 msgr2=0x557924cf92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 -- 172.21.15.86:0/3432032533 shutdown_connections 2024-04-16T05:42:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 1 -- 172.21.15.86:0/3432032533 wait complete. 2024-04-16T05:42:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.405+0000 7f3c68606740 20 client.17868 trim_cache size 0 max 16384 2024-04-16T05:42:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c68606740 1 Processor -- start 2024-04-16T05:42:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c68606740 1 -- start start 2024-04-16T05:42:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c68606740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfce50 0x557924d05a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c68606740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 0x557924d12030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c68606740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924d06350 0x557924cf25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c68606740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601af440 con 0x557924cfce50 2024-04-16T05:42:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c68606740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601b0f80 con 0x557924d06350 2024-04-16T05:42:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c68606740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601afb10 con 0x557924cfd770 2024-04-16T05:42:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfce50 0x557924d05a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfce50 0x557924d05a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41772/0 (socket says 172.21.15.86:41772) 2024-04-16T05:42:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5effd640 1 -- 172.21.15.86:0/2845947948 learned_addr learned my addr 172.21.15.86:0/2845947948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5effd640 1 -- 172.21.15.86:0/2845947948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 msgr2=0x557924d12030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5effd640 1 --2- 172.21.15.86:0/2845947948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 0x557924d12030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5effd640 1 -- 172.21.15.86:0/2845947948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924d06350 msgr2=0x557924cf25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/2845947948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924d06350 0x557924cf25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5effd640 1 --2- 172.21.15.86:0/2845947948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924d06350 0x557924cf25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5effd640 1 -- 172.21.15.86:0/2845947948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557924d022c0 con 0x557924cfce50 2024-04-16T05:42:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.423+0000 7f3c5effd640 1 --2- 172.21.15.86:0/2845947948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfce50 0x557924d05a40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3c500efc60 tx=0x7f3c5002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7e95a7d92ef7aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c557fa640 1 -- 172.21.15.86:0/2845947948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5004b040 con 0x557924cfce50 2024-04-16T05:42:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c557fa640 1 -- 172.21.15.86:0/2845947948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c500c2800 con 0x557924cfce50 2024-04-16T05:42:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c557fa640 1 -- 172.21.15.86:0/2845947948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50003710 con 0x557924cfce50 2024-04-16T05:42:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 -- 172.21.15.86:0/2845947948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfce50 msgr2=0x557924d05a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 --2- 172.21.15.86:0/2845947948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfce50 0x557924d05a40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3c500efc60 tx=0x7f3c5002bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 -- 172.21.15.86:0/2845947948 shutdown_connections 2024-04-16T05:42:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 --2- 172.21.15.86:0/2845947948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924d06350 0x557924cf25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 --2- 172.21.15.86:0/2845947948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfd770 0x557924d12030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 --2- 172.21.15.86:0/2845947948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924cfce50 0x557924d05a40 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 -- 172.21.15.86:0/2845947948 >> 172.21.15.86:0/2845947948 conn(0x557924cf7580 msgr2=0x557924cf6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 -- 172.21.15.86:0/2845947948 shutdown_connections 2024-04-16T05:42:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 -- 172.21.15.86:0/2845947948 wait complete. 2024-04-16T05:42:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.424+0000 7f3c68606740 1 -- start start 2024-04-16T05:42:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c68606740 1 Processor -- start 2024-04-16T05:42:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c68606740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfce50 0x557924d0c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c68606740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfd770 0x557924d0c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c68606740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d7b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c68606740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601b08b0 con 0x557924d06350 2024-04-16T05:42:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c68606740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601b01e0 con 0x557924cfd770 2024-04-16T05:42:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c68606740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601ad440 con 0x557924cfce50 2024-04-16T05:42:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c68606740 10 client.0 _ll_register_callbacks cb 0x7f3c4800b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d7b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d7b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41788/0 (socket says 172.21.15.86:41788) 2024-04-16T05:42:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5f7fe640 1 -- 172.21.15.86:0/19880006 learned_addr learned my addr 172.21.15.86:0/19880006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5effd640 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfce50 0x557924d0c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c567fc640 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfd770 0x557924d0c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5f7fe640 1 -- 172.21.15.86:0/19880006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfce50 msgr2=0x557924d0c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfce50 0x557924d0c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5f7fe640 1 -- 172.21.15.86:0/19880006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfd770 msgr2=0x557924d0c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfd770 0x557924d0c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5f7fe640 1 -- 172.21.15.86:0/19880006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557924d0d100 con 0x557924d06350 2024-04-16T05:42:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c5f7fe640 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d7b980 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800f7a0 tx=0x7f3c58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51145128556888f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.425+0000 7f3c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c3effd640 1 -- 172.21.15.86:0/19880006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58022070 con 0x557924d06350 2024-04-16T05:42:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c3effd640 1 -- 172.21.15.86:0/19880006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3c5801e910 con 0x557924d06350 2024-04-16T05:42:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c68606740 1 -- 172.21.15.86:0/19880006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c500c2800 con 0x557924d06350 2024-04-16T05:42:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c3effd640 1 -- 172.21.15.86:0/19880006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3c58016040 con 0x557924d06350 2024-04-16T05:42:59.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c68606740 10 client.17874 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:59.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c68606740 1 -- 172.21.15.86:0/19880006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557924d7d030 con 0x557924d06350 2024-04-16T05:42:59.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c3effd640 1 -- 172.21.15.86:0/19880006 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c500c2800 con 0x557924d06350 2024-04-16T05:42:59.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c3effd640 1 -- 172.21.15.86:0/19880006 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3c5801d3d0 con 0x557924d06350 2024-04-16T05:42:59.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c68606740 10 client.17874 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:59.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c68606740 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:59.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c68606740 20 client.17874 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:59.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c68606740 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557924d7d750 0x557924d7fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c68606740 4 client.17874 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:42:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c68606740 1 -- 172.21.15.86:0/19880006 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557924c69a10 con 0x557924d7d750 2024-04-16T05:42:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.426+0000 7f3c5effd640 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557924d7d750 0x557924d7fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c3effd640 10 client.17874 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c5effd640 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557924d7d750 0x557924d7fb40 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dee462ad4c90c41 server_cookie=86241bc440a7a7cb in_seq=0 out_seq=0 2024-04-16T05:42:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c3effd640 1 -- 172.21.15.86:0/19880006 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557924c69a10 con 0x557924d7d750 2024-04-16T05:42:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c3effd640 10 client.17874 handle_command_reply: tid=0 2024-04-16T05:42:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 client.17874 shutdown 2024-04-16T05:42:59.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 -- 172.21.15.86:0/19880006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 msgr2=0x557924d7b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d7b980 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800f7a0 tx=0x7f3c58004a40 comp rx=0 tx=0).stop 2024-04-16T05:42:59.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 -- 172.21.15.86:0/19880006 shutdown_connections 2024-04-16T05:42:59.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557924d7d750 0x557924d7fb40 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557924d06350 0x557924d7b980 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557924cfd770 0x557924d0c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c3effd640 0 client.17874 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:42:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 --2- 172.21.15.86:0/19880006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557924cfce50 0x557924d0c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 -- 172.21.15.86:0/19880006 >> 172.21.15.86:0/19880006 conn(0x557924cf7580 msgr2=0x557924d032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 -- 172.21.15.86:0/19880006 shutdown_connections 2024-04-16T05:42:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 1 -- 172.21.15.86:0/19880006 wait complete. 2024-04-16T05:42:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.427+0000 7f3c68606740 20 client.17874 trim_cache size 0 max 16384 2024-04-16T05:42:59.442 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:42:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:42:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:42:59.443 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:42:59.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 -- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3c4c0a3d20 msgr2=0x7f3c4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3c4c0a3d20 0x7f3c4c0a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800afd0 tx=0x7f3c58005f40 comp rx=0 tx=0).stop 2024-04-16T05:42:59.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 -- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c6006a620 msgr2=0x7f3c6019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c6006a620 0x7f3c6019e5a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3c50004f50 tx=0x7f3c5002ce20 comp rx=0 tx=0).stop 2024-04-16T05:42:59.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 -- 172.21.15.86:0/975610487 shutdown_connections 2024-04-16T05:42:59.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60107040 0x7f3c60198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3c4c0a3d20 0x7f3c4c0a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c6006afe0 0x7f3c6019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 --2- 172.21.15.86:0/975610487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c6006a620 0x7f3c6019e5a0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 -- 172.21.15.86:0/975610487 >> 172.21.15.86:0/975610487 conn(0x7f3c60077ee0 msgr2=0x7f3c600fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 -- 172.21.15.86:0/975610487 shutdown_connections 2024-04-16T05:42:59.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.428+0000 7f3c65940640 1 -- 172.21.15.86:0/975610487 wait complete. 2024-04-16T05:42:59.449 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:42:59.449 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:42:59.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.558+0000 7fd1e7693640 1 -- 172.21.15.86:0/3462648712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0102240 msgr2=0x7fd1e0102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.558+0000 7fd1e7693640 1 --2- 172.21.15.86:0/3462648712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0102240 0x7fd1e0102620 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009900 tx=0x7fd1dc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:59.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.558+0000 7fd1e7693640 1 -- 172.21.15.86:0/3462648712 shutdown_connections 2024-04-16T05:42:59.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.558+0000 7fd1e7693640 1 --2- 172.21.15.86:0/3462648712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e0107310 0x7fd1e010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.558+0000 7fd1e7693640 1 --2- 172.21.15.86:0/3462648712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e0102bf0 0x7fd1e0106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.558+0000 7fd1e7693640 1 --2- 172.21.15.86:0/3462648712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0102240 0x7fd1e0102620 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.558+0000 7fd1e7693640 1 -- 172.21.15.86:0/3462648712 >> 172.21.15.86:0/3462648712 conn(0x7fd1e00793c0 msgr2=0x7fd1e00797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.558+0000 7fd1e7693640 1 -- 172.21.15.86:0/3462648712 shutdown_connections 2024-04-16T05:42:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.558+0000 7fd1e7693640 1 -- 172.21.15.86:0/3462648712 wait complete. 2024-04-16T05:42:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e7693640 1 Processor -- start 2024-04-16T05:42:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e7693640 1 -- start start 2024-04-16T05:42:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e7693640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e0102240 0x7fd1e019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e7693640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0107310 0x7fd1e019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e7693640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e0198820 0x7fd1e0198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e7693640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e01124a0 con 0x7fd1e0107310 2024-04-16T05:42:59.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e7693640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0112320 con 0x7fd1e0102240 2024-04-16T05:42:59.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e7693640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0112620 con 0x7fd1e0198820 2024-04-16T05:42:59.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e4c2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0107310 0x7fd1e019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e4c2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0107310 0x7fd1e019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41808/0 (socket says 172.21.15.86:41808) 2024-04-16T05:42:59.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e4c2e640 1 -- 172.21.15.86:0/740660639 learned_addr learned my addr 172.21.15.86:0/740660639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e542f640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e0102240 0x7fd1e019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e5c30640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e0198820 0x7fd1e0198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e4c2e640 1 -- 172.21.15.86:0/740660639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e0198820 msgr2=0x7fd1e0198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e4c2e640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e0198820 0x7fd1e0198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e4c2e640 1 -- 172.21.15.86:0/740660639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e0102240 msgr2=0x7fd1e019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e4c2e640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e0102240 0x7fd1e019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e4c2e640 1 -- 172.21.15.86:0/740660639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e01993b0 con 0x7fd1e0107310 2024-04-16T05:42:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e542f640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e0102240 0x7fd1e019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.559+0000 7fd1e5c30640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e0198820 0x7fd1e0198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.560+0000 7fd1e4c2e640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0107310 0x7fd1e019ebf0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000b450 tx=0x7fd1d000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.560+0000 7fd1d67fc640 1 -- 172.21.15.86:0/740660639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0015020 con 0x7fd1e0107310 2024-04-16T05:42:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.560+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e01996a0 con 0x7fd1e0107310 2024-04-16T05:42:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.560+0000 7fd1d67fc640 1 -- 172.21.15.86:0/740660639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d0011d70 con 0x7fd1e0107310 2024-04-16T05:42:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.560+0000 7fd1d67fc640 1 -- 172.21.15.86:0/740660639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000f340 con 0x7fd1e0107310 2024-04-16T05:42:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.560+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e01a5880 con 0x7fd1e0107310 2024-04-16T05:42:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.561+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e0103710 con 0x7fd1e0107310 2024-04-16T05:42:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.562+0000 7fd1d67fc640 1 -- 172.21.15.86:0/740660639 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd1d0018020 con 0x7fd1e0107310 2024-04-16T05:42:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.562+0000 7fd1d67fc640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1cc0a3d20 0x7fd1cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.562+0000 7fd1e542f640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1cc0a3d20 0x7fd1cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.562+0000 7fd1d67fc640 1 -- 172.21.15.86:0/740660639 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fd1d00d0670 con 0x7fd1e0107310 2024-04-16T05:42:59.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.563+0000 7fd1e542f640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1cc0a3d20 0x7fd1cc0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc02cfd0 tx=0x7fd1dc000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.567+0000 7fd1d67fc640 1 -- 172.21.15.86:0/740660639 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1d001f020 con 0x7fd1e0107310 2024-04-16T05:42:59.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.783+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1e010a540 con 0x7fd1e0107310 2024-04-16T05:42:59.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.784+0000 7fd1d67fc640 1 -- 172.21.15.86:0/740660639 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fd1e01996a0 con 0x7fd1e0107310 2024-04-16T05:42:59.783 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:42:59.783 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:42:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:42:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1cc0a3d20 msgr2=0x7fd1cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1cc0a3d20 0x7fd1cc0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc02cfd0 tx=0x7fd1dc000bc0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0107310 msgr2=0x7fd1e019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0107310 0x7fd1e019ebf0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000b450 tx=0x7fd1d000b910 comp rx=0 tx=0).stop 2024-04-16T05:42:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 shutdown_connections 2024-04-16T05:42:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e0198820 0x7fd1e0198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e0107310 0x7fd1e019ebf0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1cc0a3d20 0x7fd1cc0a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 --2- 172.21.15.86:0/740660639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e0102240 0x7fd1e019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 >> 172.21.15.86:0/740660639 conn(0x7fd1e00793c0 msgr2=0x7fd1e0100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 shutdown_connections 2024-04-16T05:42:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.786+0000 7fd1e7693640 1 -- 172.21.15.86:0/740660639 wait complete. 2024-04-16T05:42:59.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:42:59.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 -- 172.21.15.86:0/1931073435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 msgr2=0x7f8f4c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/1931073435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 0x7f8f4c101510 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009900 tx=0x7f8f4002bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:59.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 -- 172.21.15.86:0/1931073435 shutdown_connections 2024-04-16T05:42:59.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/1931073435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f4c106200 0x7f8f4c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/1931073435 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f4c101ae0 0x7f8f4c105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/1931073435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 0x7f8f4c101510 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 -- 172.21.15.86:0/1931073435 >> 172.21.15.86:0/1931073435 conn(0x7f8f4c0780b0 msgr2=0x7f8f4c0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 -- 172.21.15.86:0/1931073435 shutdown_connections 2024-04-16T05:42:59.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 -- 172.21.15.86:0/1931073435 wait complete. 2024-04-16T05:42:59.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 Processor -- start 2024-04-16T05:42:59.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 -- start start 2024-04-16T05:42:59.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.904+0000 7f8f53ed3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 0x7f8f4c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f53ed3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f4c106200 0x7f8f4c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f53ed3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f4c1986a0 0x7f8f4c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f53ed3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c111290 con 0x7f8f4c101130 2024-04-16T05:42:59.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f53ed3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c111110 con 0x7f8f4c1986a0 2024-04-16T05:42:59.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f53ed3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c111410 con 0x7f8f4c106200 2024-04-16T05:42:59.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f51c6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 0x7f8f4c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f51c6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 0x7f8f4c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41824/0 (socket says 172.21.15.86:41824) 2024-04-16T05:42:59.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f51c6f640 1 -- 172.21.15.86:0/2652135126 learned_addr learned my addr 172.21.15.86:0/2652135126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f52470640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f4c1986a0 0x7f8f4c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f5146e640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f4c106200 0x7f8f4c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f51c6f640 1 -- 172.21.15.86:0/2652135126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f4c106200 msgr2=0x7f8f4c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f4c106200 0x7f8f4c19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f51c6f640 1 -- 172.21.15.86:0/2652135126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f4c1986a0 msgr2=0x7f8f4c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f4c1986a0 0x7f8f4c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f51c6f640 1 -- 172.21.15.86:0/2652135126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c1992a0 con 0x7f8f4c101130 2024-04-16T05:42:59.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f52470640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f4c1986a0 0x7f8f4c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:59.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 0x7f8f4c19e5e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8f40006010 tx=0x7f8f4002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f3effd640 1 -- 172.21.15.86:0/2652135126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40041070 con 0x7f8f4c101130 2024-04-16T05:42:59.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f53ed3640 1 -- 172.21.15.86:0/2652135126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c199530 con 0x7f8f4c101130 2024-04-16T05:42:59.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f3effd640 1 -- 172.21.15.86:0/2652135126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f40004450 con 0x7f8f4c101130 2024-04-16T05:42:59.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f3effd640 1 -- 172.21.15.86:0/2652135126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40034440 con 0x7f8f4c101130 2024-04-16T05:42:59.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.905+0000 7f8f53ed3640 1 -- 172.21.15.86:0/2652135126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c1a5670 con 0x7f8f4c101130 2024-04-16T05:42:59.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.907+0000 7f8f3effd640 1 -- 172.21.15.86:0/2652135126 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8f40044070 con 0x7f8f4c101130 2024-04-16T05:42:59.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f56b99740 1 Processor -- start 2024-04-16T05:42:59.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f56b99740 1 -- start start 2024-04-16T05:42:59.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cd4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cdce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0cdf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f56b99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5583c0c390e0 con 0x5583c0cd4780 2024-04-16T05:42:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f56b99740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5583c0bd6520 con 0x5583c0cdd360 2024-04-16T05:42:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f56b99740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5583c0c350a0 con 0x5583c0cd3e60 2024-04-16T05:42:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f3effd640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8f380a3d20 0x7f8f380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f51c6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cdce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f51c6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cdce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41826/0 (socket says 172.21.15.86:41826) 2024-04-16T05:42:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f51c6f640 1 -- 172.21.15.86:0/1716125219 learned_addr learned my addr 172.21.15.86:0/1716125219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f52470640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8f380a3d20 0x7f8f380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.908+0000 7f8f52470640 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cd4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f5146e640 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0cdf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f3effd640 1 -- 172.21.15.86:0/2652135126 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8f400f2780 con 0x7f8f4c101130 2024-04-16T05:42:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f51c6f640 1 -- 172.21.15.86:0/1716125219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 msgr2=0x5583c0cd4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cd4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f51c6f640 1 -- 172.21.15.86:0/1716125219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 msgr2=0x5583c0cdf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0cdf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f51c6f640 1 -- 172.21.15.86:0/1716125219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c0bfe8d0 con 0x5583c0cd4780 2024-04-16T05:42:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f52470640 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cd4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cdce20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8f400bae30 tx=0x7f8f400be360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91ba3fe86eb5db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f52470640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8f380a3d20 0x7f8f380a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009370 tx=0x7f8f44009590 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f37fff640 1 -- 172.21.15.86:0/1716125219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4004b070 con 0x5583c0cd4780 2024-04-16T05:42:59.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f37fff640 1 -- 172.21.15.86:0/1716125219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f400beb60 con 0x5583c0cd4780 2024-04-16T05:42:59.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.909+0000 7f8f37fff640 1 -- 172.21.15.86:0/1716125219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f400baad0 con 0x5583c0cd4780 2024-04-16T05:42:59.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 -- 172.21.15.86:0/1716125219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 msgr2=0x5583c0cdce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cdce20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8f400bae30 tx=0x7f8f400be360 comp rx=0 tx=0).stop 2024-04-16T05:42:59.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 -- 172.21.15.86:0/1716125219 shutdown_connections 2024-04-16T05:42:59.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0cdf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cdce20 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 --2- 172.21.15.86:0/1716125219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cd4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 -- 172.21.15.86:0/1716125219 >> 172.21.15.86:0/1716125219 conn(0x5583c0cce590 msgr2=0x5583c0cce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 -- 172.21.15.86:0/1716125219 shutdown_connections 2024-04-16T05:42:59.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 -- 172.21.15.86:0/1716125219 wait complete. 2024-04-16T05:42:59.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 -- start start 2024-04-16T05:42:59.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.910+0000 7f8f56b99740 1 Processor -- start 2024-04-16T05:42:59.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cda3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cd4780 0x5583c0cda930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cdd360 0x5583c0cdae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f56b99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5583c0cdff10 con 0x5583c0cdd360 2024-04-16T05:42:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f56b99740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5583c0bf24d0 con 0x5583c0cd4780 2024-04-16T05:42:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f56b99740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5583c0ce0090 con 0x5583c0cd3e60 2024-04-16T05:42:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f56b99740 10 client.0 _ll_register_callbacks cb 0x5583c0bd72c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f52470640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cdd360 0x5583c0cdae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f52470640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cdd360 0x5583c0cdae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41836/0 (socket says 172.21.15.86:41836) 2024-04-16T05:42:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f52470640 1 -- 172.21.15.86:0/3260537987 learned_addr learned my addr 172.21.15.86:0/3260537987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cda3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f5146e640 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cd4780 0x5583c0cda930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f52470640 1 -- 172.21.15.86:0/3260537987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 msgr2=0x5583c0cda3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f52470640 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cda3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f52470640 1 -- 172.21.15.86:0/3260537987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cd4780 msgr2=0x5583c0cda930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f52470640 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cd4780 0x5583c0cda930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f52470640 1 -- 172.21.15.86:0/3260537987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c094b3f0 con 0x5583c0cdd360 2024-04-16T05:42:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cda3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f5146e640 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cd4780 0x5583c0cda930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f52470640 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cdd360 0x5583c0cdae70 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400fae0 tx=0x7f8f44004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e39f1902377b09 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f357fa640 1 -- 172.21.15.86:0/3260537987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44012de0 con 0x5583c0cdd360 2024-04-16T05:42:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f357fa640 1 -- 172.21.15.86:0/3260537987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8f44003970 con 0x5583c0cdd360 2024-04-16T05:42:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f56b99740 1 -- 172.21.15.86:0/3260537987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f400beb60 con 0x5583c0cdd360 2024-04-16T05:42:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.911+0000 7f8f357fa640 1 -- 172.21.15.86:0/3260537987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f44031050 con 0x5583c0cdd360 2024-04-16T05:42:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f56b99740 10 client.17892 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f56b99740 1 -- 172.21.15.86:0/3260537987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583c0a157d0 con 0x5583c0cdd360 2024-04-16T05:42:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f357fa640 1 -- 172.21.15.86:0/3260537987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f400beb60 con 0x5583c0cdd360 2024-04-16T05:42:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f357fa640 1 -- 172.21.15.86:0/3260537987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8f4401f430 con 0x5583c0cdd360 2024-04-16T05:42:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f56b99740 10 client.17892 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f56b99740 10 client.17892 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:42:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f56b99740 20 client.17892 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f56b99740 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5583c0ced160 0x5583c0cef580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f56b99740 4 client.17892 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f56b99740 1 -- 172.21.15.86:0/3260537987 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583c0c46b00 con 0x5583c0ced160 2024-04-16T05:42:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.912+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5583c0ced160 0x5583c0cef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.913+0000 7f8f357fa640 10 client.17892 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:42:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.913+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5583c0ced160 0x5583c0cef580 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=583098fedc5817c5 server_cookie=61b1c110332eab14 in_seq=0 out_seq=0 2024-04-16T05:42:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.913+0000 7f8f357fa640 1 -- 172.21.15.86:0/3260537987 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583c0c46b00 con 0x5583c0ced160 2024-04-16T05:42:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.913+0000 7f8f357fa640 10 client.17892 handle_command_reply: tid=0 2024-04-16T05:42:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.913+0000 7f8f56b99740 1 client.17892 shutdown 2024-04-16T05:42:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.913+0000 7f8f56b99740 1 -- 172.21.15.86:0/3260537987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cdd360 msgr2=0x5583c0cdae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.913+0000 7f8f56b99740 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cdd360 0x5583c0cdae70 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400fae0 tx=0x7f8f44004ab0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f56b99740 1 -- 172.21.15.86:0/3260537987 shutdown_connections 2024-04-16T05:42:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f56b99740 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5583c0ced160 0x5583c0cef580 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f56b99740 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cdd360 0x5583c0cdae70 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f56b99740 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cd4780 0x5583c0cda930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f56b99740 1 --2- 172.21.15.86:0/3260537987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cda3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f357fa640 0 client.17892 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:42:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f56b99740 1 -- 172.21.15.86:0/3260537987 >> 172.21.15.86:0/3260537987 conn(0x5583c0cce590 msgr2=0x5583c0cd02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f56b99740 1 -- 172.21.15.86:0/3260537987 shutdown_connections 2024-04-16T05:42:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f56b99740 1 -- 172.21.15.86:0/3260537987 wait complete. 2024-04-16T05:42:59.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.914+0000 7f8f56b99740 20 client.17892 trim_cache size 0 max 16384 2024-04-16T05:42:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f56b99740 1 Processor -- start 2024-04-16T05:42:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f56b99740 1 -- start start 2024-04-16T05:42:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cdcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cd9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0ce8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f56b99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1af570 con 0x5583c0cd4780 2024-04-16T05:42:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f56b99740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1b10b0 con 0x5583c0cdd360 2024-04-16T05:42:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f56b99740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1afc40 con 0x5583c0cd3e60 2024-04-16T05:42:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f5146e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cd9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f5146e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cd9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41844/0 (socket says 172.21.15.86:41844) 2024-04-16T05:42:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.927+0000 7f8f5146e640 1 -- 172.21.15.86:0/325209518 learned_addr learned my addr 172.21.15.86:0/325209518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f52470640 1 --2- 172.21.15.86:0/325209518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0ce8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/325209518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cdcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f5146e640 1 -- 172.21.15.86:0/325209518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 msgr2=0x5583c0cdcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f5146e640 1 --2- 172.21.15.86:0/325209518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cdcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f5146e640 1 -- 172.21.15.86:0/325209518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 msgr2=0x5583c0ce8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f5146e640 1 --2- 172.21.15.86:0/325209518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0ce8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f5146e640 1 -- 172.21.15.86:0/325209518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c0cc9c30 con 0x5583c0cd4780 2024-04-16T05:42:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f5146e640 1 --2- 172.21.15.86:0/325209518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cd9570 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800eb90 tx=0x7f8f480127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9a9d63613f312c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f377fe640 1 -- 172.21.15.86:0/325209518 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48005020 con 0x5583c0cd4780 2024-04-16T05:42:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f377fe640 1 -- 172.21.15.86:0/325209518 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f480047a0 con 0x5583c0cd4780 2024-04-16T05:42:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.928+0000 7f8f377fe640 1 -- 172.21.15.86:0/325209518 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48010800 con 0x5583c0cd4780 2024-04-16T05:42:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 -- 172.21.15.86:0/325209518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 msgr2=0x5583c0cd9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 --2- 172.21.15.86:0/325209518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cd9570 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800eb90 tx=0x7f8f480127c0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 -- 172.21.15.86:0/325209518 shutdown_connections 2024-04-16T05:42:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 --2- 172.21.15.86:0/325209518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0ce8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 --2- 172.21.15.86:0/325209518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd4780 0x5583c0cd9570 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 --2- 172.21.15.86:0/325209518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd3e60 0x5583c0cdcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 -- 172.21.15.86:0/325209518 >> 172.21.15.86:0/325209518 conn(0x5583c0cce590 msgr2=0x5583c0ccf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 -- 172.21.15.86:0/325209518 shutdown_connections 2024-04-16T05:42:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 -- 172.21.15.86:0/325209518 wait complete. 2024-04-16T05:42:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 -- start start 2024-04-16T05:42:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 Processor -- start 2024-04-16T05:42:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd3e60 0x5583c0cd7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd4780 0x5583c0cd7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0cd7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1b09e0 con 0x5583c0cd3e60 2024-04-16T05:42:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1b0310 con 0x5583c0cdd360 2024-04-16T05:42:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.929+0000 7f8f56b99740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1ad570 con 0x5583c0cd4780 2024-04-16T05:42:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f56b99740 10 client.0 _ll_register_callbacks cb 0x7f8f44014f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f51c6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd3e60 0x5583c0cd7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f51c6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd3e60 0x5583c0cd7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41848/0 (socket says 172.21.15.86:41848) 2024-04-16T05:42:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f51c6f640 1 -- 172.21.15.86:0/4192394647 learned_addr learned my addr 172.21.15.86:0/4192394647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:42:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f5146e640 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd4780 0x5583c0cd7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f52470640 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0cd7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f51c6f640 1 -- 172.21.15.86:0/4192394647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd4780 msgr2=0x5583c0cd7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd4780 0x5583c0cd7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f51c6f640 1 -- 172.21.15.86:0/4192394647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 msgr2=0x5583c0cd7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0cd7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f51c6f640 1 -- 172.21.15.86:0/4192394647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c0d52860 con 0x5583c0cd3e60 2024-04-16T05:42:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f51c6f640 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd3e60 0x5583c0cd7240 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8f400be3f0 tx=0x7f8f40003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97bd4393b68f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:42:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f357fa640 1 -- 172.21.15.86:0/4192394647 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4004b040 con 0x5583c0cd3e60 2024-04-16T05:42:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f357fa640 1 -- 172.21.15.86:0/4192394647 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8f400035f0 con 0x5583c0cd3e60 2024-04-16T05:42:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f56b99740 1 -- 172.21.15.86:0/4192394647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f480047a0 con 0x5583c0cd3e60 2024-04-16T05:42:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f357fa640 1 -- 172.21.15.86:0/4192394647 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f400bdfa0 con 0x5583c0cd3e60 2024-04-16T05:42:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f56b99740 10 client.17898 fetch_fsmap learned FSMap version 29 2024-04-16T05:42:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f56b99740 1 -- 172.21.15.86:0/4192394647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583c0d53b40 con 0x5583c0cd3e60 2024-04-16T05:42:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.930+0000 7f8f357fa640 1 -- 172.21.15.86:0/4192394647 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f480047a0 con 0x5583c0cd3e60 2024-04-16T05:42:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f357fa640 1 -- 172.21.15.86:0/4192394647 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8f400d7be0 con 0x5583c0cd3e60 2024-04-16T05:42:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f56b99740 10 client.17898 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:42:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f56b99740 10 client.17898 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:42:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f56b99740 20 client.17898 populate_metadata read hostname 'smithi086' 2024-04-16T05:42:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f56b99740 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5583c0d540d0 0x5583c0d564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f56b99740 4 client.17898 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f56b99740 1 -- 172.21.15.86:0/4192394647 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583c0c40970 con 0x5583c0d540d0 2024-04-16T05:42:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f5146e640 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5583c0d540d0 0x5583c0d564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f357fa640 10 client.17898 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:42:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f5146e640 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5583c0d540d0 0x5583c0d564f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96ecc2a7480e6146 server_cookie=7b7017880a732c2c in_seq=0 out_seq=0 2024-04-16T05:42:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f357fa640 1 -- 172.21.15.86:0/4192394647 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583c0c40970 con 0x5583c0d540d0 2024-04-16T05:42:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f357fa640 10 client.17898 handle_command_reply: tid=0 2024-04-16T05:42:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.931+0000 7f8f56b99740 1 client.17898 shutdown 2024-04-16T05:42:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 -- 172.21.15.86:0/4192394647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd3e60 msgr2=0x5583c0cd7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd3e60 0x5583c0cd7240 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8f400be3f0 tx=0x7f8f40003990 comp rx=0 tx=0).stop 2024-04-16T05:42:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 -- 172.21.15.86:0/4192394647 shutdown_connections 2024-04-16T05:42:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x5583c0d540d0 0x5583c0d564f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5583c0cdd360 0x5583c0cd7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583c0cd4780 0x5583c0cd7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f357fa640 0 client.17898 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:42:59.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 --2- 172.21.15.86:0/4192394647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583c0cd3e60 0x5583c0cd7240 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 -- 172.21.15.86:0/4192394647 >> 172.21.15.86:0/4192394647 conn(0x5583c0cce590 msgr2=0x5583c0cd0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 -- 172.21.15.86:0/4192394647 shutdown_connections 2024-04-16T05:42:59.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 1 -- 172.21.15.86:0/4192394647 wait complete. 2024-04-16T05:42:59.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f56b99740 20 client.17898 trim_cache size 0 max 16384 2024-04-16T05:42:59.942 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:42:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 -- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8f380a3d20 msgr2=0x7f8f380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8f380a3d20 0x7f8f380a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009370 tx=0x7f8f44009590 comp rx=0 tx=0).stop 2024-04-16T05:42:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 -- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 msgr2=0x7f8f4c19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 0x7f8f4c19e5e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8f40006010 tx=0x7f8f4002cc70 comp rx=0 tx=0).stop 2024-04-16T05:42:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 -- 172.21.15.86:0/2652135126 shutdown_connections 2024-04-16T05:42:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8f380a3d20 0x7f8f380a61e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f4c1986a0 0x7f8f4c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f4c106200 0x7f8f4c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 --2- 172.21.15.86:0/2652135126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4c101130 0x7f8f4c19e5e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 -- 172.21.15.86:0/2652135126 >> 172.21.15.86:0/2652135126 conn(0x7f8f4c0780b0 msgr2=0x7f8f4c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 -- 172.21.15.86:0/2652135126 shutdown_connections 2024-04-16T05:42:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:42:59.932+0000 7f8f53ed3640 1 -- 172.21.15.86:0/2652135126 wait complete. 2024-04-16T05:42:59.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:43:00.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.054+0000 7f51dd900640 1 -- 172.21.15.86:0/2954746688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d810c5b0 msgr2=0x7f51d810ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.054+0000 7f51dd900640 1 --2- 172.21.15.86:0/2954746688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d810c5b0 0x7f51d810ea50 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009900 tx=0x7f51d002bd60 comp rx=0 tx=0).stop 2024-04-16T05:43:00.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.054+0000 7f51dd900640 1 -- 172.21.15.86:0/2954746688 shutdown_connections 2024-04-16T05:43:00.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.054+0000 7f51dd900640 1 --2- 172.21.15.86:0/2954746688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d810c5b0 0x7f51d810ea50 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.054+0000 7f51dd900640 1 --2- 172.21.15.86:0/2954746688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d8103b90 0x7f51d810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.054+0000 7f51dd900640 1 --2- 172.21.15.86:0/2954746688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d8103270 0x7f51d8103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.054+0000 7f51dd900640 1 -- 172.21.15.86:0/2954746688 >> 172.21.15.86:0/2954746688 conn(0x7f51d80780b0 msgr2=0x7f51d80784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.054+0000 7f51dd900640 1 -- 172.21.15.86:0/2954746688 shutdown_connections 2024-04-16T05:43:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.054+0000 7f51dd900640 1 -- 172.21.15.86:0/2954746688 wait complete. 2024-04-16T05:43:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51dd900640 1 Processor -- start 2024-04-16T05:43:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51dd900640 1 -- start start 2024-04-16T05:43:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51dd900640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d8103270 0x7f51d819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51dd900640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d8103b90 0x7f51d819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51dd900640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d810c5b0 0x7f51d8198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51dd900640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81123c0 con 0x7f51d8103270 2024-04-16T05:43:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51dd900640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f51d8112240 con 0x7f51d8103b90 2024-04-16T05:43:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51dd900640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51d8112540 con 0x7f51d810c5b0 2024-04-16T05:43:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d8103270 0x7f51d819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d8103270 0x7f51d819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41868/0 (socket says 172.21.15.86:41868) 2024-04-16T05:43:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.055+0000 7f51d6ffd640 1 -- 172.21.15.86:0/3559057697 learned_addr learned my addr 172.21.15.86:0/3559057697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51d77fe640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d810c5b0 0x7f51d8198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51d67fc640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d8103b90 0x7f51d819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51d6ffd640 1 -- 172.21.15.86:0/3559057697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d810c5b0 msgr2=0x7f51d8198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d810c5b0 0x7f51d8198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51d6ffd640 1 -- 172.21.15.86:0/3559057697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d8103b90 msgr2=0x7f51d819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d8103b90 0x7f51d819e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51d6ffd640 1 -- 172.21.15.86:0/3559057697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d8198de0 con 0x7f51d8103270 2024-04-16T05:43:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d8103270 0x7f51d819e460 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f51c800cce0 tx=0x7f51c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51dc8fe640 1 -- 172.21.15.86:0/3559057697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c800f650 con 0x7f51d8103270 2024-04-16T05:43:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51dd900640 1 -- 172.21.15.86:0/3559057697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d81990d0 con 0x7f51d8103270 2024-04-16T05:43:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51dc8fe640 1 -- 172.21.15.86:0/3559057697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c800fcd0 con 0x7f51d8103270 2024-04-16T05:43:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51dc8fe640 1 -- 172.21.15.86:0/3559057697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8018c70 con 0x7f51d8103270 2024-04-16T05:43:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.056+0000 7f51dd900640 1 -- 172.21.15.86:0/3559057697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d81a5320 con 0x7f51d8103270 2024-04-16T05:43:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.058+0000 7f51dc8fe640 1 -- 172.21.15.86:0/3559057697 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f51c800f7f0 con 0x7f51d8103270 2024-04-16T05:43:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.058+0000 7f51e05c1740 1 Processor -- start 2024-04-16T05:43:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51e05c1740 1 -- start start 2024-04-16T05:43:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e38b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38b780 0x55c87e393e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e394360 0x55c87e396750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51e05c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c87e2f00e0 con 0x55c87e38ae60 2024-04-16T05:43:00.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51e05c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c87e28d520 con 0x55c87e394360 2024-04-16T05:43:00.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51e05c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c87e2ec0a0 con 0x55c87e38b780 2024-04-16T05:43:00.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51dc8fe640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51c00a3d20 0x7f51c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e38b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e38b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41874/0 (socket says 172.21.15.86:41874) 2024-04-16T05:43:00.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d77fe640 1 -- 172.21.15.86:0/207282599 learned_addr learned my addr 172.21.15.86:0/207282599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d67fc640 1 --2- 172.21.15.86:0/207282599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e394360 0x55c87e396750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d77fe640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51c00a3d20 0x7f51c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51dc8fe640 1 -- 172.21.15.86:0/3559057697 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f51c80d1170 con 0x7f51d8103270 2024-04-16T05:43:00.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/207282599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38b780 0x55c87e393e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d77fe640 1 -- 172.21.15.86:0/207282599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38b780 msgr2=0x55c87e393e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d77fe640 1 --2- 172.21.15.86:0/207282599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38b780 0x55c87e393e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d77fe640 1 -- 172.21.15.86:0/207282599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e394360 msgr2=0x55c87e396750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d77fe640 1 --2- 172.21.15.86:0/207282599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e394360 0x55c87e396750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.059+0000 7f51d77fe640 1 -- 172.21.15.86:0/207282599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c87e2b58d0 con 0x55c87e38ae60 2024-04-16T05:43:00.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51d77fe640 1 --2- 172.21.15.86:0/207282599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e38b240 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f51d00096c0 tx=0x7f51d002de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a31d20a71269a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51bdffb640 1 -- 172.21.15.86:0/207282599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d002da20 con 0x55c87e38ae60 2024-04-16T05:43:00.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51bdffb640 1 -- 172.21.15.86:0/207282599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d0005410 con 0x55c87e38ae60 2024-04-16T05:43:00.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51bdffb640 1 -- 172.21.15.86:0/207282599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0003830 con 0x55c87e38ae60 2024-04-16T05:43:00.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51d77fe640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51c00a3d20 0x7f51c00a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f51d004a000 tx=0x7f51d004b000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51e05c1740 1 -- 172.21.15.86:0/207282599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 msgr2=0x55c87e38b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51e05c1740 1 --2- 172.21.15.86:0/207282599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e38b240 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f51d00096c0 tx=0x7f51d002de20 comp rx=0 tx=0).stop 2024-04-16T05:43:00.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51e05c1740 1 -- 172.21.15.86:0/207282599 shutdown_connections 2024-04-16T05:43:00.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51e05c1740 1 --2- 172.21.15.86:0/207282599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e394360 0x55c87e396750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51e05c1740 1 --2- 172.21.15.86:0/207282599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38b780 0x55c87e393e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51e05c1740 1 --2- 172.21.15.86:0/207282599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e38b240 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51e05c1740 1 -- 172.21.15.86:0/207282599 >> 172.21.15.86:0/207282599 conn(0x55c87e385590 msgr2=0x55c87e385970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51e05c1740 1 -- 172.21.15.86:0/207282599 shutdown_connections 2024-04-16T05:43:00.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.060+0000 7f51e05c1740 1 -- 172.21.15.86:0/207282599 wait complete. 2024-04-16T05:43:00.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51e05c1740 1 -- start start 2024-04-16T05:43:00.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51e05c1740 1 Processor -- start 2024-04-16T05:43:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38ae60 0x55c87e391460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 0x55c87e3919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e391ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51e05c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c87e396fa0 con 0x55c87e394360 2024-04-16T05:43:00.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51e05c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c87e2a94d0 con 0x55c87e38b780 2024-04-16T05:43:00.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51e05c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c87e397120 con 0x55c87e38ae60 2024-04-16T05:43:00.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51e05c1740 10 client.0 _ll_register_callbacks cb 0x55c87e28e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:00.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e391ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e391ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41876/0 (socket says 172.21.15.86:41876) 2024-04-16T05:43:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.061+0000 7f51d77fe640 1 -- 172.21.15.86:0/3203204287 learned_addr learned my addr 172.21.15.86:0/3203204287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51d67fc640 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 0x55c87e3919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38ae60 0x55c87e391460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51d77fe640 1 -- 172.21.15.86:0/3203204287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38ae60 msgr2=0x55c87e391460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51d77fe640 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38ae60 0x55c87e391460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51d77fe640 1 -- 172.21.15.86:0/3203204287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 msgr2=0x55c87e3919a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51d77fe640 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 0x55c87e3919a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51d77fe640 1 -- 172.21.15.86:0/3203204287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c87e0023f0 con 0x55c87e394360 2024-04-16T05:43:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38ae60 0x55c87e391460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51d77fe640 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e391ee0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f51d0006010 tx=0x7f51d0004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6915d68f0d1ce65f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3203204287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0059030 con 0x55c87e394360 2024-04-16T05:43:00.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3203204287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f51d00035f0 con 0x55c87e394360 2024-04-16T05:43:00.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51e05c1740 1 -- 172.21.15.86:0/3203204287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51d0005410 con 0x55c87e394360 2024-04-16T05:43:00.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3203204287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f51d002da20 con 0x55c87e394360 2024-04-16T05:43:00.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51e05c1740 10 client.17910 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:00.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51e05c1740 1 -- 172.21.15.86:0/3203204287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c87e0cc7d0 con 0x55c87e394360 2024-04-16T05:43:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3203204287 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d0005410 con 0x55c87e394360 2024-04-16T05:43:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3203204287 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f51d0043450 con 0x55c87e394360 2024-04-16T05:43:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.062+0000 7f51e05c1740 10 client.17910 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.063+0000 7f51e05c1740 10 client.17910 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:43:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.063+0000 7f51e05c1740 20 client.17910 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.063+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c87e3a41c0 0x55c87e3a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.063+0000 7f51e05c1740 4 client.17910 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.063+0000 7f51e05c1740 1 -- 172.21.15.86:0/3203204287 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c87e2fdb00 con 0x55c87e3a41c0 2024-04-16T05:43:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.063+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c87e3a41c0 0x55c87e3a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.063+0000 7f51b6ffd640 10 client.17910 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:43:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.063+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c87e3a41c0 0x55c87e3a65b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=abd51cdab960e41b server_cookie=cef47c8dee6c0fd2 in_seq=0 out_seq=0 2024-04-16T05:43:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3203204287 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c87e2fdb00 con 0x55c87e3a41c0 2024-04-16T05:43:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51b6ffd640 10 client.17910 handle_command_reply: tid=0 2024-04-16T05:43:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 client.17910 shutdown 2024-04-16T05:43:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 -- 172.21.15.86:0/3203204287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 msgr2=0x55c87e391ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e391ee0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f51d0006010 tx=0x7f51d0004db0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 -- 172.21.15.86:0/3203204287 shutdown_connections 2024-04-16T05:43:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e391ee0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 0x55c87e3919a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c87e3a41c0 0x55c87e3a65b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3203204287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38ae60 0x55c87e391460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51b6ffd640 0 client.17910 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:43:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 -- 172.21.15.86:0/3203204287 >> 172.21.15.86:0/3203204287 conn(0x55c87e385590 msgr2=0x55c87e3872e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 -- 172.21.15.86:0/3203204287 shutdown_connections 2024-04-16T05:43:00.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.064+0000 7f51e05c1740 1 -- 172.21.15.86:0/3203204287 wait complete. 2024-04-16T05:43:00.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.065+0000 7f51e05c1740 20 client.17910 trim_cache size 0 max 16384 2024-04-16T05:43:00.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51e05c1740 1 Processor -- start 2024-04-16T05:43:00.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51e05c1740 1 -- start start 2024-04-16T05:43:00.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38ae60 0x55c87e39ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38b780 0x55c87e391b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e3abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51e05c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81af4e0 con 0x55c87e394360 2024-04-16T05:43:00.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51e05c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81b1020 con 0x55c87e38ae60 2024-04-16T05:43:00.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51e05c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81afbb0 con 0x55c87e38b780 2024-04-16T05:43:00.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e3abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e3abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41880/0 (socket says 172.21.15.86:41880) 2024-04-16T05:43:00.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51d77fe640 1 -- 172.21.15.86:0/3820574122 learned_addr learned my addr 172.21.15.86:0/3820574122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51d77fe640 1 -- 172.21.15.86:0/3820574122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38b780 msgr2=0x55c87e391b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51d77fe640 1 --2- 172.21.15.86:0/3820574122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38b780 0x55c87e391b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51d77fe640 1 -- 172.21.15.86:0/3820574122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38ae60 msgr2=0x55c87e39ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51d77fe640 1 --2- 172.21.15.86:0/3820574122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38ae60 0x55c87e39ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51d77fe640 1 -- 172.21.15.86:0/3820574122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c87e38d030 con 0x55c87e394360 2024-04-16T05:43:00.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51d77fe640 1 --2- 172.21.15.86:0/3820574122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e3abb50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f51d002c2a0 tx=0x7f51d0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa32f037e3a9eaf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.078+0000 7f51bd7fa640 1 -- 172.21.15.86:0/3820574122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0059070 con 0x55c87e394360 2024-04-16T05:43:00.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51bd7fa640 1 -- 172.21.15.86:0/3820574122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d0005410 con 0x55c87e394360 2024-04-16T05:43:00.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51bd7fa640 1 -- 172.21.15.86:0/3820574122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0003b40 con 0x55c87e394360 2024-04-16T05:43:00.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 -- 172.21.15.86:0/3820574122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 msgr2=0x55c87e3abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3820574122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e3abb50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f51d002c2a0 tx=0x7f51d0005a20 comp rx=0 tx=0).stop 2024-04-16T05:43:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 -- 172.21.15.86:0/3820574122 shutdown_connections 2024-04-16T05:43:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3820574122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e394360 0x55c87e3abb50 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3820574122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e38b780 0x55c87e391b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3820574122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38ae60 0x55c87e39ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 -- 172.21.15.86:0/3820574122 >> 172.21.15.86:0/3820574122 conn(0x55c87e385590 msgr2=0x55c87e391420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 -- 172.21.15.86:0/3820574122 shutdown_connections 2024-04-16T05:43:00.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 -- 172.21.15.86:0/3820574122 wait complete. 2024-04-16T05:43:00.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.079+0000 7f51e05c1740 1 -- start start 2024-04-16T05:43:00.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51e05c1740 1 Processor -- start 2024-04-16T05:43:00.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e397e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 0x55c87e3983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51e05c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e394360 0x55c87e398900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51e05c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81b0950 con 0x55c87e38ae60 2024-04-16T05:43:00.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51e05c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81b0280 con 0x55c87e38b780 2024-04-16T05:43:00.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51e05c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81ad4e0 con 0x55c87e394360 2024-04-16T05:43:00.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51e05c1740 10 client.0 _ll_register_callbacks cb 0x7f51d0047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:00.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e397e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e397e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41886/0 (socket says 172.21.15.86:41886) 2024-04-16T05:43:00.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d6ffd640 1 -- 172.21.15.86:0/3386567347 learned_addr learned my addr 172.21.15.86:0/3386567347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d6ffd640 1 -- 172.21.15.86:0/3386567347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e394360 msgr2=0x55c87e398900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e394360 0x55c87e398900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d6ffd640 1 -- 172.21.15.86:0/3386567347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 msgr2=0x55c87e3983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d67fc640 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 0x55c87e3983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 0x55c87e3983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d6ffd640 1 -- 172.21.15.86:0/3386567347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c87e4093c0 con 0x55c87e38ae60 2024-04-16T05:43:00.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51d6ffd640 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e397e80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f51c800afd0 tx=0x7f51c809c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ead8d27ad81108 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.080+0000 7f51b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:00.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3386567347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c809cd40 con 0x55c87e38ae60 2024-04-16T05:43:00.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3386567347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f51c80993b0 con 0x55c87e38ae60 2024-04-16T05:43:00.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51e05c1740 1 -- 172.21.15.86:0/3386567347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51d0005410 con 0x55c87e38ae60 2024-04-16T05:43:00.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3386567347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f51c80a10f0 con 0x55c87e38ae60 2024-04-16T05:43:00.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51e05c1740 10 client.17916 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:00.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51e05c1740 1 -- 172.21.15.86:0/3386567347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c87e40a700 con 0x55c87e38ae60 2024-04-16T05:43:00.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3386567347 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d0005410 con 0x55c87e38ae60 2024-04-16T05:43:00.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3386567347 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f51c809cfa0 con 0x55c87e38ae60 2024-04-16T05:43:00.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51e05c1740 10 client.17916 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:00.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51e05c1740 10 client.17916 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:43:00.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51e05c1740 20 client.17916 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:00.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c87e40ad80 0x55c87e40d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51e05c1740 4 client.17916 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:00.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.081+0000 7f51e05c1740 1 -- 172.21.15.86:0/3386567347 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c87e2f7970 con 0x55c87e40ad80 2024-04-16T05:43:00.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51d67fc640 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c87e40ad80 0x55c87e40d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51d67fc640 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c87e40ad80 0x55c87e40d1a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f6a1ea8e25db9ba server_cookie=fa65b3046c04dfc8 in_seq=0 out_seq=0 2024-04-16T05:43:00.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51b6ffd640 10 client.17916 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:43:00.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51b6ffd640 1 -- 172.21.15.86:0/3386567347 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c87e2f7970 con 0x55c87e40ad80 2024-04-16T05:43:00.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51b6ffd640 10 client.17916 handle_command_reply: tid=0 2024-04-16T05:43:00.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51e05c1740 1 client.17916 shutdown 2024-04-16T05:43:00.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51e05c1740 1 -- 172.21.15.86:0/3386567347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 msgr2=0x55c87e397e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e397e80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f51c800afd0 tx=0x7f51c809c940 comp rx=0 tx=0).stop 2024-04-16T05:43:00.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51e05c1740 1 -- 172.21.15.86:0/3386567347 shutdown_connections 2024-04-16T05:43:00.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c87e40ad80 0x55c87e40d1a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.082+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c87e394360 0x55c87e398900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c87e38b780 0x55c87e3983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51b6ffd640 0 client.17916 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:43:00.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51e05c1740 1 --2- 172.21.15.86:0/3386567347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c87e38ae60 0x55c87e397e80 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51e05c1740 1 -- 172.21.15.86:0/3386567347 >> 172.21.15.86:0/3386567347 conn(0x55c87e385590 msgr2=0x55c87e391390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51e05c1740 1 -- 172.21.15.86:0/3386567347 shutdown_connections 2024-04-16T05:43:00.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51e05c1740 1 -- 172.21.15.86:0/3386567347 wait complete. 2024-04-16T05:43:00.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51e05c1740 20 client.17916 trim_cache size 0 max 16384 2024-04-16T05:43:00.126 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:43:00.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 -- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51c00a3d20 msgr2=0x7f51c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51c00a3d20 0x7f51c00a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f51d004a000 tx=0x7f51d004b000 comp rx=0 tx=0).stop 2024-04-16T05:43:00.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 -- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d8103270 msgr2=0x7f51d819e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d8103270 0x7f51d819e460 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f51c800cce0 tx=0x7f51c800a620 comp rx=0 tx=0).stop 2024-04-16T05:43:00.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 -- 172.21.15.86:0/3559057697 shutdown_connections 2024-04-16T05:43:00.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d810c5b0 0x7f51d8198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51c00a3d20 0x7f51c00a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d8103b90 0x7f51d819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 --2- 172.21.15.86:0/3559057697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d8103270 0x7f51d819e460 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 -- 172.21.15.86:0/3559057697 >> 172.21.15.86:0/3559057697 conn(0x7f51d80780b0 msgr2=0x7f51d80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 -- 172.21.15.86:0/3559057697 shutdown_connections 2024-04-16T05:43:00.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.083+0000 7f51dd900640 1 -- 172.21.15.86:0/3559057697 wait complete. 2024-04-16T05:43:00.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:43:00.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/2362556218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 msgr2=0x7f8b541008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/2362556218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 0x7f8b541008d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009930 tx=0x7f8b4402bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:00.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/2362556218 shutdown_connections 2024-04-16T05:43:00.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/2362556218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b54100e10 0x7f8b5410eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/2362556218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 0x7f8b541008d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/2362556218 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b54108080 0x7f8b54108460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/2362556218 >> 172.21.15.86:0/2362556218 conn(0x7f8b54077410 msgr2=0x7f8b540fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/2362556218 shutdown_connections 2024-04-16T05:43:00.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/2362556218 wait complete. 2024-04-16T05:43:00.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.238+0000 7f8b5b5c8640 1 Processor -- start 2024-04-16T05:43:00.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b5b5c8640 1 -- start start 2024-04-16T05:43:00.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b5b5c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 0x7f8b5419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b5b5c8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b54100e10 0x7f8b5419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b5b5c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b54108080 0x7f8b54198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b5b5c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b541123a0 con 0x7f8b54100490 2024-04-16T05:43:00.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b5b5c8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b54112220 con 0x7f8b54100e10 2024-04-16T05:43:00.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b5b5c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b54112520 con 0x7f8b54108080 2024-04-16T05:43:00.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b59364640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 0x7f8b5419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b59364640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 0x7f8b5419e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41910/0 (socket says 172.21.15.86:41910) 2024-04-16T05:43:00.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b59364640 1 -- 172.21.15.86:0/3853785081 learned_addr learned my addr 172.21.15.86:0/3853785081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b58b63640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b54100e10 0x7f8b5419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b59364640 1 -- 172.21.15.86:0/3853785081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b54108080 msgr2=0x7f8b54198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b59364640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b54108080 0x7f8b54198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b59364640 1 -- 172.21.15.86:0/3853785081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b54100e10 msgr2=0x7f8b5419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b59364640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b54100e10 0x7f8b5419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b59364640 1 -- 172.21.15.86:0/3853785081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b54199000 con 0x7f8b54100490 2024-04-16T05:43:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b58b63640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b54100e10 0x7f8b5419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b59364640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 0x7f8b5419e6c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000c890 tx=0x7f8b5000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b4a7fc640 1 -- 172.21.15.86:0/3853785081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50004220 con 0x7f8b54100490 2024-04-16T05:43:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b4a7fc640 1 -- 172.21.15.86:0/3853785081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b500043c0 con 0x7f8b54100490 2024-04-16T05:43:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.239+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/3853785081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b541992f0 con 0x7f8b54100490 2024-04-16T05:43:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.240+0000 7f8b4a7fc640 1 -- 172.21.15.86:0/3853785081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b500049e0 con 0x7f8b54100490 2024-04-16T05:43:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.240+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/3853785081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b541a5510 con 0x7f8b54100490 2024-04-16T05:43:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b4a7fc640 1 -- 172.21.15.86:0/3853785081 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b500079b0 con 0x7f8b54100490 2024-04-16T05:43:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b5e1b4740 1 Processor -- start 2024-04-16T05:43:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b5e1b4740 1 -- start start 2024-04-16T05:43:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc1790240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc1798e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc179b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564bc16f50e0 con 0x564bc178fe60 2024-04-16T05:43:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564bc1692520 con 0x564bc1790780 2024-04-16T05:43:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564bc16f10a0 con 0x564bc1799360 2024-04-16T05:43:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b4a7fc640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b400a3d20 0x7f8b400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b59b65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc1790240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b59b65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc1790240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41920/0 (socket says 172.21.15.86:41920) 2024-04-16T05:43:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.242+0000 7f8b59b65640 1 -- 172.21.15.86:0/2232207287 learned_addr learned my addr 172.21.15.86:0/2232207287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b4a7fc640 1 -- 172.21.15.86:0/3853785081 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8b500d0c80 con 0x7f8b54100490 2024-04-16T05:43:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59b65640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b400a3d20 0x7f8b400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b58b63640 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc179b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59364640 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc1798e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59b65640 1 -- 172.21.15.86:0/2232207287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 msgr2=0x564bc179b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59b65640 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc179b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59b65640 1 -- 172.21.15.86:0/2232207287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 msgr2=0x564bc1798e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59b65640 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc1798e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59b65640 1 -- 172.21.15.86:0/2232207287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc16ba8d0 con 0x564bc178fe60 2024-04-16T05:43:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59364640 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc1798e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59b65640 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc1790240 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c015900 tx=0x7f8b4c015dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c857313207642c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b3f7fe640 1 -- 172.21.15.86:0/2232207287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c022070 con 0x564bc178fe60 2024-04-16T05:43:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b3f7fe640 1 -- 172.21.15.86:0/2232207287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c015270 con 0x564bc178fe60 2024-04-16T05:43:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b3f7fe640 1 -- 172.21.15.86:0/2232207287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c014670 con 0x564bc178fe60 2024-04-16T05:43:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.243+0000 7f8b59b65640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b400a3d20 0x7f8b400a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c01f000 tx=0x7f8b4c01b000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2232207287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 msgr2=0x564bc1790240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc1790240 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c015900 tx=0x7f8b4c015dc0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2232207287 shutdown_connections 2024-04-16T05:43:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc179b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc1798e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2232207287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc1790240 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2232207287 >> 172.21.15.86:0/2232207287 conn(0x564bc178a590 msgr2=0x564bc178a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2232207287 shutdown_connections 2024-04-16T05:43:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2232207287 wait complete. 2024-04-16T05:43:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 -- start start 2024-04-16T05:43:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.244+0000 7f8b5e1b4740 1 Processor -- start 2024-04-16T05:43:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc178fe60 0x564bc17963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1790780 0x564bc1796920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc1796e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564bc179bfa0 con 0x564bc1790780 2024-04-16T05:43:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564bc16ae4d0 con 0x564bc178fe60 2024-04-16T05:43:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564bc179c120 con 0x564bc1799360 2024-04-16T05:43:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b5e1b4740 10 client.0 _ll_register_callbacks cb 0x564bc16932c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b58b63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1790780 0x564bc1796920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b58b63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1790780 0x564bc1796920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41928/0 (socket says 172.21.15.86:41928) 2024-04-16T05:43:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b58b63640 1 -- 172.21.15.86:0/2170029542 learned_addr learned my addr 172.21.15.86:0/2170029542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b59364640 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc178fe60 0x564bc17963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b59b65640 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc1796e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b58b63640 1 -- 172.21.15.86:0/2170029542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 msgr2=0x564bc1796e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b58b63640 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc1796e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b58b63640 1 -- 172.21.15.86:0/2170029542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc178fe60 msgr2=0x564bc17963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b58b63640 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc178fe60 0x564bc17963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b58b63640 1 -- 172.21.15.86:0/2170029542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc14073f0 con 0x564bc1790780 2024-04-16T05:43:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b59364640 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc178fe60 0x564bc17963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b59b65640 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc1796e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b58b63640 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1790780 0x564bc1796920 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8b44036040 tx=0x7f8b44004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ee0dfd09fc4d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.245+0000 7f8b3cff9640 1 -- 172.21.15.86:0/2170029542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44005230 con 0x564bc1790780 2024-04-16T05:43:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b3cff9640 1 -- 172.21.15.86:0/2170029542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8b440053d0 con 0x564bc1790780 2024-04-16T05:43:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2170029542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b4c015270 con 0x564bc1790780 2024-04-16T05:43:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b3cff9640 1 -- 172.21.15.86:0/2170029542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b4404e050 con 0x564bc1790780 2024-04-16T05:43:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b5e1b4740 10 client.17928 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2170029542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bc14d17d0 con 0x564bc1790780 2024-04-16T05:43:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b3cff9640 1 -- 172.21.15.86:0/2170029542 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c015270 con 0x564bc1790780 2024-04-16T05:43:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b3cff9640 1 -- 172.21.15.86:0/2170029542 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b44003850 con 0x564bc1790780 2024-04-16T05:43:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b5e1b4740 10 client.17928 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b5e1b4740 10 client.17928 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:43:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b5e1b4740 20 client.17928 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564bc17a91c0 0x564bc17ab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b5e1b4740 4 client.17928 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2170029542 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bc1702b00 con 0x564bc17a91c0 2024-04-16T05:43:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.246+0000 7f8b59364640 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564bc17a91c0 0x564bc17ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.247+0000 7f8b3cff9640 10 client.17928 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:43:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.247+0000 7f8b59364640 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564bc17a91c0 0x564bc17ab620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83a2c09c559dd3a9 server_cookie=d60a386199d19cb9 in_seq=0 out_seq=0 2024-04-16T05:43:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.247+0000 7f8b3cff9640 1 -- 172.21.15.86:0/2170029542 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bc1702b00 con 0x564bc17a91c0 2024-04-16T05:43:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.247+0000 7f8b3cff9640 10 client.17928 handle_command_reply: tid=0 2024-04-16T05:43:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 client.17928 shutdown 2024-04-16T05:43:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2170029542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1790780 msgr2=0x564bc1796920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1790780 0x564bc1796920 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8b44036040 tx=0x7f8b44004040 comp rx=0 tx=0).stop 2024-04-16T05:43:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2170029542 shutdown_connections 2024-04-16T05:43:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc1796e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564bc17a91c0 0x564bc17ab620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1790780 0x564bc1796920 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b3cff9640 0 client.17928 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:43:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/2170029542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc178fe60 0x564bc17963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2170029542 >> 172.21.15.86:0/2170029542 conn(0x564bc178a590 msgr2=0x564bc178c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2170029542 shutdown_connections 2024-04-16T05:43:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/2170029542 wait complete. 2024-04-16T05:43:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.248+0000 7f8b5e1b4740 20 client.17928 trim_cache size 0 max 16384 2024-04-16T05:43:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b5e1b4740 1 Processor -- start 2024-04-16T05:43:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b5e1b4740 1 -- start start 2024-04-16T05:43:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc178fe60 0x564bc1798b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc17a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1799360 0x564bc17895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b541af520 con 0x564bc1799360 2024-04-16T05:43:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b541b1060 con 0x564bc1790780 2024-04-16T05:43:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b541afbf0 con 0x564bc178fe60 2024-04-16T05:43:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b59b65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1799360 0x564bc17895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b59b65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1799360 0x564bc17895b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41934/0 (socket says 172.21.15.86:41934) 2024-04-16T05:43:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.261+0000 7f8b59b65640 1 -- 172.21.15.86:0/3746155839 learned_addr learned my addr 172.21.15.86:0/3746155839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b59364640 1 --2- 172.21.15.86:0/3746155839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc178fe60 0x564bc1798b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b59b65640 1 -- 172.21.15.86:0/3746155839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc178fe60 msgr2=0x564bc1798b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b59b65640 1 --2- 172.21.15.86:0/3746155839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc178fe60 0x564bc1798b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b59b65640 1 -- 172.21.15.86:0/3746155839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 msgr2=0x564bc17a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b59b65640 1 --2- 172.21.15.86:0/3746155839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc17a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b59b65640 1 -- 172.21.15.86:0/3746155839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc1794930 con 0x564bc1799360 2024-04-16T05:43:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b59364640 1 --2- 172.21.15.86:0/3746155839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc178fe60 0x564bc1798b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b59b65640 1 --2- 172.21.15.86:0/3746155839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1799360 0x564bc17895b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c012760 tx=0x7f8b4c005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2f163f73ff4a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b3effd640 1 -- 172.21.15.86:0/3746155839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c022050 con 0x564bc1799360 2024-04-16T05:43:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b3effd640 1 -- 172.21.15.86:0/3746155839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c02a070 con 0x564bc1799360 2024-04-16T05:43:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.262+0000 7f8b3effd640 1 -- 172.21.15.86:0/3746155839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c005030 con 0x564bc1799360 2024-04-16T05:43:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/3746155839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1799360 msgr2=0x564bc17895b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/3746155839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1799360 0x564bc17895b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c012760 tx=0x7f8b4c005b90 comp rx=0 tx=0).stop 2024-04-16T05:43:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/3746155839 shutdown_connections 2024-04-16T05:43:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/3746155839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc1799360 0x564bc17895b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/3746155839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc17a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/3746155839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc178fe60 0x564bc1798b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/3746155839 >> 172.21.15.86:0/3746155839 conn(0x564bc178a590 msgr2=0x564bc1788ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/3746155839 shutdown_connections 2024-04-16T05:43:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/3746155839 wait complete. 2024-04-16T05:43:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 -- start start 2024-04-16T05:43:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 Processor -- start 2024-04-16T05:43:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc17a1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc17a1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc179e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b541b0990 con 0x564bc178fe60 2024-04-16T05:43:00.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b541b02c0 con 0x564bc1790780 2024-04-16T05:43:00.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.263+0000 7f8b5e1b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b541ad520 con 0x564bc1799360 2024-04-16T05:43:00.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b5e1b4740 10 client.0 _ll_register_callbacks cb 0x7f8b44009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:00.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59364640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc17a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59364640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc17a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41946/0 (socket says 172.21.15.86:41946) 2024-04-16T05:43:00.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59364640 1 -- 172.21.15.86:0/829616287 learned_addr learned my addr 172.21.15.86:0/829616287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59b65640 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc179e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59364640 1 -- 172.21.15.86:0/829616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 msgr2=0x564bc179e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59364640 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc179e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59364640 1 -- 172.21.15.86:0/829616287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 msgr2=0x564bc17a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59364640 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc17a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59364640 1 -- 172.21.15.86:0/829616287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc179eb10 con 0x564bc178fe60 2024-04-16T05:43:00.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59b65640 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc179e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:00.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b59364640 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc17a1450 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8b50098cb0 tx=0x7f8b5009c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bdf658b5fbd64d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:00.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b3cff9640 1 -- 172.21.15.86:0/829616287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5001d040 con 0x564bc178fe60 2024-04-16T05:43:00.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b3cff9640 1 -- 172.21.15.86:0/829616287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8b50098ec0 con 0x564bc178fe60 2024-04-16T05:43:00.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/829616287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b4c02a070 con 0x564bc178fe60 2024-04-16T05:43:00.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b3cff9640 1 -- 172.21.15.86:0/829616287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b5009c850 con 0x564bc178fe60 2024-04-16T05:43:00.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b5e1b4740 10 client.17934 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:00.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/829616287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bc1810130 con 0x564bc178fe60 2024-04-16T05:43:00.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b3cff9640 1 -- 172.21.15.86:0/829616287 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c02a070 con 0x564bc178fe60 2024-04-16T05:43:00.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.264+0000 7f8b3cff9640 1 -- 172.21.15.86:0/829616287 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b500b2070 con 0x564bc178fe60 2024-04-16T05:43:00.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b5e1b4740 10 client.17934 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:00.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b5e1b4740 10 client.17934 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:43:00.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b5e1b4740 20 client.17934 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:00.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564bc1810830 0x564bc1812c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b5e1b4740 4 client.17934 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:00.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/829616287 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bc16fc970 con 0x564bc1810830 2024-04-16T05:43:00.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b58b63640 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564bc1810830 0x564bc1812c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b3cff9640 10 client.17934 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:43:00.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b58b63640 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564bc1810830 0x564bc1812c60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=888c3101359c04e5 server_cookie=7728fa4669ec2e5a in_seq=0 out_seq=0 2024-04-16T05:43:00.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b3cff9640 1 -- 172.21.15.86:0/829616287 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bc16fc970 con 0x564bc1810830 2024-04-16T05:43:00.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b3cff9640 10 client.17934 handle_command_reply: tid=0 2024-04-16T05:43:00.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b5e1b4740 1 client.17934 shutdown 2024-04-16T05:43:00.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/829616287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 msgr2=0x564bc17a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.265+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc17a1450 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8b50098cb0 tx=0x7f8b5009c450 comp rx=0 tx=0).stop 2024-04-16T05:43:00.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/829616287 shutdown_connections 2024-04-16T05:43:00.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564bc1799360 0x564bc179e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564bc1790780 0x564bc17a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x564bc1810830 0x564bc1812c60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5e1b4740 1 --2- 172.21.15.86:0/829616287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bc178fe60 0x564bc17a1450 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b3cff9640 0 client.17934 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:43:00.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/829616287 >> 172.21.15.86:0/829616287 conn(0x564bc178a590 msgr2=0x564bc1789730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/829616287 shutdown_connections 2024-04-16T05:43:00.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5e1b4740 1 -- 172.21.15.86:0/829616287 wait complete. 2024-04-16T05:43:00.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5e1b4740 20 client.17934 trim_cache size 0 max 16384 2024-04-16T05:43:00.276 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:43:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b400a3d20 msgr2=0x7f8b400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b400a3d20 0x7f8b400a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c01f000 tx=0x7f8b4c01b000 comp rx=0 tx=0).stop 2024-04-16T05:43:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 msgr2=0x7f8b5419e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 0x7f8b5419e6c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000c890 tx=0x7f8b5000cd50 comp rx=0 tx=0).stop 2024-04-16T05:43:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/3853785081 shutdown_connections 2024-04-16T05:43:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b54108080 0x7f8b54198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b400a3d20 0x7f8b400a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b54100e10 0x7f8b5419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 --2- 172.21.15.86:0/3853785081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b54100490 0x7f8b5419e6c0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/3853785081 >> 172.21.15.86:0/3853785081 conn(0x7f8b54077410 msgr2=0x7f8b540fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/3853785081 shutdown_connections 2024-04-16T05:43:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.266+0000 7f8b5b5c8640 1 -- 172.21.15.86:0/3853785081 wait complete. 2024-04-16T05:43:00.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.387+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2699244096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba8103260 msgr2=0x7f8ba8103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.387+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2699244096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba8103260 0x7f8ba8103640 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009900 tx=0x7f8b9802bcd0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.387+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2699244096 shutdown_connections 2024-04-16T05:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.387+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2699244096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba810c630 0x7f8ba810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.387+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2699244096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba8103b80 0x7f8ba810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.387+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2699244096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba8103260 0x7f8ba8103640 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.387+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2699244096 >> 172.21.15.86:0/2699244096 conn(0x7f8ba8077280 msgr2=0x7f8ba8077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.387+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2699244096 shutdown_connections 2024-04-16T05:43:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.387+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2699244096 wait complete. 2024-04-16T05:43:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8bae0eb640 1 Processor -- start 2024-04-16T05:43:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8bae0eb640 1 -- start start 2024-04-16T05:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8bae0eb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba8103260 0x7f8ba819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8bae0eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba8103b80 0x7f8ba819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8bae0eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba810c630 0x7f8ba81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8bae0eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8112420 con 0x7f8ba810c630 2024-04-16T05:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8bae0eb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81122a0 con 0x7f8ba8103260 2024-04-16T05:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8bae0eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81125a0 con 0x7f8ba8103b80 2024-04-16T05:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba810c630 0x7f8ba81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba810c630 0x7f8ba81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41966/0 (socket says 172.21.15.86:41966) 2024-04-16T05:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba7fff640 1 -- 172.21.15.86:0/2759662480 learned_addr learned my addr 172.21.15.86:0/2759662480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba8103b80 0x7f8ba819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba77fe640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba8103260 0x7f8ba819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba7fff640 1 -- 172.21.15.86:0/2759662480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba8103b80 msgr2=0x7f8ba819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba7fff640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba8103b80 0x7f8ba819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba7fff640 1 -- 172.21.15.86:0/2759662480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba8103260 msgr2=0x7f8ba819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba7fff640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba8103260 0x7f8ba819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba7fff640 1 -- 172.21.15.86:0/2759662480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba8198fb0 con 0x7f8ba810c630 2024-04-16T05:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba77fe640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba8103260 0x7f8ba819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.388+0000 7f8ba7fff640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba810c630 0x7f8ba81987e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000dac0 tx=0x7f8ba000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.389+0000 7f8ba57fa640 1 -- 172.21.15.86:0/2759662480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0017070 con 0x7f8ba810c630 2024-04-16T05:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.389+0000 7f8ba57fa640 1 -- 172.21.15.86:0/2759662480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba0003e50 con 0x7f8ba810c630 2024-04-16T05:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.389+0000 7f8ba57fa640 1 -- 172.21.15.86:0/2759662480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00106c0 con 0x7f8ba810c630 2024-04-16T05:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.389+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2759662480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba81992a0 con 0x7f8ba810c630 2024-04-16T05:43:00.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.389+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2759662480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81a54c0 con 0x7f8ba810c630 2024-04-16T05:43:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8ba57fa640 1 -- 172.21.15.86:0/2759662480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ba0003970 con 0x7f8ba810c630 2024-04-16T05:43:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8bb0db5740 1 Processor -- start 2024-04-16T05:43:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8bb0db5740 1 -- start start 2024-04-16T05:43:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c13840 0x55ef23c13c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c21230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c23b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef23b790e0 con 0x55ef23c21770 2024-04-16T05:43:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ef23b16520 con 0x55ef23c141f0 2024-04-16T05:43:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef23b750a0 con 0x55ef23c13840 2024-04-16T05:43:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c23b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c23b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41982/0 (socket says 172.21.15.86:41982) 2024-04-16T05:43:00.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8b9ffff640 1 -- 172.21.15.86:0/2441679239 learned_addr learned my addr 172.21.15.86:0/2441679239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8ba77fe640 1 --2- 172.21.15.86:0/2441679239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c21230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8ba57fa640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b940a3da0 0x7f8b940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/2441679239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c13840 0x55ef23c13c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8b9ffff640 1 -- 172.21.15.86:0/2441679239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c13840 msgr2=0x55ef23c13c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/2441679239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c13840 0x55ef23c13c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8ba7fff640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b940a3da0 0x7f8b940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8b9ffff640 1 -- 172.21.15.86:0/2441679239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 msgr2=0x55ef23c21230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/2441679239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c21230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.392+0000 7f8b9ffff640 1 -- 172.21.15.86:0/2441679239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef23b3e8d0 con 0x55ef23c21770 2024-04-16T05:43:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.393+0000 7f8ba57fa640 1 -- 172.21.15.86:0/2759662480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8ba00d08d0 con 0x7f8ba810c630 2024-04-16T05:43:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.393+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/2441679239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c23b60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000ad50 tx=0x7f8b9000a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2f665b7d17c9db server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.393+0000 7f8b9dffb640 1 -- 172.21.15.86:0/2441679239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9001c070 con 0x55ef23c21770 2024-04-16T05:43:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.393+0000 7f8ba7fff640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b940a3da0 0x7f8b940a6260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00cdd00 tx=0x7f8ba0004580 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.393+0000 7f8b9dffb640 1 -- 172.21.15.86:0/2441679239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b9000fdf0 con 0x55ef23c21770 2024-04-16T05:43:00.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.393+0000 7f8b9dffb640 1 -- 172.21.15.86:0/2441679239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90014770 con 0x55ef23c21770 2024-04-16T05:43:00.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.393+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2441679239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 msgr2=0x55ef23c23b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.393+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/2441679239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c23b60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000ad50 tx=0x7f8b9000a470 comp rx=0 tx=0).stop 2024-04-16T05:43:00.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2441679239 shutdown_connections 2024-04-16T05:43:00.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/2441679239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c23b60 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/2441679239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c21230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/2441679239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c13840 0x55ef23c13c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2441679239 >> 172.21.15.86:0/2441679239 conn(0x55ef23c0e590 msgr2=0x55ef23c0e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2441679239 shutdown_connections 2024-04-16T05:43:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2441679239 wait complete. 2024-04-16T05:43:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 -- start start 2024-04-16T05:43:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 Processor -- start 2024-04-16T05:43:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c1e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c1ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c21770 0x55ef23c1f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef23c24530 con 0x55ef23c13840 2024-04-16T05:43:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ef23b324d0 con 0x55ef23c141f0 2024-04-16T05:43:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef23c246b0 con 0x55ef23c21770 2024-04-16T05:43:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8bb0db5740 10 client.0 _ll_register_callbacks cb 0x55ef23b172c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c1e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c1e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41992/0 (socket says 172.21.15.86:41992) 2024-04-16T05:43:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.394+0000 7f8b9ffff640 1 -- 172.21.15.86:0/1128428183 learned_addr learned my addr 172.21.15.86:0/1128428183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8ba77fe640 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c1ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b9ffff640 1 -- 172.21.15.86:0/1128428183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c21770 msgr2=0x55ef23c1f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c21770 0x55ef23c1f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b9ffff640 1 -- 172.21.15.86:0/1128428183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 msgr2=0x55ef23c1ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c1ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b9ffff640 1 -- 172.21.15.86:0/1128428183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2388b3f0 con 0x55ef23c13840 2024-04-16T05:43:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c1e770 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f8b90017fd0 tx=0x7f8b90006d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2e3b1d41a3653 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b86ffd640 1 -- 172.21.15.86:0/1128428183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90028060 con 0x55ef23c13840 2024-04-16T05:43:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8bb0db5740 1 -- 172.21.15.86:0/1128428183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b9000fdf0 con 0x55ef23c13840 2024-04-16T05:43:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b86ffd640 1 -- 172.21.15.86:0/1128428183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8b90017820 con 0x55ef23c13840 2024-04-16T05:43:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b86ffd640 1 -- 172.21.15.86:0/1128428183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b9002f050 con 0x55ef23c13840 2024-04-16T05:43:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8bb0db5740 10 client.17946 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8bb0db5740 1 -- 172.21.15.86:0/1128428183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef239557d0 con 0x55ef23c13840 2024-04-16T05:43:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.395+0000 7f8b86ffd640 1 -- 172.21.15.86:0/1128428183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b9000fdf0 con 0x55ef23c13840 2024-04-16T05:43:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8b86ffd640 1 -- 172.21.15.86:0/1128428183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b90015040 con 0x55ef23c13840 2024-04-16T05:43:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8bb0db5740 10 client.17946 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8bb0db5740 10 client.17946 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:43:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8bb0db5740 20 client.17946 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55ef23c315f0 0x55ef23c339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8bb0db5740 4 client.17946 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8bb0db5740 1 -- 172.21.15.86:0/1128428183 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef23b86b00 con 0x55ef23c315f0 2024-04-16T05:43:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8ba77fe640 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55ef23c315f0 0x55ef23c339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8ba77fe640 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55ef23c315f0 0x55ef23c339e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbd434b6a25453e1 server_cookie=67a15f788881f4e0 in_seq=0 out_seq=0 2024-04-16T05:43:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.396+0000 7f8b86ffd640 10 client.17946 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:43:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.397+0000 7f8b86ffd640 1 -- 172.21.15.86:0/1128428183 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef23b86b00 con 0x55ef23c315f0 2024-04-16T05:43:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.397+0000 7f8b86ffd640 10 client.17946 handle_command_reply: tid=0 2024-04-16T05:43:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.397+0000 7f8bb0db5740 1 client.17946 shutdown 2024-04-16T05:43:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.397+0000 7f8bb0db5740 1 -- 172.21.15.86:0/1128428183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 msgr2=0x55ef23c1e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.397+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c1e770 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f8b90017fd0 tx=0x7f8b90006d60 comp rx=0 tx=0).stop 2024-04-16T05:43:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.397+0000 7f8bb0db5740 1 -- 172.21.15.86:0/1128428183 shutdown_connections 2024-04-16T05:43:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.398+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55ef23c315f0 0x55ef23c339e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.398+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c21770 0x55ef23c1f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.398+0000 7f8b86ffd640 0 client.17946 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:43:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.398+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c1ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.398+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/1128428183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c1e770 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.398+0000 7f8bb0db5740 1 -- 172.21.15.86:0/1128428183 >> 172.21.15.86:0/1128428183 conn(0x55ef23c0e590 msgr2=0x55ef23c102e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.398+0000 7f8bb0db5740 1 -- 172.21.15.86:0/1128428183 shutdown_connections 2024-04-16T05:43:00.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.398+0000 7f8bb0db5740 1 -- 172.21.15.86:0/1128428183 wait complete. 2024-04-16T05:43:00.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.398+0000 7f8bb0db5740 20 client.17946 trim_cache size 0 max 16384 2024-04-16T05:43:00.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8bb0db5740 1 Processor -- start 2024-04-16T05:43:00.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8bb0db5740 1 -- start start 2024-04-16T05:43:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c13840 0x55ef23c20ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c141f0 0x55ef23c1d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c2d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81ad3e0 con 0x55ef23c21770 2024-04-16T05:43:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81aef20 con 0x55ef23c13840 2024-04-16T05:43:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81adab0 con 0x55ef23c141f0 2024-04-16T05:43:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c2d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8ba77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c141f0 0x55ef23c1d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c2d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42002/0 (socket says 172.21.15.86:42002) 2024-04-16T05:43:00.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8b9ffff640 1 -- 172.21.15.86:0/2697323192 learned_addr learned my addr 172.21.15.86:0/2697323192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.412+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/2697323192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c13840 0x55ef23c20ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.413+0000 7f8b9ffff640 1 -- 172.21.15.86:0/2697323192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c141f0 msgr2=0x55ef23c1d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.413+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/2697323192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c141f0 0x55ef23c1d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.413+0000 7f8b9ffff640 1 -- 172.21.15.86:0/2697323192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c13840 msgr2=0x55ef23c20ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.413+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/2697323192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c13840 0x55ef23c20ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.413+0000 7f8b9ffff640 1 -- 172.21.15.86:0/2697323192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef23c33d50 con 0x55ef23c21770 2024-04-16T05:43:00.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.413+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/2697323192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c2d460 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8b900215b0 tx=0x7f8b90007fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95fc9aa79e64c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.413+0000 7f8b9d7fa640 1 -- 172.21.15.86:0/2697323192 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90028070 con 0x55ef23c21770 2024-04-16T05:43:00.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.413+0000 7f8b9d7fa640 1 -- 172.21.15.86:0/2697323192 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b90017c50 con 0x55ef23c21770 2024-04-16T05:43:00.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.413+0000 7f8b9d7fa640 1 -- 172.21.15.86:0/2697323192 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9001c040 con 0x55ef23c21770 2024-04-16T05:43:00.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2697323192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 msgr2=0x55ef23c2d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/2697323192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c2d460 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8b900215b0 tx=0x7f8b90007fe0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2697323192 shutdown_connections 2024-04-16T05:43:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/2697323192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c21770 0x55ef23c2d460 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/2697323192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c141f0 0x55ef23c1d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/2697323192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c13840 0x55ef23c20ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2697323192 >> 172.21.15.86:0/2697323192 conn(0x55ef23c0e590 msgr2=0x55ef23c10220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2697323192 shutdown_connections 2024-04-16T05:43:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 -- 172.21.15.86:0/2697323192 wait complete. 2024-04-16T05:43:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.414+0000 7f8bb0db5740 1 -- start start 2024-04-16T05:43:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8bb0db5740 1 Processor -- start 2024-04-16T05:43:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c24ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c25430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8bb0db5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c21770 0x55ef23c293e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81ae850 con 0x55ef23c13840 2024-04-16T05:43:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81ae180 con 0x55ef23c141f0 2024-04-16T05:43:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8bb0db5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81ab3e0 con 0x55ef23c21770 2024-04-16T05:43:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8bb0db5740 10 client.0 _ll_register_callbacks cb 0x7f8b9001a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c24ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c24ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42012/0 (socket says 172.21.15.86:42012) 2024-04-16T05:43:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8b9ffff640 1 -- 172.21.15.86:0/999890544 learned_addr learned my addr 172.21.15.86:0/999890544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8ba77fe640 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c25430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c21770 0x55ef23c293e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8b9ffff640 1 -- 172.21.15.86:0/999890544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c21770 msgr2=0x55ef23c293e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c21770 0x55ef23c293e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8b9ffff640 1 -- 172.21.15.86:0/999890544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 msgr2=0x55ef23c25430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c25430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.415+0000 7f8b9ffff640 1 -- 172.21.15.86:0/999890544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef23c29a20 con 0x55ef23c13840 2024-04-16T05:43:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8b9ffff640 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c24ef0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8b90016000 tx=0x7f8b9000a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cee4fd84fd444a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8b86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8b86ffd640 1 -- 172.21.15.86:0/999890544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90044060 con 0x55ef23c13840 2024-04-16T05:43:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8b86ffd640 1 -- 172.21.15.86:0/999890544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8b9000fa40 con 0x55ef23c13840 2024-04-16T05:43:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8bb0db5740 1 -- 172.21.15.86:0/999890544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b90017c50 con 0x55ef23c13840 2024-04-16T05:43:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8b86ffd640 1 -- 172.21.15.86:0/999890544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b9001c050 con 0x55ef23c13840 2024-04-16T05:43:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8bb0db5740 10 client.17952 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8bb0db5740 1 -- 172.21.15.86:0/999890544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef23c97350 con 0x55ef23c13840 2024-04-16T05:43:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8b86ffd640 1 -- 172.21.15.86:0/999890544 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b90017c50 con 0x55ef23c13840 2024-04-16T05:43:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.416+0000 7f8b86ffd640 1 -- 172.21.15.86:0/999890544 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b900143e0 con 0x55ef23c13840 2024-04-16T05:43:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.417+0000 7f8bb0db5740 10 client.17952 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.417+0000 7f8bb0db5740 10 client.17952 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:43:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.417+0000 7f8bb0db5740 20 client.17952 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.417+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55ef23c979b0 0x55ef23c99da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.417+0000 7f8bb0db5740 4 client.17952 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.417+0000 7f8bb0db5740 1 -- 172.21.15.86:0/999890544 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef23b80970 con 0x55ef23c979b0 2024-04-16T05:43:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.417+0000 7f8ba77fe640 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55ef23c979b0 0x55ef23c99da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.417+0000 7f8ba77fe640 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55ef23c979b0 0x55ef23c99da0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddc4fa53e70ccc40 server_cookie=f32b4f7f743704ec in_seq=0 out_seq=0 2024-04-16T05:43:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.417+0000 7f8b86ffd640 10 client.17952 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:43:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8b86ffd640 1 -- 172.21.15.86:0/999890544 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef23b80970 con 0x55ef23c979b0 2024-04-16T05:43:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8b86ffd640 10 client.17952 handle_command_reply: tid=0 2024-04-16T05:43:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 client.17952 shutdown 2024-04-16T05:43:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 -- 172.21.15.86:0/999890544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 msgr2=0x55ef23c24ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c24ef0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8b90016000 tx=0x7f8b9000a470 comp rx=0 tx=0).stop 2024-04-16T05:43:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 -- 172.21.15.86:0/999890544 shutdown_connections 2024-04-16T05:43:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55ef23c979b0 0x55ef23c99da0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8b86ffd640 0 client.17952 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:43:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef23c21770 0x55ef23c293e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ef23c141f0 0x55ef23c25430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 --2- 172.21.15.86:0/999890544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef23c13840 0x55ef23c24ef0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 -- 172.21.15.86:0/999890544 >> 172.21.15.86:0/999890544 conn(0x55ef23c0e590 msgr2=0x55ef23c0f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 -- 172.21.15.86:0/999890544 shutdown_connections 2024-04-16T05:43:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 1 -- 172.21.15.86:0/999890544 wait complete. 2024-04-16T05:43:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.418+0000 7f8bb0db5740 20 client.17952 trim_cache size 0 max 16384 2024-04-16T05:43:00.426 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:43:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b940a3da0 msgr2=0x7f8b940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b940a3da0 0x7f8b940a6260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00cdd00 tx=0x7f8ba0004580 comp rx=0 tx=0).stop 2024-04-16T05:43:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba810c630 msgr2=0x7f8ba81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba810c630 0x7f8ba81987e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000dac0 tx=0x7f8ba000df80 comp rx=0 tx=0).stop 2024-04-16T05:43:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2759662480 shutdown_connections 2024-04-16T05:43:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba810c630 0x7f8ba81987e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba8103b80 0x7f8ba819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b940a3da0 0x7f8b940a6260 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 --2- 172.21.15.86:0/2759662480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba8103260 0x7f8ba819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2759662480 >> 172.21.15.86:0/2759662480 conn(0x7f8ba8077280 msgr2=0x7f8ba80ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2759662480 shutdown_connections 2024-04-16T05:43:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.419+0000 7f8bae0eb640 1 -- 172.21.15.86:0/2759662480 wait complete. 2024-04-16T05:43:00.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:00 smithi086 ceph-mon[32028]: pgmap v236: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:43:00.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:00 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/740660639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:00.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:43:00.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:00 smithi196 ceph-mon[36373]: pgmap v236: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:43:00.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:00 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/740660639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.544+0000 7f96b77a8640 1 -- 172.21.15.86:0/1515102833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0101170 msgr2=0x7f96b0101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.544+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1515102833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0101170 0x7f96b0101550 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009ca0 tx=0x7f96ac02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.544+0000 7f96b77a8640 1 -- 172.21.15.86:0/1515102833 shutdown_connections 2024-04-16T05:43:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.544+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1515102833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96b0106240 0x7f96b010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1515102833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b0101b20 0x7f96b0105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1515102833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0101170 0x7f96b0101550 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 -- 172.21.15.86:0/1515102833 >> 172.21.15.86:0/1515102833 conn(0x7f96b0077280 msgr2=0x7f96b0077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 -- 172.21.15.86:0/1515102833 shutdown_connections 2024-04-16T05:43:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 -- 172.21.15.86:0/1515102833 wait complete. 2024-04-16T05:43:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 Processor -- start 2024-04-16T05:43:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 -- start start 2024-04-16T05:43:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b0101170 0x7f96b019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96b0106240 0x7f96b019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0198730 0x7f96b0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0111370 con 0x7f96b0198730 2024-04-16T05:43:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01111f0 con 0x7f96b0106240 2024-04-16T05:43:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.545+0000 7f96b77a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01114f0 con 0x7f96b0101170 2024-04-16T05:43:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5d45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0198730 0x7f96b0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5d45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0198730 0x7f96b0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42040/0 (socket says 172.21.15.86:42040) 2024-04-16T05:43:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5544640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b0101170 0x7f96b019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5d45640 1 -- 172.21.15.86:0/1789969505 learned_addr learned my addr 172.21.15.86:0/1789969505 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b4d43640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96b0106240 0x7f96b019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5d45640 1 -- 172.21.15.86:0/1789969505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b0101170 msgr2=0x7f96b019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5d45640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b0101170 0x7f96b019e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5d45640 1 -- 172.21.15.86:0/1789969505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96b0106240 msgr2=0x7f96b019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5d45640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96b0106240 0x7f96b019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5d45640 1 -- 172.21.15.86:0/1789969505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b0199330 con 0x7f96b0198730 2024-04-16T05:43:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b5d45640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0198730 0x7f96b0198bb0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009fd0 tx=0x7f96a800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96a67fc640 1 -- 172.21.15.86:0/1789969505 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8011830 con 0x7f96b0198730 2024-04-16T05:43:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96a67fc640 1 -- 172.21.15.86:0/1789969505 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a8016070 con 0x7f96b0198730 2024-04-16T05:43:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b77a8640 1 -- 172.21.15.86:0/1789969505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b01a5410 con 0x7f96b0198730 2024-04-16T05:43:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96a67fc640 1 -- 172.21.15.86:0/1789969505 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a800f2e0 con 0x7f96b0198730 2024-04-16T05:43:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.546+0000 7f96b77a8640 1 -- 172.21.15.86:0/1789969505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b01a5810 con 0x7f96b0198730 2024-04-16T05:43:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.549+0000 7f96a67fc640 1 -- 172.21.15.86:0/1789969505 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f96a80119d0 con 0x7f96b0198730 2024-04-16T05:43:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.549+0000 7f96ba38e740 1 Processor -- start 2024-04-16T05:43:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.549+0000 7f96ba38e740 1 -- start start 2024-04-16T05:43:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06e7e60 0x55dbe06e8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06f1360 0x55dbe06f3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96ba38e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dbe064d0e0 con 0x55dbe06e8780 2024-04-16T05:43:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96ba38e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dbe05ea520 con 0x55dbe06f1360 2024-04-16T05:43:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96ba38e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dbe06490a0 con 0x55dbe06e7e60 2024-04-16T05:43:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b5544640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b5544640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42050/0 (socket says 172.21.15.86:42050) 2024-04-16T05:43:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b5544640 1 -- 172.21.15.86:0/1821981140 learned_addr learned my addr 172.21.15.86:0/1821981140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b4d43640 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06e7e60 0x55dbe06e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b4d43640 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06f1360 0x55dbe06f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b5544640 1 -- 172.21.15.86:0/1821981140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06e7e60 msgr2=0x55dbe06e8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b5544640 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06e7e60 0x55dbe06e8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b5544640 1 -- 172.21.15.86:0/1821981140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06f1360 msgr2=0x55dbe06f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b5544640 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06f1360 0x55dbe06f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b5544640 1 -- 172.21.15.86:0/1821981140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe06128d0 con 0x55dbe06e8780 2024-04-16T05:43:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b4d43640 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06e7e60 0x55dbe06e8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96a67fc640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f969c0a3da0 0x7f969c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b5544640 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f0e20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f96ac02c700 tx=0x7f96ac009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f730fc1b852ec224 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96b4d43640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f969c0a3da0 0x7f969c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f96a67fc640 1 -- 172.21.15.86:0/1789969505 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f96a80cffe0 con 0x7f96b0198730 2024-04-16T05:43:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f969b7fe640 1 -- 172.21.15.86:0/1821981140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac043070 con 0x55dbe06e8780 2024-04-16T05:43:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f969b7fe640 1 -- 172.21.15.86:0/1821981140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96ac032c90 con 0x55dbe06e8780 2024-04-16T05:43:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.550+0000 7f969b7fe640 1 -- 172.21.15.86:0/1821981140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac039040 con 0x55dbe06e8780 2024-04-16T05:43:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96ba38e740 1 -- 172.21.15.86:0/1821981140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 msgr2=0x55dbe06f0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96b4d43640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f969c0a3da0 0x7f969c0a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55dbe06e3720 tx=0x7f96a000a250 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96ba38e740 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f0e20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f96ac02c700 tx=0x7f96ac009920 comp rx=0 tx=0).stop 2024-04-16T05:43:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96ba38e740 1 -- 172.21.15.86:0/1821981140 shutdown_connections 2024-04-16T05:43:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96ba38e740 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06f1360 0x55dbe06f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96ba38e740 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f0e20 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96ba38e740 1 --2- 172.21.15.86:0/1821981140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06e7e60 0x55dbe06e8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96ba38e740 1 -- 172.21.15.86:0/1821981140 >> 172.21.15.86:0/1821981140 conn(0x55dbe06e2590 msgr2=0x55dbe06e2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96ba38e740 1 -- 172.21.15.86:0/1821981140 shutdown_connections 2024-04-16T05:43:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.551+0000 7f96ba38e740 1 -- 172.21.15.86:0/1821981140 wait complete. 2024-04-16T05:43:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96ba38e740 1 -- start start 2024-04-16T05:43:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96ba38e740 1 Processor -- start 2024-04-16T05:43:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e7e60 0x55dbe06ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e8780 0x55dbe06ee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06eed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96ba38e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dbe06f4060 con 0x55dbe06e7e60 2024-04-16T05:43:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96ba38e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dbe06064d0 con 0x55dbe06e8780 2024-04-16T05:43:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96ba38e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dbe06f41e0 con 0x55dbe06f1360 2024-04-16T05:43:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96ba38e740 10 client.0 _ll_register_callbacks cb 0x55dbe05eb2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96b5544640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e7e60 0x55dbe06ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96b5544640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e7e60 0x55dbe06ee2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42060/0 (socket says 172.21.15.86:42060) 2024-04-16T05:43:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96b5544640 1 -- 172.21.15.86:0/2171783292 learned_addr learned my addr 172.21.15.86:0/2171783292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96b5544640 1 -- 172.21.15.86:0/2171783292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 msgr2=0x55dbe06eed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96b5544640 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96b5544640 1 -- 172.21.15.86:0/2171783292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e8780 msgr2=0x55dbe06ee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96b5544640 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e8780 0x55dbe06ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.552+0000 7f96b5544640 1 -- 172.21.15.86:0/2171783292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe035f3f0 con 0x55dbe06e7e60 2024-04-16T05:43:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96b5544640 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e7e60 0x55dbe06ee2f0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f96ac037040 tx=0x7f96ac0326f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdf07bab76e03ef server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f9698ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f9698ff9640 1 -- 172.21.15.86:0/2171783292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac04c030 con 0x55dbe06e7e60 2024-04-16T05:43:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f9698ff9640 1 -- 172.21.15.86:0/2171783292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f96ac048450 con 0x55dbe06e7e60 2024-04-16T05:43:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96ba38e740 1 -- 172.21.15.86:0/2171783292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96ac032c90 con 0x55dbe06e7e60 2024-04-16T05:43:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f9698ff9640 1 -- 172.21.15.86:0/2171783292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96ac059050 con 0x55dbe06e7e60 2024-04-16T05:43:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96ba38e740 10 client.17964 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96ba38e740 1 -- 172.21.15.86:0/2171783292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dbe04297d0 con 0x55dbe06e7e60 2024-04-16T05:43:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f9698ff9640 1 -- 172.21.15.86:0/2171783292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96ac032c90 con 0x55dbe06e7e60 2024-04-16T05:43:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f9698ff9640 1 -- 172.21.15.86:0/2171783292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f96ac034a00 con 0x55dbe06e7e60 2024-04-16T05:43:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96ba38e740 10 client.17964 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96ba38e740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96ba38e740 20 client.17964 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55dbe0701070 0x55dbe0703460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96ba38e740 4 client.17964 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.553+0000 7f96ba38e740 1 -- 172.21.15.86:0/2171783292 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbe065ab00 con 0x55dbe0701070 2024-04-16T05:43:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.554+0000 7f96b4d43640 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55dbe0701070 0x55dbe0703460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.554+0000 7f9698ff9640 10 client.17964 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.554+0000 7f96b4d43640 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55dbe0701070 0x55dbe0703460 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436624d3c0602b5e server_cookie=4d880b3b0715502 in_seq=0 out_seq=0 2024-04-16T05:43:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f9698ff9640 1 -- 172.21.15.86:0/2171783292 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbe065ab00 con 0x55dbe0701070 2024-04-16T05:43:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f9698ff9640 10 client.17964 handle_command_reply: tid=0 2024-04-16T05:43:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 client.17964 shutdown 2024-04-16T05:43:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 -- 172.21.15.86:0/2171783292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e7e60 msgr2=0x55dbe06ee2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e7e60 0x55dbe06ee2f0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f96ac037040 tx=0x7f96ac0326f0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 -- 172.21.15.86:0/2171783292 shutdown_connections 2024-04-16T05:43:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55dbe0701070 0x55dbe0703460 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06eed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f9698ff9640 0 client.17964 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:00.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e8780 0x55dbe06ee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2171783292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e7e60 0x55dbe06ee2f0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 -- 172.21.15.86:0/2171783292 >> 172.21.15.86:0/2171783292 conn(0x55dbe06e2590 msgr2=0x55dbe06e42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 -- 172.21.15.86:0/2171783292 shutdown_connections 2024-04-16T05:43:00.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 1 -- 172.21.15.86:0/2171783292 wait complete. 2024-04-16T05:43:00.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.555+0000 7f96ba38e740 20 client.17964 trim_cache size 0 max 16384 2024-04-16T05:43:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96ba38e740 1 Processor -- start 2024-04-16T05:43:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96ba38e740 1 -- start start 2024-04-16T05:43:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 0x55dbe06f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06fd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96ba38e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01af590 con 0x55dbe06e8780 2024-04-16T05:43:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96ba38e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01b10d0 con 0x55dbe06e7e60 2024-04-16T05:43:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96ba38e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01afc60 con 0x55dbe06f1360 2024-04-16T05:43:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96b4d43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06fd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96b4d43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06fd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42074/0 (socket says 172.21.15.86:42074) 2024-04-16T05:43:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96b4d43640 1 -- 172.21.15.86:0/2744495078 learned_addr learned my addr 172.21.15.86:0/2744495078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96b4d43640 1 -- 172.21.15.86:0/2744495078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 msgr2=0x55dbe06dd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96b4d43640 1 --2- 172.21.15.86:0/2744495078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06dd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96b4d43640 1 -- 172.21.15.86:0/2744495078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 msgr2=0x55dbe06f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96b4d43640 1 --2- 172.21.15.86:0/2744495078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 0x55dbe06f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.568+0000 7f96b4d43640 1 -- 172.21.15.86:0/2744495078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe06ed2a0 con 0x55dbe06e8780 2024-04-16T05:43:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96b4d43640 1 --2- 172.21.15.86:0/2744495078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06fd010 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f96b0199f80 tx=0x7f96a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ed89d2623348d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f969affd640 1 -- 172.21.15.86:0/2744495078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a002a030 con 0x55dbe06e8780 2024-04-16T05:43:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f969affd640 1 -- 172.21.15.86:0/2744495078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a0004020 con 0x55dbe06e8780 2024-04-16T05:43:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f969affd640 1 -- 172.21.15.86:0/2744495078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0026810 con 0x55dbe06e8780 2024-04-16T05:43:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96ba38e740 1 -- 172.21.15.86:0/2744495078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 msgr2=0x55dbe06fd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2744495078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06fd010 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f96b0199f80 tx=0x7f96a0014000 comp rx=0 tx=0).stop 2024-04-16T05:43:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96ba38e740 1 -- 172.21.15.86:0/2744495078 shutdown_connections 2024-04-16T05:43:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2744495078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2744495078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06fd010 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96ba38e740 1 --2- 172.21.15.86:0/2744495078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 0x55dbe06f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96ba38e740 1 -- 172.21.15.86:0/2744495078 >> 172.21.15.86:0/2744495078 conn(0x55dbe06e2590 msgr2=0x55dbe06e4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96ba38e740 1 -- 172.21.15.86:0/2744495078 shutdown_connections 2024-04-16T05:43:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.569+0000 7f96ba38e740 1 -- 172.21.15.86:0/2744495078 wait complete. 2024-04-16T05:43:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96ba38e740 1 -- start start 2024-04-16T05:43:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96ba38e740 1 Processor -- start 2024-04-16T05:43:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 0x55dbe06f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96ba38e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96ba38e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01b0a00 con 0x55dbe06e8780 2024-04-16T05:43:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96ba38e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01b0330 con 0x55dbe06e7e60 2024-04-16T05:43:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96ba38e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01ad590 con 0x55dbe06f1360 2024-04-16T05:43:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96ba38e740 10 client.0 _ll_register_callbacks cb 0x7f96ac032450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b4d43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b4d43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42088/0 (socket says 172.21.15.86:42088) 2024-04-16T05:43:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b4d43640 1 -- 172.21.15.86:0/255163209 learned_addr learned my addr 172.21.15.86:0/255163209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b5d45640 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b5544640 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 0x55dbe06f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b4d43640 1 -- 172.21.15.86:0/255163209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 msgr2=0x55dbe06f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b4d43640 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b4d43640 1 -- 172.21.15.86:0/255163209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 msgr2=0x55dbe06f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b4d43640 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 0x55dbe06f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b4d43640 1 -- 172.21.15.86:0/255163209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe0766650 con 0x55dbe06e8780 2024-04-16T05:43:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b5544640 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 0x55dbe06f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.570+0000 7f96b5d45640 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96b4d43640 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f57a0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f96a0016040 tx=0x7f96a00264a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f85a993a271616 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f9698ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f9698ff9640 1 -- 172.21.15.86:0/255163209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a002a040 con 0x55dbe06e8780 2024-04-16T05:43:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f9698ff9640 1 -- 172.21.15.86:0/255163209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f96a0026690 con 0x55dbe06e8780 2024-04-16T05:43:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96ba38e740 1 -- 172.21.15.86:0/255163209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a0004020 con 0x55dbe06e8780 2024-04-16T05:43:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f9698ff9640 1 -- 172.21.15.86:0/255163209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96a0040050 con 0x55dbe06e8780 2024-04-16T05:43:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96ba38e740 10 client.17970 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96ba38e740 1 -- 172.21.15.86:0/255163209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dbe0767950 con 0x55dbe06e8780 2024-04-16T05:43:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f9698ff9640 1 -- 172.21.15.86:0/255163209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a0004020 con 0x55dbe06e8780 2024-04-16T05:43:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f9698ff9640 1 -- 172.21.15.86:0/255163209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f96a002f460 con 0x55dbe06e8780 2024-04-16T05:43:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96ba38e740 10 client.17970 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96ba38e740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96ba38e740 20 client.17970 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96ba38e740 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55dbe0768030 0x55dbe076a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96ba38e740 4 client.17970 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.571+0000 7f96ba38e740 1 -- 172.21.15.86:0/255163209 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbe0654970 con 0x55dbe0768030 2024-04-16T05:43:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96b5544640 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55dbe0768030 0x55dbe076a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f9698ff9640 10 client.17970 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96b5544640 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55dbe0768030 0x55dbe076a450 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=159974e3c52e995 server_cookie=a9cecbc0680d2284 in_seq=0 out_seq=0 2024-04-16T05:43:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f9698ff9640 1 -- 172.21.15.86:0/255163209 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbe0654970 con 0x55dbe0768030 2024-04-16T05:43:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f9698ff9640 10 client.17970 handle_command_reply: tid=0 2024-04-16T05:43:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 client.17970 shutdown 2024-04-16T05:43:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 -- 172.21.15.86:0/255163209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 msgr2=0x55dbe06f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f57a0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f96a0016040 tx=0x7f96a00264a0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 -- 172.21.15.86:0/255163209 shutdown_connections 2024-04-16T05:43:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55dbe0768030 0x55dbe076a450 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbe06f1360 0x55dbe06f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f9698ff9640 0 client.17970 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe06e8780 0x55dbe06f57a0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 --2- 172.21.15.86:0/255163209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dbe06e7e60 0x55dbe06f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 -- 172.21.15.86:0/255163209 >> 172.21.15.86:0/255163209 conn(0x55dbe06e2590 msgr2=0x55dbe06e3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 -- 172.21.15.86:0/255163209 shutdown_connections 2024-04-16T05:43:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 1 -- 172.21.15.86:0/255163209 wait complete. 2024-04-16T05:43:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.572+0000 7f96ba38e740 20 client.17970 trim_cache size 0 max 16384 2024-04-16T05:43:00.582 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:43:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 -- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f969c0a3da0 msgr2=0x7f969c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f969c0a3da0 0x7f969c0a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55dbe06e3720 tx=0x7f96a000a250 comp rx=0 tx=0).stop 2024-04-16T05:43:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 -- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0198730 msgr2=0x7f96b0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0198730 0x7f96b0198bb0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009fd0 tx=0x7f96a800b8e0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 -- 172.21.15.86:0/1789969505 shutdown_connections 2024-04-16T05:43:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b0198730 0x7f96b0198bb0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f969c0a3da0 0x7f969c0a6260 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96b0106240 0x7f96b019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 --2- 172.21.15.86:0/1789969505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b0101170 0x7f96b019e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 -- 172.21.15.86:0/1789969505 >> 172.21.15.86:0/1789969505 conn(0x7f96b0077280 msgr2=0x7f96b006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 -- 172.21.15.86:0/1789969505 shutdown_connections 2024-04-16T05:43:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:00.573+0000 7f96b77a8640 1 -- 172.21.15.86:0/1789969505 wait complete. 2024-04-16T05:43:00.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:00 smithi194 ceph-mon[35865]: pgmap v236: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:43:00.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:00 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/740660639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:01.586 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:43:01.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:43:01.587 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:43:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.702+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3556000199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4105f50 msgr2=0x7fdca4106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.702+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3556000199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4105f50 0x7fdca4106330 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009900 tx=0x7fdc9402bc80 comp rx=0 tx=0).stop 2024-04-16T05:43:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.702+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3556000199 shutdown_connections 2024-04-16T05:43:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.702+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3556000199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdca4101b80 0x7fdca410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.702+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3556000199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca41011e0 0x7fdca4101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.702+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3556000199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4105f50 0x7fdca4106330 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.702+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3556000199 >> 172.21.15.86:0/3556000199 conn(0x7fdca4077ee0 msgr2=0x7fdca40782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.702+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3556000199 shutdown_connections 2024-04-16T05:43:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.702+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3556000199 wait complete. 2024-04-16T05:43:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca8b9d640 1 Processor -- start 2024-04-16T05:43:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca8b9d640 1 -- start start 2024-04-16T05:43:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca8b9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdca41011e0 0x7fdca4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca8b9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4101b80 0x7fdca4199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca8b9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4105f50 0x7fdca41a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca8b9d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41113f0 con 0x7fdca4101b80 2024-04-16T05:43:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca8b9d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4111270 con 0x7fdca41011e0 2024-04-16T05:43:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca8b9d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4111570 con 0x7fdca4105f50 2024-04-16T05:43:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4101b80 0x7fdca4199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4101b80 0x7fdca4199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59028/0 (socket says 172.21.15.86:59028) 2024-04-16T05:43:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdc9bfff640 1 -- 172.21.15.86:0/3869250188 learned_addr learned my addr 172.21.15.86:0/3869250188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca259c640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdca41011e0 0x7fdca4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdc9bfff640 1 -- 172.21.15.86:0/3869250188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4105f50 msgr2=0x7fdca41a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdc9bfff640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4105f50 0x7fdca41a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdc9bfff640 1 -- 172.21.15.86:0/3869250188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdca41011e0 msgr2=0x7fdca4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdc9bfff640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdca41011e0 0x7fdca4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdc9bfff640 1 -- 172.21.15.86:0/3869250188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41a4e10 con 0x7fdca4101b80 2024-04-16T05:43:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.703+0000 7fdca259c640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdca41011e0 0x7fdca4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.704+0000 7fdc9bfff640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4101b80 0x7fdca4199d90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00d770 tx=0x7fdc8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.704+0000 7fdc9b7fe640 1 -- 172.21.15.86:0/3869250188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c017070 con 0x7fdca4101b80 2024-04-16T05:43:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.704+0000 7fdc9b7fe640 1 -- 172.21.15.86:0/3869250188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c005500 con 0x7fdca4101b80 2024-04-16T05:43:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.704+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3869250188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca41a4fe0 con 0x7fdca4101b80 2024-04-16T05:43:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.704+0000 7fdc9b7fe640 1 -- 172.21.15.86:0/3869250188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c004e00 con 0x7fdca4101b80 2024-04-16T05:43:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.704+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3869250188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca41a5450 con 0x7fdca4101b80 2024-04-16T05:43:01.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.706+0000 7fdc9b7fe640 1 -- 172.21.15.86:0/3869250188 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc8c004fa0 con 0x7fdca4101b80 2024-04-16T05:43:01.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.706+0000 7fdcab783740 1 Processor -- start 2024-04-16T05:43:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdcab783740 1 -- start start 2024-04-16T05:43:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdcab783740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a228354ab0 0x55a228354e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdcab783740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a22835da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdcab783740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 0x55a2283603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdcab783740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2282b9e90 con 0x55a228354ab0 2024-04-16T05:43:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdcab783740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a2282574c0 con 0x55a22835dfb0 2024-04-16T05:43:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdcab783740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a2282b5e50 con 0x55a2283553d0 2024-04-16T05:43:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a228354ab0 0x55a228354e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a228354ab0 0x55a228354e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59034/0 (socket says 172.21.15.86:59034) 2024-04-16T05:43:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 -- 172.21.15.86:0/833652307 learned_addr learned my addr 172.21.15.86:0/833652307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdc9b7fe640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc900a3d20 0x7fdc900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca259c640 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a22835da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdc9bfff640 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 0x55a2283603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 -- 172.21.15.86:0/833652307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 msgr2=0x55a22835da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a22835da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 -- 172.21.15.86:0/833652307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 msgr2=0x55a2283603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 0x55a2283603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 -- 172.21.15.86:0/833652307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2280947e0 con 0x55a228354ab0 2024-04-16T05:43:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdc9bfff640 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 0x55a2283603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdc9b7fe640 1 -- 172.21.15.86:0/3869250188 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fdc8c0d06f0 con 0x7fdca4101b80 2024-04-16T05:43:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc900a3d20 0x7fdc900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.707+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a228354ab0 0x55a228354e90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009e90 tx=0x7fdc9c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3608bc4b16e18967 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdc98ff9640 1 -- 172.21.15.86:0/833652307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00ee10 con 0x55a228354ab0 2024-04-16T05:43:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdc98ff9640 1 -- 172.21.15.86:0/833652307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c01ba30 con 0x55a228354ab0 2024-04-16T05:43:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdc98ff9640 1 -- 172.21.15.86:0/833652307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c011900 con 0x55a228354ab0 2024-04-16T05:43:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc900a3d20 0x7fdc900a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c01f000 tx=0x7fdc9c020000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdcab783740 1 -- 172.21.15.86:0/833652307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a228354ab0 msgr2=0x55a228354e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdcab783740 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a228354ab0 0x55a228354e90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009e90 tx=0x7fdc9c00e6c0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdcab783740 1 -- 172.21.15.86:0/833652307 shutdown_connections 2024-04-16T05:43:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdcab783740 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 0x55a2283603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdcab783740 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a22835da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdcab783740 1 --2- 172.21.15.86:0/833652307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a228354ab0 0x55a228354e90 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdcab783740 1 -- 172.21.15.86:0/833652307 >> 172.21.15.86:0/833652307 conn(0x55a22834f1e0 msgr2=0x55a22834f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdcab783740 1 -- 172.21.15.86:0/833652307 shutdown_connections 2024-04-16T05:43:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.708+0000 7fdcab783740 1 -- 172.21.15.86:0/833652307 wait complete. 2024-04-16T05:43:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdcab783740 1 -- start start 2024-04-16T05:43:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdcab783740 1 Processor -- start 2024-04-16T05:43:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdcab783740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a228354ab0 0x55a22835b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdcab783740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2283553d0 0x55a22835b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdcab783740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 0x55a22835bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdcab783740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a228360da0 con 0x55a2283553d0 2024-04-16T05:43:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdcab783740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a228242410 con 0x55a22835dfb0 2024-04-16T05:43:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdcab783740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a228360f20 con 0x55a228354ab0 2024-04-16T05:43:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdcab783740 10 client.0 _ll_register_callbacks cb 0x55a22823db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2283553d0 0x55a22835b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2283553d0 0x55a22835b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59040/0 (socket says 172.21.15.86:59040) 2024-04-16T05:43:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdc9bfff640 1 -- 172.21.15.86:0/1064584582 learned_addr learned my addr 172.21.15.86:0/1064584582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdca259c640 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a228354ab0 0x55a22835b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 0x55a22835bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdc9bfff640 1 -- 172.21.15.86:0/1064584582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a228354ab0 msgr2=0x55a22835b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdc9bfff640 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a228354ab0 0x55a22835b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdc9bfff640 1 -- 172.21.15.86:0/1064584582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 msgr2=0x55a22835bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdc9bfff640 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 0x55a22835bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.709+0000 7fdc9bfff640 1 -- 172.21.15.86:0/1064584582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2280f4f70 con 0x55a2283553d0 2024-04-16T05:43:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdc9bfff640 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2283553d0 0x55a22835b570 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c09c250 tx=0x7fdc8c09bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed72084ffd4dd53 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdc89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdc89ffb640 1 -- 172.21.15.86:0/1064584582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c020040 con 0x55a2283553d0 2024-04-16T05:43:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdc89ffb640 1 -- 172.21.15.86:0/1064584582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fdc8c0cdcf0 con 0x55a2283553d0 2024-04-16T05:43:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdcab783740 1 -- 172.21.15.86:0/1064584582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9c01ba30 con 0x55a2283553d0 2024-04-16T05:43:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdc89ffb640 1 -- 172.21.15.86:0/1064584582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc8c0a4f30 con 0x55a2283553d0 2024-04-16T05:43:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdcab783740 10 client.17982 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdcab783740 1 -- 172.21.15.86:0/1064584582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a227f8c170 con 0x55a2283553d0 2024-04-16T05:43:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdc89ffb640 1 -- 172.21.15.86:0/1064584582 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c01ba30 con 0x55a2283553d0 2024-04-16T05:43:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdc89ffb640 1 -- 172.21.15.86:0/1064584582 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc8c098930 con 0x55a2283553d0 2024-04-16T05:43:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdcab783740 10 client.17982 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdcab783740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdcab783740 20 client.17982 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdcab783740 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a22836e3d0 0x55a2283707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdcab783740 4 client.17982 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.710+0000 7fdcab783740 1 -- 172.21.15.86:0/1064584582 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2282c78b0 con 0x55a22836e3d0 2024-04-16T05:43:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.711+0000 7fdca259c640 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a22836e3d0 0x55a2283707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.711+0000 7fdca259c640 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a22836e3d0 0x55a2283707c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc11ca06b7bdbd9 server_cookie=faf9035278b200eb in_seq=0 out_seq=0 2024-04-16T05:43:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.711+0000 7fdc89ffb640 10 client.17982 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdc89ffb640 1 -- 172.21.15.86:0/1064584582 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2282c78b0 con 0x55a22836e3d0 2024-04-16T05:43:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdc89ffb640 10 client.17982 handle_command_reply: tid=0 2024-04-16T05:43:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 client.17982 shutdown 2024-04-16T05:43:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 -- 172.21.15.86:0/1064584582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2283553d0 msgr2=0x55a22835b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2283553d0 0x55a22835b570 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c09c250 tx=0x7fdc8c09bce0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 -- 172.21.15.86:0/1064584582 shutdown_connections 2024-04-16T05:43:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a22836e3d0 0x55a2283707c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a22835dfb0 0x55a22835bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2283553d0 0x55a22835b570 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdc89ffb640 0 client.17982 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 --2- 172.21.15.86:0/1064584582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a228354ab0 0x55a22835b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 -- 172.21.15.86:0/1064584582 >> 172.21.15.86:0/1064584582 conn(0x55a22834f1e0 msgr2=0x55a228350f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 -- 172.21.15.86:0/1064584582 shutdown_connections 2024-04-16T05:43:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 1 -- 172.21.15.86:0/1064584582 wait complete. 2024-04-16T05:43:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.712+0000 7fdcab783740 20 client.17982 trim_cache size 0 max 16384 2024-04-16T05:43:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.729+0000 7fdcab783740 1 Processor -- start 2024-04-16T05:43:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.729+0000 7fdcab783740 1 -- start start 2024-04-16T05:43:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdcab783740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 0x55a22835d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdcab783740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a228369c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdcab783740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228376090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdcab783740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41af5f0 con 0x55a22835dfb0 2024-04-16T05:43:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdcab783740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41b1130 con 0x55a228354ab0 2024-04-16T05:43:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdcab783740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41afcc0 con 0x55a2283553d0 2024-04-16T05:43:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228376090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228376090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59046/0 (socket says 172.21.15.86:59046) 2024-04-16T05:43:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca2d9d640 1 -- 172.21.15.86:0/3237332159 learned_addr learned my addr 172.21.15.86:0/3237332159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca259c640 1 --2- 172.21.15.86:0/3237332159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 0x55a22835d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca2d9d640 1 -- 172.21.15.86:0/3237332159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 msgr2=0x55a228369c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/3237332159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a228369c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca2d9d640 1 -- 172.21.15.86:0/3237332159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 msgr2=0x55a22835d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/3237332159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 0x55a22835d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca2d9d640 1 -- 172.21.15.86:0/3237332159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a22834fdb0 con 0x55a22835dfb0 2024-04-16T05:43:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/3237332159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228376090 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0094e0 tx=0x7fdc9c012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc8a1539c8d0ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdc8bfff640 1 -- 172.21.15.86:0/3237332159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c034070 con 0x55a22835dfb0 2024-04-16T05:43:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdc8bfff640 1 -- 172.21.15.86:0/3237332159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c00ee10 con 0x55a22835dfb0 2024-04-16T05:43:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.730+0000 7fdc8bfff640 1 -- 172.21.15.86:0/3237332159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c01b8d0 con 0x55a22835dfb0 2024-04-16T05:43:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 -- 172.21.15.86:0/3237332159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 msgr2=0x55a228376090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 --2- 172.21.15.86:0/3237332159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228376090 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0094e0 tx=0x7fdc9c012f90 comp rx=0 tx=0).stop 2024-04-16T05:43:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 -- 172.21.15.86:0/3237332159 shutdown_connections 2024-04-16T05:43:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 --2- 172.21.15.86:0/3237332159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228376090 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 --2- 172.21.15.86:0/3237332159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a228369c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 --2- 172.21.15.86:0/3237332159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 0x55a22835d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 -- 172.21.15.86:0/3237332159 >> 172.21.15.86:0/3237332159 conn(0x55a22834f1e0 msgr2=0x55a22835b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 -- 172.21.15.86:0/3237332159 shutdown_connections 2024-04-16T05:43:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 -- 172.21.15.86:0/3237332159 wait complete. 2024-04-16T05:43:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 -- start start 2024-04-16T05:43:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 Processor -- start 2024-04-16T05:43:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 0x55a228366170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a2283666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228363140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41b0a60 con 0x55a22835dfb0 2024-04-16T05:43:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41b0390 con 0x55a228354ab0 2024-04-16T05:43:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.731+0000 7fdcab783740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41ad5f0 con 0x55a2283553d0 2024-04-16T05:43:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdcab783740 10 client.0 _ll_register_callbacks cb 0x7fdc8c098630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228363140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228363140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59056/0 (socket says 172.21.15.86:59056) 2024-04-16T05:43:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca2d9d640 1 -- 172.21.15.86:0/3836955359 learned_addr learned my addr 172.21.15.86:0/3836955359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdc9bfff640 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a2283666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca259c640 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 0x55a228366170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca2d9d640 1 -- 172.21.15.86:0/3836955359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 msgr2=0x55a2283666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a2283666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca2d9d640 1 -- 172.21.15.86:0/3836955359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 msgr2=0x55a228366170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 0x55a228366170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca2d9d640 1 -- 172.21.15.86:0/3836955359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a228363730 con 0x55a22835dfb0 2024-04-16T05:43:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdca2d9d640 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228363140 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c032040 tx=0x7fdc9c0097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e9483ed5688e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdc89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdc89ffb640 1 -- 172.21.15.86:0/3836955359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c03f060 con 0x55a22835dfb0 2024-04-16T05:43:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdc89ffb640 1 -- 172.21.15.86:0/3836955359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fdc9c01ba60 con 0x55a22835dfb0 2024-04-16T05:43:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdcab783740 1 -- 172.21.15.86:0/3836955359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a2282c1720 con 0x55a22835dfb0 2024-04-16T05:43:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdc89ffb640 1 -- 172.21.15.86:0/3836955359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc9c034070 con 0x55a22835dfb0 2024-04-16T05:43:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdcab783740 10 client.17988 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdcab783740 1 -- 172.21.15.86:0/3836955359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2283d5230 con 0x55a22835dfb0 2024-04-16T05:43:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.732+0000 7fdc89ffb640 1 -- 172.21.15.86:0/3836955359 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a2282c1720 con 0x55a22835dfb0 2024-04-16T05:43:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdc89ffb640 1 -- 172.21.15.86:0/3836955359 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc9c00f040 con 0x55a22835dfb0 2024-04-16T05:43:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdcab783740 10 client.17988 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdcab783740 10 client.17988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdcab783740 20 client.17988 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdcab783740 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a2283d5730 0x55a2283d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdcab783740 4 client.17988 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:43:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdcab783740 1 -- 172.21.15.86:0/3836955359 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a22823dbe0 con 0x55a2283d5730 2024-04-16T05:43:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdca259c640 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a2283d5730 0x55a2283d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdc89ffb640 10 client.17988 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.733+0000 7fdca259c640 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a2283d5730 0x55a2283d7b50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ce054a0246275a server_cookie=acdddb00ff358fcc in_seq=0 out_seq=0 2024-04-16T05:43:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdc89ffb640 1 -- 172.21.15.86:0/3836955359 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a22823dbe0 con 0x55a2283d5730 2024-04-16T05:43:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdc89ffb640 10 client.17988 handle_command_reply: tid=0 2024-04-16T05:43:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdcab783740 1 client.17988 shutdown 2024-04-16T05:43:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdcab783740 1 -- 172.21.15.86:0/3836955359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 msgr2=0x55a228363140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdcab783740 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228363140 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c032040 tx=0x7fdc9c0097c0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdcab783740 1 -- 172.21.15.86:0/3836955359 shutdown_connections 2024-04-16T05:43:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdcab783740 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55a2283d5730 0x55a2283d7b50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdcab783740 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a22835dfb0 0x55a228363140 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdcab783740 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2283553d0 0x55a2283666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.734+0000 7fdcab783740 1 --2- 172.21.15.86:0/3836955359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a228354ab0 0x55a228366170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdcab783740 1 -- 172.21.15.86:0/3836955359 >> 172.21.15.86:0/3836955359 conn(0x55a22834f1e0 msgr2=0x55a22834d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdc89ffb640 0 client.17988 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdcab783740 1 -- 172.21.15.86:0/3836955359 shutdown_connections 2024-04-16T05:43:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdcab783740 1 -- 172.21.15.86:0/3836955359 wait complete. 2024-04-16T05:43:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdcab783740 20 client.17988 trim_cache size 0 max 16384 2024-04-16T05:43:01.744 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:43:01.744 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:43:01.744 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "e306f177-8b77-4420-908c-488fd1242429", 2024-04-16T05:43:01.744 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:43:01.744 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:43:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc900a3d20 msgr2=0x7fdc900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc900a3d20 0x7fdc900a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c01f000 tx=0x7fdc9c020000 comp rx=0 tx=0).stop 2024-04-16T05:43:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4101b80 msgr2=0x7fdca4199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4101b80 0x7fdca4199d90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00d770 tx=0x7fdc8c00dc30 comp rx=0 tx=0).stop 2024-04-16T05:43:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3869250188 shutdown_connections 2024-04-16T05:43:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4105f50 0x7fdca41a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdca4101b80 0x7fdca4199d90 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc900a3d20 0x7fdc900a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 --2- 172.21.15.86:0/3869250188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdca41011e0 0x7fdca4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3869250188 >> 172.21.15.86:0/3869250188 conn(0x7fdca4077ee0 msgr2=0x7fdca40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3869250188 shutdown_connections 2024-04-16T05:43:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.735+0000 7fdca8b9d640 1 -- 172.21.15.86:0/3869250188 wait complete. 2024-04-16T05:43:01.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:43:01.747 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:43:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.855+0000 7f0eacb22640 1 -- 172.21.15.86:0/3631050997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8101140 msgr2=0x7f0ea8101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.855+0000 7f0eacb22640 1 --2- 172.21.15.86:0/3631050997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8101140 0x7f0ea8101520 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009ca0 tx=0x7f0ea002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.855+0000 7f0eacb22640 1 -- 172.21.15.86:0/3631050997 shutdown_connections 2024-04-16T05:43:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.855+0000 7f0eacb22640 1 --2- 172.21.15.86:0/3631050997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ea8106210 0x7f0ea810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.855+0000 7f0eacb22640 1 --2- 172.21.15.86:0/3631050997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea8101af0 0x7f0ea8105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.855+0000 7f0eacb22640 1 --2- 172.21.15.86:0/3631050997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8101140 0x7f0ea8101520 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.855+0000 7f0eacb22640 1 -- 172.21.15.86:0/3631050997 >> 172.21.15.86:0/3631050997 conn(0x7f0ea8078000 msgr2=0x7f0ea8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.855+0000 7f0eacb22640 1 -- 172.21.15.86:0/3631050997 shutdown_connections 2024-04-16T05:43:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.855+0000 7f0eacb22640 1 -- 172.21.15.86:0/3631050997 wait complete. 2024-04-16T05:43:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0eacb22640 1 Processor -- start 2024-04-16T05:43:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0eacb22640 1 -- start start 2024-04-16T05:43:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0eacb22640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ea8101140 0x7f0ea8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0eacb22640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8106210 0x7f0ea8199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0eacb22640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea819a150 0x7f0ea81a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0eacb22640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81112a0 con 0x7f0ea8106210 2024-04-16T05:43:01.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0eacb22640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8111120 con 0x7f0ea8101140 2024-04-16T05:43:01.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0eacb22640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8111420 con 0x7f0ea819a150 2024-04-16T05:43:01.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8106210 0x7f0ea8199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8106210 0x7f0ea8199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59082/0 (socket says 172.21.15.86:59082) 2024-04-16T05:43:01.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea5d9b640 1 -- 172.21.15.86:0/2112202613 learned_addr learned my addr 172.21.15.86:0/2112202613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea659c640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ea8101140 0x7f0ea8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea5d9b640 1 -- 172.21.15.86:0/2112202613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea819a150 msgr2=0x7f0ea81a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea819a150 0x7f0ea81a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea5d9b640 1 -- 172.21.15.86:0/2112202613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ea8101140 msgr2=0x7f0ea8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ea8101140 0x7f0ea8199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea5d9b640 1 -- 172.21.15.86:0/2112202613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea81a4f00 con 0x7f0ea8106210 2024-04-16T05:43:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.856+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8106210 0x7f0ea8199bc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c970 tx=0x7f0e9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.857+0000 7f0e9f7fe640 1 -- 172.21.15.86:0/2112202613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90014070 con 0x7f0ea8106210 2024-04-16T05:43:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.857+0000 7f0eacb22640 1 -- 172.21.15.86:0/2112202613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea81a51f0 con 0x7f0ea8106210 2024-04-16T05:43:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.857+0000 7f0e9f7fe640 1 -- 172.21.15.86:0/2112202613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e90004440 con 0x7f0ea8106210 2024-04-16T05:43:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.857+0000 7f0e9f7fe640 1 -- 172.21.15.86:0/2112202613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9000f450 con 0x7f0ea8106210 2024-04-16T05:43:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.857+0000 7f0eacb22640 1 -- 172.21.15.86:0/2112202613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea81a5790 con 0x7f0ea8106210 2024-04-16T05:43:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.859+0000 7f0e9f7fe640 1 -- 172.21.15.86:0/2112202613 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0e90017070 con 0x7f0ea8106210 2024-04-16T05:43:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.859+0000 7f0eaf7df740 1 Processor -- start 2024-04-16T05:43:01.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.859+0000 7f0eaf7df740 1 -- start start 2024-04-16T05:43:01.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.859+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca96b80 0x55e3fca96f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca974a0 0x55e3fcaa4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fcaa6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc9fc180 con 0x55e3fca974a0 2024-04-16T05:43:01.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc999360 con 0x55e3fca96b80 2024-04-16T05:43:01.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc9f8140 con 0x55e3fcaa4760 2024-04-16T05:43:01.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca974a0 0x55e3fcaa4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca974a0 0x55e3fcaa4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59096/0 (socket says 172.21.15.86:59096) 2024-04-16T05:43:01.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea659c640 1 -- 172.21.15.86:0/3002193622 learned_addr learned my addr 172.21.15.86:0/3002193622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0e9f7fe640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0e940a3d20 0x7f0e940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea6d9d640 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca96b80 0x55e3fca96f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fcaa6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea6d9d640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0e940a3d20 0x7f0e940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea659c640 1 -- 172.21.15.86:0/3002193622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 msgr2=0x55e3fcaa6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea659c640 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fcaa6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea659c640 1 -- 172.21.15.86:0/3002193622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca96b80 msgr2=0x55e3fca96f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea659c640 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca96b80 0x55e3fca96f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea659c640 1 -- 172.21.15.86:0/3002193622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fc7d87d0 con 0x55e3fca974a0 2024-04-16T05:43:01.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fcaa6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:01.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0e9f7fe640 1 -- 172.21.15.86:0/2112202613 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f0e900d0a00 con 0x7f0ea8106210 2024-04-16T05:43:01.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea659c640 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca974a0 0x55e3fcaa4220 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002c700 tx=0x7f0ea0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca2a9d343ed1322 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0ea6d9d640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0e940a3d20 0x7f0e940a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800ec50 tx=0x7f0e98005a10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0e9cff9640 1 -- 172.21.15.86:0/3002193622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0043070 con 0x55e3fca974a0 2024-04-16T05:43:01.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0e9cff9640 1 -- 172.21.15.86:0/3002193622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea0032c90 con 0x55e3fca974a0 2024-04-16T05:43:01.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.860+0000 7f0e9cff9640 1 -- 172.21.15.86:0/3002193622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0039040 con 0x55e3fca974a0 2024-04-16T05:43:01.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3002193622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca974a0 msgr2=0x55e3fcaa4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca974a0 0x55e3fcaa4220 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002c700 tx=0x7f0ea0009920 comp rx=0 tx=0).stop 2024-04-16T05:43:01.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3002193622 shutdown_connections 2024-04-16T05:43:01.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fcaa6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca974a0 0x55e3fcaa4220 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3002193622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca96b80 0x55e3fca96f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3002193622 >> 172.21.15.86:0/3002193622 conn(0x55e3fca91580 msgr2=0x55e3fca91960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3002193622 shutdown_connections 2024-04-16T05:43:01.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3002193622 wait complete. 2024-04-16T05:43:01.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.861+0000 7f0eaf7df740 1 -- start start 2024-04-16T05:43:01.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0eaf7df740 1 Processor -- start 2024-04-16T05:43:01.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca974a0 0x55e3fcaa1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fcaa21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3fcaa74c0 con 0x55e3fca96b80 2024-04-16T05:43:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc9b52c0 con 0x55e3fca974a0 2024-04-16T05:43:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3fcaa7640 con 0x55e3fcaa4760 2024-04-16T05:43:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0eaf7df740 10 client.0 _ll_register_callbacks cb 0x55e3fc99c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59108/0 (socket says 172.21.15.86:59108) 2024-04-16T05:43:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea659c640 1 -- 172.21.15.86:0/3047828455 learned_addr learned my addr 172.21.15.86:0/3047828455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca974a0 0x55e3fcaa1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea6d9d640 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fcaa21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea659c640 1 -- 172.21.15.86:0/3047828455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 msgr2=0x55e3fcaa21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea659c640 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fcaa21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea659c640 1 -- 172.21.15.86:0/3047828455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca974a0 msgr2=0x55e3fcaa1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea659c640 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca974a0 0x55e3fcaa1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea659c640 1 -- 172.21.15.86:0/3047828455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fc70e3f0 con 0x55e3fca96b80 2024-04-16T05:43:01.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0ea659c640 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa1760 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0000f80 tx=0x7f0ea0003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89d432b16d7da1c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0e8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:01.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0e8dffb640 1 -- 172.21.15.86:0/3047828455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea004c030 con 0x55e3fca96b80 2024-04-16T05:43:01.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0e8dffb640 1 -- 172.21.15.86:0/3047828455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f0ea0032960 con 0x55e3fca96b80 2024-04-16T05:43:01.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.862+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3047828455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea0032c90 con 0x55e3fca96b80 2024-04-16T05:43:01.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0e8dffb640 1 -- 172.21.15.86:0/3047828455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ea0059050 con 0x55e3fca96b80 2024-04-16T05:43:01.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0eaf7df740 10 client.18000 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:01.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3047828455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3fc6ce0f0 con 0x55e3fca96b80 2024-04-16T05:43:01.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0e8dffb640 1 -- 172.21.15.86:0/3047828455 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea0032c90 con 0x55e3fca96b80 2024-04-16T05:43:01.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0e8dffb640 1 -- 172.21.15.86:0/3047828455 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ea00487e0 con 0x55e3fca96b80 2024-04-16T05:43:01.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0eaf7df740 10 client.18000 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:01.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0eaf7df740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:01.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0eaf7df740 20 client.18000 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:01.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e3fcab4850 0x55e3fcab6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0eaf7df740 4 client.18000 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:01.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3047828455 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3fca09ba0 con 0x55e3fcab4850 2024-04-16T05:43:01.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.863+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e3fcab4850 0x55e3fcab6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0e8dffb640 10 client.18000 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:01.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e3fcab4850 0x55e3fcab6c40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8356e90ef0641bf4 server_cookie=7c47cca42a7b1cc7 in_seq=0 out_seq=0 2024-04-16T05:43:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0e8dffb640 1 -- 172.21.15.86:0/3047828455 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3fca09ba0 con 0x55e3fcab4850 2024-04-16T05:43:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0e8dffb640 10 client.18000 handle_command_reply: tid=0 2024-04-16T05:43:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0eaf7df740 1 client.18000 shutdown 2024-04-16T05:43:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3047828455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 msgr2=0x55e3fcaa1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa1760 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0000f80 tx=0x7f0ea0003e30 comp rx=0 tx=0).stop 2024-04-16T05:43:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3047828455 shutdown_connections 2024-04-16T05:43:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e3fcab4850 0x55e3fcab6c40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fcaa21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0e8dffb640 0 client.18000 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca974a0 0x55e3fcaa1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/3047828455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa1760 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.864+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3047828455 >> 172.21.15.86:0/3047828455 conn(0x55e3fca91580 msgr2=0x55e3fca932d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.865+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3047828455 shutdown_connections 2024-04-16T05:43:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.865+0000 7f0eaf7df740 1 -- 172.21.15.86:0/3047828455 wait complete. 2024-04-16T05:43:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.865+0000 7f0eaf7df740 20 client.18000 trim_cache size 0 max 16384 2024-04-16T05:43:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0eaf7df740 1 Processor -- start 2024-04-16T05:43:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0eaf7df740 1 -- start start 2024-04-16T05:43:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca974a0 0x55e3fcab03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fca905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81ad310 con 0x55e3fca96b80 2024-04-16T05:43:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81aee50 con 0x55e3fca974a0 2024-04-16T05:43:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81ad9e0 con 0x55e3fcaa4760 2024-04-16T05:43:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59112/0 (socket says 172.21.15.86:59112) 2024-04-16T05:43:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea659c640 1 -- 172.21.15.86:0/385284568 learned_addr learned my addr 172.21.15.86:0/385284568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea6d9d640 1 --2- 172.21.15.86:0/385284568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fca905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea659c640 1 -- 172.21.15.86:0/385284568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 msgr2=0x55e3fca905e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea659c640 1 --2- 172.21.15.86:0/385284568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fca905e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea659c640 1 -- 172.21.15.86:0/385284568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca974a0 msgr2=0x55e3fcab03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea659c640 1 --2- 172.21.15.86:0/385284568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca974a0 0x55e3fcab03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea659c640 1 -- 172.21.15.86:0/385284568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fca8cc20 con 0x55e3fca96b80 2024-04-16T05:43:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea6d9d640 1 --2- 172.21.15.86:0/385284568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fca905e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.882+0000 7f0ea659c640 1 --2- 172.21.15.86:0/385284568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa3e90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0ea004b040 tx=0x7f0ea00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191baad06a6c045 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0e8ffff640 1 -- 172.21.15.86:0/385284568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea004c070 con 0x55e3fca96b80 2024-04-16T05:43:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0e8ffff640 1 -- 172.21.15.86:0/385284568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea0048930 con 0x55e3fca96b80 2024-04-16T05:43:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0e8ffff640 1 -- 172.21.15.86:0/385284568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0039070 con 0x55e3fca96b80 2024-04-16T05:43:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 -- 172.21.15.86:0/385284568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 msgr2=0x55e3fcaa3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/385284568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa3e90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0ea004b040 tx=0x7f0ea00021d0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 -- 172.21.15.86:0/385284568 shutdown_connections 2024-04-16T05:43:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/385284568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fcaa4760 0x55e3fca905e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/385284568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fca974a0 0x55e3fcab03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/385284568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaa3e90 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 -- 172.21.15.86:0/385284568 >> 172.21.15.86:0/385284568 conn(0x55e3fca91580 msgr2=0x55e3fca8fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 -- 172.21.15.86:0/385284568 shutdown_connections 2024-04-16T05:43:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 -- 172.21.15.86:0/385284568 wait complete. 2024-04-16T05:43:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.883+0000 7f0eaf7df740 1 -- start start 2024-04-16T05:43:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0eaf7df740 1 Processor -- start 2024-04-16T05:43:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaaa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fca974a0 0x55e3fcaaa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0eaf7df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fcaa4760 0x55e3fcaaaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81ae780 con 0x55e3fca96b80 2024-04-16T05:43:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81ae0b0 con 0x55e3fcaa4760 2024-04-16T05:43:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0eaf7df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81ab310 con 0x55e3fca974a0 2024-04-16T05:43:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0eaf7df740 10 client.0 _ll_register_callbacks cb 0x7f0ea0032450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:01.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaaa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaaa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59118/0 (socket says 172.21.15.86:59118) 2024-04-16T05:43:01.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea659c640 1 -- 172.21.15.86:0/826370753 learned_addr learned my addr 172.21.15.86:0/826370753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:01.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea6d9d640 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fcaa4760 0x55e3fcaaaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea659c640 1 -- 172.21.15.86:0/826370753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fca974a0 msgr2=0x55e3fcaaa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea659c640 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fca974a0 0x55e3fcaaa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea659c640 1 -- 172.21.15.86:0/826370753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fcaa4760 msgr2=0x55e3fcaaaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:01.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea659c640 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fcaa4760 0x55e3fcaaaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea659c640 1 -- 172.21.15.86:0/826370753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fcb19ce0 con 0x55e3fca96b80 2024-04-16T05:43:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea6d9d640 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fcaa4760 0x55e3fcaaaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0ea659c640 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaaa4b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002c630 tx=0x7f0ea0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3a8ce409061799 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0e8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0e8dffb640 1 -- 172.21.15.86:0/826370753 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea004c070 con 0x55e3fca96b80 2024-04-16T05:43:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0e8dffb640 1 -- 172.21.15.86:0/826370753 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f0ea0004b40 con 0x55e3fca96b80 2024-04-16T05:43:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.884+0000 7f0eaf7df740 1 -- 172.21.15.86:0/826370753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea0048930 con 0x55e3fca96b80 2024-04-16T05:43:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0e8dffb640 1 -- 172.21.15.86:0/826370753 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ea0039040 con 0x55e3fca96b80 2024-04-16T05:43:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0eaf7df740 10 client.18006 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0eaf7df740 1 -- 172.21.15.86:0/826370753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3fcb1af20 con 0x55e3fca96b80 2024-04-16T05:43:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0e8dffb640 1 -- 172.21.15.86:0/826370753 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea0048930 con 0x55e3fca96b80 2024-04-16T05:43:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0e8dffb640 1 -- 172.21.15.86:0/826370753 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ea0005440 con 0x55e3fca96b80 2024-04-16T05:43:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0eaf7df740 10 client.18006 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0eaf7df740 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0eaf7df740 20 client.18006 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e3fcb1b3f0 0x55e3fcb1d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0eaf7df740 4 client.18006 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:43:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0eaf7df740 1 -- 172.21.15.86:0/826370753 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e3fca03a10 con 0x55e3fcb1b3f0 2024-04-16T05:43:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.885+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e3fcb1b3f0 0x55e3fcb1d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0ea5d9b640 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e3fcb1b3f0 0x55e3fcb1d830 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b85f4ea790cf3d3a server_cookie=5fa5dd83864dfcde in_seq=0 out_seq=0 2024-04-16T05:43:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0e8dffb640 10 client.18006 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0e8dffb640 1 -- 172.21.15.86:0/826370753 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e3fca03a10 con 0x55e3fcb1b3f0 2024-04-16T05:43:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0e8dffb640 10 client.18006 handle_command_reply: tid=0 2024-04-16T05:43:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 client.18006 shutdown 2024-04-16T05:43:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 -- 172.21.15.86:0/826370753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 msgr2=0x55e3fcaaa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaaa4b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002c630 tx=0x7f0ea0005d80 comp rx=0 tx=0).stop 2024-04-16T05:43:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 -- 172.21.15.86:0/826370753 shutdown_connections 2024-04-16T05:43:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e3fcb1b3f0 0x55e3fcb1d830 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e3fcaa4760 0x55e3fcaaaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fca974a0 0x55e3fcaaa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0e8dffb640 0 client.18006 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 --2- 172.21.15.86:0/826370753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3fca96b80 0x55e3fcaaa4b0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 -- 172.21.15.86:0/826370753 >> 172.21.15.86:0/826370753 conn(0x55e3fca91580 msgr2=0x55e3fca90760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 -- 172.21.15.86:0/826370753 shutdown_connections 2024-04-16T05:43:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 1 -- 172.21.15.86:0/826370753 wait complete. 2024-04-16T05:43:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eaf7df740 20 client.18006 trim_cache size 0 max 16384 2024-04-16T05:43:01.896 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:43:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:43:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:43:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "e306f177-8b77-4420-908c-488fd1242429": { 2024-04-16T05:43:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:43:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e306f177-8b77-4420-908c-488fd1242429", 2024-04-16T05:43:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:43:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:43:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:43:01.897 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:43:01.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eacb22640 1 -- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0e940a3d20 msgr2=0x7f0e940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eacb22640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0e940a3d20 0x7f0e940a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800ec50 tx=0x7f0e98005a10 comp rx=0 tx=0).stop 2024-04-16T05:43:01.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eacb22640 1 -- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8106210 msgr2=0x7f0ea8199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:01.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.886+0000 7f0eacb22640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8106210 0x7f0ea8199bc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c970 tx=0x7f0e9000ce30 comp rx=0 tx=0).stop 2024-04-16T05:43:01.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.887+0000 7f0eacb22640 1 -- 172.21.15.86:0/2112202613 shutdown_connections 2024-04-16T05:43:01.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.887+0000 7f0eacb22640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea819a150 0x7f0ea81a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.887+0000 7f0eacb22640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea8106210 0x7f0ea8199bc0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.887+0000 7f0eacb22640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0e940a3d20 0x7f0e940a61e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.887+0000 7f0eacb22640 1 --2- 172.21.15.86:0/2112202613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ea8101140 0x7f0ea8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:01.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.887+0000 7f0eacb22640 1 -- 172.21.15.86:0/2112202613 >> 172.21.15.86:0/2112202613 conn(0x7f0ea8078000 msgr2=0x7f0ea80fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:01.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.887+0000 7f0eacb22640 1 -- 172.21.15.86:0/2112202613 shutdown_connections 2024-04-16T05:43:01.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:01.887+0000 7f0eacb22640 1 -- 172.21.15.86:0/2112202613 wait complete. 2024-04-16T05:43:01.901 INFO:tasks.cephfs.filesystem:scrub status for tag:e306f177-8b77-4420-908c-488fd1242429 - {'path': '/', 'tag': 'e306f177-8b77-4420-908c-488fd1242429', 'options': 'recursive,force'} 2024-04-16T05:43:02.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:02 smithi086 ceph-mon[32028]: pgmap v237: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:02.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:02 smithi196 ceph-mon[36373]: pgmap v237: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:02.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:02 smithi194 ceph-mon[35865]: pgmap v237: 129 pgs: 129 active+clean; 1.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:03.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:03 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:43:03.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:03 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:03.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:03 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:43:03.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:03 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:03.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:03 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:43:03.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:03 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:03.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:03 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:43:03.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:03 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:03.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:03 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:43:03.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:03 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:03.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:03 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:43:03.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:03 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:04.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:04 smithi086 ceph-mon[32028]: pgmap v238: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:43:04.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:04 smithi196 ceph-mon[36373]: pgmap v238: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:43:04.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:04 smithi194 ceph-mon[35865]: pgmap v238: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:43:05.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:05 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:43:05.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:05 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:43:05.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:05 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:43:06.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:06 smithi086 ceph-mon[32028]: pgmap v239: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:43:06.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:06 smithi196 ceph-mon[36373]: pgmap v239: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:43:06.557 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:06 smithi194 ceph-mon[35865]: pgmap v239: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:43:08.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:08 smithi086 ceph-mon[32028]: pgmap v240: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:43:08.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:08 smithi196 ceph-mon[36373]: pgmap v240: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:43:08.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:08 smithi194 ceph-mon[35865]: pgmap v240: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-16T05:43:10.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:10 smithi086 ceph-mon[32028]: pgmap v241: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:43:10.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:10 smithi196 ceph-mon[36373]: pgmap v241: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:43:10.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:10 smithi194 ceph-mon[35865]: pgmap v241: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:43:11.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:11 smithi086 ceph-mon[32028]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T05:43:11.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:11 smithi196 ceph-mon[36373]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T05:43:11.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:11 smithi194 ceph-mon[35865]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T05:43:12.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:12 smithi196 ceph-mon[36373]: pgmap v243: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 32 op/s 2024-04-16T05:43:12.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:12 smithi194 ceph-mon[35865]: pgmap v243: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 32 op/s 2024-04-16T05:43:12.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:12 smithi086 ceph-mon[32028]: pgmap v243: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 32 op/s 2024-04-16T05:43:14.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:14 smithi196 ceph-mon[36373]: pgmap v244: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:43:14.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:14 smithi194 ceph-mon[35865]: pgmap v244: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:43:14.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:14 smithi086 ceph-mon[32028]: pgmap v244: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:43:16.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:15 smithi196 ceph-mon[36373]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T05:43:16.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:15 smithi196 ceph-mon[36373]: pgmap v246: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:43:16.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:15 smithi194 ceph-mon[35865]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T05:43:16.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:15 smithi194 ceph-mon[35865]: pgmap v246: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:43:16.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:15 smithi086 ceph-mon[32028]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T05:43:16.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:15 smithi086 ceph-mon[32028]: pgmap v246: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:43:18.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:18 smithi194 ceph-mon[35865]: pgmap v247: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:43:18.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:18 smithi086 ceph-mon[32028]: pgmap v247: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:43:18.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:18 smithi196 ceph-mon[36373]: pgmap v247: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:43:20.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:20 smithi194 ceph-mon[35865]: pgmap v248: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-16T05:43:20.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:20 smithi086 ceph-mon[32028]: pgmap v248: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-16T05:43:20.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:20 smithi196 ceph-mon[36373]: pgmap v248: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-16T05:43:22.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:22 smithi194 ceph-mon[35865]: pgmap v249: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:43:22.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:22 smithi086 ceph-mon[32028]: pgmap v249: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:43:22.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:22 smithi196 ceph-mon[36373]: pgmap v249: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:43:24.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:24 smithi194 ceph-mon[35865]: pgmap v250: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:43:24.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:24 smithi086 ceph-mon[32028]: pgmap v250: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:43:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:24 smithi196 ceph-mon[36373]: pgmap v250: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:43:26.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:26 smithi194 ceph-mon[35865]: pgmap v251: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:43:26.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:26 smithi086 ceph-mon[32028]: pgmap v251: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:43:26.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:26 smithi196 ceph-mon[36373]: pgmap v251: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:43:28.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:28 smithi194 ceph-mon[35865]: pgmap v252: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:43:28.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:28 smithi086 ceph-mon[32028]: pgmap v252: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:43:28.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:28 smithi196 ceph-mon[36373]: pgmap v252: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:43:29.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:29 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:43:29.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:29 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:43:29.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:29 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:43:30.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:30 smithi086 ceph-mon[32028]: pgmap v253: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:43:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:30 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:43:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:30 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:43:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:30 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:43:30.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:30 smithi196 ceph-mon[36373]: pgmap v253: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:43:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:30 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:43:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:30 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:43:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:30 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:43:30.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:30 smithi194 ceph-mon[35865]: pgmap v253: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:43:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:30 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:43:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:30 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:43:30.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:30 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:43:31.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:43:31.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:43:32.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.022+0000 7f276a288640 1 -- 172.21.15.86:0/60957523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764107310 msgr2=0x7f276410eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.022+0000 7f276a288640 1 --2- 172.21.15.86:0/60957523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764107310 0x7f276410eee0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f275c009ca0 tx=0x7f275c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.022+0000 7f276a288640 1 -- 172.21.15.86:0/60957523 shutdown_connections 2024-04-16T05:43:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.022+0000 7f276a288640 1 --2- 172.21.15.86:0/60957523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764107310 0x7f276410eee0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.022+0000 7f276a288640 1 --2- 172.21.15.86:0/60957523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764102bf0 0x7f2764106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.022+0000 7f276a288640 1 --2- 172.21.15.86:0/60957523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2764102240 0x7f2764102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.022+0000 7f276a288640 1 -- 172.21.15.86:0/60957523 >> 172.21.15.86:0/60957523 conn(0x7f2764079450 msgr2=0x7f2764079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.022+0000 7f276a288640 1 -- 172.21.15.86:0/60957523 shutdown_connections 2024-04-16T05:43:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.022+0000 7f276a288640 1 -- 172.21.15.86:0/60957523 wait complete. 2024-04-16T05:43:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f276a288640 1 Processor -- start 2024-04-16T05:43:32.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f276a288640 1 -- start start 2024-04-16T05:43:32.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f276a288640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764102240 0x7f276419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f276a288640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764107310 0x7f276419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f276a288640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27641986f0 0x7f2764198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f276a288640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27641123d0 con 0x7f2764102240 2024-04-16T05:43:32.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f276a288640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2764112250 con 0x7f27641986f0 2024-04-16T05:43:32.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f276a288640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2764112550 con 0x7f2764107310 2024-04-16T05:43:32.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f2763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764102240 0x7f276419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.023+0000 7f2763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764102240 0x7f276419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56484/0 (socket says 172.21.15.86:56484) 2024-04-16T05:43:32.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f2763fff640 1 -- 172.21.15.86:0/634399966 learned_addr learned my addr 172.21.15.86:0/634399966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f27637fe640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764107310 0x7f276419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f2768825640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27641986f0 0x7f2764198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f2763fff640 1 -- 172.21.15.86:0/634399966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764107310 msgr2=0x7f276419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f2763fff640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764107310 0x7f276419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f2763fff640 1 -- 172.21.15.86:0/634399966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27641986f0 msgr2=0x7f2764198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f2763fff640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27641986f0 0x7f2764198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f2763fff640 1 -- 172.21.15.86:0/634399966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27641992f0 con 0x7f2764102240 2024-04-16T05:43:32.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f2768825640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27641986f0 0x7f2764198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:32.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f2763fff640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764102240 0x7f276419e630 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f275400d950 tx=0x7f275400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f27617fa640 1 -- 172.21.15.86:0/634399966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754017030 con 0x7f2764102240 2024-04-16T05:43:32.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f27617fa640 1 -- 172.21.15.86:0/634399966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2754004020 con 0x7f2764102240 2024-04-16T05:43:32.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f276a288640 1 -- 172.21.15.86:0/634399966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27641a53d0 con 0x7f2764102240 2024-04-16T05:43:32.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f27617fa640 1 -- 172.21.15.86:0/634399966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27540047a0 con 0x7f2764102240 2024-04-16T05:43:32.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.024+0000 7f276a288640 1 -- 172.21.15.86:0/634399966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27641a57d0 con 0x7f2764102240 2024-04-16T05:43:32.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f27617fa640 1 -- 172.21.15.86:0/634399966 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27540041d0 con 0x7f2764102240 2024-04-16T05:43:32.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f276ce7c740 1 Processor -- start 2024-04-16T05:43:32.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f276ce7c740 1 -- start start 2024-04-16T05:43:32.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f06e50 0x557355f07230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f07770 0x557355f0fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f10350 0x557355f12740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f276ce7c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557355e6c180 con 0x557355f10350 2024-04-16T05:43:32.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f276ce7c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557355e09360 con 0x557355f07770 2024-04-16T05:43:32.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f276ce7c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557355e68140 con 0x557355f06e50 2024-04-16T05:43:32.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.027+0000 7f27637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f10350 0x557355f12740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27617fa640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f274c0a3d20 0x7f274c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f10350 0x557355f12740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56496/0 (socket says 172.21.15.86:56496) 2024-04-16T05:43:32.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27637fe640 1 -- 172.21.15.86:0/1655333583 learned_addr learned my addr 172.21.15.86:0/1655333583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f2763fff640 1 --2- 172.21.15.86:0/1655333583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f07770 0x557355f0fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f2768825640 1 --2- 172.21.15.86:0/1655333583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f06e50 0x557355f07230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f2768825640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f274c0a3d20 0x7f274c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27637fe640 1 -- 172.21.15.86:0/1655333583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f06e50 msgr2=0x557355f07230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27637fe640 1 --2- 172.21.15.86:0/1655333583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f06e50 0x557355f07230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27637fe640 1 -- 172.21.15.86:0/1655333583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f07770 msgr2=0x557355f0fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27637fe640 1 --2- 172.21.15.86:0/1655333583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f07770 0x557355f0fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27617fa640 1 -- 172.21.15.86:0/634399966 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f27540d0680 con 0x7f2764102240 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27637fe640 1 -- 172.21.15.86:0/1655333583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557355c487d0 con 0x557355f10350 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f27637fe640 1 --2- 172.21.15.86:0/1655333583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f10350 0x557355f12740 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f275800bbb0 tx=0x7f275800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638cc947d1b37386 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f2768825640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f274c0a3d20 0x7f274c0a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f275c004af0 tx=0x7f275c0095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f274a7fc640 1 -- 172.21.15.86:0/1655333583 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758015070 con 0x557355f10350 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f274a7fc640 1 -- 172.21.15.86:0/1655333583 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f275801a940 con 0x557355f10350 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.028+0000 7f274a7fc640 1 -- 172.21.15.86:0/1655333583 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758011920 con 0x557355f10350 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 -- 172.21.15.86:0/1655333583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f10350 msgr2=0x557355f12740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 --2- 172.21.15.86:0/1655333583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f10350 0x557355f12740 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f275800bbb0 tx=0x7f275800f730 comp rx=0 tx=0).stop 2024-04-16T05:43:32.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 -- 172.21.15.86:0/1655333583 shutdown_connections 2024-04-16T05:43:32.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 --2- 172.21.15.86:0/1655333583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f10350 0x557355f12740 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 --2- 172.21.15.86:0/1655333583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f07770 0x557355f0fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 --2- 172.21.15.86:0/1655333583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f06e50 0x557355f07230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 -- 172.21.15.86:0/1655333583 >> 172.21.15.86:0/1655333583 conn(0x557355f01580 msgr2=0x557355f01960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 -- 172.21.15.86:0/1655333583 shutdown_connections 2024-04-16T05:43:32.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 -- 172.21.15.86:0/1655333583 wait complete. 2024-04-16T05:43:32.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.029+0000 7f276ce7c740 1 -- start start 2024-04-16T05:43:32.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f276ce7c740 1 Processor -- start 2024-04-16T05:43:32.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f0d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f07770 0x557355f0d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f0dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f276ce7c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557355f13140 con 0x557355f06e50 2024-04-16T05:43:32.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f276ce7c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557355e252c0 con 0x557355f10350 2024-04-16T05:43:32.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f276ce7c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557355f132c0 con 0x557355f07770 2024-04-16T05:43:32.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f276ce7c740 10 client.0 _ll_register_callbacks cb 0x557355e0c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f0d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f0d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56512/0 (socket says 172.21.15.86:56512) 2024-04-16T05:43:32.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2763fff640 1 -- 172.21.15.86:0/4271117231 learned_addr learned my addr 172.21.15.86:0/4271117231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2768825640 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f0dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f27637fe640 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f07770 0x557355f0d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2763fff640 1 -- 172.21.15.86:0/4271117231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f07770 msgr2=0x557355f0d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2763fff640 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f07770 0x557355f0d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2763fff640 1 -- 172.21.15.86:0/4271117231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 msgr2=0x557355f0dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2763fff640 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f0dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2763fff640 1 -- 172.21.15.86:0/4271117231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557355b7e3f0 con 0x557355f06e50 2024-04-16T05:43:32.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f2763fff640 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f0d320 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f275409bf50 tx=0x7f275406ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ccbc447d87dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.030+0000 7f27437fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f27437fe640 1 -- 172.21.15.86:0/4271117231 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275409bbb0 con 0x557355f06e50 2024-04-16T05:43:32.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f27437fe640 1 -- 172.21.15.86:0/4271117231 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f27540988c0 con 0x557355f06e50 2024-04-16T05:43:32.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f276ce7c740 1 -- 172.21.15.86:0/4271117231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f275801a940 con 0x557355f06e50 2024-04-16T05:43:32.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f27437fe640 1 -- 172.21.15.86:0/4271117231 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2754066d60 con 0x557355f06e50 2024-04-16T05:43:32.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f276ce7c740 10 client.18018 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f276ce7c740 1 -- 172.21.15.86:0/4271117231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557355b3e0f0 con 0x557355f06e50 2024-04-16T05:43:32.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f27437fe640 1 -- 172.21.15.86:0/4271117231 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f275801a940 con 0x557355f06e50 2024-04-16T05:43:32.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f27437fe640 1 -- 172.21.15.86:0/4271117231 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2754019310 con 0x557355f06e50 2024-04-16T05:43:32.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f276ce7c740 10 client.18018 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f276ce7c740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:32.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f276ce7c740 20 client.18018 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f276ce7c740 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557355f206d0 0x557355f22b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f276ce7c740 4 client.18018 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:32.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f276ce7c740 1 -- 172.21.15.86:0/4271117231 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557355e79ba0 con 0x557355f206d0 2024-04-16T05:43:32.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.031+0000 7f27637fe640 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557355f206d0 0x557355f22b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.032+0000 7f27637fe640 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557355f206d0 0x557355f22b20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68b2c864c03b0e30 server_cookie=95308c55f7daf76f in_seq=0 out_seq=0 2024-04-16T05:43:32.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.032+0000 7f27437fe640 10 client.18018 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:32.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f27437fe640 1 -- 172.21.15.86:0/4271117231 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557355e79ba0 con 0x557355f206d0 2024-04-16T05:43:32.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f27437fe640 10 client.18018 handle_command_reply: tid=0 2024-04-16T05:43:32.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 client.18018 shutdown 2024-04-16T05:43:32.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 -- 172.21.15.86:0/4271117231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 msgr2=0x557355f0d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f0d320 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f275409bf50 tx=0x7f275406ac00 comp rx=0 tx=0).stop 2024-04-16T05:43:32.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 -- 172.21.15.86:0/4271117231 shutdown_connections 2024-04-16T05:43:32.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557355f206d0 0x557355f22b20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f0dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f07770 0x557355f0d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 --2- 172.21.15.86:0/4271117231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f0d320 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f27437fe640 0 client.18018 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:32.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 -- 172.21.15.86:0/4271117231 >> 172.21.15.86:0/4271117231 conn(0x557355f01580 msgr2=0x557355f032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 -- 172.21.15.86:0/4271117231 shutdown_connections 2024-04-16T05:43:32.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 1 -- 172.21.15.86:0/4271117231 wait complete. 2024-04-16T05:43:32.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.033+0000 7f276ce7c740 20 client.18018 trim_cache size 0 max 16384 2024-04-16T05:43:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.050+0000 7f276ce7c740 1 Processor -- start 2024-04-16T05:43:32.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.050+0000 7f276ce7c740 1 -- start start 2024-04-16T05:43:32.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.050+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f1c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.050+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f07770 0x557355f0dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.050+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.050+0000 7f276ce7c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27641af640 con 0x557355f06e50 2024-04-16T05:43:32.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.050+0000 7f276ce7c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27641b1180 con 0x557355f10350 2024-04-16T05:43:32.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.050+0000 7f276ce7c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27641afd10 con 0x557355f07770 2024-04-16T05:43:32.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f1c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f1c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56524/0 (socket says 172.21.15.86:56524) 2024-04-16T05:43:32.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2763fff640 1 -- 172.21.15.86:0/2471441481 learned_addr learned my addr 172.21.15.86:0/2471441481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2763fff640 1 -- 172.21.15.86:0/2471441481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f07770 msgr2=0x557355f0dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2763fff640 1 --2- 172.21.15.86:0/2471441481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f07770 0x557355f0dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2763fff640 1 -- 172.21.15.86:0/2471441481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 msgr2=0x557355f283f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2763fff640 1 --2- 172.21.15.86:0/2471441481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2763fff640 1 -- 172.21.15.86:0/2471441481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557355f12740 con 0x557355f06e50 2024-04-16T05:43:32.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2763fff640 1 --2- 172.21.15.86:0/2471441481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f1c060 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f27540ce000 tx=0x7f275409a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a551b6269c1330 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2749ffb640 1 -- 172.21.15.86:0/2471441481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275409bbb0 con 0x557355f06e50 2024-04-16T05:43:32.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2749ffb640 1 -- 172.21.15.86:0/2471441481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f275409c950 con 0x557355f06e50 2024-04-16T05:43:32.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.051+0000 7f2749ffb640 1 -- 172.21.15.86:0/2471441481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275409c020 con 0x557355f06e50 2024-04-16T05:43:32.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 -- 172.21.15.86:0/2471441481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 msgr2=0x557355f1c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2471441481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f1c060 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f27540ce000 tx=0x7f275409a760 comp rx=0 tx=0).stop 2024-04-16T05:43:32.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 -- 172.21.15.86:0/2471441481 shutdown_connections 2024-04-16T05:43:32.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2471441481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f283f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2471441481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f07770 0x557355f0dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2471441481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f06e50 0x557355f1c060 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 -- 172.21.15.86:0/2471441481 >> 172.21.15.86:0/2471441481 conn(0x557355f01580 msgr2=0x557355f00760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 -- 172.21.15.86:0/2471441481 shutdown_connections 2024-04-16T05:43:32.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 -- 172.21.15.86:0/2471441481 wait complete. 2024-04-16T05:43:32.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 -- start start 2024-04-16T05:43:32.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 Processor -- start 2024-04-16T05:43:32.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.052+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f06e50 0x557355f187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f07770 0x557355f15b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f276ce7c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f16080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f276ce7c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27641b0ab0 con 0x557355f07770 2024-04-16T05:43:32.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f276ce7c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27641b03e0 con 0x557355f10350 2024-04-16T05:43:32.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f276ce7c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27641ad640 con 0x557355f06e50 2024-04-16T05:43:32.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f276ce7c740 10 client.0 _ll_register_callbacks cb 0x7f275400ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f07770 0x557355f15b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f07770 0x557355f15b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56532/0 (socket says 172.21.15.86:56532) 2024-04-16T05:43:32.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27637fe640 1 -- 172.21.15.86:0/2583627392 learned_addr learned my addr 172.21.15.86:0/2583627392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27637fe640 1 -- 172.21.15.86:0/2583627392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f06e50 msgr2=0x557355f187b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27637fe640 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f06e50 0x557355f187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27637fe640 1 -- 172.21.15.86:0/2583627392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 msgr2=0x557355f16080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f2768825640 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f16080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27637fe640 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f16080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27637fe640 1 -- 172.21.15.86:0/2583627392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557355f166f0 con 0x557355f07770 2024-04-16T05:43:32.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f2768825640 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f16080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:32.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27637fe640 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f07770 0x557355f15b40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x557355f02710 tx=0x7f2758005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7aafd9d573855bd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27437fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27437fe640 1 -- 172.21.15.86:0/2583627392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275802d060 con 0x557355f07770 2024-04-16T05:43:32.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27437fe640 1 -- 172.21.15.86:0/2583627392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f2758005240 con 0x557355f07770 2024-04-16T05:43:32.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f276ce7c740 1 -- 172.21.15.86:0/2583627392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f275409c950 con 0x557355f07770 2024-04-16T05:43:32.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f27437fe640 1 -- 172.21.15.86:0/2583627392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2758034050 con 0x557355f07770 2024-04-16T05:43:32.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f276ce7c740 10 client.18024 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.053+0000 7f276ce7c740 1 -- 172.21.15.86:0/2583627392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557355f86ba0 con 0x557355f07770 2024-04-16T05:43:32.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f27437fe640 1 -- 172.21.15.86:0/2583627392 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f275409c950 con 0x557355f07770 2024-04-16T05:43:32.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f27437fe640 1 -- 172.21.15.86:0/2583627392 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2758004640 con 0x557355f07770 2024-04-16T05:43:32.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f276ce7c740 10 client.18024 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f276ce7c740 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:32.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f276ce7c740 20 client.18024 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557355f87040 0x557355f89430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f276ce7c740 4 client.18024 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f276ce7c740 1 -- 172.21.15.86:0/2583627392 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557355e73a10 con 0x557355f87040 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f2763fff640 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557355f87040 0x557355f89430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f27437fe640 10 client.18024 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f2763fff640 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557355f87040 0x557355f89430 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891e4da5248038e3 server_cookie=e92817f8e9df1a46 in_seq=0 out_seq=0 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f27437fe640 1 -- 172.21.15.86:0/2583627392 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557355e73a10 con 0x557355f87040 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f27437fe640 10 client.18024 handle_command_reply: tid=0 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.054+0000 7f276ce7c740 1 client.18024 shutdown 2024-04-16T05:43:32.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 -- 172.21.15.86:0/2583627392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f07770 msgr2=0x557355f15b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f07770 0x557355f15b40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x557355f02710 tx=0x7f2758005ce0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 -- 172.21.15.86:0/2583627392 shutdown_connections 2024-04-16T05:43:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x557355f87040 0x557355f89430 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557355f10350 0x557355f16080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f27437fe640 0 client.18024 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557355f07770 0x557355f15b40 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 --2- 172.21.15.86:0/2583627392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557355f06e50 0x557355f187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 -- 172.21.15.86:0/2583627392 >> 172.21.15.86:0/2583627392 conn(0x557355f01580 msgr2=0x557355f02740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 -- 172.21.15.86:0/2583627392 shutdown_connections 2024-04-16T05:43:32.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 1 -- 172.21.15.86:0/2583627392 wait complete. 2024-04-16T05:43:32.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276ce7c740 20 client.18024 trim_cache size 0 max 16384 2024-04-16T05:43:32.063 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:43:32.063 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:43:32.063 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:43:32.063 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:43:32.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 -- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f274c0a3d20 msgr2=0x7f274c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f274c0a3d20 0x7f274c0a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f275c004af0 tx=0x7f275c0095c0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 -- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764102240 msgr2=0x7f276419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764102240 0x7f276419e630 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f275400d950 tx=0x7f275400de10 comp rx=0 tx=0).stop 2024-04-16T05:43:32.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 -- 172.21.15.86:0/634399966 shutdown_connections 2024-04-16T05:43:32.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f274c0a3d20 0x7f274c0a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27641986f0 0x7f2764198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764107310 0x7f276419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 --2- 172.21.15.86:0/634399966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764102240 0x7f276419e630 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.055+0000 7f276a288640 1 -- 172.21.15.86:0/634399966 >> 172.21.15.86:0/634399966 conn(0x7f2764079450 msgr2=0x7f27640ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.056+0000 7f276a288640 1 -- 172.21.15.86:0/634399966 shutdown_connections 2024-04-16T05:43:32.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.056+0000 7f276a288640 1 -- 172.21.15.86:0/634399966 wait complete. 2024-04-16T05:43:32.066 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:43:32.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:43:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 -- 172.21.15.86:0/3158988986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c101b20 msgr2=0x7fd04c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 --2- 172.21.15.86:0/3158988986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c101b20 0x7fd04c105b60 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd034009ca0 tx=0x7fd03402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 -- 172.21.15.86:0/3158988986 shutdown_connections 2024-04-16T05:43:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 --2- 172.21.15.86:0/3158988986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd04c106240 0x7fd04c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 --2- 172.21.15.86:0/3158988986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c101b20 0x7fd04c105b60 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 --2- 172.21.15.86:0/3158988986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c101170 0x7fd04c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 -- 172.21.15.86:0/3158988986 >> 172.21.15.86:0/3158988986 conn(0x7fd04c077280 msgr2=0x7fd04c077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 -- 172.21.15.86:0/3158988986 shutdown_connections 2024-04-16T05:43:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 -- 172.21.15.86:0/3158988986 wait complete. 2024-04-16T05:43:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.174+0000 7fd05300c640 1 Processor -- start 2024-04-16T05:43:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd05300c640 1 -- start start 2024-04-16T05:43:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd05300c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c101170 0x7fd04c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd05300c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c106240 0x7fd04c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd05300c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd04c19a280 0x7fd04c1a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd05300c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c111330 con 0x7fd04c106240 2024-04-16T05:43:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd05300c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1111b0 con 0x7fd04c19a280 2024-04-16T05:43:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd05300c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1114b0 con 0x7fd04c101170 2024-04-16T05:43:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c106240 0x7fd04c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c106240 0x7fd04c199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56558/0 (socket says 172.21.15.86:56558) 2024-04-16T05:43:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd04bfff640 1 -- 172.21.15.86:0/1046696753 learned_addr learned my addr 172.21.15.86:0/1046696753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd050da8640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c101170 0x7fd04c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd04bfff640 1 -- 172.21.15.86:0/1046696753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c101170 msgr2=0x7fd04c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd04bfff640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c101170 0x7fd04c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd0515a9640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd04c19a280 0x7fd04c1a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd04bfff640 1 -- 172.21.15.86:0/1046696753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd04c19a280 msgr2=0x7fd04c1a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd04bfff640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd04c19a280 0x7fd04c1a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd04bfff640 1 -- 172.21.15.86:0/1046696753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c1a5030 con 0x7fd04c106240 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd04bfff640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c106240 0x7fd04c199cf0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd034005cc0 tx=0x7fd034032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd049ffb640 1 -- 172.21.15.86:0/1046696753 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034041020 con 0x7fd04c106240 2024-04-16T05:43:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c1a52c0 con 0x7fd04c106240 2024-04-16T05:43:32.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.175+0000 7fd049ffb640 1 -- 172.21.15.86:0/1046696753 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd034034cd0 con 0x7fd04c106240 2024-04-16T05:43:32.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.176+0000 7fd049ffb640 1 -- 172.21.15.86:0/1046696753 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034039040 con 0x7fd04c106240 2024-04-16T05:43:32.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.176+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c1a5800 con 0x7fd04c106240 2024-04-16T05:43:32.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.177+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c069270 con 0x7fd04c106240 2024-04-16T05:43:32.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.178+0000 7fd049ffb640 1 -- 172.21.15.86:0/1046696753 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd034044020 con 0x7fd04c106240 2024-04-16T05:43:32.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.179+0000 7fd049ffb640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd0380a3d20 0x7fd0380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.179+0000 7fd049ffb640 1 -- 172.21.15.86:0/1046696753 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fd0340f2de0 con 0x7fd04c106240 2024-04-16T05:43:32.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.179+0000 7fd050da8640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd0380a3d20 0x7fd0380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.179+0000 7fd050da8640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd0380a3d20 0x7fd0380a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009890 tx=0x7fd03c008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.182+0000 7fd049ffb640 1 -- 172.21.15.86:0/1046696753 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd03404b020 con 0x7fd04c106240 2024-04-16T05:43:32.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:32 smithi086 ceph-mon[32028]: pgmap v254: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:32.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.399+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04c109470 con 0x7fd04c106240 2024-04-16T05:43:32.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.400+0000 7fd049ffb640 1 -- 172.21.15.86:0/1046696753 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fd0340bf1c0 con 0x7fd04c106240 2024-04-16T05:43:32.398 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:43:32.399 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:43:32.399 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:43:32.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd0380a3d20 msgr2=0x7fd0380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd0380a3d20 0x7fd0380a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009890 tx=0x7fd03c008040 comp rx=0 tx=0).stop 2024-04-16T05:43:32.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c106240 msgr2=0x7fd04c199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c106240 0x7fd04c199cf0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd034005cc0 tx=0x7fd034032f40 comp rx=0 tx=0).stop 2024-04-16T05:43:32.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 shutdown_connections 2024-04-16T05:43:32.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd0380a3d20 0x7fd0380a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd04c19a280 0x7fd04c1a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c106240 0x7fd04c199cf0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 --2- 172.21.15.86:0/1046696753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c101170 0x7fd04c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 >> 172.21.15.86:0/1046696753 conn(0x7fd04c077280 msgr2=0x7fd04c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 shutdown_connections 2024-04-16T05:43:32.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.402+0000 7fd05300c640 1 -- 172.21.15.86:0/1046696753 wait complete. 2024-04-16T05:43:32.412 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:43:32.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:32 smithi196 ceph-mon[36373]: pgmap v254: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:32.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.520+0000 7fd6c8567640 1 -- 172.21.15.86:0/3871694512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c01072f0 msgr2=0x7fd6c010eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.520+0000 7fd6c8567640 1 --2- 172.21.15.86:0/3871694512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c01072f0 0x7fd6c010eec0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009ca0 tx=0x7fd6b802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.520+0000 7fd6c8567640 1 -- 172.21.15.86:0/3871694512 shutdown_connections 2024-04-16T05:43:32.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.520+0000 7fd6c8567640 1 --2- 172.21.15.86:0/3871694512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c01072f0 0x7fd6c010eec0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.520+0000 7fd6c8567640 1 --2- 172.21.15.86:0/3871694512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c0102bd0 0x7fd6c0106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.520+0000 7fd6c8567640 1 --2- 172.21.15.86:0/3871694512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c0102220 0x7fd6c0102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.520+0000 7fd6c8567640 1 -- 172.21.15.86:0/3871694512 >> 172.21.15.86:0/3871694512 conn(0x7fd6c0079450 msgr2=0x7fd6c0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.520+0000 7fd6c8567640 1 -- 172.21.15.86:0/3871694512 shutdown_connections 2024-04-16T05:43:32.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.520+0000 7fd6c8567640 1 -- 172.21.15.86:0/3871694512 wait complete. 2024-04-16T05:43:32.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c8567640 1 Processor -- start 2024-04-16T05:43:32.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c8567640 1 -- start start 2024-04-16T05:43:32.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c8567640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c0102220 0x7fd6c019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c8567640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c01072f0 0x7fd6c019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c8567640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c0198650 0x7fd6c0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c8567640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01123b0 con 0x7fd6c0102220 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c8567640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0112230 con 0x7fd6c01072f0 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c8567640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0112530 con 0x7fd6c0198650 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c6303640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c0102220 0x7fd6c019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c6303640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c0102220 0x7fd6c019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56572/0 (socket says 172.21.15.86:56572) 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.521+0000 7fd6c6303640 1 -- 172.21.15.86:0/195027568 learned_addr learned my addr 172.21.15.86:0/195027568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c6b04640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c0198650 0x7fd6c0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c6303640 1 -- 172.21.15.86:0/195027568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c0198650 msgr2=0x7fd6c0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c6303640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c0198650 0x7fd6c0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c6303640 1 -- 172.21.15.86:0/195027568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c01072f0 msgr2=0x7fd6c019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c5b02640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c01072f0 0x7fd6c019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c6303640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c01072f0 0x7fd6c019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c6303640 1 -- 172.21.15.86:0/195027568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c0199170 con 0x7fd6c0102220 2024-04-16T05:43:32.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c6b04640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c0198650 0x7fd6c0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:32.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c6303640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c0102220 0x7fd6c019e450 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c920 tx=0x7fd6bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6b77fe640 1 -- 172.21.15.86:0/195027568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc007c70 con 0x7fd6c0102220 2024-04-16T05:43:32.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6b77fe640 1 -- 172.21.15.86:0/195027568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc007e10 con 0x7fd6c0102220 2024-04-16T05:43:32.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c8567640 1 -- 172.21.15.86:0/195027568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c0199460 con 0x7fd6c0102220 2024-04-16T05:43:32.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6b77fe640 1 -- 172.21.15.86:0/195027568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc018890 con 0x7fd6c0102220 2024-04-16T05:43:32.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.522+0000 7fd6c8567640 1 -- 172.21.15.86:0/195027568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c01a56b0 con 0x7fd6c0102220 2024-04-16T05:43:32.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.524+0000 7fd6b77fe640 1 -- 172.21.15.86:0/195027568 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6bc018a70 con 0x7fd6c0102220 2024-04-16T05:43:32.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.524+0000 7fd6cb22f740 1 Processor -- start 2024-04-16T05:43:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6cb22f740 1 -- start start 2024-04-16T05:43:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a128e60 0x55f89a129240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 0x55f89a131e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a132360 0x55f89a134750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f89a08e0e0 con 0x55f89a132360 2024-04-16T05:43:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f89a02b520 con 0x55f89a129780 2024-04-16T05:43:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f89a08a0a0 con 0x55f89a128e60 2024-04-16T05:43:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6b77fe640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6ac0a3da0 0x7fd6ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c5b02640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a132360 0x55f89a134750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c5b02640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a132360 0x55f89a134750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56574/0 (socket says 172.21.15.86:56574) 2024-04-16T05:43:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c5b02640 1 -- 172.21.15.86:0/3462296999 learned_addr learned my addr 172.21.15.86:0/3462296999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c6b04640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6ac0a3da0 0x7fd6ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c6b04640 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a128e60 0x55f89a129240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6b77fe640 1 -- 172.21.15.86:0/195027568 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fd6bc0d0d50 con 0x7fd6c0102220 2024-04-16T05:43:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c6303640 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 0x55f89a131e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c5b02640 1 -- 172.21.15.86:0/3462296999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a128e60 msgr2=0x55f89a129240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c5b02640 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a128e60 0x55f89a129240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c5b02640 1 -- 172.21.15.86:0/3462296999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 msgr2=0x55f89a131e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c5b02640 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 0x55f89a131e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c5b02640 1 -- 172.21.15.86:0/3462296999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89a0538d0 con 0x55f89a132360 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.525+0000 7fd6c6303640 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 0x55f89a131e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.526+0000 7fd6c5b02640 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a132360 0x55f89a134750 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b990 tx=0x7fd6b000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbd662f900d9c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.526+0000 7fd6c6b04640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6ac0a3da0 0x7fd6ac0a6260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8032530 tx=0x7fd6b8004a60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.526+0000 7fd6b4bfe640 1 -- 172.21.15.86:0/3462296999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000ddd0 con 0x55f89a132360 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.526+0000 7fd6b4bfe640 1 -- 172.21.15.86:0/3462296999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6b0017900 con 0x55f89a132360 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.526+0000 7fd6b4bfe640 1 -- 172.21.15.86:0/3462296999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000fa00 con 0x55f89a132360 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 -- 172.21.15.86:0/3462296999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a132360 msgr2=0x55f89a134750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a132360 0x55f89a134750 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b990 tx=0x7fd6b000be50 comp rx=0 tx=0).stop 2024-04-16T05:43:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 -- 172.21.15.86:0/3462296999 shutdown_connections 2024-04-16T05:43:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a132360 0x55f89a134750 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 0x55f89a131e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/3462296999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a128e60 0x55f89a129240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 -- 172.21.15.86:0/3462296999 >> 172.21.15.86:0/3462296999 conn(0x55f89a123590 msgr2=0x55f89a123970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 -- 172.21.15.86:0/3462296999 shutdown_connections 2024-04-16T05:43:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 -- 172.21.15.86:0/3462296999 wait complete. 2024-04-16T05:43:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 -- start start 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 Processor -- start 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a128e60 0x55f89a12f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a129780 0x55f89a12f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a132360 0x55f89a12ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f89a134fa0 con 0x55f89a129780 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f89a0474d0 con 0x55f89a128e60 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f89a135120 con 0x55f89a132360 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6cb22f740 10 client.0 _ll_register_callbacks cb 0x55f89a02c2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6c5b02640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a129780 0x55f89a12f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6c5b02640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a129780 0x55f89a12f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56588/0 (socket says 172.21.15.86:56588) 2024-04-16T05:43:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.527+0000 7fd6c5b02640 1 -- 172.21.15.86:0/852610643 learned_addr learned my addr 172.21.15.86:0/852610643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6c6b04640 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a132360 0x55f89a12ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6c5b02640 1 -- 172.21.15.86:0/852610643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a132360 msgr2=0x55f89a12ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6c5b02640 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a132360 0x55f89a12ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6c5b02640 1 -- 172.21.15.86:0/852610643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a128e60 msgr2=0x55f89a12f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6c5b02640 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a128e60 0x55f89a12f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6c5b02640 1 -- 172.21.15.86:0/852610643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f899da03f0 con 0x55f89a129780 2024-04-16T05:43:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6c5b02640 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a129780 0x55f89a12f9d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0019040 tx=0x7fd6b00174a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6dcb9880ec1a632 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/852610643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0027020 con 0x55f89a129780 2024-04-16T05:43:32.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/852610643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fd6b0017cb0 con 0x55f89a129780 2024-04-16T05:43:32.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6cb22f740 1 -- 172.21.15.86:0/852610643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6b0017900 con 0x55f89a129780 2024-04-16T05:43:32.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/852610643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6b0032050 con 0x55f89a129780 2024-04-16T05:43:32.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6cb22f740 10 client.18042 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6cb22f740 1 -- 172.21.15.86:0/852610643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f899e6a7d0 con 0x55f89a129780 2024-04-16T05:43:32.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.528+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/852610643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6b0017900 con 0x55f89a129780 2024-04-16T05:43:32.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/852610643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6b0023890 con 0x55f89a129780 2024-04-16T05:43:32.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6cb22f740 10 client.18042 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6cb22f740 10 client.18042 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:43:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6cb22f740 20 client.18042 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55f89a1422e0 0x55f89a1446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6cb22f740 4 client.18042 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6cb22f740 1 -- 172.21.15.86:0/852610643 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f89a09bb00 con 0x55f89a1422e0 2024-04-16T05:43:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6c6303640 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55f89a1422e0 0x55f89a1446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6a9ffb640 10 client.18042 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:43:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.529+0000 7fd6c6303640 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55f89a1422e0 0x55f89a1446d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9c5a888f864320f server_cookie=1d05f436efb11c52 in_seq=0 out_seq=0 2024-04-16T05:43:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/852610643 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f89a09bb00 con 0x55f89a1422e0 2024-04-16T05:43:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6a9ffb640 10 client.18042 handle_command_reply: tid=0 2024-04-16T05:43:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 client.18042 shutdown 2024-04-16T05:43:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 -- 172.21.15.86:0/852610643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a129780 msgr2=0x55f89a12f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a129780 0x55f89a12f9d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0019040 tx=0x7fd6b00174a0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 -- 172.21.15.86:0/852610643 shutdown_connections 2024-04-16T05:43:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55f89a1422e0 0x55f89a1446d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a132360 0x55f89a12ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a129780 0x55f89a12f9d0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6a9ffb640 0 client.18042 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:43:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/852610643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a128e60 0x55f89a12f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 -- 172.21.15.86:0/852610643 >> 172.21.15.86:0/852610643 conn(0x55f89a123590 msgr2=0x55f89a1252e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 -- 172.21.15.86:0/852610643 shutdown_connections 2024-04-16T05:43:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 1 -- 172.21.15.86:0/852610643 wait complete. 2024-04-16T05:43:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.530+0000 7fd6cb22f740 20 client.18042 trim_cache size 0 max 16384 2024-04-16T05:43:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.543+0000 7fd6cb22f740 1 Processor -- start 2024-04-16T05:43:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6cb22f740 1 -- start start 2024-04-16T05:43:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 0x55f89a131680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a132360 0x55f89a12ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01af3f0 con 0x55f89a128e60 2024-04-16T05:43:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01b0f30 con 0x55f89a129780 2024-04-16T05:43:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01afac0 con 0x55f89a132360 2024-04-16T05:43:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6c6303640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6c6303640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56600/0 (socket says 172.21.15.86:56600) 2024-04-16T05:43:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6c6303640 1 -- 172.21.15.86:0/2361628158 learned_addr learned my addr 172.21.15.86:0/2361628158 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6c6303640 1 -- 172.21.15.86:0/2361628158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a132360 msgr2=0x55f89a12ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6c6303640 1 --2- 172.21.15.86:0/2361628158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a132360 0x55f89a12ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6c6303640 1 -- 172.21.15.86:0/2361628158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 msgr2=0x55f89a131680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6c6303640 1 --2- 172.21.15.86:0/2361628158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 0x55f89a131680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6c6303640 1 -- 172.21.15.86:0/2361628158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89a11ee60 con 0x55f89a128e60 2024-04-16T05:43:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6c6303640 1 --2- 172.21.15.86:0/2361628158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13dfb0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc098d80 tx=0x7fd6bc00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db76a5ec27376d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6abfff640 1 -- 172.21.15.86:0/2361628158 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc085e60 con 0x55f89a128e60 2024-04-16T05:43:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6abfff640 1 -- 172.21.15.86:0/2361628158 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc09a9c0 con 0x55f89a128e60 2024-04-16T05:43:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.544+0000 7fd6abfff640 1 -- 172.21.15.86:0/2361628158 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc098f90 con 0x55f89a128e60 2024-04-16T05:43:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 -- 172.21.15.86:0/2361628158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 msgr2=0x55f89a13dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/2361628158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13dfb0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc098d80 tx=0x7fd6bc00ff30 comp rx=0 tx=0).stop 2024-04-16T05:43:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 -- 172.21.15.86:0/2361628158 shutdown_connections 2024-04-16T05:43:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/2361628158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a132360 0x55f89a12ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/2361628158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a129780 0x55f89a131680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/2361628158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13dfb0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 -- 172.21.15.86:0/2361628158 >> 172.21.15.86:0/2361628158 conn(0x55f89a123590 msgr2=0x55f89a122730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 -- 172.21.15.86:0/2361628158 shutdown_connections 2024-04-16T05:43:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 -- 172.21.15.86:0/2361628158 wait complete. 2024-04-16T05:43:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 -- start start 2024-04-16T05:43:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.545+0000 7fd6cb22f740 1 Processor -- start 2024-04-16T05:43:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a129780 0x55f89a13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a132360 0x55f89a12b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01b0860 con 0x55f89a128e60 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01b0190 con 0x55f89a132360 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01ad3f0 con 0x55f89a129780 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 10 client.0 _ll_register_callbacks cb 0x7fd6b000b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6303640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6303640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56614/0 (socket says 172.21.15.86:56614) 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6303640 1 -- 172.21.15.86:0/446744351 learned_addr learned my addr 172.21.15.86:0/446744351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6b04640 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a132360 0x55f89a12b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6303640 1 -- 172.21.15.86:0/446744351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a129780 msgr2=0x55f89a13aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6303640 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a129780 0x55f89a13aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6303640 1 -- 172.21.15.86:0/446744351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a132360 msgr2=0x55f89a12b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6303640 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a132360 0x55f89a12b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6303640 1 -- 172.21.15.86:0/446744351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89a12c150 con 0x55f89a128e60 2024-04-16T05:43:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6c6303640 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13a4c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc020040 tx=0x7fd6bc0d1050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb800a06b851da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/446744351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc080060 con 0x55f89a128e60 2024-04-16T05:43:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/446744351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fd6bc00fb30 con 0x55f89a128e60 2024-04-16T05:43:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 1 -- 172.21.15.86:0/446744351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6bc09a9c0 con 0x55f89a128e60 2024-04-16T05:43:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/446744351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6bc085e60 con 0x55f89a128e60 2024-04-16T05:43:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 10 client.18048 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6cb22f740 1 -- 172.21.15.86:0/446744351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f89a1a8120 con 0x55f89a128e60 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.546+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/446744351 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc09a9c0 con 0x55f89a128e60 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/446744351 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6bc0a5100 con 0x55f89a128e60 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6cb22f740 10 client.18048 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6cb22f740 10 client.18048 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6cb22f740 20 client.18048 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55f89a1a8940 0x55f89a1aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6cb22f740 4 client.18048 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6cb22f740 1 -- 172.21.15.86:0/446744351 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f89a095970 con 0x55f89a1a8940 2024-04-16T05:43:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6c5b02640 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55f89a1a8940 0x55f89a1aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6c5b02640 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55f89a1a8940 0x55f89a1aad50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53e8e1636ca86dee server_cookie=53bbe9e6dd042e43 in_seq=0 out_seq=0 2024-04-16T05:43:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6a9ffb640 10 client.18048 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:43:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6a9ffb640 1 -- 172.21.15.86:0/446744351 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f89a095970 con 0x55f89a1a8940 2024-04-16T05:43:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.547+0000 7fd6a9ffb640 10 client.18048 handle_command_reply: tid=0 2024-04-16T05:43:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 client.18048 shutdown 2024-04-16T05:43:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 -- 172.21.15.86:0/446744351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 msgr2=0x55f89a13a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13a4c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc020040 tx=0x7fd6bc0d1050 comp rx=0 tx=0).stop 2024-04-16T05:43:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 -- 172.21.15.86:0/446744351 shutdown_connections 2024-04-16T05:43:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55f89a1a8940 0x55f89a1aad50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f89a132360 0x55f89a12b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89a129780 0x55f89a13aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 --2- 172.21.15.86:0/446744351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f89a128e60 0x55f89a13a4c0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 -- 172.21.15.86:0/446744351 >> 172.21.15.86:0/446744351 conn(0x55f89a123590 msgr2=0x55f89a12f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6a9ffb640 0 client.18048 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:43:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 -- 172.21.15.86:0/446744351 shutdown_connections 2024-04-16T05:43:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 1 -- 172.21.15.86:0/446744351 wait complete. 2024-04-16T05:43:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6cb22f740 20 client.18048 trim_cache size 0 max 16384 2024-04-16T05:43:32.556 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:43:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 -- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6ac0a3da0 msgr2=0x7fd6ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6ac0a3da0 0x7fd6ac0a6260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8032530 tx=0x7fd6b8004a60 comp rx=0 tx=0).stop 2024-04-16T05:43:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 -- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c0102220 msgr2=0x7fd6c019e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c0102220 0x7fd6c019e450 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c920 tx=0x7fd6bc00cde0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 -- 172.21.15.86:0/195027568 shutdown_connections 2024-04-16T05:43:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c0198650 0x7fd6c0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6ac0a3da0 0x7fd6ac0a6260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c01072f0 0x7fd6c019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 --2- 172.21.15.86:0/195027568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c0102220 0x7fd6c019e450 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 -- 172.21.15.86:0/195027568 >> 172.21.15.86:0/195027568 conn(0x7fd6c0079450 msgr2=0x7fd6c00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 -- 172.21.15.86:0/195027568 shutdown_connections 2024-04-16T05:43:32.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.548+0000 7fd6c8567640 1 -- 172.21.15.86:0/195027568 wait complete. 2024-04-16T05:43:32.559 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:43:32.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:32 smithi194 ceph-mon[35865]: pgmap v254: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.666+0000 7f2b11241640 1 -- 172.21.15.86:0/2452411133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c101130 msgr2=0x7f2b0c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.666+0000 7f2b11241640 1 --2- 172.21.15.86:0/2452411133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c101130 0x7f2b0c101510 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009ca0 tx=0x7f2afc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.666+0000 7f2b11241640 1 -- 172.21.15.86:0/2452411133 shutdown_connections 2024-04-16T05:43:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.666+0000 7f2b11241640 1 --2- 172.21.15.86:0/2452411133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0c106200 0x7f2b0c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.666+0000 7f2b11241640 1 --2- 172.21.15.86:0/2452411133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c101ae0 0x7f2b0c105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.666+0000 7f2b11241640 1 --2- 172.21.15.86:0/2452411133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c101130 0x7f2b0c101510 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.666+0000 7f2b11241640 1 -- 172.21.15.86:0/2452411133 >> 172.21.15.86:0/2452411133 conn(0x7f2b0c0780b0 msgr2=0x7f2b0c0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.666+0000 7f2b11241640 1 -- 172.21.15.86:0/2452411133 shutdown_connections 2024-04-16T05:43:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.667+0000 7f2b11241640 1 -- 172.21.15.86:0/2452411133 wait complete. 2024-04-16T05:43:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.667+0000 7f2b11241640 1 Processor -- start 2024-04-16T05:43:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.667+0000 7f2b11241640 1 -- start start 2024-04-16T05:43:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b11241640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c101130 0x7f2b0c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b11241640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c106200 0x7f2b0c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b11241640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0c19a170 0x7f2b0c1a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b11241640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c111290 con 0x7f2b0c106200 2024-04-16T05:43:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b11241640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c111110 con 0x7f2b0c101130 2024-04-16T05:43:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b11241640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c111410 con 0x7f2b0c19a170 2024-04-16T05:43:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c106200 0x7f2b0c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c106200 0x7f2b0c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56622/0 (socket says 172.21.15.86:56622) 2024-04-16T05:43:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0a59c640 1 -- 172.21.15.86:0/175389142 learned_addr learned my addr 172.21.15.86:0/175389142 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c101130 0x7f2b0c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0c19a170 0x7f2b0c1a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0a59c640 1 -- 172.21.15.86:0/175389142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0c19a170 msgr2=0x7f2b0c1a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0a59c640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0c19a170 0x7f2b0c1a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0a59c640 1 -- 172.21.15.86:0/175389142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c101130 msgr2=0x7f2b0c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0a59c640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c101130 0x7f2b0c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0a59c640 1 -- 172.21.15.86:0/175389142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c1a4ef0 con 0x7f2b0c106200 2024-04-16T05:43:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0c19a170 0x7f2b0c1a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b0a59c640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c106200 0x7f2b0c199c30 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2af400d770 tx=0x7f2af400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b03fff640 1 -- 172.21.15.86:0/175389142 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4017070 con 0x7f2b0c106200 2024-04-16T05:43:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b03fff640 1 -- 172.21.15.86:0/175389142 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4009c30 con 0x7f2b0c106200 2024-04-16T05:43:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b11241640 1 -- 172.21.15.86:0/175389142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c1a51e0 con 0x7f2b0c106200 2024-04-16T05:43:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b03fff640 1 -- 172.21.15.86:0/175389142 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af400f620 con 0x7f2b0c106200 2024-04-16T05:43:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.668+0000 7f2b11241640 1 -- 172.21.15.86:0/175389142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c1a5730 con 0x7f2b0c106200 2024-04-16T05:43:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.671+0000 7f2b03fff640 1 -- 172.21.15.86:0/175389142 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2af401a070 con 0x7f2b0c106200 2024-04-16T05:43:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.671+0000 7f2b13e21740 1 Processor -- start 2024-04-16T05:43:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.671+0000 7f2b13e21740 1 -- start start 2024-04-16T05:43:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.671+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee5a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.671+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee5a780 0x5566aee62e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.671+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 0x5566aee65750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.671+0000 7f2b13e21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566aedbf0e0 con 0x5566aee59e60 2024-04-16T05:43:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.671+0000 7f2b13e21740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5566aed5c520 con 0x5566aee63360 2024-04-16T05:43:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.671+0000 7f2b13e21740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566aedbb0a0 con 0x5566aee5a780 2024-04-16T05:43:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee5a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee5a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56624/0 (socket says 172.21.15.86:56624) 2024-04-16T05:43:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 -- 172.21.15.86:0/1690849106 learned_addr learned my addr 172.21.15.86:0/1690849106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/1690849106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee5a780 0x5566aee62e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 -- 172.21.15.86:0/1690849106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee5a780 msgr2=0x5566aee62e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/1690849106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee5a780 0x5566aee62e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b03fff640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2af80a3d20 0x7f2af80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 -- 172.21.15.86:0/1690849106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 msgr2=0x5566aee65750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0a59c640 1 --2- 172.21.15.86:0/1690849106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 0x5566aee65750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/1690849106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 0x5566aee65750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 -- 172.21.15.86:0/1690849106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566aed848d0 con 0x5566aee59e60 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b03fff640 1 -- 172.21.15.86:0/175389142 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f2af40d06f0 con 0x7f2b0c106200 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/1690849106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee5a240 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400bb00 tx=0x7f2b0400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57abe0ad4ba91b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2af80a3d20 0x7f2af80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b017fa640 1 -- 172.21.15.86:0/1690849106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04017620 con 0x5566aee59e60 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b017fa640 1 -- 172.21.15.86:0/1690849106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b04017da0 con 0x5566aee59e60 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.672+0000 7f2b017fa640 1 -- 172.21.15.86:0/1690849106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04016630 con 0x5566aee59e60 2024-04-16T05:43:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 -- 172.21.15.86:0/1690849106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 msgr2=0x5566aee5a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2af80a3d20 0x7f2af80a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b04018000 tx=0x7f2b04016da0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1690849106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee5a240 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400bb00 tx=0x7f2b0400d340 comp rx=0 tx=0).stop 2024-04-16T05:43:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 -- 172.21.15.86:0/1690849106 shutdown_connections 2024-04-16T05:43:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1690849106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 0x5566aee65750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1690849106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee5a780 0x5566aee62e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1690849106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee5a240 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 -- 172.21.15.86:0/1690849106 >> 172.21.15.86:0/1690849106 conn(0x5566aee54590 msgr2=0x5566aee54970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 -- 172.21.15.86:0/1690849106 shutdown_connections 2024-04-16T05:43:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 -- 172.21.15.86:0/1690849106 wait complete. 2024-04-16T05:43:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.673+0000 7f2b13e21740 1 -- start start 2024-04-16T05:43:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b13e21740 1 Processor -- start 2024-04-16T05:43:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 0x5566aee60380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee5a780 0x5566aee608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee60e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b13e21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566aee66000 con 0x5566aee63360 2024-04-16T05:43:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b13e21740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5566aed784d0 con 0x5566aee5a780 2024-04-16T05:43:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b13e21740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566aee66180 con 0x5566aee59e60 2024-04-16T05:43:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b13e21740 10 client.0 _ll_register_callbacks cb 0x5566aed5d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee60e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee60e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56638/0 (socket says 172.21.15.86:56638) 2024-04-16T05:43:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0b59e640 1 -- 172.21.15.86:0/1713543364 learned_addr learned my addr 172.21.15.86:0/1713543364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 0x5566aee60380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0a59c640 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee5a780 0x5566aee608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0b59e640 1 -- 172.21.15.86:0/1713543364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 msgr2=0x5566aee60380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 0x5566aee60380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0b59e640 1 -- 172.21.15.86:0/1713543364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee5a780 msgr2=0x5566aee608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee5a780 0x5566aee608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0b59e640 1 -- 172.21.15.86:0/1713543364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566aead13f0 con 0x5566aee63360 2024-04-16T05:43:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee60e00 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400d380 tx=0x7f2b04022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ec472394b777b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2af27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.674+0000 7f2af27fc640 1 -- 172.21.15.86:0/1713543364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400b6f0 con 0x5566aee63360 2024-04-16T05:43:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2af27fc640 1 -- 172.21.15.86:0/1713543364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f2b040163d0 con 0x5566aee63360 2024-04-16T05:43:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b13e21740 1 -- 172.21.15.86:0/1713543364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b04017da0 con 0x5566aee63360 2024-04-16T05:43:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2af27fc640 1 -- 172.21.15.86:0/1713543364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b0403b050 con 0x5566aee63360 2024-04-16T05:43:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b13e21740 10 client.18060 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b13e21740 1 -- 172.21.15.86:0/1713543364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566aeb9b7d0 con 0x5566aee63360 2024-04-16T05:43:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2af27fc640 1 -- 172.21.15.86:0/1713543364 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b04017da0 con 0x5566aee63360 2024-04-16T05:43:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2af27fc640 1 -- 172.21.15.86:0/1713543364 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b040174b0 con 0x5566aee63360 2024-04-16T05:43:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b13e21740 10 client.18060 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b13e21740 10 client.18060 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:43:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b13e21740 20 client.18060 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5566aee73310 0x5566aee75700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b13e21740 4 client.18060 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b13e21740 1 -- 172.21.15.86:0/1713543364 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566aedccb00 con 0x5566aee73310 2024-04-16T05:43:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.675+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5566aee73310 0x5566aee75700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.676+0000 7f2af27fc640 10 client.18060 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:43:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.676+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5566aee73310 0x5566aee75700 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d9376e9227a4c66 server_cookie=cdd911439497eba6 in_seq=0 out_seq=0 2024-04-16T05:43:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.676+0000 7f2af27fc640 1 -- 172.21.15.86:0/1713543364 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566aedccb00 con 0x5566aee73310 2024-04-16T05:43:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2af27fc640 10 client.18060 handle_command_reply: tid=0 2024-04-16T05:43:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 client.18060 shutdown 2024-04-16T05:43:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 -- 172.21.15.86:0/1713543364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 msgr2=0x5566aee60e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee60e00 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400d380 tx=0x7f2b04022000 comp rx=0 tx=0).stop 2024-04-16T05:43:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 -- 172.21.15.86:0/1713543364 shutdown_connections 2024-04-16T05:43:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee60e00 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee5a780 0x5566aee608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5566aee73310 0x5566aee75700 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 --2- 172.21.15.86:0/1713543364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 0x5566aee60380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2af27fc640 0 client.18060 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:43:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 -- 172.21.15.86:0/1713543364 >> 172.21.15.86:0/1713543364 conn(0x5566aee54590 msgr2=0x5566aee562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 -- 172.21.15.86:0/1713543364 shutdown_connections 2024-04-16T05:43:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 1 -- 172.21.15.86:0/1713543364 wait complete. 2024-04-16T05:43:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.677+0000 7f2b13e21740 20 client.18060 trim_cache size 0 max 16384 2024-04-16T05:43:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b13e21740 1 Processor -- start 2024-04-16T05:43:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b13e21740 1 -- start start 2024-04-16T05:43:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee6f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee5a780 0x5566aee60c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 0x5566aee7ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b13e21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1af580 con 0x5566aee59e60 2024-04-16T05:43:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b13e21740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1b10c0 con 0x5566aee63360 2024-04-16T05:43:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b13e21740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1afc50 con 0x5566aee5a780 2024-04-16T05:43:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee6f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee6f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56646/0 (socket says 172.21.15.86:56646) 2024-04-16T05:43:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b0ad9d640 1 -- 172.21.15.86:0/2446349931 learned_addr learned my addr 172.21.15.86:0/2446349931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.690+0000 7f2b0ad9d640 1 -- 172.21.15.86:0/2446349931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee5a780 msgr2=0x5566aee60c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/2446349931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee5a780 0x5566aee60c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b0ad9d640 1 -- 172.21.15.86:0/2446349931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 msgr2=0x5566aee7ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/2446349931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 0x5566aee7ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/2446349931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 0x5566aee7ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b0ad9d640 1 -- 172.21.15.86:0/2446349931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566aee557c0 con 0x5566aee59e60 2024-04-16T05:43:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/2446349931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 0x5566aee7ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/2446349931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee6f000 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2afc004af0 tx=0x7f2afc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59eedda03a7694be server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b00ff9640 1 -- 172.21.15.86:0/2446349931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc04f070 con 0x5566aee59e60 2024-04-16T05:43:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b00ff9640 1 -- 172.21.15.86:0/2446349931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2afc0098a0 con 0x5566aee59e60 2024-04-16T05:43:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.691+0000 7f2b00ff9640 1 -- 172.21.15.86:0/2446349931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc039040 con 0x5566aee59e60 2024-04-16T05:43:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 -- 172.21.15.86:0/2446349931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 msgr2=0x5566aee6f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2446349931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee6f000 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2afc004af0 tx=0x7f2afc005c50 comp rx=0 tx=0).stop 2024-04-16T05:43:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 -- 172.21.15.86:0/2446349931 shutdown_connections 2024-04-16T05:43:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2446349931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee63360 0x5566aee7ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2446349931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee5a780 0x5566aee60c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2446349931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee59e60 0x5566aee6f000 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 -- 172.21.15.86:0/2446349931 >> 172.21.15.86:0/2446349931 conn(0x5566aee54590 msgr2=0x5566aee56270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 -- 172.21.15.86:0/2446349931 shutdown_connections 2024-04-16T05:43:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 -- 172.21.15.86:0/2446349931 wait complete. 2024-04-16T05:43:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 -- start start 2024-04-16T05:43:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 Processor -- start 2024-04-16T05:43:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 0x5566aee6b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee5a780 0x5566aee66d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1b09f0 con 0x5566aee63360 2024-04-16T05:43:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1b0320 con 0x5566aee5a780 2024-04-16T05:43:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1ad580 con 0x5566aee59e60 2024-04-16T05:43:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b13e21740 10 client.0 _ll_register_callbacks cb 0x7f2b0400b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b0b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b0b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee672a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56660/0 (socket says 172.21.15.86:56660) 2024-04-16T05:43:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.692+0000 7f2b0b59e640 1 -- 172.21.15.86:0/2761057032 learned_addr learned my addr 172.21.15.86:0/2761057032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 0x5566aee6b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b0b59e640 1 -- 172.21.15.86:0/2761057032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 msgr2=0x5566aee6b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 0x5566aee6b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b0b59e640 1 -- 172.21.15.86:0/2761057032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee5a780 msgr2=0x5566aee66d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee5a780 0x5566aee66d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b0b59e640 1 -- 172.21.15.86:0/2761057032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566aee67ad0 con 0x5566aee63360 2024-04-16T05:43:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b0b59e640 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee672a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2b04017850 tx=0x7f2b0400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba046c0f8895f6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2af27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2af27fc640 1 -- 172.21.15.86:0/2761057032 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04043030 con 0x5566aee63360 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2af27fc640 1 -- 172.21.15.86:0/2761057032 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f2b04033070 con 0x5566aee63360 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b13e21740 1 -- 172.21.15.86:0/2761057032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2afc0098a0 con 0x5566aee63360 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2af27fc640 1 -- 172.21.15.86:0/2761057032 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b0404d050 con 0x5566aee63360 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b13e21740 10 client.18066 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b13e21740 1 -- 172.21.15.86:0/2761057032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566aeed9610 con 0x5566aee63360 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2af27fc640 1 -- 172.21.15.86:0/2761057032 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2afc0098a0 con 0x5566aee63360 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2af27fc640 1 -- 172.21.15.86:0/2761057032 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b04017490 con 0x5566aee63360 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b13e21740 10 client.18066 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b13e21740 10 client.18066 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:43:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b13e21740 20 client.18066 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.693+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5566aeed9d80 0x5566aeedc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.694+0000 7f2b13e21740 4 client.18066 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.694+0000 7f2b13e21740 1 -- 172.21.15.86:0/2761057032 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566aedc6970 con 0x5566aeed9d80 2024-04-16T05:43:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.694+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5566aeed9d80 0x5566aeedc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.694+0000 7f2b0ad9d640 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5566aeed9d80 0x5566aeedc1c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f6d99951b45f170c server_cookie=7afa0bb800485193 in_seq=0 out_seq=0 2024-04-16T05:43:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.694+0000 7f2af27fc640 10 client.18066 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:43:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2af27fc640 1 -- 172.21.15.86:0/2761057032 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566aedc6970 con 0x5566aeed9d80 2024-04-16T05:43:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2af27fc640 10 client.18066 handle_command_reply: tid=0 2024-04-16T05:43:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 client.18066 shutdown 2024-04-16T05:43:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 -- 172.21.15.86:0/2761057032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 msgr2=0x5566aee672a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee672a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2b04017850 tx=0x7f2b0400df60 comp rx=0 tx=0).stop 2024-04-16T05:43:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 -- 172.21.15.86:0/2761057032 shutdown_connections 2024-04-16T05:43:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566aee63360 0x5566aee672a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566aee5a780 0x5566aee66d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x5566aeed9d80 0x5566aeedc1c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 --2- 172.21.15.86:0/2761057032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566aee59e60 0x5566aee6b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2af27fc640 0 client.18066 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:43:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 -- 172.21.15.86:0/2761057032 >> 172.21.15.86:0/2761057032 conn(0x5566aee54590 msgr2=0x5566aee55990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 -- 172.21.15.86:0/2761057032 shutdown_connections 2024-04-16T05:43:32.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 1 -- 172.21.15.86:0/2761057032 wait complete. 2024-04-16T05:43:32.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b13e21740 20 client.18066 trim_cache size 0 max 16384 2024-04-16T05:43:32.703 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:43:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 -- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2af80a3d20 msgr2=0x7f2af80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2af80a3d20 0x7f2af80a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b04018000 tx=0x7f2b04016da0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 -- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c106200 msgr2=0x7f2b0c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c106200 0x7f2b0c199c30 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2af400d770 tx=0x7f2af400dc30 comp rx=0 tx=0).stop 2024-04-16T05:43:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 -- 172.21.15.86:0/175389142 shutdown_connections 2024-04-16T05:43:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0c19a170 0x7f2b0c1a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0c106200 0x7f2b0c199c30 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2af80a3d20 0x7f2af80a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 --2- 172.21.15.86:0/175389142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c101130 0x7f2b0c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 -- 172.21.15.86:0/175389142 >> 172.21.15.86:0/175389142 conn(0x7f2b0c0780b0 msgr2=0x7f2b0c0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.695+0000 7f2b11241640 1 -- 172.21.15.86:0/175389142 shutdown_connections 2024-04-16T05:43:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.696+0000 7f2b11241640 1 -- 172.21.15.86:0/175389142 wait complete. 2024-04-16T05:43:32.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:43:32.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.813+0000 7f2c311d5640 1 -- 172.21.15.86:0/1034955747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c101130 msgr2=0x7f2c2c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.813+0000 7f2c311d5640 1 --2- 172.21.15.86:0/1034955747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c101130 0x7f2c2c101510 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009900 tx=0x7f2c1c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.813+0000 7f2c311d5640 1 -- 172.21.15.86:0/1034955747 shutdown_connections 2024-04-16T05:43:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.813+0000 7f2c311d5640 1 --2- 172.21.15.86:0/1034955747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c2c106200 0x7f2c2c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.813+0000 7f2c311d5640 1 --2- 172.21.15.86:0/1034955747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c2c101ae0 0x7f2c2c105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.813+0000 7f2c311d5640 1 --2- 172.21.15.86:0/1034955747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c101130 0x7f2c2c101510 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.813+0000 7f2c311d5640 1 -- 172.21.15.86:0/1034955747 >> 172.21.15.86:0/1034955747 conn(0x7f2c2c0780b0 msgr2=0x7f2c2c0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.813+0000 7f2c311d5640 1 -- 172.21.15.86:0/1034955747 shutdown_connections 2024-04-16T05:43:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.813+0000 7f2c311d5640 1 -- 172.21.15.86:0/1034955747 wait complete. 2024-04-16T05:43:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.814+0000 7f2c311d5640 1 Processor -- start 2024-04-16T05:43:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.814+0000 7f2c311d5640 1 -- start start 2024-04-16T05:43:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c311d5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c2c101130 0x7f2c2c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c311d5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c2c106200 0x7f2c2c1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c311d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c197ee0 0x7f2c2c1a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c311d5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c111290 con 0x7f2c2c197ee0 2024-04-16T05:43:32.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c311d5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c111110 con 0x7f2c2c101130 2024-04-16T05:43:32.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c311d5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c111410 con 0x7f2c2c106200 2024-04-16T05:43:32.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c197ee0 0x7f2c2c1a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c197ee0 0x7f2c2c1a6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56672/0 (socket says 172.21.15.86:56672) 2024-04-16T05:43:32.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2b59e640 1 -- 172.21.15.86:0/3965687421 learned_addr learned my addr 172.21.15.86:0/3965687421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c2c101130 0x7f2c2c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c2c106200 0x7f2c2c1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2b59e640 1 -- 172.21.15.86:0/3965687421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c2c106200 msgr2=0x7f2c2c1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2b59e640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c2c106200 0x7f2c2c1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2b59e640 1 -- 172.21.15.86:0/3965687421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c2c101130 msgr2=0x7f2c2c197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2b59e640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c2c101130 0x7f2c2c197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2b59e640 1 -- 172.21.15.86:0/3965687421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c1a70a0 con 0x7f2c2c197ee0 2024-04-16T05:43:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c2b59e640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c197ee0 0x7f2c2c1a6ae0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400ead0 tx=0x7f2c2400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c23fff640 1 -- 172.21.15.86:0/3965687421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24004d20 con 0x7f2c2c197ee0 2024-04-16T05:43:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c23fff640 1 -- 172.21.15.86:0/3965687421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c24005020 con 0x7f2c2c197ee0 2024-04-16T05:43:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c311d5640 1 -- 172.21.15.86:0/3965687421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c1a7390 con 0x7f2c2c197ee0 2024-04-16T05:43:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c23fff640 1 -- 172.21.15.86:0/3965687421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c240116d0 con 0x7f2c2c197ee0 2024-04-16T05:43:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.815+0000 7f2c311d5640 1 -- 172.21.15.86:0/3965687421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c1a7890 con 0x7f2c2c197ee0 2024-04-16T05:43:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.817+0000 7f2c23fff640 1 -- 172.21.15.86:0/3965687421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c24005560 con 0x7f2c2c197ee0 2024-04-16T05:43:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c33db6740 1 Processor -- start 2024-04-16T05:43:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c33db6740 1 -- start start 2024-04-16T05:43:32.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89927e60 0x560f89928240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f89930e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89931360 0x560f89933750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c33db6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f8988d0e0 con 0x560f89927e60 2024-04-16T05:43:32.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c33db6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560f8982a520 con 0x560f89931360 2024-04-16T05:43:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c33db6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f898890a0 con 0x560f89928780 2024-04-16T05:43:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c23fff640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c180a3d20 0x7f2c180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c2a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89927e60 0x560f89928240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c2a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89927e60 0x560f89928240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56686/0 (socket says 172.21.15.86:56686) 2024-04-16T05:43:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c2a59c640 1 -- 172.21.15.86:0/876930257 learned_addr learned my addr 172.21.15.86:0/876930257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c2b59e640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c180a3d20 0x7f2c180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.818+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/876930257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f89930e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c23fff640 1 -- 172.21.15.86:0/3965687421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f2c240d0ce0 con 0x7f2c2c197ee0 2024-04-16T05:43:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c2a59c640 1 -- 172.21.15.86:0/876930257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 msgr2=0x560f89930e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/876930257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f89930e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c2a59c640 1 -- 172.21.15.86:0/876930257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89931360 msgr2=0x560f89933750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/876930257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89931360 0x560f89933750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c2a59c640 1 -- 172.21.15.86:0/876930257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f898528d0 con 0x560f89927e60 2024-04-16T05:43:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c2b59e640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c180a3d20 0x7f2c180a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c240ce110 tx=0x7f2c240054d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/876930257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89927e60 0x560f89928240 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400ad00 tx=0x7f2c140117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc574f949efa150c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c217fa640 1 -- 172.21.15.86:0/876930257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14012450 con 0x560f89927e60 2024-04-16T05:43:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c217fa640 1 -- 172.21.15.86:0/876930257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c14012ad0 con 0x560f89927e60 2024-04-16T05:43:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.819+0000 7f2c217fa640 1 -- 172.21.15.86:0/876930257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400e950 con 0x560f89927e60 2024-04-16T05:43:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 -- 172.21.15.86:0/876930257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89927e60 msgr2=0x560f89928240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 --2- 172.21.15.86:0/876930257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89927e60 0x560f89928240 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400ad00 tx=0x7f2c140117b0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 -- 172.21.15.86:0/876930257 shutdown_connections 2024-04-16T05:43:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 --2- 172.21.15.86:0/876930257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89931360 0x560f89933750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 --2- 172.21.15.86:0/876930257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f89930e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 --2- 172.21.15.86:0/876930257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89927e60 0x560f89928240 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 -- 172.21.15.86:0/876930257 >> 172.21.15.86:0/876930257 conn(0x560f89922590 msgr2=0x560f89922970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 -- 172.21.15.86:0/876930257 shutdown_connections 2024-04-16T05:43:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 -- 172.21.15.86:0/876930257 wait complete. 2024-04-16T05:43:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 -- start start 2024-04-16T05:43:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.820+0000 7f2c33db6740 1 Processor -- start 2024-04-16T05:43:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f8992e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f8992ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f8992ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f89933fa0 con 0x560f89931360 2024-04-16T05:43:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560f898464d0 con 0x560f89927e60 2024-04-16T05:43:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f89934120 con 0x560f89928780 2024-04-16T05:43:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 10 client.0 _ll_register_callbacks cb 0x560f8982b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f8992ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f8992ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56698/0 (socket says 172.21.15.86:56698) 2024-04-16T05:43:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 -- 172.21.15.86:0/4019762216 learned_addr learned my addr 172.21.15.86:0/4019762216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f8992ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f8992e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 -- 172.21.15.86:0/4019762216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 msgr2=0x560f8992ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f8992ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 -- 172.21.15.86:0/4019762216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 msgr2=0x560f8992e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f8992e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 -- 172.21.15.86:0/4019762216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f8959f3f0 con 0x560f89931360 2024-04-16T05:43:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f8992ef90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400a350 tx=0x7f2c1400e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8df5e7f3def5a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c127fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c127fc640 1 -- 172.21.15.86:0/4019762216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14012920 con 0x560f89931360 2024-04-16T05:43:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c127fc640 1 -- 172.21.15.86:0/4019762216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f2c1400e900 con 0x560f89931360 2024-04-16T05:43:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 1 -- 172.21.15.86:0/4019762216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c14012ad0 con 0x560f89931360 2024-04-16T05:43:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c127fc640 1 -- 172.21.15.86:0/4019762216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c1400eb00 con 0x560f89931360 2024-04-16T05:43:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 10 client.18078 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.821+0000 7f2c33db6740 1 -- 172.21.15.86:0/4019762216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f896697d0 con 0x560f89931360 2024-04-16T05:43:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.822+0000 7f2c127fc640 1 -- 172.21.15.86:0/4019762216 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c14012ad0 con 0x560f89931360 2024-04-16T05:43:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.822+0000 7f2c127fc640 1 -- 172.21.15.86:0/4019762216 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c1400f420 con 0x560f89931360 2024-04-16T05:43:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.822+0000 7f2c33db6740 10 client.18078 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.822+0000 7f2c33db6740 10 client.18078 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:43:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.822+0000 7f2c33db6740 20 client.18078 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.822+0000 7f2c33db6740 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x560f89941220 0x560f89943610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.822+0000 7f2c33db6740 4 client.18078 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.822+0000 7f2c33db6740 1 -- 172.21.15.86:0/4019762216 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f8989ab00 con 0x560f89941220 2024-04-16T05:43:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.822+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x560f89941220 0x560f89943610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.823+0000 7f2c127fc640 10 client.18078 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:43:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.823+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x560f89941220 0x560f89943610 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e513b78f4eacec9 server_cookie=6e1ded1424f97cfa in_seq=0 out_seq=0 2024-04-16T05:43:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.823+0000 7f2c127fc640 1 -- 172.21.15.86:0/4019762216 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f8989ab00 con 0x560f89941220 2024-04-16T05:43:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.823+0000 7f2c127fc640 10 client.18078 handle_command_reply: tid=0 2024-04-16T05:43:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.823+0000 7f2c33db6740 1 client.18078 shutdown 2024-04-16T05:43:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.823+0000 7f2c33db6740 1 -- 172.21.15.86:0/4019762216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 msgr2=0x560f8992ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.823+0000 7f2c33db6740 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f8992ef90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400a350 tx=0x7f2c1400e660 comp rx=0 tx=0).stop 2024-04-16T05:43:32.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c33db6740 1 -- 172.21.15.86:0/4019762216 shutdown_connections 2024-04-16T05:43:32.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c33db6740 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x560f89941220 0x560f89943610 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c33db6740 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f8992ef90 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c127fc640 0 client.18078 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:43:32.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c33db6740 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f8992ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c33db6740 1 --2- 172.21.15.86:0/4019762216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f8992e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c33db6740 1 -- 172.21.15.86:0/4019762216 >> 172.21.15.86:0/4019762216 conn(0x560f89922590 msgr2=0x560f899242e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c33db6740 1 -- 172.21.15.86:0/4019762216 shutdown_connections 2024-04-16T05:43:32.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c33db6740 1 -- 172.21.15.86:0/4019762216 wait complete. 2024-04-16T05:43:32.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.824+0000 7f2c33db6740 20 client.18078 trim_cache size 0 max 16384 2024-04-16T05:43:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c33db6740 1 Processor -- start 2024-04-16T05:43:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c33db6740 1 -- start start 2024-04-16T05:43:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f8992d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f8992ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f89948d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c33db6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1b1730 con 0x560f89931360 2024-04-16T05:43:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c33db6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1b3270 con 0x560f89927e60 2024-04-16T05:43:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c33db6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1b1e00 con 0x560f89928780 2024-04-16T05:43:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c2a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f89948d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c2a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f89948d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56700/0 (socket says 172.21.15.86:56700) 2024-04-16T05:43:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c2a59c640 1 -- 172.21.15.86:0/588062081 learned_addr learned my addr 172.21.15.86:0/588062081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/588062081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f8992ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c2a59c640 1 -- 172.21.15.86:0/588062081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 msgr2=0x560f8992ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.837+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/588062081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f8992ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c2a59c640 1 -- 172.21.15.86:0/588062081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 msgr2=0x560f8992d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/588062081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f8992d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c2a59c640 1 -- 172.21.15.86:0/588062081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f89927090 con 0x560f89931360 2024-04-16T05:43:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/588062081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f8992ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/588062081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f89948d30 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2c140124f0 tx=0x7f2c1400ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323409d3241286c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c20ff9640 1 -- 172.21.15.86:0/588062081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400e900 con 0x560f89931360 2024-04-16T05:43:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c20ff9640 1 -- 172.21.15.86:0/588062081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c14011cb0 con 0x560f89931360 2024-04-16T05:43:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c20ff9640 1 -- 172.21.15.86:0/588062081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14005070 con 0x560f89931360 2024-04-16T05:43:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c33db6740 1 -- 172.21.15.86:0/588062081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 msgr2=0x560f89948d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.838+0000 7f2c33db6740 1 --2- 172.21.15.86:0/588062081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f89948d30 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2c140124f0 tx=0x7f2c1400ac80 comp rx=0 tx=0).stop 2024-04-16T05:43:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 -- 172.21.15.86:0/588062081 shutdown_connections 2024-04-16T05:43:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 --2- 172.21.15.86:0/588062081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89931360 0x560f89948d30 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 --2- 172.21.15.86:0/588062081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89928780 0x560f8992ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 --2- 172.21.15.86:0/588062081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f8992d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 -- 172.21.15.86:0/588062081 >> 172.21.15.86:0/588062081 conn(0x560f89922590 msgr2=0x560f8992e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 -- 172.21.15.86:0/588062081 shutdown_connections 2024-04-16T05:43:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 -- 172.21.15.86:0/588062081 wait complete. 2024-04-16T05:43:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 -- start start 2024-04-16T05:43:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 Processor -- start 2024-04-16T05:43:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f89939400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89928780 0x560f89939940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89931360 0x560f8992a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1b2ba0 con 0x560f89928780 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1b24d0 con 0x560f89927e60 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1af730 con 0x560f89931360 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c33db6740 10 client.0 _ll_register_callbacks cb 0x7f2c1c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c2ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89928780 0x560f89939940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c2ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89928780 0x560f89939940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56706/0 (socket says 172.21.15.86:56706) 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.839+0000 7f2c2ad9d640 1 -- 172.21.15.86:0/3449314480 learned_addr learned my addr 172.21.15.86:0/3449314480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c2ad9d640 1 -- 172.21.15.86:0/3449314480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89931360 msgr2=0x560f8992a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89931360 0x560f8992a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c2ad9d640 1 -- 172.21.15.86:0/3449314480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 msgr2=0x560f89939400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f89939400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c2ad9d640 1 -- 172.21.15.86:0/3449314480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f8992b140 con 0x560f89928780 2024-04-16T05:43:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c2ad9d640 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89928780 0x560f89939940 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009900 tx=0x7f2c1c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df43c1ed35d60ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c127fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c127fc640 1 -- 172.21.15.86:0/3449314480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c04c030 con 0x560f89928780 2024-04-16T05:43:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c127fc640 1 -- 172.21.15.86:0/3449314480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f2c1c004040 con 0x560f89928780 2024-04-16T05:43:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c33db6740 1 -- 172.21.15.86:0/3449314480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c14011cb0 con 0x560f89928780 2024-04-16T05:43:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c127fc640 1 -- 172.21.15.86:0/3449314480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c1c059050 con 0x560f89928780 2024-04-16T05:43:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c33db6740 10 client.18084 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c33db6740 1 -- 172.21.15.86:0/3449314480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f899a6fc0 con 0x560f89928780 2024-04-16T05:43:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c127fc640 1 -- 172.21.15.86:0/3449314480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c14011cb0 con 0x560f89928780 2024-04-16T05:43:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c127fc640 1 -- 172.21.15.86:0/3449314480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c1c004860 con 0x560f89928780 2024-04-16T05:43:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c33db6740 10 client.18084 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c33db6740 10 client.18084 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:43:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c33db6740 20 client.18084 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c33db6740 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x560f89941220 0x560f899a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c33db6740 4 client.18084 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.840+0000 7f2c33db6740 1 -- 172.21.15.86:0/3449314480 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f89894970 con 0x560f89941220 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x560f89941220 0x560f899a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c2a59c640 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x560f89941220 0x560f899a97b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3131d643390cee20 server_cookie=70f30b5c85135a57 in_seq=0 out_seq=0 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c127fc640 10 client.18084 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c127fc640 1 -- 172.21.15.86:0/3449314480 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f89894970 con 0x560f89941220 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c127fc640 10 client.18084 handle_command_reply: tid=0 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 client.18084 shutdown 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 -- 172.21.15.86:0/3449314480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89928780 msgr2=0x560f89939940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89928780 0x560f89939940 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009900 tx=0x7f2c1c005c00 comp rx=0 tx=0).stop 2024-04-16T05:43:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 -- 172.21.15.86:0/3449314480 shutdown_connections 2024-04-16T05:43:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f89931360 0x560f8992a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x560f89941220 0x560f899a97b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f89928780 0x560f89939940 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c127fc640 0 client.18084 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:43:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 --2- 172.21.15.86:0/3449314480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f89927e60 0x560f89939400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 -- 172.21.15.86:0/3449314480 >> 172.21.15.86:0/3449314480 conn(0x560f89922590 msgr2=0x560f8992e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 -- 172.21.15.86:0/3449314480 shutdown_connections 2024-04-16T05:43:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.841+0000 7f2c33db6740 1 -- 172.21.15.86:0/3449314480 wait complete. 2024-04-16T05:43:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c33db6740 20 client.18084 trim_cache size 0 max 16384 2024-04-16T05:43:32.850 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:43:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 -- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c180a3d20 msgr2=0x7f2c180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c180a3d20 0x7f2c180a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c240ce110 tx=0x7f2c240054d0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 -- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c197ee0 msgr2=0x7f2c2c1a6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c197ee0 0x7f2c2c1a6ae0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400ead0 tx=0x7f2c2400eed0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 -- 172.21.15.86:0/3965687421 shutdown_connections 2024-04-16T05:43:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2c197ee0 0x7f2c2c1a6ae0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c2c106200 0x7f2c2c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f2c180a3d20 0x7f2c180a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 --2- 172.21.15.86:0/3965687421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c2c101130 0x7f2c2c197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 -- 172.21.15.86:0/3965687421 >> 172.21.15.86:0/3965687421 conn(0x7f2c2c0780b0 msgr2=0x7f2c2c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 -- 172.21.15.86:0/3965687421 shutdown_connections 2024-04-16T05:43:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.842+0000 7f2c311d5640 1 -- 172.21.15.86:0/3965687421 wait complete. 2024-04-16T05:43:32.853 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:43:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 -- 172.21.15.86:0/1445759635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8107040 msgr2=0x7fb0a8107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/1445759635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8107040 0x7fb0a8107420 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fb098009900 tx=0x7fb09802bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 -- 172.21.15.86:0/1445759635 shutdown_connections 2024-04-16T05:43:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/1445759635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0a8101bf0 0x7fb0a810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/1445759635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a8101250 0x7fb0a81016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/1445759635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8107040 0x7fb0a8107420 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 -- 172.21.15.86:0/1445759635 >> 172.21.15.86:0/1445759635 conn(0x7fb0a8076d00 msgr2=0x7fb0a8077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 -- 172.21.15.86:0/1445759635 shutdown_connections 2024-04-16T05:43:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 -- 172.21.15.86:0/1445759635 wait complete. 2024-04-16T05:43:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.962+0000 7fb0adf0d640 1 Processor -- start 2024-04-16T05:43:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0adf0d640 1 -- start start 2024-04-16T05:43:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0adf0d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0a8101250 0x7fb0a819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0adf0d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8101bf0 0x7fb0a819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0adf0d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a8107040 0x7fb0a81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0adf0d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81112f0 con 0x7fb0a8101bf0 2024-04-16T05:43:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0adf0d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8111170 con 0x7fb0a8101250 2024-04-16T05:43:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0adf0d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8111470 con 0x7fb0a8107040 2024-04-16T05:43:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8101bf0 0x7fb0a819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8101bf0 0x7fb0a819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56710/0 (socket says 172.21.15.86:56710) 2024-04-16T05:43:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a6ffd640 1 -- 172.21.15.86:0/2715912738 learned_addr learned my addr 172.21.15.86:0/2715912738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0a8101250 0x7fb0a819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a7fff640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a8107040 0x7fb0a81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a6ffd640 1 -- 172.21.15.86:0/2715912738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a8107040 msgr2=0x7fb0a81975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a8107040 0x7fb0a81975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a6ffd640 1 -- 172.21.15.86:0/2715912738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0a8101250 msgr2=0x7fb0a819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0a8101250 0x7fb0a819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a6ffd640 1 -- 172.21.15.86:0/2715912738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a8197d30 con 0x7fb0a8101bf0 2024-04-16T05:43:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a7fff640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a8107040 0x7fb0a81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.963+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0a8101250 0x7fb0a819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.964+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8101bf0 0x7fb0a819d940 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fb09000cce0 tx=0x7fb09000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.964+0000 7fb0a4ff9640 1 -- 172.21.15.86:0/2715912738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090004b20 con 0x7fb0a8101bf0 2024-04-16T05:43:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.964+0000 7fb0a4ff9640 1 -- 172.21.15.86:0/2715912738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb090004cc0 con 0x7fb0a8101bf0 2024-04-16T05:43:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.964+0000 7fb0adf0d640 1 -- 172.21.15.86:0/2715912738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a8198020 con 0x7fb0a8101bf0 2024-04-16T05:43:32.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.964+0000 7fb0a4ff9640 1 -- 172.21.15.86:0/2715912738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09000f8c0 con 0x7fb0a8101bf0 2024-04-16T05:43:32.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.964+0000 7fb0adf0d640 1 -- 172.21.15.86:0/2715912738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a806a680 con 0x7fb0a8101bf0 2024-04-16T05:43:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0a4ff9640 1 -- 172.21.15.86:0/2715912738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb09000fa60 con 0x7fb0a8101bf0 2024-04-16T05:43:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0b0bcb740 1 Processor -- start 2024-04-16T05:43:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0b0bcb740 1 -- start start 2024-04-16T05:43:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 0x556bad0d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 0x556bad0e1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0e4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bad03e0e0 con 0x556bad0e2360 2024-04-16T05:43:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556bacfdb520 con 0x556bad0d9780 2024-04-16T05:43:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bad03a0a0 con 0x556bad0d8e60 2024-04-16T05:43:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0e4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0a4ff9640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb0940a3d20 0x7fb0940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0e4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56716/0 (socket says 172.21.15.86:56716) 2024-04-16T05:43:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.966+0000 7fb0a6ffd640 1 -- 172.21.15.86:0/3627702882 learned_addr learned my addr 172.21.15.86:0/3627702882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a7fff640 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 0x556bad0d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a7fff640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb0940a3d20 0x7fb0940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 0x556bad0e1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a6ffd640 1 -- 172.21.15.86:0/3627702882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 msgr2=0x556bad0d9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 0x556bad0d9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a6ffd640 1 -- 172.21.15.86:0/3627702882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 msgr2=0x556bad0e1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 0x556bad0e1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a6ffd640 1 -- 172.21.15.86:0/3627702882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bad0038d0 con 0x556bad0e2360 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a4ff9640 1 -- 172.21.15.86:0/2715912738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fb0900d0d80 con 0x7fb0a8101bf0 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 0x556bad0e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0e4750 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb0900a5780 tx=0x7fb090005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42929687c168248d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb0a7fff640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb0940a3d20 0x7fb0940a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005dd0 tx=0x7fb0a0005d60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb09dffb640 1 -- 172.21.15.86:0/3627702882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09001d070 con 0x556bad0e2360 2024-04-16T05:43:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb09dffb640 1 -- 172.21.15.86:0/3627702882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0900a55c0 con 0x556bad0e2360 2024-04-16T05:43:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.967+0000 7fb09dffb640 1 -- 172.21.15.86:0/3627702882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0900ce380 con 0x556bad0e2360 2024-04-16T05:43:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3627702882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 msgr2=0x556bad0e4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0e4750 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb0900a5780 tx=0x7fb090005a00 comp rx=0 tx=0).stop 2024-04-16T05:43:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3627702882 shutdown_connections 2024-04-16T05:43:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0e4750 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 0x556bad0e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3627702882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 0x556bad0d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3627702882 >> 172.21.15.86:0/3627702882 conn(0x556bad0d3590 msgr2=0x556bad0d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3627702882 shutdown_connections 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3627702882 wait complete. 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.968+0000 7fb0b0bcb740 1 -- start start 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0b0bcb740 1 Processor -- start 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 0x556bad0df460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 0x556bad0df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bad0e4fa0 con 0x556bad0e2360 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556bacff74d0 con 0x556bad0d9780 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bad0e5120 con 0x556bad0d8e60 2024-04-16T05:43:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0b0bcb740 10 client.0 _ll_register_callbacks cb 0x556bacfdc2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0dfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56728/0 (socket says 172.21.15.86:56728) 2024-04-16T05:43:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a7fff640 1 -- 172.21.15.86:0/3144421923 learned_addr learned my addr 172.21.15.86:0/3144421923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 0x556bad0df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 0x556bad0df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a7fff640 1 -- 172.21.15.86:0/3144421923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 msgr2=0x556bad0df460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a7fff640 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 0x556bad0df460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a7fff640 1 -- 172.21.15.86:0/3144421923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 msgr2=0x556bad0df9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a7fff640 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 0x556bad0df9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.969+0000 7fb0a7fff640 1 -- 172.21.15.86:0/3144421923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bacd503f0 con 0x556bad0e2360 2024-04-16T05:43:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0a7fff640 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0dfee0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0010fd0 tx=0x7fb0a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8656408177f33f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb086ffd640 1 -- 172.21.15.86:0/3144421923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0025070 con 0x556bad0e2360 2024-04-16T05:43:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3144421923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556bad04bb00 con 0x556bad0e2360 2024-04-16T05:43:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb086ffd640 1 -- 172.21.15.86:0/3144421923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fb0a001f860 con 0x556bad0e2360 2024-04-16T05:43:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb086ffd640 1 -- 172.21.15.86:0/3144421923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0a0031050 con 0x556bad0e2360 2024-04-16T05:43:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0b0bcb740 10 client.18096 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3144421923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bace1a7d0 con 0x556bad0e2360 2024-04-16T05:43:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb086ffd640 1 -- 172.21.15.86:0/3144421923 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556bad04bb00 con 0x556bad0e2360 2024-04-16T05:43:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb086ffd640 1 -- 172.21.15.86:0/3144421923 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb0a001e850 con 0x556bad0e2360 2024-04-16T05:43:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0b0bcb740 10 client.18096 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0b0bcb740 10 client.18096 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:43:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0b0bcb740 20 client.18096 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x556bad0f2200 0x556bad0f4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0b0bcb740 4 client.18096 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.970+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3144421923 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bad045970 con 0x556bad0f2200 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.971+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x556bad0f2200 0x556bad0f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.971+0000 7fb086ffd640 10 client.18096 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.971+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x556bad0f2200 0x556bad0f4660 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=775fffe821b573c1 server_cookie=a8f42fa74afaf5c4 in_seq=0 out_seq=0 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb086ffd640 1 -- 172.21.15.86:0/3144421923 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556bad045970 con 0x556bad0f2200 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb086ffd640 10 client.18096 handle_command_reply: tid=0 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 client.18096 shutdown 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3144421923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 msgr2=0x556bad0dfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0dfee0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0010fd0 tx=0x7fb0a00046c0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3144421923 shutdown_connections 2024-04-16T05:43:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x556bad0f2200 0x556bad0f4660 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0e2360 0x556bad0dfee0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0d9780 0x556bad0df9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb086ffd640 0 client.18096 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:43:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3144421923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d8e60 0x556bad0df460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3144421923 >> 172.21.15.86:0/3144421923 conn(0x556bad0d3590 msgr2=0x556bad0d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3144421923 shutdown_connections 2024-04-16T05:43:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3144421923 wait complete. 2024-04-16T05:43:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.972+0000 7fb0b0bcb740 20 client.18096 trim_cache size 0 max 16384 2024-04-16T05:43:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0b0bcb740 1 Processor -- start 2024-04-16T05:43:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0b0bcb740 1 -- start start 2024-04-16T05:43:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ded60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 0x556bad0edef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0e2360 0x556bad0e1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81af560 con 0x556bad0d8e60 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81b10a0 con 0x556bad0e2360 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81afc30 con 0x556bad0d9780 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0a77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ded60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0a77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ded60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56744/0 (socket says 172.21.15.86:56744) 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0a77fe640 1 -- 172.21.15.86:0/3544840277 learned_addr learned my addr 172.21.15.86:0/3544840277 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0a77fe640 1 -- 172.21.15.86:0/3544840277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 msgr2=0x556bad0edef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/3544840277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 0x556bad0edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0a77fe640 1 -- 172.21.15.86:0/3544840277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0e2360 msgr2=0x556bad0e1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/3544840277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0e2360 0x556bad0e1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0a77fe640 1 -- 172.21.15.86:0/3544840277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bad0d4750 con 0x556bad0d8e60 2024-04-16T05:43:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.986+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/3544840277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ded60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb098000c00 tx=0x7fb098005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29004b9700c4cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb09d7fa640 1 -- 172.21.15.86:0/3544840277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09802cd00 con 0x556bad0d8e60 2024-04-16T05:43:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb09d7fa640 1 -- 172.21.15.86:0/3544840277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0980046c0 con 0x556bad0d8e60 2024-04-16T05:43:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb09d7fa640 1 -- 172.21.15.86:0/3544840277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098005380 con 0x556bad0d8e60 2024-04-16T05:43:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3544840277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 msgr2=0x556bad0ded60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3544840277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ded60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb098000c00 tx=0x7fb098005d50 comp rx=0 tx=0).stop 2024-04-16T05:43:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3544840277 shutdown_connections 2024-04-16T05:43:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3544840277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0e2360 0x556bad0e1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3544840277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 0x556bad0edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/3544840277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ded60 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3544840277 >> 172.21.15.86:0/3544840277 conn(0x556bad0d3590 msgr2=0x556bad0d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3544840277 shutdown_connections 2024-04-16T05:43:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.987+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/3544840277 wait complete. 2024-04-16T05:43:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0b0bcb740 1 -- start start 2024-04-16T05:43:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0b0bcb740 1 Processor -- start 2024-04-16T05:43:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 0x556bad0ea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0b0bcb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0e2360 0x556bad0dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81b09d0 con 0x556bad0d8e60 2024-04-16T05:43:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81b0300 con 0x556bad0e2360 2024-04-16T05:43:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0b0bcb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81ad560 con 0x556bad0d9780 2024-04-16T05:43:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0b0bcb740 10 client.0 _ll_register_callbacks cb 0x7fb0a0010c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ea400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56760/0 (socket says 172.21.15.86:56760) 2024-04-16T05:43:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a77fe640 1 -- 172.21.15.86:0/1310168063 learned_addr learned my addr 172.21.15.86:0/1310168063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 0x556bad0ea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a77fe640 1 -- 172.21.15.86:0/1310168063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 msgr2=0x556bad0ea940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 0x556bad0ea940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a77fe640 1 -- 172.21.15.86:0/1310168063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0e2360 msgr2=0x556bad0dbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0e2360 0x556bad0dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a77fe640 1 -- 172.21.15.86:0/1310168063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bad0dc490 con 0x556bad0d8e60 2024-04-16T05:43:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.988+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 0x556bad0ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0a77fe640 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ea400 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb09804a040 tx=0x7fb09802cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb724c0fcbf94c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb086ffd640 1 -- 172.21.15.86:0/1310168063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098004c80 con 0x556bad0d8e60 2024-04-16T05:43:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb086ffd640 1 -- 172.21.15.86:0/1310168063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fb098005100 con 0x556bad0d8e60 2024-04-16T05:43:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/1310168063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0980046c0 con 0x556bad0d8e60 2024-04-16T05:43:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb086ffd640 1 -- 172.21.15.86:0/1310168063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb098005300 con 0x556bad0d8e60 2024-04-16T05:43:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0b0bcb740 10 client.18102 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/1310168063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bad158360 con 0x556bad0d8e60 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb086ffd640 1 -- 172.21.15.86:0/1310168063 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0980046c0 con 0x556bad0d8e60 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb086ffd640 1 -- 172.21.15.86:0/1310168063 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb09805b030 con 0x556bad0d8e60 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0b0bcb740 10 client.18102 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0b0bcb740 10 client.18102 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0b0bcb740 20 client.18102 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x556bad158a10 0x556bad15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0b0bcb740 4 client.18102 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.989+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/1310168063 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bacfc16b0 con 0x556bad158a10 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.990+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x556bad158a10 0x556bad15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.990+0000 7fb086ffd640 10 client.18102 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:43:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.990+0000 7fb0a6ffd640 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x556bad158a10 0x556bad15ae30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=768353ff75f3c82a server_cookie=ffd1cad4cd73791d in_seq=0 out_seq=0 2024-04-16T05:43:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.990+0000 7fb086ffd640 1 -- 172.21.15.86:0/1310168063 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bacfc16b0 con 0x556bad158a10 2024-04-16T05:43:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.990+0000 7fb086ffd640 10 client.18102 handle_command_reply: tid=0 2024-04-16T05:43:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.990+0000 7fb0b0bcb740 1 client.18102 shutdown 2024-04-16T05:43:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.990+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/1310168063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 msgr2=0x556bad0ea400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.990+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ea400 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb09804a040 tx=0x7fb09802cbf0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/1310168063 shutdown_connections 2024-04-16T05:43:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x556bad158a10 0x556bad15ae30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bad0e2360 0x556bad0dbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb086ffd640 0 client.18102 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:43:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad0d9780 0x556bad0ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0b0bcb740 1 --2- 172.21.15.86:0/1310168063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bad0d8e60 0x556bad0ea400 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/1310168063 >> 172.21.15.86:0/1310168063 conn(0x556bad0d3590 msgr2=0x556bad0d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/1310168063 shutdown_connections 2024-04-16T05:43:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0b0bcb740 1 -- 172.21.15.86:0/1310168063 wait complete. 2024-04-16T05:43:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0b0bcb740 20 client.18102 trim_cache size 0 max 16384 2024-04-16T05:43:32.998 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:43:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 -- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb0940a3d20 msgr2=0x7fb0940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb0940a3d20 0x7fb0940a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005dd0 tx=0x7fb0a0005d60 comp rx=0 tx=0).stop 2024-04-16T05:43:33.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 -- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8101bf0 msgr2=0x7fb0a819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8101bf0 0x7fb0a819d940 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fb09000cce0 tx=0x7fb09000a620 comp rx=0 tx=0).stop 2024-04-16T05:43:33.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 -- 172.21.15.86:0/2715912738 shutdown_connections 2024-04-16T05:43:33.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a8107040 0x7fb0a81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8101bf0 0x7fb0a819d940 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb0940a3d20 0x7fb0940a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 --2- 172.21.15.86:0/2715912738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0a8101250 0x7fb0a819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 -- 172.21.15.86:0/2715912738 >> 172.21.15.86:0/2715912738 conn(0x7fb0a8076d00 msgr2=0x7fb0a8100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:33.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 -- 172.21.15.86:0/2715912738 shutdown_connections 2024-04-16T05:43:33.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:32.991+0000 7fb0adf0d640 1 -- 172.21.15.86:0/2715912738 wait complete. 2024-04-16T05:43:33.002 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:43:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.110+0000 7f608ffc8640 1 -- 172.21.15.86:0/3277703641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088102280 msgr2=0x7f6088105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.110+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3277703641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088102280 0x7f6088105b30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f607c009900 tx=0x7f607c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.110+0000 7f608ffc8640 1 -- 172.21.15.86:0/3277703641 shutdown_connections 2024-04-16T05:43:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.110+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3277703641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6088106210 0x7f608810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.110+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3277703641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088102280 0x7f6088105b30 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.110+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3277703641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60881018d0 0x7f6088101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 -- 172.21.15.86:0/3277703641 >> 172.21.15.86:0/3277703641 conn(0x7f6088077410 msgr2=0x7f60880fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:33.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 -- 172.21.15.86:0/3277703641 shutdown_connections 2024-04-16T05:43:33.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 -- 172.21.15.86:0/3277703641 wait complete. 2024-04-16T05:43:33.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 Processor -- start 2024-04-16T05:43:33.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 -- start start 2024-04-16T05:43:33.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60881018d0 0x7f608819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6088106210 0x7f608819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088198810 0x7f6088198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6088111320 con 0x7f6088198810 2024-04-16T05:43:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f60881111a0 con 0x7f6088106210 2024-04-16T05:43:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608ffc8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60881114a0 con 0x7f60881018d0 2024-04-16T05:43:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.111+0000 7f608e565640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088198810 0x7f6088198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608e565640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088198810 0x7f6088198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56768/0 (socket says 172.21.15.86:56768) 2024-04-16T05:43:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608e565640 1 -- 172.21.15.86:0/3115617717 learned_addr learned my addr 172.21.15.86:0/3115617717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608dd64640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60881018d0 0x7f608819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608e565640 1 -- 172.21.15.86:0/3115617717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60881018d0 msgr2=0x7f608819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608e565640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60881018d0 0x7f608819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608e565640 1 -- 172.21.15.86:0/3115617717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6088106210 msgr2=0x7f608819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608d563640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6088106210 0x7f608819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608e565640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6088106210 0x7f608819eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608e565640 1 -- 172.21.15.86:0/3115617717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60881993a0 con 0x7f6088198810 2024-04-16T05:43:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608e565640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088198810 0x7f6088198cc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f608000cad0 tx=0x7f608000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f6086ffd640 1 -- 172.21.15.86:0/3115617717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608000ae20 con 0x7f6088198810 2024-04-16T05:43:33.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f6086ffd640 1 -- 172.21.15.86:0/3115617717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6080005020 con 0x7f6088198810 2024-04-16T05:43:33.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f6086ffd640 1 -- 172.21.15.86:0/3115617717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608000f6f0 con 0x7f6088198810 2024-04-16T05:43:33.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608ffc8640 1 -- 172.21.15.86:0/3115617717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6088199690 con 0x7f6088198810 2024-04-16T05:43:33.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.112+0000 7f608ffc8640 1 -- 172.21.15.86:0/3115617717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60881a57d0 con 0x7f6088198810 2024-04-16T05:43:33.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.114+0000 7f6086ffd640 1 -- 172.21.15.86:0/3115617717 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f60800051d0 con 0x7f6088198810 2024-04-16T05:43:33.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f6092bb3740 1 Processor -- start 2024-04-16T05:43:33.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f6092bb3740 1 -- start start 2024-04-16T05:43:33.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d77430 0x5591a0d84230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d84770 0x5591a0d86b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f6092bb3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591a0cdc0e0 con 0x5591a0d76b10 2024-04-16T05:43:33.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f6092bb3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5591a0c79520 con 0x5591a0d84770 2024-04-16T05:43:33.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f6092bb3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5591a0cd80a0 con 0x5591a0d77430 2024-04-16T05:43:33.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f6086ffd640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f60700a3da0 0x7f60700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f608d563640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f608d563640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d76ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56782/0 (socket says 172.21.15.86:56782) 2024-04-16T05:43:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f608d563640 1 -- 172.21.15.86:0/3871513840 learned_addr learned my addr 172.21.15.86:0/3871513840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f608e565640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f60700a3da0 0x7f60700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.115+0000 7f608dd64640 1 --2- 172.21.15.86:0/3871513840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d77430 0x5591a0d84230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f6086ffd640 1 -- 172.21.15.86:0/3115617717 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f60800d0f60 con 0x7f6088198810 2024-04-16T05:43:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f608d563640 1 -- 172.21.15.86:0/3871513840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d77430 msgr2=0x5591a0d84230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f608d563640 1 --2- 172.21.15.86:0/3871513840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d77430 0x5591a0d84230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f608d563640 1 -- 172.21.15.86:0/3871513840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d84770 msgr2=0x5591a0d86b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f608d563640 1 --2- 172.21.15.86:0/3871513840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d84770 0x5591a0d86b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f608d563640 1 -- 172.21.15.86:0/3871513840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a0ca18d0 con 0x5591a0d76b10 2024-04-16T05:43:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f608dd64640 1 --2- 172.21.15.86:0/3871513840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d77430 0x5591a0d84230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f608d563640 1 --2- 172.21.15.86:0/3871513840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d76ef0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f607c036040 tx=0x7f607c02c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc911470ff692e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f608e565640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f60700a3da0 0x7f60700a6260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f608009c830 tx=0x7f60800bd040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f606ffff640 1 -- 172.21.15.86:0/3871513840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c043020 con 0x5591a0d76b10 2024-04-16T05:43:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f606ffff640 1 -- 172.21.15.86:0/3871513840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f607c02ca80 con 0x5591a0d76b10 2024-04-16T05:43:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.116+0000 7f606ffff640 1 -- 172.21.15.86:0/3871513840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c0046c0 con 0x5591a0d76b10 2024-04-16T05:43:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 -- 172.21.15.86:0/3871513840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 msgr2=0x5591a0d76ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 --2- 172.21.15.86:0/3871513840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d76ef0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f607c036040 tx=0x7f607c02c6a0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 -- 172.21.15.86:0/3871513840 shutdown_connections 2024-04-16T05:43:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 --2- 172.21.15.86:0/3871513840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d84770 0x5591a0d86b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 --2- 172.21.15.86:0/3871513840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d77430 0x5591a0d84230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 --2- 172.21.15.86:0/3871513840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d76ef0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 -- 172.21.15.86:0/3871513840 >> 172.21.15.86:0/3871513840 conn(0x5591a0d71590 msgr2=0x5591a0d71970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 -- 172.21.15.86:0/3871513840 shutdown_connections 2024-04-16T05:43:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 -- 172.21.15.86:0/3871513840 wait complete. 2024-04-16T05:43:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 -- start start 2024-04-16T05:43:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 Processor -- start 2024-04-16T05:43:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 0x5591a0d81780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.117+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d81cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d82200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f6092bb3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591a0d87550 con 0x5591a0d84770 2024-04-16T05:43:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f6092bb3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5591a0c954d0 con 0x5591a0d77430 2024-04-16T05:43:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f6092bb3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5591a0d876d0 con 0x5591a0d76b10 2024-04-16T05:43:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f6092bb3740 10 client.0 _ll_register_callbacks cb 0x5591a0c7a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d82200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d82200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56794/0 (socket says 172.21.15.86:56794) 2024-04-16T05:43:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 -- 172.21.15.86:0/405690487 learned_addr learned my addr 172.21.15.86:0/405690487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 0x5591a0d81780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608dd64640 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d81cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 -- 172.21.15.86:0/405690487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 msgr2=0x5591a0d81780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 0x5591a0d81780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 -- 172.21.15.86:0/405690487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 msgr2=0x5591a0d81cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d81cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 -- 172.21.15.86:0/405690487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a09ee3f0 con 0x5591a0d84770 2024-04-16T05:43:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f608d563640 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d82200 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f607c02ce60 tx=0x7f607c0054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8704afc2ed4ded68 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f606d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f606d7fa640 1 -- 172.21.15.86:0/405690487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c043040 con 0x5591a0d84770 2024-04-16T05:43:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f606d7fa640 1 -- 172.21.15.86:0/405690487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f607c004ad0 con 0x5591a0d84770 2024-04-16T05:43:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f6092bb3740 1 -- 172.21.15.86:0/405690487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f607c02ca80 con 0x5591a0d84770 2024-04-16T05:43:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f606d7fa640 1 -- 172.21.15.86:0/405690487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f607c004cd0 con 0x5591a0d84770 2024-04-16T05:43:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f6092bb3740 10 client.18114 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.118+0000 7f6092bb3740 1 -- 172.21.15.86:0/405690487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591a0ab87d0 con 0x5591a0d84770 2024-04-16T05:43:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.119+0000 7f606d7fa640 1 -- 172.21.15.86:0/405690487 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f607c02ca80 con 0x5591a0d84770 2024-04-16T05:43:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.119+0000 7f606d7fa640 1 -- 172.21.15.86:0/405690487 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f607c03f4e0 con 0x5591a0d84770 2024-04-16T05:43:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.119+0000 7f6092bb3740 10 client.18114 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.119+0000 7f6092bb3740 10 client.18114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.119+0000 7f6092bb3740 20 client.18114 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.119+0000 7f6092bb3740 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5591a0d945a0 0x5591a0d96990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.119+0000 7f6092bb3740 4 client.18114 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.119+0000 7f6092bb3740 1 -- 172.21.15.86:0/405690487 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591a0ce9b00 con 0x5591a0d945a0 2024-04-16T05:43:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.119+0000 7f608dd64640 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5591a0d945a0 0x5591a0d96990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.120+0000 7f608dd64640 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5591a0d945a0 0x5591a0d96990 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a4d0b9b677550 server_cookie=da3578df7052bf6c in_seq=0 out_seq=0 2024-04-16T05:43:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.120+0000 7f606d7fa640 10 client.18114 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.120+0000 7f606d7fa640 1 -- 172.21.15.86:0/405690487 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591a0ce9b00 con 0x5591a0d945a0 2024-04-16T05:43:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.120+0000 7f606d7fa640 10 client.18114 handle_command_reply: tid=0 2024-04-16T05:43:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.120+0000 7f6092bb3740 1 client.18114 shutdown 2024-04-16T05:43:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.120+0000 7f6092bb3740 1 -- 172.21.15.86:0/405690487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 msgr2=0x5591a0d82200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.120+0000 7f6092bb3740 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d82200 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f607c02ce60 tx=0x7f607c0054c0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f6092bb3740 1 -- 172.21.15.86:0/405690487 shutdown_connections 2024-04-16T05:43:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f6092bb3740 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5591a0d945a0 0x5591a0d96990 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f6092bb3740 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d82200 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f606d7fa640 0 client.18114 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f6092bb3740 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d81cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f6092bb3740 1 --2- 172.21.15.86:0/405690487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 0x5591a0d81780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f6092bb3740 1 -- 172.21.15.86:0/405690487 >> 172.21.15.86:0/405690487 conn(0x5591a0d71590 msgr2=0x5591a0d732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f6092bb3740 1 -- 172.21.15.86:0/405690487 shutdown_connections 2024-04-16T05:43:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f6092bb3740 1 -- 172.21.15.86:0/405690487 wait complete. 2024-04-16T05:43:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.121+0000 7f6092bb3740 20 client.18114 trim_cache size 0 max 16384 2024-04-16T05:43:33.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f6092bb3740 1 Processor -- start 2024-04-16T05:43:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f6092bb3740 1 -- start start 2024-04-16T05:43:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d83eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d7e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d84770 0x5591a0d7ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f6092bb3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60881ad370 con 0x5591a0d76b10 2024-04-16T05:43:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f6092bb3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f60881aeeb0 con 0x5591a0d77430 2024-04-16T05:43:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f6092bb3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60881ada40 con 0x5591a0d84770 2024-04-16T05:43:33.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608d563640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d83eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608d563640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d83eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56808/0 (socket says 172.21.15.86:56808) 2024-04-16T05:43:33.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608d563640 1 -- 172.21.15.86:0/1304516881 learned_addr learned my addr 172.21.15.86:0/1304516881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:33.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608dd64640 1 --2- 172.21.15.86:0/1304516881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d7e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608d563640 1 -- 172.21.15.86:0/1304516881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d84770 msgr2=0x5591a0d7ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:33.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608d563640 1 --2- 172.21.15.86:0/1304516881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d84770 0x5591a0d7ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608d563640 1 -- 172.21.15.86:0/1304516881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 msgr2=0x5591a0d7e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:33.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608d563640 1 --2- 172.21.15.86:0/1304516881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d7e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608d563640 1 -- 172.21.15.86:0/1304516881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a0d76570 con 0x5591a0d76b10 2024-04-16T05:43:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f608d563640 1 --2- 172.21.15.86:0/1304516881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d83eb0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f607c000c00 tx=0x7f607c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8f9d83951f8916 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f606f7fe640 1 -- 172.21.15.86:0/1304516881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c043040 con 0x5591a0d76b10 2024-04-16T05:43:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f606f7fe640 1 -- 172.21.15.86:0/1304516881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f607c050070 con 0x5591a0d76b10 2024-04-16T05:43:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.134+0000 7f606f7fe640 1 -- 172.21.15.86:0/1304516881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c0048d0 con 0x5591a0d76b10 2024-04-16T05:43:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 -- 172.21.15.86:0/1304516881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 msgr2=0x5591a0d83eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1304516881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d83eb0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f607c000c00 tx=0x7f607c005e20 comp rx=0 tx=0).stop 2024-04-16T05:43:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 -- 172.21.15.86:0/1304516881 shutdown_connections 2024-04-16T05:43:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1304516881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d84770 0x5591a0d7ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1304516881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d7e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1304516881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d76b10 0x5591a0d83eb0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 -- 172.21.15.86:0/1304516881 >> 172.21.15.86:0/1304516881 conn(0x5591a0d71590 msgr2=0x5591a0d73270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 -- 172.21.15.86:0/1304516881 shutdown_connections 2024-04-16T05:43:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 -- 172.21.15.86:0/1304516881 wait complete. 2024-04-16T05:43:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 -- start start 2024-04-16T05:43:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.135+0000 7f6092bb3740 1 Processor -- start 2024-04-16T05:43:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 0x5591a0d99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d99f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f6092bb3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d96fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f6092bb3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60881ae7e0 con 0x5591a0d84770 2024-04-16T05:43:33.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f6092bb3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f60881ae110 con 0x5591a0d77430 2024-04-16T05:43:33.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f6092bb3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60881ab370 con 0x5591a0d76b10 2024-04-16T05:43:33.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f6092bb3740 10 client.0 _ll_register_callbacks cb 0x7f607c02c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:33.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d96fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d96fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56810/0 (socket says 172.21.15.86:56810) 2024-04-16T05:43:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 -- 172.21.15.86:0/1333182767 learned_addr learned my addr 172.21.15.86:0/1333182767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608dd64640 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d99f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 0x5591a0d99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 -- 172.21.15.86:0/1333182767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 msgr2=0x5591a0d99a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 0x5591a0d99a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 -- 172.21.15.86:0/1333182767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 msgr2=0x5591a0d99f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d99f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 -- 172.21.15.86:0/1333182767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a0d977a0 con 0x5591a0d84770 2024-04-16T05:43:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f608d563640 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d96fa0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f607c0059d0 tx=0x7f607c02cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50859b93a53e14a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f606d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f606d7fa640 1 -- 172.21.15.86:0/1333182767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c061040 con 0x5591a0d84770 2024-04-16T05:43:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f606d7fa640 1 -- 172.21.15.86:0/1333182767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f607c0048d0 con 0x5591a0d84770 2024-04-16T05:43:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.136+0000 7f6092bb3740 1 -- 172.21.15.86:0/1333182767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f607c050070 con 0x5591a0d84770 2024-04-16T05:43:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f606d7fa640 1 -- 172.21.15.86:0/1333182767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f607c043040 con 0x5591a0d84770 2024-04-16T05:43:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f6092bb3740 10 client.18120 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f6092bb3740 1 -- 172.21.15.86:0/1333182767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591a0dfa410 con 0x5591a0d84770 2024-04-16T05:43:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f606d7fa640 1 -- 172.21.15.86:0/1333182767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f607c050070 con 0x5591a0d84770 2024-04-16T05:43:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f606d7fa640 1 -- 172.21.15.86:0/1333182767 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f607c067020 con 0x5591a0d84770 2024-04-16T05:43:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f6092bb3740 10 client.18120 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f6092bb3740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f6092bb3740 20 client.18120 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5591a0dfaa50 0x5591a0dfce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f6092bb3740 4 client.18120 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f6092bb3740 1 -- 172.21.15.86:0/1333182767 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591a0ce3970 con 0x5591a0dfaa50 2024-04-16T05:43:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.137+0000 7f608dd64640 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5591a0dfaa50 0x5591a0dfce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f606d7fa640 10 client.18120 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f608dd64640 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5591a0dfaa50 0x5591a0dfce70 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea774868d08c2a58 server_cookie=d290650ac1753ea8 in_seq=0 out_seq=0 2024-04-16T05:43:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f606d7fa640 1 -- 172.21.15.86:0/1333182767 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591a0ce3970 con 0x5591a0dfaa50 2024-04-16T05:43:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f606d7fa640 10 client.18120 handle_command_reply: tid=0 2024-04-16T05:43:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 client.18120 shutdown 2024-04-16T05:43:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 -- 172.21.15.86:0/1333182767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 msgr2=0x5591a0d96fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d96fa0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f607c0059d0 tx=0x7f607c02cd60 comp rx=0 tx=0).stop 2024-04-16T05:43:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 -- 172.21.15.86:0/1333182767 shutdown_connections 2024-04-16T05:43:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5591a0dfaa50 0x5591a0dfce70 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a0d84770 0x5591a0d96fa0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f606d7fa640 0 client.18120 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:33.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591a0d77430 0x5591a0d99f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 --2- 172.21.15.86:0/1333182767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591a0d76b10 0x5591a0d99a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 -- 172.21.15.86:0/1333182767 >> 172.21.15.86:0/1333182767 conn(0x5591a0d71590 msgr2=0x5591a0d81720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:33.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 -- 172.21.15.86:0/1333182767 shutdown_connections 2024-04-16T05:43:33.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 1 -- 172.21.15.86:0/1333182767 wait complete. 2024-04-16T05:43:33.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.138+0000 7f6092bb3740 20 client.18120 trim_cache size 0 max 16384 2024-04-16T05:43:33.147 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:43:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 -- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f60700a3da0 msgr2=0x7f60700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f60700a3da0 0x7f60700a6260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f608009c830 tx=0x7f60800bd040 comp rx=0 tx=0).stop 2024-04-16T05:43:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 -- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088198810 msgr2=0x7f6088198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088198810 0x7f6088198cc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f608000cad0 tx=0x7f608000cf90 comp rx=0 tx=0).stop 2024-04-16T05:43:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 -- 172.21.15.86:0/3115617717 shutdown_connections 2024-04-16T05:43:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6088198810 0x7f6088198cc0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f60700a3da0 0x7f60700a6260 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6088106210 0x7f608819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 --2- 172.21.15.86:0/3115617717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60881018d0 0x7f608819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:33.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 -- 172.21.15.86:0/3115617717 >> 172.21.15.86:0/3115617717 conn(0x7f6088077410 msgr2=0x7f6088100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:33.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 -- 172.21.15.86:0/3115617717 shutdown_connections 2024-04-16T05:43:33.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:33.139+0000 7f608ffc8640 1 -- 172.21.15.86:0/3115617717 wait complete. 2024-04-16T05:43:33.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:33 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1046696753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:33.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:33 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1046696753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:33.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:33 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1046696753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:34.151 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:43:34.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:43:34.152 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:43:34.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.273+0000 7f47470d5640 1 -- 172.21.15.86:0/237156274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740102240 msgr2=0x7f4740102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.273+0000 7f47470d5640 1 --2- 172.21.15.86:0/237156274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740102240 0x7f4740102620 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f473c009900 tx=0x7f473c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 -- 172.21.15.86:0/237156274 shutdown_connections 2024-04-16T05:43:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 --2- 172.21.15.86:0/237156274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4740107310 0x7f474010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 --2- 172.21.15.86:0/237156274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4740102bf0 0x7f4740106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 --2- 172.21.15.86:0/237156274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740102240 0x7f4740102620 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 -- 172.21.15.86:0/237156274 >> 172.21.15.86:0/237156274 conn(0x7f47400fda40 msgr2=0x7f47400ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 -- 172.21.15.86:0/237156274 shutdown_connections 2024-04-16T05:43:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 -- 172.21.15.86:0/237156274 wait complete. 2024-04-16T05:43:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 Processor -- start 2024-04-16T05:43:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 -- start start 2024-04-16T05:43:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4740102240 0x7f474019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740107310 0x7f474019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4740198730 0x7f4740198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47401123f0 con 0x7f4740107310 2024-04-16T05:43:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4740112270 con 0x7f4740102240 2024-04-16T05:43:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.274+0000 7f47470d5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4740112570 con 0x7f4740198730 2024-04-16T05:43:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4737fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740107310 0x7f474019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4737fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740107310 0x7f474019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56830/0 (socket says 172.21.15.86:56830) 2024-04-16T05:43:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4737fff640 1 -- 172.21.15.86:0/151909260 learned_addr learned my addr 172.21.15.86:0/151909260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4745672640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4740198730 0x7f4740198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4737fff640 1 -- 172.21.15.86:0/151909260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4740198730 msgr2=0x7f4740198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4737fff640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4740198730 0x7f4740198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4737fff640 1 -- 172.21.15.86:0/151909260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4740102240 msgr2=0x7f474019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4744e71640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4740102240 0x7f474019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4737fff640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4740102240 0x7f474019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4737fff640 1 -- 172.21.15.86:0/151909260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4740199330 con 0x7f4740107310 2024-04-16T05:43:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4745672640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4740198730 0x7f4740198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4737fff640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740107310 0x7f474019ebb0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f473000c970 tx=0x7f473000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4735ffb640 1 -- 172.21.15.86:0/151909260 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730007bb0 con 0x7f4740107310 2024-04-16T05:43:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4735ffb640 1 -- 172.21.15.86:0/151909260 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4730007d50 con 0x7f4740107310 2024-04-16T05:43:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f47470d5640 1 -- 172.21.15.86:0/151909260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47401a5410 con 0x7f4740107310 2024-04-16T05:43:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f4735ffb640 1 -- 172.21.15.86:0/151909260 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473000f450 con 0x7f4740107310 2024-04-16T05:43:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.275+0000 7f47470d5640 1 -- 172.21.15.86:0/151909260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47401a5870 con 0x7f4740107310 2024-04-16T05:43:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.277+0000 7f4735ffb640 1 -- 172.21.15.86:0/151909260 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f47300046c0 con 0x7f4740107310 2024-04-16T05:43:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4749d95740 1 Processor -- start 2024-04-16T05:43:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4749d95740 1 -- start start 2024-04-16T05:43:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4735ffb640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47280a3da0 0x7f47280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4749d95740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d37e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4749d95740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d40a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4749d95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4749d95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555605c9ce90 con 0x555605d40fb0 2024-04-16T05:43:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4749d95740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555605c3a4c0 con 0x555605d37ab0 2024-04-16T05:43:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4749d95740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555605c98e50 con 0x555605d383d0 2024-04-16T05:43:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4745672640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47280a3da0 0x7f47280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4735ffb640 1 -- 172.21.15.86:0/151909260 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f47300d0c60 con 0x7f4740107310 2024-04-16T05:43:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.278+0000 7f4745672640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4745672640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56840/0 (socket says 172.21.15.86:56840) 2024-04-16T05:43:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4745672640 1 -- 172.21.15.86:0/3161534072 learned_addr learned my addr 172.21.15.86:0/3161534072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4737fff640 1 --2- 172.21.15.86:0/3161534072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d40a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4744e71640 1 --2- 172.21.15.86:0/3161534072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d37e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4745672640 1 -- 172.21.15.86:0/3161534072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 msgr2=0x555605d40a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4745672640 1 --2- 172.21.15.86:0/3161534072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d40a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4745672640 1 -- 172.21.15.86:0/3161534072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 msgr2=0x555605d37e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4745672640 1 --2- 172.21.15.86:0/3161534072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d37e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4745672640 1 -- 172.21.15.86:0/3161534072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555605a777e0 con 0x555605d40fb0 2024-04-16T05:43:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4745672640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47280a3da0 0x7f47280a6260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4738012ae0 tx=0x7f473800ad30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4745672640 1 --2- 172.21.15.86:0/3161534072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d433a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f473800fc60 tx=0x7f4738019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92730ab033bb88a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4726ffd640 1 -- 172.21.15.86:0/3161534072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738022070 con 0x555605d40fb0 2024-04-16T05:43:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4726ffd640 1 -- 172.21.15.86:0/3161534072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473800dde0 con 0x555605d40fb0 2024-04-16T05:43:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.279+0000 7f4726ffd640 1 -- 172.21.15.86:0/3161534072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47380105d0 con 0x555605d40fb0 2024-04-16T05:43:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 -- 172.21.15.86:0/3161534072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 msgr2=0x555605d433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 --2- 172.21.15.86:0/3161534072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d433a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f473800fc60 tx=0x7f4738019000 comp rx=0 tx=0).stop 2024-04-16T05:43:34.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 -- 172.21.15.86:0/3161534072 shutdown_connections 2024-04-16T05:43:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 --2- 172.21.15.86:0/3161534072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d433a0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 --2- 172.21.15.86:0/3161534072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d40a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 --2- 172.21.15.86:0/3161534072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d37e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 -- 172.21.15.86:0/3161534072 >> 172.21.15.86:0/3161534072 conn(0x555605d321e0 msgr2=0x555605d325c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 -- 172.21.15.86:0/3161534072 shutdown_connections 2024-04-16T05:43:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 -- 172.21.15.86:0/3161534072 wait complete. 2024-04-16T05:43:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 -- start start 2024-04-16T05:43:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.280+0000 7f4749d95740 1 Processor -- start 2024-04-16T05:43:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4749d95740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d3e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4749d95740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d3e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4749d95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d3eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4749d95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555605d43d40 con 0x555605d40fb0 2024-04-16T05:43:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4749d95740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555605c25410 con 0x555605d37ab0 2024-04-16T05:43:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4749d95740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555605d43ec0 con 0x555605d383d0 2024-04-16T05:43:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4749d95740 10 client.0 _ll_register_callbacks cb 0x555605c20b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4745672640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d3eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4745672640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d3eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56852/0 (socket says 172.21.15.86:56852) 2024-04-16T05:43:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4745672640 1 -- 172.21.15.86:0/1798129744 learned_addr learned my addr 172.21.15.86:0/1798129744 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4744e71640 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d3e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4745672640 1 -- 172.21.15.86:0/1798129744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 msgr2=0x555605d3e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4737fff640 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d3e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4745672640 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d3e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4745672640 1 -- 172.21.15.86:0/1798129744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 msgr2=0x555605d3e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4745672640 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d3e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4745672640 1 -- 172.21.15.86:0/1798129744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555605ad7f70 con 0x555605d40fb0 2024-04-16T05:43:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4744e71640 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d3e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f4745672640 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d3eaa0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f473801eab0 tx=0x7f473800f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a343394397beccab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.281+0000 7f471ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f471ffff640 1 -- 172.21.15.86:0/1798129744 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738035060 con 0x555605d40fb0 2024-04-16T05:43:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f471ffff640 1 -- 172.21.15.86:0/1798129744 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f4738027450 con 0x555605d40fb0 2024-04-16T05:43:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4749d95740 1 -- 172.21.15.86:0/1798129744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473800dde0 con 0x555605d40fb0 2024-04-16T05:43:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f471ffff640 1 -- 172.21.15.86:0/1798129744 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4738022040 con 0x555605d40fb0 2024-04-16T05:43:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4749d95740 10 client.18132 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4749d95740 1 -- 172.21.15.86:0/1798129744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55560596f170 con 0x555605d40fb0 2024-04-16T05:43:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f471ffff640 1 -- 172.21.15.86:0/1798129744 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473800dde0 con 0x555605d40fb0 2024-04-16T05:43:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f471ffff640 1 -- 172.21.15.86:0/1798129744 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f473802f030 con 0x555605d40fb0 2024-04-16T05:43:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4749d95740 10 client.18132 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4749d95740 10 client.18132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4749d95740 20 client.18132 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4749d95740 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555605d51340 0x555605d53730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4749d95740 4 client.18132 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4749d95740 1 -- 172.21.15.86:0/1798129744 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555605caa8b0 con 0x555605d51340 2024-04-16T05:43:34.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.282+0000 7f4744e71640 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555605d51340 0x555605d53730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.283+0000 7f4744e71640 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555605d51340 0x555605d53730 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2695c43acab55349 server_cookie=1dd6dd239345b8ab in_seq=0 out_seq=0 2024-04-16T05:43:34.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.283+0000 7f471ffff640 10 client.18132 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:34.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f471ffff640 1 -- 172.21.15.86:0/1798129744 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555605caa8b0 con 0x555605d51340 2024-04-16T05:43:34.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f471ffff640 10 client.18132 handle_command_reply: tid=0 2024-04-16T05:43:34.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 client.18132 shutdown 2024-04-16T05:43:34.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 -- 172.21.15.86:0/1798129744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 msgr2=0x555605d3eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d3eaa0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f473801eab0 tx=0x7f473800f880 comp rx=0 tx=0).stop 2024-04-16T05:43:34.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 -- 172.21.15.86:0/1798129744 shutdown_connections 2024-04-16T05:43:34.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555605d51340 0x555605d53730 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605d3eaa0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d3e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f471ffff640 0 client.18132 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:34.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 --2- 172.21.15.86:0/1798129744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d3e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 -- 172.21.15.86:0/1798129744 >> 172.21.15.86:0/1798129744 conn(0x555605d321e0 msgr2=0x555605d33f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 -- 172.21.15.86:0/1798129744 shutdown_connections 2024-04-16T05:43:34.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 1 -- 172.21.15.86:0/1798129744 wait complete. 2024-04-16T05:43:34.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.284+0000 7f4749d95740 20 client.18132 trim_cache size 0 max 16384 2024-04-16T05:43:34.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4749d95740 1 Processor -- start 2024-04-16T05:43:34.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4749d95740 1 -- start start 2024-04-16T05:43:34.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4749d95740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d37ab0 0x555605d4cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4749d95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d383d0 0x555605d3d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4749d95740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d40fb0 0x555605d58e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4749d95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47401af570 con 0x555605d383d0 2024-04-16T05:43:34.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4749d95740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f47401b10b0 con 0x555605d40fb0 2024-04-16T05:43:34.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4749d95740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47401afc40 con 0x555605d37ab0 2024-04-16T05:43:34.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4737fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d383d0 0x555605d3d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4737fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d383d0 0x555605d3d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56862/0 (socket says 172.21.15.86:56862) 2024-04-16T05:43:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4737fff640 1 -- 172.21.15.86:0/2784806648 learned_addr learned my addr 172.21.15.86:0/2784806648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4745672640 1 --2- 172.21.15.86:0/2784806648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d40fb0 0x555605d58e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4737fff640 1 -- 172.21.15.86:0/2784806648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d37ab0 msgr2=0x555605d4cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4737fff640 1 --2- 172.21.15.86:0/2784806648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d37ab0 0x555605d4cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4737fff640 1 -- 172.21.15.86:0/2784806648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d40fb0 msgr2=0x555605d58e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4737fff640 1 --2- 172.21.15.86:0/2784806648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d40fb0 0x555605d58e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4737fff640 1 -- 172.21.15.86:0/2784806648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555605d433a0 con 0x555605d383d0 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f4737fff640 1 --2- 172.21.15.86:0/2784806648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d383d0 0x555605d3d7d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4730000c00 tx=0x7f473009c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d548c41fe3687c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f47267fc640 1 -- 172.21.15.86:0/2784806648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473001d070 con 0x555605d383d0 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f47267fc640 1 -- 172.21.15.86:0/2784806648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4730003610 con 0x555605d383d0 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.302+0000 7f47267fc640 1 -- 172.21.15.86:0/2784806648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47300ce260 con 0x555605d383d0 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 -- 172.21.15.86:0/2784806648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d383d0 msgr2=0x555605d3d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 --2- 172.21.15.86:0/2784806648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d383d0 0x555605d3d7d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4730000c00 tx=0x7f473009c190 comp rx=0 tx=0).stop 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 -- 172.21.15.86:0/2784806648 shutdown_connections 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 --2- 172.21.15.86:0/2784806648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d40fb0 0x555605d58e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 --2- 172.21.15.86:0/2784806648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d383d0 0x555605d3d7d0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 --2- 172.21.15.86:0/2784806648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d37ab0 0x555605d4cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 -- 172.21.15.86:0/2784806648 >> 172.21.15.86:0/2784806648 conn(0x555605d321e0 msgr2=0x555605d3e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 -- 172.21.15.86:0/2784806648 shutdown_connections 2024-04-16T05:43:34.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 -- 172.21.15.86:0/2784806648 wait complete. 2024-04-16T05:43:34.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 -- start start 2024-04-16T05:43:34.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.303+0000 7f4749d95740 1 Processor -- start 2024-04-16T05:43:34.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4749d95740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4749d95740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d45a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4749d95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605db0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4749d95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47401b09e0 con 0x555605d40fb0 2024-04-16T05:43:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4749d95740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f47401b0310 con 0x555605d37ab0 2024-04-16T05:43:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4749d95740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47401ad570 con 0x555605d383d0 2024-04-16T05:43:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4749d95740 10 client.0 _ll_register_callbacks cb 0x7f4730019f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4745672640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605db0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4745672640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605db0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56864/0 (socket says 172.21.15.86:56864) 2024-04-16T05:43:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4745672640 1 -- 172.21.15.86:0/2151134038 learned_addr learned my addr 172.21.15.86:0/2151134038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4744e71640 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4737fff640 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d45a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4745672640 1 -- 172.21.15.86:0/2151134038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 msgr2=0x555605d45a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4745672640 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d45a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4745672640 1 -- 172.21.15.86:0/2151134038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 msgr2=0x555605d454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4745672640 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4745672640 1 -- 172.21.15.86:0/2151134038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555605d31800 con 0x555605d40fb0 2024-04-16T05:43:34.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4744e71640 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:34.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f4745672640 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605db0700 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f473801ed90 tx=0x7f473800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b8159758431f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f471ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:34.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f471ffff640 1 -- 172.21.15.86:0/2151134038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738035040 con 0x555605d40fb0 2024-04-16T05:43:34.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.304+0000 7f471ffff640 1 -- 172.21.15.86:0/2151134038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f473801e850 con 0x555605d40fb0 2024-04-16T05:43:34.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4749d95740 1 -- 172.21.15.86:0/2151134038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4730003610 con 0x555605d40fb0 2024-04-16T05:43:34.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f471ffff640 1 -- 172.21.15.86:0/2151134038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f473801eae0 con 0x555605d40fb0 2024-04-16T05:43:34.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4749d95740 10 client.18138 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:34.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4749d95740 1 -- 172.21.15.86:0/2151134038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555605db7f90 con 0x555605d40fb0 2024-04-16T05:43:34.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f471ffff640 1 -- 172.21.15.86:0/2151134038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4730003610 con 0x555605d40fb0 2024-04-16T05:43:34.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f471ffff640 1 -- 172.21.15.86:0/2151134038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f473802f040 con 0x555605d40fb0 2024-04-16T05:43:34.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4749d95740 10 client.18138 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:34.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4749d95740 10 client.18138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4749d95740 20 client.18138 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4749d95740 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555605db86d0 0x555605dbaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4749d95740 4 client.18138 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:43:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4749d95740 1 -- 172.21.15.86:0/2151134038 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555605ca4720 con 0x555605db86d0 2024-04-16T05:43:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.305+0000 7f4744e71640 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555605db86d0 0x555605dbaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.306+0000 7f4744e71640 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555605db86d0 0x555605dbaaf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b81d52723d0ec server_cookie=c35971413cd258a8 in_seq=0 out_seq=0 2024-04-16T05:43:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.306+0000 7f471ffff640 10 client.18138 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:34.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f471ffff640 1 -- 172.21.15.86:0/2151134038 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555605ca4720 con 0x555605db86d0 2024-04-16T05:43:34.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f471ffff640 10 client.18138 handle_command_reply: tid=0 2024-04-16T05:43:34.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f4749d95740 1 client.18138 shutdown 2024-04-16T05:43:34.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f4749d95740 1 -- 172.21.15.86:0/2151134038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 msgr2=0x555605db0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f4749d95740 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605db0700 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f473801ed90 tx=0x7f473800f730 comp rx=0 tx=0).stop 2024-04-16T05:43:34.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f4749d95740 1 -- 172.21.15.86:0/2151134038 shutdown_connections 2024-04-16T05:43:34.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f4749d95740 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x555605db86d0 0x555605dbaaf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f4749d95740 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555605d40fb0 0x555605db0700 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f471ffff640 0 client.18138 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:34.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f4749d95740 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555605d383d0 0x555605d45a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f4749d95740 1 --2- 172.21.15.86:0/2151134038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555605d37ab0 0x555605d454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.307+0000 7f4749d95740 1 -- 172.21.15.86:0/2151134038 >> 172.21.15.86:0/2151134038 conn(0x555605d321e0 msgr2=0x555605d33530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f4749d95740 1 -- 172.21.15.86:0/2151134038 shutdown_connections 2024-04-16T05:43:34.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f4749d95740 1 -- 172.21.15.86:0/2151134038 wait complete. 2024-04-16T05:43:34.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f4749d95740 20 client.18138 trim_cache size 0 max 16384 2024-04-16T05:43:34.315 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:43:34.315 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:43:34.315 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "4a62a2d7-274d-4d15-b052-918147710e0b", 2024-04-16T05:43:34.316 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:43:34.316 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:43:34.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 -- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47280a3da0 msgr2=0x7f47280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47280a3da0 0x7f47280a6260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4738012ae0 tx=0x7f473800ad30 comp rx=0 tx=0).stop 2024-04-16T05:43:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 -- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740107310 msgr2=0x7f474019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740107310 0x7f474019ebb0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f473000c970 tx=0x7f473000ce30 comp rx=0 tx=0).stop 2024-04-16T05:43:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 -- 172.21.15.86:0/151909260 shutdown_connections 2024-04-16T05:43:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4740198730 0x7f4740198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740107310 0x7f474019ebb0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f47280a3da0 0x7f47280a6260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 --2- 172.21.15.86:0/151909260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4740102240 0x7f474019e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 -- 172.21.15.86:0/151909260 >> 172.21.15.86:0/151909260 conn(0x7f47400fda40 msgr2=0x7f47400ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 -- 172.21.15.86:0/151909260 shutdown_connections 2024-04-16T05:43:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.308+0000 7f47470d5640 1 -- 172.21.15.86:0/151909260 wait complete. 2024-04-16T05:43:34.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:43:34.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:43:34.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 -- 172.21.15.86:0/2411430980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 msgr2=0x7fea24100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 --2- 172.21.15.86:0/2411430980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 0x7fea24100ad0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fea1400aa40 tx=0x7fea1402cf50 comp rx=0 tx=0).stop 2024-04-16T05:43:34.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 -- 172.21.15.86:0/2411430980 shutdown_connections 2024-04-16T05:43:34.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 --2- 172.21.15.86:0/2411430980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea24101010 0x7fea2410d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 --2- 172.21.15.86:0/2411430980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 0x7fea24100ad0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 --2- 172.21.15.86:0/2411430980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24106660 0x7fea24106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 -- 172.21.15.86:0/2411430980 >> 172.21.15.86:0/2411430980 conn(0x7fea24077490 msgr2=0x7fea240fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 -- 172.21.15.86:0/2411430980 shutdown_connections 2024-04-16T05:43:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 -- 172.21.15.86:0/2411430980 wait complete. 2024-04-16T05:43:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 Processor -- start 2024-04-16T05:43:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.428+0000 7fea2ba37640 1 -- start start 2024-04-16T05:43:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea2ba37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 0x7fea2419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea2ba37640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea24101010 0x7fea2419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea2ba37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24106660 0x7fea241986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea2ba37640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea24110190 con 0x7fea24100650 2024-04-16T05:43:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea2ba37640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea24110010 con 0x7fea24101010 2024-04-16T05:43:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea2ba37640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea24110310 con 0x7fea24106660 2024-04-16T05:43:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea297d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 0x7fea2419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea297d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 0x7fea2419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56884/0 (socket says 172.21.15.86:56884) 2024-04-16T05:43:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea297d3640 1 -- 172.21.15.86:0/129606638 learned_addr learned my addr 172.21.15.86:0/129606638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea28fd2640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea24101010 0x7fea2419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea297d3640 1 -- 172.21.15.86:0/129606638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24106660 msgr2=0x7fea241986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea29fd4640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24106660 0x7fea241986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea297d3640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24106660 0x7fea241986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea297d3640 1 -- 172.21.15.86:0/129606638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea24101010 msgr2=0x7fea2419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea297d3640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea24101010 0x7fea2419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea297d3640 1 -- 172.21.15.86:0/129606638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea24198f90 con 0x7fea24100650 2024-04-16T05:43:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea28fd2640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea24101010 0x7fea2419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea29fd4640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24106660 0x7fea241986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea297d3640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 0x7fea2419e610 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fea2000c970 tx=0x7fea2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea1a7fc640 1 -- 172.21.15.86:0/129606638 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20007bb0 con 0x7fea24100650 2024-04-16T05:43:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea1a7fc640 1 -- 172.21.15.86:0/129606638 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea20007d50 con 0x7fea24100650 2024-04-16T05:43:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea2ba37640 1 -- 172.21.15.86:0/129606638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea24199280 con 0x7fea24100650 2024-04-16T05:43:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea1a7fc640 1 -- 172.21.15.86:0/129606638 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2000f450 con 0x7fea24100650 2024-04-16T05:43:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.429+0000 7fea2ba37640 1 -- 172.21.15.86:0/129606638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea241a5590 con 0x7fea24100650 2024-04-16T05:43:34.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.432+0000 7fea1a7fc640 1 -- 172.21.15.86:0/129606638 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea20005020 con 0x7fea24100650 2024-04-16T05:43:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.432+0000 7fea2e61c740 1 Processor -- start 2024-04-16T05:43:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.432+0000 7fea2e61c740 1 -- start start 2024-04-16T05:43:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.432+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09f7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.432+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.432+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0a00350 0x55e5e0a02740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.432+0000 7fea2e61c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5e095c180 con 0x55e5e09f7770 2024-04-16T05:43:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.432+0000 7fea2e61c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e5e08f9360 con 0x55e5e09f6e50 2024-04-16T05:43:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.432+0000 7fea2e61c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5e0958140 con 0x55e5e0a00350 2024-04-16T05:43:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea297d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea297d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56888/0 (socket says 172.21.15.86:56888) 2024-04-16T05:43:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea1a7fc640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea100a3d20 0x7fea100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea297d3640 1 -- 172.21.15.86:0/3059039680 learned_addr learned my addr 172.21.15.86:0/3059039680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea28fd2640 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0a00350 0x55e5e0a02740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea29fd4640 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea29fd4640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea100a3d20 0x7fea100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea1a7fc640 1 -- 172.21.15.86:0/129606638 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fea200d0a20 con 0x7fea24100650 2024-04-16T05:43:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea297d3640 1 -- 172.21.15.86:0/3059039680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0a00350 msgr2=0x55e5e0a02740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea297d3640 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0a00350 0x55e5e0a02740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea297d3640 1 -- 172.21.15.86:0/3059039680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 msgr2=0x55e5e09f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea297d3640 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea297d3640 1 -- 172.21.15.86:0/3059039680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e07387d0 con 0x55e5e09f7770 2024-04-16T05:43:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea29fd4640 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea297d3640 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09ffe10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fea2009a260 tx=0x7fea2009c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d229ab58133999 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea29fd4640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea100a3d20 0x7fea100a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00afd0 tx=0x7fea1c005f40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea0f7fe640 1 -- 172.21.15.86:0/3059039680 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2001d070 con 0x55e5e09f7770 2024-04-16T05:43:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea0f7fe640 1 -- 172.21.15.86:0/3059039680 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2009ccf0 con 0x55e5e09f7770 2024-04-16T05:43:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.433+0000 7fea0f7fe640 1 -- 172.21.15.86:0/3059039680 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea200ccee0 con 0x55e5e09f7770 2024-04-16T05:43:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 -- 172.21.15.86:0/3059039680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 msgr2=0x55e5e09ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09ffe10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fea2009a260 tx=0x7fea2009c4f0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 -- 172.21.15.86:0/3059039680 shutdown_connections 2024-04-16T05:43:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0a00350 0x55e5e0a02740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09ffe10 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 --2- 172.21.15.86:0/3059039680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 -- 172.21.15.86:0/3059039680 >> 172.21.15.86:0/3059039680 conn(0x55e5e09f1580 msgr2=0x55e5e09f1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 -- 172.21.15.86:0/3059039680 shutdown_connections 2024-04-16T05:43:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 -- 172.21.15.86:0/3059039680 wait complete. 2024-04-16T05:43:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 -- start start 2024-04-16T05:43:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.434+0000 7fea2e61c740 1 Processor -- start 2024-04-16T05:43:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09fd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0a00350 0x55e5e09fde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea2e61c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5e0a030e0 con 0x55e5e09f7770 2024-04-16T05:43:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea2e61c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e5e09152c0 con 0x55e5e09f6e50 2024-04-16T05:43:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea2e61c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5e0a03260 con 0x55e5e0a00350 2024-04-16T05:43:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea2e61c740 10 client.0 _ll_register_callbacks cb 0x55e5e08fc090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea28fd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea28fd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09fd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56904/0 (socket says 172.21.15.86:56904) 2024-04-16T05:43:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea28fd2640 1 -- 172.21.15.86:0/1824296917 learned_addr learned my addr 172.21.15.86:0/1824296917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea28fd2640 1 -- 172.21.15.86:0/1824296917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0a00350 msgr2=0x55e5e09fde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea28fd2640 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0a00350 0x55e5e09fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea28fd2640 1 -- 172.21.15.86:0/1824296917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 msgr2=0x55e5e09fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea297d3640 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea28fd2640 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea28fd2640 1 -- 172.21.15.86:0/1824296917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e066e3f0 con 0x55e5e09f7770 2024-04-16T05:43:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea28fd2640 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09fd950 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fea14037040 tx=0x7fea1402de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb59fe9c55510fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.435+0000 7fea0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea0cff9640 1 -- 172.21.15.86:0/1824296917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14043070 con 0x55e5e09f7770 2024-04-16T05:43:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea0cff9640 1 -- 172.21.15.86:0/1824296917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fea14004040 con 0x55e5e09f7770 2024-04-16T05:43:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea2e61c740 1 -- 172.21.15.86:0/1824296917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2009ccf0 con 0x55e5e09f7770 2024-04-16T05:43:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea0cff9640 1 -- 172.21.15.86:0/1824296917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea1404f050 con 0x55e5e09f7770 2024-04-16T05:43:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea2e61c740 10 client.18150 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea2e61c740 1 -- 172.21.15.86:0/1824296917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5e062e0f0 con 0x55e5e09f7770 2024-04-16T05:43:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea0cff9640 1 -- 172.21.15.86:0/1824296917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2009ccf0 con 0x55e5e09f7770 2024-04-16T05:43:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea0cff9640 1 -- 172.21.15.86:0/1824296917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fea1400b040 con 0x55e5e09f7770 2024-04-16T05:43:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea2e61c740 10 client.18150 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea2e61c740 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea2e61c740 20 client.18150 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea2e61c740 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0a106a0 0x55e5e0a12b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea2e61c740 4 client.18150 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.436+0000 7fea2e61c740 1 -- 172.21.15.86:0/1824296917 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5e0969ba0 con 0x55e5e0a106a0 2024-04-16T05:43:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.437+0000 7fea297d3640 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0a106a0 0x55e5e0a12b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.437+0000 7fea0cff9640 10 client.18150 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.437+0000 7fea297d3640 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0a106a0 0x55e5e0a12b60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f1159cbfc365d2 server_cookie=7b9172f325df9be2 in_seq=0 out_seq=0 2024-04-16T05:43:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea0cff9640 1 -- 172.21.15.86:0/1824296917 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5e0969ba0 con 0x55e5e0a106a0 2024-04-16T05:43:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea0cff9640 10 client.18150 handle_command_reply: tid=0 2024-04-16T05:43:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 client.18150 shutdown 2024-04-16T05:43:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 -- 172.21.15.86:0/1824296917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 msgr2=0x55e5e09fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09fd950 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fea14037040 tx=0x7fea1402de70 comp rx=0 tx=0).stop 2024-04-16T05:43:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 -- 172.21.15.86:0/1824296917 shutdown_connections 2024-04-16T05:43:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0a106a0 0x55e5e0a12b60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e0a00350 0x55e5e09fde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e09f7770 0x55e5e09fd950 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea0cff9640 0 client.18150 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 --2- 172.21.15.86:0/1824296917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 -- 172.21.15.86:0/1824296917 >> 172.21.15.86:0/1824296917 conn(0x55e5e09f1580 msgr2=0x55e5e09f32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 -- 172.21.15.86:0/1824296917 shutdown_connections 2024-04-16T05:43:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 1 -- 172.21.15.86:0/1824296917 wait complete. 2024-04-16T05:43:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.438+0000 7fea2e61c740 20 client.18150 trim_cache size 0 max 16384 2024-04-16T05:43:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.455+0000 7fea2e61c740 1 Processor -- start 2024-04-16T05:43:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.455+0000 7fea2e61c740 1 -- start start 2024-04-16T05:43:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e09f7770 0x55e5e0a0bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e09fc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea2e61c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea241af570 con 0x55e5e0a00350 2024-04-16T05:43:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea2e61c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea241b10b0 con 0x55e5e09f6e50 2024-04-16T05:43:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea2e61c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea241afc40 con 0x55e5e09f7770 2024-04-16T05:43:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea29fd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e09fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea29fd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e09fc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56908/0 (socket says 172.21.15.86:56908) 2024-04-16T05:43:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea29fd4640 1 -- 172.21.15.86:0/462909121 learned_addr learned my addr 172.21.15.86:0/462909121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea29fd4640 1 -- 172.21.15.86:0/462909121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e09f7770 msgr2=0x55e5e0a0bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea29fd4640 1 --2- 172.21.15.86:0/462909121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e09f7770 0x55e5e0a0bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea29fd4640 1 -- 172.21.15.86:0/462909121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 msgr2=0x55e5e09ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea297d3640 1 --2- 172.21.15.86:0/462909121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea29fd4640 1 --2- 172.21.15.86:0/462909121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09ffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea29fd4640 1 -- 172.21.15.86:0/462909121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e09f7bb0 con 0x55e5e0a00350 2024-04-16T05:43:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea297d3640 1 --2- 172.21.15.86:0/462909121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea29fd4640 1 --2- 172.21.15.86:0/462909121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e09fc570 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fea1c012310 tx=0x7fea1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db056e2296f13045 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea0effd640 1 -- 172.21.15.86:0/462909121 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c020030 con 0x55e5e0a00350 2024-04-16T05:43:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea0effd640 1 -- 172.21.15.86:0/462909121 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c005020 con 0x55e5e0a00350 2024-04-16T05:43:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.456+0000 7fea0effd640 1 -- 172.21.15.86:0/462909121 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c016040 con 0x55e5e0a00350 2024-04-16T05:43:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 -- 172.21.15.86:0/462909121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 msgr2=0x55e5e09fc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 --2- 172.21.15.86:0/462909121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e09fc570 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fea1c012310 tx=0x7fea1c004ab0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 -- 172.21.15.86:0/462909121 shutdown_connections 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 --2- 172.21.15.86:0/462909121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e09fc570 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 --2- 172.21.15.86:0/462909121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e09f7770 0x55e5e0a0bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 --2- 172.21.15.86:0/462909121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f6e50 0x55e5e09ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 -- 172.21.15.86:0/462909121 >> 172.21.15.86:0/462909121 conn(0x55e5e09f1580 msgr2=0x55e5e09f29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 -- 172.21.15.86:0/462909121 shutdown_connections 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 -- 172.21.15.86:0/462909121 wait complete. 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 -- start start 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.457+0000 7fea2e61c740 1 Processor -- start 2024-04-16T05:43:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e09f6e50 0x55e5e0a08680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f7770 0x55e5e0a05a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e0a05fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea241b09e0 con 0x55e5e0a00350 2024-04-16T05:43:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea241b0310 con 0x55e5e09f7770 2024-04-16T05:43:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea241ad570 con 0x55e5e09f6e50 2024-04-16T05:43:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 10 client.0 _ll_register_callbacks cb 0x7fea1402db00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea29fd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e0a05fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea29fd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e0a05fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56916/0 (socket says 172.21.15.86:56916) 2024-04-16T05:43:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea29fd4640 1 -- 172.21.15.86:0/2691674593 learned_addr learned my addr 172.21.15.86:0/2691674593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:43:34.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea28fd2640 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f7770 0x55e5e0a05a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea297d3640 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e09f6e50 0x55e5e0a08680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea29fd4640 1 -- 172.21.15.86:0/2691674593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e09f6e50 msgr2=0x55e5e0a08680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea29fd4640 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e09f6e50 0x55e5e0a08680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea29fd4640 1 -- 172.21.15.86:0/2691674593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f7770 msgr2=0x55e5e0a05a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:34.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea29fd4640 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f7770 0x55e5e0a05a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea29fd4640 1 -- 172.21.15.86:0/2691674593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e0a06590 con 0x55e5e0a00350 2024-04-16T05:43:34.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea29fd4640 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e0a05fa0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fea1c004a80 tx=0x7fea1c005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f109f4ddc5a343 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:34.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:43:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea0cff9640 1 -- 172.21.15.86:0/2691674593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c032060 con 0x55e5e0a00350 2024-04-16T05:43:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea0cff9640 1 -- 172.21.15.86:0/2691674593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fea1c004d20 con 0x55e5e0a00350 2024-04-16T05:43:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 1 -- 172.21.15.86:0/2691674593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1c005020 con 0x55e5e0a00350 2024-04-16T05:43:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea0cff9640 1 -- 172.21.15.86:0/2691674593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea1c039050 con 0x55e5e0a00350 2024-04-16T05:43:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 10 client.18156 fetch_fsmap learned FSMap version 29 2024-04-16T05:43:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.458+0000 7fea2e61c740 1 -- 172.21.15.86:0/2691674593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5e0a76a30 con 0x55e5e0a00350 2024-04-16T05:43:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea0cff9640 1 -- 172.21.15.86:0/2691674593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c005020 con 0x55e5e0a00350 2024-04-16T05:43:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea0cff9640 1 -- 172.21.15.86:0/2691674593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fea1c02c030 con 0x55e5e0a00350 2024-04-16T05:43:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea2e61c740 10 client.18156 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:43:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea2e61c740 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea2e61c740 20 client.18156 populate_metadata read hostname 'smithi086' 2024-04-16T05:43:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea2e61c740 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0a770f0 0x55e5e0a79510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea2e61c740 4 client.18156 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:43:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea2e61c740 1 -- 172.21.15.86:0/2691674593 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5e0963a10 con 0x55e5e0a770f0 2024-04-16T05:43:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea297d3640 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0a770f0 0x55e5e0a79510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea0cff9640 10 client.18156 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.459+0000 7fea297d3640 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0a770f0 0x55e5e0a79510 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8ea3d1b311ecd4 server_cookie=42c0218c1cb5a189 in_seq=0 out_seq=0 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea0cff9640 1 -- 172.21.15.86:0/2691674593 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e5e0963a10 con 0x55e5e0a770f0 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea0cff9640 10 client.18156 handle_command_reply: tid=0 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 client.18156 shutdown 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 -- 172.21.15.86:0/2691674593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 msgr2=0x55e5e0a05fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e0a05fa0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fea1c004a80 tx=0x7fea1c005400 comp rx=0 tx=0).stop 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 -- 172.21.15.86:0/2691674593 shutdown_connections 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e5e0a770f0 0x55e5e0a79510 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5e0a00350 0x55e5e0a05fa0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea0cff9640 0 client.18156 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:43:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e5e09f7770 0x55e5e0a05a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 --2- 172.21.15.86:0/2691674593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5e09f6e50 0x55e5e0a08680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 -- 172.21.15.86:0/2691674593 >> 172.21.15.86:0/2691674593 conn(0x55e5e09f1580 msgr2=0x55e5e09f0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 -- 172.21.15.86:0/2691674593 shutdown_connections 2024-04-16T05:43:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 1 -- 172.21.15.86:0/2691674593 wait complete. 2024-04-16T05:43:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2e61c740 20 client.18156 trim_cache size 0 max 16384 2024-04-16T05:43:34.469 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:43:34.470 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:43:34.470 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:43:34.470 INFO:teuthology.orchestra.run.smithi086.stdout: "4a62a2d7-274d-4d15-b052-918147710e0b": { 2024-04-16T05:43:34.470 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:43:34.470 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "4a62a2d7-274d-4d15-b052-918147710e0b", 2024-04-16T05:43:34.470 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:43:34.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:43:34.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:43:34.471 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:43:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2ba37640 1 -- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea100a3d20 msgr2=0x7fea100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2ba37640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea100a3d20 0x7fea100a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00afd0 tx=0x7fea1c005f40 comp rx=0 tx=0).stop 2024-04-16T05:43:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2ba37640 1 -- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 msgr2=0x7fea2419e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.460+0000 7fea2ba37640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 0x7fea2419e610 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fea2000c970 tx=0x7fea2000ce30 comp rx=0 tx=0).stop 2024-04-16T05:43:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.461+0000 7fea2ba37640 1 -- 172.21.15.86:0/129606638 shutdown_connections 2024-04-16T05:43:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.461+0000 7fea2ba37640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24106660 0x7fea241986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.461+0000 7fea2ba37640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fea100a3d20 0x7fea100a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.461+0000 7fea2ba37640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea24101010 0x7fea2419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.461+0000 7fea2ba37640 1 --2- 172.21.15.86:0/129606638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24100650 0x7fea2419e610 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.461+0000 7fea2ba37640 1 -- 172.21.15.86:0/129606638 >> 172.21.15.86:0/129606638 conn(0x7fea24077490 msgr2=0x7fea24103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.461+0000 7fea2ba37640 1 -- 172.21.15.86:0/129606638 shutdown_connections 2024-04-16T05:43:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:43:34.461+0000 7fea2ba37640 1 -- 172.21.15.86:0/129606638 wait complete. 2024-04-16T05:43:34.473 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:34 smithi086 ceph-mon[32028]: pgmap v255: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:43:34.474 INFO:tasks.cephfs.filesystem:scrub status for tag:4a62a2d7-274d-4d15-b052-918147710e0b - {'path': '/', 'tag': '4a62a2d7-274d-4d15-b052-918147710e0b', 'options': 'recursive,force'} 2024-04-16T05:43:34.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:34 smithi196 ceph-mon[36373]: pgmap v255: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:43:34.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:34 smithi194 ceph-mon[35865]: pgmap v255: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:43:35.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:35 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:43:35.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:35 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:35.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:35 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:43:35.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:35 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:35.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:35 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:43:35.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:35 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:43:35.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:35 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:35.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:35 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:43:35.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:35 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:35.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:35 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:43:35.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:35 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:43:35.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:35 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:35.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:35 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:43:35.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:35 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:35.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:35 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:43:36.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:36 smithi086 ceph-mon[32028]: pgmap v256: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:36.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:36 smithi196 ceph-mon[36373]: pgmap v256: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:36.557 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:36 smithi194 ceph-mon[35865]: pgmap v256: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:38.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:38 smithi086 ceph-mon[32028]: pgmap v257: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:38.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:38 smithi196 ceph-mon[36373]: pgmap v257: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:38.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:38 smithi194 ceph-mon[35865]: pgmap v257: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:40.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:40 smithi086 ceph-mon[32028]: pgmap v258: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:43:40.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:40 smithi196 ceph-mon[36373]: pgmap v258: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:43:40.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:40 smithi194 ceph-mon[35865]: pgmap v258: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:43:42.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:42 smithi086 ceph-mon[32028]: pgmap v259: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:43:42.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:42 smithi196 ceph-mon[36373]: pgmap v259: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:43:42.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:42 smithi194 ceph-mon[35865]: pgmap v259: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:43:44.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:44 smithi086 ceph-mon[32028]: pgmap v260: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:43:44.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:44 smithi196 ceph-mon[36373]: pgmap v260: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:43:44.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:44 smithi194 ceph-mon[35865]: pgmap v260: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:43:46.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:46 smithi086 ceph-mon[32028]: pgmap v261: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:43:46.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:46 smithi196 ceph-mon[36373]: pgmap v261: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:43:46.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:46 smithi194 ceph-mon[35865]: pgmap v261: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:43:48.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:48 smithi086 ceph-mon[32028]: pgmap v262: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:43:48.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:48 smithi196 ceph-mon[36373]: pgmap v262: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:43:48.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:48 smithi194 ceph-mon[35865]: pgmap v262: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:43:50.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:50 smithi086 ceph-mon[32028]: pgmap v263: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:43:50.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:50 smithi196 ceph-mon[36373]: pgmap v263: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:43:50.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:50 smithi194 ceph-mon[35865]: pgmap v263: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:43:52.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:52 smithi086 ceph-mon[32028]: pgmap v264: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:52.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:52 smithi196 ceph-mon[36373]: pgmap v264: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:52.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:52 smithi194 ceph-mon[35865]: pgmap v264: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:54.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:54 smithi086 ceph-mon[32028]: pgmap v265: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:43:54.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:54 smithi196 ceph-mon[36373]: pgmap v265: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:43:54.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:54 smithi194 ceph-mon[35865]: pgmap v265: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:43:56.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:56 smithi086 ceph-mon[32028]: pgmap v266: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:56.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:56 smithi196 ceph-mon[36373]: pgmap v266: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:56.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:56 smithi194 ceph-mon[35865]: pgmap v266: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:58.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:43:58 smithi086 ceph-mon[32028]: pgmap v267: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:58.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:43:58 smithi196 ceph-mon[36373]: pgmap v267: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:43:58.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:43:58 smithi194 ceph-mon[35865]: pgmap v267: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:00.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:00 smithi086 ceph-mon[32028]: pgmap v268: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:44:00.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:00 smithi196 ceph-mon[36373]: pgmap v268: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:44:00.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:00 smithi194 ceph-mon[35865]: pgmap v268: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:44:02.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:02 smithi196 ceph-mon[36373]: pgmap v269: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:02.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:02 smithi194 ceph-mon[35865]: pgmap v269: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:02.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:02 smithi086 ceph-mon[32028]: pgmap v269: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:04.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:44:04.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:44:04.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:04 smithi196 ceph-mon[36373]: pgmap v270: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:44:04.505 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:04 smithi086 ceph-mon[32028]: pgmap v270: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:44:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.596+0000 7f372ecba640 1 -- 172.21.15.86:0/1722026122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 msgr2=0x7f372810dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.596+0000 7f372ecba640 1 --2- 172.21.15.86:0/1722026122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 0x7f372810dda0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f3720009900 tx=0x7f372002bc80 comp rx=0 tx=0).stop 2024-04-16T05:44:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.596+0000 7f372ecba640 1 -- 172.21.15.86:0/1722026122 shutdown_connections 2024-04-16T05:44:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.596+0000 7f372ecba640 1 --2- 172.21.15.86:0/1722026122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 0x7f372810dda0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.596+0000 7f372ecba640 1 --2- 172.21.15.86:0/1722026122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37281011e0 0x7f3728101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.596+0000 7f372ecba640 1 --2- 172.21.15.86:0/1722026122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3728105f50 0x7f3728106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.596+0000 7f372ecba640 1 -- 172.21.15.86:0/1722026122 >> 172.21.15.86:0/1722026122 conn(0x7f3728077ee0 msgr2=0x7f37280782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.596+0000 7f372ecba640 1 -- 172.21.15.86:0/1722026122 shutdown_connections 2024-04-16T05:44:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.596+0000 7f372ecba640 1 -- 172.21.15.86:0/1722026122 wait complete. 2024-04-16T05:44:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f372ecba640 1 Processor -- start 2024-04-16T05:44:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f372ecba640 1 -- start start 2024-04-16T05:44:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f372ecba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37281011e0 0x7f372819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f372ecba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 0x7f372819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f372ecba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3728105f50 0x7f3728197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f372ecba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3728111450 con 0x7f3728101b80 2024-04-16T05:44:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f372ecba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37281112d0 con 0x7f3728105f50 2024-04-16T05:44:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f372ecba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37281115d0 con 0x7f37281011e0 2024-04-16T05:44:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f3727fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 0x7f372819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f3727fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 0x7f372819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48032/0 (socket says 172.21.15.86:48032) 2024-04-16T05:44:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f3727fff640 1 -- 172.21.15.86:0/495774414 learned_addr learned my addr 172.21.15.86:0/495774414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f372d257640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3728105f50 0x7f3728197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f3727fff640 1 -- 172.21.15.86:0/495774414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37281011e0 msgr2=0x7f372819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f3727fff640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37281011e0 0x7f372819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f3727fff640 1 -- 172.21.15.86:0/495774414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3728105f50 msgr2=0x7f3728197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f3727fff640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3728105f50 0x7f3728197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f3727fff640 1 -- 172.21.15.86:0/495774414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3728197f50 con 0x7f3728101b80 2024-04-16T05:44:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.597+0000 7f3727fff640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 0x7f372819dac0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f371800cce0 tx=0x7f371800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.598+0000 7f3725ffb640 1 -- 172.21.15.86:0/495774414 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371800f650 con 0x7f3728101b80 2024-04-16T05:44:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.598+0000 7f372ecba640 1 -- 172.21.15.86:0/495774414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3728198240 con 0x7f3728101b80 2024-04-16T05:44:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.598+0000 7f3725ffb640 1 -- 172.21.15.86:0/495774414 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371800fcd0 con 0x7f3728101b80 2024-04-16T05:44:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.598+0000 7f3725ffb640 1 -- 172.21.15.86:0/495774414 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718018c70 con 0x7f3728101b80 2024-04-16T05:44:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.598+0000 7f372ecba640 1 -- 172.21.15.86:0/495774414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37281a4500 con 0x7f3728101b80 2024-04-16T05:44:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f3725ffb640 1 -- 172.21.15.86:0/495774414 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f371800f7f0 con 0x7f3728101b80 2024-04-16T05:44:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f373197e740 1 Processor -- start 2024-04-16T05:44:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f373197e740 1 -- start start 2024-04-16T05:44:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f373197e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e7bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f373197e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e87020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f373197e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17e89950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f373197e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f17de1180 con 0x560f17e7bb10 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f373197e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560f17d7e360 con 0x560f17e7c430 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f373197e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f17ddd140 con 0x560f17e87560 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f3725ffb640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37140a3d20 0x7f37140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.601+0000 7f372d257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e7bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e7bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48048/0 (socket says 172.21.15.86:48048) 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 -- 172.21.15.86:0/839017761 learned_addr learned my addr 172.21.15.86:0/839017761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f3727fff640 1 --2- 172.21.15.86:0/839017761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17e89950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37140a3d20 0x7f37140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372ca56640 1 --2- 172.21.15.86:0/839017761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e87020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f3725ffb640 1 -- 172.21.15.86:0/495774414 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f37180d1170 con 0x7f3728101b80 2024-04-16T05:44:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 -- 172.21.15.86:0/839017761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 msgr2=0x560f17e89950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 --2- 172.21.15.86:0/839017761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17e89950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 -- 172.21.15.86:0/839017761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 msgr2=0x560f17e87020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 --2- 172.21.15.86:0/839017761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e87020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 -- 172.21.15.86:0/839017761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f17bbd7d0 con 0x560f17e7bb10 2024-04-16T05:44:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 --2- 172.21.15.86:0/839017761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e7bef0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3720009900 tx=0x7f372002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785982cd17d079f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:04.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f372d257640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37140a3d20 0x7f37140a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3720035000 tx=0x7f372003b000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f371effd640 1 -- 172.21.15.86:0/839017761 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720004310 con 0x560f17e7bb10 2024-04-16T05:44:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f371effd640 1 -- 172.21.15.86:0/839017761 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3720050070 con 0x560f17e7bb10 2024-04-16T05:44:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.602+0000 7f371effd640 1 -- 172.21.15.86:0/839017761 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720003850 con 0x560f17e7bb10 2024-04-16T05:44:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 -- 172.21.15.86:0/839017761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 msgr2=0x560f17e7bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 --2- 172.21.15.86:0/839017761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e7bef0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3720009900 tx=0x7f372002c920 comp rx=0 tx=0).stop 2024-04-16T05:44:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 -- 172.21.15.86:0/839017761 shutdown_connections 2024-04-16T05:44:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 --2- 172.21.15.86:0/839017761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17e89950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 --2- 172.21.15.86:0/839017761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e87020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 --2- 172.21.15.86:0/839017761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e7bef0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 -- 172.21.15.86:0/839017761 >> 172.21.15.86:0/839017761 conn(0x560f17e76580 msgr2=0x560f17e76960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 -- 172.21.15.86:0/839017761 shutdown_connections 2024-04-16T05:44:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 -- 172.21.15.86:0/839017761 wait complete. 2024-04-16T05:44:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.603+0000 7f373197e740 1 -- start start 2024-04-16T05:44:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f373197e740 1 Processor -- start 2024-04-16T05:44:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f373197e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7bb10 0x560f17e845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f373197e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e7c430 0x560f17e84b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f373197e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e87560 0x560f17e85060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f373197e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f17e8a2f0 con 0x560f17e87560 2024-04-16T05:44:04.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f373197e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560f17d9a2c0 con 0x560f17e7bb10 2024-04-16T05:44:04.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f373197e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f17e8a470 con 0x560f17e7c430 2024-04-16T05:44:04.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f373197e740 10 client.0 _ll_register_callbacks cb 0x560f17d81090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:04.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372d257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e87560 0x560f17e85060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372d257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e87560 0x560f17e85060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48050/0 (socket says 172.21.15.86:48050) 2024-04-16T05:44:04.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372d257640 1 -- 172.21.15.86:0/630674195 learned_addr learned my addr 172.21.15.86:0/630674195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:04.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372ca56640 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7bb10 0x560f17e845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f3727fff640 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e7c430 0x560f17e84b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372d257640 1 -- 172.21.15.86:0/630674195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e7c430 msgr2=0x560f17e84b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372d257640 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e7c430 0x560f17e84b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372d257640 1 -- 172.21.15.86:0/630674195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7bb10 msgr2=0x560f17e845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372d257640 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7bb10 0x560f17e845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372d257640 1 -- 172.21.15.86:0/630674195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f17af33f0 con 0x560f17e87560 2024-04-16T05:44:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f372ca56640 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7bb10 0x560f17e845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.604+0000 7f3727fff640 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e7c430 0x560f17e84b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f372d257640 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e87560 0x560f17e85060 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3720004080 tx=0x7f37200038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ba12c95d68f2f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f3707fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f3707fff640 1 -- 172.21.15.86:0/630674195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720059060 con 0x560f17e87560 2024-04-16T05:44:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f3707fff640 1 -- 172.21.15.86:0/630674195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f3720004310 con 0x560f17e87560 2024-04-16T05:44:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f373197e740 1 -- 172.21.15.86:0/630674195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3720050070 con 0x560f17e87560 2024-04-16T05:44:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f3707fff640 1 -- 172.21.15.86:0/630674195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3720060050 con 0x560f17e87560 2024-04-16T05:44:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f373197e740 10 client.18168 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f373197e740 1 -- 172.21.15.86:0/630674195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f17ab30f0 con 0x560f17e87560 2024-04-16T05:44:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f3707fff640 1 -- 172.21.15.86:0/630674195 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3720050070 con 0x560f17e87560 2024-04-16T05:44:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f3707fff640 1 -- 172.21.15.86:0/630674195 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3720034920 con 0x560f17e87560 2024-04-16T05:44:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f373197e740 10 client.18168 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f373197e740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f373197e740 20 client.18168 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f373197e740 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560f17e977a0 0x560f17e99b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f373197e740 4 client.18168 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.605+0000 7f373197e740 1 -- 172.21.15.86:0/630674195 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f17deeba0 con 0x560f17e977a0 2024-04-16T05:44:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.606+0000 7f372ca56640 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560f17e977a0 0x560f17e99b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.606+0000 7f3707fff640 10 client.18168 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.606+0000 7f372ca56640 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560f17e977a0 0x560f17e99b90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46900e341d12073 server_cookie=5a59bb5988d057a5 in_seq=0 out_seq=0 2024-04-16T05:44:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f3707fff640 1 -- 172.21.15.86:0/630674195 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f17deeba0 con 0x560f17e977a0 2024-04-16T05:44:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f3707fff640 10 client.18168 handle_command_reply: tid=0 2024-04-16T05:44:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 client.18168 shutdown 2024-04-16T05:44:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 -- 172.21.15.86:0/630674195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e87560 msgr2=0x560f17e85060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e87560 0x560f17e85060 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3720004080 tx=0x7f37200038a0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 -- 172.21.15.86:0/630674195 shutdown_connections 2024-04-16T05:44:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560f17e977a0 0x560f17e99b90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e87560 0x560f17e85060 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e7c430 0x560f17e84b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f3707fff640 0 client.18168 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 --2- 172.21.15.86:0/630674195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7bb10 0x560f17e845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 -- 172.21.15.86:0/630674195 >> 172.21.15.86:0/630674195 conn(0x560f17e76580 msgr2=0x560f17e782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 -- 172.21.15.86:0/630674195 shutdown_connections 2024-04-16T05:44:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 1 -- 172.21.15.86:0/630674195 wait complete. 2024-04-16T05:44:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.607+0000 7f373197e740 20 client.18168 trim_cache size 0 max 16384 2024-04-16T05:44:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.624+0000 7f373197e740 1 Processor -- start 2024-04-16T05:44:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.624+0000 7f373197e740 1 -- start start 2024-04-16T05:44:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f373197e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e86d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f373197e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e84e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f373197e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17e755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f373197e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ad280 con 0x560f17e7bb10 2024-04-16T05:44:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f373197e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37281aedc0 con 0x560f17e7c430 2024-04-16T05:44:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f373197e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ad950 con 0x560f17e87560 2024-04-16T05:44:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f372ca56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e86d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f372ca56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e86d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48058/0 (socket says 172.21.15.86:48058) 2024-04-16T05:44:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f372ca56640 1 -- 172.21.15.86:0/4249845192 learned_addr learned my addr 172.21.15.86:0/4249845192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f372ca56640 1 -- 172.21.15.86:0/4249845192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 msgr2=0x560f17e755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f372ca56640 1 --2- 172.21.15.86:0/4249845192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17e755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f372ca56640 1 -- 172.21.15.86:0/4249845192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 msgr2=0x560f17e84e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f372ca56640 1 --2- 172.21.15.86:0/4249845192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e84e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f372ca56640 1 -- 172.21.15.86:0/4249845192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f17e7c260 con 0x560f17e7bb10 2024-04-16T05:44:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f372ca56640 1 --2- 172.21.15.86:0/4249845192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e86d10 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f37100097d0 tx=0x7f3710009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca37309431d326b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f371e7fc640 1 -- 172.21.15.86:0/4249845192 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710016e60 con 0x560f17e7bb10 2024-04-16T05:44:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f371e7fc640 1 -- 172.21.15.86:0/4249845192 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371001f030 con 0x560f17e7bb10 2024-04-16T05:44:04.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.625+0000 7f371e7fc640 1 -- 172.21.15.86:0/4249845192 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710014610 con 0x560f17e7bb10 2024-04-16T05:44:04.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 -- 172.21.15.86:0/4249845192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 msgr2=0x560f17e86d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 --2- 172.21.15.86:0/4249845192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e86d10 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f37100097d0 tx=0x7f3710009110 comp rx=0 tx=0).stop 2024-04-16T05:44:04.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 -- 172.21.15.86:0/4249845192 shutdown_connections 2024-04-16T05:44:04.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 --2- 172.21.15.86:0/4249845192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17e755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 --2- 172.21.15.86:0/4249845192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e84e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 --2- 172.21.15.86:0/4249845192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e86d10 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 -- 172.21.15.86:0/4249845192 >> 172.21.15.86:0/4249845192 conn(0x560f17e76580 msgr2=0x560f17e74ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 -- 172.21.15.86:0/4249845192 shutdown_connections 2024-04-16T05:44:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 -- 172.21.15.86:0/4249845192 wait complete. 2024-04-16T05:44:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 -- start start 2024-04-16T05:44:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.626+0000 7f373197e740 1 Processor -- start 2024-04-16T05:44:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f373197e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e9bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f373197e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e9c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f373197e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17efb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f373197e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ae6f0 con 0x560f17e7bb10 2024-04-16T05:44:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f373197e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ae020 con 0x560f17e7c430 2024-04-16T05:44:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f373197e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ab280 con 0x560f17e87560 2024-04-16T05:44:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f373197e740 10 client.0 _ll_register_callbacks cb 0x7f3710006f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372ca56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e9bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372ca56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e9bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48070/0 (socket says 172.21.15.86:48070) 2024-04-16T05:44:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372ca56640 1 -- 172.21.15.86:0/624779427 learned_addr learned my addr 172.21.15.86:0/624779427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372d257640 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17efb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f3727fff640 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e9c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372ca56640 1 -- 172.21.15.86:0/624779427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 msgr2=0x560f17efb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372ca56640 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17efb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372ca56640 1 -- 172.21.15.86:0/624779427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 msgr2=0x560f17e9c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372ca56640 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e9c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372ca56640 1 -- 172.21.15.86:0/624779427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f17efbf70 con 0x560f17e7bb10 2024-04-16T05:44:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f372ca56640 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e9bd70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f3710023cc0 tx=0x7f3710017e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d28618297d8dd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.627+0000 7f3707fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f3707fff640 1 -- 172.21.15.86:0/624779427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710009e90 con 0x560f17e7bb10 2024-04-16T05:44:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f3707fff640 1 -- 172.21.15.86:0/624779427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f3710005a30 con 0x560f17e7bb10 2024-04-16T05:44:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f373197e740 1 -- 172.21.15.86:0/624779427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371001f030 con 0x560f17e7bb10 2024-04-16T05:44:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f3707fff640 1 -- 172.21.15.86:0/624779427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3710037050 con 0x560f17e7bb10 2024-04-16T05:44:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f373197e740 10 client.18174 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f373197e740 1 -- 172.21.15.86:0/624779427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f17efd260 con 0x560f17e7bb10 2024-04-16T05:44:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f3707fff640 1 -- 172.21.15.86:0/624779427 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371001f030 con 0x560f17e7bb10 2024-04-16T05:44:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f3707fff640 1 -- 172.21.15.86:0/624779427 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3710024720 con 0x560f17e7bb10 2024-04-16T05:44:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f373197e740 10 client.18174 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f373197e740 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f373197e740 20 client.18174 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f373197e740 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560f17efd8c0 0x560f17effce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f373197e740 4 client.18174 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:44:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f373197e740 1 -- 172.21.15.86:0/624779427 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f17de8a10 con 0x560f17efd8c0 2024-04-16T05:44:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.628+0000 7f3727fff640 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560f17efd8c0 0x560f17effce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f3707fff640 10 client.18174 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f3727fff640 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560f17efd8c0 0x560f17effce0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbc52eeeae2a511 server_cookie=229ebd5f3ccb0145 in_seq=0 out_seq=0 2024-04-16T05:44:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f3707fff640 1 -- 172.21.15.86:0/624779427 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f17de8a10 con 0x560f17efd8c0 2024-04-16T05:44:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f3707fff640 10 client.18174 handle_command_reply: tid=0 2024-04-16T05:44:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 client.18174 shutdown 2024-04-16T05:44:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 -- 172.21.15.86:0/624779427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 msgr2=0x560f17e9bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e9bd70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f3710023cc0 tx=0x7f3710017e70 comp rx=0 tx=0).stop 2024-04-16T05:44:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 -- 172.21.15.86:0/624779427 shutdown_connections 2024-04-16T05:44:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560f17efd8c0 0x560f17effce0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f17e87560 0x560f17efb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f17e7c430 0x560f17e9c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f3707fff640 0 client.18174 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 --2- 172.21.15.86:0/624779427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f17e7bb10 0x560f17e9bd70 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 -- 172.21.15.86:0/624779427 >> 172.21.15.86:0/624779427 conn(0x560f17e76580 msgr2=0x560f17e75760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 -- 172.21.15.86:0/624779427 shutdown_connections 2024-04-16T05:44:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 1 -- 172.21.15.86:0/624779427 wait complete. 2024-04-16T05:44:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.629+0000 7f373197e740 20 client.18174 trim_cache size 0 max 16384 2024-04-16T05:44:04.638 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:44:04.638 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:44:04.638 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:44:04.639 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:44:04.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 -- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37140a3d20 msgr2=0x7f37140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37140a3d20 0x7f37140a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3720035000 tx=0x7f372003b000 comp rx=0 tx=0).stop 2024-04-16T05:44:04.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 -- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 msgr2=0x7f372819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 0x7f372819dac0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f371800cce0 tx=0x7f371800a620 comp rx=0 tx=0).stop 2024-04-16T05:44:04.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 -- 172.21.15.86:0/495774414 shutdown_connections 2024-04-16T05:44:04.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f37140a3d20 0x7f37140a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3728105f50 0x7f3728197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3728101b80 0x7f372819dac0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 --2- 172.21.15.86:0/495774414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37281011e0 0x7f372819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 -- 172.21.15.86:0/495774414 >> 172.21.15.86:0/495774414 conn(0x7f3728077ee0 msgr2=0x7f37280fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 -- 172.21.15.86:0/495774414 shutdown_connections 2024-04-16T05:44:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.630+0000 7f372ecba640 1 -- 172.21.15.86:0/495774414 wait complete. 2024-04-16T05:44:04.641 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:44:04.642 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:44:04.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:04 smithi194 ceph-mon[35865]: pgmap v270: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:44:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.787+0000 7fc13db75640 1 -- 172.21.15.86:0/882525205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc138100e10 msgr2=0x7fc1381011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.787+0000 7fc13db75640 1 --2- 172.21.15.86:0/882525205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc138100e10 0x7fc1381011f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc128009900 tx=0x7fc12802bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.787+0000 7fc13db75640 1 -- 172.21.15.86:0/882525205 shutdown_connections 2024-04-16T05:44:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.787+0000 7fc13db75640 1 --2- 172.21.15.86:0/882525205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc13810e9c0 0x7fc138110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.787+0000 7fc13db75640 1 --2- 172.21.15.86:0/882525205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc138101730 0x7fc13810e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.787+0000 7fc13db75640 1 --2- 172.21.15.86:0/882525205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc138100e10 0x7fc1381011f0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.787+0000 7fc13db75640 1 -- 172.21.15.86:0/882525205 >> 172.21.15.86:0/882525205 conn(0x7fc138078000 msgr2=0x7fc138078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.787+0000 7fc13db75640 1 -- 172.21.15.86:0/882525205 shutdown_connections 2024-04-16T05:44:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.787+0000 7fc13db75640 1 -- 172.21.15.86:0/882525205 wait complete. 2024-04-16T05:44:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc13db75640 1 Processor -- start 2024-04-16T05:44:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc13db75640 1 -- start start 2024-04-16T05:44:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc13db75640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc138100e10 0x7fc1381a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc13db75640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc138101730 0x7fc1381a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc13db75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc13810e9c0 0x7fc13819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc13db75640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1381145e0 con 0x7fc13810e9c0 2024-04-16T05:44:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc13db75640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc138114460 con 0x7fc138101730 2024-04-16T05:44:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc13db75640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc138114760 con 0x7fc138100e10 2024-04-16T05:44:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc137fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc13810e9c0 0x7fc13819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc137fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc13810e9c0 0x7fc13819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48086/0 (socket says 172.21.15.86:48086) 2024-04-16T05:44:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc137fff640 1 -- 172.21.15.86:0/2343411775 learned_addr learned my addr 172.21.15.86:0/2343411775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc12ffff640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc138101730 0x7fc1381a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc1377fe640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc138100e10 0x7fc1381a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc137fff640 1 -- 172.21.15.86:0/2343411775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc138100e10 msgr2=0x7fc1381a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc137fff640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc138100e10 0x7fc1381a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc137fff640 1 -- 172.21.15.86:0/2343411775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc138101730 msgr2=0x7fc1381a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:04.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc137fff640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc138101730 0x7fc1381a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:04.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc137fff640 1 -- 172.21.15.86:0/2343411775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13819b080 con 0x7fc13810e9c0 2024-04-16T05:44:04.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc12ffff640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc138101730 0x7fc1381a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:04.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.788+0000 7fc137fff640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc13810e9c0 0x7fc13819a900 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc13000eb40 tx=0x7fc13000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:04.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.789+0000 7fc1357fa640 1 -- 172.21.15.86:0/2343411775 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130004b20 con 0x7fc13810e9c0 2024-04-16T05:44:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.789+0000 7fc1357fa640 1 -- 172.21.15.86:0/2343411775 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc130004cc0 con 0x7fc13810e9c0 2024-04-16T05:44:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.789+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13819b370 con 0x7fc13810e9c0 2024-04-16T05:44:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.789+0000 7fc1357fa640 1 -- 172.21.15.86:0/2343411775 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130010850 con 0x7fc13810e9c0 2024-04-16T05:44:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.789+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1381a7610 con 0x7fc13810e9c0 2024-04-16T05:44:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.790+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc138068e20 con 0x7fc13810e9c0 2024-04-16T05:44:04.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.791+0000 7fc1357fa640 1 -- 172.21.15.86:0/2343411775 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc130005020 con 0x7fc13810e9c0 2024-04-16T05:44:04.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.791+0000 7fc1357fa640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1240a3d20 0x7fc1240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:04.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.791+0000 7fc1377fe640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1240a3d20 0x7fc1240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:04.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.791+0000 7fc1357fa640 1 -- 172.21.15.86:0/2343411775 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fc1300d14b0 con 0x7fc13810e9c0 2024-04-16T05:44:04.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.792+0000 7fc1377fe640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1240a3d20 0x7fc1240a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc128006010 tx=0x7fc12802c630 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:04.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:04.796+0000 7fc1357fa640 1 -- 172.21.15.86:0/2343411775 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc13009d780 con 0x7fc13810e9c0 2024-04-16T05:44:05.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.014+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc13802d620 con 0x7fc13810e9c0 2024-04-16T05:44:05.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.015+0000 7fc1357fa640 1 -- 172.21.15.86:0/2343411775 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fc13009d080 con 0x7fc13810e9c0 2024-04-16T05:44:05.014 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:44:05.014 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:44:05.014 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:44:05.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1240a3d20 msgr2=0x7fc1240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1240a3d20 0x7fc1240a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc128006010 tx=0x7fc12802c630 comp rx=0 tx=0).stop 2024-04-16T05:44:05.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc13810e9c0 msgr2=0x7fc13819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc13810e9c0 0x7fc13819a900 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc13000eb40 tx=0x7fc13000c2d0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 shutdown_connections 2024-04-16T05:44:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc13810e9c0 0x7fc13819a900 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc1240a3d20 0x7fc1240a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc138101730 0x7fc1381a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 --2- 172.21.15.86:0/2343411775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc138100e10 0x7fc1381a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 >> 172.21.15.86:0/2343411775 conn(0x7fc138078000 msgr2=0x7fc13810fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.017+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 shutdown_connections 2024-04-16T05:44:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.018+0000 7fc13db75640 1 -- 172.21.15.86:0/2343411775 wait complete. 2024-04-16T05:44:05.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:44:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 -- 172.21.15.86:0/630697761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 msgr2=0x7f63dc10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 --2- 172.21.15.86:0/630697761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 0x7f63dc10ddd0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009ca0 tx=0x7f63d402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 -- 172.21.15.86:0/630697761 shutdown_connections 2024-04-16T05:44:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 --2- 172.21.15.86:0/630697761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 0x7f63dc10ddd0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 --2- 172.21.15.86:0/630697761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63dc101ae0 0x7f63dc105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 --2- 172.21.15.86:0/630697761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63dc101130 0x7f63dc101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 -- 172.21.15.86:0/630697761 >> 172.21.15.86:0/630697761 conn(0x7f63dc078080 msgr2=0x7f63dc078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 -- 172.21.15.86:0/630697761 shutdown_connections 2024-04-16T05:44:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 -- 172.21.15.86:0/630697761 wait complete. 2024-04-16T05:44:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.136+0000 7f63e4037640 1 Processor -- start 2024-04-16T05:44:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e4037640 1 -- start start 2024-04-16T05:44:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e4037640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63dc101130 0x7f63dc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e4037640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 0x7f63dc199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e4037640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63dc19a170 0x7f63dc1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e4037640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1112c0 con 0x7f63dc106200 2024-04-16T05:44:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e4037640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc111140 con 0x7f63dc19a170 2024-04-16T05:44:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e4037640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc111440 con 0x7f63dc101130 2024-04-16T05:44:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e15d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 0x7f63dc199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e15d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 0x7f63dc199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48110/0 (socket says 172.21.15.86:48110) 2024-04-16T05:44:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e15d2640 1 -- 172.21.15.86:0/231161944 learned_addr learned my addr 172.21.15.86:0/231161944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e25d4640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63dc19a170 0x7f63dc1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e1dd3640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63dc101130 0x7f63dc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e15d2640 1 -- 172.21.15.86:0/231161944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63dc101130 msgr2=0x7f63dc1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e15d2640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63dc101130 0x7f63dc1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e15d2640 1 -- 172.21.15.86:0/231161944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63dc19a170 msgr2=0x7f63dc1a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e15d2640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63dc19a170 0x7f63dc1a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e15d2640 1 -- 172.21.15.86:0/231161944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc1a4f20 con 0x7f63dc106200 2024-04-16T05:44:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e25d4640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63dc19a170 0x7f63dc1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e1dd3640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63dc101130 0x7f63dc1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e15d2640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 0x7f63dc199be0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f63d8005fd0 tx=0x7f63d800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63ceffd640 1 -- 172.21.15.86:0/231161944 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8017070 con 0x7f63dc106200 2024-04-16T05:44:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63ceffd640 1 -- 172.21.15.86:0/231161944 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d8009c30 con 0x7f63dc106200 2024-04-16T05:44:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e4037640 1 -- 172.21.15.86:0/231161944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc1a5210 con 0x7f63dc106200 2024-04-16T05:44:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63ceffd640 1 -- 172.21.15.86:0/231161944 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8010460 con 0x7f63dc106200 2024-04-16T05:44:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.137+0000 7f63e4037640 1 -- 172.21.15.86:0/231161944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc1a5760 con 0x7f63dc106200 2024-04-16T05:44:05.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.140+0000 7f63ceffd640 1 -- 172.21.15.86:0/231161944 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f63d8003e70 con 0x7f63dc106200 2024-04-16T05:44:05.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.140+0000 7f63e6c1a740 1 Processor -- start 2024-04-16T05:44:05.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.140+0000 7f63e6c1a740 1 -- start start 2024-04-16T05:44:05.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9bb10 0x55b65af9bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9c430 0x55b65afa9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65afabb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b65af010e0 con 0x55b65af9bb10 2024-04-16T05:44:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b65ae9e520 con 0x55b65af9c430 2024-04-16T05:44:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b65aefd0a0 con 0x55b65afa9770 2024-04-16T05:44:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63ceffd640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f63c80a3d20 0x7f63c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9bb10 0x55b65af9bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9bb10 0x55b65af9bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48122/0 (socket says 172.21.15.86:48122) 2024-04-16T05:44:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 -- 172.21.15.86:0/2868779129 learned_addr learned my addr 172.21.15.86:0/2868779129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f63c80a3d20 0x7f63c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63ceffd640 1 -- 172.21.15.86:0/231161944 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f63d80d0870 con 0x7f63dc106200 2024-04-16T05:44:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e15d2640 1 --2- 172.21.15.86:0/2868779129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65afabb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e1dd3640 1 --2- 172.21.15.86:0/2868779129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9c430 0x55b65afa9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 -- 172.21.15.86:0/2868779129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 msgr2=0x55b65afabb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 --2- 172.21.15.86:0/2868779129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65afabb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 -- 172.21.15.86:0/2868779129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9c430 msgr2=0x55b65afa9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 --2- 172.21.15.86:0/2868779129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9c430 0x55b65afa9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 -- 172.21.15.86:0/2868779129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65aec68d0 con 0x55b65af9bb10 2024-04-16T05:44:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 --2- 172.21.15.86:0/2868779129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9bb10 0x55b65af9bef0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f63d4032530 tx=0x7f63d4032690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1428b678abf764 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.141+0000 7f63e25d4640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f63c80a3d20 0x7f63c80a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009ca0 tx=0x7f63d4041000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63c7fff640 1 -- 172.21.15.86:0/2868779129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4051070 con 0x55b65af9bb10 2024-04-16T05:44:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63c7fff640 1 -- 172.21.15.86:0/2868779129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d4048070 con 0x55b65af9bb10 2024-04-16T05:44:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63c7fff640 1 -- 172.21.15.86:0/2868779129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4039040 con 0x55b65af9bb10 2024-04-16T05:44:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63e6c1a740 1 -- 172.21.15.86:0/2868779129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9bb10 msgr2=0x55b65af9bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/2868779129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9bb10 0x55b65af9bef0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f63d4032530 tx=0x7f63d4032690 comp rx=0 tx=0).stop 2024-04-16T05:44:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63e6c1a740 1 -- 172.21.15.86:0/2868779129 shutdown_connections 2024-04-16T05:44:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/2868779129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65afabb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/2868779129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9c430 0x55b65afa9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/2868779129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9bb10 0x55b65af9bef0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63e6c1a740 1 -- 172.21.15.86:0/2868779129 >> 172.21.15.86:0/2868779129 conn(0x55b65af96590 msgr2=0x55b65af96970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63e6c1a740 1 -- 172.21.15.86:0/2868779129 shutdown_connections 2024-04-16T05:44:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.142+0000 7f63e6c1a740 1 -- 172.21.15.86:0/2868779129 wait complete. 2024-04-16T05:44:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e6c1a740 1 -- start start 2024-04-16T05:44:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e6c1a740 1 Processor -- start 2024-04-16T05:44:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9bb10 0x55b65afa67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65afa7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b65afac550 con 0x55b65af9c430 2024-04-16T05:44:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b65aeba4d0 con 0x55b65af9bb10 2024-04-16T05:44:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b65afac6d0 con 0x55b65afa9770 2024-04-16T05:44:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e6c1a740 10 client.0 _ll_register_callbacks cb 0x55b65ae9f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e15d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e15d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48138/0 (socket says 172.21.15.86:48138) 2024-04-16T05:44:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e15d2640 1 -- 172.21.15.86:0/3785526733 learned_addr learned my addr 172.21.15.86:0/3785526733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.143+0000 7f63e25d4640 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65afa7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e15d2640 1 -- 172.21.15.86:0/3785526733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 msgr2=0x55b65afa7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e15d2640 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65afa7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e15d2640 1 -- 172.21.15.86:0/3785526733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9bb10 msgr2=0x55b65afa67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e15d2640 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9bb10 0x55b65afa67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e15d2640 1 -- 172.21.15.86:0/3785526733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65ac133f0 con 0x55b65af9c430 2024-04-16T05:44:05.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e15d2640 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa6d00 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f63d80cdca0 tx=0x7f63d809cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbde5e0e386bbba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63c57fa640 1 -- 172.21.15.86:0/3785526733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ce070 con 0x55b65af9c430 2024-04-16T05:44:05.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63c57fa640 1 -- 172.21.15.86:0/3785526733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f63d80a4c20 con 0x55b65af9c430 2024-04-16T05:44:05.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e6c1a740 1 -- 172.21.15.86:0/3785526733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d4048070 con 0x55b65af9c430 2024-04-16T05:44:05.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63c57fa640 1 -- 172.21.15.86:0/3785526733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63d80a07f0 con 0x55b65af9c430 2024-04-16T05:44:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e6c1a740 10 client.18192 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e6c1a740 1 -- 172.21.15.86:0/3785526733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b65acdd7d0 con 0x55b65af9c430 2024-04-16T05:44:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63c57fa640 1 -- 172.21.15.86:0/3785526733 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d4048070 con 0x55b65af9c430 2024-04-16T05:44:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63c57fa640 1 -- 172.21.15.86:0/3785526733 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f63d80d0420 con 0x55b65af9c430 2024-04-16T05:44:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e6c1a740 10 client.18192 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e6c1a740 10 client.18192 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:44:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e6c1a740 20 client.18192 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55b65afb98b0 0x55b65afbbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e6c1a740 4 client.18192 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.144+0000 7f63e6c1a740 1 -- 172.21.15.86:0/3785526733 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b65af0eb00 con 0x55b65afb98b0 2024-04-16T05:44:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.145+0000 7f63e1dd3640 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55b65afb98b0 0x55b65afbbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.145+0000 7f63e1dd3640 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55b65afb98b0 0x55b65afbbd00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f4478ad438bcaba2 server_cookie=1045e9aa7ac81d21 in_seq=0 out_seq=0 2024-04-16T05:44:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.145+0000 7f63c57fa640 10 client.18192 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:44:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63c57fa640 1 -- 172.21.15.86:0/3785526733 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b65af0eb00 con 0x55b65afb98b0 2024-04-16T05:44:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63c57fa640 10 client.18192 handle_command_reply: tid=0 2024-04-16T05:44:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 client.18192 shutdown 2024-04-16T05:44:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 -- 172.21.15.86:0/3785526733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 msgr2=0x55b65afa6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa6d00 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f63d80cdca0 tx=0x7f63d809cc00 comp rx=0 tx=0).stop 2024-04-16T05:44:05.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 -- 172.21.15.86:0/3785526733 shutdown_connections 2024-04-16T05:44:05.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55b65afb98b0 0x55b65afbbd00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65afa7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63c57fa640 0 client.18192 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:44:05.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa6d00 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/3785526733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9bb10 0x55b65afa67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 -- 172.21.15.86:0/3785526733 >> 172.21.15.86:0/3785526733 conn(0x55b65af96590 msgr2=0x55b65af982e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 -- 172.21.15.86:0/3785526733 shutdown_connections 2024-04-16T05:44:05.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 1 -- 172.21.15.86:0/3785526733 wait complete. 2024-04-16T05:44:05.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.146+0000 7f63e6c1a740 20 client.18192 trim_cache size 0 max 16384 2024-04-16T05:44:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e6c1a740 1 Processor -- start 2024-04-16T05:44:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e6c1a740 1 -- start start 2024-04-16T05:44:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9bb10 0x55b65afa8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afb5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65af955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1ad2f0 con 0x55b65af9c430 2024-04-16T05:44:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1aee30 con 0x55b65af9bb10 2024-04-16T05:44:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1ad9c0 con 0x55b65afa9770 2024-04-16T05:44:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e15d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afb5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e15d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afb5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48140/0 (socket says 172.21.15.86:48140) 2024-04-16T05:44:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e15d2640 1 -- 172.21.15.86:0/928350082 learned_addr learned my addr 172.21.15.86:0/928350082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e15d2640 1 -- 172.21.15.86:0/928350082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 msgr2=0x55b65af955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e15d2640 1 --2- 172.21.15.86:0/928350082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65af955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e15d2640 1 -- 172.21.15.86:0/928350082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9bb10 msgr2=0x55b65afa8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e15d2640 1 --2- 172.21.15.86:0/928350082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9bb10 0x55b65afa8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.159+0000 7f63e15d2640 1 -- 172.21.15.86:0/928350082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65afb8400 con 0x55b65af9c430 2024-04-16T05:44:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e15d2640 1 --2- 172.21.15.86:0/928350082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afb5450 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009610 tx=0x7f63d800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5534c720a6da0d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63c77fe640 1 -- 172.21.15.86:0/928350082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ba070 con 0x55b65af9c430 2024-04-16T05:44:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63c77fe640 1 -- 172.21.15.86:0/928350082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d80b5e40 con 0x55b65af9c430 2024-04-16T05:44:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63c77fe640 1 -- 172.21.15.86:0/928350082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80a4c20 con 0x55b65af9c430 2024-04-16T05:44:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e6c1a740 1 -- 172.21.15.86:0/928350082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 msgr2=0x55b65afb5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/928350082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afb5450 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009610 tx=0x7f63d800bf60 comp rx=0 tx=0).stop 2024-04-16T05:44:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e6c1a740 1 -- 172.21.15.86:0/928350082 shutdown_connections 2024-04-16T05:44:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/928350082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65afa9770 0x55b65af955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/928350082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afb5450 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/928350082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65af9bb10 0x55b65afa8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e6c1a740 1 -- 172.21.15.86:0/928350082 >> 172.21.15.86:0/928350082 conn(0x55b65af96590 msgr2=0x55b65af94ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e6c1a740 1 -- 172.21.15.86:0/928350082 shutdown_connections 2024-04-16T05:44:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.160+0000 7f63e6c1a740 1 -- 172.21.15.86:0/928350082 wait complete. 2024-04-16T05:44:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e6c1a740 1 -- start start 2024-04-16T05:44:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e6c1a740 1 Processor -- start 2024-04-16T05:44:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65af9bb10 0x55b65afa3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e6c1a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65afa9770 0x55b65afa41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1ae760 con 0x55b65af9c430 2024-04-16T05:44:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1ae090 con 0x55b65afa9770 2024-04-16T05:44:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e6c1a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1ab2f0 con 0x55b65af9bb10 2024-04-16T05:44:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e6c1a740 10 client.0 _ll_register_callbacks cb 0x7f63d809bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e15d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e15d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48142/0 (socket says 172.21.15.86:48142) 2024-04-16T05:44:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e15d2640 1 -- 172.21.15.86:0/4287507652 learned_addr learned my addr 172.21.15.86:0/4287507652 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e15d2640 1 -- 172.21.15.86:0/4287507652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65af9bb10 msgr2=0x55b65afa3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e15d2640 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65af9bb10 0x55b65afa3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e15d2640 1 -- 172.21.15.86:0/4287507652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65afa9770 msgr2=0x55b65afa41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.161+0000 7f63e15d2640 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65afa9770 0x55b65afa41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e15d2640 1 -- 172.21.15.86:0/4287507652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65b01f110 con 0x55b65af9c430 2024-04-16T05:44:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e15d2640 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa3c90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f63d8098e10 tx=0x7f63d80a4e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d9a2890316d9c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63c57fa640 1 -- 172.21.15.86:0/4287507652 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ba030 con 0x55b65af9c430 2024-04-16T05:44:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63c57fa640 1 -- 172.21.15.86:0/4287507652 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f63d80bb120 con 0x55b65af9c430 2024-04-16T05:44:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e6c1a740 1 -- 172.21.15.86:0/4287507652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b65af08970 con 0x55b65af9c430 2024-04-16T05:44:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63c57fa640 1 -- 172.21.15.86:0/4287507652 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63d809bf80 con 0x55b65af9c430 2024-04-16T05:44:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e6c1a740 10 client.18198 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e6c1a740 1 -- 172.21.15.86:0/4287507652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b65b020410 con 0x55b65af9c430 2024-04-16T05:44:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63c57fa640 1 -- 172.21.15.86:0/4287507652 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b65af08970 con 0x55b65af9c430 2024-04-16T05:44:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63c57fa640 1 -- 172.21.15.86:0/4287507652 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f63d809ba20 con 0x55b65af9c430 2024-04-16T05:44:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e6c1a740 10 client.18198 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e6c1a740 10 client.18198 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:44:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e6c1a740 20 client.18198 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55b65b0207a0 0x55b65b022bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e6c1a740 4 client.18198 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.162+0000 7f63e6c1a740 1 -- 172.21.15.86:0/4287507652 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b65ae846b0 con 0x55b65b0207a0 2024-04-16T05:44:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e1dd3640 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55b65b0207a0 0x55b65b022bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e1dd3640 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55b65b0207a0 0x55b65b022bc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee753c0aeba0c2ea server_cookie=f9cabf1974691c3b in_seq=0 out_seq=0 2024-04-16T05:44:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63c57fa640 10 client.18198 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:44:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63c57fa640 1 -- 172.21.15.86:0/4287507652 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b65ae846b0 con 0x55b65b0207a0 2024-04-16T05:44:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63c57fa640 10 client.18198 handle_command_reply: tid=0 2024-04-16T05:44:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 client.18198 shutdown 2024-04-16T05:44:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 -- 172.21.15.86:0/4287507652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 msgr2=0x55b65afa3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa3c90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f63d8098e10 tx=0x7f63d80a4e30 comp rx=0 tx=0).stop 2024-04-16T05:44:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 -- 172.21.15.86:0/4287507652 shutdown_connections 2024-04-16T05:44:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55b65b0207a0 0x55b65b022bc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b65afa9770 0x55b65afa41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65af9c430 0x55b65afa3c90 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63c57fa640 0 client.18198 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:44:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 --2- 172.21.15.86:0/4287507652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65af9bb10 0x55b65afa3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 -- 172.21.15.86:0/4287507652 >> 172.21.15.86:0/4287507652 conn(0x55b65af96590 msgr2=0x55b65af95730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.163+0000 7f63e6c1a740 1 -- 172.21.15.86:0/4287507652 shutdown_connections 2024-04-16T05:44:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e6c1a740 1 -- 172.21.15.86:0/4287507652 wait complete. 2024-04-16T05:44:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e6c1a740 20 client.18198 trim_cache size 0 max 16384 2024-04-16T05:44:05.172 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:05.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 -- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f63c80a3d20 msgr2=0x7f63c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f63c80a3d20 0x7f63c80a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009ca0 tx=0x7f63d4041000 comp rx=0 tx=0).stop 2024-04-16T05:44:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 -- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 msgr2=0x7f63dc199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 0x7f63dc199be0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f63d8005fd0 tx=0x7f63d800dde0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 -- 172.21.15.86:0/231161944 shutdown_connections 2024-04-16T05:44:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f63c80a3d20 0x7f63c80a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63dc19a170 0x7f63dc1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63dc106200 0x7f63dc199be0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 --2- 172.21.15.86:0/231161944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63dc101130 0x7f63dc1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 -- 172.21.15.86:0/231161944 >> 172.21.15.86:0/231161944 conn(0x7f63dc078080 msgr2=0x7f63dc0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 -- 172.21.15.86:0/231161944 shutdown_connections 2024-04-16T05:44:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.164+0000 7f63e4037640 1 -- 172.21.15.86:0/231161944 wait complete. 2024-04-16T05:44:05.175 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:44:05.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.284+0000 7f9800ac5640 1 -- 172.21.15.86:0/4032938771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc102250 msgr2=0x7f97fc102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.284+0000 7f9800ac5640 1 --2- 172.21.15.86:0/4032938771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc102250 0x7f97fc102630 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009900 tx=0x7f97ec02bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:05.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.284+0000 7f9800ac5640 1 -- 172.21.15.86:0/4032938771 shutdown_connections 2024-04-16T05:44:05.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.284+0000 7f9800ac5640 1 --2- 172.21.15.86:0/4032938771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97fc107320 0x7f97fc10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.284+0000 7f9800ac5640 1 --2- 172.21.15.86:0/4032938771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc102c00 0x7f97fc106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.284+0000 7f9800ac5640 1 --2- 172.21.15.86:0/4032938771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc102250 0x7f97fc102630 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.284+0000 7f9800ac5640 1 -- 172.21.15.86:0/4032938771 >> 172.21.15.86:0/4032938771 conn(0x7f97fc079490 msgr2=0x7f97fc0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.284+0000 7f9800ac5640 1 -- 172.21.15.86:0/4032938771 shutdown_connections 2024-04-16T05:44:05.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.284+0000 7f9800ac5640 1 -- 172.21.15.86:0/4032938771 wait complete. 2024-04-16T05:44:05.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f9800ac5640 1 Processor -- start 2024-04-16T05:44:05.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f9800ac5640 1 -- start start 2024-04-16T05:44:05.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f9800ac5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc102250 0x7f97fc19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f9800ac5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc107320 0x7f97fc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f9800ac5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97fc198750 0x7f97fc198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f9800ac5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1123d0 con 0x7f97fc107320 2024-04-16T05:44:05.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f9800ac5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc112250 con 0x7f97fc198750 2024-04-16T05:44:05.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f9800ac5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc112550 con 0x7f97fc102250 2024-04-16T05:44:05.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc107320 0x7f97fc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc107320 0x7f97fc19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48154/0 (socket says 172.21.15.86:48154) 2024-04-16T05:44:05.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f97f3fff640 1 -- 172.21.15.86:0/2405646602 learned_addr learned my addr 172.21.15.86:0/2405646602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f97fa59c640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc102250 0x7f97fc19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f97f3fff640 1 -- 172.21.15.86:0/2405646602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc102250 msgr2=0x7f97fc19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f97f3fff640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc102250 0x7f97fc19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f97f3fff640 1 -- 172.21.15.86:0/2405646602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97fc198750 msgr2=0x7f97fc198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f97f3fff640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97fc198750 0x7f97fc198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.285+0000 7f97f3fff640 1 -- 172.21.15.86:0/2405646602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc199350 con 0x7f97fc107320 2024-04-16T05:44:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.286+0000 7f97f3fff640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc107320 0x7f97fc19eb80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c970 tx=0x7f97e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.286+0000 7f97f37fe640 1 -- 172.21.15.86:0/2405646602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4007bb0 con 0x7f97fc107320 2024-04-16T05:44:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.286+0000 7f9800ac5640 1 -- 172.21.15.86:0/2405646602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc1a53e0 con 0x7f97fc107320 2024-04-16T05:44:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.286+0000 7f97f37fe640 1 -- 172.21.15.86:0/2405646602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e4007d50 con 0x7f97fc107320 2024-04-16T05:44:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.286+0000 7f97f37fe640 1 -- 172.21.15.86:0/2405646602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40187a0 con 0x7f97fc107320 2024-04-16T05:44:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.286+0000 7f9800ac5640 1 -- 172.21.15.86:0/2405646602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc1a57e0 con 0x7f97fc107320 2024-04-16T05:44:05.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.288+0000 7f97f37fe640 1 -- 172.21.15.86:0/2405646602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f97e400f450 con 0x7f97fc107320 2024-04-16T05:44:05.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.288+0000 7f9803785740 1 Processor -- start 2024-04-16T05:44:05.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f9803785740 1 -- start start 2024-04-16T05:44:05.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f9803785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6ce60 0x560a2df6d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f9803785740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 0x560a2df75e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f9803785740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df78750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f9803785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a2ded20e0 con 0x560a2df6ce60 2024-04-16T05:44:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f9803785740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560a2de6f520 con 0x560a2df76360 2024-04-16T05:44:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f9803785740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560a2dece0a0 con 0x560a2df6d780 2024-04-16T05:44:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6ce60 0x560a2df6d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6ce60 0x560a2df6d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48164/0 (socket says 172.21.15.86:48164) 2024-04-16T05:44:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 -- 172.21.15.86:0/1596197067 learned_addr learned my addr 172.21.15.86:0/1596197067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97f37fe640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97e80a3d20 0x7f97e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97e80a3d20 0x7f97e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97f3fff640 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df78750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fa59c640 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 0x560a2df75e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 -- 172.21.15.86:0/1596197067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 msgr2=0x560a2df75e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 0x560a2df75e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 -- 172.21.15.86:0/1596197067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 msgr2=0x560a2df78750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df78750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97f37fe640 1 -- 172.21.15.86:0/2405646602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f97e40d0cc0 con 0x7f97fc107320 2024-04-16T05:44:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 -- 172.21.15.86:0/1596197067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a2de978d0 con 0x560a2df6ce60 2024-04-16T05:44:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fa59c640 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 0x560a2df75e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97f3fff640 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df78750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.289+0000 7f97fad9d640 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6ce60 0x560a2df6d240 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f97f4015890 tx=0x7f97f4015d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd9bfce221b934c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f97f0ff9640 1 -- 172.21.15.86:0/1596197067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4022070 con 0x560a2df6ce60 2024-04-16T05:44:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f97f0ff9640 1 -- 172.21.15.86:0/1596197067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f400f8c0 con 0x560a2df6ce60 2024-04-16T05:44:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f97f0ff9640 1 -- 172.21.15.86:0/1596197067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4010840 con 0x560a2df6ce60 2024-04-16T05:44:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f9803785740 1 -- 172.21.15.86:0/1596197067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6ce60 msgr2=0x560a2df6d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f97fad9d640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97e80a3d20 0x7f97e80a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f97f401f000 tx=0x7f97f401b000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f9803785740 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6ce60 0x560a2df6d240 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f97f4015890 tx=0x7f97f4015d50 comp rx=0 tx=0).stop 2024-04-16T05:44:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f9803785740 1 -- 172.21.15.86:0/1596197067 shutdown_connections 2024-04-16T05:44:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f9803785740 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df78750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f9803785740 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 0x560a2df75e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f9803785740 1 --2- 172.21.15.86:0/1596197067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6ce60 0x560a2df6d240 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f9803785740 1 -- 172.21.15.86:0/1596197067 >> 172.21.15.86:0/1596197067 conn(0x560a2df67590 msgr2=0x560a2df67970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.290+0000 7f9803785740 1 -- 172.21.15.86:0/1596197067 shutdown_connections 2024-04-16T05:44:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 1 -- 172.21.15.86:0/1596197067 wait complete. 2024-04-16T05:44:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 1 -- start start 2024-04-16T05:44:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 1 Processor -- start 2024-04-16T05:44:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6ce60 0x560a2df732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df73d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a2df79060 con 0x560a2df6d780 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560a2de8b4d0 con 0x560a2df76360 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560a2df791e0 con 0x560a2df6ce60 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f9803785740 10 client.0 _ll_register_callbacks cb 0x560a2de702c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df73800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48180/0 (socket says 172.21.15.86:48180) 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97f3fff640 1 -- 172.21.15.86:0/1783455270 learned_addr learned my addr 172.21.15.86:0/1783455270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97fad9d640 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df73d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97fa59c640 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6ce60 0x560a2df732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97f3fff640 1 -- 172.21.15.86:0/1783455270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6ce60 msgr2=0x560a2df732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97f3fff640 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6ce60 0x560a2df732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97f3fff640 1 -- 172.21.15.86:0/1783455270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 msgr2=0x560a2df73d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97f3fff640 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df73d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97f3fff640 1 -- 172.21.15.86:0/1783455270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2dbe43f0 con 0x560a2df6d780 2024-04-16T05:44:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97fad9d640 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df73d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.291+0000 7f97fa59c640 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6ce60 0x560a2df732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f97f3fff640 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df73800 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f97e40ce640 tx=0x7f97e4008a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9066b5874c972f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f97e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1783455270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40d2070 con 0x560a2df6d780 2024-04-16T05:44:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1783455270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f97e4098fb0 con 0x560a2df6d780 2024-04-16T05:44:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f9803785740 1 -- 172.21.15.86:0/1783455270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f400f8c0 con 0x560a2df6d780 2024-04-16T05:44:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1783455270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97e40131b0 con 0x560a2df6d780 2024-04-16T05:44:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f9803785740 10 client.18210 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f9803785740 1 -- 172.21.15.86:0/1783455270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a2dcae7d0 con 0x560a2df6d780 2024-04-16T05:44:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1783455270 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f400f8c0 con 0x560a2df6d780 2024-04-16T05:44:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1783455270 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97e4020030 con 0x560a2df6d780 2024-04-16T05:44:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f9803785740 10 client.18210 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f9803785740 10 client.18210 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:44:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f9803785740 20 client.18210 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f9803785740 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560a2df861f0 0x560a2df885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f9803785740 4 client.18210 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.292+0000 7f9803785740 1 -- 172.21.15.86:0/1783455270 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a2dedfb00 con 0x560a2df861f0 2024-04-16T05:44:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.293+0000 7f97fa59c640 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560a2df861f0 0x560a2df885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.293+0000 7f97e1ffb640 10 client.18210 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:44:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.293+0000 7f97fa59c640 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560a2df861f0 0x560a2df885e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=972fc7b527040197 server_cookie=ed12bf691f61d4aa in_seq=0 out_seq=0 2024-04-16T05:44:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1783455270 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a2dedfb00 con 0x560a2df861f0 2024-04-16T05:44:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f97e1ffb640 10 client.18210 handle_command_reply: tid=0 2024-04-16T05:44:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 client.18210 shutdown 2024-04-16T05:44:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 -- 172.21.15.86:0/1783455270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 msgr2=0x560a2df73800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df73800 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f97e40ce640 tx=0x7f97e4008a00 comp rx=0 tx=0).stop 2024-04-16T05:44:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 -- 172.21.15.86:0/1783455270 shutdown_connections 2024-04-16T05:44:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df76360 0x560a2df73d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df73800 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560a2df861f0 0x560a2df885e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 --2- 172.21.15.86:0/1783455270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6ce60 0x560a2df732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f97e1ffb640 0 client.18210 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:44:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 -- 172.21.15.86:0/1783455270 >> 172.21.15.86:0/1783455270 conn(0x560a2df67590 msgr2=0x560a2df692e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 -- 172.21.15.86:0/1783455270 shutdown_connections 2024-04-16T05:44:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 1 -- 172.21.15.86:0/1783455270 wait complete. 2024-04-16T05:44:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.294+0000 7f9803785740 20 client.18210 trim_cache size 0 max 16384 2024-04-16T05:44:05.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.307+0000 7f9803785740 1 Processor -- start 2024-04-16T05:44:05.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.307+0000 7f9803785740 1 -- start start 2024-04-16T05:44:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.307+0000 7f9803785740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 0x560a2df69470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.307+0000 7f9803785740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 0x560a2df750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.307+0000 7f9803785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df76360 0x560a2df8de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.307+0000 7f9803785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1af520 con 0x560a2df76360 2024-04-16T05:44:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.307+0000 7f9803785740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1b1060 con 0x560a2df6ce60 2024-04-16T05:44:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.307+0000 7f9803785740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1afbf0 con 0x560a2df6d780 2024-04-16T05:44:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df76360 0x560a2df8de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df76360 0x560a2df8de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48196/0 (socket says 172.21.15.86:48196) 2024-04-16T05:44:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fad9d640 1 -- 172.21.15.86:0/2904958587 learned_addr learned my addr 172.21.15.86:0/2904958587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fa59c640 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 0x560a2df69470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97f3fff640 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 0x560a2df750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fad9d640 1 -- 172.21.15.86:0/2904958587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 msgr2=0x560a2df750f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fad9d640 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 0x560a2df750f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fad9d640 1 -- 172.21.15.86:0/2904958587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 msgr2=0x560a2df69470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fad9d640 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 0x560a2df69470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fad9d640 1 -- 172.21.15.86:0/2904958587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a2df673c0 con 0x560a2df76360 2024-04-16T05:44:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fa59c640 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 0x560a2df69470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97fad9d640 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df76360 0x560a2df8de90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009f90 tx=0x7f97f4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17096fd47d2d583 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97e3fff640 1 -- 172.21.15.86:0/2904958587 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4022040 con 0x560a2df76360 2024-04-16T05:44:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97e3fff640 1 -- 172.21.15.86:0/2904958587 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f4015040 con 0x560a2df76360 2024-04-16T05:44:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.308+0000 7f97e3fff640 1 -- 172.21.15.86:0/2904958587 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f40107f0 con 0x560a2df76360 2024-04-16T05:44:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 -- 172.21.15.86:0/2904958587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df76360 msgr2=0x560a2df8de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df76360 0x560a2df8de90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009f90 tx=0x7f97f4005e30 comp rx=0 tx=0).stop 2024-04-16T05:44:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 -- 172.21.15.86:0/2904958587 shutdown_connections 2024-04-16T05:44:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df76360 0x560a2df8de90 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df6d780 0x560a2df750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 --2- 172.21.15.86:0/2904958587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 0x560a2df69470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 -- 172.21.15.86:0/2904958587 >> 172.21.15.86:0/2904958587 conn(0x560a2df67590 msgr2=0x560a2df749e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 -- 172.21.15.86:0/2904958587 shutdown_connections 2024-04-16T05:44:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 -- 172.21.15.86:0/2904958587 wait complete. 2024-04-16T05:44:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 -- start start 2024-04-16T05:44:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 Processor -- start 2024-04-16T05:44:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 0x560a2df7a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df7a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df76360 0x560a2df7aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1b0990 con 0x560a2df6d780 2024-04-16T05:44:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1b02c0 con 0x560a2df6ce60 2024-04-16T05:44:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1ad520 con 0x560a2df76360 2024-04-16T05:44:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f9803785740 10 client.0 _ll_register_callbacks cb 0x7f97e40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df7a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df7a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48204/0 (socket says 172.21.15.86:48204) 2024-04-16T05:44:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f97f3fff640 1 -- 172.21.15.86:0/1611260575 learned_addr learned my addr 172.21.15.86:0/1611260575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.309+0000 7f97fad9d640 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df76360 0x560a2df7aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97fa59c640 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 0x560a2df7a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97f3fff640 1 -- 172.21.15.86:0/1611260575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df76360 msgr2=0x560a2df7aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97f3fff640 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df76360 0x560a2df7aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97f3fff640 1 -- 172.21.15.86:0/1611260575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 msgr2=0x560a2df7a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97f3fff640 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 0x560a2df7a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97f3fff640 1 -- 172.21.15.86:0/1611260575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2dfeb800 con 0x560a2df6d780 2024-04-16T05:44:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97fad9d640 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df76360 0x560a2df7aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97f3fff640 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df7a5b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f97e4004770 tx=0x7f97e400fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba867e5abdb25b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1611260575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4098be0 con 0x560a2df6d780 2024-04-16T05:44:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1611260575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f97e409ddc0 con 0x560a2df6d780 2024-04-16T05:44:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f9803785740 1 -- 172.21.15.86:0/1611260575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f4015040 con 0x560a2df6d780 2024-04-16T05:44:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1611260575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97e40a0c40 con 0x560a2df6d780 2024-04-16T05:44:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f9803785740 10 client.18216 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f9803785740 1 -- 172.21.15.86:0/1611260575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a2dfecb00 con 0x560a2df6d780 2024-04-16T05:44:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1611260575 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f4015040 con 0x560a2df6d780 2024-04-16T05:44:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1611260575 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97e4020050 con 0x560a2df6d780 2024-04-16T05:44:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f9803785740 10 client.18216 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f9803785740 10 client.18216 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:44:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f9803785740 20 client.18216 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f9803785740 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560a2dfed120 0x560a2dfef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f9803785740 4 client.18216 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.310+0000 7f9803785740 1 -- 172.21.15.86:0/1611260575 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a2ded9970 con 0x560a2dfed120 2024-04-16T05:44:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f97fa59c640 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560a2dfed120 0x560a2dfef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f97e1ffb640 10 client.18216 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:44:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f97fa59c640 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560a2dfed120 0x560a2dfef540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dcfdc6103db92571 server_cookie=46d1be5d4d0bed1f in_seq=0 out_seq=0 2024-04-16T05:44:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f97e1ffb640 1 -- 172.21.15.86:0/1611260575 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a2ded9970 con 0x560a2dfed120 2024-04-16T05:44:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f97e1ffb640 10 client.18216 handle_command_reply: tid=0 2024-04-16T05:44:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 client.18216 shutdown 2024-04-16T05:44:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 -- 172.21.15.86:0/1611260575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 msgr2=0x560a2df7a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df7a5b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f97e4004770 tx=0x7f97e400fc40 comp rx=0 tx=0).stop 2024-04-16T05:44:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 -- 172.21.15.86:0/1611260575 shutdown_connections 2024-04-16T05:44:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x560a2dfed120 0x560a2dfef540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a2df76360 0x560a2df7aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a2df6d780 0x560a2df7a5b0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f97e1ffb640 0 client.18216 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:44:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 --2- 172.21.15.86:0/1611260575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a2df6ce60 0x560a2df7a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 -- 172.21.15.86:0/1611260575 >> 172.21.15.86:0/1611260575 conn(0x560a2df67590 msgr2=0x560a2df66730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.311+0000 7f9803785740 1 -- 172.21.15.86:0/1611260575 shutdown_connections 2024-04-16T05:44:05.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9803785740 1 -- 172.21.15.86:0/1611260575 wait complete. 2024-04-16T05:44:05.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9803785740 20 client.18216 trim_cache size 0 max 16384 2024-04-16T05:44:05.321 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:05.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 -- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97e80a3d20 msgr2=0x7f97e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97e80a3d20 0x7f97e80a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f97f401f000 tx=0x7f97f401b000 comp rx=0 tx=0).stop 2024-04-16T05:44:05.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 -- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc107320 msgr2=0x7f97fc19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc107320 0x7f97fc19eb80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c970 tx=0x7f97e400ce30 comp rx=0 tx=0).stop 2024-04-16T05:44:05.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 -- 172.21.15.86:0/2405646602 shutdown_connections 2024-04-16T05:44:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f97e80a3d20 0x7f97e80a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97fc198750 0x7f97fc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97fc107320 0x7f97fc19eb80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 --2- 172.21.15.86:0/2405646602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc102250 0x7f97fc19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 -- 172.21.15.86:0/2405646602 >> 172.21.15.86:0/2405646602 conn(0x7f97fc079490 msgr2=0x7f97fc0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 -- 172.21.15.86:0/2405646602 shutdown_connections 2024-04-16T05:44:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.312+0000 7f9800ac5640 1 -- 172.21.15.86:0/2405646602 wait complete. 2024-04-16T05:44:05.324 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:05 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2343411775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:05.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:44:05.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.431+0000 7f4cebf96640 1 -- 172.21.15.86:0/3249503265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101970 msgr2=0x7f4ce4105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.431+0000 7f4cebf96640 1 --2- 172.21.15.86:0/3249503265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101970 0x7f4ce4105e30 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009930 tx=0x7f4cd402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.431+0000 7f4cebf96640 1 -- 172.21.15.86:0/3249503265 shutdown_connections 2024-04-16T05:44:05.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.431+0000 7f4cebf96640 1 --2- 172.21.15.86:0/3249503265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ce4106450 0x7f4ce4106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.431+0000 7f4cebf96640 1 --2- 172.21.15.86:0/3249503265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101970 0x7f4ce4105e30 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.431+0000 7f4cebf96640 1 --2- 172.21.15.86:0/3249503265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ce4101050 0x7f4ce4101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.431+0000 7f4cebf96640 1 -- 172.21.15.86:0/3249503265 >> 172.21.15.86:0/3249503265 conn(0x7f4ce4077280 msgr2=0x7f4ce4077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 -- 172.21.15.86:0/3249503265 shutdown_connections 2024-04-16T05:44:05.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 -- 172.21.15.86:0/3249503265 wait complete. 2024-04-16T05:44:05.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 Processor -- start 2024-04-16T05:44:05.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 -- start start 2024-04-16T05:44:05.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101050 0x7f4ce419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ce4101970 0x7f4ce419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ce4106450 0x7f4ce41a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41189e0 con 0x7f4ce4101050 2024-04-16T05:44:05.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4118860 con 0x7f4ce4101970 2024-04-16T05:44:05.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4cebf96640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4118b60 con 0x7f4ce4106450 2024-04-16T05:44:05.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.432+0000 7f4ce9d32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101050 0x7f4ce419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9d32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101050 0x7f4ce419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48230/0 (socket says 172.21.15.86:48230) 2024-04-16T05:44:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9d32640 1 -- 172.21.15.86:0/1029482697 learned_addr learned my addr 172.21.15.86:0/1029482697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4cea533640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ce4106450 0x7f4ce41a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9d32640 1 -- 172.21.15.86:0/1029482697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ce4106450 msgr2=0x7f4ce41a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ce4106450 0x7f4ce41a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9d32640 1 -- 172.21.15.86:0/1029482697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ce4101970 msgr2=0x7f4ce419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9531640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ce4101970 0x7f4ce419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ce4101970 0x7f4ce419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9d32640 1 -- 172.21.15.86:0/1029482697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce41a3ea0 con 0x7f4ce4101050 2024-04-16T05:44:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9531640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ce4101970 0x7f4ce419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101050 0x7f4ce419ef00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c890 tx=0x7f4ce000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4cdaffd640 1 -- 172.21.15.86:0/1029482697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0004220 con 0x7f4ce4101050 2024-04-16T05:44:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4cdaffd640 1 -- 172.21.15.86:0/1029482697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce00043c0 con 0x7f4ce4101050 2024-04-16T05:44:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4cebf96640 1 -- 172.21.15.86:0/1029482697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce41a4190 con 0x7f4ce4101050 2024-04-16T05:44:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4cdaffd640 1 -- 172.21.15.86:0/1029482697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00049e0 con 0x7f4ce4101050 2024-04-16T05:44:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.433+0000 7f4cebf96640 1 -- 172.21.15.86:0/1029482697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce41aba20 con 0x7f4ce4101050 2024-04-16T05:44:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.435+0000 7f4cdaffd640 1 -- 172.21.15.86:0/1029482697 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ce00079b0 con 0x7f4ce4101050 2024-04-16T05:44:05.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ceeb77740 1 Processor -- start 2024-04-16T05:44:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ceeb77740 1 -- start start 2024-04-16T05:44:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c98544a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c98544a780 0x55c985452e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c985455750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9853af0e0 con 0x55c985449e60 2024-04-16T05:44:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c98534c520 con 0x55c985453360 2024-04-16T05:44:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c9853ab0a0 con 0x55c98544a780 2024-04-16T05:44:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cdaffd640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cea533640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c98544a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cea533640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c98544a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48240/0 (socket says 172.21.15.86:48240) 2024-04-16T05:44:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cea533640 1 -- 172.21.15.86:0/1121189362 learned_addr learned my addr 172.21.15.86:0/1121189362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cdaffd640 1 -- 172.21.15.86:0/1029482697 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f4ce00d0cb0 con 0x7f4ce4101050 2024-04-16T05:44:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ce9531640 1 --2- 172.21.15.86:0/1121189362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c985455750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cea533640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/1121189362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c98544a780 0x55c985452e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cea533640 1 -- 172.21.15.86:0/1121189362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c98544a780 msgr2=0x55c985452e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cea533640 1 --2- 172.21.15.86:0/1121189362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c98544a780 0x55c985452e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cea533640 1 -- 172.21.15.86:0/1121189362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 msgr2=0x55c985455750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cea533640 1 --2- 172.21.15.86:0/1121189362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c985455750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.436+0000 7f4cea533640 1 -- 172.21.15.86:0/1121189362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9853748d0 con 0x55c985449e60 2024-04-16T05:44:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.437+0000 7f4cea533640 1 --2- 172.21.15.86:0/1121189362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c98544a240 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc014cf0 tx=0x7f4cdc011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1337e7ab29b81cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.437+0000 7f4cea533640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009fd0 tx=0x7f4cdc016000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.437+0000 7f4ccffff640 1 -- 172.21.15.86:0/1121189362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc009ab0 con 0x55c985449e60 2024-04-16T05:44:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.437+0000 7f4ccffff640 1 -- 172.21.15.86:0/1121189362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cdc014640 con 0x55c985449e60 2024-04-16T05:44:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.437+0000 7f4ccffff640 1 -- 172.21.15.86:0/1121189362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc005020 con 0x55c985449e60 2024-04-16T05:44:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 -- 172.21.15.86:0/1121189362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 msgr2=0x55c98544a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/1121189362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c98544a240 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc014cf0 tx=0x7f4cdc011750 comp rx=0 tx=0).stop 2024-04-16T05:44:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 -- 172.21.15.86:0/1121189362 shutdown_connections 2024-04-16T05:44:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/1121189362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c985455750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/1121189362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c98544a780 0x55c985452e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/1121189362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c98544a240 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 -- 172.21.15.86:0/1121189362 >> 172.21.15.86:0/1121189362 conn(0x55c985444590 msgr2=0x55c985444970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 -- 172.21.15.86:0/1121189362 shutdown_connections 2024-04-16T05:44:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 -- 172.21.15.86:0/1121189362 wait complete. 2024-04-16T05:44:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 -- start start 2024-04-16T05:44:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 Processor -- start 2024-04-16T05:44:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c985449e60 0x55c9854504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c9854509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c985450f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c985455fa0 con 0x55c98544a780 2024-04-16T05:44:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c9853684d0 con 0x55c985453360 2024-04-16T05:44:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.438+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c985456120 con 0x55c985449e60 2024-04-16T05:44:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ceeb77740 10 client.0 _ll_register_callbacks cb 0x55c98534d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9531640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c9854509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9531640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c9854509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48248/0 (socket says 172.21.15.86:48248) 2024-04-16T05:44:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9531640 1 -- 172.21.15.86:0/2465768488 learned_addr learned my addr 172.21.15.86:0/2465768488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c985449e60 0x55c9854504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4cea533640 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c985450f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9531640 1 -- 172.21.15.86:0/2465768488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c985449e60 msgr2=0x55c9854504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9531640 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c985449e60 0x55c9854504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9531640 1 -- 172.21.15.86:0/2465768488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 msgr2=0x55c985450f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9531640 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c985450f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9531640 1 -- 172.21.15.86:0/2465768488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9850c13f0 con 0x55c98544a780 2024-04-16T05:44:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ce9531640 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c9854509e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009900 tx=0x7f4cd4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386f4db444f090dd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2465768488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4042070 con 0x55c98544a780 2024-04-16T05:44:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2465768488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f4cd4005030 con 0x55c98544a780 2024-04-16T05:44:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2465768488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cdc014640 con 0x55c98544a780 2024-04-16T05:44:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2465768488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4cd4005230 con 0x55c98544a780 2024-04-16T05:44:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ceeb77740 10 client.18228 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.439+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2465768488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c98518b7d0 con 0x55c98544a780 2024-04-16T05:44:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.440+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2465768488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cdc014640 con 0x55c98544a780 2024-04-16T05:44:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.440+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2465768488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4cd4003680 con 0x55c98544a780 2024-04-16T05:44:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.440+0000 7f4ceeb77740 10 client.18228 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.440+0000 7f4ceeb77740 10 client.18228 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:44:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.440+0000 7f4ceeb77740 20 client.18228 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.440+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c985463350 0x55c985465760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.440+0000 7f4ceeb77740 4 client.18228 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.440+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2465768488 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9853bcb00 con 0x55c985463350 2024-04-16T05:44:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.440+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c985463350 0x55c985465760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.441+0000 7f4ccd7fa640 10 client.18228 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:44:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.441+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c985463350 0x55c985465760 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=950f4c29ea67e695 server_cookie=9fe9d40a611b1a1c in_seq=0 out_seq=0 2024-04-16T05:44:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.441+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2465768488 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9853bcb00 con 0x55c985463350 2024-04-16T05:44:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.441+0000 7f4ccd7fa640 10 client.18228 handle_command_reply: tid=0 2024-04-16T05:44:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.441+0000 7f4ceeb77740 1 client.18228 shutdown 2024-04-16T05:44:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.441+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2465768488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 msgr2=0x55c9854509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.441+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c9854509e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009900 tx=0x7f4cd4005bd0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2465768488 shutdown_connections 2024-04-16T05:44:05.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c985450f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c985463350 0x55c985465760 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c9854509e0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ccd7fa640 0 client.18228 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:44:05.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2465768488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c985449e60 0x55c9854504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2465768488 >> 172.21.15.86:0/2465768488 conn(0x55c985444590 msgr2=0x55c9854462e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2465768488 shutdown_connections 2024-04-16T05:44:05.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2465768488 wait complete. 2024-04-16T05:44:05.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.442+0000 7f4ceeb77740 20 client.18228 trim_cache size 0 max 16384 2024-04-16T05:44:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ceeb77740 1 Processor -- start 2024-04-16T05:44:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ceeb77740 1 -- start start 2024-04-16T05:44:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c985452bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c98544a780 0x55c98545eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c98543f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41b5c40 con 0x55c985449e60 2024-04-16T05:44:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41b7780 con 0x55c985453360 2024-04-16T05:44:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41b6310 con 0x55c98544a780 2024-04-16T05:44:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ce9d32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c985452bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ce9d32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c985452bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48262/0 (socket says 172.21.15.86:48262) 2024-04-16T05:44:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ce9d32640 1 -- 172.21.15.86:0/2839809359 learned_addr learned my addr 172.21.15.86:0/2839809359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ce9d32640 1 -- 172.21.15.86:0/2839809359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c98544a780 msgr2=0x55c98545eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2839809359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c98544a780 0x55c98545eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4cea533640 1 --2- 172.21.15.86:0/2839809359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c98543f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ce9d32640 1 -- 172.21.15.86:0/2839809359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 msgr2=0x55c98543f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2839809359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c98543f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4ce9d32640 1 -- 172.21.15.86:0/2839809359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98544f470 con 0x55c985449e60 2024-04-16T05:44:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.455+0000 7f4cea533640 1 --2- 172.21.15.86:0/2839809359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c98543f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2839809359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c985452bd0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00ce0e0 tx=0x7f4ce001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8696afb2060d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ccf7fe640 1 -- 172.21.15.86:0/2839809359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce001d040 con 0x55c985449e60 2024-04-16T05:44:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ccf7fe640 1 -- 172.21.15.86:0/2839809359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce009ab80 con 0x55c985449e60 2024-04-16T05:44:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ccf7fe640 1 -- 172.21.15.86:0/2839809359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00870b0 con 0x55c985449e60 2024-04-16T05:44:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2839809359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 msgr2=0x55c985452bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2839809359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c985452bd0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00ce0e0 tx=0x7f4ce001a040 comp rx=0 tx=0).stop 2024-04-16T05:44:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2839809359 shutdown_connections 2024-04-16T05:44:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2839809359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985453360 0x55c98543f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2839809359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c98544a780 0x55c98545eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2839809359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c985449e60 0x55c985452bd0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2839809359 >> 172.21.15.86:0/2839809359 conn(0x55c985444590 msgr2=0x55c985443730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2839809359 shutdown_connections 2024-04-16T05:44:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.456+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2839809359 wait complete. 2024-04-16T05:44:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ceeb77740 1 -- start start 2024-04-16T05:44:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ceeb77740 1 Processor -- start 2024-04-16T05:44:05.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985449e60 0x55c98545b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c98545b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ceeb77740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c985453360 0x55c985458410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41b70b0 con 0x55c98544a780 2024-04-16T05:44:05.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41b69e0 con 0x55c985449e60 2024-04-16T05:44:05.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ceeb77740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41b3c40 con 0x55c985453360 2024-04-16T05:44:05.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ceeb77740 10 client.0 _ll_register_callbacks cb 0x7f4cd402cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ce9531640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c98545b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ce9531640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c98545b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48274/0 (socket says 172.21.15.86:48274) 2024-04-16T05:44:05.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ce9531640 1 -- 172.21.15.86:0/2386312795 learned_addr learned my addr 172.21.15.86:0/2386312795 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985449e60 0x55c98545b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ce9531640 1 -- 172.21.15.86:0/2386312795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c985453360 msgr2=0x55c985458410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ce9531640 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c985453360 0x55c985458410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ce9531640 1 -- 172.21.15.86:0/2386312795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985449e60 msgr2=0x55c98545b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ce9531640 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985449e60 0x55c98545b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.457+0000 7f4ce9531640 1 -- 172.21.15.86:0/2386312795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c985458c40 con 0x55c98544a780 2024-04-16T05:44:05.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985449e60 0x55c98545b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ce9531640 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c98545b940 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006010 tx=0x7f4cd402c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bda37267784462 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2386312795 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4042070 con 0x55c98544a780 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2386312795 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f4cd40055d0 con 0x55c98544a780 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2386312795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce009ab80 con 0x55c98544a780 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2386312795 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4cd4009520 con 0x55c98544a780 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ceeb77740 10 client.18234 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2386312795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9854ca3c0 con 0x55c98544a780 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2386312795 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce009ab80 con 0x55c98544a780 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2386312795 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4cd4034440 con 0x55c98544a780 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ceeb77740 10 client.18234 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ceeb77740 10 client.18234 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:44:05.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ceeb77740 20 client.18234 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c9854ca9a0 0x55c9854ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ceeb77740 4 client.18234 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.458+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2386312795 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9853b6970 con 0x55c9854ca9a0 2024-04-16T05:44:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c9854ca9a0 0x55c9854ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ce9d32640 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c9854ca9a0 0x55c9854ccd90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6f43ab65caa38f server_cookie=7f3406866462811a in_seq=0 out_seq=0 2024-04-16T05:44:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ccd7fa640 10 client.18234 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:44:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ccd7fa640 1 -- 172.21.15.86:0/2386312795 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9853b6970 con 0x55c9854ca9a0 2024-04-16T05:44:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ccd7fa640 10 client.18234 handle_command_reply: tid=0 2024-04-16T05:44:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ceeb77740 1 client.18234 shutdown 2024-04-16T05:44:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2386312795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 msgr2=0x55c98545b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c98545b940 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006010 tx=0x7f4cd402c470 comp rx=0 tx=0).stop 2024-04-16T05:44:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2386312795 shutdown_connections 2024-04-16T05:44:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c985453360 0x55c985458410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c9854ca9a0 0x55c9854ccd90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98544a780 0x55c98545b940 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ceeb77740 1 --2- 172.21.15.86:0/2386312795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c985449e60 0x55c98545b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ccd7fa640 0 client.18234 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:44:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.459+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2386312795 >> 172.21.15.86:0/2386312795 conn(0x55c985444590 msgr2=0x55c985450440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2386312795 shutdown_connections 2024-04-16T05:44:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4ceeb77740 1 -- 172.21.15.86:0/2386312795 wait complete. 2024-04-16T05:44:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4ceeb77740 20 client.18234 trim_cache size 0 max 16384 2024-04-16T05:44:05.468 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 -- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4cd00a3d20 msgr2=0x7f4cd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009fd0 tx=0x7f4cdc016000 comp rx=0 tx=0).stop 2024-04-16T05:44:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 -- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101050 msgr2=0x7f4ce419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101050 0x7f4ce419ef00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c890 tx=0x7f4ce000cd50 comp rx=0 tx=0).stop 2024-04-16T05:44:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 -- 172.21.15.86:0/1029482697 shutdown_connections 2024-04-16T05:44:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ce4106450 0x7f4ce41a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ce4101970 0x7f4ce419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 --2- 172.21.15.86:0/1029482697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce4101050 0x7f4ce419ef00 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 -- 172.21.15.86:0/1029482697 >> 172.21.15.86:0/1029482697 conn(0x7f4ce4077280 msgr2=0x7f4ce4103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 -- 172.21.15.86:0/1029482697 shutdown_connections 2024-04-16T05:44:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.460+0000 7f4cebf96640 1 -- 172.21.15.86:0/1029482697 wait complete. 2024-04-16T05:44:05.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:44:05.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:05 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2343411775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 -- 172.21.15.86:0/4067443220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34106200 msgr2=0x7f4a3410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 --2- 172.21.15.86:0/4067443220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34106200 0x7f4a3410ddd0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009ca0 tx=0x7f4a2c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 -- 172.21.15.86:0/4067443220 shutdown_connections 2024-04-16T05:44:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 --2- 172.21.15.86:0/4067443220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34106200 0x7f4a3410ddd0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 --2- 172.21.15.86:0/4067443220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a34101ae0 0x7f4a34105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 --2- 172.21.15.86:0/4067443220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a34101130 0x7f4a34101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 -- 172.21.15.86:0/4067443220 >> 172.21.15.86:0/4067443220 conn(0x7f4a340780b0 msgr2=0x7f4a340784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 -- 172.21.15.86:0/4067443220 shutdown_connections 2024-04-16T05:44:05.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 -- 172.21.15.86:0/4067443220 wait complete. 2024-04-16T05:44:05.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.579+0000 7f4a3b132640 1 Processor -- start 2024-04-16T05:44:05.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a3b132640 1 -- start start 2024-04-16T05:44:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a3b132640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34101130 0x7f4a3419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a3b132640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a34106200 0x7f4a3419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a3b132640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a341985d0 0x7f4a34198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a3b132640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341112a0 con 0x7f4a34101130 2024-04-16T05:44:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a3b132640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34111120 con 0x7f4a34106200 2024-04-16T05:44:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a3b132640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34111420 con 0x7f4a341985d0 2024-04-16T05:44:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a38ece640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34101130 0x7f4a3419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a38ece640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34101130 0x7f4a3419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48298/0 (socket says 172.21.15.86:48298) 2024-04-16T05:44:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a38ece640 1 -- 172.21.15.86:0/1483566653 learned_addr learned my addr 172.21.15.86:0/1483566653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a33fff640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a34106200 0x7f4a3419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a396cf640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a341985d0 0x7f4a34198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a38ece640 1 -- 172.21.15.86:0/1483566653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a341985d0 msgr2=0x7f4a34198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a38ece640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a341985d0 0x7f4a34198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a38ece640 1 -- 172.21.15.86:0/1483566653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a34106200 msgr2=0x7f4a3419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a38ece640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a34106200 0x7f4a3419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a38ece640 1 -- 172.21.15.86:0/1483566653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a341990f0 con 0x7f4a34101130 2024-04-16T05:44:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a33fff640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a34106200 0x7f4a3419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a396cf640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a341985d0 0x7f4a34198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a38ece640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34101130 0x7f4a3419e420 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9f0 tx=0x7f4a2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a31ffb640 1 -- 172.21.15.86:0/1483566653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800cd80 con 0x7f4a34101130 2024-04-16T05:44:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.580+0000 7f4a31ffb640 1 -- 172.21.15.86:0/1483566653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a28010910 con 0x7f4a34101130 2024-04-16T05:44:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.581+0000 7f4a3b132640 1 -- 172.21.15.86:0/1483566653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a341993e0 con 0x7f4a34101130 2024-04-16T05:44:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.581+0000 7f4a31ffb640 1 -- 172.21.15.86:0/1483566653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28018990 con 0x7f4a34101130 2024-04-16T05:44:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.581+0000 7f4a3b132640 1 -- 172.21.15.86:0/1483566653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a341a5680 con 0x7f4a34101130 2024-04-16T05:44:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a31ffb640 1 -- 172.21.15.86:0/1483566653 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a28010ac0 con 0x7f4a34101130 2024-04-16T05:44:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a3ddf2740 1 Processor -- start 2024-04-16T05:44:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a3ddf2740 1 -- start start 2024-04-16T05:44:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559eb48210e0 con 0x559eb48c5360 2024-04-16T05:44:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559eb47be520 con 0x559eb48bc780 2024-04-16T05:44:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559eb481d0a0 con 0x559eb48bbe60 2024-04-16T05:44:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a33fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a33fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48310/0 (socket says 172.21.15.86:48310) 2024-04-16T05:44:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a33fff640 1 -- 172.21.15.86:0/1805001783 learned_addr learned my addr 172.21.15.86:0/1805001783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a396cf640 1 --2- 172.21.15.86:0/1805001783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a38ece640 1 --2- 172.21.15.86:0/1805001783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a31ffb640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a240a3da0 0x7f4a240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a33fff640 1 -- 172.21.15.86:0/1805001783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 msgr2=0x559eb48bc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a33fff640 1 --2- 172.21.15.86:0/1805001783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48bc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a33fff640 1 -- 172.21.15.86:0/1805001783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 msgr2=0x559eb48c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a33fff640 1 --2- 172.21.15.86:0/1805001783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a33fff640 1 -- 172.21.15.86:0/1805001783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb47e68d0 con 0x559eb48c5360 2024-04-16T05:44:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.584+0000 7f4a396cf640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a240a3da0 0x7f4a240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.585+0000 7f4a31ffb640 1 -- 172.21.15.86:0/1483566653 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f4a280d0fb0 con 0x7f4a34101130 2024-04-16T05:44:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.585+0000 7f4a33fff640 1 --2- 172.21.15.86:0/1805001783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c7750 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000bb20 tx=0x7f4a2000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22dd3b9cc974f7cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.585+0000 7f4a1effd640 1 -- 172.21.15.86:0/1805001783 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20017420 con 0x559eb48c5360 2024-04-16T05:44:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.585+0000 7f4a1effd640 1 -- 172.21.15.86:0/1805001783 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a20017aa0 con 0x559eb48c5360 2024-04-16T05:44:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.585+0000 7f4a1effd640 1 -- 172.21.15.86:0/1805001783 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000fb90 con 0x559eb48c5360 2024-04-16T05:44:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.585+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/1805001783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 msgr2=0x559eb48c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.585+0000 7f4a396cf640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a240a3da0 0x7f4a240a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005e30 tx=0x7f4a2c0095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.585+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/1805001783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c7750 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000bb20 tx=0x7f4a2000d340 comp rx=0 tx=0).stop 2024-04-16T05:44:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/1805001783 shutdown_connections 2024-04-16T05:44:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/1805001783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c7750 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/1805001783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/1805001783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48bc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/1805001783 >> 172.21.15.86:0/1805001783 conn(0x559eb48b6590 msgr2=0x559eb48b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/1805001783 shutdown_connections 2024-04-16T05:44:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/1805001783 wait complete. 2024-04-16T05:44:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 -- start start 2024-04-16T05:44:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 Processor -- start 2024-04-16T05:44:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48c2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48c2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559eb48c7fa0 con 0x559eb48c5360 2024-04-16T05:44:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559eb47da4d0 con 0x559eb48bc780 2024-04-16T05:44:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559eb48c8120 con 0x559eb48bbe60 2024-04-16T05:44:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a3ddf2740 10 client.0 _ll_register_callbacks cb 0x559eb47bf2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.586+0000 7f4a396cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a396cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48324/0 (socket says 172.21.15.86:48324) 2024-04-16T05:44:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a396cf640 1 -- 172.21.15.86:0/2636787666 learned_addr learned my addr 172.21.15.86:0/2636787666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a38ece640 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a33fff640 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a396cf640 1 -- 172.21.15.86:0/2636787666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 msgr2=0x559eb48c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a396cf640 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48c2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a396cf640 1 -- 172.21.15.86:0/2636787666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 msgr2=0x559eb48c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a396cf640 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48c2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a396cf640 1 -- 172.21.15.86:0/2636787666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb45333f0 con 0x559eb48c5360 2024-04-16T05:44:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a396cf640 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c2ec0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009ca0 tx=0x7f4a2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f955c8a4f88640e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a13fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a13fff640 1 -- 172.21.15.86:0/2636787666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c04f020 con 0x559eb48c5360 2024-04-16T05:44:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a13fff640 1 -- 172.21.15.86:0/2636787666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f4a2c0328d0 con 0x559eb48c5360 2024-04-16T05:44:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2636787666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a20017aa0 con 0x559eb48c5360 2024-04-16T05:44:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a13fff640 1 -- 172.21.15.86:0/2636787666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a2c05a050 con 0x559eb48c5360 2024-04-16T05:44:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a3ddf2740 10 client.18246 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2636787666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559eb45fd7d0 con 0x559eb48c5360 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.587+0000 7f4a13fff640 1 -- 172.21.15.86:0/2636787666 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a20017aa0 con 0x559eb48c5360 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a13fff640 1 -- 172.21.15.86:0/2636787666 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a2c04b6a0 con 0x559eb48c5360 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a3ddf2740 10 client.18246 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a3ddf2740 10 client.18246 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a3ddf2740 20 client.18246 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559eb48d5350 0x559eb48d77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a3ddf2740 4 client.18246 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2636787666 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eb482eb00 con 0x559eb48d5350 2024-04-16T05:44:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a38ece640 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559eb48d5350 0x559eb48d77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a13fff640 10 client.18246 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:44:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.588+0000 7f4a38ece640 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559eb48d5350 0x559eb48d77a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=858dd1fe2d79577 server_cookie=c6a8854528ec3135 in_seq=0 out_seq=0 2024-04-16T05:44:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a13fff640 1 -- 172.21.15.86:0/2636787666 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559eb482eb00 con 0x559eb48d5350 2024-04-16T05:44:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a13fff640 10 client.18246 handle_command_reply: tid=0 2024-04-16T05:44:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 client.18246 shutdown 2024-04-16T05:44:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2636787666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 msgr2=0x559eb48c2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c2ec0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009ca0 tx=0x7f4a2c004a40 comp rx=0 tx=0).stop 2024-04-16T05:44:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2636787666 shutdown_connections 2024-04-16T05:44:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559eb48d5350 0x559eb48d77a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48c2ec0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a13fff640 0 client.18246 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:44:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48c2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2636787666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2636787666 >> 172.21.15.86:0/2636787666 conn(0x559eb48b6590 msgr2=0x559eb48b82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2636787666 shutdown_connections 2024-04-16T05:44:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.589+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2636787666 wait complete. 2024-04-16T05:44:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.590+0000 7f4a3ddf2740 20 client.18246 trim_cache size 0 max 16384 2024-04-16T05:44:05.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.602+0000 7f4a3ddf2740 1 Processor -- start 2024-04-16T05:44:05.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.602+0000 7f4a3ddf2740 1 -- start start 2024-04-16T05:44:05.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48dd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341af470 con 0x559eb48c5360 2024-04-16T05:44:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341b0fb0 con 0x559eb48bc780 2024-04-16T05:44:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341afb40 con 0x559eb48bbe60 2024-04-16T05:44:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a396cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a396cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48dd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48340/0 (socket says 172.21.15.86:48340) 2024-04-16T05:44:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a396cf640 1 -- 172.21.15.86:0/3566978284 learned_addr learned my addr 172.21.15.86:0/3566978284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a38ece640 1 --2- 172.21.15.86:0/3566978284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a396cf640 1 -- 172.21.15.86:0/3566978284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 msgr2=0x559eb48c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a396cf640 1 --2- 172.21.15.86:0/3566978284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a396cf640 1 -- 172.21.15.86:0/3566978284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 msgr2=0x559eb48d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a396cf640 1 --2- 172.21.15.86:0/3566978284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a396cf640 1 -- 172.21.15.86:0/3566978284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb48d7e20 con 0x559eb48c5360 2024-04-16T05:44:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a396cf640 1 --2- 172.21.15.86:0/3566978284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48dd070 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c04bac0 tx=0x7f4a2c034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67665a4243c78f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a1e7fc640 1 -- 172.21.15.86:0/3566978284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c04f030 con 0x559eb48c5360 2024-04-16T05:44:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a1e7fc640 1 -- 172.21.15.86:0/3566978284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a2c063070 con 0x559eb48c5360 2024-04-16T05:44:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.603+0000 7f4a1e7fc640 1 -- 172.21.15.86:0/3566978284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c039030 con 0x559eb48c5360 2024-04-16T05:44:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/3566978284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 msgr2=0x559eb48dd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/3566978284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48dd070 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c04bac0 tx=0x7f4a2c034440 comp rx=0 tx=0).stop 2024-04-16T05:44:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/3566978284 shutdown_connections 2024-04-16T05:44:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/3566978284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48dd070 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/3566978284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/3566978284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/3566978284 >> 172.21.15.86:0/3566978284 conn(0x559eb48b6590 msgr2=0x559eb48b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/3566978284 shutdown_connections 2024-04-16T05:44:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/3566978284 wait complete. 2024-04-16T05:44:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 -- start start 2024-04-16T05:44:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.604+0000 7f4a3ddf2740 1 Processor -- start 2024-04-16T05:44:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48bed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a3ddf2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341b08e0 con 0x559eb48c5360 2024-04-16T05:44:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341b0210 con 0x559eb48bc780 2024-04-16T05:44:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a3ddf2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341ad470 con 0x559eb48bbe60 2024-04-16T05:44:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a3ddf2740 10 client.0 _ll_register_callbacks cb 0x7f4a2c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a396cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a396cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48bf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48342/0 (socket says 172.21.15.86:48342) 2024-04-16T05:44:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a396cf640 1 -- 172.21.15.86:0/2599371986 learned_addr learned my addr 172.21.15.86:0/2599371986 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a33fff640 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48bed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a38ece640 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a396cf640 1 -- 172.21.15.86:0/2599371986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 msgr2=0x559eb48cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a396cf640 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a396cf640 1 -- 172.21.15.86:0/2599371986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 msgr2=0x559eb48bed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a396cf640 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48bed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a396cf640 1 -- 172.21.15.86:0/2599371986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb48bf9d0 con 0x559eb48c5360 2024-04-16T05:44:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a33fff640 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48bed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a38ece640 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a396cf640 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48bf290 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c033000 tx=0x7f4a2c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a313c7c642f4c5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a13fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a13fff640 1 -- 172.21.15.86:0/2599371986 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c06c060 con 0x559eb48c5360 2024-04-16T05:44:05.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a13fff640 1 -- 172.21.15.86:0/2599371986 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f4a2c004240 con 0x559eb48c5360 2024-04-16T05:44:05.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.605+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2599371986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2c063070 con 0x559eb48c5360 2024-04-16T05:44:05.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a13fff640 1 -- 172.21.15.86:0/2599371986 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a2c073050 con 0x559eb48c5360 2024-04-16T05:44:05.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a3ddf2740 10 client.18252 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2599371986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559eb493b810 con 0x559eb48c5360 2024-04-16T05:44:05.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a13fff640 1 -- 172.21.15.86:0/2599371986 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a2c063070 con 0x559eb48c5360 2024-04-16T05:44:05.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a13fff640 1 -- 172.21.15.86:0/2599371986 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a2c0346f0 con 0x559eb48c5360 2024-04-16T05:44:05.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a3ddf2740 10 client.18252 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a3ddf2740 10 client.18252 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:44:05.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a3ddf2740 20 client.18252 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559eb493be20 0x559eb493e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a3ddf2740 4 client.18252 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:05.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2599371986 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559eb4828970 con 0x559eb493be20 2024-04-16T05:44:05.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.606+0000 7f4a38ece640 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559eb493be20 0x559eb493e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a13fff640 10 client.18252 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:44:05.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a38ece640 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559eb493be20 0x559eb493e210 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=906f66b1c1c32527 server_cookie=7c6828e0c308bbe5 in_seq=0 out_seq=0 2024-04-16T05:44:05.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a13fff640 1 -- 172.21.15.86:0/2599371986 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559eb4828970 con 0x559eb493be20 2024-04-16T05:44:05.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a13fff640 10 client.18252 handle_command_reply: tid=0 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 client.18252 shutdown 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2599371986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 msgr2=0x559eb48bf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48bf290 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c033000 tx=0x7f4a2c032d50 comp rx=0 tx=0).stop 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2599371986 shutdown_connections 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x559eb493be20 0x559eb493e210 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559eb48c5360 0x559eb48bf290 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a13fff640 0 client.18252 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559eb48bc780 0x559eb48bed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 --2- 172.21.15.86:0/2599371986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eb48bbe60 0x559eb48cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2599371986 >> 172.21.15.86:0/2599371986 conn(0x559eb48b6590 msgr2=0x559eb48b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2599371986 shutdown_connections 2024-04-16T05:44:05.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 1 -- 172.21.15.86:0/2599371986 wait complete. 2024-04-16T05:44:05.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.607+0000 7f4a3ddf2740 20 client.18252 trim_cache size 0 max 16384 2024-04-16T05:44:05.616 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 -- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a240a3da0 msgr2=0x7f4a240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a240a3da0 0x7f4a240a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005e30 tx=0x7f4a2c0095c0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 -- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34101130 msgr2=0x7f4a3419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34101130 0x7f4a3419e420 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9f0 tx=0x7f4a2800eeb0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 -- 172.21.15.86:0/1483566653 shutdown_connections 2024-04-16T05:44:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a341985d0 0x7f4a34198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a240a3da0 0x7f4a240a6260 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a34106200 0x7f4a3419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 --2- 172.21.15.86:0/1483566653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a34101130 0x7f4a3419e420 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 -- 172.21.15.86:0/1483566653 >> 172.21.15.86:0/1483566653 conn(0x7f4a340780b0 msgr2=0x7f4a34069370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 -- 172.21.15.86:0/1483566653 shutdown_connections 2024-04-16T05:44:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.608+0000 7f4a3b132640 1 -- 172.21.15.86:0/1483566653 wait complete. 2024-04-16T05:44:05.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:44:05.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:05 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2343411775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.727+0000 7f4301d1c640 1 -- 172.21.15.86:0/2158928663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 msgr2=0x7f42fc10dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.727+0000 7f4301d1c640 1 --2- 172.21.15.86:0/2158928663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 0x7f42fc10dde0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009900 tx=0x7f42f402bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.727+0000 7f4301d1c640 1 -- 172.21.15.86:0/2158928663 shutdown_connections 2024-04-16T05:44:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.727+0000 7f4301d1c640 1 --2- 172.21.15.86:0/2158928663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 0x7f42fc10dde0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.727+0000 7f4301d1c640 1 --2- 172.21.15.86:0/2158928663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42fc101af0 0x7f42fc105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.727+0000 7f4301d1c640 1 --2- 172.21.15.86:0/2158928663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42fc101140 0x7f42fc101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.727+0000 7f4301d1c640 1 -- 172.21.15.86:0/2158928663 >> 172.21.15.86:0/2158928663 conn(0x7f42fc077280 msgr2=0x7f42fc077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.727+0000 7f4301d1c640 1 -- 172.21.15.86:0/2158928663 shutdown_connections 2024-04-16T05:44:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.727+0000 7f4301d1c640 1 -- 172.21.15.86:0/2158928663 wait complete. 2024-04-16T05:44:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f4301d1c640 1 Processor -- start 2024-04-16T05:44:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f4301d1c640 1 -- start start 2024-04-16T05:44:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f4301d1c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42fc101140 0x7f42fc199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f4301d1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 0x7f42fc199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f4301d1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42fc19a140 0x7f42fc1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f4301d1c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1112d0 con 0x7f42fc106210 2024-04-16T05:44:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f4301d1c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc111150 con 0x7f42fc101140 2024-04-16T05:44:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f4301d1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc111450 con 0x7f42fc19a140 2024-04-16T05:44:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f42faffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 0x7f42fc199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f42faffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 0x7f42fc199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48350/0 (socket says 172.21.15.86:48350) 2024-04-16T05:44:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.728+0000 7f42faffd640 1 -- 172.21.15.86:0/209845139 learned_addr learned my addr 172.21.15.86:0/209845139 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42fbfff640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42fc19a140 0x7f42fc1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42faffd640 1 -- 172.21.15.86:0/209845139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42fc19a140 msgr2=0x7f42fc1a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42faffd640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42fc19a140 0x7f42fc1a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42faffd640 1 -- 172.21.15.86:0/209845139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42fc101140 msgr2=0x7f42fc199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42faffd640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42fc101140 0x7f42fc199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42faffd640 1 -- 172.21.15.86:0/209845139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc1a4ef0 con 0x7f42fc106210 2024-04-16T05:44:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42fbfff640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42fc19a140 0x7f42fc1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42faffd640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 0x7f42fc199bb0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f42f000c970 tx=0x7f42f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42f8ff9640 1 -- 172.21.15.86:0/209845139 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0007bb0 con 0x7f42fc106210 2024-04-16T05:44:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f4301d1c640 1 -- 172.21.15.86:0/209845139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42fc1a51e0 con 0x7f42fc106210 2024-04-16T05:44:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42f8ff9640 1 -- 172.21.15.86:0/209845139 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f0007d50 con 0x7f42fc106210 2024-04-16T05:44:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f42f8ff9640 1 -- 172.21.15.86:0/209845139 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f00187a0 con 0x7f42fc106210 2024-04-16T05:44:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.729+0000 7f4301d1c640 1 -- 172.21.15.86:0/209845139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42fc1a5730 con 0x7f42fc106210 2024-04-16T05:44:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f43049e9740 1 Processor -- start 2024-04-16T05:44:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f42f8ff9640 1 -- 172.21.15.86:0/209845139 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f42f000f450 con 0x7f42fc106210 2024-04-16T05:44:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f43049e9740 1 -- start start 2024-04-16T05:44:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f43049e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d57e60 0x561bd4d58240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f43049e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 0x561bd4d60e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f43049e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 0x561bd4d63750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f43049e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bd4cbd0e0 con 0x561bd4d57e60 2024-04-16T05:44:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f43049e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561bd4c5a520 con 0x561bd4d58780 2024-04-16T05:44:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f43049e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bd4cb90a0 con 0x561bd4d61360 2024-04-16T05:44:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d57e60 0x561bd4d58240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d57e60 0x561bd4d58240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48366/0 (socket says 172.21.15.86:48366) 2024-04-16T05:44:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.732+0000 7f42fbfff640 1 -- 172.21.15.86:0/260104022 learned_addr learned my addr 172.21.15.86:0/260104022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42faffd640 1 --2- 172.21.15.86:0/260104022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 0x561bd4d63750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/260104022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 0x561bd4d60e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42fbfff640 1 -- 172.21.15.86:0/260104022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 msgr2=0x561bd4d63750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42fbfff640 1 --2- 172.21.15.86:0/260104022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 0x561bd4d63750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42fbfff640 1 -- 172.21.15.86:0/260104022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 msgr2=0x561bd4d60e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42fbfff640 1 --2- 172.21.15.86:0/260104022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 0x561bd4d60e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42fbfff640 1 -- 172.21.15.86:0/260104022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd4c828d0 con 0x561bd4d57e60 2024-04-16T05:44:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42f8ff9640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f42e40a3d20 0x7f42e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42fbfff640 1 --2- 172.21.15.86:0/260104022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d57e60 0x561bd4d58240 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f42f4006010 tx=0x7f42f4005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be7fface8e725f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f42e40a3d20 0x7f42e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42f8ff9640 1 -- 172.21.15.86:0/209845139 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f42f00d0dc0 con 0x7f42fc106210 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42e1ffb640 1 -- 172.21.15.86:0/260104022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f402ca80 con 0x561bd4d57e60 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42e1ffb640 1 -- 172.21.15.86:0/260104022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f402cc20 con 0x561bd4d57e60 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.733+0000 7f42e1ffb640 1 -- 172.21.15.86:0/260104022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4004920 con 0x561bd4d57e60 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 -- 172.21.15.86:0/260104022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d57e60 msgr2=0x561bd4d58240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f42e40a3d20 0x7f42e40a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f42ec005c40 tx=0x7f42ec005bd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 --2- 172.21.15.86:0/260104022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d57e60 0x561bd4d58240 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f42f4006010 tx=0x7f42f4005e60 comp rx=0 tx=0).stop 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 -- 172.21.15.86:0/260104022 shutdown_connections 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 --2- 172.21.15.86:0/260104022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 0x561bd4d63750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 --2- 172.21.15.86:0/260104022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 0x561bd4d60e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 --2- 172.21.15.86:0/260104022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d57e60 0x561bd4d58240 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 -- 172.21.15.86:0/260104022 >> 172.21.15.86:0/260104022 conn(0x561bd4d52590 msgr2=0x561bd4d52970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 -- 172.21.15.86:0/260104022 shutdown_connections 2024-04-16T05:44:05.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 -- 172.21.15.86:0/260104022 wait complete. 2024-04-16T05:44:05.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.734+0000 7f43049e9740 1 -- start start 2024-04-16T05:44:05.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f43049e9740 1 Processor -- start 2024-04-16T05:44:05.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f43049e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d57e60 0x561bd4d5e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f43049e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d5e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f43049e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 0x561bd4d5eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f43049e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bd4d63fa0 con 0x561bd4d58780 2024-04-16T05:44:05.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f43049e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561bd4c764d0 con 0x561bd4d57e60 2024-04-16T05:44:05.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f43049e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bd4d64120 con 0x561bd4d61360 2024-04-16T05:44:05.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f43049e9740 10 client.0 _ll_register_callbacks cb 0x561bd4c5b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42faffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d5e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42faffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d5e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48382/0 (socket says 172.21.15.86:48382) 2024-04-16T05:44:05.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42faffd640 1 -- 172.21.15.86:0/1699692066 learned_addr learned my addr 172.21.15.86:0/1699692066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d57e60 0x561bd4d5e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42fbfff640 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 0x561bd4d5eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42faffd640 1 -- 172.21.15.86:0/1699692066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 msgr2=0x561bd4d5eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42faffd640 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 0x561bd4d5eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42faffd640 1 -- 172.21.15.86:0/1699692066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d57e60 msgr2=0x561bd4d5e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42faffd640 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d57e60 0x561bd4d5e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42faffd640 1 -- 172.21.15.86:0/1699692066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd49cf3f0 con 0x561bd4d58780 2024-04-16T05:44:05.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d57e60 0x561bd4d5e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:05.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42faffd640 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d5e970 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f42f00ce1a0 tx=0x7f42f000a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19858a7eb679618b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42daffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42daffd640 1 -- 172.21.15.86:0/1699692066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f009c810 con 0x561bd4d58780 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.735+0000 7f42daffd640 1 -- 172.21.15.86:0/1699692066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f42f00990b0 con 0x561bd4d58780 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f43049e9740 1 -- 172.21.15.86:0/1699692066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f402cc20 con 0x561bd4d58780 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f42daffd640 1 -- 172.21.15.86:0/1699692066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42f009c3a0 con 0x561bd4d58780 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f43049e9740 10 client.18264 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f43049e9740 1 -- 172.21.15.86:0/1699692066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bd4a997d0 con 0x561bd4d58780 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f42daffd640 1 -- 172.21.15.86:0/1699692066 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f402cc20 con 0x561bd4d58780 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f42daffd640 1 -- 172.21.15.86:0/1699692066 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f42f0020030 con 0x561bd4d58780 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f43049e9740 10 client.18264 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f43049e9740 10 client.18264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:05.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f43049e9740 20 client.18264 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f43049e9740 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561bd4d71320 0x561bd4d73710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f43049e9740 4 client.18264 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:05.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f43049e9740 1 -- 172.21.15.86:0/1699692066 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd4ccab00 con 0x561bd4d71320 2024-04-16T05:44:05.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.736+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561bd4d71320 0x561bd4d73710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.737+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561bd4d71320 0x561bd4d73710 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6333e6163bbf725c server_cookie=b993c2154109e643 in_seq=0 out_seq=0 2024-04-16T05:44:05.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.737+0000 7f42daffd640 10 client.18264 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:05.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.737+0000 7f42daffd640 1 -- 172.21.15.86:0/1699692066 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561bd4ccab00 con 0x561bd4d71320 2024-04-16T05:44:05.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.737+0000 7f42daffd640 10 client.18264 handle_command_reply: tid=0 2024-04-16T05:44:05.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.737+0000 7f43049e9740 1 client.18264 shutdown 2024-04-16T05:44:05.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 -- 172.21.15.86:0/1699692066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 msgr2=0x561bd4d5e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d5e970 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f42f00ce1a0 tx=0x7f42f000a540 comp rx=0 tx=0).stop 2024-04-16T05:44:05.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 -- 172.21.15.86:0/1699692066 shutdown_connections 2024-04-16T05:44:05.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561bd4d71320 0x561bd4d73710 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d61360 0x561bd4d5eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d5e970 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f42daffd640 0 client.18264 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:05.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 --2- 172.21.15.86:0/1699692066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d57e60 0x561bd4d5e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 -- 172.21.15.86:0/1699692066 >> 172.21.15.86:0/1699692066 conn(0x561bd4d52590 msgr2=0x561bd4d542e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 -- 172.21.15.86:0/1699692066 shutdown_connections 2024-04-16T05:44:05.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 1 -- 172.21.15.86:0/1699692066 wait complete. 2024-04-16T05:44:05.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.738+0000 7f43049e9740 20 client.18264 trim_cache size 0 max 16384 2024-04-16T05:44:05.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f43049e9740 1 Processor -- start 2024-04-16T05:44:05.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f43049e9740 1 -- start start 2024-04-16T05:44:05.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f43049e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 0x561bd4d60b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f43049e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d6cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f43049e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d61360 0x561bd4d4d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f43049e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1af3f0 con 0x561bd4d58780 2024-04-16T05:44:05.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f43049e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b0f30 con 0x561bd4d61360 2024-04-16T05:44:05.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f43049e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1afac0 con 0x561bd4d57e60 2024-04-16T05:44:05.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f42faffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d6cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f42faffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d6cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48398/0 (socket says 172.21.15.86:48398) 2024-04-16T05:44:05.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f42faffd640 1 -- 172.21.15.86:0/3926012112 learned_addr learned my addr 172.21.15.86:0/3926012112 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f42faffd640 1 -- 172.21.15.86:0/3926012112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 msgr2=0x561bd4d60b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f42faffd640 1 --2- 172.21.15.86:0/3926012112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 0x561bd4d60b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f42faffd640 1 -- 172.21.15.86:0/3926012112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d61360 msgr2=0x561bd4d4d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f42faffd640 1 --2- 172.21.15.86:0/3926012112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d61360 0x561bd4d4d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.751+0000 7f42faffd640 1 -- 172.21.15.86:0/3926012112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd4d5d3e0 con 0x561bd4d58780 2024-04-16T05:44:05.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f42faffd640 1 --2- 172.21.15.86:0/3926012112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d6cf90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f42f009c9d0 tx=0x7f42f000fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed85ac4a0ff7a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f42e17fa640 1 -- 172.21.15.86:0/3926012112 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f009c580 con 0x561bd4d58780 2024-04-16T05:44:05.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f42e17fa640 1 -- 172.21.15.86:0/3926012112 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f0087430 con 0x561bd4d58780 2024-04-16T05:44:05.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f42e17fa640 1 -- 172.21.15.86:0/3926012112 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f000f740 con 0x561bd4d58780 2024-04-16T05:44:05.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f43049e9740 1 -- 172.21.15.86:0/3926012112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 msgr2=0x561bd4d6cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f43049e9740 1 --2- 172.21.15.86:0/3926012112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d6cf90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f42f009c9d0 tx=0x7f42f000fe00 comp rx=0 tx=0).stop 2024-04-16T05:44:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f43049e9740 1 -- 172.21.15.86:0/3926012112 shutdown_connections 2024-04-16T05:44:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f43049e9740 1 --2- 172.21.15.86:0/3926012112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d61360 0x561bd4d4d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f43049e9740 1 --2- 172.21.15.86:0/3926012112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d58780 0x561bd4d6cf90 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f43049e9740 1 --2- 172.21.15.86:0/3926012112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 0x561bd4d60b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f43049e9740 1 -- 172.21.15.86:0/3926012112 >> 172.21.15.86:0/3926012112 conn(0x561bd4d52590 msgr2=0x561bd4d54250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f43049e9740 1 -- 172.21.15.86:0/3926012112 shutdown_connections 2024-04-16T05:44:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.752+0000 7f43049e9740 1 -- 172.21.15.86:0/3926012112 wait complete. 2024-04-16T05:44:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f43049e9740 1 -- start start 2024-04-16T05:44:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f43049e9740 1 Processor -- start 2024-04-16T05:44:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f43049e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 0x561bd4dd0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f43049e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 0x561bd4dd1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f43049e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d61360 0x561bd4d656b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f43049e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b0860 con 0x561bd4d61360 2024-04-16T05:44:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f43049e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b0190 con 0x561bd4d58780 2024-04-16T05:44:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f43049e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1ad3f0 con 0x561bd4d57e60 2024-04-16T05:44:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f43049e9740 10 client.0 _ll_register_callbacks cb 0x7f42f0021f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d61360 0x561bd4d656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d61360 0x561bd4d656b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48410/0 (socket says 172.21.15.86:48410) 2024-04-16T05:44:05.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fbfff640 1 -- 172.21.15.86:0/1063612241 learned_addr learned my addr 172.21.15.86:0/1063612241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:05.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 0x561bd4dd0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42faffd640 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 0x561bd4dd1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fbfff640 1 -- 172.21.15.86:0/1063612241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 msgr2=0x561bd4dd0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fbfff640 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 0x561bd4dd0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fbfff640 1 -- 172.21.15.86:0/1063612241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 msgr2=0x561bd4dd1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:05.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fbfff640 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 0x561bd4dd1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fbfff640 1 -- 172.21.15.86:0/1063612241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd4d585b0 con 0x561bd4d61360 2024-04-16T05:44:05.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.753+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 0x561bd4dd0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f42fbfff640 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d61360 0x561bd4d656b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f42f402c170 tx=0x7f42f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b8bb5325455bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f42daffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f42daffd640 1 -- 172.21.15.86:0/1063612241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f404d060 con 0x561bd4d61360 2024-04-16T05:44:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f42daffd640 1 -- 172.21.15.86:0/1063612241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f42f4005240 con 0x561bd4d61360 2024-04-16T05:44:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f43049e9740 1 -- 172.21.15.86:0/1063612241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f0087430 con 0x561bd4d61360 2024-04-16T05:44:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f42daffd640 1 -- 172.21.15.86:0/1063612241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42f4054050 con 0x561bd4d61360 2024-04-16T05:44:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f43049e9740 10 client.18270 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f43049e9740 1 -- 172.21.15.86:0/1063612241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bd4dd8140 con 0x561bd4d61360 2024-04-16T05:44:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f42daffd640 1 -- 172.21.15.86:0/1063612241 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f0087430 con 0x561bd4d61360 2024-04-16T05:44:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f42daffd640 1 -- 172.21.15.86:0/1063612241 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f42f40035f0 con 0x561bd4d61360 2024-04-16T05:44:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f43049e9740 10 client.18270 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f43049e9740 10 client.18270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f43049e9740 20 client.18270 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f43049e9740 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561bd4dd84d0 0x561bd4dda8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f43049e9740 4 client.18270 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.754+0000 7f43049e9740 1 -- 172.21.15.86:0/1063612241 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bd4cc4970 con 0x561bd4dd84d0 2024-04-16T05:44:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561bd4dd84d0 0x561bd4dda8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f42daffd640 10 client.18270 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f42fb7fe640 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561bd4dd84d0 0x561bd4dda8c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c0f477d0e015838 server_cookie=b11c47052a86d94e in_seq=0 out_seq=0 2024-04-16T05:44:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f42daffd640 1 -- 172.21.15.86:0/1063612241 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bd4cc4970 con 0x561bd4dd84d0 2024-04-16T05:44:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f42daffd640 10 client.18270 handle_command_reply: tid=0 2024-04-16T05:44:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 client.18270 shutdown 2024-04-16T05:44:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 -- 172.21.15.86:0/1063612241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d61360 msgr2=0x561bd4d656b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d61360 0x561bd4d656b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f42f402c170 tx=0x7f42f4004270 comp rx=0 tx=0).stop 2024-04-16T05:44:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 -- 172.21.15.86:0/1063612241 shutdown_connections 2024-04-16T05:44:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x561bd4dd84d0 0x561bd4dda8c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd4d61360 0x561bd4d656b0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bd4d58780 0x561bd4dd1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f42daffd640 0 client.18270 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 --2- 172.21.15.86:0/1063612241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd4d57e60 0x561bd4dd0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 -- 172.21.15.86:0/1063612241 >> 172.21.15.86:0/1063612241 conn(0x561bd4d52590 msgr2=0x561bd4d5e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 -- 172.21.15.86:0/1063612241 shutdown_connections 2024-04-16T05:44:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.755+0000 7f43049e9740 1 -- 172.21.15.86:0/1063612241 wait complete. 2024-04-16T05:44:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f43049e9740 20 client.18270 trim_cache size 0 max 16384 2024-04-16T05:44:05.765 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 -- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f42e40a3d20 msgr2=0x7f42e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f42e40a3d20 0x7f42e40a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f42ec005c40 tx=0x7f42ec005bd0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 -- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 msgr2=0x7f42fc199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 0x7f42fc199bb0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f42f000c970 tx=0x7f42f000ce30 comp rx=0 tx=0).stop 2024-04-16T05:44:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 -- 172.21.15.86:0/209845139 shutdown_connections 2024-04-16T05:44:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42fc19a140 0x7f42fc1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106210 0x7f42fc199bb0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f42e40a3d20 0x7f42e40a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 --2- 172.21.15.86:0/209845139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42fc101140 0x7f42fc199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 -- 172.21.15.86:0/209845139 >> 172.21.15.86:0/209845139 conn(0x7f42fc077280 msgr2=0x7f42fc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 -- 172.21.15.86:0/209845139 shutdown_connections 2024-04-16T05:44:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:05.756+0000 7f4301d1c640 1 -- 172.21.15.86:0/209845139 wait complete. 2024-04-16T05:44:06.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:06 smithi196 ceph-mon[36373]: pgmap v271: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:44:06.557 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:06 smithi194 ceph-mon[35865]: pgmap v271: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:44:06.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:06 smithi086 ceph-mon[32028]: pgmap v271: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:44:06.768 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:44:06.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:44:06.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:44:06.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.890+0000 7f8b3c406640 1 -- 172.21.15.86:0/4218211187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 msgr2=0x7f8b3410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:06.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.890+0000 7f8b3c406640 1 --2- 172.21.15.86:0/4218211187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 0x7f8b3410dd30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009cd0 tx=0x7f8b2c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.890+0000 7f8b3c406640 1 -- 172.21.15.86:0/4218211187 shutdown_connections 2024-04-16T05:44:06.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.890+0000 7f8b3c406640 1 --2- 172.21.15.86:0/4218211187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 0x7f8b3410dd30 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.890+0000 7f8b3c406640 1 --2- 172.21.15.86:0/4218211187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b34101250 0x7f8b341016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.890+0000 7f8b3c406640 1 --2- 172.21.15.86:0/4218211187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b341070e0 0x7f8b341074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.890+0000 7f8b3c406640 1 -- 172.21.15.86:0/4218211187 >> 172.21.15.86:0/4218211187 conn(0x7f8b34076d00 msgr2=0x7f8b34077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:06.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.890+0000 7f8b3c406640 1 -- 172.21.15.86:0/4218211187 shutdown_connections 2024-04-16T05:44:06.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.890+0000 7f8b3c406640 1 -- 172.21.15.86:0/4218211187 wait complete. 2024-04-16T05:44:06.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3c406640 1 Processor -- start 2024-04-16T05:44:06.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3c406640 1 -- start start 2024-04-16T05:44:06.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3c406640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34101250 0x7f8b3419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3c406640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 0x7f8b3419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3c406640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b341070e0 0x7f8b34198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3c406640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b341112e0 con 0x7f8b34101bf0 2024-04-16T05:44:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3c406640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b34111160 con 0x7f8b341070e0 2024-04-16T05:44:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3c406640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b34111460 con 0x7f8b34101250 2024-04-16T05:44:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b399a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 0x7f8b3419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b399a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 0x7f8b3419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48424/0 (socket says 172.21.15.86:48424) 2024-04-16T05:44:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b399a1640 1 -- 172.21.15.86:0/3711216852 learned_addr learned my addr 172.21.15.86:0/3711216852 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b341070e0 0x7f8b34198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34101250 0x7f8b3419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b399a1640 1 -- 172.21.15.86:0/3711216852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34101250 msgr2=0x7f8b3419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34101250 0x7f8b3419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b399a1640 1 -- 172.21.15.86:0/3711216852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b341070e0 msgr2=0x7f8b34198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.891+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b341070e0 0x7f8b34198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.892+0000 7f8b399a1640 1 -- 172.21.15.86:0/3711216852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b34198f30 con 0x7f8b34101bf0 2024-04-16T05:44:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.892+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b341070e0 0x7f8b34198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.892+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 0x7f8b3419eaf0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c920 tx=0x7f8b3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.892+0000 7f8b277fe640 1 -- 172.21.15.86:0/3711216852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30007c70 con 0x7f8b34101bf0 2024-04-16T05:44:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.892+0000 7f8b277fe640 1 -- 172.21.15.86:0/3711216852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b30007e10 con 0x7f8b34101bf0 2024-04-16T05:44:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.892+0000 7f8b3c406640 1 -- 172.21.15.86:0/3711216852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b34199220 con 0x7f8b34101bf0 2024-04-16T05:44:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.892+0000 7f8b277fe640 1 -- 172.21.15.86:0/3711216852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30018890 con 0x7f8b34101bf0 2024-04-16T05:44:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.892+0000 7f8b3c406640 1 -- 172.21.15.86:0/3711216852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b341a5530 con 0x7f8b34101bf0 2024-04-16T05:44:06.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.894+0000 7f8b277fe640 1 -- 172.21.15.86:0/3711216852 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b30018a70 con 0x7f8b34101bf0 2024-04-16T05:44:06.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b3eff2740 1 Processor -- start 2024-04-16T05:44:06.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b3eff2740 1 -- start start 2024-04-16T05:44:06.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 0x55ae25df6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b277fe640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b200a3d20 0x7f8b200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25dfffb0 0x55ae25e023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae25d5be90 con 0x55ae25df73d0 2024-04-16T05:44:06.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ae25cf94c0 con 0x55ae25dfffb0 2024-04-16T05:44:06.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae25d57e50 con 0x55ae25df6ab0 2024-04-16T05:44:06.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b399a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b200a3d20 0x7f8b200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b399a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dffa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48438/0 (socket says 172.21.15.86:48438) 2024-04-16T05:44:06.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b399a1640 1 -- 172.21.15.86:0/636765609 learned_addr learned my addr 172.21.15.86:0/636765609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:06.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.895+0000 7f8b277fe640 1 -- 172.21.15.86:0/3711216852 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f8b300d0e00 con 0x7f8b34101bf0 2024-04-16T05:44:06.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25dfffb0 0x55ae25e023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 0x55ae25df6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b399a1640 1 -- 172.21.15.86:0/636765609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 msgr2=0x55ae25df6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:06.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b399a1640 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 0x55ae25df6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b399a1640 1 -- 172.21.15.86:0/636765609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25dfffb0 msgr2=0x55ae25e023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:06.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b399a1640 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25dfffb0 0x55ae25e023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b399a1640 1 -- 172.21.15.86:0/636765609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae25b367e0 con 0x55ae25df73d0 2024-04-16T05:44:06.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25dfffb0 0x55ae25e023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b399a1640 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dffa70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8b3009c960 tx=0x7f8b30013720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83798f2bb568f77a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b200a3d20 0x7f8b200a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8b280058c0 tx=0x7f8b28005c60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b24bfe640 1 -- 172.21.15.86:0/636765609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3009af90 con 0x55ae25df73d0 2024-04-16T05:44:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b24bfe640 1 -- 172.21.15.86:0/636765609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3009d0d0 con 0x55ae25df73d0 2024-04-16T05:44:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.896+0000 7f8b24bfe640 1 -- 172.21.15.86:0/636765609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3009a020 con 0x55ae25df73d0 2024-04-16T05:44:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 -- 172.21.15.86:0/636765609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 msgr2=0x55ae25dffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dffa70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8b3009c960 tx=0x7f8b30013720 comp rx=0 tx=0).stop 2024-04-16T05:44:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 -- 172.21.15.86:0/636765609 shutdown_connections 2024-04-16T05:44:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25dfffb0 0x55ae25e023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dffa70 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/636765609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 0x55ae25df6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 -- 172.21.15.86:0/636765609 >> 172.21.15.86:0/636765609 conn(0x55ae25df11e0 msgr2=0x55ae25df15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:06.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 -- 172.21.15.86:0/636765609 shutdown_connections 2024-04-16T05:44:06.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 -- 172.21.15.86:0/636765609 wait complete. 2024-04-16T05:44:06.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 -- start start 2024-04-16T05:44:06.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.897+0000 7f8b3eff2740 1 Processor -- start 2024-04-16T05:44:06.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df6ab0 0x55ae25dfd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dfd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 0x55ae25dfdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae25e02da0 con 0x55ae25df73d0 2024-04-16T05:44:06.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ae25ce4410 con 0x55ae25df6ab0 2024-04-16T05:44:06.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae25e02f20 con 0x55ae25dfffb0 2024-04-16T05:44:06.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 10 client.0 _ll_register_callbacks cb 0x55ae25cdfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:06.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b399a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dfd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b399a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dfd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48448/0 (socket says 172.21.15.86:48448) 2024-04-16T05:44:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b399a1640 1 -- 172.21.15.86:0/3425469052 learned_addr learned my addr 172.21.15.86:0/3425469052 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df6ab0 0x55ae25dfd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 0x55ae25dfdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b399a1640 1 -- 172.21.15.86:0/3425469052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 msgr2=0x55ae25dfdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 0x55ae25dfdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b399a1640 1 -- 172.21.15.86:0/3425469052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df6ab0 msgr2=0x55ae25dfd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df6ab0 0x55ae25dfd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b399a1640 1 -- 172.21.15.86:0/3425469052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae25b96f70 con 0x55ae25df73d0 2024-04-16T05:44:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 0x55ae25dfdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:06.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dfd610 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8b300ce230 tx=0x7f8b3009aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c98472b50881dd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:06.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:06.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3425469052 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3009a200 con 0x55ae25df73d0 2024-04-16T05:44:06.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3425469052 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f8b3009cc30 con 0x55ae25df73d0 2024-04-16T05:44:06.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3425469052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b3009d0d0 con 0x55ae25df73d0 2024-04-16T05:44:06.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3425469052 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b3009a8b0 con 0x55ae25df73d0 2024-04-16T05:44:06.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 10 client.18282 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:06.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.898+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3425469052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae25a2e170 con 0x55ae25df73d0 2024-04-16T05:44:06.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3425469052 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3009d0d0 con 0x55ae25df73d0 2024-04-16T05:44:06.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3425469052 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b300672a0 con 0x55ae25df73d0 2024-04-16T05:44:06.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b3eff2740 10 client.18282 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:06.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b3eff2740 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:06.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b3eff2740 20 client.18282 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:06.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ae25e102b0 0x55ae25e126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b3eff2740 4 client.18282 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:06.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3425469052 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae25d698b0 con 0x55ae25e102b0 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ae25e102b0 0x55ae25e126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ae25e102b0 0x55ae25e126a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8800a9fda73032d3 server_cookie=cdf49d7a354e5703 in_seq=0 out_seq=0 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.899+0000 7f8b1dffb640 10 client.18282 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3425469052 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae25d698b0 con 0x55ae25e102b0 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b1dffb640 10 client.18282 handle_command_reply: tid=0 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 client.18282 shutdown 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3425469052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 msgr2=0x55ae25dfd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dfd610 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8b300ce230 tx=0x7f8b3009aee0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3425469052 shutdown_connections 2024-04-16T05:44:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ae25e102b0 0x55ae25e126a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 0x55ae25dfdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df73d0 0x55ae25dfd610 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3425469052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df6ab0 0x55ae25dfd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b1dffb640 0 client.18282 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3425469052 >> 172.21.15.86:0/3425469052 conn(0x55ae25df11e0 msgr2=0x55ae25df2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3425469052 shutdown_connections 2024-04-16T05:44:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3425469052 wait complete. 2024-04-16T05:44:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.900+0000 7f8b3eff2740 20 client.18282 trim_cache size 0 max 16384 2024-04-16T05:44:06.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3eff2740 1 Processor -- start 2024-04-16T05:44:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3eff2740 1 -- start start 2024-04-16T05:44:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 0x55ae25dff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 0x55ae25dfd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25dfffb0 0x55ae25e156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b341af4f0 con 0x55ae25dfffb0 2024-04-16T05:44:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b341b1030 con 0x55ae25df73d0 2024-04-16T05:44:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b341afbc0 con 0x55ae25df6ab0 2024-04-16T05:44:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a9a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25dfffb0 0x55ae25e156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a9a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25dfffb0 0x55ae25e156c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48450/0 (socket says 172.21.15.86:48450) 2024-04-16T05:44:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a9a3640 1 -- 172.21.15.86:0/3089711563 learned_addr learned my addr 172.21.15.86:0/3089711563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 0x55ae25dff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 0x55ae25dfd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a9a3640 1 -- 172.21.15.86:0/3089711563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 msgr2=0x55ae25dff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 0x55ae25dff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a9a3640 1 -- 172.21.15.86:0/3089711563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 msgr2=0x55ae25dfd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 0x55ae25dfd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a9a3640 1 -- 172.21.15.86:0/3089711563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae25df7200 con 0x55ae25dfffb0 2024-04-16T05:44:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 0x55ae25dff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.918+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25dfffb0 0x55ae25e156c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009cd0 tx=0x7f8b2c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74fa87f00431f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b1ffff640 1 -- 172.21.15.86:0/3089711563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c039040 con 0x55ae25dfffb0 2024-04-16T05:44:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b1ffff640 1 -- 172.21.15.86:0/3089711563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c003970 con 0x55ae25dfffb0 2024-04-16T05:44:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b1ffff640 1 -- 172.21.15.86:0/3089711563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c004560 con 0x55ae25dfffb0 2024-04-16T05:44:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3089711563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25dfffb0 msgr2=0x55ae25e156c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25dfffb0 0x55ae25e156c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009cd0 tx=0x7f8b2c0098e0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3089711563 shutdown_connections 2024-04-16T05:44:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25dfffb0 0x55ae25e156c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 0x55ae25dfd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3089711563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25df6ab0 0x55ae25dff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3089711563 >> 172.21.15.86:0/3089711563 conn(0x55ae25df11e0 msgr2=0x55ae25dfc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3089711563 shutdown_connections 2024-04-16T05:44:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3089711563 wait complete. 2024-04-16T05:44:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.919+0000 7f8b3eff2740 1 -- start start 2024-04-16T05:44:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3eff2740 1 Processor -- start 2024-04-16T05:44:06.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df6ab0 0x55ae25e134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 0x55ae25e139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3eff2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 0x55ae25e75490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b341b0960 con 0x55ae25df6ab0 2024-04-16T05:44:06.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b341b0290 con 0x55ae25df73d0 2024-04-16T05:44:06.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3eff2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b341ad4f0 con 0x55ae25dfffb0 2024-04-16T05:44:06.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3eff2740 10 client.0 _ll_register_callbacks cb 0x7f8b30098d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:06.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a1a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df6ab0 0x55ae25e134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a1a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df6ab0 0x55ae25e134b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48460/0 (socket says 172.21.15.86:48460) 2024-04-16T05:44:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a1a2640 1 -- 172.21.15.86:0/3781927981 learned_addr learned my addr 172.21.15.86:0/3781927981 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 0x55ae25e139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a9a3640 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 0x55ae25e75490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a1a2640 1 -- 172.21.15.86:0/3781927981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 msgr2=0x55ae25e75490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 0x55ae25e75490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a1a2640 1 -- 172.21.15.86:0/3781927981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 msgr2=0x55ae25e139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 0x55ae25e139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a1a2640 1 -- 172.21.15.86:0/3781927981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae25e759d0 con 0x55ae25df6ab0 2024-04-16T05:44:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b3a1a2640 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df6ab0 0x55ae25e134b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800bfd0 tx=0x7f8b28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87171871de317b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.920+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3781927981 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b280267d0 con 0x55ae25df6ab0 2024-04-16T05:44:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3781927981 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f8b2800f420 con 0x55ae25df6ab0 2024-04-16T05:44:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3781927981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b2c003970 con 0x55ae25df6ab0 2024-04-16T05:44:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3781927981 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b28037050 con 0x55ae25df6ab0 2024-04-16T05:44:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b3eff2740 10 client.18288 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3781927981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae25e76bb0 con 0x55ae25df6ab0 2024-04-16T05:44:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3781927981 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c003970 con 0x55ae25df6ab0 2024-04-16T05:44:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3781927981 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b2802e650 con 0x55ae25df6ab0 2024-04-16T05:44:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b3eff2740 10 client.18288 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b3eff2740 10 client.18288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b3eff2740 20 client.18288 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ae25e772e0 0x55ae25e79720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b3eff2740 4 client.18288 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:44:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3781927981 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ae25d63720 con 0x55ae25e772e0 2024-04-16T05:44:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.921+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ae25e772e0 0x55ae25e79720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:06.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.922+0000 7f8b1dffb640 10 client.18288 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:06.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.922+0000 7f8b399a1640 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ae25e772e0 0x55ae25e79720 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72d395d176ff4f7 server_cookie=3f92d49c833e90e5 in_seq=0 out_seq=0 2024-04-16T05:44:06.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b1dffb640 1 -- 172.21.15.86:0/3781927981 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ae25d63720 con 0x55ae25e772e0 2024-04-16T05:44:06.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b1dffb640 10 client.18288 handle_command_reply: tid=0 2024-04-16T05:44:06.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 client.18288 shutdown 2024-04-16T05:44:06.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3781927981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df6ab0 msgr2=0x55ae25e134b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:06.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df6ab0 0x55ae25e134b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800bfd0 tx=0x7f8b28004730 comp rx=0 tx=0).stop 2024-04-16T05:44:06.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3781927981 shutdown_connections 2024-04-16T05:44:06.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55ae25e772e0 0x55ae25e79720 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae25dfffb0 0x55ae25e75490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b1dffb640 0 client.18288 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae25df73d0 0x55ae25e139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 --2- 172.21.15.86:0/3781927981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae25df6ab0 0x55ae25e134b0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3781927981 >> 172.21.15.86:0/3781927981 conn(0x55ae25df11e0 msgr2=0x55ae25dfb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3781927981 shutdown_connections 2024-04-16T05:44:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 1 -- 172.21.15.86:0/3781927981 wait complete. 2024-04-16T05:44:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.923+0000 7f8b3eff2740 20 client.18288 trim_cache size 0 max 16384 2024-04-16T05:44:06.932 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:44:06.932 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:44:06.932 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "96ac9bef-e7ea-4c92-96c6-e495c0b2d103", 2024-04-16T05:44:06.932 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:44:06.932 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:44:06.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 -- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b200a3d20 msgr2=0x7f8b200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:06.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b200a3d20 0x7f8b200a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8b280058c0 tx=0x7f8b28005c60 comp rx=0 tx=0).stop 2024-04-16T05:44:06.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 -- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 msgr2=0x7f8b3419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:06.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 0x7f8b3419eaf0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c920 tx=0x7f8b3000cde0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 -- 172.21.15.86:0/3711216852 shutdown_connections 2024-04-16T05:44:06.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b200a3d20 0x7f8b200a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b341070e0 0x7f8b34198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b34101bf0 0x7f8b3419eaf0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 --2- 172.21.15.86:0/3711216852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34101250 0x7f8b3419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:06.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 -- 172.21.15.86:0/3711216852 >> 172.21.15.86:0/3711216852 conn(0x7f8b34076d00 msgr2=0x7f8b34100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:06.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 -- 172.21.15.86:0/3711216852 shutdown_connections 2024-04-16T05:44:06.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:06.924+0000 7f8b3c406640 1 -- 172.21.15.86:0/3711216852 wait complete. 2024-04-16T05:44:06.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:44:06.935 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:44:07.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.043+0000 7fb7808f2640 1 -- 172.21.15.86:0/233263942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c101250 msgr2=0x7fb77c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:07.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.043+0000 7fb7808f2640 1 --2- 172.21.15.86:0/233263942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c101250 0x7fb77c1016b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb770009930 tx=0x7fb77002bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:07.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.043+0000 7fb7808f2640 1 -- 172.21.15.86:0/233263942 shutdown_connections 2024-04-16T05:44:07.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.043+0000 7fb7808f2640 1 --2- 172.21.15.86:0/233263942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb77c101bf0 0x7fb77c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.043+0000 7fb7808f2640 1 --2- 172.21.15.86:0/233263942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c101250 0x7fb77c1016b0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.043+0000 7fb7808f2640 1 --2- 172.21.15.86:0/233263942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb77c1070e0 0x7fb77c1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.043+0000 7fb7808f2640 1 -- 172.21.15.86:0/233263942 >> 172.21.15.86:0/233263942 conn(0x7fb77c076d00 msgr2=0x7fb77c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:07.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.043+0000 7fb7808f2640 1 -- 172.21.15.86:0/233263942 shutdown_connections 2024-04-16T05:44:07.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.043+0000 7fb7808f2640 1 -- 172.21.15.86:0/233263942 wait complete. 2024-04-16T05:44:07.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb7808f2640 1 Processor -- start 2024-04-16T05:44:07.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb7808f2640 1 -- start start 2024-04-16T05:44:07.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb7808f2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb77c101250 0x7fb77c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb7808f2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb77c101bf0 0x7fb77c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb7808f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c1070e0 0x7fb77c1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb7808f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1112e0 con 0x7fb77c1070e0 2024-04-16T05:44:07.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb7808f2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c111160 con 0x7fb77c101bf0 2024-04-16T05:44:07.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb7808f2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c111460 con 0x7fb77c101250 2024-04-16T05:44:07.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb77ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c1070e0 0x7fb77c1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb77ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c1070e0 0x7fb77c1a47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48478/0 (socket says 172.21.15.86:48478) 2024-04-16T05:44:07.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb77ad9d640 1 -- 172.21.15.86:0/3307517593 learned_addr learned my addr 172.21.15.86:0/3307517593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:07.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb779d9b640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb77c101bf0 0x7fb77c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb77ad9d640 1 -- 172.21.15.86:0/3307517593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb77c101250 msgr2=0x7fb77c199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:07.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb77ad9d640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb77c101250 0x7fb77c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb77ad9d640 1 -- 172.21.15.86:0/3307517593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb77c101bf0 msgr2=0x7fb77c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:07.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.044+0000 7fb77ad9d640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb77c101bf0 0x7fb77c199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.045+0000 7fb77ad9d640 1 -- 172.21.15.86:0/3307517593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c1a4d00 con 0x7fb77c1070e0 2024-04-16T05:44:07.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.045+0000 7fb77ad9d640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c1070e0 0x7fb77c1a47c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb77400cce0 tx=0x7fb77400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:07.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.045+0000 7fb76b7fe640 1 -- 172.21.15.86:0/3307517593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77400f450 con 0x7fb77c1070e0 2024-04-16T05:44:07.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.045+0000 7fb7808f2640 1 -- 172.21.15.86:0/3307517593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c1a4ed0 con 0x7fb77c1070e0 2024-04-16T05:44:07.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.045+0000 7fb76b7fe640 1 -- 172.21.15.86:0/3307517593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77400fad0 con 0x7fb77c1070e0 2024-04-16T05:44:07.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.045+0000 7fb76b7fe640 1 -- 172.21.15.86:0/3307517593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774018c10 con 0x7fb77c1070e0 2024-04-16T05:44:07.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.045+0000 7fb7808f2640 1 -- 172.21.15.86:0/3307517593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c1a5340 con 0x7fb77c1070e0 2024-04-16T05:44:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.047+0000 7fb76b7fe640 1 -- 172.21.15.86:0/3307517593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb77400f5f0 con 0x7fb77c1070e0 2024-04-16T05:44:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.047+0000 7fb7835b8740 1 Processor -- start 2024-04-16T05:44:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.047+0000 7fb7835b8740 1 -- start start 2024-04-16T05:44:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bd6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bdf350 0x55f2b4be1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb7835b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2b4b3b180 con 0x55f2b4bd6770 2024-04-16T05:44:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb7835b8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f2b4ad8360 con 0x55f2b4bdf350 2024-04-16T05:44:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb7835b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2b4b37140 con 0x55f2b4bd5e50 2024-04-16T05:44:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb76b7fe640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7640a3d20 0x7fb7640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48488/0 (socket says 172.21.15.86:48488) 2024-04-16T05:44:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77a59c640 1 -- 172.21.15.86:0/4086016811 learned_addr learned my addr 172.21.15.86:0/4086016811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77ad9d640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7640a3d20 0x7fb7640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb779d9b640 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bd6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb779d9b640 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bdf350 0x55f2b4be1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb76b7fe640 1 -- 172.21.15.86:0/3307517593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fb7740d1360 con 0x7fb77c1070e0 2024-04-16T05:44:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77a59c640 1 -- 172.21.15.86:0/4086016811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd5e50 msgr2=0x55f2b4bd6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77a59c640 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bd6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77a59c640 1 -- 172.21.15.86:0/4086016811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bdf350 msgr2=0x55f2b4be1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77a59c640 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bdf350 0x55f2b4be1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77a59c640 1 -- 172.21.15.86:0/4086016811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b49177d0 con 0x55f2b4bd6770 2024-04-16T05:44:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb779d9b640 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bdf350 0x55f2b4be1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77a59c640 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdee10 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00a960 tx=0x7fb76c00ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25513715d8aa964a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb77ad9d640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7640a3d20 0x7fb7640a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb774099390 tx=0x7fb77406e970 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.048+0000 7fb768ff9640 1 -- 172.21.15.86:0/4086016811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c00cd80 con 0x55f2b4bd6770 2024-04-16T05:44:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb768ff9640 1 -- 172.21.15.86:0/4086016811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76c00f930 con 0x55f2b4bd6770 2024-04-16T05:44:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb768ff9640 1 -- 172.21.15.86:0/4086016811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c018a80 con 0x55f2b4bd6770 2024-04-16T05:44:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb7835b8740 1 -- 172.21.15.86:0/4086016811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 msgr2=0x55f2b4bdee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb7835b8740 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdee10 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00a960 tx=0x7fb76c00ae20 comp rx=0 tx=0).stop 2024-04-16T05:44:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb7835b8740 1 -- 172.21.15.86:0/4086016811 shutdown_connections 2024-04-16T05:44:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb7835b8740 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bdf350 0x55f2b4be1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb7835b8740 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdee10 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb7835b8740 1 --2- 172.21.15.86:0/4086016811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bd6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb7835b8740 1 -- 172.21.15.86:0/4086016811 >> 172.21.15.86:0/4086016811 conn(0x55f2b4bd0580 msgr2=0x55f2b4bd0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb7835b8740 1 -- 172.21.15.86:0/4086016811 shutdown_connections 2024-04-16T05:44:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.049+0000 7fb7835b8740 1 -- 172.21.15.86:0/4086016811 wait complete. 2024-04-16T05:44:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb7835b8740 1 -- start start 2024-04-16T05:44:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb7835b8740 1 Processor -- start 2024-04-16T05:44:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bdc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bdf350 0x55f2b4bdce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb7835b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2b4be2140 con 0x55f2b4bdf350 2024-04-16T05:44:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb7835b8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f2b4af42c0 con 0x55f2b4bd5e50 2024-04-16T05:44:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb7835b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2b4be22c0 con 0x55f2b4bd6770 2024-04-16T05:44:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb7835b8740 10 client.0 _ll_register_callbacks cb 0x55f2b4adb090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb779d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bdf350 0x55f2b4bdce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb779d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bdf350 0x55f2b4bdce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48504/0 (socket says 172.21.15.86:48504) 2024-04-16T05:44:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb779d9b640 1 -- 172.21.15.86:0/3340866001 learned_addr learned my addr 172.21.15.86:0/3340866001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb779d9b640 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bdc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb77a59c640 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb779d9b640 1 -- 172.21.15.86:0/3340866001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd6770 msgr2=0x55f2b4bdc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb779d9b640 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb779d9b640 1 -- 172.21.15.86:0/3340866001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd5e50 msgr2=0x55f2b4bdc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb779d9b640 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bdc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb779d9b640 1 -- 172.21.15.86:0/3340866001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b484d3f0 con 0x55f2b4bdf350 2024-04-16T05:44:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.050+0000 7fb77a59c640 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb779d9b640 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bdf350 0x55f2b4bdce70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb770009900 tx=0x7fb770009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edff88a08ef260c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb761ffb640 1 -- 172.21.15.86:0/3340866001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7700044b0 con 0x55f2b4bdf350 2024-04-16T05:44:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb761ffb640 1 -- 172.21.15.86:0/3340866001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fb77003a040 con 0x55f2b4bdf350 2024-04-16T05:44:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb7835b8740 1 -- 172.21.15.86:0/3340866001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb76c00f930 con 0x55f2b4bdf350 2024-04-16T05:44:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb761ffb640 1 -- 172.21.15.86:0/3340866001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb770053050 con 0x55f2b4bdf350 2024-04-16T05:44:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb7835b8740 10 client.18300 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb7835b8740 1 -- 172.21.15.86:0/3340866001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2b480d0f0 con 0x55f2b4bdf350 2024-04-16T05:44:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb761ffb640 1 -- 172.21.15.86:0/3340866001 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76c00f930 con 0x55f2b4bdf350 2024-04-16T05:44:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb761ffb640 1 -- 172.21.15.86:0/3340866001 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7700426b0 con 0x55f2b4bdf350 2024-04-16T05:44:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb7835b8740 10 client.18300 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb7835b8740 10 client.18300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb7835b8740 20 client.18300 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb7835b8740 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f2b4bef730 0x55f2b4bf1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb7835b8740 4 client.18300 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.051+0000 7fb7835b8740 1 -- 172.21.15.86:0/3340866001 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b4b48ba0 con 0x55f2b4bef730 2024-04-16T05:44:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.052+0000 7fb77a59c640 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f2b4bef730 0x55f2b4bf1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.052+0000 7fb77a59c640 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f2b4bef730 0x55f2b4bf1bb0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2022c5e395514286 server_cookie=374bbb283caa8f3a in_seq=0 out_seq=0 2024-04-16T05:44:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.052+0000 7fb761ffb640 10 client.18300 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb761ffb640 1 -- 172.21.15.86:0/3340866001 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2b4b48ba0 con 0x55f2b4bef730 2024-04-16T05:44:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb761ffb640 10 client.18300 handle_command_reply: tid=0 2024-04-16T05:44:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 client.18300 shutdown 2024-04-16T05:44:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 -- 172.21.15.86:0/3340866001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bdf350 msgr2=0x55f2b4bdce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bdf350 0x55f2b4bdce70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb770009900 tx=0x7fb770009260 comp rx=0 tx=0).stop 2024-04-16T05:44:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 -- 172.21.15.86:0/3340866001 shutdown_connections 2024-04-16T05:44:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f2b4bef730 0x55f2b4bf1bb0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bdf350 0x55f2b4bdce70 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb761ffb640 0 client.18300 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bd6770 0x55f2b4bdc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 --2- 172.21.15.86:0/3340866001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bdc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 -- 172.21.15.86:0/3340866001 >> 172.21.15.86:0/3340866001 conn(0x55f2b4bd0580 msgr2=0x55f2b4bd22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 -- 172.21.15.86:0/3340866001 shutdown_connections 2024-04-16T05:44:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 1 -- 172.21.15.86:0/3340866001 wait complete. 2024-04-16T05:44:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.053+0000 7fb7835b8740 20 client.18300 trim_cache size 0 max 16384 2024-04-16T05:44:07.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb7835b8740 1 Processor -- start 2024-04-16T05:44:07.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb7835b8740 1 -- start start 2024-04-16T05:44:07.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bdeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4beb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bdf350 0x55f2b4bd0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb7835b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1af520 con 0x55f2b4bd6770 2024-04-16T05:44:07.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb7835b8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1b1060 con 0x55f2b4bd5e50 2024-04-16T05:44:07.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb7835b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1afbf0 con 0x55f2b4bdf350 2024-04-16T05:44:07.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb77a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4beb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb77a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4beb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48512/0 (socket says 172.21.15.86:48512) 2024-04-16T05:44:07.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.070+0000 7fb77a59c640 1 -- 172.21.15.86:0/2257423276 learned_addr learned my addr 172.21.15.86:0/2257423276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:07.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb77a59c640 1 -- 172.21.15.86:0/2257423276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bdf350 msgr2=0x55f2b4bd0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:07.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb77a59c640 1 --2- 172.21.15.86:0/2257423276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bdf350 0x55f2b4bd0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb77a59c640 1 -- 172.21.15.86:0/2257423276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd5e50 msgr2=0x55f2b4bdeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:07.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb77a59c640 1 --2- 172.21.15.86:0/2257423276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bdeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb77a59c640 1 -- 172.21.15.86:0/2257423276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b4bda960 con 0x55f2b4bd6770 2024-04-16T05:44:07.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb77a59c640 1 --2- 172.21.15.86:0/2257423276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4beb000 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb76c000c00 tx=0x7fb76c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60138359c1dc6f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:07.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb763fff640 1 -- 172.21.15.86:0/2257423276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0272d0 con 0x55f2b4bd6770 2024-04-16T05:44:07.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb763fff640 1 -- 172.21.15.86:0/2257423276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76c01e040 con 0x55f2b4bd6770 2024-04-16T05:44:07.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb763fff640 1 -- 172.21.15.86:0/2257423276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0188f0 con 0x55f2b4bd6770 2024-04-16T05:44:07.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb7835b8740 1 -- 172.21.15.86:0/2257423276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 msgr2=0x55f2b4beb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:07.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.071+0000 7fb7835b8740 1 --2- 172.21.15.86:0/2257423276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4beb000 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb76c000c00 tx=0x7fb76c00cd10 comp rx=0 tx=0).stop 2024-04-16T05:44:07.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 -- 172.21.15.86:0/2257423276 shutdown_connections 2024-04-16T05:44:07.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 --2- 172.21.15.86:0/2257423276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bdf350 0x55f2b4bd0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 --2- 172.21.15.86:0/2257423276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd6770 0x55f2b4beb000 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 --2- 172.21.15.86:0/2257423276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd5e50 0x55f2b4bdeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 -- 172.21.15.86:0/2257423276 >> 172.21.15.86:0/2257423276 conn(0x55f2b4bd0580 msgr2=0x55f2b4bceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:07.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 -- 172.21.15.86:0/2257423276 shutdown_connections 2024-04-16T05:44:07.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 -- 172.21.15.86:0/2257423276 wait complete. 2024-04-16T05:44:07.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 -- start start 2024-04-16T05:44:07.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 Processor -- start 2024-04-16T05:44:07.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd5e50 0x55f2b4be5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd6770 0x55f2b4be56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bdf350 0x55f2b4be5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1b0990 con 0x55f2b4bd5e50 2024-04-16T05:44:07.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1b02c0 con 0x55f2b4bd6770 2024-04-16T05:44:07.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1ad520 con 0x55f2b4bdf350 2024-04-16T05:44:07.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb7835b8740 10 client.0 _ll_register_callbacks cb 0x7fb77002cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:07.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb779d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd5e50 0x55f2b4be5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb779d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd5e50 0x55f2b4be5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48528/0 (socket says 172.21.15.86:48528) 2024-04-16T05:44:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb779d9b640 1 -- 172.21.15.86:0/984551653 learned_addr learned my addr 172.21.15.86:0/984551653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb77a59c640 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd6770 0x55f2b4be56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb779d9b640 1 -- 172.21.15.86:0/984551653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bdf350 msgr2=0x55f2b4be5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb779d9b640 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bdf350 0x55f2b4be5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb779d9b640 1 -- 172.21.15.86:0/984551653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd6770 msgr2=0x55f2b4be56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb779d9b640 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd6770 0x55f2b4be56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.072+0000 7fb779d9b640 1 -- 172.21.15.86:0/984551653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b4c54f50 con 0x55f2b4bd5e50 2024-04-16T05:44:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb77a59c640 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd6770 0x55f2b4be56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:07.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb779d9b640 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd5e50 0x55f2b4be5190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb770001eb0 tx=0x7fb77002cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c09b56232d55a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:07.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:07.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb761ffb640 1 -- 172.21.15.86:0/984551653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770046020 con 0x55f2b4bd5e50 2024-04-16T05:44:07.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb761ffb640 1 -- 172.21.15.86:0/984551653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fb770042650 con 0x55f2b4bd5e50 2024-04-16T05:44:07.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb7835b8740 1 -- 172.21.15.86:0/984551653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb76c01e040 con 0x55f2b4bd5e50 2024-04-16T05:44:07.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb761ffb640 1 -- 172.21.15.86:0/984551653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7700044b0 con 0x55f2b4bd5e50 2024-04-16T05:44:07.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb7835b8740 10 client.18306 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:07.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb7835b8740 1 -- 172.21.15.86:0/984551653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2b4c56240 con 0x55f2b4bd5e50 2024-04-16T05:44:07.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb761ffb640 1 -- 172.21.15.86:0/984551653 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76c01e040 con 0x55f2b4bd5e50 2024-04-16T05:44:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb761ffb640 1 -- 172.21.15.86:0/984551653 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7700049c0 con 0x55f2b4bd5e50 2024-04-16T05:44:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb7835b8740 10 client.18306 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb7835b8740 10 client.18306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb7835b8740 20 client.18306 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb7835b8740 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f2b4bef730 0x55f2b4c588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb7835b8740 4 client.18306 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:44:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.073+0000 7fb7835b8740 1 -- 172.21.15.86:0/984551653 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f2b4b42a10 con 0x55f2b4bef730 2024-04-16T05:44:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb77a59c640 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f2b4bef730 0x55f2b4c588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb77a59c640 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f2b4bef730 0x55f2b4c588f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449b27b4ba2c24a5 server_cookie=5642d7dd23ca5f5 in_seq=0 out_seq=0 2024-04-16T05:44:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb761ffb640 10 client.18306 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb761ffb640 1 -- 172.21.15.86:0/984551653 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f2b4b42a10 con 0x55f2b4bef730 2024-04-16T05:44:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb761ffb640 10 client.18306 handle_command_reply: tid=0 2024-04-16T05:44:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 client.18306 shutdown 2024-04-16T05:44:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 -- 172.21.15.86:0/984551653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd5e50 msgr2=0x55f2b4be5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd5e50 0x55f2b4be5190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb770001eb0 tx=0x7fb77002cab0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 -- 172.21.15.86:0/984551653 shutdown_connections 2024-04-16T05:44:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55f2b4bef730 0x55f2b4c588f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2b4bdf350 0x55f2b4be5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b4bd6770 0x55f2b4be56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb761ffb640 0 client.18306 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 --2- 172.21.15.86:0/984551653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b4bd5e50 0x55f2b4be5190 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 -- 172.21.15.86:0/984551653 >> 172.21.15.86:0/984551653 conn(0x55f2b4bd0580 msgr2=0x55f2b4bcf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 -- 172.21.15.86:0/984551653 shutdown_connections 2024-04-16T05:44:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.074+0000 7fb7835b8740 1 -- 172.21.15.86:0/984551653 wait complete. 2024-04-16T05:44:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7835b8740 20 client.18306 trim_cache size 0 max 16384 2024-04-16T05:44:07.084 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:44:07.084 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:44:07.084 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:44:07.084 INFO:teuthology.orchestra.run.smithi086.stdout: "96ac9bef-e7ea-4c92-96c6-e495c0b2d103": { 2024-04-16T05:44:07.084 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:44:07.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "96ac9bef-e7ea-4c92-96c6-e495c0b2d103", 2024-04-16T05:44:07.085 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:44:07.085 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:44:07.085 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:44:07.085 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:44:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 -- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7640a3d20 msgr2=0x7fb7640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7640a3d20 0x7fb7640a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb774099390 tx=0x7fb77406e970 comp rx=0 tx=0).stop 2024-04-16T05:44:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 -- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c1070e0 msgr2=0x7fb77c1a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c1070e0 0x7fb77c1a47c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb77400cce0 tx=0x7fb77400a620 comp rx=0 tx=0).stop 2024-04-16T05:44:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 -- 172.21.15.86:0/3307517593 shutdown_connections 2024-04-16T05:44:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77c1070e0 0x7fb77c1a47c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb7640a3d20 0x7fb7640a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb77c101bf0 0x7fb77c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 --2- 172.21.15.86:0/3307517593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb77c101250 0x7fb77c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 -- 172.21.15.86:0/3307517593 >> 172.21.15.86:0/3307517593 conn(0x7fb77c076d00 msgr2=0x7fb77c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 -- 172.21.15.86:0/3307517593 shutdown_connections 2024-04-16T05:44:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:07.075+0000 7fb7808f2640 1 -- 172.21.15.86:0/3307517593 wait complete. 2024-04-16T05:44:07.088 INFO:tasks.cephfs.filesystem:scrub status for tag:96ac9bef-e7ea-4c92-96c6-e495c0b2d103 - {'path': '/', 'tag': '96ac9bef-e7ea-4c92-96c6-e495c0b2d103', 'options': 'recursive,force'} 2024-04-16T05:44:08.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:08 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:44:08.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:08 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:08.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:08 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:44:08.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:08 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:08.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:08 smithi196 ceph-mon[36373]: pgmap v272: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:44:08.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:08 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:44:08.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:08 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:08.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:08 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:44:08.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:08 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:08.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:08 smithi194 ceph-mon[35865]: pgmap v272: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:44:08.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:08 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:44:08.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:08 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:08.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:08 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:44:08.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:08 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:08.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:08 smithi086 ceph-mon[32028]: pgmap v272: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:44:10.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:10 smithi194 ceph-mon[35865]: pgmap v273: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:44:10.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:10 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:44:10.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:10 smithi086 ceph-mon[32028]: pgmap v273: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:44:10.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:10 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:44:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:10 smithi196 ceph-mon[36373]: pgmap v273: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-16T05:44:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:10 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:44:11.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:11 smithi194 ceph-mon[35865]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T05:44:11.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:11 smithi086 ceph-mon[32028]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T05:44:11.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:11 smithi196 ceph-mon[36373]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T05:44:12.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:12 smithi086 ceph-mon[32028]: pgmap v275: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 32 op/s 2024-04-16T05:44:12.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:12 smithi194 ceph-mon[35865]: pgmap v275: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 32 op/s 2024-04-16T05:44:12.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:12 smithi196 ceph-mon[36373]: pgmap v275: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 32 op/s 2024-04-16T05:44:14.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:14 smithi194 ceph-mon[35865]: pgmap v276: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-16T05:44:14.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:14 smithi086 ceph-mon[32028]: pgmap v276: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-16T05:44:14.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:14 smithi196 ceph-mon[36373]: pgmap v276: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-16T05:44:16.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:15 smithi194 ceph-mon[35865]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T05:44:16.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:15 smithi194 ceph-mon[35865]: pgmap v278: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-16T05:44:16.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:15 smithi086 ceph-mon[32028]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T05:44:16.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:15 smithi086 ceph-mon[32028]: pgmap v278: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-16T05:44:16.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:15 smithi196 ceph-mon[36373]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T05:44:16.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:15 smithi196 ceph-mon[36373]: pgmap v278: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-16T05:44:18.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:18 smithi194 ceph-mon[35865]: pgmap v279: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:44:18.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:18 smithi086 ceph-mon[32028]: pgmap v279: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:44:18.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:18 smithi196 ceph-mon[36373]: pgmap v279: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:44:20.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:20 smithi086 ceph-mon[32028]: pgmap v280: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 231 B/s wr, 11 op/s 2024-04-16T05:44:20.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:20 smithi196 ceph-mon[36373]: pgmap v280: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 231 B/s wr, 11 op/s 2024-04-16T05:44:20.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:20 smithi194 ceph-mon[35865]: pgmap v280: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 231 B/s wr, 11 op/s 2024-04-16T05:44:22.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:22 smithi086 ceph-mon[32028]: pgmap v281: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:44:22.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:22 smithi196 ceph-mon[36373]: pgmap v281: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:44:22.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:22 smithi194 ceph-mon[35865]: pgmap v281: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:44:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:24 smithi086 ceph-mon[32028]: pgmap v282: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:44:24.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:24 smithi196 ceph-mon[36373]: pgmap v282: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:44:24.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:24 smithi194 ceph-mon[35865]: pgmap v282: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:44:26.406 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:25 smithi194 ceph-mon[35865]: pgmap v283: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:44:26.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:25 smithi086 ceph-mon[32028]: pgmap v283: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:44:26.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:25 smithi196 ceph-mon[36373]: pgmap v283: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:44:28.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:28 smithi086 ceph-mon[32028]: pgmap v284: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:44:28.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:28 smithi196 ceph-mon[36373]: pgmap v284: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:44:28.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:28 smithi194 ceph-mon[35865]: pgmap v284: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:44:30.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:30 smithi086 ceph-mon[32028]: pgmap v285: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:44:30.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:30 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:44:30.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:30 smithi196 ceph-mon[36373]: pgmap v285: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:44:30.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:30 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:44:30.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:30 smithi194 ceph-mon[35865]: pgmap v285: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-16T05:44:30.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:30 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:44:31.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:31 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:44:31.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:31 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:44:31.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:31 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:44:31.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:31 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:44:31.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:31 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:44:31.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:31 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:44:31.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:31 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:44:31.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:31 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:44:31.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:31 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:44:32.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:32 smithi194 ceph-mon[35865]: pgmap v286: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:32.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:32 smithi086 ceph-mon[32028]: pgmap v286: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:32.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:32 smithi196 ceph-mon[36373]: pgmap v286: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:34.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:34 smithi194 ceph-mon[35865]: pgmap v287: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:44:34.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:34 smithi086 ceph-mon[32028]: pgmap v287: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:44:34.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:34 smithi196 ceph-mon[36373]: pgmap v287: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:44:36.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:36 smithi194 ceph-mon[35865]: pgmap v288: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:36.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:36 smithi086 ceph-mon[32028]: pgmap v288: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:36.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:36 smithi196 ceph-mon[36373]: pgmap v288: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:37.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:44:37.089 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:44:37.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.209+0000 7f98962eb640 1 -- 172.21.15.86:0/1945702847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101bf0 msgr2=0x7f989010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.209+0000 7f98962eb640 1 --2- 172.21.15.86:0/1945702847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101bf0 0x7f989010dd30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9888009cd0 tx=0x7f988802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.209+0000 7f98962eb640 1 -- 172.21.15.86:0/1945702847 shutdown_connections 2024-04-16T05:44:37.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.209+0000 7f98962eb640 1 --2- 172.21.15.86:0/1945702847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101bf0 0x7f989010dd30 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.209+0000 7f98962eb640 1 --2- 172.21.15.86:0/1945702847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9890101250 0x7f98901016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.209+0000 7f98962eb640 1 --2- 172.21.15.86:0/1945702847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98901070e0 0x7f98901074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.209+0000 7f98962eb640 1 -- 172.21.15.86:0/1945702847 >> 172.21.15.86:0/1945702847 conn(0x7f9890076d00 msgr2=0x7f9890077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.209+0000 7f98962eb640 1 -- 172.21.15.86:0/1945702847 shutdown_connections 2024-04-16T05:44:37.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.209+0000 7f98962eb640 1 -- 172.21.15.86:0/1945702847 wait complete. 2024-04-16T05:44:37.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.210+0000 7f98962eb640 1 Processor -- start 2024-04-16T05:44:37.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.210+0000 7f98962eb640 1 -- start start 2024-04-16T05:44:37.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.210+0000 7f98962eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101250 0x7f989019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f98962eb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9890101bf0 0x7f989019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f98962eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98901070e0 0x7f9890198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f98962eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98901112e0 con 0x7f9890101250 2024-04-16T05:44:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f98962eb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9890111160 con 0x7f9890101bf0 2024-04-16T05:44:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f98962eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9890111460 con 0x7f98901070e0 2024-04-16T05:44:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101250 0x7f989019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101250 0x7f989019e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60716/0 (socket says 172.21.15.86:60716) 2024-04-16T05:44:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988ffff640 1 -- 172.21.15.86:0/2345090363 learned_addr learned my addr 172.21.15.86:0/2345090363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988f7fe640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9890101bf0 0x7f989019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f9894888640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98901070e0 0x7f9890198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988ffff640 1 -- 172.21.15.86:0/2345090363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98901070e0 msgr2=0x7f9890198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988ffff640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98901070e0 0x7f9890198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988ffff640 1 -- 172.21.15.86:0/2345090363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9890101bf0 msgr2=0x7f989019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988ffff640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9890101bf0 0x7f989019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988ffff640 1 -- 172.21.15.86:0/2345090363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9890198f50 con 0x7f9890101250 2024-04-16T05:44:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988ffff640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101250 0x7f989019e5d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f988000d770 tx=0x7f988000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988d7fa640 1 -- 172.21.15.86:0/2345090363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880017070 con 0x7f9890101250 2024-04-16T05:44:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f98962eb640 1 -- 172.21.15.86:0/2345090363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9890199240 con 0x7f9890101250 2024-04-16T05:44:37.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988d7fa640 1 -- 172.21.15.86:0/2345090363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9880005500 con 0x7f9890101250 2024-04-16T05:44:37.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f988d7fa640 1 -- 172.21.15.86:0/2345090363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880004e00 con 0x7f9890101250 2024-04-16T05:44:37.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.211+0000 7f98962eb640 1 -- 172.21.15.86:0/2345090363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a5550 con 0x7f9890101250 2024-04-16T05:44:37.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.214+0000 7f988d7fa640 1 -- 172.21.15.86:0/2345090363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9880004fa0 con 0x7f9890101250 2024-04-16T05:44:37.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.214+0000 7f9898fb4740 1 Processor -- start 2024-04-16T05:44:37.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.214+0000 7f9898fb4740 1 -- start start 2024-04-16T05:44:37.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.214+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 0x55d3a985e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.214+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9866e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.214+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9869740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.214+0000 7f9898fb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3a97c3180 con 0x55d3a9867350 2024-04-16T05:44:37.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.214+0000 7f9898fb4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d3a9760360 con 0x55d3a985e770 2024-04-16T05:44:37.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.214+0000 7f9898fb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3a97bf140 con 0x55d3a985de50 2024-04-16T05:44:37.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988d7fa640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f98780a3d20 0x7f98780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9869740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9869740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60730/0 (socket says 172.21.15.86:60730) 2024-04-16T05:44:37.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988f7fe640 1 -- 172.21.15.86:0/324156067 learned_addr learned my addr 172.21.15.86:0/324156067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988ffff640 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9866e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f9894888640 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 0x55d3a985e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f9894888640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f98780a3d20 0x7f98780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988d7fa640 1 -- 172.21.15.86:0/2345090363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f98800d06b0 con 0x7f9890101250 2024-04-16T05:44:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988f7fe640 1 -- 172.21.15.86:0/324156067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 msgr2=0x55d3a985e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988f7fe640 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 0x55d3a985e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988f7fe640 1 -- 172.21.15.86:0/324156067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 msgr2=0x55d3a9866e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988f7fe640 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9866e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988f7fe640 1 -- 172.21.15.86:0/324156067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a959f7d0 con 0x55d3a9867350 2024-04-16T05:44:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988ffff640 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9866e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f988f7fe640 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9869740 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f988400bcb0 tx=0x7f98840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21ef81ad5ed534c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f9894888640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f98780a3d20 0x7f98780a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f988802c630 tx=0x7f98880095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f98767fc640 1 -- 172.21.15.86:0/324156067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884004030 con 0x55d3a9867350 2024-04-16T05:44:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f98767fc640 1 -- 172.21.15.86:0/324156067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98840041d0 con 0x55d3a9867350 2024-04-16T05:44:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.215+0000 7f98767fc640 1 -- 172.21.15.86:0/324156067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98840117b0 con 0x55d3a9867350 2024-04-16T05:44:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 -- 172.21.15.86:0/324156067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 msgr2=0x55d3a9869740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9869740 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f988400bcb0 tx=0x7f98840107a0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 -- 172.21.15.86:0/324156067 shutdown_connections 2024-04-16T05:44:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9869740 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9866e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 --2- 172.21.15.86:0/324156067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 0x55d3a985e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 -- 172.21.15.86:0/324156067 >> 172.21.15.86:0/324156067 conn(0x55d3a9858580 msgr2=0x55d3a9858960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 -- 172.21.15.86:0/324156067 shutdown_connections 2024-04-16T05:44:37.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 -- 172.21.15.86:0/324156067 wait complete. 2024-04-16T05:44:37.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.216+0000 7f9898fb4740 1 -- start start 2024-04-16T05:44:37.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9898fb4740 1 Processor -- start 2024-04-16T05:44:37.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 0x55d3a9864340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9864880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9864dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9898fb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3a986a140 con 0x55d3a9867350 2024-04-16T05:44:37.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9898fb4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d3a977c2c0 con 0x55d3a985e770 2024-04-16T05:44:37.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9898fb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3a986a2c0 con 0x55d3a985de50 2024-04-16T05:44:37.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9898fb4740 10 client.0 _ll_register_callbacks cb 0x55d3a9763090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:37.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9894888640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9864dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9894888640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9864dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60736/0 (socket says 172.21.15.86:60736) 2024-04-16T05:44:37.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9894888640 1 -- 172.21.15.86:0/2009613337 learned_addr learned my addr 172.21.15.86:0/2009613337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f988ffff640 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 0x55d3a9864340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f988f7fe640 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9864880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9894888640 1 -- 172.21.15.86:0/2009613337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 msgr2=0x55d3a9864340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9894888640 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 0x55d3a9864340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9894888640 1 -- 172.21.15.86:0/2009613337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 msgr2=0x55d3a9864880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9894888640 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9864880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f9894888640 1 -- 172.21.15.86:0/2009613337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a94d53f0 con 0x55d3a9867350 2024-04-16T05:44:37.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.217+0000 7f988f7fe640 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9864880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:37.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9894888640 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9864dc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9888036000 tx=0x7f988802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50942f979f31a18b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f986f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:37.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f986f7fe640 1 -- 172.21.15.86:0/2009613337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888052060 con 0x55d3a9867350 2024-04-16T05:44:37.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f986f7fe640 1 -- 172.21.15.86:0/2009613337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f9888004790 con 0x55d3a9867350 2024-04-16T05:44:37.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9898fb4740 1 -- 172.21.15.86:0/2009613337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98840041d0 con 0x55d3a9867350 2024-04-16T05:44:37.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f986f7fe640 1 -- 172.21.15.86:0/2009613337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9888039070 con 0x55d3a9867350 2024-04-16T05:44:37.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9898fb4740 10 client.18318 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:37.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9898fb4740 1 -- 172.21.15.86:0/2009613337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3a94950f0 con 0x55d3a9867350 2024-04-16T05:44:37.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f986f7fe640 1 -- 172.21.15.86:0/2009613337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98840041d0 con 0x55d3a9867350 2024-04-16T05:44:37.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f986f7fe640 1 -- 172.21.15.86:0/2009613337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9888034cb0 con 0x55d3a9867350 2024-04-16T05:44:37.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9898fb4740 10 client.18318 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:37.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9898fb4740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:37.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9898fb4740 20 client.18318 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:37.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9898fb4740 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d3a9877730 0x55d3a9879bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9898fb4740 4 client.18318 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:37.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.218+0000 7f9898fb4740 1 -- 172.21.15.86:0/2009613337 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3a97d0ba0 con 0x55d3a9877730 2024-04-16T05:44:37.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.219+0000 7f988ffff640 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d3a9877730 0x55d3a9879bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.219+0000 7f986f7fe640 10 client.18318 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:37.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.219+0000 7f988ffff640 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d3a9877730 0x55d3a9879bb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ba39c8bd679c9a server_cookie=e9a01665f3c06d43 in_seq=0 out_seq=0 2024-04-16T05:44:37.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f986f7fe640 1 -- 172.21.15.86:0/2009613337 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3a97d0ba0 con 0x55d3a9877730 2024-04-16T05:44:37.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f986f7fe640 10 client.18318 handle_command_reply: tid=0 2024-04-16T05:44:37.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 client.18318 shutdown 2024-04-16T05:44:37.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 -- 172.21.15.86:0/2009613337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 msgr2=0x55d3a9864dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9864dc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9888036000 tx=0x7f988802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 -- 172.21.15.86:0/2009613337 shutdown_connections 2024-04-16T05:44:37.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d3a9877730 0x55d3a9879bb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a9867350 0x55d3a9864dc0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a9864880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f986f7fe640 0 client.18318 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:37.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 --2- 172.21.15.86:0/2009613337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a985de50 0x55d3a9864340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 -- 172.21.15.86:0/2009613337 >> 172.21.15.86:0/2009613337 conn(0x55d3a9858580 msgr2=0x55d3a985a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 -- 172.21.15.86:0/2009613337 shutdown_connections 2024-04-16T05:44:37.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 1 -- 172.21.15.86:0/2009613337 wait complete. 2024-04-16T05:44:37.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.220+0000 7f9898fb4740 20 client.18318 trim_cache size 0 max 16384 2024-04-16T05:44:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f9898fb4740 1 Processor -- start 2024-04-16T05:44:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f9898fb4740 1 -- start start 2024-04-16T05:44:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985de50 0x55d3a9866a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985e770 0x55d3a9873060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a9867350 0x55d3a9858960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f9898fb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98901af570 con 0x55d3a985e770 2024-04-16T05:44:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f9898fb4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f98901b10b0 con 0x55d3a985de50 2024-04-16T05:44:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f9898fb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98901afc40 con 0x55d3a9867350 2024-04-16T05:44:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985e770 0x55d3a9873060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985e770 0x55d3a9873060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60740/0 (socket says 172.21.15.86:60740) 2024-04-16T05:44:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988f7fe640 1 -- 172.21.15.86:0/617129675 learned_addr learned my addr 172.21.15.86:0/617129675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f9894888640 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a9867350 0x55d3a9858960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988ffff640 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985de50 0x55d3a9866a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988f7fe640 1 -- 172.21.15.86:0/617129675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a9867350 msgr2=0x55d3a9858960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988f7fe640 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a9867350 0x55d3a9858960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988f7fe640 1 -- 172.21.15.86:0/617129675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985de50 msgr2=0x55d3a9866a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988f7fe640 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985de50 0x55d3a9866a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988f7fe640 1 -- 172.21.15.86:0/617129675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a9859150 con 0x55d3a985e770 2024-04-16T05:44:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988ffff640 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985de50 0x55d3a9866a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.238+0000 7f988f7fe640 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985e770 0x55d3a9873060 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9884011010 tx=0x7f988400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e3e1bff066e6cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9875ffb640 1 -- 172.21.15.86:0/617129675 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988400b7f0 con 0x55d3a985e770 2024-04-16T05:44:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9875ffb640 1 -- 172.21.15.86:0/617129675 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9884005520 con 0x55d3a985e770 2024-04-16T05:44:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9875ffb640 1 -- 172.21.15.86:0/617129675 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884011440 con 0x55d3a985e770 2024-04-16T05:44:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9898fb4740 1 -- 172.21.15.86:0/617129675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985e770 msgr2=0x55d3a9873060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9898fb4740 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985e770 0x55d3a9873060 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9884011010 tx=0x7f988400bf30 comp rx=0 tx=0).stop 2024-04-16T05:44:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9898fb4740 1 -- 172.21.15.86:0/617129675 shutdown_connections 2024-04-16T05:44:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9898fb4740 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a9867350 0x55d3a9858960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9898fb4740 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985e770 0x55d3a9873060 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9898fb4740 1 --2- 172.21.15.86:0/617129675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985de50 0x55d3a9866a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9898fb4740 1 -- 172.21.15.86:0/617129675 >> 172.21.15.86:0/617129675 conn(0x55d3a9858580 msgr2=0x55d3a9856ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9898fb4740 1 -- 172.21.15.86:0/617129675 shutdown_connections 2024-04-16T05:44:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.239+0000 7f9898fb4740 1 -- 172.21.15.86:0/617129675 wait complete. 2024-04-16T05:44:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f9898fb4740 1 -- start start 2024-04-16T05:44:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f9898fb4740 1 Processor -- start 2024-04-16T05:44:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985de50 0x55d3a986ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a986d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f9898fb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a9867350 0x55d3a986d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f9898fb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98901b09e0 con 0x55d3a985de50 2024-04-16T05:44:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f9898fb4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f98901b0310 con 0x55d3a985e770 2024-04-16T05:44:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f9898fb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98901ad570 con 0x55d3a9867350 2024-04-16T05:44:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f9898fb4740 10 client.0 _ll_register_callbacks cb 0x7f9888009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f988ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985de50 0x55d3a986ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f988ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985de50 0x55d3a986ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60744/0 (socket says 172.21.15.86:60744) 2024-04-16T05:44:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f988ffff640 1 -- 172.21.15.86:0/769739574 learned_addr learned my addr 172.21.15.86:0/769739574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f988f7fe640 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a986d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f988ffff640 1 -- 172.21.15.86:0/769739574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a9867350 msgr2=0x55d3a986d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f988ffff640 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a9867350 0x55d3a986d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f988ffff640 1 -- 172.21.15.86:0/769739574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 msgr2=0x55d3a986d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f988ffff640 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a986d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.240+0000 7f988ffff640 1 -- 172.21.15.86:0/769739574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a98dcd00 con 0x55d3a985de50 2024-04-16T05:44:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f988ffff640 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985de50 0x55d3a986ced0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f988009bf20 tx=0x7f988006bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ef1d759d324fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f986f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f986f7fe640 1 -- 172.21.15.86:0/769739574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880020040 con 0x55d3a985de50 2024-04-16T05:44:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f986f7fe640 1 -- 172.21.15.86:0/769739574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f9880009cc0 con 0x55d3a985de50 2024-04-16T05:44:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f9898fb4740 1 -- 172.21.15.86:0/769739574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9884005520 con 0x55d3a985de50 2024-04-16T05:44:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f986f7fe640 1 -- 172.21.15.86:0/769739574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f988009c220 con 0x55d3a985de50 2024-04-16T05:44:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f9898fb4740 10 client.18324 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f9898fb4740 1 -- 172.21.15.86:0/769739574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3a98dde90 con 0x55d3a985de50 2024-04-16T05:44:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f986f7fe640 1 -- 172.21.15.86:0/769739574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9884005520 con 0x55d3a985de50 2024-04-16T05:44:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f986f7fe640 1 -- 172.21.15.86:0/769739574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f98800cdc50 con 0x55d3a985de50 2024-04-16T05:44:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f9898fb4740 10 client.18324 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f9898fb4740 10 client.18324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f9898fb4740 20 client.18324 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f9898fb4740 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d3a9877730 0x55d3a98e0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f9898fb4740 4 client.18324 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:44:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.241+0000 7f9898fb4740 1 -- 172.21.15.86:0/769739574 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d3a97caa10 con 0x55d3a9877730 2024-04-16T05:44:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f988f7fe640 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d3a9877730 0x55d3a98e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f986f7fe640 10 client.18324 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f988f7fe640 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d3a9877730 0x55d3a98e0590 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc383fb27110b9 server_cookie=1f230a4bee663f8f in_seq=0 out_seq=0 2024-04-16T05:44:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f986f7fe640 1 -- 172.21.15.86:0/769739574 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d3a97caa10 con 0x55d3a9877730 2024-04-16T05:44:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f986f7fe640 10 client.18324 handle_command_reply: tid=0 2024-04-16T05:44:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 client.18324 shutdown 2024-04-16T05:44:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 -- 172.21.15.86:0/769739574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985de50 msgr2=0x55d3a986ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985de50 0x55d3a986ced0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f988009bf20 tx=0x7f988006bb00 comp rx=0 tx=0).stop 2024-04-16T05:44:37.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 -- 172.21.15.86:0/769739574 shutdown_connections 2024-04-16T05:44:37.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55d3a9877730 0x55d3a98e0590 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3a9867350 0x55d3a986d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f986f7fe640 0 client.18324 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:37.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3a985e770 0x55d3a986d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 --2- 172.21.15.86:0/769739574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3a985de50 0x55d3a986ced0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 -- 172.21.15.86:0/769739574 >> 172.21.15.86:0/769739574 conn(0x55d3a9858580 msgr2=0x55d3a9857760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 -- 172.21.15.86:0/769739574 shutdown_connections 2024-04-16T05:44:37.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 1 -- 172.21.15.86:0/769739574 wait complete. 2024-04-16T05:44:37.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.242+0000 7f9898fb4740 20 client.18324 trim_cache size 0 max 16384 2024-04-16T05:44:37.254 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:44:37.254 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:44:37.254 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:44:37.254 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:44:37.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 -- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f98780a3d20 msgr2=0x7f98780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f98780a3d20 0x7f98780a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f988802c630 tx=0x7f98880095c0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 -- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101250 msgr2=0x7f989019e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101250 0x7f989019e5d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f988000d770 tx=0x7f988000dc30 comp rx=0 tx=0).stop 2024-04-16T05:44:37.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 -- 172.21.15.86:0/2345090363 shutdown_connections 2024-04-16T05:44:37.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98901070e0 0x7f9890198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f98780a3d20 0x7f98780a61e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9890101bf0 0x7f989019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 --2- 172.21.15.86:0/2345090363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890101250 0x7f989019e5d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 -- 172.21.15.86:0/2345090363 >> 172.21.15.86:0/2345090363 conn(0x7f9890076d00 msgr2=0x7f9890100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 -- 172.21.15.86:0/2345090363 shutdown_connections 2024-04-16T05:44:37.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.243+0000 7f98962eb640 1 -- 172.21.15.86:0/2345090363 wait complete. 2024-04-16T05:44:37.257 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:44:37.258 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:44:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 -- 172.21.15.86:0/3955847558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 msgr2=0x7f5200103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 --2- 172.21.15.86:0/3955847558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 0x7f5200103670 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009900 tx=0x7f51f002bd60 comp rx=0 tx=0).stop 2024-04-16T05:44:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 -- 172.21.15.86:0/3955847558 shutdown_connections 2024-04-16T05:44:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 --2- 172.21.15.86:0/3955847558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f520010c5d0 0x7f520010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 --2- 172.21.15.86:0/3955847558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103bb0 0x7f520010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 --2- 172.21.15.86:0/3955847558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 0x7f5200103670 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 -- 172.21.15.86:0/3955847558 >> 172.21.15.86:0/3955847558 conn(0x7f5200078000 msgr2=0x7f5200078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 -- 172.21.15.86:0/3955847558 shutdown_connections 2024-04-16T05:44:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 -- 172.21.15.86:0/3955847558 wait complete. 2024-04-16T05:44:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 Processor -- start 2024-04-16T05:44:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 -- start start 2024-04-16T05:44:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.366+0000 7f5208693640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 0x7f520019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f5208693640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103bb0 0x7f520019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f5208693640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f520010c5d0 0x7f5200198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f5208693640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52001123e0 con 0x7f5200103290 2024-04-16T05:44:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f5208693640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5200112260 con 0x7f520010c5d0 2024-04-16T05:44:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f5208693640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5200112560 con 0x7f5200103bb0 2024-04-16T05:44:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f520642f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 0x7f520019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f520642f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 0x7f520019e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60766/0 (socket says 172.21.15.86:60766) 2024-04-16T05:44:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f520642f640 1 -- 172.21.15.86:0/2899552664 learned_addr learned my addr 172.21.15.86:0/2899552664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f5205c2e640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103bb0 0x7f520019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f520642f640 1 -- 172.21.15.86:0/2899552664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103bb0 msgr2=0x7f520019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f520642f640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103bb0 0x7f520019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f520642f640 1 -- 172.21.15.86:0/2899552664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f520010c5d0 msgr2=0x7f5200198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f520642f640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f520010c5d0 0x7f5200198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f520642f640 1 -- 172.21.15.86:0/2899552664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5200198f00 con 0x7f5200103290 2024-04-16T05:44:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f5205c2e640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103bb0 0x7f520019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f520642f640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 0x7f520019e580 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f51f0004f50 tx=0x7f51f002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f51f77fe640 1 -- 172.21.15.86:0/2899552664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0043020 con 0x7f5200103290 2024-04-16T05:44:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f51f77fe640 1 -- 172.21.15.86:0/2899552664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f00054e0 con 0x7f5200103290 2024-04-16T05:44:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f51f77fe640 1 -- 172.21.15.86:0/2899552664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0003de0 con 0x7f5200103290 2024-04-16T05:44:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5200199190 con 0x7f5200103290 2024-04-16T05:44:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.367+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001a5500 con 0x7f5200103290 2024-04-16T05:44:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.368+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5200069990 con 0x7f5200103290 2024-04-16T05:44:37.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.369+0000 7f51f77fe640 1 -- 172.21.15.86:0/2899552664 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f51f0046020 con 0x7f5200103290 2024-04-16T05:44:37.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.370+0000 7f51f77fe640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51ec0a3da0 0x7f51ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.370+0000 7f5205c2e640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51ec0a3da0 0x7f51ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.370+0000 7f51f77fe640 1 -- 172.21.15.86:0/2899552664 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f51f00f3350 con 0x7f5200103290 2024-04-16T05:44:37.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.370+0000 7f5205c2e640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51ec0a3da0 0x7f51ec0a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5200199e90 tx=0x7f51fc0082f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.374+0000 7f51f77fe640 1 -- 172.21.15.86:0/2899552664 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51f004d020 con 0x7f5200103290 2024-04-16T05:44:37.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.591+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52001a56d0 con 0x7f5200103290 2024-04-16T05:44:37.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.592+0000 7f51f77fe640 1 -- 172.21.15.86:0/2899552664 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7f51f00f0e20 con 0x7f5200103290 2024-04-16T05:44:37.590 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:44:37.590 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:44:37.591 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:44:37.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.593+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51ec0a3da0 msgr2=0x7f51ec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.593+0000 7f5208693640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51ec0a3da0 0x7f51ec0a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5200199e90 tx=0x7f51fc0082f0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 msgr2=0x7f520019e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 0x7f520019e580 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f51f0004f50 tx=0x7f51f002c860 comp rx=0 tx=0).stop 2024-04-16T05:44:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 shutdown_connections 2024-04-16T05:44:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f51ec0a3da0 0x7f51ec0a6260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f520010c5d0 0x7f5200198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103bb0 0x7f520019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 --2- 172.21.15.86:0/2899552664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200103290 0x7f520019e580 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 >> 172.21.15.86:0/2899552664 conn(0x7f5200078000 msgr2=0x7f52000fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 shutdown_connections 2024-04-16T05:44:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.594+0000 7f5208693640 1 -- 172.21.15.86:0/2899552664 wait complete. 2024-04-16T05:44:37.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:44:37.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.710+0000 7fe4f4550640 1 -- 172.21.15.86:0/2663158510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 msgr2=0x7fe4ec10ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.710+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2663158510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 0x7fe4ec10ea50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009900 tx=0x7fe4e402bd60 comp rx=0 tx=0).stop 2024-04-16T05:44:37.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 -- 172.21.15.86:0/2663158510 shutdown_connections 2024-04-16T05:44:37.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2663158510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 0x7fe4ec10ea50 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2663158510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4ec103b90 0x7fe4ec10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2663158510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4ec103270 0x7fe4ec103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 -- 172.21.15.86:0/2663158510 >> 172.21.15.86:0/2663158510 conn(0x7fe4ec0780b0 msgr2=0x7fe4ec0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 -- 172.21.15.86:0/2663158510 shutdown_connections 2024-04-16T05:44:37.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 -- 172.21.15.86:0/2663158510 wait complete. 2024-04-16T05:44:37.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 Processor -- start 2024-04-16T05:44:37.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 -- start start 2024-04-16T05:44:37.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4ec103270 0x7fe4ec19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4ec103b90 0x7fe4ec19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 0x7fe4ec198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec1123e0 con 0x7fe4ec10c5b0 2024-04-16T05:44:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec112260 con 0x7fe4ec103270 2024-04-16T05:44:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.711+0000 7fe4f4550640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec112560 con 0x7fe4ec103b90 2024-04-16T05:44:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f2aed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 0x7fe4ec198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f2aed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 0x7fe4ec198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60780/0 (socket says 172.21.15.86:60780) 2024-04-16T05:44:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f2aed640 1 -- 172.21.15.86:0/2110060165 learned_addr learned my addr 172.21.15.86:0/2110060165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4ec103270 0x7fe4ec19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4ec103b90 0x7fe4ec19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f2aed640 1 -- 172.21.15.86:0/2110060165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4ec103b90 msgr2=0x7fe4ec19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f2aed640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4ec103b90 0x7fe4ec19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f2aed640 1 -- 172.21.15.86:0/2110060165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4ec103270 msgr2=0x7fe4ec19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f2aed640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4ec103270 0x7fe4ec19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f2aed640 1 -- 172.21.15.86:0/2110060165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ec198f40 con 0x7fe4ec10c5b0 2024-04-16T05:44:37.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4ec103b90 0x7fe4ec19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:37.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f2aed640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 0x7fe4ec198680 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe4e402c370 tx=0x7fe4e402dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4eb7fe640 1 -- 172.21.15.86:0/2110060165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4043020 con 0x7fe4ec10c5b0 2024-04-16T05:44:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4eb7fe640 1 -- 172.21.15.86:0/2110060165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e40054e0 con 0x7fe4ec10c5b0 2024-04-16T05:44:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f4550640 1 -- 172.21.15.86:0/2110060165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ec1991d0 con 0x7fe4ec10c5b0 2024-04-16T05:44:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4eb7fe640 1 -- 172.21.15.86:0/2110060165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4003c20 con 0x7fe4ec10c5b0 2024-04-16T05:44:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.712+0000 7fe4f4550640 1 -- 172.21.15.86:0/2110060165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ec1a5540 con 0x7fe4ec10c5b0 2024-04-16T05:44:37.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.714+0000 7fe4eb7fe640 1 -- 172.21.15.86:0/2110060165 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4e4046020 con 0x7fe4ec10c5b0 2024-04-16T05:44:37.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f720e740 1 Processor -- start 2024-04-16T05:44:37.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f720e740 1 -- start start 2024-04-16T05:44:37.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 0x55e26dcf2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf2780 0x55e26dcfae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcfb360 0x55e26dcfd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f720e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e26dc570e0 con 0x55e26dcfb360 2024-04-16T05:44:37.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f720e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e26dbf4520 con 0x55e26dcf1e60 2024-04-16T05:44:37.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f720e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e26dc530a0 con 0x55e26dcf2780 2024-04-16T05:44:37.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f1aeb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcfb360 0x55e26dcfd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f1aeb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcfb360 0x55e26dcfd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60796/0 (socket says 172.21.15.86:60796) 2024-04-16T05:44:37.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f1aeb640 1 -- 172.21.15.86:0/3098709634 learned_addr learned my addr 172.21.15.86:0/3098709634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4eb7fe640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4d40a3d20 0x7fe4d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf2780 0x55e26dcfae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.715+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 0x55e26dcf2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4f1aeb640 1 -- 172.21.15.86:0/3098709634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf2780 msgr2=0x55e26dcfae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf2780 0x55e26dcfae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4f2aed640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4d40a3d20 0x7fe4d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4f1aeb640 1 -- 172.21.15.86:0/3098709634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 msgr2=0x55e26dcf2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 0x55e26dcf2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4f1aeb640 1 -- 172.21.15.86:0/3098709634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26dc1c8d0 con 0x55e26dcfb360 2024-04-16T05:44:37.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf2780 0x55e26dcfae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:37.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4eb7fe640 1 -- 172.21.15.86:0/2110060165 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fe4e40f2dc0 con 0x7fe4ec10c5b0 2024-04-16T05:44:37.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcfb360 0x55e26dcfd750 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000be60 tx=0x7fe4e000b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e95974f63abcc81 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4e8bfe640 1 -- 172.21.15.86:0/3098709634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e001b450 con 0x55e26dcfb360 2024-04-16T05:44:37.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4f2aed640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4d40a3d20 0x7fe4d40a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40f0740 tx=0x7fe4e4004ee0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4e8bfe640 1 -- 172.21.15.86:0/3098709634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e001bad0 con 0x55e26dcfb360 2024-04-16T05:44:37.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.716+0000 7fe4e8bfe640 1 -- 172.21.15.86:0/3098709634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0012ce0 con 0x55e26dcfb360 2024-04-16T05:44:37.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 -- 172.21.15.86:0/3098709634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcfb360 msgr2=0x55e26dcfd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcfb360 0x55e26dcfd750 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000be60 tx=0x7fe4e000b5a0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 -- 172.21.15.86:0/3098709634 shutdown_connections 2024-04-16T05:44:37.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcfb360 0x55e26dcfd750 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf2780 0x55e26dcfae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 --2- 172.21.15.86:0/3098709634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 0x55e26dcf2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 -- 172.21.15.86:0/3098709634 >> 172.21.15.86:0/3098709634 conn(0x55e26dcec590 msgr2=0x55e26dcec970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 -- 172.21.15.86:0/3098709634 shutdown_connections 2024-04-16T05:44:37.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 -- 172.21.15.86:0/3098709634 wait complete. 2024-04-16T05:44:37.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 -- start start 2024-04-16T05:44:37.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.717+0000 7fe4f720e740 1 Processor -- start 2024-04-16T05:44:37.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf1e60 0x55e26dcf8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf2780 0x55e26dcf89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcfb360 0x55e26dcf8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e26dcfe000 con 0x55e26dcf1e60 2024-04-16T05:44:37.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e26dc104d0 con 0x55e26dcf2780 2024-04-16T05:44:37.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e26dcfe180 con 0x55e26dcfb360 2024-04-16T05:44:37.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 10 client.0 _ll_register_callbacks cb 0x55e26dbf52c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:37.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f1aeb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf1e60 0x55e26dcf8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f1aeb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf1e60 0x55e26dcf8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60802/0 (socket says 172.21.15.86:60802) 2024-04-16T05:44:37.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f1aeb640 1 -- 172.21.15.86:0/259002689 learned_addr learned my addr 172.21.15.86:0/259002689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf2780 0x55e26dcf89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f1aeb640 1 -- 172.21.15.86:0/259002689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcfb360 msgr2=0x55e26dcf8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcfb360 0x55e26dcf8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f1aeb640 1 -- 172.21.15.86:0/259002689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf2780 msgr2=0x55e26dcf89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf2780 0x55e26dcf89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f1aeb640 1 -- 172.21.15.86:0/259002689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26d9693f0 con 0x55e26dcf1e60 2024-04-16T05:44:37.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf1e60 0x55e26dcf8490 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0012860 tx=0x7fe4e00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa112d0873051998 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:37.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/259002689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e001b710 con 0x55e26dcf1e60 2024-04-16T05:44:37.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/259002689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fe4e001b8b0 con 0x55e26dcf1e60 2024-04-16T05:44:37.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 1 -- 172.21.15.86:0/259002689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e001bad0 con 0x55e26dcf1e60 2024-04-16T05:44:37.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/259002689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4e0012960 con 0x55e26dcf1e60 2024-04-16T05:44:37.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 10 client.18342 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:37.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.718+0000 7fe4f720e740 1 -- 172.21.15.86:0/259002689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e26da337d0 con 0x55e26dcf1e60 2024-04-16T05:44:37.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/259002689 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e001bad0 con 0x55e26dcf1e60 2024-04-16T05:44:37.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/259002689 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe4e000f040 con 0x55e26dcf1e60 2024-04-16T05:44:37.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4f720e740 10 client.18342 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:37.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4f720e740 10 client.18342 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:44:37.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4f720e740 20 client.18342 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:37.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4f720e740 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55e26dd0b370 0x55e26dd0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4f720e740 4 client.18342 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:37.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4f720e740 1 -- 172.21.15.86:0/259002689 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e26dc64b00 con 0x55e26dd0b370 2024-04-16T05:44:37.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55e26dd0b370 0x55e26dd0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4d1ffb640 10 client.18342 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:44:37.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.719+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55e26dd0b370 0x55e26dd0d760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1823cb14cb44cd7c server_cookie=f12c55c566b8a51c in_seq=0 out_seq=0 2024-04-16T05:44:37.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/259002689 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e26dc64b00 con 0x55e26dd0b370 2024-04-16T05:44:37.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4d1ffb640 10 client.18342 handle_command_reply: tid=0 2024-04-16T05:44:37.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 client.18342 shutdown 2024-04-16T05:44:37.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 -- 172.21.15.86:0/259002689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf1e60 msgr2=0x55e26dcf8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf1e60 0x55e26dcf8490 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0012860 tx=0x7fe4e00058e0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 -- 172.21.15.86:0/259002689 shutdown_connections 2024-04-16T05:44:37.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55e26dd0b370 0x55e26dd0d760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcfb360 0x55e26dcf8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4d1ffb640 0 client.18342 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:44:37.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf2780 0x55e26dcf89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 --2- 172.21.15.86:0/259002689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf1e60 0x55e26dcf8490 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 -- 172.21.15.86:0/259002689 >> 172.21.15.86:0/259002689 conn(0x55e26dcec590 msgr2=0x55e26dcee2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 -- 172.21.15.86:0/259002689 shutdown_connections 2024-04-16T05:44:37.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.720+0000 7fe4f720e740 1 -- 172.21.15.86:0/259002689 wait complete. 2024-04-16T05:44:37.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.721+0000 7fe4f720e740 20 client.18342 trim_cache size 0 max 16384 2024-04-16T05:44:37.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f720e740 1 Processor -- start 2024-04-16T05:44:37.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f720e740 1 -- start start 2024-04-16T05:44:37.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf1e60 0x55e26dcfabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dd12e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcfb360 0x55e26dd13390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f720e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec1af550 con 0x55e26dcf2780 2024-04-16T05:44:37.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f720e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec1b1090 con 0x55e26dcfb360 2024-04-16T05:44:37.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f720e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec1afc20 con 0x55e26dcf1e60 2024-04-16T05:44:37.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f22ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dd12e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f22ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dd12e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60808/0 (socket says 172.21.15.86:60808) 2024-04-16T05:44:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f22ec640 1 -- 172.21.15.86:0/3158469758 learned_addr learned my addr 172.21.15.86:0/3158469758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f22ec640 1 -- 172.21.15.86:0/3158469758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf1e60 msgr2=0x55e26dcfabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/3158469758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf1e60 0x55e26dcfabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f22ec640 1 -- 172.21.15.86:0/3158469758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcfb360 msgr2=0x55e26dd13390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/3158469758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcfb360 0x55e26dd13390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f22ec640 1 -- 172.21.15.86:0/3158469758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26dcf6a00 con 0x55e26dcf2780 2024-04-16T05:44:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/3158469758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dd12e50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x55e26dcf7cf0 tx=0x7fe4dc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e945ff2c3fd30822 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.734+0000 7fe4d3fff640 1 -- 172.21.15.86:0/3158469758 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc01e070 con 0x55e26dcf2780 2024-04-16T05:44:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4d3fff640 1 -- 172.21.15.86:0/3158469758 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc003e50 con 0x55e26dcf2780 2024-04-16T05:44:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4d3fff640 1 -- 172.21.15.86:0/3158469758 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc01a850 con 0x55e26dcf2780 2024-04-16T05:44:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 -- 172.21.15.86:0/3158469758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 msgr2=0x55e26dd12e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 --2- 172.21.15.86:0/3158469758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dd12e50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x55e26dcf7cf0 tx=0x7fe4dc00a650 comp rx=0 tx=0).stop 2024-04-16T05:44:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 -- 172.21.15.86:0/3158469758 shutdown_connections 2024-04-16T05:44:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 --2- 172.21.15.86:0/3158469758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcfb360 0x55e26dd13390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 --2- 172.21.15.86:0/3158469758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dd12e50 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 --2- 172.21.15.86:0/3158469758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcf1e60 0x55e26dcfabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 -- 172.21.15.86:0/3158469758 >> 172.21.15.86:0/3158469758 conn(0x55e26dcec590 msgr2=0x55e26dce75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 -- 172.21.15.86:0/3158469758 shutdown_connections 2024-04-16T05:44:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 -- 172.21.15.86:0/3158469758 wait complete. 2024-04-16T05:44:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 -- start start 2024-04-16T05:44:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.735+0000 7fe4f720e740 1 Processor -- start 2024-04-16T05:44:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 0x55e26dd039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dcf5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f720e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcfb360 0x55e26dcf5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f720e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec1b09c0 con 0x55e26dcf2780 2024-04-16T05:44:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f720e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec1b02f0 con 0x55e26dcf1e60 2024-04-16T05:44:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f720e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec1ad550 con 0x55e26dcfb360 2024-04-16T05:44:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f720e740 10 client.0 _ll_register_callbacks cb 0x7fe4e000bc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f22ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dcf5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f22ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dcf5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60816/0 (socket says 172.21.15.86:60816) 2024-04-16T05:44:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f22ec640 1 -- 172.21.15.86:0/2296937270 learned_addr learned my addr 172.21.15.86:0/2296937270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 0x55e26dd039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcfb360 0x55e26dcf5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f22ec640 1 -- 172.21.15.86:0/2296937270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcfb360 msgr2=0x55e26dcf5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcfb360 0x55e26dcf5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f22ec640 1 -- 172.21.15.86:0/2296937270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 msgr2=0x55e26dd039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 0x55e26dd039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f22ec640 1 -- 172.21.15.86:0/2296937270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26dcf5e70 con 0x55e26dcf2780 2024-04-16T05:44:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 0x55e26dd039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f22ec640 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dcf5030 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b010 tx=0x7fe4dc012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700fe18084fdabd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/2296937270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc01e040 con 0x55e26dcf2780 2024-04-16T05:44:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/2296937270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fe4dc004020 con 0x55e26dcf2780 2024-04-16T05:44:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.736+0000 7fe4f720e740 1 -- 172.21.15.86:0/2296937270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4dc003e50 con 0x55e26dcf2780 2024-04-16T05:44:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/2296937270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4dc035050 con 0x55e26dcf2780 2024-04-16T05:44:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f720e740 10 client.18348 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f720e740 1 -- 172.21.15.86:0/2296937270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e26dd717c0 con 0x55e26dcf2780 2024-04-16T05:44:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/2296937270 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc003e50 con 0x55e26dcf2780 2024-04-16T05:44:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/2296937270 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe4dc01abc0 con 0x55e26dcf2780 2024-04-16T05:44:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f720e740 10 client.18348 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f720e740 10 client.18348 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:44:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f720e740 20 client.18348 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f720e740 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55e26dd71eb0 0x55e26dd742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f720e740 4 client.18348 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f720e740 1 -- 172.21.15.86:0/2296937270 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e26dc5e970 con 0x55e26dd71eb0 2024-04-16T05:44:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55e26dd71eb0 0x55e26dd742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4d1ffb640 10 client.18348 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:44:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.737+0000 7fe4f1aeb640 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55e26dd71eb0 0x55e26dd742d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1744019806799774 server_cookie=a2917076ea99d9b2 in_seq=0 out_seq=0 2024-04-16T05:44:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4d1ffb640 1 -- 172.21.15.86:0/2296937270 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e26dc5e970 con 0x55e26dd71eb0 2024-04-16T05:44:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4d1ffb640 10 client.18348 handle_command_reply: tid=0 2024-04-16T05:44:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 client.18348 shutdown 2024-04-16T05:44:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 -- 172.21.15.86:0/2296937270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 msgr2=0x55e26dcf5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dcf5030 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b010 tx=0x7fe4dc012ef0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 -- 172.21.15.86:0/2296937270 shutdown_connections 2024-04-16T05:44:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x55e26dd71eb0 0x55e26dd742d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e26dcfb360 0x55e26dcf5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26dcf2780 0x55e26dcf5030 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4d1ffb640 0 client.18348 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:44:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 --2- 172.21.15.86:0/2296937270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e26dcf1e60 0x55e26dd039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 -- 172.21.15.86:0/2296937270 >> 172.21.15.86:0/2296937270 conn(0x55e26dcec590 msgr2=0x55e26dced910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 -- 172.21.15.86:0/2296937270 shutdown_connections 2024-04-16T05:44:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 1 -- 172.21.15.86:0/2296937270 wait complete. 2024-04-16T05:44:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f720e740 20 client.18348 trim_cache size 0 max 16384 2024-04-16T05:44:37.747 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:37.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f4550640 1 -- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4d40a3d20 msgr2=0x7fe4d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4d40a3d20 0x7fe4d40a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40f0740 tx=0x7fe4e4004ee0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f4550640 1 -- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 msgr2=0x7fe4ec198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 0x7fe4ec198680 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe4e402c370 tx=0x7fe4e402dd40 comp rx=0 tx=0).stop 2024-04-16T05:44:37.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.738+0000 7fe4f4550640 1 -- 172.21.15.86:0/2110060165 shutdown_connections 2024-04-16T05:44:37.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.739+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4ec10c5b0 0x7fe4ec198680 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.739+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4ec103b90 0x7fe4ec19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.739+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe4d40a3d20 0x7fe4d40a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.739+0000 7fe4f4550640 1 --2- 172.21.15.86:0/2110060165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4ec103270 0x7fe4ec19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.739+0000 7fe4f4550640 1 -- 172.21.15.86:0/2110060165 >> 172.21.15.86:0/2110060165 conn(0x7fe4ec0780b0 msgr2=0x7fe4ec0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.739+0000 7fe4f4550640 1 -- 172.21.15.86:0/2110060165 shutdown_connections 2024-04-16T05:44:37.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.739+0000 7fe4f4550640 1 -- 172.21.15.86:0/2110060165 wait complete. 2024-04-16T05:44:37.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:44:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.859+0000 7fb2fa463640 1 -- 172.21.15.86:0/3453665682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 msgr2=0x7fb2f4101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.859+0000 7fb2fa463640 1 --2- 172.21.15.86:0/3453665682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 0x7fb2f4101510 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009ca0 tx=0x7fb2e402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.860+0000 7fb2fa463640 1 -- 172.21.15.86:0/3453665682 shutdown_connections 2024-04-16T05:44:37.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.860+0000 7fb2fa463640 1 --2- 172.21.15.86:0/3453665682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f4106200 0x7fb2f410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.860+0000 7fb2fa463640 1 --2- 172.21.15.86:0/3453665682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2f4101ae0 0x7fb2f4105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.860+0000 7fb2fa463640 1 --2- 172.21.15.86:0/3453665682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 0x7fb2f4101510 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.860+0000 7fb2fa463640 1 -- 172.21.15.86:0/3453665682 >> 172.21.15.86:0/3453665682 conn(0x7fb2f40780b0 msgr2=0x7fb2f40784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.860+0000 7fb2fa463640 1 -- 172.21.15.86:0/3453665682 shutdown_connections 2024-04-16T05:44:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.860+0000 7fb2fa463640 1 -- 172.21.15.86:0/3453665682 wait complete. 2024-04-16T05:44:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.860+0000 7fb2fa463640 1 Processor -- start 2024-04-16T05:44:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.860+0000 7fb2fa463640 1 -- start start 2024-04-16T05:44:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2fa463640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 0x7fb2f419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2fa463640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2f4106200 0x7fb2f419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2fa463640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f4197610 0x7fb2f4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2fa463640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41112b0 con 0x7fb2f4101130 2024-04-16T05:44:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2fa463640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4111130 con 0x7fb2f4197610 2024-04-16T05:44:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2fa463640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4111430 con 0x7fb2f4106200 2024-04-16T05:44:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 0x7fb2f419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 0x7fb2f419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60840/0 (socket says 172.21.15.86:60840) 2024-04-16T05:44:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f3fff640 1 -- 172.21.15.86:0/2865334393 learned_addr learned my addr 172.21.15.86:0/2865334393 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f4197610 0x7fb2f4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f3fff640 1 -- 172.21.15.86:0/2865334393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2f4106200 msgr2=0x7fb2f419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2f4106200 0x7fb2f419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f3fff640 1 -- 172.21.15.86:0/2865334393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f4197610 msgr2=0x7fb2f4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f4197610 0x7fb2f4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f3fff640 1 -- 172.21.15.86:0/2865334393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f4198130 con 0x7fb2f4101130 2024-04-16T05:44:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f4197610 0x7fb2f4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 0x7fb2f419d460 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4005e30 tx=0x7fb2e4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f17fa640 1 -- 172.21.15.86:0/2865334393 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4039040 con 0x7fb2f4101130 2024-04-16T05:44:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f17fa640 1 -- 172.21.15.86:0/2865334393 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e4044070 con 0x7fb2f4101130 2024-04-16T05:44:37.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2fa463640 1 -- 172.21.15.86:0/2865334393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f41983c0 con 0x7fb2f4101130 2024-04-16T05:44:37.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.861+0000 7fb2f17fa640 1 -- 172.21.15.86:0/2865334393 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40042a0 con 0x7fb2f4101130 2024-04-16T05:44:37.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.862+0000 7fb2fa463640 1 -- 172.21.15.86:0/2865334393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f41a45b0 con 0x7fb2f4101130 2024-04-16T05:44:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2f17fa640 1 -- 172.21.15.86:0/2865334393 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2e4004440 con 0x7fb2f4101130 2024-04-16T05:44:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2fd04e740 1 Processor -- start 2024-04-16T05:44:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2fd04e740 1 -- start start 2024-04-16T05:44:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233070 0x55fef0233450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233990 0x55fef023c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 0x55fef023e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2f17fa640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fef01960e0 con 0x55fef0233070 2024-04-16T05:44:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fef0133520 con 0x55fef0233990 2024-04-16T05:44:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fef01920a0 con 0x55fef023c570 2024-04-16T05:44:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2f8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233070 0x55fef0233450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2f8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233070 0x55fef0233450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60842/0 (socket says 172.21.15.86:60842) 2024-04-16T05:44:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2f8a00640 1 -- 172.21.15.86:0/3370885268 learned_addr learned my addr 172.21.15.86:0/3370885268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2f17fa640 1 -- 172.21.15.86:0/2865334393 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fb2e40f2ef0 con 0x7fb2f4101130 2024-04-16T05:44:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.864+0000 7fb2f37fe640 1 --2- 172.21.15.86:0/3370885268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 0x55fef023e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/3370885268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233990 0x55fef023c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2f8a00640 1 -- 172.21.15.86:0/3370885268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 msgr2=0x55fef023e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/3370885268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 0x55fef023e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2f8a00640 1 -- 172.21.15.86:0/3370885268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233990 msgr2=0x55fef023c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/3370885268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233990 0x55fef023c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2f8a00640 1 -- 172.21.15.86:0/3370885268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef015b8d0 con 0x55fef0233070 2024-04-16T05:44:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/3370885268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233070 0x55fef0233450 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005fd0 tx=0x7fb2ec0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf53209d77f0807 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005d50 tx=0x7fb2ec015000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2ea7fc640 1 -- 172.21.15.86:0/3370885268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec00bd40 con 0x55fef0233070 2024-04-16T05:44:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2ea7fc640 1 -- 172.21.15.86:0/3370885268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2ec005020 con 0x55fef0233070 2024-04-16T05:44:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.865+0000 7fb2ea7fc640 1 -- 172.21.15.86:0/3370885268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0049e0 con 0x55fef0233070 2024-04-16T05:44:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3370885268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233070 msgr2=0x55fef0233450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3370885268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233070 0x55fef0233450 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005fd0 tx=0x7fb2ec0107a0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3370885268 shutdown_connections 2024-04-16T05:44:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3370885268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 0x55fef023e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3370885268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233990 0x55fef023c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3370885268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233070 0x55fef0233450 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3370885268 >> 172.21.15.86:0/3370885268 conn(0x55fef022d7a0 msgr2=0x55fef022db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3370885268 shutdown_connections 2024-04-16T05:44:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3370885268 wait complete. 2024-04-16T05:44:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 -- start start 2024-04-16T05:44:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.866+0000 7fb2fd04e740 1 Processor -- start 2024-04-16T05:44:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233070 0x55fef02395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef0239af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 0x55fef023a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fef023f300 con 0x55fef0233990 2024-04-16T05:44:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fef014f4d0 con 0x55fef0233070 2024-04-16T05:44:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fef023f480 con 0x55fef023c570 2024-04-16T05:44:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 10 client.0 _ll_register_callbacks cb 0x55fef01342c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef0239af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef0239af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60852/0 (socket says 172.21.15.86:60852) 2024-04-16T05:44:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f37fe640 1 -- 172.21.15.86:0/3646367930 learned_addr learned my addr 172.21.15.86:0/3646367930 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233070 0x55fef02395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 0x55fef023a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f37fe640 1 -- 172.21.15.86:0/3646367930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 msgr2=0x55fef023a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f37fe640 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 0x55fef023a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f37fe640 1 -- 172.21.15.86:0/3646367930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233070 msgr2=0x55fef02395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f37fe640 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233070 0x55fef02395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f37fe640 1 -- 172.21.15.86:0/3646367930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feefea83f0 con 0x55fef0233990 2024-04-16T05:44:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233070 0x55fef02395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2f37fe640 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef0239af0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00bd30 tx=0x7fb2dc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd881f00113ae34 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2d37fe640 1 -- 172.21.15.86:0/3646367930 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc015070 con 0x55fef0233990 2024-04-16T05:44:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2d37fe640 1 -- 172.21.15.86:0/3646367930 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fb2dc011850 con 0x55fef0233990 2024-04-16T05:44:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3646367930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2ec005020 con 0x55fef0233990 2024-04-16T05:44:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2d37fe640 1 -- 172.21.15.86:0/3646367930 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2dc023050 con 0x55fef0233990 2024-04-16T05:44:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 10 client.18360 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.867+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3646367930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55feeff727d0 con 0x55fef0233990 2024-04-16T05:44:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.868+0000 7fb2d37fe640 1 -- 172.21.15.86:0/3646367930 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2ec005020 con 0x55fef0233990 2024-04-16T05:44:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.868+0000 7fb2d37fe640 1 -- 172.21.15.86:0/3646367930 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb2dc01ab00 con 0x55fef0233990 2024-04-16T05:44:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.868+0000 7fb2fd04e740 10 client.18360 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.868+0000 7fb2fd04e740 10 client.18360 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:44:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.868+0000 7fb2fd04e740 20 client.18360 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.868+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55fef024c580 0x55fef024e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.868+0000 7fb2fd04e740 4 client.18360 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.868+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3646367930 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fef01a3b00 con 0x55fef024c580 2024-04-16T05:44:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.868+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55fef024c580 0x55fef024e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2d37fe640 10 client.18360 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:44:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55fef024c580 0x55fef024e9b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=736c0293a28bf7c6 server_cookie=5f1e20cbe97f4722 in_seq=0 out_seq=0 2024-04-16T05:44:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2d37fe640 1 -- 172.21.15.86:0/3646367930 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fef01a3b00 con 0x55fef024c580 2024-04-16T05:44:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2d37fe640 10 client.18360 handle_command_reply: tid=0 2024-04-16T05:44:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2fd04e740 1 client.18360 shutdown 2024-04-16T05:44:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3646367930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 msgr2=0x55fef0239af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef0239af0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00bd30 tx=0x7fb2dc00f730 comp rx=0 tx=0).stop 2024-04-16T05:44:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3646367930 shutdown_connections 2024-04-16T05:44:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55fef024c580 0x55fef024e9b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef023c570 0x55fef023a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2d37fe640 0 client.18360 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:44:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef0239af0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/3646367930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233070 0x55fef02395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.869+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3646367930 >> 172.21.15.86:0/3646367930 conn(0x55fef022d7a0 msgr2=0x55fef022a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.870+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3646367930 shutdown_connections 2024-04-16T05:44:37.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.870+0000 7fb2fd04e740 1 -- 172.21.15.86:0/3646367930 wait complete. 2024-04-16T05:44:37.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.870+0000 7fb2fd04e740 20 client.18360 trim_cache size 0 max 16384 2024-04-16T05:44:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2fd04e740 1 Processor -- start 2024-04-16T05:44:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2fd04e740 1 -- start start 2024-04-16T05:44:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 0x55fef023bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233990 0x55fef02481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef023c570 0x55fef02265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41ad370 con 0x55fef023c570 2024-04-16T05:44:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41aeeb0 con 0x55fef0233990 2024-04-16T05:44:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41ada40 con 0x55fef0233070 2024-04-16T05:44:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef023c570 0x55fef02265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef023c570 0x55fef02265c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60854/0 (socket says 172.21.15.86:60854) 2024-04-16T05:44:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f8a00640 1 -- 172.21.15.86:0/167666339 learned_addr learned my addr 172.21.15.86:0/167666339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/167666339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 0x55fef023bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f8a00640 1 -- 172.21.15.86:0/167666339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 msgr2=0x55fef023bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/167666339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 0x55fef023bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f8a00640 1 -- 172.21.15.86:0/167666339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233990 msgr2=0x55fef02481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/167666339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233990 0x55fef02481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f8a00640 1 -- 172.21.15.86:0/167666339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef0238560 con 0x55fef023c570 2024-04-16T05:44:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.883+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/167666339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef023c570 0x55fef02265c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00efd0 tx=0x7fb2ec00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41754f6e738f8989 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2e9ffb640 1 -- 172.21.15.86:0/167666339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec00bd40 con 0x55fef023c570 2024-04-16T05:44:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2e9ffb640 1 -- 172.21.15.86:0/167666339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2ec004020 con 0x55fef023c570 2024-04-16T05:44:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2e9ffb640 1 -- 172.21.15.86:0/167666339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec003970 con 0x55fef023c570 2024-04-16T05:44:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 -- 172.21.15.86:0/167666339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef023c570 msgr2=0x55fef02265c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/167666339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef023c570 0x55fef02265c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00efd0 tx=0x7fb2ec00eb60 comp rx=0 tx=0).stop 2024-04-16T05:44:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 -- 172.21.15.86:0/167666339 shutdown_connections 2024-04-16T05:44:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/167666339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef023c570 0x55fef02265c0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/167666339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef0233990 0x55fef02481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/167666339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 0x55fef023bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 -- 172.21.15.86:0/167666339 >> 172.21.15.86:0/167666339 conn(0x55fef022d7a0 msgr2=0x55fef022a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 -- 172.21.15.86:0/167666339 shutdown_connections 2024-04-16T05:44:37.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 -- 172.21.15.86:0/167666339 wait complete. 2024-04-16T05:44:37.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.884+0000 7fb2fd04e740 1 -- start start 2024-04-16T05:44:37.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2fd04e740 1 Processor -- start 2024-04-16T05:44:37.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 0x55fef0244490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef02449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2fd04e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef023c570 0x55fef02408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41ae7e0 con 0x55fef0233990 2024-04-16T05:44:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41ae110 con 0x55fef023c570 2024-04-16T05:44:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2fd04e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41ab370 con 0x55fef0233070 2024-04-16T05:44:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2fd04e740 10 client.0 _ll_register_callbacks cb 0x7fb2dc005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef02449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef02449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60858/0 (socket says 172.21.15.86:60858) 2024-04-16T05:44:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f37fe640 1 -- 172.21.15.86:0/2232177880 learned_addr learned my addr 172.21.15.86:0/2232177880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef023c570 0x55fef02408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f37fe640 1 -- 172.21.15.86:0/2232177880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 msgr2=0x55fef0244490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 0x55fef0244490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f37fe640 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 0x55fef0244490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f37fe640 1 -- 172.21.15.86:0/2232177880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef023c570 msgr2=0x55fef02408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:37.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f37fe640 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef023c570 0x55fef02408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f37fe640 1 -- 172.21.15.86:0/2232177880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef02410f0 con 0x55fef0233990 2024-04-16T05:44:37.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 0x55fef0244490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:37.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f8a00640 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef023c570 0x55fef02408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:37.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2f37fe640 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef02449d0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc010420 tx=0x7fb2dc005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443f5d69271f07f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:37.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.885+0000 7fb2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:37.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2d37fe640 1 -- 172.21.15.86:0/2232177880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc015070 con 0x55fef0233990 2024-04-16T05:44:37.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2d37fe640 1 -- 172.21.15.86:0/2232177880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fb2dc00b440 con 0x55fef0233990 2024-04-16T05:44:37.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2fd04e740 1 -- 172.21.15.86:0/2232177880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2ec004020 con 0x55fef0233990 2024-04-16T05:44:37.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2d37fe640 1 -- 172.21.15.86:0/2232177880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2dc00fc30 con 0x55fef0233990 2024-04-16T05:44:37.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2fd04e740 10 client.18366 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:37.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2fd04e740 1 -- 172.21.15.86:0/2232177880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fef02b35e0 con 0x55fef0233990 2024-04-16T05:44:37.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2d37fe640 1 -- 172.21.15.86:0/2232177880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2ec004020 con 0x55fef0233990 2024-04-16T05:44:37.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2d37fe640 1 -- 172.21.15.86:0/2232177880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb2dc0119d0 con 0x55fef0233990 2024-04-16T05:44:37.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2fd04e740 10 client.18366 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:37.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2fd04e740 10 client.18366 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:44:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2fd04e740 20 client.18366 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55fef02b3d50 0x55fef02b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2fd04e740 4 client.18366 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2fd04e740 1 -- 172.21.15.86:0/2232177880 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fef019d970 con 0x55fef02b3d50 2024-04-16T05:44:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.886+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55fef02b3d50 0x55fef02b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2f3fff640 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55fef02b3d50 0x55fef02b6170 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26e6c05dd3241eba server_cookie=48dadded8ed6007b in_seq=0 out_seq=0 2024-04-16T05:44:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2d37fe640 10 client.18366 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:44:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2d37fe640 1 -- 172.21.15.86:0/2232177880 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fef019d970 con 0x55fef02b3d50 2024-04-16T05:44:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2d37fe640 10 client.18366 handle_command_reply: tid=0 2024-04-16T05:44:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 client.18366 shutdown 2024-04-16T05:44:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 -- 172.21.15.86:0/2232177880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 msgr2=0x55fef02449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef02449d0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc010420 tx=0x7fb2dc005ec0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 -- 172.21.15.86:0/2232177880 shutdown_connections 2024-04-16T05:44:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fef023c570 0x55fef02408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef0233990 0x55fef02449d0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55fef02b3d50 0x55fef02b6170 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 --2- 172.21.15.86:0/2232177880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fef0233070 0x55fef0244490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 -- 172.21.15.86:0/2232177880 >> 172.21.15.86:0/2232177880 conn(0x55fef022d7a0 msgr2=0x55fef0239550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2d37fe640 0 client.18366 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:44:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 -- 172.21.15.86:0/2232177880 shutdown_connections 2024-04-16T05:44:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.887+0000 7fb2fd04e740 1 -- 172.21.15.86:0/2232177880 wait complete. 2024-04-16T05:44:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fd04e740 20 client.18366 trim_cache size 0 max 16384 2024-04-16T05:44:37.900 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 -- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb2e00a3d20 msgr2=0x7fb2e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005d50 tx=0x7fb2ec015000 comp rx=0 tx=0).stop 2024-04-16T05:44:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 -- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 msgr2=0x7fb2f419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 0x7fb2f419d460 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4005e30 tx=0x7fb2e4032f20 comp rx=0 tx=0).stop 2024-04-16T05:44:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 -- 172.21.15.86:0/2865334393 shutdown_connections 2024-04-16T05:44:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f4197610 0x7fb2f4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2f4106200 0x7fb2f419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 --2- 172.21.15.86:0/2865334393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f4101130 0x7fb2f419d460 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 -- 172.21.15.86:0/2865334393 >> 172.21.15.86:0/2865334393 conn(0x7fb2f40780b0 msgr2=0x7fb2f40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 -- 172.21.15.86:0/2865334393 shutdown_connections 2024-04-16T05:44:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:37.888+0000 7fb2fa463640 1 -- 172.21.15.86:0/2865334393 wait complete. 2024-04-16T05:44:37.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:44:38.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3249111290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101fe0 msgr2=0x7fe9c410dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3249111290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101fe0 0x7fe9c410dd90 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009900 tx=0x7fe9bc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3249111290 shutdown_connections 2024-04-16T05:44:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3249111290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101fe0 0x7fe9c410dd90 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3249111290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9c4101620 0x7fe9c4101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3249111290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c4105f00 0x7fe9c41062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3249111290 >> 172.21.15.86:0/3249111290 conn(0x7fe9c4077460 msgr2=0x7fe9c40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3249111290 shutdown_connections 2024-04-16T05:44:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3249111290 wait complete. 2024-04-16T05:44:38.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 Processor -- start 2024-04-16T05:44:38.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.013+0000 7fe9c93e8640 1 -- start start 2024-04-16T05:44:38.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c93e8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101620 0x7fe9c4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c93e8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9c4101fe0 0x7fe9c4197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c93e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c4105f00 0x7fe9c419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c93e8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4111280 con 0x7fe9c4101620 2024-04-16T05:44:38.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c93e8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4111100 con 0x7fe9c4101fe0 2024-04-16T05:44:38.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c93e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4111400 con 0x7fe9c4105f00 2024-04-16T05:44:38.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101620 0x7fe9c4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101620 0x7fe9c4197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60868/0 (socket says 172.21.15.86:60868) 2024-04-16T05:44:38.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c2ffd640 1 -- 172.21.15.86:0/3900499244 learned_addr learned my addr 172.21.15.86:0/3900499244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c27fc640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9c4101fe0 0x7fe9c4197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c4105f00 0x7fe9c419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c2ffd640 1 -- 172.21.15.86:0/3900499244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c4105f00 msgr2=0x7fe9c419bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c4105f00 0x7fe9c419bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c2ffd640 1 -- 172.21.15.86:0/3900499244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9c4101fe0 msgr2=0x7fe9c4197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9c4101fe0 0x7fe9c4197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c2ffd640 1 -- 172.21.15.86:0/3900499244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c419c6e0 con 0x7fe9c4101620 2024-04-16T05:44:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101620 0x7fe9c4197740 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400cce0 tx=0x7fe9b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9bbfff640 1 -- 172.21.15.86:0/3900499244 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4004b20 con 0x7fe9c4101620 2024-04-16T05:44:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9bbfff640 1 -- 172.21.15.86:0/3900499244 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b4004cc0 con 0x7fe9c4101620 2024-04-16T05:44:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3900499244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c419c9d0 con 0x7fe9c4101620 2024-04-16T05:44:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.014+0000 7fe9bbfff640 1 -- 172.21.15.86:0/3900499244 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b400f8c0 con 0x7fe9c4101620 2024-04-16T05:44:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.015+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3900499244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c41a4260 con 0x7fe9c4101620 2024-04-16T05:44:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.017+0000 7fe9bbfff640 1 -- 172.21.15.86:0/3900499244 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9b400fa60 con 0x7fe9c4101620 2024-04-16T05:44:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.017+0000 7fe9cbfcc740 1 Processor -- start 2024-04-16T05:44:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.017+0000 7fe9cbfcc740 1 -- start start 2024-04-16T05:44:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dc2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 0x55e8f7dce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7dd0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8f7d280e0 con 0x55e8f7dc2b20 2024-04-16T05:44:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e8f7cc5520 con 0x55e8f7dc3440 2024-04-16T05:44:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8f7d240a0 con 0x55e8f7dce570 2024-04-16T05:44:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9bbfff640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dc2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dc2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60882/0 (socket says 172.21.15.86:60882) 2024-04-16T05:44:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 -- 172.21.15.86:0/3530595752 learned_addr learned my addr 172.21.15.86:0/3530595752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c27fc640 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7dd0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9bbfff640 1 -- 172.21.15.86:0/3900499244 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fe9b40d0c30 con 0x7fe9c4101620 2024-04-16T05:44:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 0x55e8f7dce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 -- 172.21.15.86:0/3530595752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 msgr2=0x55e8f7dd0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7dd0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 -- 172.21.15.86:0/3530595752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 msgr2=0x55e8f7dce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 0x55e8f7dce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 -- 172.21.15.86:0/3530595752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f7ced8d0 con 0x55e8f7dc2b20 2024-04-16T05:44:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c27fc640 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7dd0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dc2f00 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc035000 tx=0x7fe9bc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e88a7ece16f848a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.018+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc02c9b0 tx=0x7fe9bc03a000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9b97fa640 1 -- 172.21.15.86:0/3530595752 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc02c2b0 con 0x55e8f7dc2b20 2024-04-16T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9b97fa640 1 -- 172.21.15.86:0/3530595752 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9bc050070 con 0x55e8f7dc2b20 2024-04-16T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9b97fa640 1 -- 172.21.15.86:0/3530595752 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc043c30 con 0x55e8f7dc2b20 2024-04-16T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3530595752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 msgr2=0x55e8f7dc2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dc2f00 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc035000 tx=0x7fe9bc004610 comp rx=0 tx=0).stop 2024-04-16T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3530595752 shutdown_connections 2024-04-16T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7dd0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 0x55e8f7dce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/3530595752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dc2f00 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3530595752 >> 172.21.15.86:0/3530595752 conn(0x55e8f7dbd590 msgr2=0x55e8f7dbd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.019+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3530595752 shutdown_connections 2024-04-16T05:44:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3530595752 wait complete. 2024-04-16T05:44:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 1 -- start start 2024-04-16T05:44:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 1 Processor -- start 2024-04-16T05:44:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 0x55e8f7dcbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7dcc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8f7dd1300 con 0x55e8f7dce570 2024-04-16T05:44:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e8f7ce14d0 con 0x55e8f7dc2b20 2024-04-16T05:44:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8f7dd1480 con 0x55e8f7dc3440 2024-04-16T05:44:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9cbfcc740 10 client.0 _ll_register_callbacks cb 0x55e8f7cc62c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7dcc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7dcc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60886/0 (socket says 172.21.15.86:60886) 2024-04-16T05:44:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9c37fe640 1 -- 172.21.15.86:0/947337446 learned_addr learned my addr 172.21.15.86:0/947337446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9c27fc640 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 0x55e8f7dcbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9c37fe640 1 -- 172.21.15.86:0/947337446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 msgr2=0x55e8f7dcbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 0x55e8f7dcbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9c37fe640 1 -- 172.21.15.86:0/947337446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 msgr2=0x55e8f7dcb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.020+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9c37fe640 1 -- 172.21.15.86:0/947337446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f7a3a3f0 con 0x55e8f7dce570 2024-04-16T05:44:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7dcc050 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc02bc70 tx=0x7fe9bc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b88409bee734175 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/947337446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc059060 con 0x55e8f7dce570 2024-04-16T05:44:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/947337446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fe9bc043bc0 con 0x55e8f7dce570 2024-04-16T05:44:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/947337446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9bc050070 con 0x55e8f7dce570 2024-04-16T05:44:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/947337446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9bc02c2b0 con 0x55e8f7dce570 2024-04-16T05:44:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9cbfcc740 10 client.18378 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/947337446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e8f7b047d0 con 0x55e8f7dce570 2024-04-16T05:44:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/947337446 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9bc050070 con 0x55e8f7dce570 2024-04-16T05:44:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/947337446 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe9bc004ae0 con 0x55e8f7dce570 2024-04-16T05:44:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9cbfcc740 10 client.18378 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9cbfcc740 10 client.18378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:44:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9cbfcc740 20 client.18378 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e8f7dde400 0x55e8f7de07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9cbfcc740 4 client.18378 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.021+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/947337446 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8f7d35b00 con 0x55e8f7dde400 2024-04-16T05:44:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.022+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e8f7dde400 0x55e8f7de07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.022+0000 7fe9aa7fc640 10 client.18378 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:44:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.022+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e8f7dde400 0x55e8f7de07f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2feabecef9b72d6 server_cookie=b38a232a15ef6d7e in_seq=0 out_seq=0 2024-04-16T05:44:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/947337446 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8f7d35b00 con 0x55e8f7dde400 2024-04-16T05:44:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9aa7fc640 10 client.18378 handle_command_reply: tid=0 2024-04-16T05:44:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 client.18378 shutdown 2024-04-16T05:44:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/947337446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 msgr2=0x55e8f7dcc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7dcc050 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc02bc70 tx=0x7fe9bc005a20 comp rx=0 tx=0).stop 2024-04-16T05:44:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/947337446 shutdown_connections 2024-04-16T05:44:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e8f7dde400 0x55e8f7de07f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7dcc050 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 0x55e8f7dcbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9aa7fc640 0 client.18378 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:44:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/947337446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/947337446 >> 172.21.15.86:0/947337446 conn(0x55e8f7dbd590 msgr2=0x55e8f7dbf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/947337446 shutdown_connections 2024-04-16T05:44:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/947337446 wait complete. 2024-04-16T05:44:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.023+0000 7fe9cbfcc740 20 client.18378 trim_cache size 0 max 16384 2024-04-16T05:44:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.036+0000 7fe9cbfcc740 1 Processor -- start 2024-04-16T05:44:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.036+0000 7fe9cbfcc740 1 -- start start 2024-04-16T05:44:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 0x55e8f7dcbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7de3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41ad350 con 0x55e8f7dc2b20 2024-04-16T05:44:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41aee90 con 0x55e8f7dc3440 2024-04-16T05:44:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41ada20 con 0x55e8f7dce570 2024-04-16T05:44:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60898/0 (socket says 172.21.15.86:60898) 2024-04-16T05:44:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c2ffd640 1 -- 172.21.15.86:0/3390056256 learned_addr learned my addr 172.21.15.86:0/3390056256 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c27fc640 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 0x55e8f7dcbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7de3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c2ffd640 1 -- 172.21.15.86:0/3390056256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 msgr2=0x55e8f7de3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7de3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c2ffd640 1 -- 172.21.15.86:0/3390056256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 msgr2=0x55e8f7dcbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 0x55e8f7dcbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c2ffd640 1 -- 172.21.15.86:0/3390056256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f7dc71a0 con 0x55e8f7dc2b20 2024-04-16T05:44:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7de3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcdd00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40ce5b0 tx=0x7fe9b40983c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24d64f6850c066d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9b8ff9640 1 -- 172.21.15.86:0/3390056256 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b401d040 con 0x55e8f7dc2b20 2024-04-16T05:44:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9b8ff9640 1 -- 172.21.15.86:0/3390056256 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b409c800 con 0x55e8f7dc2b20 2024-04-16T05:44:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.037+0000 7fe9b8ff9640 1 -- 172.21.15.86:0/3390056256 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40870b0 con 0x55e8f7dc2b20 2024-04-16T05:44:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3390056256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 msgr2=0x55e8f7dcdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcdd00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40ce5b0 tx=0x7fe9b40983c0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3390056256 shutdown_connections 2024-04-16T05:44:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dce570 0x55e8f7de3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc3440 0x55e8f7dcbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/3390056256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dc2b20 0x55e8f7dcdd00 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3390056256 >> 172.21.15.86:0/3390056256 conn(0x55e8f7dbd590 msgr2=0x55e8f7dbc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3390056256 shutdown_connections 2024-04-16T05:44:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/3390056256 wait complete. 2024-04-16T05:44:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 -- start start 2024-04-16T05:44:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.038+0000 7fe9cbfcc740 1 Processor -- start 2024-04-16T05:44:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 0x55e8f7de3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 0x55e8f7de1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7de1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41ae7c0 con 0x55e8f7dce570 2024-04-16T05:44:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41ae0f0 con 0x55e8f7dc2b20 2024-04-16T05:44:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41ab350 con 0x55e8f7dc3440 2024-04-16T05:44:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 10 client.0 _ll_register_callbacks cb 0x7fe9b40072a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7de1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7de1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60906/0 (socket says 172.21.15.86:60906) 2024-04-16T05:44:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c37fe640 1 -- 172.21.15.86:0/2234534296 learned_addr learned my addr 172.21.15.86:0/2234534296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 0x55e8f7de3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c27fc640 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 0x55e8f7de1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c37fe640 1 -- 172.21.15.86:0/2234534296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 msgr2=0x55e8f7de1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 0x55e8f7de1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c37fe640 1 -- 172.21.15.86:0/2234534296 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 msgr2=0x55e8f7de3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 0x55e8f7de3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c37fe640 1 -- 172.21.15.86:0/2234534296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f7de1ea0 con 0x55e8f7dce570 2024-04-16T05:44:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 0x55e8f7de3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9c37fe640 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7de1670 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc02f020 tx=0x7fe9bc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462f2aebec9a7acf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/2234534296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc047020 con 0x55e8f7dce570 2024-04-16T05:44:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/2234534296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fe9bc0042d0 con 0x55e8f7dce570 2024-04-16T05:44:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/2234534296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b409c800 con 0x55e8f7dce570 2024-04-16T05:44:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/2234534296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9bc069050 con 0x55e8f7dce570 2024-04-16T05:44:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 10 client.18384 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.039+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/2234534296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e8f7e44650 con 0x55e8f7dce570 2024-04-16T05:44:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.040+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/2234534296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b409c800 con 0x55e8f7dce570 2024-04-16T05:44:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.040+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/2234534296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe9bc0053f0 con 0x55e8f7dce570 2024-04-16T05:44:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.040+0000 7fe9cbfcc740 10 client.18384 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.040+0000 7fe9cbfcc740 10 client.18384 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:44:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.040+0000 7fe9cbfcc740 20 client.18384 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.040+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e8f7e44df0 0x55e8f7e47210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.040+0000 7fe9cbfcc740 4 client.18384 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.040+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/2234534296 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8f7d2f970 con 0x55e8f7e44df0 2024-04-16T05:44:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.040+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e8f7e44df0 0x55e8f7e47210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9aa7fc640 10 client.18384 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:44:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9c2ffd640 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e8f7e44df0 0x55e8f7e47210 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2e700a36c1da50e server_cookie=ed45f64a6345168e in_seq=0 out_seq=0 2024-04-16T05:44:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9aa7fc640 1 -- 172.21.15.86:0/2234534296 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8f7d2f970 con 0x55e8f7e44df0 2024-04-16T05:44:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9aa7fc640 10 client.18384 handle_command_reply: tid=0 2024-04-16T05:44:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 client.18384 shutdown 2024-04-16T05:44:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/2234534296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 msgr2=0x55e8f7de1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7de1670 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc02f020 tx=0x7fe9bc003cd0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/2234534296 shutdown_connections 2024-04-16T05:44:38.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55e8f7e44df0 0x55e8f7e47210 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f7dce570 0x55e8f7de1670 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9aa7fc640 0 client.18384 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:44:38.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f7dc3440 0x55e8f7de1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 --2- 172.21.15.86:0/2234534296 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8f7dc2b20 0x55e8f7de3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/2234534296 >> 172.21.15.86:0/2234534296 conn(0x55e8f7dbd590 msgr2=0x55e8f7dbbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/2234534296 shutdown_connections 2024-04-16T05:44:38.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 1 -- 172.21.15.86:0/2234534296 wait complete. 2024-04-16T05:44:38.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.041+0000 7fe9cbfcc740 20 client.18384 trim_cache size 0 max 16384 2024-04-16T05:44:38.051 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:38.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe9b00a3d20 msgr2=0x7fe9b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc02c9b0 tx=0x7fe9bc03a000 comp rx=0 tx=0).stop 2024-04-16T05:44:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101620 msgr2=0x7fe9c4197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101620 0x7fe9c4197740 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400cce0 tx=0x7fe9b400a620 comp rx=0 tx=0).stop 2024-04-16T05:44:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3900499244 shutdown_connections 2024-04-16T05:44:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c4105f00 0x7fe9c419bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9c4101fe0 0x7fe9c4197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 --2- 172.21.15.86:0/3900499244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c4101620 0x7fe9c4197740 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3900499244 >> 172.21.15.86:0/3900499244 conn(0x7fe9c4077460 msgr2=0x7fe9c4103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3900499244 shutdown_connections 2024-04-16T05:44:38.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.042+0000 7fe9c93e8640 1 -- 172.21.15.86:0/3900499244 wait complete. 2024-04-16T05:44:38.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:44:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.162+0000 7f96c78c8640 1 -- 172.21.15.86:0/3285662584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0101160 msgr2=0x7f96c0101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.162+0000 7f96c78c8640 1 --2- 172.21.15.86:0/3285662584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0101160 0x7f96c0101540 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009900 tx=0x7f96b402bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.162+0000 7f96c78c8640 1 -- 172.21.15.86:0/3285662584 shutdown_connections 2024-04-16T05:44:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.162+0000 7f96c78c8640 1 --2- 172.21.15.86:0/3285662584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96c0106230 0x7f96c010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.162+0000 7f96c78c8640 1 --2- 172.21.15.86:0/3285662584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96c0101b10 0x7f96c0105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.162+0000 7f96c78c8640 1 --2- 172.21.15.86:0/3285662584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0101160 0x7f96c0101540 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.162+0000 7f96c78c8640 1 -- 172.21.15.86:0/3285662584 >> 172.21.15.86:0/3285662584 conn(0x7f96c0077280 msgr2=0x7f96c0077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.162+0000 7f96c78c8640 1 -- 172.21.15.86:0/3285662584 shutdown_connections 2024-04-16T05:44:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.162+0000 7f96c78c8640 1 -- 172.21.15.86:0/3285662584 wait complete. 2024-04-16T05:44:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c78c8640 1 Processor -- start 2024-04-16T05:44:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c78c8640 1 -- start start 2024-04-16T05:44:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c78c8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96c0101160 0x7f96c019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c78c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96c0106230 0x7f96c019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c78c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0198600 0x7f96c0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c78c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0112400 con 0x7f96c0198600 2024-04-16T05:44:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c78c8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0112280 con 0x7f96c0101160 2024-04-16T05:44:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c78c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0112580 con 0x7f96c0106230 2024-04-16T05:44:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5e65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0198600 0x7f96c0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5e65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0198600 0x7f96c0198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60922/0 (socket says 172.21.15.86:60922) 2024-04-16T05:44:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5e65640 1 -- 172.21.15.86:0/2353270322 learned_addr learned my addr 172.21.15.86:0/2353270322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5664640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96c0101160 0x7f96c019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5e65640 1 -- 172.21.15.86:0/2353270322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96c0106230 msgr2=0x7f96c019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c4e63640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96c0106230 0x7f96c019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5e65640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96c0106230 0x7f96c019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5e65640 1 -- 172.21.15.86:0/2353270322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96c0101160 msgr2=0x7f96c019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5e65640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96c0101160 0x7f96c019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5e65640 1 -- 172.21.15.86:0/2353270322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c0199120 con 0x7f96c0198600 2024-04-16T05:44:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.163+0000 7f96c5664640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96c0101160 0x7f96c019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.164+0000 7f96c5e65640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0198600 0x7f96c0198ab0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f96b800eaa0 tx=0x7f96b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.164+0000 7f96b27fc640 1 -- 172.21.15.86:0/2353270322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b800cdb0 con 0x7f96c0198600 2024-04-16T05:44:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.164+0000 7f96b27fc640 1 -- 172.21.15.86:0/2353270322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b8005020 con 0x7f96c0198600 2024-04-16T05:44:38.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.164+0000 7f96c78c8640 1 -- 172.21.15.86:0/2353270322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c0199410 con 0x7f96c0198600 2024-04-16T05:44:38.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.164+0000 7f96b27fc640 1 -- 172.21.15.86:0/2353270322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b80116b0 con 0x7f96c0198600 2024-04-16T05:44:38.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.164+0000 7f96c78c8640 1 -- 172.21.15.86:0/2353270322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c01a5610 con 0x7f96c0198600 2024-04-16T05:44:38.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.166+0000 7f96b27fc640 1 -- 172.21.15.86:0/2353270322 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f96b8005560 con 0x7f96c0198600 2024-04-16T05:44:38.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.166+0000 7f96ca58d740 1 Processor -- start 2024-04-16T05:44:38.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96ca58d740 1 -- start start 2024-04-16T05:44:38.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96b27fc640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f96ac0a3d20 0x7f96ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974b9b10 0x55db974b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 0x55db974c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96ca58d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db9741f0e0 con 0x55db974ba430 2024-04-16T05:44:38.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96ca58d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55db973bc520 con 0x55db974c7770 2024-04-16T05:44:38.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96ca58d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db9741b0a0 con 0x55db974b9b10 2024-04-16T05:44:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f96ac0a3d20 0x7f96ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96b27fc640 1 -- 172.21.15.86:0/2353270322 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f96b80d0e60 con 0x7f96c0198600 2024-04-16T05:44:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60926/0 (socket says 172.21.15.86:60926) 2024-04-16T05:44:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 -- 172.21.15.86:0/939163855 learned_addr learned my addr 172.21.15.86:0/939163855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c5e65640 1 --2- 172.21.15.86:0/939163855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 0x55db974c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c5664640 1 --2- 172.21.15.86:0/939163855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974b9b10 0x55db974b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 -- 172.21.15.86:0/939163855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974b9b10 msgr2=0x55db974b9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 --2- 172.21.15.86:0/939163855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974b9b10 0x55db974b9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 -- 172.21.15.86:0/939163855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 msgr2=0x55db974c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 --2- 172.21.15.86:0/939163855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 0x55db974c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 -- 172.21.15.86:0/939163855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db973e48d0 con 0x55db974ba430 2024-04-16T05:44:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.167+0000 7f96c4e63640 1 --2- 172.21.15.86:0/939163855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974c7230 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f96bc012a80 tx=0x7f96bc012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b339430d1bdfcdc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96c4e63640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f96ac0a3d20 0x7f96ac0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00f7f0 tx=0x7f96bc01a000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ab7fe640 1 -- 172.21.15.86:0/939163855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc020070 con 0x55db974ba430 2024-04-16T05:44:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ab7fe640 1 -- 172.21.15.86:0/939163855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96bc012450 con 0x55db974ba430 2024-04-16T05:44:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ab7fe640 1 -- 172.21.15.86:0/939163855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc01c8b0 con 0x55db974ba430 2024-04-16T05:44:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ca58d740 1 -- 172.21.15.86:0/939163855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 msgr2=0x55db974c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ca58d740 1 --2- 172.21.15.86:0/939163855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974c7230 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f96bc012a80 tx=0x7f96bc012f40 comp rx=0 tx=0).stop 2024-04-16T05:44:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ca58d740 1 -- 172.21.15.86:0/939163855 shutdown_connections 2024-04-16T05:44:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ca58d740 1 --2- 172.21.15.86:0/939163855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 0x55db974c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ca58d740 1 --2- 172.21.15.86:0/939163855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974c7230 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ca58d740 1 --2- 172.21.15.86:0/939163855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974b9b10 0x55db974b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ca58d740 1 -- 172.21.15.86:0/939163855 >> 172.21.15.86:0/939163855 conn(0x55db974b4590 msgr2=0x55db974b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ca58d740 1 -- 172.21.15.86:0/939163855 shutdown_connections 2024-04-16T05:44:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.168+0000 7f96ca58d740 1 -- 172.21.15.86:0/939163855 wait complete. 2024-04-16T05:44:38.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96ca58d740 1 -- start start 2024-04-16T05:44:38.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96ca58d740 1 Processor -- start 2024-04-16T05:44:38.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974ba430 0x55db974c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 0x55db974c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96ca58d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db974ca550 con 0x55db974b9b10 2024-04-16T05:44:38.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96ca58d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55db973d84d0 con 0x55db974c7770 2024-04-16T05:44:38.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96ca58d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db974ca6d0 con 0x55db974ba430 2024-04-16T05:44:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96ca58d740 10 client.0 _ll_register_callbacks cb 0x55db973bd2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96c5664640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96c5664640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60928/0 (socket says 172.21.15.86:60928) 2024-04-16T05:44:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96c5664640 1 -- 172.21.15.86:0/3736608284 learned_addr learned my addr 172.21.15.86:0/3736608284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96c4e63640 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974ba430 0x55db974c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.169+0000 7f96c5e65640 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 0x55db974c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96c5664640 1 -- 172.21.15.86:0/3736608284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974ba430 msgr2=0x55db974c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96c5664640 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974ba430 0x55db974c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96c5664640 1 -- 172.21.15.86:0/3736608284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 msgr2=0x55db974c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96c5664640 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 0x55db974c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96c5664640 1 -- 172.21.15.86:0/3736608284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db971313f0 con 0x55db974b9b10 2024-04-16T05:44:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96c5664640 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974c47d0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f96b402c170 tx=0x7f96b40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6768f3c69952bef server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96a8ff9640 1 -- 172.21.15.86:0/3736608284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4005230 con 0x55db974b9b10 2024-04-16T05:44:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96ca58d740 1 -- 172.21.15.86:0/3736608284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96bc012450 con 0x55db974b9b10 2024-04-16T05:44:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96a8ff9640 1 -- 172.21.15.86:0/3736608284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f96b40053d0 con 0x55db974b9b10 2024-04-16T05:44:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96a8ff9640 1 -- 172.21.15.86:0/3736608284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96b404e050 con 0x55db974b9b10 2024-04-16T05:44:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96ca58d740 10 client.18396 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96ca58d740 1 -- 172.21.15.86:0/3736608284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55db971fb7d0 con 0x55db974b9b10 2024-04-16T05:44:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96a8ff9640 1 -- 172.21.15.86:0/3736608284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96bc012450 con 0x55db974b9b10 2024-04-16T05:44:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96a8ff9640 1 -- 172.21.15.86:0/3736608284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f96b4003790 con 0x55db974b9b10 2024-04-16T05:44:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96ca58d740 10 client.18396 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96ca58d740 10 client.18396 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:44:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96ca58d740 20 client.18396 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.170+0000 7f96ca58d740 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55db974d77b0 0x55db974d9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.171+0000 7f96ca58d740 4 client.18396 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.171+0000 7f96ca58d740 1 -- 172.21.15.86:0/3736608284 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db9742cb00 con 0x55db974d77b0 2024-04-16T05:44:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.171+0000 7f96c4e63640 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55db974d77b0 0x55db974d9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.171+0000 7f96a8ff9640 10 client.18396 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:44:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.171+0000 7f96c4e63640 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55db974d77b0 0x55db974d9c30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd2c6591e6d2e808 server_cookie=d864a6658e41784b in_seq=0 out_seq=0 2024-04-16T05:44:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96a8ff9640 1 -- 172.21.15.86:0/3736608284 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db9742cb00 con 0x55db974d77b0 2024-04-16T05:44:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96a8ff9640 10 client.18396 handle_command_reply: tid=0 2024-04-16T05:44:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 client.18396 shutdown 2024-04-16T05:44:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 -- 172.21.15.86:0/3736608284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 msgr2=0x55db974c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974c47d0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f96b402c170 tx=0x7f96b40040c0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 -- 172.21.15.86:0/3736608284 shutdown_connections 2024-04-16T05:44:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55db974d77b0 0x55db974d9c30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974c7770 0x55db974c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974ba430 0x55db974c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96a8ff9640 0 client.18396 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:44:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 --2- 172.21.15.86:0/3736608284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974c47d0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 -- 172.21.15.86:0/3736608284 >> 172.21.15.86:0/3736608284 conn(0x55db974b4590 msgr2=0x55db974b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 -- 172.21.15.86:0/3736608284 shutdown_connections 2024-04-16T05:44:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 1 -- 172.21.15.86:0/3736608284 wait complete. 2024-04-16T05:44:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.172+0000 7f96ca58d740 20 client.18396 trim_cache size 0 max 16384 2024-04-16T05:44:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.185+0000 7f96ca58d740 1 Processor -- start 2024-04-16T05:44:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.185+0000 7f96ca58d740 1 -- start start 2024-04-16T05:44:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.185+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.185+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974ba430 0x55db974d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.185+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 0x55db974c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.185+0000 7f96ca58d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ad170 con 0x55db974b9b10 2024-04-16T05:44:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.185+0000 7f96ca58d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01aecb0 con 0x55db974ba430 2024-04-16T05:44:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.185+0000 7f96ca58d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ad840 con 0x55db974c7770 2024-04-16T05:44:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5664640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5664640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60930/0 (socket says 172.21.15.86:60930) 2024-04-16T05:44:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5664640 1 -- 172.21.15.86:0/175308239 learned_addr learned my addr 172.21.15.86:0/175308239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5e65640 1 --2- 172.21.15.86:0/175308239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 0x55db974c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5664640 1 -- 172.21.15.86:0/175308239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 msgr2=0x55db974c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5664640 1 --2- 172.21.15.86:0/175308239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 0x55db974c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5664640 1 -- 172.21.15.86:0/175308239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974ba430 msgr2=0x55db974d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5664640 1 --2- 172.21.15.86:0/175308239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974ba430 0x55db974d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5664640 1 -- 172.21.15.86:0/175308239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db974ba260 con 0x55db974b9b10 2024-04-16T05:44:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96c5664640 1 --2- 172.21.15.86:0/175308239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974b3970 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f96b402cc20 tx=0x7f96b402cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2141b4063e7550 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96aaffd640 1 -- 172.21.15.86:0/175308239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b40035f0 con 0x55db974b9b10 2024-04-16T05:44:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96aaffd640 1 -- 172.21.15.86:0/175308239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b4005550 con 0x55db974b9b10 2024-04-16T05:44:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.186+0000 7f96aaffd640 1 -- 172.21.15.86:0/175308239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4004920 con 0x55db974b9b10 2024-04-16T05:44:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 -- 172.21.15.86:0/175308239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 msgr2=0x55db974b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 --2- 172.21.15.86:0/175308239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974b3970 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f96b402cc20 tx=0x7f96b402cc50 comp rx=0 tx=0).stop 2024-04-16T05:44:38.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 -- 172.21.15.86:0/175308239 shutdown_connections 2024-04-16T05:44:38.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 --2- 172.21.15.86:0/175308239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 0x55db974c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 --2- 172.21.15.86:0/175308239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974ba430 0x55db974d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 --2- 172.21.15.86:0/175308239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974b9b10 0x55db974b3970 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 -- 172.21.15.86:0/175308239 >> 172.21.15.86:0/175308239 conn(0x55db974b4590 msgr2=0x55db974c5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 -- 172.21.15.86:0/175308239 shutdown_connections 2024-04-16T05:44:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 -- 172.21.15.86:0/175308239 wait complete. 2024-04-16T05:44:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 -- start start 2024-04-16T05:44:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 Processor -- start 2024-04-16T05:44:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974b9b10 0x55db974cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 0x55db974cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ae5e0 con 0x55db974ba430 2024-04-16T05:44:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01adf10 con 0x55db974b9b10 2024-04-16T05:44:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.187+0000 7f96ca58d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ab170 con 0x55db974c7770 2024-04-16T05:44:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96ca58d740 10 client.0 _ll_register_callbacks cb 0x7f96b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c4e63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c4e63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60940/0 (socket says 172.21.15.86:60940) 2024-04-16T05:44:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c4e63640 1 -- 172.21.15.86:0/1565259999 learned_addr learned my addr 172.21.15.86:0/1565259999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c5e65640 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 0x55db974cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c5664640 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974b9b10 0x55db974cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c4e63640 1 -- 172.21.15.86:0/1565259999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 msgr2=0x55db974cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c4e63640 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 0x55db974cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c4e63640 1 -- 172.21.15.86:0/1565259999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974b9b10 msgr2=0x55db974cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c4e63640 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974b9b10 0x55db974cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c4e63640 1 -- 172.21.15.86:0/1565259999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db974cbed0 con 0x55db974ba430 2024-04-16T05:44:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96c4e63640 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974cb120 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f96bc01c480 tx=0x7f96bc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ea66514a20e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96a8ff9640 1 -- 172.21.15.86:0/1565259999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc020040 con 0x55db974ba430 2024-04-16T05:44:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96a8ff9640 1 -- 172.21.15.86:0/1565259999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f96bc030420 con 0x55db974ba430 2024-04-16T05:44:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96ca58d740 1 -- 172.21.15.86:0/1565259999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b4005550 con 0x55db974ba430 2024-04-16T05:44:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96a8ff9640 1 -- 172.21.15.86:0/1565259999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96bc03e050 con 0x55db974ba430 2024-04-16T05:44:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96ca58d740 10 client.18402 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96ca58d740 1 -- 172.21.15.86:0/1565259999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55db9753de10 con 0x55db974ba430 2024-04-16T05:44:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.188+0000 7f96a8ff9640 1 -- 172.21.15.86:0/1565259999 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b4005550 con 0x55db974ba430 2024-04-16T05:44:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.189+0000 7f96a8ff9640 1 -- 172.21.15.86:0/1565259999 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f96bc01b070 con 0x55db974ba430 2024-04-16T05:44:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.189+0000 7f96ca58d740 10 client.18402 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.189+0000 7f96ca58d740 10 client.18402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:44:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.189+0000 7f96ca58d740 20 client.18402 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.189+0000 7f96ca58d740 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55db9753e310 0x55db97540730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.189+0000 7f96ca58d740 4 client.18402 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:38.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.189+0000 7f96ca58d740 1 -- 172.21.15.86:0/1565259999 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db97426970 con 0x55db9753e310 2024-04-16T05:44:38.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.189+0000 7f96c5664640 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55db9753e310 0x55db97540730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96c5664640 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55db9753e310 0x55db97540730 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d46684955b7e3e13 server_cookie=96c41f2ef0a876c7 in_seq=0 out_seq=0 2024-04-16T05:44:38.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96a8ff9640 10 client.18402 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:44:38.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96a8ff9640 1 -- 172.21.15.86:0/1565259999 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db97426970 con 0x55db9753e310 2024-04-16T05:44:38.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96a8ff9640 10 client.18402 handle_command_reply: tid=0 2024-04-16T05:44:38.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 client.18402 shutdown 2024-04-16T05:44:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 -- 172.21.15.86:0/1565259999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 msgr2=0x55db974cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974cb120 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f96bc01c480 tx=0x7f96bc01c4b0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 -- 172.21.15.86:0/1565259999 shutdown_connections 2024-04-16T05:44:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55db9753e310 0x55db97540730 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db974c7770 0x55db974cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db974ba430 0x55db974cb120 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96a8ff9640 0 client.18402 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:44:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 --2- 172.21.15.86:0/1565259999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55db974b9b10 0x55db974cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 -- 172.21.15.86:0/1565259999 >> 172.21.15.86:0/1565259999 conn(0x55db974b4590 msgr2=0x55db974c5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 -- 172.21.15.86:0/1565259999 shutdown_connections 2024-04-16T05:44:38.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 1 -- 172.21.15.86:0/1565259999 wait complete. 2024-04-16T05:44:38.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.190+0000 7f96ca58d740 20 client.18402 trim_cache size 0 max 16384 2024-04-16T05:44:38.202 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:38.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 -- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f96ac0a3d20 msgr2=0x7f96ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f96ac0a3d20 0x7f96ac0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00f7f0 tx=0x7f96bc01a000 comp rx=0 tx=0).stop 2024-04-16T05:44:38.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 -- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0198600 msgr2=0x7f96c0198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0198600 0x7f96c0198ab0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f96b800eaa0 tx=0x7f96b800ef60 comp rx=0 tx=0).stop 2024-04-16T05:44:38.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 -- 172.21.15.86:0/2353270322 shutdown_connections 2024-04-16T05:44:38.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c0198600 0x7f96c0198ab0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96c0106230 0x7f96c019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f96ac0a3d20 0x7f96ac0a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 --2- 172.21.15.86:0/2353270322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96c0101160 0x7f96c019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 -- 172.21.15.86:0/2353270322 >> 172.21.15.86:0/2353270322 conn(0x7f96c0077280 msgr2=0x7f96c00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 -- 172.21.15.86:0/2353270322 shutdown_connections 2024-04-16T05:44:38.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.191+0000 7f96c78c8640 1 -- 172.21.15.86:0/2353270322 wait complete. 2024-04-16T05:44:38.205 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:44:38.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.313+0000 7f330ec35640 1 -- 172.21.15.86:0/1483420954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3308107320 msgr2=0x7f330810eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.313+0000 7f330ec35640 1 --2- 172.21.15.86:0/1483420954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3308107320 0x7f330810eef0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3300009ca0 tx=0x7f330002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.313+0000 7f330ec35640 1 -- 172.21.15.86:0/1483420954 shutdown_connections 2024-04-16T05:44:38.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.313+0000 7f330ec35640 1 --2- 172.21.15.86:0/1483420954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3308107320 0x7f330810eef0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.313+0000 7f330ec35640 1 --2- 172.21.15.86:0/1483420954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3308102c00 0x7f3308106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.313+0000 7f330ec35640 1 --2- 172.21.15.86:0/1483420954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3308102250 0x7f3308102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.313+0000 7f330ec35640 1 -- 172.21.15.86:0/1483420954 >> 172.21.15.86:0/1483420954 conn(0x7f3308079490 msgr2=0x7f33080798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.313+0000 7f330ec35640 1 -- 172.21.15.86:0/1483420954 shutdown_connections 2024-04-16T05:44:38.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.313+0000 7f330ec35640 1 -- 172.21.15.86:0/1483420954 wait complete. 2024-04-16T05:44:38.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330ec35640 1 Processor -- start 2024-04-16T05:44:38.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330ec35640 1 -- start start 2024-04-16T05:44:38.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330ec35640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3308102250 0x7f330819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330ec35640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3308107320 0x7f330819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330ec35640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33081985c0 0x7f3308198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330ec35640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33081123c0 con 0x7f33081985c0 2024-04-16T05:44:38.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330ec35640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3308112240 con 0x7f3308102250 2024-04-16T05:44:38.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330ec35640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3308112540 con 0x7f3308107320 2024-04-16T05:44:38.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330d1d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33081985c0 0x7f3308198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330d1d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33081985c0 0x7f3308198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60958/0 (socket says 172.21.15.86:60958) 2024-04-16T05:44:38.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330d1d2640 1 -- 172.21.15.86:0/3358565077 learned_addr learned my addr 172.21.15.86:0/3358565077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330c9d1640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3308102250 0x7f330819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330d1d2640 1 -- 172.21.15.86:0/3358565077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3308107320 msgr2=0x7f330819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330d1d2640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3308107320 0x7f330819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330d1d2640 1 -- 172.21.15.86:0/3358565077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3308102250 msgr2=0x7f330819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330d1d2640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3308102250 0x7f330819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330d1d2640 1 -- 172.21.15.86:0/3358565077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33081990e0 con 0x7f33081985c0 2024-04-16T05:44:38.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.314+0000 7f330d1d2640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33081985c0 0x7f3308198a70 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f330002c700 tx=0x7f3300032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.315+0000 7f32fdffb640 1 -- 172.21.15.86:0/3358565077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300039040 con 0x7f33081985c0 2024-04-16T05:44:38.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.315+0000 7f32fdffb640 1 -- 172.21.15.86:0/3358565077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3300044070 con 0x7f33081985c0 2024-04-16T05:44:38.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.315+0000 7f330ec35640 1 -- 172.21.15.86:0/3358565077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3308199370 con 0x7f33081985c0 2024-04-16T05:44:38.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.315+0000 7f32fdffb640 1 -- 172.21.15.86:0/3358565077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33000042a0 con 0x7f33081985c0 2024-04-16T05:44:38.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.315+0000 7f330ec35640 1 -- 172.21.15.86:0/3358565077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33081a5560 con 0x7f33081985c0 2024-04-16T05:44:38.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f32fdffb640 1 -- 172.21.15.86:0/3358565077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3300004440 con 0x7f33081985c0 2024-04-16T05:44:38.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f3311824740 1 Processor -- start 2024-04-16T05:44:38.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f3311824740 1 -- start start 2024-04-16T05:44:38.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f3311824740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0761de60 0x558c0761e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f3311824740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f3311824740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c07627360 0x558c07629750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f3311824740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c075830e0 con 0x558c0761e780 2024-04-16T05:44:38.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f3311824740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558c07520520 con 0x558c07627360 2024-04-16T05:44:38.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f3311824740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c0757f0a0 con 0x558c0761de60 2024-04-16T05:44:38.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f330c9d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f330c9d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60964/0 (socket says 172.21.15.86:60964) 2024-04-16T05:44:38.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f330c9d1640 1 -- 172.21.15.86:0/2806321049 learned_addr learned my addr 172.21.15.86:0/2806321049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f32fffff640 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0761de60 0x558c0761e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f32fffff640 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c07627360 0x558c07629750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f32fdffb640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32f00a3d20 0x7f32f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f330c9d1640 1 -- 172.21.15.86:0/2806321049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0761de60 msgr2=0x558c0761e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0761de60 0x558c0761e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f330c9d1640 1 -- 172.21.15.86:0/2806321049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c07627360 msgr2=0x558c07629750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c07627360 0x558c07629750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f330c9d1640 1 -- 172.21.15.86:0/2806321049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c075488d0 con 0x558c0761e780 2024-04-16T05:44:38.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.318+0000 7f32fffff640 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c07627360 0x558c07629750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:38.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.319+0000 7f330d1d2640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32f00a3d20 0x7f32f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.319+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626e20 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f330400bb10 tx=0x7f330400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18d4bbf009d551d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.319+0000 7f32fdffb640 1 -- 172.21.15.86:0/3358565077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f33000f2f40 con 0x7f33081985c0 2024-04-16T05:44:38.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.319+0000 7f330d1d2640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32f00a3d20 0x7f32f00a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f33000f0480 tx=0x7f330002c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.322+0000 7f32eeffd640 1 -- 172.21.15.86:0/2806321049 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304017420 con 0x558c0761e780 2024-04-16T05:44:38.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.322+0000 7f32eeffd640 1 -- 172.21.15.86:0/2806321049 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3304017aa0 con 0x558c0761e780 2024-04-16T05:44:38.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.322+0000 7f32eeffd640 1 -- 172.21.15.86:0/2806321049 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330400faa0 con 0x558c0761e780 2024-04-16T05:44:38.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 -- 172.21.15.86:0/2806321049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 msgr2=0x558c07626e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626e20 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f330400bb10 tx=0x7f330400d340 comp rx=0 tx=0).stop 2024-04-16T05:44:38.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 -- 172.21.15.86:0/2806321049 shutdown_connections 2024-04-16T05:44:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c07627360 0x558c07629750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626e20 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 --2- 172.21.15.86:0/2806321049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0761de60 0x558c0761e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 -- 172.21.15.86:0/2806321049 >> 172.21.15.86:0/2806321049 conn(0x558c07618590 msgr2=0x558c07618970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 -- 172.21.15.86:0/2806321049 shutdown_connections 2024-04-16T05:44:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 -- 172.21.15.86:0/2806321049 wait complete. 2024-04-16T05:44:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 -- start start 2024-04-16T05:44:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 Processor -- start 2024-04-16T05:44:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c07624400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.323+0000 7f3311824740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 0x558c07624940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f3311824740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c07624e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f3311824740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c0762a0c0 con 0x558c0761de60 2024-04-16T05:44:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f3311824740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558c0753c4d0 con 0x558c0761e780 2024-04-16T05:44:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f3311824740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c0762a240 con 0x558c07627360 2024-04-16T05:44:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f3311824740 10 client.0 _ll_register_callbacks cb 0x558c075212c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c07624400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c07624400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60974/0 (socket says 172.21.15.86:60974) 2024-04-16T05:44:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 -- 172.21.15.86:0/2411863319 learned_addr learned my addr 172.21.15.86:0/2411863319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c07624e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 0x558c07624940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 -- 172.21.15.86:0/2411863319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 msgr2=0x558c07624e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c07624e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 -- 172.21.15.86:0/2411863319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 msgr2=0x558c07624940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 0x558c07624940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 -- 172.21.15.86:0/2411863319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c072953f0 con 0x558c0761de60 2024-04-16T05:44:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 0x558c07624940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32fffff640 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c07624400 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f32f800cfa0 tx=0x7f32f8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3d7b0dcbc08846 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32e7fff640 1 -- 172.21.15.86:0/2411863319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f800daa0 con 0x558c0761de60 2024-04-16T05:44:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f3311824740 1 -- 172.21.15.86:0/2411863319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3304017aa0 con 0x558c0761de60 2024-04-16T05:44:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32e7fff640 1 -- 172.21.15.86:0/2411863319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f32f800dc40 con 0x558c0761de60 2024-04-16T05:44:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f32e7fff640 1 -- 172.21.15.86:0/2411863319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32f8026050 con 0x558c0761de60 2024-04-16T05:44:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f3311824740 10 client.18414 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.324+0000 7f3311824740 1 -- 172.21.15.86:0/2411863319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c0735f7d0 con 0x558c0761de60 2024-04-16T05:44:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f32e7fff640 1 -- 172.21.15.86:0/2411863319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3304017aa0 con 0x558c0761de60 2024-04-16T05:44:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f32e7fff640 1 -- 172.21.15.86:0/2411863319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f32f801e4a0 con 0x558c0761de60 2024-04-16T05:44:38.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f3311824740 10 client.18414 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:38.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f3311824740 10 client.18414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:38.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f3311824740 20 client.18414 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:38.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f3311824740 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c07637350 0x558c07639780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f3311824740 4 client.18414 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:38.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f3311824740 1 -- 172.21.15.86:0/2411863319 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c07590b00 con 0x558c07637350 2024-04-16T05:44:38.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c07637350 0x558c07639780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c07637350 0x558c07639780 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4dadf1d87ae381 server_cookie=d3d596b007dd08a7 in_seq=0 out_seq=0 2024-04-16T05:44:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.325+0000 7f32e7fff640 10 client.18414 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f32e7fff640 1 -- 172.21.15.86:0/2411863319 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c07590b00 con 0x558c07637350 2024-04-16T05:44:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f32e7fff640 10 client.18414 handle_command_reply: tid=0 2024-04-16T05:44:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 client.18414 shutdown 2024-04-16T05:44:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 -- 172.21.15.86:0/2411863319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 msgr2=0x558c07624400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c07624400 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f32f800cfa0 tx=0x7f32f8014040 comp rx=0 tx=0).stop 2024-04-16T05:44:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 -- 172.21.15.86:0/2411863319 shutdown_connections 2024-04-16T05:44:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c07637350 0x558c07639780 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c07624e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 0x558c07624940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 --2- 172.21.15.86:0/2411863319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c07624400 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f32e7fff640 0 client.18414 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:38.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 -- 172.21.15.86:0/2411863319 >> 172.21.15.86:0/2411863319 conn(0x558c07618590 msgr2=0x558c0761a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 -- 172.21.15.86:0/2411863319 shutdown_connections 2024-04-16T05:44:38.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.326+0000 7f3311824740 1 -- 172.21.15.86:0/2411863319 wait complete. 2024-04-16T05:44:38.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.327+0000 7f3311824740 20 client.18414 trim_cache size 0 max 16384 2024-04-16T05:44:38.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f3311824740 1 Processor -- start 2024-04-16T05:44:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f3311824740 1 -- start start 2024-04-16T05:44:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f3311824740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761de60 0x558c07633040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f3311824740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f3311824740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c0763f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f3311824740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33081af5b0 con 0x558c0761e780 2024-04-16T05:44:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f3311824740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b10f0 con 0x558c0761de60 2024-04-16T05:44:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f3311824740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33081afc80 con 0x558c07627360 2024-04-16T05:44:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f330c9d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f330c9d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60980/0 (socket says 172.21.15.86:60980) 2024-04-16T05:44:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f330c9d1640 1 -- 172.21.15.86:0/2000850601 learned_addr learned my addr 172.21.15.86:0/2000850601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f330c9d1640 1 -- 172.21.15.86:0/2000850601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 msgr2=0x558c0763f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2000850601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c0763f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f330c9d1640 1 -- 172.21.15.86:0/2000850601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761de60 msgr2=0x558c07633040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2000850601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761de60 0x558c07633040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f330c9d1640 1 -- 172.21.15.86:0/2000850601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c07639e00 con 0x558c0761e780 2024-04-16T05:44:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f330c9d1640 1 --2- 172.21.15.86:0/2000850601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626230 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f330400bb10 tx=0x7f330400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64975fa689b5890e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.340+0000 7f32ee7fc640 1 -- 172.21.15.86:0/2000850601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330402c070 con 0x558c0761e780 2024-04-16T05:44:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f32ee7fc640 1 -- 172.21.15.86:0/2000850601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3304027d50 con 0x558c0761e780 2024-04-16T05:44:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f32ee7fc640 1 -- 172.21.15.86:0/2000850601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33040055d0 con 0x558c0761e780 2024-04-16T05:44:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 -- 172.21.15.86:0/2000850601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 msgr2=0x558c07626230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 --2- 172.21.15.86:0/2000850601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626230 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f330400bb10 tx=0x7f330400d340 comp rx=0 tx=0).stop 2024-04-16T05:44:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 -- 172.21.15.86:0/2000850601 shutdown_connections 2024-04-16T05:44:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 --2- 172.21.15.86:0/2000850601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c0763f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 --2- 172.21.15.86:0/2000850601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761e780 0x558c07626230 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 --2- 172.21.15.86:0/2000850601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761de60 0x558c07633040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 -- 172.21.15.86:0/2000850601 >> 172.21.15.86:0/2000850601 conn(0x558c07618590 msgr2=0x558c07625b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 -- 172.21.15.86:0/2000850601 shutdown_connections 2024-04-16T05:44:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 -- 172.21.15.86:0/2000850601 wait complete. 2024-04-16T05:44:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 -- start start 2024-04-16T05:44:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.341+0000 7f3311824740 1 Processor -- start 2024-04-16T05:44:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f3311824740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c0762f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f3311824740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 0x558c0762ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f3311824740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c0762b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f3311824740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b0a20 con 0x558c0761de60 2024-04-16T05:44:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f3311824740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b0350 con 0x558c0761e780 2024-04-16T05:44:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f3311824740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33081ad5b0 con 0x558c07627360 2024-04-16T05:44:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f3311824740 10 client.0 _ll_register_callbacks cb 0x7f32f800c660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c0762f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c0762f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60984/0 (socket says 172.21.15.86:60984) 2024-04-16T05:44:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32fffff640 1 -- 172.21.15.86:0/975508117 learned_addr learned my addr 172.21.15.86:0/975508117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32fffff640 1 -- 172.21.15.86:0/975508117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 msgr2=0x558c0762b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32fffff640 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c0762b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32fffff640 1 -- 172.21.15.86:0/975508117 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 msgr2=0x558c0762ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32fffff640 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 0x558c0762ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32fffff640 1 -- 172.21.15.86:0/975508117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0762b610 con 0x558c0761de60 2024-04-16T05:44:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32fffff640 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c0762f680 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f32f8005cb0 tx=0x7f32f80126b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93dc4d1079201b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32e7fff640 1 -- 172.21.15.86:0/975508117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f802f060 con 0x558c0761de60 2024-04-16T05:44:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.342+0000 7f32e7fff640 1 -- 172.21.15.86:0/975508117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f32f801eca0 con 0x558c0761de60 2024-04-16T05:44:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f3311824740 1 -- 172.21.15.86:0/975508117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3304027d50 con 0x558c0761de60 2024-04-16T05:44:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f32e7fff640 1 -- 172.21.15.86:0/975508117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32f8036050 con 0x558c0761de60 2024-04-16T05:44:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f3311824740 10 client.18420 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f3311824740 1 -- 172.21.15.86:0/975508117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c0769d6e0 con 0x558c0761de60 2024-04-16T05:44:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f32e7fff640 1 -- 172.21.15.86:0/975508117 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3304027d50 con 0x558c0761de60 2024-04-16T05:44:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f32e7fff640 1 -- 172.21.15.86:0/975508117 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f32f802a6f0 con 0x558c0761de60 2024-04-16T05:44:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f3311824740 10 client.18420 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f3311824740 10 client.18420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f3311824740 20 client.18420 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f3311824740 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c0769de90 0x558c076a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f3311824740 4 client.18420 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f3311824740 1 -- 172.21.15.86:0/975508117 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c0758a970 con 0x558c0769de90 2024-04-16T05:44:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.343+0000 7f330c9d1640 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c0769de90 0x558c076a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f32e7fff640 10 client.18420 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f330c9d1640 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c0769de90 0x558c076a02d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4666823ece48f060 server_cookie=af4fc4817e9fd2f3 in_seq=0 out_seq=0 2024-04-16T05:44:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f32e7fff640 1 -- 172.21.15.86:0/975508117 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c0758a970 con 0x558c0769de90 2024-04-16T05:44:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f32e7fff640 10 client.18420 handle_command_reply: tid=0 2024-04-16T05:44:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 client.18420 shutdown 2024-04-16T05:44:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 -- 172.21.15.86:0/975508117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 msgr2=0x558c0762f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c0762f680 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f32f8005cb0 tx=0x7f32f80126b0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 -- 172.21.15.86:0/975508117 shutdown_connections 2024-04-16T05:44:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x558c0769de90 0x558c076a02d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c07627360 0x558c0762b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558c0761e780 0x558c0762ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f32e7fff640 0 client.18420 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 --2- 172.21.15.86:0/975508117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c0761de60 0x558c0762f680 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 -- 172.21.15.86:0/975508117 >> 172.21.15.86:0/975508117 conn(0x558c07618590 msgr2=0x558c07617730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 -- 172.21.15.86:0/975508117 shutdown_connections 2024-04-16T05:44:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 1 -- 172.21.15.86:0/975508117 wait complete. 2024-04-16T05:44:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.344+0000 7f3311824740 20 client.18420 trim_cache size 0 max 16384 2024-04-16T05:44:38.353 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:44:38.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 -- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32f00a3d20 msgr2=0x7f32f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32f00a3d20 0x7f32f00a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f33000f0480 tx=0x7f330002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 -- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33081985c0 msgr2=0x7f3308198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:38.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33081985c0 0x7f3308198a70 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f330002c700 tx=0x7f3300032f20 comp rx=0 tx=0).stop 2024-04-16T05:44:38.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 -- 172.21.15.86:0/3358565077 shutdown_connections 2024-04-16T05:44:38.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33081985c0 0x7f3308198a70 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3308107320 0x7f330819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f32f00a3d20 0x7f32f00a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 --2- 172.21.15.86:0/3358565077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3308102250 0x7f330819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:38.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 -- 172.21.15.86:0/3358565077 >> 172.21.15.86:0/3358565077 conn(0x7f3308079490 msgr2=0x7f33080ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:38.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 -- 172.21.15.86:0/3358565077 shutdown_connections 2024-04-16T05:44:38.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:38.345+0000 7f330ec35640 1 -- 172.21.15.86:0/3358565077 wait complete. 2024-04-16T05:44:38.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:38 smithi086 ceph-mon[32028]: pgmap v289: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:38.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:38 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2899552664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:38.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:38 smithi194 ceph-mon[35865]: pgmap v289: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:38.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:38 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2899552664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:38.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:38 smithi196 ceph-mon[36373]: pgmap v289: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:38.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:38 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2899552664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:39.356 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:44:39.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:44:39.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:44:39.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.479+0000 7f4ab660d640 1 -- 172.21.15.86:0/1400080966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0106fe0 msgr2=0x7f4ab01073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.479+0000 7f4ab660d640 1 --2- 172.21.15.86:0/1400080966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0106fe0 0x7f4ab01073c0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009930 tx=0x7f4aa002bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:39.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.479+0000 7f4ab660d640 1 -- 172.21.15.86:0/1400080966 shutdown_connections 2024-04-16T05:44:39.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.479+0000 7f4ab660d640 1 --2- 172.21.15.86:0/1400080966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ab0100ec0 0x7f4ab010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.479+0000 7f4ab660d640 1 --2- 172.21.15.86:0/1400080966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ab0100500 0x7f4ab0100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.479+0000 7f4ab660d640 1 --2- 172.21.15.86:0/1400080966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0106fe0 0x7f4ab01073c0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.479+0000 7f4ab660d640 1 -- 172.21.15.86:0/1400080966 >> 172.21.15.86:0/1400080966 conn(0x7f4ab0077490 msgr2=0x7f4ab00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.479+0000 7f4ab660d640 1 -- 172.21.15.86:0/1400080966 shutdown_connections 2024-04-16T05:44:39.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.479+0000 7f4ab660d640 1 -- 172.21.15.86:0/1400080966 wait complete. 2024-04-16T05:44:39.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.480+0000 7f4ab660d640 1 Processor -- start 2024-04-16T05:44:39.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.480+0000 7f4ab660d640 1 -- start start 2024-04-16T05:44:39.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.480+0000 7f4ab660d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0100500 0x7f4ab019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.480+0000 7f4ab660d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ab0100ec0 0x7f4ab019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.480+0000 7f4ab660d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ab0106fe0 0x7f4ab01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.480+0000 7f4ab660d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0112470 con 0x7f4ab0100500 2024-04-16T05:44:39.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.480+0000 7f4ab660d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01122f0 con 0x7f4ab0100ec0 2024-04-16T05:44:39.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.480+0000 7f4ab660d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01125f0 con 0x7f4ab0106fe0 2024-04-16T05:44:39.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0100500 0x7f4ab019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0100500 0x7f4ab019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32768/0 (socket says 172.21.15.86:32768) 2024-04-16T05:44:39.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaffff640 1 -- 172.21.15.86:0/4085271366 learned_addr learned my addr 172.21.15.86:0/4085271366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaf7fe640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ab0100ec0 0x7f4ab019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4ab4baa640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ab0106fe0 0x7f4ab01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaffff640 1 -- 172.21.15.86:0/4085271366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ab0106fe0 msgr2=0x7f4ab01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaffff640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ab0106fe0 0x7f4ab01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaffff640 1 -- 172.21.15.86:0/4085271366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ab0100ec0 msgr2=0x7f4ab019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaffff640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ab0100ec0 0x7f4ab019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaffff640 1 -- 172.21.15.86:0/4085271366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab0198fa0 con 0x7f4ab0100500 2024-04-16T05:44:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4ab4baa640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ab0106fe0 0x7f4ab01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aaffff640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0100500 0x7f4ab019e610 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0004f50 tx=0x7f4aa002ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aad7fa640 1 -- 172.21.15.86:0/4085271366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0041070 con 0x7f4ab0100500 2024-04-16T05:44:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4ab660d640 1 -- 172.21.15.86:0/4085271366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab0199230 con 0x7f4ab0100500 2024-04-16T05:44:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aad7fa640 1 -- 172.21.15.86:0/4085271366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa00044d0 con 0x7f4ab0100500 2024-04-16T05:44:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4aad7fa640 1 -- 172.21.15.86:0/4085271366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0034440 con 0x7f4ab0100500 2024-04-16T05:44:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.481+0000 7f4ab660d640 1 -- 172.21.15.86:0/4085271366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab01a5460 con 0x7f4ab0100500 2024-04-16T05:44:39.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.484+0000 7f4aad7fa640 1 -- 172.21.15.86:0/4085271366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4aa0044070 con 0x7f4ab0100500 2024-04-16T05:44:39.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.484+0000 7f4ab91f7740 1 Processor -- start 2024-04-16T05:44:39.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.484+0000 7f4ab91f7740 1 -- start start 2024-04-16T05:44:39.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341009e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341012a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 0x55e3410153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e340f6ee90 con 0x55e34100a3d0 2024-04-16T05:44:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e340f0c4c0 con 0x55e341009ab0 2024-04-16T05:44:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e340f6ae50 con 0x55e341012fb0 2024-04-16T05:44:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341012a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341012a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32782/0 (socket says 172.21.15.86:32782) 2024-04-16T05:44:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaffff640 1 -- 172.21.15.86:0/458390536 learned_addr learned my addr 172.21.15.86:0/458390536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaf7fe640 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 0x55e3410153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aad7fa640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a980a3d20 0x7f4a980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4ab4baa640 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341009e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaffff640 1 -- 172.21.15.86:0/458390536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 msgr2=0x55e3410153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaffff640 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 0x55e3410153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaffff640 1 -- 172.21.15.86:0/458390536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 msgr2=0x55e341009e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaffff640 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341009e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaffff640 1 -- 172.21.15.86:0/458390536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e340d497e0 con 0x55e34100a3d0 2024-04-16T05:44:39.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaf7fe640 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 0x55e3410153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:39.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4ab4baa640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a980a3d20 0x7f4a980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aad7fa640 1 -- 172.21.15.86:0/4085271366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f4aa00f27a0 con 0x7f4ab0100500 2024-04-16T05:44:39.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4aaffff640 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341012a70 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00efba0 tx=0x7f4aa004d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdaeed92f82648f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.485+0000 7f4a967fc640 1 -- 172.21.15.86:0/458390536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa00bdfc0 con 0x55e34100a3d0 2024-04-16T05:44:39.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab4baa640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a980a3d20 0x7f4a980a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8005fd0 tx=0x7f4aa8004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4a967fc640 1 -- 172.21.15.86:0/458390536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa00beb80 con 0x55e34100a3d0 2024-04-16T05:44:39.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4a967fc640 1 -- 172.21.15.86:0/458390536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa00bdde0 con 0x55e34100a3d0 2024-04-16T05:44:39.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab91f7740 1 -- 172.21.15.86:0/458390536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 msgr2=0x55e341012a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341012a70 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00efba0 tx=0x7f4aa004d000 comp rx=0 tx=0).stop 2024-04-16T05:44:39.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab91f7740 1 -- 172.21.15.86:0/458390536 shutdown_connections 2024-04-16T05:44:39.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 0x55e3410153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341012a70 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/458390536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341009e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab91f7740 1 -- 172.21.15.86:0/458390536 >> 172.21.15.86:0/458390536 conn(0x55e3410041e0 msgr2=0x55e3410045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab91f7740 1 -- 172.21.15.86:0/458390536 shutdown_connections 2024-04-16T05:44:39.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.486+0000 7f4ab91f7740 1 -- 172.21.15.86:0/458390536 wait complete. 2024-04-16T05:44:39.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4ab91f7740 1 -- start start 2024-04-16T05:44:39.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4ab91f7740 1 Processor -- start 2024-04-16T05:44:39.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341010030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341010570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 0x55e341010ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e341015b60 con 0x55e34100a3d0 2024-04-16T05:44:39.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e340ef7410 con 0x55e341009ab0 2024-04-16T05:44:39.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e341015ce0 con 0x55e341012fb0 2024-04-16T05:44:39.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4ab91f7740 10 client.0 _ll_register_callbacks cb 0x55e340ef2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:39.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4aaf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341010570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4aaf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341010570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32796/0 (socket says 172.21.15.86:32796) 2024-04-16T05:44:39.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4aaf7fe640 1 -- 172.21.15.86:0/2864260946 learned_addr learned my addr 172.21.15.86:0/2864260946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.487+0000 7f4aaffff640 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341010030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4aaf7fe640 1 -- 172.21.15.86:0/2864260946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 msgr2=0x55e341010ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4ab4baa640 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 0x55e341010ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4aaf7fe640 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 0x55e341010ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4aaf7fe640 1 -- 172.21.15.86:0/2864260946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 msgr2=0x55e341010030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4aaf7fe640 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341010030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4aaf7fe640 1 -- 172.21.15.86:0/2864260946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e340da9f70 con 0x55e34100a3d0 2024-04-16T05:44:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4aaf7fe640 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341010570 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4010680 tx=0x7f4aa4010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebbdeabe543c644 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4a8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:39.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2864260946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400eb30 con 0x55e34100a3d0 2024-04-16T05:44:39.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2864260946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f4aa400ecd0 con 0x55e34100a3d0 2024-04-16T05:44:39.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2864260946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa00beb80 con 0x55e34100a3d0 2024-04-16T05:44:39.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2864260946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4aa4025050 con 0x55e34100a3d0 2024-04-16T05:44:39.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4ab91f7740 10 client.18432 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:39.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2864260946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e340c41170 con 0x55e34100a3d0 2024-04-16T05:44:39.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2864260946 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa00beb80 con 0x55e34100a3d0 2024-04-16T05:44:39.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.488+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2864260946 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4aa4011440 con 0x55e34100a3d0 2024-04-16T05:44:39.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.489+0000 7f4ab91f7740 10 client.18432 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:39.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.489+0000 7f4ab91f7740 10 client.18432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:39.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.489+0000 7f4ab91f7740 20 client.18432 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:39.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.489+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e341023090 0x55e3410254c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.489+0000 7f4ab91f7740 4 client.18432 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:39.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.489+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2864260946 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e340f7c8b0 con 0x55e341023090 2024-04-16T05:44:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.489+0000 7f4aaffff640 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e341023090 0x55e3410254c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.489+0000 7f4a8f7fe640 10 client.18432 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.489+0000 7f4aaffff640 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e341023090 0x55e3410254c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f406a2c1f11661 server_cookie=428d2b5379fb8b80 in_seq=0 out_seq=0 2024-04-16T05:44:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2864260946 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e340f7c8b0 con 0x55e341023090 2024-04-16T05:44:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4a8f7fe640 10 client.18432 handle_command_reply: tid=0 2024-04-16T05:44:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 client.18432 shutdown 2024-04-16T05:44:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2864260946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 msgr2=0x55e341010570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341010570 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4010680 tx=0x7f4aa4010b40 comp rx=0 tx=0).stop 2024-04-16T05:44:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2864260946 shutdown_connections 2024-04-16T05:44:39.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e341023090 0x55e3410254c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341012fb0 0x55e341010ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4a8f7fe640 0 client.18432 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:39.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341010570 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2864260946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341010030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2864260946 >> 172.21.15.86:0/2864260946 conn(0x55e3410041e0 msgr2=0x55e341005f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2864260946 shutdown_connections 2024-04-16T05:44:39.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2864260946 wait complete. 2024-04-16T05:44:39.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.490+0000 7f4ab91f7740 20 client.18432 trim_cache size 0 max 16384 2024-04-16T05:44:39.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab91f7740 1 Processor -- start 2024-04-16T05:44:39.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab91f7740 1 -- start start 2024-04-16T05:44:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341012760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e34100a3d0 0x55e34101e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e341012fb0 0x55e34101b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01af510 con 0x55e341012fb0 2024-04-16T05:44:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01b1050 con 0x55e341009ab0 2024-04-16T05:44:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01afbe0 con 0x55e34100a3d0 2024-04-16T05:44:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab4baa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e341012fb0 0x55e34101b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab4baa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e341012fb0 0x55e34101b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32800/0 (socket says 172.21.15.86:32800) 2024-04-16T05:44:39.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab4baa640 1 -- 172.21.15.86:0/2951729511 learned_addr learned my addr 172.21.15.86:0/2951729511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab4baa640 1 -- 172.21.15.86:0/2951729511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e34100a3d0 msgr2=0x55e34101e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab4baa640 1 --2- 172.21.15.86:0/2951729511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e34100a3d0 0x55e34101e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab4baa640 1 -- 172.21.15.86:0/2951729511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 msgr2=0x55e341012760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab4baa640 1 --2- 172.21.15.86:0/2951729511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341012760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.508+0000 7f4ab4baa640 1 -- 172.21.15.86:0/2951729511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e34101fe50 con 0x55e341012fb0 2024-04-16T05:44:39.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab4baa640 1 --2- 172.21.15.86:0/2951729511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e341012fb0 0x55e34101b230 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f4aa801d5f0 tx=0x7f4aa8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1790d305ec0b6d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4a95ffb640 1 -- 172.21.15.86:0/2951729511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8011910 con 0x55e341012fb0 2024-04-16T05:44:39.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4a95ffb640 1 -- 172.21.15.86:0/2951729511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa801a420 con 0x55e341012fb0 2024-04-16T05:44:39.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4a95ffb640 1 -- 172.21.15.86:0/2951729511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8010630 con 0x55e341012fb0 2024-04-16T05:44:39.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2951729511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e341012fb0 msgr2=0x55e34101b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2951729511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e341012fb0 0x55e34101b230 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f4aa801d5f0 tx=0x7f4aa8012000 comp rx=0 tx=0).stop 2024-04-16T05:44:39.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2951729511 shutdown_connections 2024-04-16T05:44:39.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2951729511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e341012fb0 0x55e34101b230 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2951729511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e34100a3d0 0x55e34101e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2951729511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341009ab0 0x55e341012760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2951729511 >> 172.21.15.86:0/2951729511 conn(0x55e3410041e0 msgr2=0x55e341010880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2951729511 shutdown_connections 2024-04-16T05:44:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.509+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2951729511 wait complete. 2024-04-16T05:44:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4ab91f7740 1 -- start start 2024-04-16T05:44:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4ab91f7740 1 Processor -- start 2024-04-16T05:44:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341009ab0 0x55e3410869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341086f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4ab91f7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341012fb0 0x55e34100ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01b0980 con 0x55e34100a3d0 2024-04-16T05:44:39.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01b02b0 con 0x55e341012fb0 2024-04-16T05:44:39.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4ab91f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01ad510 con 0x55e341009ab0 2024-04-16T05:44:39.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4ab91f7740 10 client.0 _ll_register_callbacks cb 0x7f4aa4004140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:39.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4aaf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341086f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4aaf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341086f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32814/0 (socket says 172.21.15.86:32814) 2024-04-16T05:44:39.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4aaf7fe640 1 -- 172.21.15.86:0/2366650945 learned_addr learned my addr 172.21.15.86:0/2366650945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4aaffff640 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341009ab0 0x55e3410869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4aaf7fe640 1 -- 172.21.15.86:0/2366650945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341009ab0 msgr2=0x55e3410869f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4aaf7fe640 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341009ab0 0x55e3410869f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4aaf7fe640 1 -- 172.21.15.86:0/2366650945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341012fb0 msgr2=0x55e34100ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4aaf7fe640 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341012fb0 0x55e34100ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.510+0000 7f4aaf7fe640 1 -- 172.21.15.86:0/2366650945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e34100d200 con 0x55e34100a3d0 2024-04-16T05:44:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4aaf7fe640 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341086f30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4011890 tx=0x7f4aa40116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd2df22db263b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4a8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2366650945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4022040 con 0x55e34100a3d0 2024-04-16T05:44:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2366650945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f4aa40055c0 con 0x55e34100a3d0 2024-04-16T05:44:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2366650945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa801a420 con 0x55e34100a3d0 2024-04-16T05:44:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2366650945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4aa4011c90 con 0x55e34100a3d0 2024-04-16T05:44:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4ab91f7740 10 client.18438 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2366650945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e34108f1c0 con 0x55e34100a3d0 2024-04-16T05:44:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2366650945 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa801a420 con 0x55e34100a3d0 2024-04-16T05:44:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2366650945 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4aa401c610 con 0x55e34100a3d0 2024-04-16T05:44:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4ab91f7740 10 client.18438 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4ab91f7740 10 client.18438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4ab91f7740 20 client.18438 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e34108f990 0x55e341091db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4ab91f7740 4 client.18438 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:44:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.511+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2366650945 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e340f76720 con 0x55e34108f990 2024-04-16T05:44:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.512+0000 7f4aaffff640 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e34108f990 0x55e341091db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.512+0000 7f4a8f7fe640 10 client.18438 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.512+0000 7f4aaffff640 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e34108f990 0x55e341091db0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93b90602e191a8f0 server_cookie=da7c785dd99655c6 in_seq=0 out_seq=0 2024-04-16T05:44:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4a8f7fe640 1 -- 172.21.15.86:0/2366650945 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e340f76720 con 0x55e34108f990 2024-04-16T05:44:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4a8f7fe640 10 client.18438 handle_command_reply: tid=0 2024-04-16T05:44:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4ab91f7740 1 client.18438 shutdown 2024-04-16T05:44:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2366650945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 msgr2=0x55e341086f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341086f30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4011890 tx=0x7f4aa40116f0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2366650945 shutdown_connections 2024-04-16T05:44:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e34108f990 0x55e341091db0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e341012fb0 0x55e34100ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34100a3d0 0x55e341086f30 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4ab91f7740 1 --2- 172.21.15.86:0/2366650945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e341009ab0 0x55e3410869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4a8f7fe640 0 client.18438 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.513+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2366650945 >> 172.21.15.86:0/2366650945 conn(0x55e3410041e0 msgr2=0x55e3410053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2366650945 shutdown_connections 2024-04-16T05:44:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab91f7740 1 -- 172.21.15.86:0/2366650945 wait complete. 2024-04-16T05:44:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab91f7740 20 client.18438 trim_cache size 0 max 16384 2024-04-16T05:44:39.522 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:44:39.522 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:44:39.522 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "b9a6d921-4491-4f4a-8049-80886c9c0146", 2024-04-16T05:44:39.522 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:44:39.522 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:44:39.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 -- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a980a3d20 msgr2=0x7f4a980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a980a3d20 0x7f4a980a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8005fd0 tx=0x7f4aa8004a40 comp rx=0 tx=0).stop 2024-04-16T05:44:39.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 -- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0100500 msgr2=0x7f4ab019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0100500 0x7f4ab019e610 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0004f50 tx=0x7f4aa002ce20 comp rx=0 tx=0).stop 2024-04-16T05:44:39.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 -- 172.21.15.86:0/4085271366 shutdown_connections 2024-04-16T05:44:39.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ab0106fe0 0x7f4ab01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4a980a3d20 0x7f4a980a61e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ab0100ec0 0x7f4ab019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 --2- 172.21.15.86:0/4085271366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab0100500 0x7f4ab019e610 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 -- 172.21.15.86:0/4085271366 >> 172.21.15.86:0/4085271366 conn(0x7f4ab0077490 msgr2=0x7f4ab0105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 -- 172.21.15.86:0/4085271366 shutdown_connections 2024-04-16T05:44:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.514+0000 7f4ab660d640 1 -- 172.21.15.86:0/4085271366 wait complete. 2024-04-16T05:44:39.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:44:39.526 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:44:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.635+0000 7f275e524640 1 -- 172.21.15.86:0/245604674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758102bf0 msgr2=0x7f2758106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.635+0000 7f275e524640 1 --2- 172.21.15.86:0/245604674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758102bf0 0x7f2758106c30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f274c009900 tx=0x7f274c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.635+0000 7f275e524640 1 -- 172.21.15.86:0/245604674 shutdown_connections 2024-04-16T05:44:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.635+0000 7f275e524640 1 --2- 172.21.15.86:0/245604674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2758107310 0x7f275810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.635+0000 7f275e524640 1 --2- 172.21.15.86:0/245604674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758102bf0 0x7f2758106c30 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.635+0000 7f275e524640 1 --2- 172.21.15.86:0/245604674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2758102240 0x7f2758102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.635+0000 7f275e524640 1 -- 172.21.15.86:0/245604674 >> 172.21.15.86:0/245604674 conn(0x7f2758079450 msgr2=0x7f2758079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.635+0000 7f275e524640 1 -- 172.21.15.86:0/245604674 shutdown_connections 2024-04-16T05:44:39.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.635+0000 7f275e524640 1 -- 172.21.15.86:0/245604674 wait complete. 2024-04-16T05:44:39.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275e524640 1 Processor -- start 2024-04-16T05:44:39.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275e524640 1 -- start start 2024-04-16T05:44:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275e524640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2758102240 0x7f275819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275e524640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2758107310 0x7f275819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275e524640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758198680 0x7f2758198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275e524640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2758112380 con 0x7f2758198680 2024-04-16T05:44:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275e524640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2758112200 con 0x7f2758107310 2024-04-16T05:44:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275e524640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2758112500 con 0x7f2758102240 2024-04-16T05:44:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275cac1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758198680 0x7f2758198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275cac1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758198680 0x7f2758198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32834/0 (socket says 172.21.15.86:32834) 2024-04-16T05:44:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275cac1640 1 -- 172.21.15.86:0/2832886247 learned_addr learned my addr 172.21.15.86:0/2832886247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f27577fe640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2758107310 0x7f275819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f2757fff640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2758102240 0x7f275819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275cac1640 1 -- 172.21.15.86:0/2832886247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2758102240 msgr2=0x7f275819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275cac1640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2758102240 0x7f275819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275cac1640 1 -- 172.21.15.86:0/2832886247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2758107310 msgr2=0x7f275819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275cac1640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2758107310 0x7f275819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f275cac1640 1 -- 172.21.15.86:0/2832886247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2758199280 con 0x7f2758198680 2024-04-16T05:44:39.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.636+0000 7f2757fff640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2758102240 0x7f275819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:39.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.637+0000 7f275cac1640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758198680 0x7f2758198b00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f275000cce0 tx=0x7f275000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.637+0000 7f27557fa640 1 -- 172.21.15.86:0/2832886247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275000f450 con 0x7f2758198680 2024-04-16T05:44:39.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.637+0000 7f27557fa640 1 -- 172.21.15.86:0/2832886247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f275000fad0 con 0x7f2758198680 2024-04-16T05:44:39.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.637+0000 7f275e524640 1 -- 172.21.15.86:0/2832886247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27581a5360 con 0x7f2758198680 2024-04-16T05:44:39.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.637+0000 7f27557fa640 1 -- 172.21.15.86:0/2832886247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750018c10 con 0x7f2758198680 2024-04-16T05:44:39.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.637+0000 7f275e524640 1 -- 172.21.15.86:0/2832886247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27581a5710 con 0x7f2758198680 2024-04-16T05:44:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.639+0000 7f27557fa640 1 -- 172.21.15.86:0/2832886247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f275000f5f0 con 0x7f2758198680 2024-04-16T05:44:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.639+0000 7f27611ef740 1 Processor -- start 2024-04-16T05:44:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.639+0000 7f27611ef740 1 -- start start 2024-04-16T05:44:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27611ef740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 0x5619b6743230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27611ef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6743770 0x5619b674be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27611ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b674e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27611ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5619b66a8180 con 0x5619b674c350 2024-04-16T05:44:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27611ef740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5619b6645360 con 0x5619b6742e50 2024-04-16T05:44:39.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27611ef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5619b66a4140 con 0x5619b6743770 2024-04-16T05:44:39.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27557fa640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27400a3d20 0x7f27400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b674e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b674e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32846/0 (socket says 172.21.15.86:32846) 2024-04-16T05:44:39.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27577fe640 1 -- 172.21.15.86:0/2844833060 learned_addr learned my addr 172.21.15.86:0/2844833060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f2757fff640 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6743770 0x5619b674be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27577fe640 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 0x5619b6743230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27557fa640 1 -- 172.21.15.86:0/2832886247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f27500d12d0 con 0x7f2758198680 2024-04-16T05:44:39.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f275cac1640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27400a3d20 0x7f27400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27577fe640 1 -- 172.21.15.86:0/2844833060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6743770 msgr2=0x5619b674be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27577fe640 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6743770 0x5619b674be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27577fe640 1 -- 172.21.15.86:0/2844833060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 msgr2=0x5619b6743230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.640+0000 7f27577fe640 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 0x5619b6743230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.641+0000 7f27577fe640 1 -- 172.21.15.86:0/2844833060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b64847d0 con 0x5619b674c350 2024-04-16T05:44:39.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.641+0000 7f2757fff640 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6743770 0x5619b674be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:44:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.641+0000 7f27577fe640 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b674e740 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f274c005ab0 tx=0x7f274c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e95a8408c81c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.641+0000 7f275cac1640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27400a3d20 0x7f27400a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f275009ce30 tx=0x7f275000ff60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.641+0000 7f273e7fc640 1 -- 172.21.15.86:0/2844833060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c02ca40 con 0x5619b674c350 2024-04-16T05:44:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.641+0000 7f273e7fc640 1 -- 172.21.15.86:0/2844833060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274c02cbe0 con 0x5619b674c350 2024-04-16T05:44:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.641+0000 7f273e7fc640 1 -- 172.21.15.86:0/2844833060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c005170 con 0x5619b674c350 2024-04-16T05:44:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 -- 172.21.15.86:0/2844833060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 msgr2=0x5619b674e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b674e740 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f274c005ab0 tx=0x7f274c0058b0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 -- 172.21.15.86:0/2844833060 shutdown_connections 2024-04-16T05:44:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b674e740 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6743770 0x5619b674be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 --2- 172.21.15.86:0/2844833060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 0x5619b6743230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 -- 172.21.15.86:0/2844833060 >> 172.21.15.86:0/2844833060 conn(0x5619b673d580 msgr2=0x5619b673d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 -- 172.21.15.86:0/2844833060 shutdown_connections 2024-04-16T05:44:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 -- 172.21.15.86:0/2844833060 wait complete. 2024-04-16T05:44:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 -- start start 2024-04-16T05:44:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 Processor -- start 2024-04-16T05:44:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 0x5619b6749450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 0x5619b6749990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.642+0000 7f27611ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b6749ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27611ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5619b674f0e0 con 0x5619b674c350 2024-04-16T05:44:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27611ef740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5619b66612c0 con 0x5619b6743770 2024-04-16T05:44:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27611ef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5619b674f260 con 0x5619b6742e50 2024-04-16T05:44:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27611ef740 10 client.0 _ll_register_callbacks cb 0x5619b6648090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b6749ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b6749ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32858/0 (socket says 172.21.15.86:32858) 2024-04-16T05:44:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 -- 172.21.15.86:0/1343393469 learned_addr learned my addr 172.21.15.86:0/1343393469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f2757fff640 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 0x5619b6749990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 0x5619b6749450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 -- 172.21.15.86:0/1343393469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 msgr2=0x5619b6749450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 0x5619b6749450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 -- 172.21.15.86:0/1343393469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 msgr2=0x5619b6749990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 0x5619b6749990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 -- 172.21.15.86:0/1343393469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b63ba3f0 con 0x5619b674c350 2024-04-16T05:44:39.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f2757fff640 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 0x5619b6749990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:39.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27577fe640 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b6749ed0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x5619b674ed10 tx=0x7f274c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9e3d883d3c1c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27377fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:39.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27377fe640 1 -- 172.21.15.86:0/1343393469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c051060 con 0x5619b674c350 2024-04-16T05:44:39.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27611ef740 1 -- 172.21.15.86:0/1343393469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274c02cbe0 con 0x5619b674c350 2024-04-16T05:44:39.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27377fe640 1 -- 172.21.15.86:0/1343393469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f274c005350 con 0x5619b674c350 2024-04-16T05:44:39.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27377fe640 1 -- 172.21.15.86:0/1343393469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f274c005550 con 0x5619b674c350 2024-04-16T05:44:39.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27611ef740 10 client.18450 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:39.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27611ef740 1 -- 172.21.15.86:0/1343393469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619b637a0f0 con 0x5619b674c350 2024-04-16T05:44:39.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.643+0000 7f27377fe640 1 -- 172.21.15.86:0/1343393469 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274c02cbe0 con 0x5619b674c350 2024-04-16T05:44:39.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f27377fe640 1 -- 172.21.15.86:0/1343393469 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f274c041580 con 0x5619b674c350 2024-04-16T05:44:39.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f27611ef740 10 client.18450 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:39.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f27611ef740 10 client.18450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:39.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f27611ef740 20 client.18450 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:39.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f27611ef740 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619b675c610 0x5619b675ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f27611ef740 4 client.18450 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:39.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f27611ef740 1 -- 172.21.15.86:0/1343393469 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619b66b5ba0 con 0x5619b675c610 2024-04-16T05:44:39.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f2757fff640 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619b675c610 0x5619b675ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f27377fe640 10 client.18450 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:39.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.644+0000 7f2757fff640 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619b675c610 0x5619b675ea00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48cffed87b89c55e server_cookie=68406f311b87ea58 in_seq=0 out_seq=0 2024-04-16T05:44:39.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27377fe640 1 -- 172.21.15.86:0/1343393469 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619b66b5ba0 con 0x5619b675c610 2024-04-16T05:44:39.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27377fe640 10 client.18450 handle_command_reply: tid=0 2024-04-16T05:44:39.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 client.18450 shutdown 2024-04-16T05:44:39.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 -- 172.21.15.86:0/1343393469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 msgr2=0x5619b6749ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b6749ed0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x5619b674ed10 tx=0x7f274c005c90 comp rx=0 tx=0).stop 2024-04-16T05:44:39.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 -- 172.21.15.86:0/1343393469 shutdown_connections 2024-04-16T05:44:39.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619b675c610 0x5619b675ea00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b6749ed0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 0x5619b6749990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27377fe640 0 client.18450 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 --2- 172.21.15.86:0/1343393469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 0x5619b6749450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 -- 172.21.15.86:0/1343393469 >> 172.21.15.86:0/1343393469 conn(0x5619b673d580 msgr2=0x5619b673f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 -- 172.21.15.86:0/1343393469 shutdown_connections 2024-04-16T05:44:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.645+0000 7f27611ef740 1 -- 172.21.15.86:0/1343393469 wait complete. 2024-04-16T05:44:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.646+0000 7f27611ef740 20 client.18450 trim_cache size 0 max 16384 2024-04-16T05:44:39.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27611ef740 1 Processor -- start 2024-04-16T05:44:39.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27611ef740 1 -- start start 2024-04-16T05:44:39.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27611ef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 0x5619b674bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27611ef740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 0x5619b673c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27611ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b673e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27611ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27581ad1b0 con 0x5619b674c350 2024-04-16T05:44:39.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27611ef740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27581aecf0 con 0x5619b6743770 2024-04-16T05:44:39.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27611ef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27581ad880 con 0x5619b6742e50 2024-04-16T05:44:39.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b673e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b673e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32870/0 (socket says 172.21.15.86:32870) 2024-04-16T05:44:39.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27577fe640 1 -- 172.21.15.86:0/3006420998 learned_addr learned my addr 172.21.15.86:0/3006420998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f2757fff640 1 --2- 172.21.15.86:0/3006420998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 0x5619b673c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27577fe640 1 --2- 172.21.15.86:0/3006420998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 0x5619b674bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27577fe640 1 -- 172.21.15.86:0/3006420998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 msgr2=0x5619b674bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27577fe640 1 --2- 172.21.15.86:0/3006420998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 0x5619b674bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27577fe640 1 -- 172.21.15.86:0/3006420998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 msgr2=0x5619b673c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27577fe640 1 --2- 172.21.15.86:0/3006420998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 0x5619b673c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.663+0000 7f27577fe640 1 -- 172.21.15.86:0/3006420998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b6745020 con 0x5619b674c350 2024-04-16T05:44:39.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27577fe640 1 --2- 172.21.15.86:0/3006420998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b673e960 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f274c02c170 tx=0x7f274c043fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b79cda4858938b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f273dffb640 1 -- 172.21.15.86:0/3006420998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c061070 con 0x5619b674c350 2024-04-16T05:44:39.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f273dffb640 1 -- 172.21.15.86:0/3006420998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274c009520 con 0x5619b674c350 2024-04-16T05:44:39.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f273dffb640 1 -- 172.21.15.86:0/3006420998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c051040 con 0x5619b674c350 2024-04-16T05:44:39.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27611ef740 1 -- 172.21.15.86:0/3006420998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 msgr2=0x5619b673e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27611ef740 1 --2- 172.21.15.86:0/3006420998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b673e960 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f274c02c170 tx=0x7f274c043fa0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27611ef740 1 -- 172.21.15.86:0/3006420998 shutdown_connections 2024-04-16T05:44:39.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27611ef740 1 --2- 172.21.15.86:0/3006420998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b674c350 0x5619b673e960 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27611ef740 1 --2- 172.21.15.86:0/3006420998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6743770 0x5619b673c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27611ef740 1 --2- 172.21.15.86:0/3006420998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b6742e50 0x5619b674bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27611ef740 1 -- 172.21.15.86:0/3006420998 >> 172.21.15.86:0/3006420998 conn(0x5619b673d580 msgr2=0x5619b673bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27611ef740 1 -- 172.21.15.86:0/3006420998 shutdown_connections 2024-04-16T05:44:39.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.664+0000 7f27611ef740 1 -- 172.21.15.86:0/3006420998 wait complete. 2024-04-16T05:44:39.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27611ef740 1 -- start start 2024-04-16T05:44:39.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27611ef740 1 Processor -- start 2024-04-16T05:44:39.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27611ef740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 0x5619b6761630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27611ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b6743770 0x5619b6761b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27611ef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b674c350 0x5619b67b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27611ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27581ae620 con 0x5619b6743770 2024-04-16T05:44:39.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27611ef740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27581adf50 con 0x5619b6742e50 2024-04-16T05:44:39.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27611ef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27581ab1b0 con 0x5619b674c350 2024-04-16T05:44:39.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27611ef740 10 client.0 _ll_register_callbacks cb 0x7f274800f6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:39.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f2757fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b6743770 0x5619b6761b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f2757fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b6743770 0x5619b6761b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32880/0 (socket says 172.21.15.86:32880) 2024-04-16T05:44:39.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f2757fff640 1 -- 172.21.15.86:0/3919763383 learned_addr learned my addr 172.21.15.86:0/3919763383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:44:39.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27577fe640 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 0x5619b6761630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27577fe640 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b674c350 0x5619b67b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f2757fff640 1 -- 172.21.15.86:0/3919763383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b674c350 msgr2=0x5619b67b9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f2757fff640 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b674c350 0x5619b67b9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f2757fff640 1 -- 172.21.15.86:0/3919763383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 msgr2=0x5619b6761630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:39.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f2757fff640 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 0x5619b6761630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f2757fff640 1 -- 172.21.15.86:0/3919763383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b67ba4f0 con 0x5619b6743770 2024-04-16T05:44:39.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.665+0000 7f27577fe640 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b674c350 0x5619b67b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:39.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f2757fff640 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b6743770 0x5619b6761b70 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2748004700 tx=0x7f274800fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09591a294c502cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:39.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27377fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:44:39.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27377fe640 1 -- 172.21.15.86:0/3919763383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748005050 con 0x5619b6743770 2024-04-16T05:44:39.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27611ef740 1 -- 172.21.15.86:0/3919763383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274c009520 con 0x5619b6743770 2024-04-16T05:44:39.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27377fe640 1 -- 172.21.15.86:0/3919763383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f27480051f0 con 0x5619b6743770 2024-04-16T05:44:39.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27377fe640 1 -- 172.21.15.86:0/3919763383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27480053f0 con 0x5619b6743770 2024-04-16T05:44:39.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27611ef740 10 client.18456 fetch_fsmap learned FSMap version 29 2024-04-16T05:44:39.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27611ef740 1 -- 172.21.15.86:0/3919763383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619b67c2b30 con 0x5619b6743770 2024-04-16T05:44:39.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27377fe640 1 -- 172.21.15.86:0/3919763383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274c009520 con 0x5619b6743770 2024-04-16T05:44:39.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27377fe640 1 -- 172.21.15.86:0/3919763383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2748023020 con 0x5619b6743770 2024-04-16T05:44:39.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27611ef740 10 client.18456 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:44:39.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27611ef740 10 client.18456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:39.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27611ef740 20 client.18456 populate_metadata read hostname 'smithi086' 2024-04-16T05:44:39.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27611ef740 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619b67c32e0 0x5619b67c5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:39.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27611ef740 4 client.18456 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:44:39.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.666+0000 7f27611ef740 1 -- 172.21.15.86:0/3919763383 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619b66afa10 con 0x5619b67c32e0 2024-04-16T05:44:39.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27577fe640 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619b67c32e0 0x5619b67c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:39.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27377fe640 10 client.18456 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:39.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27577fe640 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619b67c32e0 0x5619b67c5700 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10069b20037f9ea server_cookie=ae9c81d60fb0ca28 in_seq=0 out_seq=0 2024-04-16T05:44:39.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27377fe640 1 -- 172.21.15.86:0/3919763383 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5619b66afa10 con 0x5619b67c32e0 2024-04-16T05:44:39.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27377fe640 10 client.18456 handle_command_reply: tid=0 2024-04-16T05:44:39.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 client.18456 shutdown 2024-04-16T05:44:39.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 -- 172.21.15.86:0/3919763383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b6743770 msgr2=0x5619b6761b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b6743770 0x5619b6761b70 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2748004700 tx=0x7f274800fc80 comp rx=0 tx=0).stop 2024-04-16T05:44:39.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 -- 172.21.15.86:0/3919763383 shutdown_connections 2024-04-16T05:44:39.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x5619b67c32e0 0x5619b67c5700 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619b674c350 0x5619b67b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619b6743770 0x5619b6761b70 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27377fe640 0 client.18456 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:44:39.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 --2- 172.21.15.86:0/3919763383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619b6742e50 0x5619b6761630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 -- 172.21.15.86:0/3919763383 >> 172.21.15.86:0/3919763383 conn(0x5619b673d580 msgr2=0x5619b6749ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 -- 172.21.15.86:0/3919763383 shutdown_connections 2024-04-16T05:44:39.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 1 -- 172.21.15.86:0/3919763383 wait complete. 2024-04-16T05:44:39.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.667+0000 7f27611ef740 20 client.18456 trim_cache size 0 max 16384 2024-04-16T05:44:39.676 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:44:39.677 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:44:39.677 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:44:39.677 INFO:teuthology.orchestra.run.smithi086.stdout: "b9a6d921-4491-4f4a-8049-80886c9c0146": { 2024-04-16T05:44:39.677 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:44:39.677 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "b9a6d921-4491-4f4a-8049-80886c9c0146", 2024-04-16T05:44:39.677 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:44:39.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:44:39.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:44:39.677 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:44:39.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 -- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27400a3d20 msgr2=0x7f27400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27400a3d20 0x7f27400a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f275009ce30 tx=0x7f275000ff60 comp rx=0 tx=0).stop 2024-04-16T05:44:39.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 -- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758198680 msgr2=0x7f2758198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:39.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758198680 0x7f2758198b00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f275000cce0 tx=0x7f275000a620 comp rx=0 tx=0).stop 2024-04-16T05:44:39.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 -- 172.21.15.86:0/2832886247 shutdown_connections 2024-04-16T05:44:39.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2758198680 0x7f2758198b00 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27400a3d20 0x7f27400a61e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2758107310 0x7f275819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 --2- 172.21.15.86:0/2832886247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2758102240 0x7f275819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:39.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 -- 172.21.15.86:0/2832886247 >> 172.21.15.86:0/2832886247 conn(0x7f2758079450 msgr2=0x7f27580fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:39.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 -- 172.21.15.86:0/2832886247 shutdown_connections 2024-04-16T05:44:39.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:44:39.668+0000 7f275e524640 1 -- 172.21.15.86:0/2832886247 wait complete. 2024-04-16T05:44:39.680 INFO:tasks.cephfs.filesystem:scrub status for tag:b9a6d921-4491-4f4a-8049-80886c9c0146 - {'path': '/', 'tag': 'b9a6d921-4491-4f4a-8049-80886c9c0146', 'options': 'recursive,force'} 2024-04-16T05:44:40.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:40 smithi194 ceph-mon[35865]: pgmap v290: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:44:40.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:40 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:44:40.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:40 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:40.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:40 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:44:40.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:40 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:40.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:40 smithi086 ceph-mon[32028]: pgmap v290: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:44:40.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:40 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:44:40.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:40 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:40.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:40 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:44:40.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:40 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:40.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:40 smithi196 ceph-mon[36373]: pgmap v290: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:44:40.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:40 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:44:40.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:40 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:40.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:40 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:44:40.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:40 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:42.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:42 smithi194 ceph-mon[35865]: pgmap v291: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:42.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:42 smithi086 ceph-mon[32028]: pgmap v291: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:42.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:42 smithi196 ceph-mon[36373]: pgmap v291: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:44.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:44 smithi194 ceph-mon[35865]: pgmap v292: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:44:44.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:44 smithi086 ceph-mon[32028]: pgmap v292: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:44:44.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:44 smithi196 ceph-mon[36373]: pgmap v292: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:44:45.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:45 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:44:45.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:45 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:44:45.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:45 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:44:46.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:46 smithi194 ceph-mon[35865]: pgmap v293: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:44:46.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:46 smithi086 ceph-mon[32028]: pgmap v293: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:44:46.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:46 smithi196 ceph-mon[36373]: pgmap v293: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:44:48.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:48 smithi194 ceph-mon[35865]: pgmap v294: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:44:48.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:48 smithi086 ceph-mon[32028]: pgmap v294: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:44:48.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:48 smithi196 ceph-mon[36373]: pgmap v294: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:44:50.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:50 smithi194 ceph-mon[35865]: pgmap v295: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:44:50.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:50 smithi086 ceph-mon[32028]: pgmap v295: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:44:50.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:50 smithi196 ceph-mon[36373]: pgmap v295: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:44:52.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:52 smithi194 ceph-mon[35865]: pgmap v296: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:44:52.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:52 smithi086 ceph-mon[32028]: pgmap v296: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:44:52.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:52 smithi196 ceph-mon[36373]: pgmap v296: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-16T05:44:54.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:54 smithi194 ceph-mon[35865]: pgmap v297: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:44:54.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:54 smithi086 ceph-mon[32028]: pgmap v297: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:44:54.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:54 smithi196 ceph-mon[36373]: pgmap v297: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-16T05:44:56.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:56 smithi194 ceph-mon[35865]: pgmap v298: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:56.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:56 smithi086 ceph-mon[32028]: pgmap v298: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:56.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:56 smithi196 ceph-mon[36373]: pgmap v298: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:58.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:44:58 smithi194 ceph-mon[35865]: pgmap v299: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:58.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:44:58 smithi086 ceph-mon[32028]: pgmap v299: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:44:58.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:44:58 smithi196 ceph-mon[36373]: pgmap v299: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:00.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:00 smithi194 ceph-mon[35865]: pgmap v300: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:00.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:00 smithi086 ceph-mon[32028]: pgmap v300: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:00.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:00 smithi196 ceph-mon[36373]: pgmap v300: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:02.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:02 smithi194 ceph-mon[35865]: pgmap v301: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:02.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:02 smithi086 ceph-mon[32028]: pgmap v301: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:02.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:02 smithi196 ceph-mon[36373]: pgmap v301: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:04.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:04 smithi194 ceph-mon[35865]: pgmap v302: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:45:04.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:04 smithi086 ceph-mon[32028]: pgmap v302: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:45:04.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:04 smithi196 ceph-mon[36373]: pgmap v302: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:45:06.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:06 smithi194 ceph-mon[35865]: pgmap v303: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:45:06.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:06 smithi086 ceph-mon[32028]: pgmap v303: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:45:06.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:06 smithi196 ceph-mon[36373]: pgmap v303: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:45:08.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:08 smithi194 ceph-mon[35865]: pgmap v304: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:45:08.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:08 smithi086 ceph-mon[32028]: pgmap v304: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:45:08.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:08 smithi196 ceph-mon[36373]: pgmap v304: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:45:09.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:45:09.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:45:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 -- 172.21.15.86:0/1350155927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0850107390 msgr2=0x7f085010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 --2- 172.21.15.86:0/1350155927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0850107390 0x7f085010ef60 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0848009900 tx=0x7f084802bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 -- 172.21.15.86:0/1350155927 shutdown_connections 2024-04-16T05:45:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 --2- 172.21.15.86:0/1350155927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0850107390 0x7f085010ef60 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 --2- 172.21.15.86:0/1350155927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0850102c70 0x7f0850106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 --2- 172.21.15.86:0/1350155927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08501022c0 0x7f08501026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 -- 172.21.15.86:0/1350155927 >> 172.21.15.86:0/1350155927 conn(0x7f08500fdb60 msgr2=0x7f08500fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 -- 172.21.15.86:0/1350155927 shutdown_connections 2024-04-16T05:45:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 -- 172.21.15.86:0/1350155927 wait complete. 2024-04-16T05:45:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 Processor -- start 2024-04-16T05:45:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.802+0000 7f08579a8640 1 -- start start 2024-04-16T05:45:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f08579a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08501022c0 0x7f085019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f08579a8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0850107390 0x7f085019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f08579a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501986b0 0x7f0850198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f08579a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0850112430 con 0x7f08501986b0 2024-04-16T05:45:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f08579a8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08501122b0 con 0x7f0850107390 2024-04-16T05:45:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f08579a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08501125b0 con 0x7f08501022c0 2024-04-16T05:45:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855f45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501986b0 0x7f0850198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855f45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501986b0 0x7f0850198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48010/0 (socket says 172.21.15.86:48010) 2024-04-16T05:45:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855f45640 1 -- 172.21.15.86:0/2376948703 learned_addr learned my addr 172.21.15.86:0/2376948703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0854f43640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0850107390 0x7f085019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855744640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08501022c0 0x7f085019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855f45640 1 -- 172.21.15.86:0/2376948703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08501022c0 msgr2=0x7f085019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855f45640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08501022c0 0x7f085019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855f45640 1 -- 172.21.15.86:0/2376948703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0850107390 msgr2=0x7f085019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855f45640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0850107390 0x7f085019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855f45640 1 -- 172.21.15.86:0/2376948703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08501992b0 con 0x7f08501986b0 2024-04-16T05:45:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0854f43640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0850107390 0x7f085019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f0855f45640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501986b0 0x7f0850198b30 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0848009230 tx=0x7f084802cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f084e7fc640 1 -- 172.21.15.86:0/2376948703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848041070 con 0x7f08501986b0 2024-04-16T05:45:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f084e7fc640 1 -- 172.21.15.86:0/2376948703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0848004450 con 0x7f08501986b0 2024-04-16T05:45:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f08579a8640 1 -- 172.21.15.86:0/2376948703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0850199540 con 0x7f08501986b0 2024-04-16T05:45:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.803+0000 7f084e7fc640 1 -- 172.21.15.86:0/2376948703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848034440 con 0x7f08501986b0 2024-04-16T05:45:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.804+0000 7f08579a8640 1 -- 172.21.15.86:0/2376948703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08501a5680 con 0x7f08501986b0 2024-04-16T05:45:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f084e7fc640 1 -- 172.21.15.86:0/2376948703 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0848044070 con 0x7f08501986b0 2024-04-16T05:45:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f085a598740 1 Processor -- start 2024-04-16T05:45:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f085a598740 1 -- start start 2024-04-16T05:45:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f085a598740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c81830 0x563828c81c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f085a598740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 0x563828c8f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f085a598740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c8f760 0x563828c91b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f085a598740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563828be7180 con 0x563828c8f760 2024-04-16T05:45:09.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f085a598740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563828b84360 con 0x563828c81830 2024-04-16T05:45:09.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f085a598740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563828be3140 con 0x563828c821e0 2024-04-16T05:45:09.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0854f43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c8f760 0x563828c91b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0854f43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c8f760 0x563828c91b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48016/0 (socket says 172.21.15.86:48016) 2024-04-16T05:45:09.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0854f43640 1 -- 172.21.15.86:0/3774034327 learned_addr learned my addr 172.21.15.86:0/3774034327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:09.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0854f43640 1 --2- 172.21.15.86:0/3774034327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c81830 0x563828c81c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f084e7fc640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08380a3d20 0x7f08380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0855744640 1 --2- 172.21.15.86:0/3774034327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 0x563828c8f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0854f43640 1 -- 172.21.15.86:0/3774034327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 msgr2=0x563828c8f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0854f43640 1 --2- 172.21.15.86:0/3774034327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 0x563828c8f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0854f43640 1 -- 172.21.15.86:0/3774034327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c81830 msgr2=0x563828c81c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0854f43640 1 --2- 172.21.15.86:0/3774034327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c81830 0x563828c81c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.807+0000 7f0854f43640 1 -- 172.21.15.86:0/3774034327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638289c37d0 con 0x563828c8f760 2024-04-16T05:45:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.808+0000 7f0855f45640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08380a3d20 0x7f08380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.808+0000 7f0854f43640 1 --2- 172.21.15.86:0/3774034327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c8f760 0x563828c91b50 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f0844005fd0 tx=0x7f084400e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c067fff6d60e769a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.808+0000 7f084e7fc640 1 -- 172.21.15.86:0/2376948703 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f08480f2740 con 0x7f08501986b0 2024-04-16T05:45:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.808+0000 7f08377fe640 1 -- 172.21.15.86:0/3774034327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844018030 con 0x563828c8f760 2024-04-16T05:45:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.808+0000 7f08377fe640 1 -- 172.21.15.86:0/3774034327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084401c8a0 con 0x563828c8f760 2024-04-16T05:45:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.808+0000 7f08377fe640 1 -- 172.21.15.86:0/3774034327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08440109d0 con 0x563828c8f760 2024-04-16T05:45:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 -- 172.21.15.86:0/3774034327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c8f760 msgr2=0x563828c91b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f0855f45640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08380a3d20 0x7f08380a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f08480badf0 tx=0x7f0848090fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 --2- 172.21.15.86:0/3774034327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c8f760 0x563828c91b50 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f0844005fd0 tx=0x7f084400e5b0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 -- 172.21.15.86:0/3774034327 shutdown_connections 2024-04-16T05:45:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 --2- 172.21.15.86:0/3774034327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c8f760 0x563828c91b50 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 --2- 172.21.15.86:0/3774034327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 0x563828c8f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 --2- 172.21.15.86:0/3774034327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c81830 0x563828c81c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 -- 172.21.15.86:0/3774034327 >> 172.21.15.86:0/3774034327 conn(0x563828c7c580 msgr2=0x563828c7c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 -- 172.21.15.86:0/3774034327 shutdown_connections 2024-04-16T05:45:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 -- 172.21.15.86:0/3774034327 wait complete. 2024-04-16T05:45:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 -- start start 2024-04-16T05:45:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.809+0000 7f085a598740 1 Processor -- start 2024-04-16T05:45:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c8cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c8f760 0x563828c8d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563828c92520 con 0x563828c81830 2024-04-16T05:45:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563828ba02c0 con 0x563828c821e0 2024-04-16T05:45:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563828c926a0 con 0x563828c8f760 2024-04-16T05:45:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 10 client.0 _ll_register_callbacks cb 0x563828b87090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48026/0 (socket says 172.21.15.86:48026) 2024-04-16T05:45:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 -- 172.21.15.86:0/2826715225 learned_addr learned my addr 172.21.15.86:0/2826715225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c8f760 0x563828c8d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0855744640 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c8cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 -- 172.21.15.86:0/2826715225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c8f760 msgr2=0x563828c8d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c8f760 0x563828c8d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 -- 172.21.15.86:0/2826715225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 msgr2=0x563828c8cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c8cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 -- 172.21.15.86:0/2826715225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638288f93f0 con 0x563828c81830 2024-04-16T05:45:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0855744640 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c8cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0854f43640 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8c800 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f084401c870 tx=0x7f084400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bd2852f2538fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0834ff9640 1 -- 172.21.15.86:0/2826715225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844018040 con 0x563828c81830 2024-04-16T05:45:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0834ff9640 1 -- 172.21.15.86:0/2826715225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f0844005040 con 0x563828c81830 2024-04-16T05:45:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 1 -- 172.21.15.86:0/2826715225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084401c8a0 con 0x563828c81830 2024-04-16T05:45:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f0834ff9640 1 -- 172.21.15.86:0/2826715225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f084402e050 con 0x563828c81830 2024-04-16T05:45:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 10 client.18468 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.810+0000 7f085a598740 1 -- 172.21.15.86:0/2826715225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638288b90f0 con 0x563828c81830 2024-04-16T05:45:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.811+0000 7f0834ff9640 1 -- 172.21.15.86:0/2826715225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084401c8a0 con 0x563828c81830 2024-04-16T05:45:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.811+0000 7f0834ff9640 1 -- 172.21.15.86:0/2826715225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f084400f040 con 0x563828c81830 2024-04-16T05:45:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.811+0000 7f085a598740 10 client.18468 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.811+0000 7f085a598740 10 client.18468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.811+0000 7f085a598740 20 client.18468 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.811+0000 7f085a598740 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563828c9fb20 0x563828ca1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.811+0000 7f085a598740 4 client.18468 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.811+0000 7f085a598740 1 -- 172.21.15.86:0/2826715225 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563828bf4ba0 con 0x563828c9fb20 2024-04-16T05:45:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.811+0000 7f0855744640 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563828c9fb20 0x563828ca1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.812+0000 7f0834ff9640 10 client.18468 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.812+0000 7f0855744640 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563828c9fb20 0x563828ca1f10 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f682eb124041ebe server_cookie=80ae7ab67b3cea25 in_seq=0 out_seq=0 2024-04-16T05:45:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.812+0000 7f0834ff9640 1 -- 172.21.15.86:0/2826715225 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563828bf4ba0 con 0x563828c9fb20 2024-04-16T05:45:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.812+0000 7f0834ff9640 10 client.18468 handle_command_reply: tid=0 2024-04-16T05:45:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.812+0000 7f085a598740 1 client.18468 shutdown 2024-04-16T05:45:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.812+0000 7f085a598740 1 -- 172.21.15.86:0/2826715225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 msgr2=0x563828c8c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8c800 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f084401c870 tx=0x7f084400ebc0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 1 -- 172.21.15.86:0/2826715225 shutdown_connections 2024-04-16T05:45:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563828c9fb20 0x563828ca1f10 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c8f760 0x563828c8d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c8cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f0834ff9640 0 client.18468 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 1 --2- 172.21.15.86:0/2826715225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8c800 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 1 -- 172.21.15.86:0/2826715225 >> 172.21.15.86:0/2826715225 conn(0x563828c7c580 msgr2=0x563828c7e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 1 -- 172.21.15.86:0/2826715225 shutdown_connections 2024-04-16T05:45:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 1 -- 172.21.15.86:0/2826715225 wait complete. 2024-04-16T05:45:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.813+0000 7f085a598740 20 client.18468 trim_cache size 0 max 16384 2024-04-16T05:45:09.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f085a598740 1 Processor -- start 2024-04-16T05:45:09.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f085a598740 1 -- start start 2024-04-16T05:45:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f085a598740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f085a598740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c9b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f085a598740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c8f760 0x563828c7b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f085a598740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08501ad2e0 con 0x563828c81830 2024-04-16T05:45:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f085a598740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08501aee20 con 0x563828c821e0 2024-04-16T05:45:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f085a598740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08501ad9b0 con 0x563828c8f760 2024-04-16T05:45:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f0854f43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f0854f43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48032/0 (socket says 172.21.15.86:48032) 2024-04-16T05:45:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.830+0000 7f0854f43640 1 -- 172.21.15.86:0/12135456 learned_addr learned my addr 172.21.15.86:0/12135456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0855744640 1 --2- 172.21.15.86:0/12135456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c9b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0854f43640 1 -- 172.21.15.86:0/12135456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c8f760 msgr2=0x563828c7b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0854f43640 1 --2- 172.21.15.86:0/12135456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c8f760 0x563828c7b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0854f43640 1 -- 172.21.15.86:0/12135456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 msgr2=0x563828c9b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0854f43640 1 --2- 172.21.15.86:0/12135456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c9b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0854f43640 1 -- 172.21.15.86:0/12135456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563828c91e80 con 0x563828c81830 2024-04-16T05:45:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0855744640 1 --2- 172.21.15.86:0/12135456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c9b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0854f43640 1 --2- 172.21.15.86:0/12135456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8ef30 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f0844006010 tx=0x7f0844012de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf409ba95890393 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0836ffd640 1 -- 172.21.15.86:0/12135456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844022030 con 0x563828c81830 2024-04-16T05:45:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0836ffd640 1 -- 172.21.15.86:0/12135456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084400e5b0 con 0x563828c81830 2024-04-16T05:45:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.831+0000 7f0836ffd640 1 -- 172.21.15.86:0/12135456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844018030 con 0x563828c81830 2024-04-16T05:45:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 -- 172.21.15.86:0/12135456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 msgr2=0x563828c8ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 --2- 172.21.15.86:0/12135456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8ef30 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f0844006010 tx=0x7f0844012de0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 -- 172.21.15.86:0/12135456 shutdown_connections 2024-04-16T05:45:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 --2- 172.21.15.86:0/12135456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c8f760 0x563828c7b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 --2- 172.21.15.86:0/12135456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c821e0 0x563828c9b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 --2- 172.21.15.86:0/12135456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c8ef30 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 -- 172.21.15.86:0/12135456 >> 172.21.15.86:0/12135456 conn(0x563828c7c580 msgr2=0x563828c7aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 -- 172.21.15.86:0/12135456 shutdown_connections 2024-04-16T05:45:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 -- 172.21.15.86:0/12135456 wait complete. 2024-04-16T05:45:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 -- start start 2024-04-16T05:45:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 Processor -- start 2024-04-16T05:45:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c97ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 0x563828c95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c8f760 0x563828c959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08501ae750 con 0x563828c81830 2024-04-16T05:45:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08501ae080 con 0x563828c8f760 2024-04-16T05:45:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.832+0000 7f085a598740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08501ab2e0 con 0x563828c821e0 2024-04-16T05:45:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f085a598740 10 client.0 _ll_register_callbacks cb 0x7f084400ef20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c97ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c97ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48044/0 (socket says 172.21.15.86:48044) 2024-04-16T05:45:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 -- 172.21.15.86:0/779186616 learned_addr learned my addr 172.21.15.86:0/779186616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0855744640 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 0x563828c95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c8f760 0x563828c959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 -- 172.21.15.86:0/779186616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 msgr2=0x563828c95270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 0x563828c95270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 -- 172.21.15.86:0/779186616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c8f760 msgr2=0x563828c959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c8f760 0x563828c959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 -- 172.21.15.86:0/779186616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563828d04f50 con 0x563828c81830 2024-04-16T05:45:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0854f43640 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c97ee0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f0844010020 tx=0x7f084400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951f22da89d8472a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0834ff9640 1 -- 172.21.15.86:0/779186616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844018070 con 0x563828c81830 2024-04-16T05:45:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0834ff9640 1 -- 172.21.15.86:0/779186616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f0844004220 con 0x563828c81830 2024-04-16T05:45:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f085a598740 1 -- 172.21.15.86:0/779186616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563828beea10 con 0x563828c81830 2024-04-16T05:45:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f0834ff9640 1 -- 172.21.15.86:0/779186616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0844022040 con 0x563828c81830 2024-04-16T05:45:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f085a598740 10 client.18474 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.833+0000 7f085a598740 1 -- 172.21.15.86:0/779186616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563828d06090 con 0x563828c81830 2024-04-16T05:45:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f0834ff9640 1 -- 172.21.15.86:0/779186616 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563828beea10 con 0x563828c81830 2024-04-16T05:45:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f0834ff9640 1 -- 172.21.15.86:0/779186616 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f08440103e0 con 0x563828c81830 2024-04-16T05:45:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f085a598740 10 client.18474 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f085a598740 10 client.18474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f085a598740 20 client.18474 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f085a598740 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563828d06550 0x563828d08940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f085a598740 4 client.18474 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:45:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f085a598740 1 -- 172.21.15.86:0/779186616 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563828b6a560 con 0x563828d06550 2024-04-16T05:45:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f0855744640 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563828d06550 0x563828d08940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f0834ff9640 10 client.18474 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.834+0000 7f0855744640 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563828d06550 0x563828d08940 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54761774d85ceb7 server_cookie=fa30ac79ba581c09 in_seq=0 out_seq=0 2024-04-16T05:45:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f0834ff9640 1 -- 172.21.15.86:0/779186616 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563828b6a560 con 0x563828d06550 2024-04-16T05:45:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f0834ff9640 10 client.18474 handle_command_reply: tid=0 2024-04-16T05:45:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 client.18474 shutdown 2024-04-16T05:45:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 -- 172.21.15.86:0/779186616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 msgr2=0x563828c97ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c97ee0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f0844010020 tx=0x7f084400eb20 comp rx=0 tx=0).stop 2024-04-16T05:45:09.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 -- 172.21.15.86:0/779186616 shutdown_connections 2024-04-16T05:45:09.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x563828d06550 0x563828d08940 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563828c8f760 0x563828c959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f0834ff9640 0 client.18474 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:09.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563828c821e0 0x563828c95270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 --2- 172.21.15.86:0/779186616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563828c81830 0x563828c97ee0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 -- 172.21.15.86:0/779186616 >> 172.21.15.86:0/779186616 conn(0x563828c7c580 msgr2=0x563828c7b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:09.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 -- 172.21.15.86:0/779186616 shutdown_connections 2024-04-16T05:45:09.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 1 -- 172.21.15.86:0/779186616 wait complete. 2024-04-16T05:45:09.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f085a598740 20 client.18474 trim_cache size 0 max 16384 2024-04-16T05:45:09.845 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:45:09.845 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:45:09.845 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:45:09.846 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:45:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 -- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08380a3d20 msgr2=0x7f08380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08380a3d20 0x7f08380a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f08480badf0 tx=0x7f0848090fd0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 -- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501986b0 msgr2=0x7f0850198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501986b0 0x7f0850198b30 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0848009230 tx=0x7f084802cbf0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 -- 172.21.15.86:0/2376948703 shutdown_connections 2024-04-16T05:45:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501986b0 0x7f0850198b30 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f08380a3d20 0x7f08380a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0850107390 0x7f085019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 --2- 172.21.15.86:0/2376948703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08501022c0 0x7f085019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.835+0000 7f08579a8640 1 -- 172.21.15.86:0/2376948703 >> 172.21.15.86:0/2376948703 conn(0x7f08500fdb60 msgr2=0x7f08500fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.836+0000 7f08579a8640 1 -- 172.21.15.86:0/2376948703 shutdown_connections 2024-04-16T05:45:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.836+0000 7f08579a8640 1 -- 172.21.15.86:0/2376948703 wait complete. 2024-04-16T05:45:09.849 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:45:09.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:45:09.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.955+0000 7fadcac74640 1 -- 172.21.15.86:0/1391167030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 msgr2=0x7fadc410dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.955+0000 7fadcac74640 1 --2- 172.21.15.86:0/1391167030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 0x7fadc410dde0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009ca0 tx=0x7fadbc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.955+0000 7fadcac74640 1 -- 172.21.15.86:0/1391167030 shutdown_connections 2024-04-16T05:45:09.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.955+0000 7fadcac74640 1 --2- 172.21.15.86:0/1391167030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 0x7fadc410dde0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.955+0000 7fadcac74640 1 --2- 172.21.15.86:0/1391167030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc4101af0 0x7fadc4105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.955+0000 7fadcac74640 1 --2- 172.21.15.86:0/1391167030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fadc4101140 0x7fadc4101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.955+0000 7fadcac74640 1 -- 172.21.15.86:0/1391167030 >> 172.21.15.86:0/1391167030 conn(0x7fadc4077280 msgr2=0x7fadc4077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.955+0000 7fadcac74640 1 -- 172.21.15.86:0/1391167030 shutdown_connections 2024-04-16T05:45:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.955+0000 7fadcac74640 1 -- 172.21.15.86:0/1391167030 wait complete. 2024-04-16T05:45:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadcac74640 1 Processor -- start 2024-04-16T05:45:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadcac74640 1 -- start start 2024-04-16T05:45:09.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadcac74640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc4101140 0x7fadc41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadcac74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 0x7fadc4199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadcac74640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fadc419a220 0x7fadc41a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadcac74640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4111320 con 0x7fadc4106210 2024-04-16T05:45:09.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadcac74640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41111a0 con 0x7fadc419a220 2024-04-16T05:45:09.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadcac74640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41114a0 con 0x7fadc4101140 2024-04-16T05:45:09.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 0x7fadc4199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 0x7fadc4199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48058/0 (socket says 172.21.15.86:48058) 2024-04-16T05:45:09.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadc3fff640 1 -- 172.21.15.86:0/2426247557 learned_addr learned my addr 172.21.15.86:0/2426247557 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:09.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadc3fff640 1 -- 172.21.15.86:0/2426247557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc4101140 msgr2=0x7fadc41997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadc8a10640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc4101140 0x7fadc41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadc3fff640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc4101140 0x7fadc41997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadc3fff640 1 -- 172.21.15.86:0/2426247557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fadc419a220 msgr2=0x7fadc41a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:09.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadc3fff640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fadc419a220 0x7fadc41a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:09.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.956+0000 7fadc3fff640 1 -- 172.21.15.86:0/2426247557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc41a4fa0 con 0x7fadc4106210 2024-04-16T05:45:09.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.957+0000 7fadc3fff640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 0x7fadc4199ce0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fadb000b450 tx=0x7fadb000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:09.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.957+0000 7fadc1ffb640 1 -- 172.21.15.86:0/2426247557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0015020 con 0x7fadc4106210 2024-04-16T05:45:09.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.957+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc41a5290 con 0x7fadc4106210 2024-04-16T05:45:09.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.957+0000 7fadc1ffb640 1 -- 172.21.15.86:0/2426247557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb0011d70 con 0x7fadc4106210 2024-04-16T05:45:09.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.957+0000 7fadc1ffb640 1 -- 172.21.15.86:0/2426247557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb000f340 con 0x7fadc4106210 2024-04-16T05:45:09.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.957+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41a5830 con 0x7fadc4106210 2024-04-16T05:45:09.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.958+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc4102610 con 0x7fadc4106210 2024-04-16T05:45:09.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.959+0000 7fadc1ffb640 1 -- 172.21.15.86:0/2426247557 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fadb0018020 con 0x7fadc4106210 2024-04-16T05:45:09.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.960+0000 7fadc1ffb640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fadb40a3d20 0x7fadb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:09.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.960+0000 7fadc8a10640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fadb40a3d20 0x7fadb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:09.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.960+0000 7fadc1ffb640 1 -- 172.21.15.86:0/2426247557 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fadb00d0600 con 0x7fadc4106210 2024-04-16T05:45:09.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.960+0000 7fadc8a10640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fadb40a3d20 0x7fadb40a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fadb80058c0 tx=0x7fadb8005c60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:09.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:09.964+0000 7fadc1ffb640 1 -- 172.21.15.86:0/2426247557 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadb001f020 con 0x7fadc4106210 2024-04-16T05:45:10.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.180+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadc4101af0 con 0x7fadc4106210 2024-04-16T05:45:10.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.181+0000 7fadc1ffb640 1 -- 172.21.15.86:0/2426247557 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fadb000d040 con 0x7fadc4106210 2024-04-16T05:45:10.179 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:45:10.180 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:45:10.180 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:45:10.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fadb40a3d20 msgr2=0x7fadb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fadb40a3d20 0x7fadb40a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fadb80058c0 tx=0x7fadb8005c60 comp rx=0 tx=0).stop 2024-04-16T05:45:10.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 msgr2=0x7fadc4199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 0x7fadc4199ce0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fadb000b450 tx=0x7fadb000b910 comp rx=0 tx=0).stop 2024-04-16T05:45:10.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 shutdown_connections 2024-04-16T05:45:10.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fadb40a3d20 0x7fadb40a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fadc419a220 0x7fadc41a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc4106210 0x7fadc4199ce0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 --2- 172.21.15.86:0/2426247557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc4101140 0x7fadc41997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 >> 172.21.15.86:0/2426247557 conn(0x7fadc4077280 msgr2=0x7fadc40fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 shutdown_connections 2024-04-16T05:45:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.183+0000 7fadcac74640 1 -- 172.21.15.86:0/2426247557 wait complete. 2024-04-16T05:45:10.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:45:10.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.300+0000 7f44b4b59640 1 -- 172.21.15.86:0/2958653253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b01073a0 msgr2=0x7f44b010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.300+0000 7f44b4b59640 1 --2- 172.21.15.86:0/2958653253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b01073a0 0x7f44b010ef20 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009ca0 tx=0x7f44a802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.301+0000 7f44b4b59640 1 -- 172.21.15.86:0/2958653253 shutdown_connections 2024-04-16T05:45:10.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.301+0000 7f44b4b59640 1 --2- 172.21.15.86:0/2958653253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b01073a0 0x7f44b010ef20 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.301+0000 7f44b4b59640 1 --2- 172.21.15.86:0/2958653253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b0101140 0x7f44b0106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.301+0000 7f44b4b59640 1 --2- 172.21.15.86:0/2958653253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b0100790 0x7f44b0100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.301+0000 7f44b4b59640 1 -- 172.21.15.86:0/2958653253 >> 172.21.15.86:0/2958653253 conn(0x7f44b00774c0 msgr2=0x7f44b00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.301+0000 7f44b4b59640 1 -- 172.21.15.86:0/2958653253 shutdown_connections 2024-04-16T05:45:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.301+0000 7f44b4b59640 1 -- 172.21.15.86:0/2958653253 wait complete. 2024-04-16T05:45:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.301+0000 7f44b4b59640 1 Processor -- start 2024-04-16T05:45:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.301+0000 7f44b4b59640 1 -- start start 2024-04-16T05:45:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44b4b59640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b0100790 0x7f44b019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44b4b59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b01073a0 0x7f44b019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44b4b59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0198690 0x7f44b0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44b4b59640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0112410 con 0x7f44b0198690 2024-04-16T05:45:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44b4b59640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0112290 con 0x7f44b0100790 2024-04-16T05:45:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44b4b59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0112590 con 0x7f44b01073a0 2024-04-16T05:45:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0198690 0x7f44b0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0198690 0x7f44b0198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48076/0 (socket says 172.21.15.86:48076) 2024-04-16T05:45:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44aed9d640 1 -- 172.21.15.86:0/4166966405 learned_addr learned my addr 172.21.15.86:0/4166966405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44ae59c640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b0100790 0x7f44b019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44aed9d640 1 -- 172.21.15.86:0/4166966405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b01073a0 msgr2=0x7f44b019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44aed9d640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b01073a0 0x7f44b019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44aed9d640 1 -- 172.21.15.86:0/4166966405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b0100790 msgr2=0x7f44b019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44aed9d640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b0100790 0x7f44b019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44aed9d640 1 -- 172.21.15.86:0/4166966405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b0199290 con 0x7f44b0198690 2024-04-16T05:45:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44ae59c640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b0100790 0x7f44b019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44aed9d640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0198690 0x7f44b0198b10 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f44a8004af0 tx=0x7f44a8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f449f7fe640 1 -- 172.21.15.86:0/4166966405 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8041020 con 0x7f44b0198690 2024-04-16T05:45:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f449f7fe640 1 -- 172.21.15.86:0/4166966405 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a8034cd0 con 0x7f44b0198690 2024-04-16T05:45:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44b4b59640 1 -- 172.21.15.86:0/4166966405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b0199520 con 0x7f44b0198690 2024-04-16T05:45:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f449f7fe640 1 -- 172.21.15.86:0/4166966405 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8039040 con 0x7f44b0198690 2024-04-16T05:45:10.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.302+0000 7f44b4b59640 1 -- 172.21.15.86:0/4166966405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b01a56b0 con 0x7f44b0198690 2024-04-16T05:45:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.305+0000 7f449f7fe640 1 -- 172.21.15.86:0/4166966405 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f44a8044020 con 0x7f44b0198690 2024-04-16T05:45:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.305+0000 7f44b7829740 1 Processor -- start 2024-04-16T05:45:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.305+0000 7f44b7829740 1 -- start start 2024-04-16T05:45:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.305+0000 7f44b7829740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a75e60 0x562262a76240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.305+0000 7f44b7829740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.305+0000 7f44b7829740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a81750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.305+0000 7f44b7829740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622629db0e0 con 0x562262a7f360 2024-04-16T05:45:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.305+0000 7f44b7829740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562262978520 con 0x562262a75e60 2024-04-16T05:45:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.305+0000 7f44b7829740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622629d70a0 con 0x562262a76780 2024-04-16T05:45:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f449f7fe640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44980a3da0 0x7f44980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a81750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a81750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48082/0 (socket says 172.21.15.86:48082) 2024-04-16T05:45:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 -- 172.21.15.86:0/3596415788 learned_addr learned my addr 172.21.15.86:0/3596415788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44aed9d640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44980a3da0 0x7f44980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44ae59c640 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a75e60 0x562262a76240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f449f7fe640 1 -- 172.21.15.86:0/4166966405 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f44a80f29f0 con 0x7f44b0198690 2024-04-16T05:45:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 -- 172.21.15.86:0/3596415788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 msgr2=0x562262a7ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 -- 172.21.15.86:0/3596415788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a75e60 msgr2=0x562262a76240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a75e60 0x562262a76240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 -- 172.21.15.86:0/3596415788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622629a08d0 con 0x562262a7f360 2024-04-16T05:45:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44ae59c640 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44aed9d640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44980a3da0 0x7f44980a6260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f44a80be3d0 tx=0x7f44a80911c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f44add9b640 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a81750 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f44a400f390 tx=0x7f44a400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33e54483529421a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f449cff9640 1 -- 172.21.15.86:0/3596415788 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4012650 con 0x562262a7f360 2024-04-16T05:45:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f449cff9640 1 -- 172.21.15.86:0/3596415788 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a400fe50 con 0x562262a7f360 2024-04-16T05:45:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.306+0000 7f449cff9640 1 -- 172.21.15.86:0/3596415788 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4011b80 con 0x562262a7f360 2024-04-16T05:45:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 -- 172.21.15.86:0/3596415788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 msgr2=0x562262a81750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a81750 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f44a400f390 tx=0x7f44a400d4a0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 -- 172.21.15.86:0/3596415788 shutdown_connections 2024-04-16T05:45:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a81750 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 --2- 172.21.15.86:0/3596415788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a75e60 0x562262a76240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 -- 172.21.15.86:0/3596415788 >> 172.21.15.86:0/3596415788 conn(0x562262a70590 msgr2=0x562262a70970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 -- 172.21.15.86:0/3596415788 shutdown_connections 2024-04-16T05:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 -- 172.21.15.86:0/3596415788 wait complete. 2024-04-16T05:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.307+0000 7f44b7829740 1 -- start start 2024-04-16T05:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44b7829740 1 Processor -- start 2024-04-16T05:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44b7829740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a75e60 0x562262a7c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44b7829740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44b7829740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a7f360 0x562262a7cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44b7829740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562262a82000 con 0x562262a75e60 2024-04-16T05:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44b7829740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5622629944d0 con 0x562262a7f360 2024-04-16T05:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44b7829740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562262a82180 con 0x562262a76780 2024-04-16T05:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44b7829740 10 client.0 _ll_register_callbacks cb 0x5622629792c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a75e60 0x562262a7c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a75e60 0x562262a7c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48092/0 (socket says 172.21.15.86:48092) 2024-04-16T05:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44add9b640 1 -- 172.21.15.86:0/320746363 learned_addr learned my addr 172.21.15.86:0/320746363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44ae59c640 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44add9b640 1 -- 172.21.15.86:0/320746363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 msgr2=0x562262a7ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44add9b640 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44add9b640 1 -- 172.21.15.86:0/320746363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a7f360 msgr2=0x562262a7cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44add9b640 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a7f360 0x562262a7cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44add9b640 1 -- 172.21.15.86:0/320746363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622626ed3f0 con 0x562262a75e60 2024-04-16T05:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44ae59c640 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f44add9b640 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a75e60 0x562262a7c4f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f44a4011440 tx=0x7f44a4011470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d392f208da57fb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f4495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.308+0000 7f4495ffb640 1 -- 172.21.15.86:0/320746363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4011b90 con 0x562262a75e60 2024-04-16T05:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f4495ffb640 1 -- 172.21.15.86:0/320746363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f44a4011d30 con 0x562262a75e60 2024-04-16T05:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44b7829740 1 -- 172.21.15.86:0/320746363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a400fe50 con 0x562262a75e60 2024-04-16T05:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f4495ffb640 1 -- 172.21.15.86:0/320746363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44a4012450 con 0x562262a75e60 2024-04-16T05:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44b7829740 10 client.18492 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44b7829740 1 -- 172.21.15.86:0/320746363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622627b77d0 con 0x562262a75e60 2024-04-16T05:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f4495ffb640 1 -- 172.21.15.86:0/320746363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a400fe50 con 0x562262a75e60 2024-04-16T05:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f4495ffb640 1 -- 172.21.15.86:0/320746363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44a401d7f0 con 0x562262a75e60 2024-04-16T05:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44b7829740 10 client.18492 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44b7829740 10 client.18492 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44b7829740 20 client.18492 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44b7829740 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x562262a8f300 0x562262a916f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44b7829740 4 client.18492 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44b7829740 1 -- 172.21.15.86:0/320746363 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622629e8b00 con 0x562262a8f300 2024-04-16T05:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.309+0000 7f44ae59c640 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x562262a8f300 0x562262a916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.310+0000 7f44ae59c640 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x562262a8f300 0x562262a916f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d713a46f38f9a90 server_cookie=174281a841930db8 in_seq=0 out_seq=0 2024-04-16T05:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.310+0000 7f4495ffb640 10 client.18492 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.310+0000 7f4495ffb640 1 -- 172.21.15.86:0/320746363 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622629e8b00 con 0x562262a8f300 2024-04-16T05:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.310+0000 7f4495ffb640 10 client.18492 handle_command_reply: tid=0 2024-04-16T05:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 client.18492 shutdown 2024-04-16T05:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 -- 172.21.15.86:0/320746363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a75e60 msgr2=0x562262a7c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a75e60 0x562262a7c4f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f44a4011440 tx=0x7f44a4011470 comp rx=0 tx=0).stop 2024-04-16T05:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 -- 172.21.15.86:0/320746363 shutdown_connections 2024-04-16T05:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x562262a8f300 0x562262a916f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a7f360 0x562262a7cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a7ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f4495ffb640 0 client.18492 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:45:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 --2- 172.21.15.86:0/320746363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a75e60 0x562262a7c4f0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 -- 172.21.15.86:0/320746363 >> 172.21.15.86:0/320746363 conn(0x562262a70590 msgr2=0x562262a722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 -- 172.21.15.86:0/320746363 shutdown_connections 2024-04-16T05:45:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 1 -- 172.21.15.86:0/320746363 wait complete. 2024-04-16T05:45:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.311+0000 7f44b7829740 20 client.18492 trim_cache size 0 max 16384 2024-04-16T05:45:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44b7829740 1 Processor -- start 2024-04-16T05:45:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44b7829740 1 -- start start 2024-04-16T05:45:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44b7829740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a75e60 0x562262a7ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44b7829740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a8b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44b7829740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a6f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44b7829740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01af560 con 0x562262a7f360 2024-04-16T05:45:10.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44b7829740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01b10a0 con 0x562262a75e60 2024-04-16T05:45:10.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44b7829740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01afc30 con 0x562262a76780 2024-04-16T05:45:10.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a6f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a6f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48100/0 (socket says 172.21.15.86:48100) 2024-04-16T05:45:10.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44add9b640 1 -- 172.21.15.86:0/1589593282 learned_addr learned my addr 172.21.15.86:0/1589593282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44ae59c640 1 --2- 172.21.15.86:0/1589593282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a8b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44add9b640 1 -- 172.21.15.86:0/1589593282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 msgr2=0x562262a8b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44add9b640 1 --2- 172.21.15.86:0/1589593282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a8b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44add9b640 1 -- 172.21.15.86:0/1589593282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a75e60 msgr2=0x562262a7ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44add9b640 1 --2- 172.21.15.86:0/1589593282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a75e60 0x562262a7ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44add9b640 1 -- 172.21.15.86:0/1589593282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562262a7aa60 con 0x562262a7f360 2024-04-16T05:45:10.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.324+0000 7f44add9b640 1 --2- 172.21.15.86:0/1589593282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a6f5b0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f44a4006010 tx=0x7f44a4012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a075f7eabff95b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f4497fff640 1 -- 172.21.15.86:0/1589593282 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a40114e0 con 0x562262a7f360 2024-04-16T05:45:10.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f4497fff640 1 -- 172.21.15.86:0/1589593282 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a4024030 con 0x562262a7f360 2024-04-16T05:45:10.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f4497fff640 1 -- 172.21.15.86:0/1589593282 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4011bd0 con 0x562262a7f360 2024-04-16T05:45:10.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 -- 172.21.15.86:0/1589593282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 msgr2=0x562262a6f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 --2- 172.21.15.86:0/1589593282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a6f5b0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f44a4006010 tx=0x7f44a4012f30 comp rx=0 tx=0).stop 2024-04-16T05:45:10.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 -- 172.21.15.86:0/1589593282 shutdown_connections 2024-04-16T05:45:10.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 --2- 172.21.15.86:0/1589593282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a6f5b0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 --2- 172.21.15.86:0/1589593282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a76780 0x562262a8b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 --2- 172.21.15.86:0/1589593282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a75e60 0x562262a7ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 -- 172.21.15.86:0/1589593282 >> 172.21.15.86:0/1589593282 conn(0x562262a70590 msgr2=0x562262a6eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 -- 172.21.15.86:0/1589593282 shutdown_connections 2024-04-16T05:45:10.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 -- 172.21.15.86:0/1589593282 wait complete. 2024-04-16T05:45:10.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.325+0000 7f44b7829740 1 -- start start 2024-04-16T05:45:10.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44b7829740 1 Processor -- start 2024-04-16T05:45:10.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44b7829740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a75e60 0x562262a878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44b7829740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a76780 0x562262a78e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44b7829740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44b7829740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01b09d0 con 0x562262a7f360 2024-04-16T05:45:10.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44b7829740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01b0300 con 0x562262a76780 2024-04-16T05:45:10.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44b7829740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01ad560 con 0x562262a75e60 2024-04-16T05:45:10.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44b7829740 10 client.0 _ll_register_callbacks cb 0x7f44a400de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a793d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48114/0 (socket says 172.21.15.86:48114) 2024-04-16T05:45:10.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 -- 172.21.15.86:0/1011547225 learned_addr learned my addr 172.21.15.86:0/1011547225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44ae59c640 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a76780 0x562262a78e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a75e60 0x562262a878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 -- 172.21.15.86:0/1011547225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a75e60 msgr2=0x562262a878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a75e60 0x562262a878a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 -- 172.21.15.86:0/1011547225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a76780 msgr2=0x562262a78e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a76780 0x562262a78e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 -- 172.21.15.86:0/1011547225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562262a79c00 con 0x562262a7f360 2024-04-16T05:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f44add9b640 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a793d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f44a4011750 tx=0x7f44a400d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0a434ad967883b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f4495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f4495ffb640 1 -- 172.21.15.86:0/1011547225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a401de30 con 0x562262a7f360 2024-04-16T05:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.326+0000 7f4495ffb640 1 -- 172.21.15.86:0/1011547225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f44a4011440 con 0x562262a7f360 2024-04-16T05:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44b7829740 1 -- 172.21.15.86:0/1011547225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a4024030 con 0x562262a7f360 2024-04-16T05:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f4495ffb640 1 -- 172.21.15.86:0/1011547225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44a400d8d0 con 0x562262a7f360 2024-04-16T05:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44b7829740 10 client.18498 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44b7829740 1 -- 172.21.15.86:0/1011547225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562262af55c0 con 0x562262a7f360 2024-04-16T05:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f4495ffb640 1 -- 172.21.15.86:0/1011547225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a4024030 con 0x562262a7f360 2024-04-16T05:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f4495ffb640 1 -- 172.21.15.86:0/1011547225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44a401d420 con 0x562262a7f360 2024-04-16T05:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44b7829740 10 client.18498 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44b7829740 10 client.18498 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44b7829740 20 client.18498 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44b7829740 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x562262af5c20 0x562262af8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44b7829740 4 client.18498 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44b7829740 1 -- 172.21.15.86:0/1011547225 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622629e2970 con 0x562262af5c20 2024-04-16T05:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.327+0000 7f44ae59c640 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x562262af5c20 0x562262af8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f4495ffb640 10 client.18498 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44ae59c640 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x562262af5c20 0x562262af8010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2512e5756319404 server_cookie=c7dfb5f7c0cb9d8d in_seq=0 out_seq=0 2024-04-16T05:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f4495ffb640 1 -- 172.21.15.86:0/1011547225 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622629e2970 con 0x562262af5c20 2024-04-16T05:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f4495ffb640 10 client.18498 handle_command_reply: tid=0 2024-04-16T05:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 client.18498 shutdown 2024-04-16T05:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 -- 172.21.15.86:0/1011547225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 msgr2=0x562262a793d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a793d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f44a4011750 tx=0x7f44a400d5c0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 -- 172.21.15.86:0/1011547225 shutdown_connections 2024-04-16T05:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x562262af5c20 0x562262af8010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562262a7f360 0x562262a793d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562262a76780 0x562262a78e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f4495ffb640 0 client.18498 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 --2- 172.21.15.86:0/1011547225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562262a75e60 0x562262a878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 -- 172.21.15.86:0/1011547225 >> 172.21.15.86:0/1011547225 conn(0x562262a70590 msgr2=0x562262a7cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 -- 172.21.15.86:0/1011547225 shutdown_connections 2024-04-16T05:45:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 1 -- 172.21.15.86:0/1011547225 wait complete. 2024-04-16T05:45:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.328+0000 7f44b7829740 20 client.18498 trim_cache size 0 max 16384 2024-04-16T05:45:10.342 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 -- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44980a3da0 msgr2=0x7f44980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44980a3da0 0x7f44980a6260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f44a80be3d0 tx=0x7f44a80911c0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 -- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0198690 msgr2=0x7f44b0198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0198690 0x7f44b0198b10 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f44a8004af0 tx=0x7f44a8032f40 comp rx=0 tx=0).stop 2024-04-16T05:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 -- 172.21.15.86:0/4166966405 shutdown_connections 2024-04-16T05:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0198690 0x7f44b0198b10 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b01073a0 0x7f44b019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44980a3da0 0x7f44980a6260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 --2- 172.21.15.86:0/4166966405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b0100790 0x7f44b019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 -- 172.21.15.86:0/4166966405 >> 172.21.15.86:0/4166966405 conn(0x7f44b00774c0 msgr2=0x7f44b01054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 -- 172.21.15.86:0/4166966405 shutdown_connections 2024-04-16T05:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.329+0000 7f44b4b59640 1 -- 172.21.15.86:0/4166966405 wait complete. 2024-04-16T05:45:10.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:45:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.456+0000 7f28c4ceb640 1 -- 172.21.15.86:0/3146902674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102bf0 msgr2=0x7f28c0106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.456+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/3146902674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102bf0 0x7f28c0106c30 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009900 tx=0x7f28b402bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.456+0000 7f28c4ceb640 1 -- 172.21.15.86:0/3146902674 shutdown_connections 2024-04-16T05:45:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.456+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/3146902674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28c0107310 0x7f28c010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.456+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/3146902674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102bf0 0x7f28c0106c30 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.456+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/3146902674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28c0102240 0x7f28c0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.456+0000 7f28c4ceb640 1 -- 172.21.15.86:0/3146902674 >> 172.21.15.86:0/3146902674 conn(0x7f28c00fdae0 msgr2=0x7f28c00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.456+0000 7f28c4ceb640 1 -- 172.21.15.86:0/3146902674 shutdown_connections 2024-04-16T05:45:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.456+0000 7f28c4ceb640 1 -- 172.21.15.86:0/3146902674 wait complete. 2024-04-16T05:45:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28c4ceb640 1 Processor -- start 2024-04-16T05:45:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28c4ceb640 1 -- start start 2024-04-16T05:45:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28c4ceb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102240 0x7f28c019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28c4ceb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28c0107310 0x7f28c019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28c4ceb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28c01987a0 0x7f28c0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28c4ceb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01123a0 con 0x7f28c0102240 2024-04-16T05:45:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28c4ceb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f28c0112220 con 0x7f28c01987a0 2024-04-16T05:45:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28c4ceb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28c0112520 con 0x7f28c0107310 2024-04-16T05:45:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102240 0x7f28c019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102240 0x7f28c019e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48132/0 (socket says 172.21.15.86:48132) 2024-04-16T05:45:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28be59c640 1 -- 172.21.15.86:0/1272433610 learned_addr learned my addr 172.21.15.86:0/1272433610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28be59c640 1 -- 172.21.15.86:0/1272433610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28c0107310 msgr2=0x7f28c019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28be59c640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28c0107310 0x7f28c019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28bed9d640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28c01987a0 0x7f28c0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28be59c640 1 -- 172.21.15.86:0/1272433610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28c01987a0 msgr2=0x7f28c0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28be59c640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28c01987a0 0x7f28c0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28be59c640 1 -- 172.21.15.86:0/1272433610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c0199330 con 0x7f28c0102240 2024-04-16T05:45:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28bed9d640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28c01987a0 0x7f28c0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.457+0000 7f28be59c640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102240 0x7f28c019e630 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f28b000ea80 tx=0x7f28b000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.458+0000 7f28af7fe640 1 -- 172.21.15.86:0/1272433610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b000ce40 con 0x7f28c0102240 2024-04-16T05:45:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.458+0000 7f28af7fe640 1 -- 172.21.15.86:0/1272433610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b0005020 con 0x7f28c0102240 2024-04-16T05:45:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.458+0000 7f28c4ceb640 1 -- 172.21.15.86:0/1272433610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c0199620 con 0x7f28c0102240 2024-04-16T05:45:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.458+0000 7f28af7fe640 1 -- 172.21.15.86:0/1272433610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0011760 con 0x7f28c0102240 2024-04-16T05:45:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.458+0000 7f28c4ceb640 1 -- 172.21.15.86:0/1272433610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c01a57b0 con 0x7f28c0102240 2024-04-16T05:45:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.460+0000 7f28af7fe640 1 -- 172.21.15.86:0/1272433610 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f28b00051d0 con 0x7f28c0102240 2024-04-16T05:45:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.460+0000 7f28c79b3740 1 Processor -- start 2024-04-16T05:45:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28c79b3740 1 -- start start 2024-04-16T05:45:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 0x556962da4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da4780 0x556962dace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 0x556962daf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28c79b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556962d090e0 con 0x556962da4780 2024-04-16T05:45:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28c79b3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556962ca6520 con 0x556962da3e60 2024-04-16T05:45:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28c79b3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556962d050a0 con 0x556962dad360 2024-04-16T05:45:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da4780 0x556962dace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da4780 0x556962dace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48146/0 (socket says 172.21.15.86:48146) 2024-04-16T05:45:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28af7fe640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f28a80a3d20 0x7f28a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28be59c640 1 -- 172.21.15.86:0/203499748 learned_addr learned my addr 172.21.15.86:0/203499748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28bed9d640 1 --2- 172.21.15.86:0/203499748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 0x556962da4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28bed9d640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f28a80a3d20 0x7f28a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28bdd9b640 1 --2- 172.21.15.86:0/203499748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 0x556962daf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28af7fe640 1 -- 172.21.15.86:0/1272433610 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f28b00d1360 con 0x7f28c0102240 2024-04-16T05:45:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28be59c640 1 -- 172.21.15.86:0/203499748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 msgr2=0x556962daf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28be59c640 1 --2- 172.21.15.86:0/203499748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 0x556962daf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28be59c640 1 -- 172.21.15.86:0/203499748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 msgr2=0x556962da4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28be59c640 1 --2- 172.21.15.86:0/203499748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 0x556962da4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.461+0000 7f28be59c640 1 -- 172.21.15.86:0/203499748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556962cce8d0 con 0x556962da4780 2024-04-16T05:45:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28be59c640 1 --2- 172.21.15.86:0/203499748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da4780 0x556962dace20 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f28b0099300 tx=0x7f28b009c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e58ea0b2579a5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28bed9d640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f28a80a3d20 0x7f28a80a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f28b8005fd0 tx=0x7f28b800d300 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28acff9640 1 -- 172.21.15.86:0/203499748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b001f070 con 0x556962da4780 2024-04-16T05:45:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28acff9640 1 -- 172.21.15.86:0/203499748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b009cea0 con 0x556962da4780 2024-04-16T05:45:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28acff9640 1 -- 172.21.15.86:0/203499748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00ce8d0 con 0x556962da4780 2024-04-16T05:45:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28c79b3740 1 -- 172.21.15.86:0/203499748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da4780 msgr2=0x556962dace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28c79b3740 1 --2- 172.21.15.86:0/203499748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da4780 0x556962dace20 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f28b0099300 tx=0x7f28b009c800 comp rx=0 tx=0).stop 2024-04-16T05:45:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28c79b3740 1 -- 172.21.15.86:0/203499748 shutdown_connections 2024-04-16T05:45:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28c79b3740 1 --2- 172.21.15.86:0/203499748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 0x556962daf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28c79b3740 1 --2- 172.21.15.86:0/203499748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da4780 0x556962dace20 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28c79b3740 1 --2- 172.21.15.86:0/203499748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 0x556962da4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.462+0000 7f28c79b3740 1 -- 172.21.15.86:0/203499748 >> 172.21.15.86:0/203499748 conn(0x556962d9e590 msgr2=0x556962d9e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 -- 172.21.15.86:0/203499748 shutdown_connections 2024-04-16T05:45:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 -- 172.21.15.86:0/203499748 wait complete. 2024-04-16T05:45:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 -- start start 2024-04-16T05:45:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 Processor -- start 2024-04-16T05:45:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da3e60 0x556962daa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da4780 0x556962daa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 0x556962daaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556962daffa0 con 0x556962da3e60 2024-04-16T05:45:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556962cc24d0 con 0x556962da4780 2024-04-16T05:45:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556962db0120 con 0x556962dad360 2024-04-16T05:45:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28c79b3740 10 client.0 _ll_register_callbacks cb 0x556962ca72c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da3e60 0x556962daa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da3e60 0x556962daa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48160/0 (socket says 172.21.15.86:48160) 2024-04-16T05:45:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28be59c640 1 -- 172.21.15.86:0/1785157807 learned_addr learned my addr 172.21.15.86:0/1785157807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.463+0000 7f28bed9d640 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 0x556962daaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28be59c640 1 -- 172.21.15.86:0/1785157807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 msgr2=0x556962daaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28be59c640 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 0x556962daaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28bdd9b640 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da4780 0x556962daa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28be59c640 1 -- 172.21.15.86:0/1785157807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da4780 msgr2=0x556962daa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28be59c640 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da4780 0x556962daa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28be59c640 1 -- 172.21.15.86:0/1785157807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556962a1b3f0 con 0x556962da3e60 2024-04-16T05:45:10.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28bed9d640 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 0x556962daaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:10.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28be59c640 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da3e60 0x556962daa480 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f28b009ce30 tx=0x7f28b009d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4946ff4083a513 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:10.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28a5ffb640 1 -- 172.21.15.86:0/1785157807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00b6060 con 0x556962da3e60 2024-04-16T05:45:10.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28a5ffb640 1 -- 172.21.15.86:0/1785157807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f28b00d1120 con 0x556962da3e60 2024-04-16T05:45:10.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28c79b3740 1 -- 172.21.15.86:0/1785157807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b009cea0 con 0x556962da3e60 2024-04-16T05:45:10.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28a5ffb640 1 -- 172.21.15.86:0/1785157807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28b001f070 con 0x556962da3e60 2024-04-16T05:45:10.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28c79b3740 10 client.18510 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28c79b3740 1 -- 172.21.15.86:0/1785157807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556962ae57d0 con 0x556962da3e60 2024-04-16T05:45:10.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28a5ffb640 1 -- 172.21.15.86:0/1785157807 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b009cea0 con 0x556962da3e60 2024-04-16T05:45:10.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28a5ffb640 1 -- 172.21.15.86:0/1785157807 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f28b000f040 con 0x556962da3e60 2024-04-16T05:45:10.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28c79b3740 10 client.18510 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28c79b3740 10 client.18510 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:45:10.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28c79b3740 20 client.18510 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28c79b3740 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x556962dbd190 0x556962dbf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28c79b3740 4 client.18510 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.464+0000 7f28c79b3740 1 -- 172.21.15.86:0/1785157807 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556962d16b00 con 0x556962dbd190 2024-04-16T05:45:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.465+0000 7f28bdd9b640 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x556962dbd190 0x556962dbf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.465+0000 7f28bdd9b640 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x556962dbd190 0x556962dbf580 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=760d9d0a174d443a server_cookie=3372fa40b54c51dd in_seq=0 out_seq=0 2024-04-16T05:45:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.465+0000 7f28a5ffb640 10 client.18510 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:45:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28a5ffb640 1 -- 172.21.15.86:0/1785157807 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556962d16b00 con 0x556962dbd190 2024-04-16T05:45:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28a5ffb640 10 client.18510 handle_command_reply: tid=0 2024-04-16T05:45:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 client.18510 shutdown 2024-04-16T05:45:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 -- 172.21.15.86:0/1785157807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da3e60 msgr2=0x556962daa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da3e60 0x556962daa480 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f28b009ce30 tx=0x7f28b009d660 comp rx=0 tx=0).stop 2024-04-16T05:45:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 -- 172.21.15.86:0/1785157807 shutdown_connections 2024-04-16T05:45:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x556962dbd190 0x556962dbf580 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962dad360 0x556962daaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da4780 0x556962daa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 --2- 172.21.15.86:0/1785157807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962da3e60 0x556962daa480 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28a5ffb640 0 client.18510 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:45:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 -- 172.21.15.86:0/1785157807 >> 172.21.15.86:0/1785157807 conn(0x556962d9e590 msgr2=0x556962da02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 -- 172.21.15.86:0/1785157807 shutdown_connections 2024-04-16T05:45:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 1 -- 172.21.15.86:0/1785157807 wait complete. 2024-04-16T05:45:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.466+0000 7f28c79b3740 20 client.18510 trim_cache size 0 max 16384 2024-04-16T05:45:10.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:10 smithi086 ceph-mon[32028]: pgmap v305: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:45:10.480 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:10 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2426247557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:45:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.479+0000 7f28c79b3740 1 Processor -- start 2024-04-16T05:45:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.479+0000 7f28c79b3740 1 -- start start 2024-04-16T05:45:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.479+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 0x556962da9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.479+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da4780 0x556962dac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.479+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dc25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.479+0000 7f28c79b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01af800 con 0x556962dad360 2024-04-16T05:45:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.479+0000 7f28c79b3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01b1340 con 0x556962da3e60 2024-04-16T05:45:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.479+0000 7f28c79b3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01afed0 con 0x556962da4780 2024-04-16T05:45:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dc25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dc25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48172/0 (socket says 172.21.15.86:48172) 2024-04-16T05:45:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bed9d640 1 -- 172.21.15.86:0/2992694577 learned_addr learned my addr 172.21.15.86:0/2992694577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bdd9b640 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da4780 0x556962dac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28be59c640 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 0x556962da9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bed9d640 1 -- 172.21.15.86:0/2992694577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da4780 msgr2=0x556962dac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bed9d640 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da4780 0x556962dac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bed9d640 1 -- 172.21.15.86:0/2992694577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 msgr2=0x556962da9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bed9d640 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 0x556962da9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bed9d640 1 -- 172.21.15.86:0/2992694577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556962dafac0 con 0x556962dad360 2024-04-16T05:45:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bdd9b640 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da4780 0x556962dac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28bed9d640 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dc25a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f28b8005c90 tx=0x7f28b801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbb15c3c839f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28a7fff640 1 -- 172.21.15.86:0/2992694577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8005220 con 0x556962dad360 2024-04-16T05:45:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28a7fff640 1 -- 172.21.15.86:0/2992694577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b8004c80 con 0x556962dad360 2024-04-16T05:45:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.480+0000 7f28a7fff640 1 -- 172.21.15.86:0/2992694577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b80039d0 con 0x556962dad360 2024-04-16T05:45:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 -- 172.21.15.86:0/2992694577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 msgr2=0x556962dc25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dc25a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f28b8005c90 tx=0x7f28b801d000 comp rx=0 tx=0).stop 2024-04-16T05:45:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 -- 172.21.15.86:0/2992694577 shutdown_connections 2024-04-16T05:45:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dc25a0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da4780 0x556962dac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 --2- 172.21.15.86:0/2992694577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da3e60 0x556962da9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 -- 172.21.15.86:0/2992694577 >> 172.21.15.86:0/2992694577 conn(0x556962d9e590 msgr2=0x556962dabba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 -- 172.21.15.86:0/2992694577 shutdown_connections 2024-04-16T05:45:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 -- 172.21.15.86:0/2992694577 wait complete. 2024-04-16T05:45:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 -- start start 2024-04-16T05:45:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 Processor -- start 2024-04-16T05:45:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da3e60 0x556962dc21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da4780 0x556962dc2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dbf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01b0c70 con 0x556962dad360 2024-04-16T05:45:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01b05a0 con 0x556962da4780 2024-04-16T05:45:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.481+0000 7f28c79b3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01ad7e0 con 0x556962da3e60 2024-04-16T05:45:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28c79b3740 10 client.0 _ll_register_callbacks cb 0x7f28b800def0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dbf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dbf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48186/0 (socket says 172.21.15.86:48186) 2024-04-16T05:45:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28bed9d640 1 -- 172.21.15.86:0/3380575281 learned_addr learned my addr 172.21.15.86:0/3380575281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28be59c640 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da3e60 0x556962dc21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28bed9d640 1 -- 172.21.15.86:0/3380575281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da3e60 msgr2=0x556962dc21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28bed9d640 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da3e60 0x556962dc21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28bed9d640 1 -- 172.21.15.86:0/3380575281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da4780 msgr2=0x556962dc2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28bed9d640 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da4780 0x556962dc2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28bed9d640 1 -- 172.21.15.86:0/3380575281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556962dbfe20 con 0x556962dad360 2024-04-16T05:45:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28bed9d640 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dbf6e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f28b8010d40 tx=0x7f28b80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c916c019c4e3f2a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28a5ffb640 1 -- 172.21.15.86:0/3380575281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b803a060 con 0x556962dad360 2024-04-16T05:45:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28c79b3740 1 -- 172.21.15.86:0/3380575281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b8004c80 con 0x556962dad360 2024-04-16T05:45:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28a5ffb640 1 -- 172.21.15.86:0/3380575281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f28b8004940 con 0x556962dad360 2024-04-16T05:45:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28a5ffb640 1 -- 172.21.15.86:0/3380575281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28b8041050 con 0x556962dad360 2024-04-16T05:45:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28c79b3740 10 client.18516 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28c79b3740 1 -- 172.21.15.86:0/3380575281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556962e27180 con 0x556962dad360 2024-04-16T05:45:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.482+0000 7f28a5ffb640 1 -- 172.21.15.86:0/3380575281 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b8004c80 con 0x556962dad360 2024-04-16T05:45:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28a5ffb640 1 -- 172.21.15.86:0/3380575281 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f28b8003c50 con 0x556962dad360 2024-04-16T05:45:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28c79b3740 10 client.18516 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28c79b3740 10 client.18516 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:45:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28c79b3740 20 client.18516 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28c79b3740 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x556962e278c0 0x556962e29ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28c79b3740 4 client.18516 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28c79b3740 1 -- 172.21.15.86:0/3380575281 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556962d10970 con 0x556962e278c0 2024-04-16T05:45:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28be59c640 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x556962e278c0 0x556962e29ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28a5ffb640 10 client.18516 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:45:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.483+0000 7f28be59c640 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x556962e278c0 0x556962e29ce0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77c2df26d687395d server_cookie=68a8897bff13e571 in_seq=0 out_seq=0 2024-04-16T05:45:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28a5ffb640 1 -- 172.21.15.86:0/3380575281 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556962d10970 con 0x556962e278c0 2024-04-16T05:45:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28a5ffb640 10 client.18516 handle_command_reply: tid=0 2024-04-16T05:45:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 client.18516 shutdown 2024-04-16T05:45:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 -- 172.21.15.86:0/3380575281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 msgr2=0x556962dbf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dbf6e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f28b8010d40 tx=0x7f28b80035f0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 -- 172.21.15.86:0/3380575281 shutdown_connections 2024-04-16T05:45:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556962dad360 0x556962dbf6e0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556962da4780 0x556962dc2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x556962e278c0 0x556962e29ce0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 --2- 172.21.15.86:0/3380575281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556962da3e60 0x556962dc21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28a5ffb640 0 client.18516 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:45:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 -- 172.21.15.86:0/3380575281 >> 172.21.15.86:0/3380575281 conn(0x556962d9e590 msgr2=0x556962da01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 -- 172.21.15.86:0/3380575281 shutdown_connections 2024-04-16T05:45:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 1 -- 172.21.15.86:0/3380575281 wait complete. 2024-04-16T05:45:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c79b3740 20 client.18516 trim_cache size 0 max 16384 2024-04-16T05:45:10.497 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:10.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 -- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f28a80a3d20 msgr2=0x7f28a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f28a80a3d20 0x7f28a80a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f28b8005fd0 tx=0x7f28b800d300 comp rx=0 tx=0).stop 2024-04-16T05:45:10.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 -- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102240 msgr2=0x7f28c019e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102240 0x7f28c019e630 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f28b000ea80 tx=0x7f28b000ef40 comp rx=0 tx=0).stop 2024-04-16T05:45:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 -- 172.21.15.86:0/1272433610 shutdown_connections 2024-04-16T05:45:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f28a80a3d20 0x7f28a80a61e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28c01987a0 0x7f28c0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28c0107310 0x7f28c019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 --2- 172.21.15.86:0/1272433610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c0102240 0x7f28c019e630 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 -- 172.21.15.86:0/1272433610 >> 172.21.15.86:0/1272433610 conn(0x7f28c00fdae0 msgr2=0x7f28c00ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 -- 172.21.15.86:0/1272433610 shutdown_connections 2024-04-16T05:45:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.484+0000 7f28c4ceb640 1 -- 172.21.15.86:0/1272433610 wait complete. 2024-04-16T05:45:10.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:45:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.609+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3060846050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 msgr2=0x7f1cb810eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.609+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3060846050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 0x7f1cb810eeb0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009930 tx=0x7f1cb002bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.609+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3060846050 shutdown_connections 2024-04-16T05:45:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.609+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3060846050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 0x7f1cb810eeb0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.609+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3060846050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cb8100490 0x7f1cb81008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.609+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3060846050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cb8108080 0x7f1cb8108460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.609+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3060846050 >> 172.21.15.86:0/3060846050 conn(0x7f1cb8077410 msgr2=0x7f1cb80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.609+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3060846050 shutdown_connections 2024-04-16T05:45:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.609+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3060846050 wait complete. 2024-04-16T05:45:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cc01aa640 1 Processor -- start 2024-04-16T05:45:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cc01aa640 1 -- start start 2024-04-16T05:45:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cc01aa640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cb8100490 0x7f1cb819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cc01aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 0x7f1cb819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cc01aa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cb8108080 0x7f1cb8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cc01aa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81123a0 con 0x7f1cb8100e10 2024-04-16T05:45:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cc01aa640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8112220 con 0x7f1cb8100490 2024-04-16T05:45:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cc01aa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8112520 con 0x7f1cb8108080 2024-04-16T05:45:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cbd745640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 0x7f1cb819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cbd745640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 0x7f1cb819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42460/0 (socket says 172.21.15.86:42460) 2024-04-16T05:45:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cbd745640 1 -- 172.21.15.86:0/3195426122 learned_addr learned my addr 172.21.15.86:0/3195426122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.610+0000 7f1cbe747640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cb8108080 0x7f1cb8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1cbd745640 1 -- 172.21.15.86:0/3195426122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cb8108080 msgr2=0x7f1cb8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cb8100490 0x7f1cb819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1cbd745640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cb8108080 0x7f1cb8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1cbd745640 1 -- 172.21.15.86:0/3195426122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cb8100490 msgr2=0x7f1cb819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1cbd745640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cb8100490 0x7f1cb819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1cbd745640 1 -- 172.21.15.86:0/3195426122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb8198fc0 con 0x7f1cb8100e10 2024-04-16T05:45:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1cbd745640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 0x7f1cb819eb80 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800d950 tx=0x7f1ca800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1caeffd640 1 -- 172.21.15.86:0/3195426122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8017070 con 0x7f1cb8100e10 2024-04-16T05:45:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1caeffd640 1 -- 172.21.15.86:0/3195426122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca8009c30 con 0x7f1cb8100e10 2024-04-16T05:45:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3195426122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb81992b0 con 0x7f1cb8100e10 2024-04-16T05:45:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1caeffd640 1 -- 172.21.15.86:0/3195426122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800f6b0 con 0x7f1cb8100e10 2024-04-16T05:45:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.611+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3195426122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb81a55c0 con 0x7f1cb8100e10 2024-04-16T05:45:10.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1caeffd640 1 -- 172.21.15.86:0/3195426122 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1ca80109d0 con 0x7f1cb8100e10 2024-04-16T05:45:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cc2d8b740 1 Processor -- start 2024-04-16T05:45:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cc2d8b740 1 -- start start 2024-04-16T05:45:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993eb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5993450e0 con 0x55c5993e9360 2024-04-16T05:45:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c5992e2520 con 0x55c5993e0780 2024-04-16T05:45:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c5993410a0 con 0x55c5993dfe60 2024-04-16T05:45:10.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cbd745640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.614+0000 7f1cbd745640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993eb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42470/0 (socket says 172.21.15.86:42470) 2024-04-16T05:45:10.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbd745640 1 -- 172.21.15.86:0/709634884 learned_addr learned my addr 172.21.15.86:0/709634884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1caeffd640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbe747640 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbd745640 1 -- 172.21.15.86:0/709634884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 msgr2=0x55c5993e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbd745640 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbd745640 1 -- 172.21.15.86:0/709634884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 msgr2=0x55c5993e8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbd745640 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbd745640 1 -- 172.21.15.86:0/709634884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59930a8d0 con 0x55c5993e9360 2024-04-16T05:45:10.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbe747640 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbe747640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1caeffd640 1 -- 172.21.15.86:0/3195426122 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f1ca80d0900 con 0x7f1cb8100e10 2024-04-16T05:45:10.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbd745640 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993eb750 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80cdb70 tx=0x7f1ca8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92308dca9a99fd88 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1ca3fff640 1 -- 172.21.15.86:0/709634884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8020020 con 0x55c5993e9360 2024-04-16T05:45:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1ca3fff640 1 -- 172.21.15.86:0/709634884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca8009de0 con 0x55c5993e9360 2024-04-16T05:45:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1cbe747640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006010 tx=0x7f1cb0009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.615+0000 7f1ca3fff640 1 -- 172.21.15.86:0/709634884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca80071c0 con 0x55c5993e9360 2024-04-16T05:45:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/709634884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 msgr2=0x55c5993eb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993eb750 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80cdb70 tx=0x7f1ca8022000 comp rx=0 tx=0).stop 2024-04-16T05:45:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/709634884 shutdown_connections 2024-04-16T05:45:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993eb750 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/709634884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/709634884 >> 172.21.15.86:0/709634884 conn(0x55c5993da590 msgr2=0x55c5993da970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/709634884 shutdown_connections 2024-04-16T05:45:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/709634884 wait complete. 2024-04-16T05:45:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.616+0000 7f1cc2d8b740 1 -- start start 2024-04-16T05:45:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cc2d8b740 1 Processor -- start 2024-04-16T05:45:10.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e0780 0x55c5993e6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5993e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5993ebfa0 con 0x55c5993e0780 2024-04-16T05:45:10.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c5992fe4d0 con 0x55c5993e9360 2024-04-16T05:45:10.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c5993ec120 con 0x55c5993dfe60 2024-04-16T05:45:10.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cc2d8b740 10 client.0 _ll_register_callbacks cb 0x55c5992e32c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbd745640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e0780 0x55c5993e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbd745640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e0780 0x55c5993e6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42476/0 (socket says 172.21.15.86:42476) 2024-04-16T05:45:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbd745640 1 -- 172.21.15.86:0/1260960563 learned_addr learned my addr 172.21.15.86:0/1260960563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbe747640 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5993e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbd745640 1 -- 172.21.15.86:0/1260960563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 msgr2=0x55c5993e6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbd745640 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbd745640 1 -- 172.21.15.86:0/1260960563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 msgr2=0x55c5993e6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbd745640 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5993e6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbd745640 1 -- 172.21.15.86:0/1260960563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5990573f0 con 0x55c5993e0780 2024-04-16T05:45:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.617+0000 7f1cbe747640 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5993e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cbd745640 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e0780 0x55c5993e6850 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800bfa0 tx=0x7f1ca809c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8986a816a8f2729d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1ca17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1260960563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8020030 con 0x55c5993e0780 2024-04-16T05:45:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1260960563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f1ca809a780 con 0x55c5993e0780 2024-04-16T05:45:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1260960563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca8009de0 con 0x55c5993e0780 2024-04-16T05:45:10.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1260960563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ca809c480 con 0x55c5993e0780 2024-04-16T05:45:10.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cc2d8b740 10 client.18528 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1260960563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5991217d0 con 0x55c5993e0780 2024-04-16T05:45:10.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1260960563 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca8009de0 con 0x55c5993e0780 2024-04-16T05:45:10.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1260960563 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1ca8098af0 con 0x55c5993e0780 2024-04-16T05:45:10.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cc2d8b740 10 client.18528 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cc2d8b740 10 client.18528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:45:10.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cc2d8b740 20 client.18528 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c5993f8fd0 0x55c5993fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cc2d8b740 4 client.18528 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:10.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.618+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1260960563 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c599352b00 con 0x55c5993f8fd0 2024-04-16T05:45:10.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.619+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c5993f8fd0 0x55c5993fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.619+0000 7f1ca17fa640 10 client.18528 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:45:10.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.619+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c5993f8fd0 0x55c5993fb3c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44abef13550b2187 server_cookie=e4539867c72ac0c2 in_seq=0 out_seq=0 2024-04-16T05:45:10.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1260960563 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c599352b00 con 0x55c5993f8fd0 2024-04-16T05:45:10.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1ca17fa640 10 client.18528 handle_command_reply: tid=0 2024-04-16T05:45:10.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 client.18528 shutdown 2024-04-16T05:45:10.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1260960563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e0780 msgr2=0x55c5993e6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e0780 0x55c5993e6850 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800bfa0 tx=0x7f1ca809c340 comp rx=0 tx=0).stop 2024-04-16T05:45:10.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1260960563 shutdown_connections 2024-04-16T05:45:10.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5993e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c5993f8fd0 0x55c5993fb3c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e0780 0x55c5993e6850 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1260960563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1ca17fa640 0 client.18528 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:45:10.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1260960563 >> 172.21.15.86:0/1260960563 conn(0x55c5993da590 msgr2=0x55c5993dc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1260960563 shutdown_connections 2024-04-16T05:45:10.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1260960563 wait complete. 2024-04-16T05:45:10.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.620+0000 7f1cc2d8b740 20 client.18528 trim_cache size 0 max 16384 2024-04-16T05:45:10.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.637+0000 7f1cc2d8b740 1 Processor -- start 2024-04-16T05:45:10.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.637+0000 7f1cc2d8b740 1 -- start start 2024-04-16T05:45:10.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993dfe60 0x55c5993e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993e0780 0x55c5993f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5994009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81af770 con 0x55c5993dfe60 2024-04-16T05:45:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81b12b0 con 0x55c5993e9360 2024-04-16T05:45:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81afe40 con 0x55c5993e0780 2024-04-16T05:45:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cbe747640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5994009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cbdf46640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993dfe60 0x55c5993e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cbdf46640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993dfe60 0x55c5993e8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42480/0 (socket says 172.21.15.86:42480) 2024-04-16T05:45:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cbdf46640 1 -- 172.21.15.86:0/3704925072 learned_addr learned my addr 172.21.15.86:0/3704925072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.638+0000 7f1cbd745640 1 --2- 172.21.15.86:0/3704925072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993e0780 0x55c5993f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1cbdf46640 1 -- 172.21.15.86:0/3704925072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993e0780 msgr2=0x55c5993f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/3704925072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993e0780 0x55c5993f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1cbdf46640 1 -- 172.21.15.86:0/3704925072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 msgr2=0x55c5994009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/3704925072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5994009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1cbdf46640 1 -- 172.21.15.86:0/3704925072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5993da3c0 con 0x55c5993dfe60 2024-04-16T05:45:10.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1cbe747640 1 --2- 172.21.15.86:0/3704925072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5994009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:45:10.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/3704925072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993dfe60 0x55c5993e8a40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x55c5993fb960 tx=0x7f1cb400b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571f423e2df03ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1ca37fe640 1 -- 172.21.15.86:0/3704925072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4018bc0 con 0x55c5993dfe60 2024-04-16T05:45:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1ca37fe640 1 -- 172.21.15.86:0/3704925072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb4018d60 con 0x55c5993dfe60 2024-04-16T05:45:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.639+0000 7f1ca37fe640 1 -- 172.21.15.86:0/3704925072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400f680 con 0x55c5993dfe60 2024-04-16T05:45:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.640+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/3704925072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993dfe60 msgr2=0x55c5993e8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.640+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/3704925072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993dfe60 0x55c5993e8a40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x55c5993fb960 tx=0x7f1cb400b490 comp rx=0 tx=0).stop 2024-04-16T05:45:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.640+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/3704925072 shutdown_connections 2024-04-16T05:45:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.640+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/3704925072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e9360 0x55c5994009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.640+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/3704925072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993e0780 0x55c5993f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.640+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/3704925072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993dfe60 0x55c5993e8a40 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.640+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/3704925072 >> 172.21.15.86:0/3704925072 conn(0x55c5993da590 msgr2=0x55c5993d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.640+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/3704925072 shutdown_connections 2024-04-16T05:45:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/3704925072 wait complete. 2024-04-16T05:45:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 1 -- start start 2024-04-16T05:45:10.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 1 Processor -- start 2024-04-16T05:45:10.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81b0be0 con 0x55c5993e9360 2024-04-16T05:45:10.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81b0510 con 0x55c5993e0780 2024-04-16T05:45:10.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81ad750 con 0x55c5993dfe60 2024-04-16T05:45:10.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.641+0000 7f1cc2d8b740 10 client.0 _ll_register_callbacks cb 0x7f1ca80985e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbd745640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbd745640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41790/0 (socket says 172.21.15.86:41790) 2024-04-16T05:45:10.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbd745640 1 -- 172.21.15.86:0/1390133865 learned_addr learned my addr 172.21.15.86:0/1390133865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbe747640 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbd745640 1 -- 172.21.15.86:0/1390133865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 msgr2=0x55c5993e2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbd745640 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbd745640 1 -- 172.21.15.86:0/1390133865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 msgr2=0x55c5993e3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbd745640 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993e3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbd745640 1 -- 172.21.15.86:0/1390133865 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c59945e2e0 con 0x55c5993e0780 2024-04-16T05:45:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1cbd745640 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e3520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8056180 tx=0x7f1ca80980f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a822425e21e002d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.642+0000 7f1ca17fa640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-16T05:45:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.643+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1390133865 <== mon.2 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8020070 con 0x55c5993e0780 2024-04-16T05:45:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.643+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1390133865 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb4018d60 con 0x55c5993e0780 2024-04-16T05:45:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.643+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1390133865 <== mon.2 v2:172.21.15.194:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f1ca8089030 con 0x55c5993e0780 2024-04-16T05:45:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.643+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1390133865 <== mon.2 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ca80ce0c0 con 0x55c5993e0780 2024-04-16T05:45:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.643+0000 7f1cc2d8b740 10 client.26795 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.643+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1390133865 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c59945f5c0 con 0x55c5993e0780 2024-04-16T05:45:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.643+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1390133865 <== mon.2 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb4018d60 con 0x55c5993e0780 2024-04-16T05:45:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1390133865 <== mon.2 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1ca809c6e0 con 0x55c5993e0780 2024-04-16T05:45:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1cc2d8b740 10 client.26795 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1cc2d8b740 10 client.26795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:45:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1cc2d8b740 20 client.26795 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c59945fb60 0x55c599461f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1cc2d8b740 4 client.26795 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1390133865 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c59934c970 con 0x55c59945fb60 2024-04-16T05:45:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c59945fb60 0x55c599461f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1ca17fa640 10 client.26795 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:45:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.644+0000 7f1cbdf46640 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c59945fb60 0x55c599461f80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34f69c2c8134a3fd server_cookie=eeb6aabc7a32aad in_seq=0 out_seq=0 2024-04-16T05:45:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1ca17fa640 1 -- 172.21.15.86:0/1390133865 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c59934c970 con 0x55c59945fb60 2024-04-16T05:45:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1ca17fa640 10 client.26795 handle_command_reply: tid=0 2024-04-16T05:45:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 client.26795 shutdown 2024-04-16T05:45:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1390133865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 msgr2=0x55c5993e3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e3520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8056180 tx=0x7f1ca80980f0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1390133865 shutdown_connections 2024-04-16T05:45:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x55c59945fb60 0x55c599461f80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5993e9360 0x55c5993e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c5993e0780 0x55c5993e3520 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 --2- 172.21.15.86:0/1390133865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5993dfe60 0x55c5993e2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1390133865 >> 172.21.15.86:0/1390133865 conn(0x55c5993da590 msgr2=0x55c5993e62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1ca17fa640 0 client.26795 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:45:10.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1390133865 shutdown_connections 2024-04-16T05:45:10.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 1 -- 172.21.15.86:0/1390133865 wait complete. 2024-04-16T05:45:10.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc2d8b740 20 client.26795 trim_cache size 0 max 16384 2024-04-16T05:45:10.652 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:10.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1ca40a3d20 msgr2=0x7f1ca40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006010 tx=0x7f1cb0009260 comp rx=0 tx=0).stop 2024-04-16T05:45:10.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 msgr2=0x7f1cb819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.645+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 0x7f1cb819eb80 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800d950 tx=0x7f1ca800de10 comp rx=0 tx=0).stop 2024-04-16T05:45:10.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.646+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3195426122 shutdown_connections 2024-04-16T05:45:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.646+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cb8108080 0x7f1cb8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.646+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb8100e10 0x7f1cb819eb80 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.646+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.646+0000 7f1cc01aa640 1 --2- 172.21.15.86:0/3195426122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cb8100490 0x7f1cb819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.646+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3195426122 >> 172.21.15.86:0/3195426122 conn(0x7f1cb8077410 msgr2=0x7f1cb80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.646+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3195426122 shutdown_connections 2024-04-16T05:45:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.646+0000 7f1cc01aa640 1 -- 172.21.15.86:0/3195426122 wait complete. 2024-04-16T05:45:10.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:45:10.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:10 smithi194 ceph-mon[35865]: pgmap v305: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:45:10.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:10 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2426247557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:45:10.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:10 smithi196 ceph-mon[36373]: pgmap v305: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:45:10.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:10 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2426247557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:45:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 -- 172.21.15.86:0/3538309243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd444103b90 msgr2=0x7fd44410c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 --2- 172.21.15.86:0/3538309243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd444103b90 0x7fd44410c070 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd438009900 tx=0x7fd43802bd60 comp rx=0 tx=0).stop 2024-04-16T05:45:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 -- 172.21.15.86:0/3538309243 shutdown_connections 2024-04-16T05:45:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 --2- 172.21.15.86:0/3538309243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd44410c5b0 0x7fd44410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 --2- 172.21.15.86:0/3538309243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd444103b90 0x7fd44410c070 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 --2- 172.21.15.86:0/3538309243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444103270 0x7fd444103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 -- 172.21.15.86:0/3538309243 >> 172.21.15.86:0/3538309243 conn(0x7fd4440780b0 msgr2=0x7fd4440784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 -- 172.21.15.86:0/3538309243 shutdown_connections 2024-04-16T05:45:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 -- 172.21.15.86:0/3538309243 wait complete. 2024-04-16T05:45:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 Processor -- start 2024-04-16T05:45:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.767+0000 7fd44c13d640 1 -- start start 2024-04-16T05:45:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44c13d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd444103270 0x7fd44419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44c13d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444103b90 0x7fd44419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44c13d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44410c5b0 0x7fd4441986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44c13d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441123e0 con 0x7fd44410c5b0 2024-04-16T05:45:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44c13d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd444112260 con 0x7fd444103270 2024-04-16T05:45:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44c13d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd444112560 con 0x7fd444103b90 2024-04-16T05:45:10.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44a6da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44410c5b0 0x7fd4441986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44a6da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44410c5b0 0x7fd4441986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42492/0 (socket says 172.21.15.86:42492) 2024-04-16T05:45:10.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44a6da640 1 -- 172.21.15.86:0/2242643870 learned_addr learned my addr 172.21.15.86:0/2242643870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd449ed9640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd444103270 0x7fd44419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44a6da640 1 -- 172.21.15.86:0/2242643870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444103b90 msgr2=0x7fd44419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44a6da640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444103b90 0x7fd44419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44a6da640 1 -- 172.21.15.86:0/2242643870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd444103270 msgr2=0x7fd44419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44a6da640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd444103270 0x7fd44419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44a6da640 1 -- 172.21.15.86:0/2242643870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd444198f80 con 0x7fd44410c5b0 2024-04-16T05:45:10.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44a6da640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44410c5b0 0x7fd4441986c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00d750 tx=0x7fd43c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd442ffd640 1 -- 172.21.15.86:0/2242643870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c017070 con 0x7fd44410c5b0 2024-04-16T05:45:10.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd442ffd640 1 -- 172.21.15.86:0/2242643870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43c005500 con 0x7fd44410c5b0 2024-04-16T05:45:10.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44c13d640 1 -- 172.21.15.86:0/2242643870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd444199270 con 0x7fd44410c5b0 2024-04-16T05:45:10.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd442ffd640 1 -- 172.21.15.86:0/2242643870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c004d90 con 0x7fd44410c5b0 2024-04-16T05:45:10.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.768+0000 7fd44c13d640 1 -- 172.21.15.86:0/2242643870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4441a5580 con 0x7fd44410c5b0 2024-04-16T05:45:10.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.770+0000 7fd442ffd640 1 -- 172.21.15.86:0/2242643870 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd43c004f30 con 0x7fd44410c5b0 2024-04-16T05:45:10.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd44ee04740 1 Processor -- start 2024-04-16T05:45:10.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd44ee04740 1 -- start start 2024-04-16T05:45:10.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1da240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1e2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd44ee04740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55620a13f0e0 con 0x55620a1e3360 2024-04-16T05:45:10.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd44ee04740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55620a0dc520 con 0x55620a1d9e60 2024-04-16T05:45:10.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd44ee04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55620a13b0a0 con 0x55620a1da780 2024-04-16T05:45:10.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd4496d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd442ffd640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd4496d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42502/0 (socket says 172.21.15.86:42502) 2024-04-16T05:45:10.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd4496d8640 1 -- 172.21.15.86:0/396970042 learned_addr learned my addr 172.21.15.86:0/396970042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd449ed9640 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1e2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd44a6da640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.771+0000 7fd4496d8640 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd4496d8640 1 -- 172.21.15.86:0/396970042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 msgr2=0x55620a1e2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd4496d8640 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1e2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd4496d8640 1 -- 172.21.15.86:0/396970042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 msgr2=0x55620a1da240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd4496d8640 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1da240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd442ffd640 1 -- 172.21.15.86:0/2242643870 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fd43c0d0660 con 0x7fd44410c5b0 2024-04-16T05:45:10.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd4496d8640 1 -- 172.21.15.86:0/396970042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620a1048d0 con 0x55620a1e3360 2024-04-16T05:45:10.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd449ed9640 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1e2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:10.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd4496d8640 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e5750 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd4380057c0 tx=0x7fd438005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a90eeacd81a170 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd44a6da640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0922d0 tx=0x7fd43c06ecd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd42bfff640 1 -- 172.21.15.86:0/396970042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4380043c0 con 0x55620a1e3360 2024-04-16T05:45:10.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd42bfff640 1 -- 172.21.15.86:0/396970042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd438046070 con 0x55620a1e3360 2024-04-16T05:45:10.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.772+0000 7fd42bfff640 1 -- 172.21.15.86:0/396970042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438034440 con 0x55620a1e3360 2024-04-16T05:45:10.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 -- 172.21.15.86:0/396970042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 msgr2=0x55620a1e5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e5750 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd4380057c0 tx=0x7fd438005b50 comp rx=0 tx=0).stop 2024-04-16T05:45:10.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 -- 172.21.15.86:0/396970042 shutdown_connections 2024-04-16T05:45:10.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e5750 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1e2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 --2- 172.21.15.86:0/396970042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1da240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 -- 172.21.15.86:0/396970042 >> 172.21.15.86:0/396970042 conn(0x55620a1d4590 msgr2=0x55620a1d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 -- 172.21.15.86:0/396970042 shutdown_connections 2024-04-16T05:45:10.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 -- 172.21.15.86:0/396970042 wait complete. 2024-04-16T05:45:10.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 -- start start 2024-04-16T05:45:10.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.773+0000 7fd44ee04740 1 Processor -- start 2024-04-16T05:45:10.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1e09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd44ee04740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55620a1e6000 con 0x55620a1e3360 2024-04-16T05:45:10.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd44ee04740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55620a0f84d0 con 0x55620a1d9e60 2024-04-16T05:45:10.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd44ee04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55620a1e6180 con 0x55620a1da780 2024-04-16T05:45:10.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd44ee04740 10 client.0 _ll_register_callbacks cb 0x55620a0dd2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4496d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4496d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42518/0 (socket says 172.21.15.86:42518) 2024-04-16T05:45:10.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4496d8640 1 -- 172.21.15.86:0/376942068 learned_addr learned my addr 172.21.15.86:0/376942068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd449ed9640 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1e09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4496d8640 1 -- 172.21.15.86:0/376942068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 msgr2=0x55620a1e09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4496d8640 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1e09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4496d8640 1 -- 172.21.15.86:0/376942068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 msgr2=0x55620a1e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4496d8640 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4496d8640 1 -- 172.21.15.86:0/376942068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556209e513f0 con 0x55620a1e3360 2024-04-16T05:45:10.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4496d8640 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e0f10 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fd43802c2a0 tx=0x7fd4380041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a74fad8e22ee0b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4297fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:10.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4297fa640 1 -- 172.21.15.86:0/376942068 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438003cd0 con 0x55620a1e3360 2024-04-16T05:45:10.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd44ee04740 1 -- 172.21.15.86:0/376942068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd438046070 con 0x55620a1e3360 2024-04-16T05:45:10.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.774+0000 7fd4297fa640 1 -- 172.21.15.86:0/376942068 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fd43802c9b0 con 0x55620a1e3360 2024-04-16T05:45:10.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd4297fa640 1 -- 172.21.15.86:0/376942068 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd43805a050 con 0x55620a1e3360 2024-04-16T05:45:10.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd44ee04740 10 client.18546 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd44ee04740 1 -- 172.21.15.86:0/376942068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556209f1b7d0 con 0x55620a1e3360 2024-04-16T05:45:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd4297fa640 1 -- 172.21.15.86:0/376942068 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd438046070 con 0x55620a1e3360 2024-04-16T05:45:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd4297fa640 1 -- 172.21.15.86:0/376942068 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd438040040 con 0x55620a1e3360 2024-04-16T05:45:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd44ee04740 10 client.18546 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd44ee04740 10 client.18546 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:45:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd44ee04740 20 client.18546 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd44ee04740 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55620a1f3250 0x55620a1f5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd44ee04740 4 client.18546 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd44ee04740 1 -- 172.21.15.86:0/376942068 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55620a14cb00 con 0x55620a1f3250 2024-04-16T05:45:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.775+0000 7fd449ed9640 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55620a1f3250 0x55620a1f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.776+0000 7fd4297fa640 10 client.18546 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:45:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.776+0000 7fd449ed9640 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55620a1f3250 0x55620a1f5640 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a2b2f9aa9d8704d server_cookie=c0394f2ae416f772 in_seq=0 out_seq=0 2024-04-16T05:45:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.776+0000 7fd4297fa640 1 -- 172.21.15.86:0/376942068 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55620a14cb00 con 0x55620a1f3250 2024-04-16T05:45:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.776+0000 7fd4297fa640 10 client.18546 handle_command_reply: tid=0 2024-04-16T05:45:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.776+0000 7fd44ee04740 1 client.18546 shutdown 2024-04-16T05:45:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 -- 172.21.15.86:0/376942068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 msgr2=0x55620a1e0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e0f10 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fd43802c2a0 tx=0x7fd4380041c0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 -- 172.21.15.86:0/376942068 shutdown_connections 2024-04-16T05:45:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55620a1f3250 0x55620a1f5640 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1e0f10 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1e09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 --2- 172.21.15.86:0/376942068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd4297fa640 0 client.18546 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:45:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 -- 172.21.15.86:0/376942068 >> 172.21.15.86:0/376942068 conn(0x55620a1d4590 msgr2=0x55620a1d62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 -- 172.21.15.86:0/376942068 shutdown_connections 2024-04-16T05:45:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 1 -- 172.21.15.86:0/376942068 wait complete. 2024-04-16T05:45:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.777+0000 7fd44ee04740 20 client.18546 trim_cache size 0 max 16384 2024-04-16T05:45:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd44ee04740 1 Processor -- start 2024-04-16T05:45:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd44ee04740 1 -- start start 2024-04-16T05:45:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd44ee04740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441af5f0 con 0x55620a1e3360 2024-04-16T05:45:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd44ee04740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441b1130 con 0x55620a1d9e60 2024-04-16T05:45:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd44ee04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441afcc0 con 0x55620a1da780 2024-04-16T05:45:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd4496d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd4496d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1d2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42522/0 (socket says 172.21.15.86:42522) 2024-04-16T05:45:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd4496d8640 1 -- 172.21.15.86:0/4261459688 learned_addr learned my addr 172.21.15.86:0/4261459688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd4496d8640 1 -- 172.21.15.86:0/4261459688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 msgr2=0x55620a1f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd4496d8640 1 --2- 172.21.15.86:0/4261459688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd4496d8640 1 -- 172.21.15.86:0/4261459688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 msgr2=0x55620a1e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd4496d8640 1 --2- 172.21.15.86:0/4261459688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.790+0000 7fd4496d8640 1 -- 172.21.15.86:0/4261459688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620a1cfc30 con 0x55620a1e3360 2024-04-16T05:45:10.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd4496d8640 1 --2- 172.21.15.86:0/4261459688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1d2ea0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd43802c950 tx=0x7fd438005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e720cb5c979818d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd42b7fe640 1 -- 172.21.15.86:0/4261459688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438005d50 con 0x55620a1e3360 2024-04-16T05:45:10.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd42b7fe640 1 -- 172.21.15.86:0/4261459688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd438046030 con 0x55620a1e3360 2024-04-16T05:45:10.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd42b7fe640 1 -- 172.21.15.86:0/4261459688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43802c3e0 con 0x55620a1e3360 2024-04-16T05:45:10.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd44ee04740 1 -- 172.21.15.86:0/4261459688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 msgr2=0x55620a1d2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd44ee04740 1 --2- 172.21.15.86:0/4261459688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1d2ea0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd43802c950 tx=0x7fd438005b70 comp rx=0 tx=0).stop 2024-04-16T05:45:10.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd44ee04740 1 -- 172.21.15.86:0/4261459688 shutdown_connections 2024-04-16T05:45:10.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd44ee04740 1 --2- 172.21.15.86:0/4261459688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1e3360 0x55620a1d2ea0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd44ee04740 1 --2- 172.21.15.86:0/4261459688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1da780 0x55620a1f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd44ee04740 1 --2- 172.21.15.86:0/4261459688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1d9e60 0x55620a1e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd44ee04740 1 -- 172.21.15.86:0/4261459688 >> 172.21.15.86:0/4261459688 conn(0x55620a1d4590 msgr2=0x55620a1eed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd44ee04740 1 -- 172.21.15.86:0/4261459688 shutdown_connections 2024-04-16T05:45:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.791+0000 7fd44ee04740 1 -- 172.21.15.86:0/4261459688 wait complete. 2024-04-16T05:45:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd44ee04740 1 -- start start 2024-04-16T05:45:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd44ee04740 1 Processor -- start 2024-04-16T05:45:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1d9e60 0x55620a254a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1da780 0x55620a254f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd44ee04740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1e3360 0x55620a1dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd44ee04740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441b0a60 con 0x55620a1da780 2024-04-16T05:45:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd44ee04740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441b0390 con 0x55620a1e3360 2024-04-16T05:45:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd44ee04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441ad5d0 con 0x55620a1d9e60 2024-04-16T05:45:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd44ee04740 10 client.0 _ll_register_callbacks cb 0x7fd43802c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd449ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1da780 0x55620a254f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd449ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1da780 0x55620a254f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42526/0 (socket says 172.21.15.86:42526) 2024-04-16T05:45:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd449ed9640 1 -- 172.21.15.86:0/608056818 learned_addr learned my addr 172.21.15.86:0/608056818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd4496d8640 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1d9e60 0x55620a254a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd449ed9640 1 -- 172.21.15.86:0/608056818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1d9e60 msgr2=0x55620a254a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd449ed9640 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1d9e60 0x55620a254a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd449ed9640 1 -- 172.21.15.86:0/608056818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1e3360 msgr2=0x55620a1dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd4496d8640 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1e3360 0x55620a1dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd449ed9640 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1e3360 0x55620a1dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd449ed9640 1 -- 172.21.15.86:0/608056818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620a1ddab0 con 0x55620a1da780 2024-04-16T05:45:10.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.792+0000 7fd4496d8640 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1e3360 0x55620a1dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd449ed9640 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1da780 0x55620a254f50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x55620a1dfcf0 tx=0x7fd434011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da775a1c39c5416 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd4297fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:10.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd4297fa640 1 -- 172.21.15.86:0/608056818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434004b20 con 0x55620a1da780 2024-04-16T05:45:10.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd4297fa640 1 -- 172.21.15.86:0/608056818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fd434004cc0 con 0x55620a1da780 2024-04-16T05:45:10.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd44ee04740 1 -- 172.21.15.86:0/608056818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd438046030 con 0x55620a1da780 2024-04-16T05:45:10.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd4297fa640 1 -- 172.21.15.86:0/608056818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd434027050 con 0x55620a1da780 2024-04-16T05:45:10.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd44ee04740 10 client.18552 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd44ee04740 1 -- 172.21.15.86:0/608056818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55620a25d340 con 0x55620a1da780 2024-04-16T05:45:10.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd4297fa640 1 -- 172.21.15.86:0/608056818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd438046030 con 0x55620a1da780 2024-04-16T05:45:10.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd4297fa640 1 -- 172.21.15.86:0/608056818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd434005020 con 0x55620a1da780 2024-04-16T05:45:10.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd44ee04740 10 client.18552 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd44ee04740 10 client.18552 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:45:10.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd44ee04740 20 client.18552 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd44ee04740 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55620a25da90 0x55620a25feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd44ee04740 4 client.18552 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:10.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd44ee04740 1 -- 172.21.15.86:0/608056818 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55620a146970 con 0x55620a25da90 2024-04-16T05:45:10.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.793+0000 7fd4496d8640 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55620a25da90 0x55620a25feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd4496d8640 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55620a25da90 0x55620a25feb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d920ea681b7a6945 server_cookie=4135c6345df93084 in_seq=0 out_seq=0 2024-04-16T05:45:10.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd4297fa640 10 client.18552 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:45:10.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd4297fa640 1 -- 172.21.15.86:0/608056818 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55620a146970 con 0x55620a25da90 2024-04-16T05:45:10.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd4297fa640 10 client.18552 handle_command_reply: tid=0 2024-04-16T05:45:10.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 client.18552 shutdown 2024-04-16T05:45:10.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 -- 172.21.15.86:0/608056818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1da780 msgr2=0x55620a254f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1da780 0x55620a254f50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x55620a1dfcf0 tx=0x7fd434011b00 comp rx=0 tx=0).stop 2024-04-16T05:45:10.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 -- 172.21.15.86:0/608056818 shutdown_connections 2024-04-16T05:45:10.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55620a25da90 0x55620a25feb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55620a1e3360 0x55620a1dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd4297fa640 0 client.18552 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:45:10.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55620a1da780 0x55620a254f50 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 --2- 172.21.15.86:0/608056818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620a1d9e60 0x55620a254a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 -- 172.21.15.86:0/608056818 >> 172.21.15.86:0/608056818 conn(0x55620a1d4590 msgr2=0x55620a1dec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 -- 172.21.15.86:0/608056818 shutdown_connections 2024-04-16T05:45:10.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 1 -- 172.21.15.86:0/608056818 wait complete. 2024-04-16T05:45:10.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.794+0000 7fd44ee04740 20 client.18552 trim_cache size 0 max 16384 2024-04-16T05:45:10.807 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:10.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 -- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd42c0a3d20 msgr2=0x7fd42c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0922d0 tx=0x7fd43c06ecd0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 -- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44410c5b0 msgr2=0x7fd4441986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44410c5b0 0x7fd4441986c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00d750 tx=0x7fd43c00dc10 comp rx=0 tx=0).stop 2024-04-16T05:45:10.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 -- 172.21.15.86:0/2242643870 shutdown_connections 2024-04-16T05:45:10.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44410c5b0 0x7fd4441986c0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444103b90 0x7fd44419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 --2- 172.21.15.86:0/2242643870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd444103270 0x7fd44419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 -- 172.21.15.86:0/2242643870 >> 172.21.15.86:0/2242643870 conn(0x7fd4440780b0 msgr2=0x7fd4440fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 -- 172.21.15.86:0/2242643870 shutdown_connections 2024-04-16T05:45:10.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.795+0000 7fd44c13d640 1 -- 172.21.15.86:0/2242643870 wait complete. 2024-04-16T05:45:10.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:45:10.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 -- 172.21.15.86:0/4004112946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434102220 msgr2=0x7f8434102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 --2- 172.21.15.86:0/4004112946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434102220 0x7f8434102600 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f8424009ca0 tx=0x7f842402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 -- 172.21.15.86:0/4004112946 shutdown_connections 2024-04-16T05:45:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 --2- 172.21.15.86:0/4004112946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84341072f0 0x7f843410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 --2- 172.21.15.86:0/4004112946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8434102bd0 0x7f8434106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 --2- 172.21.15.86:0/4004112946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434102220 0x7f8434102600 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 -- 172.21.15.86:0/4004112946 >> 172.21.15.86:0/4004112946 conn(0x7f84340fdac0 msgr2=0x7f84340ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 -- 172.21.15.86:0/4004112946 shutdown_connections 2024-04-16T05:45:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 -- 172.21.15.86:0/4004112946 wait complete. 2024-04-16T05:45:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 Processor -- start 2024-04-16T05:45:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 -- start start 2024-04-16T05:45:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8434102220 0x7f843419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84341072f0 0x7f843419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8434198730 0x7f8434198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84341123b0 con 0x7f84341072f0 2024-04-16T05:45:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8434112230 con 0x7f8434198730 2024-04-16T05:45:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.918+0000 7f843ade2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8434112530 con 0x7f8434102220 2024-04-16T05:45:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8433fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84341072f0 0x7f843419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8433fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84341072f0 0x7f843419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42546/0 (socket says 172.21.15.86:42546) 2024-04-16T05:45:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8433fff640 1 -- 172.21.15.86:0/2152737502 learned_addr learned my addr 172.21.15.86:0/2152737502 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8438b7e640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8434102220 0x7f843419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f843937f640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8434198730 0x7f8434198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8433fff640 1 -- 172.21.15.86:0/2152737502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8434102220 msgr2=0x7f843419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8433fff640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8434102220 0x7f843419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8433fff640 1 -- 172.21.15.86:0/2152737502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8434198730 msgr2=0x7f8434198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8433fff640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8434198730 0x7f8434198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8433fff640 1 -- 172.21.15.86:0/2152737502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8434199330 con 0x7f84341072f0 2024-04-16T05:45:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f843937f640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8434198730 0x7f8434198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8433fff640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84341072f0 0x7f843419ebb0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f841c00c970 tx=0x7f841c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8431ffb640 1 -- 172.21.15.86:0/2152737502 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c007bb0 con 0x7f84341072f0 2024-04-16T05:45:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8431ffb640 1 -- 172.21.15.86:0/2152737502 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c007d50 con 0x7f84341072f0 2024-04-16T05:45:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f843ade2640 1 -- 172.21.15.86:0/2152737502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84341a5410 con 0x7f84341072f0 2024-04-16T05:45:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f8431ffb640 1 -- 172.21.15.86:0/2152737502 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c00f450 con 0x7f84341072f0 2024-04-16T05:45:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.919+0000 7f843ade2640 1 -- 172.21.15.86:0/2152737502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84341a5810 con 0x7f84341072f0 2024-04-16T05:45:10.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.921+0000 7f8431ffb640 1 -- 172.21.15.86:0/2152737502 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f841c004f90 con 0x7f84341072f0 2024-04-16T05:45:10.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f843d9ce740 1 Processor -- start 2024-04-16T05:45:10.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f843d9ce740 1 -- start start 2024-04-16T05:45:10.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f8431ffb640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84200a3d20 0x7f84200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x556980900240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x556980908e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x55698090b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f843d9ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5569808650e0 con 0x556980900780 2024-04-16T05:45:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f843d9ce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556980802520 con 0x5569808ffe60 2024-04-16T05:45:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f843d9ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5569808610a0 con 0x556980909360 2024-04-16T05:45:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f843937f640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84200a3d20 0x7f84200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f8433fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x556980908e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f8431ffb640 1 -- 172.21.15.86:0/2152737502 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f841c0d0a40 con 0x7f84341072f0 2024-04-16T05:45:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f8433fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x556980908e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42548/0 (socket says 172.21.15.86:42548) 2024-04-16T05:45:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.922+0000 7f8433fff640 1 -- 172.21.15.86:0/2733634809 learned_addr learned my addr 172.21.15.86:0/2733634809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f8438b7e640 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x556980900240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f8433fff640 1 -- 172.21.15.86:0/2733634809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 msgr2=0x55698090b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f843937f640 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x55698090b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f8433fff640 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x55698090b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f8433fff640 1 -- 172.21.15.86:0/2733634809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 msgr2=0x556980900240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f8433fff640 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x556980900240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f8433fff640 1 -- 172.21.15.86:0/2733634809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55698082a8d0 con 0x556980900780 2024-04-16T05:45:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f8438b7e640 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x556980900240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f843937f640 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x55698090b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f843937f640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84200a3d20 0x7f84200a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f842c009c00 tx=0x7f842c009680 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f8433fff640 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x556980908e20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f841c098f90 tx=0x7f841c09c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95674c488be78b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f842affd640 1 -- 172.21.15.86:0/2733634809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c01d030 con 0x556980900780 2024-04-16T05:45:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f842affd640 1 -- 172.21.15.86:0/2733634809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c09ccc0 con 0x556980900780 2024-04-16T05:45:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.923+0000 7f842affd640 1 -- 172.21.15.86:0/2733634809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c098c30 con 0x556980900780 2024-04-16T05:45:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 -- 172.21.15.86:0/2733634809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 msgr2=0x556980908e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x556980908e20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f841c098f90 tx=0x7f841c09c1c0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 -- 172.21.15.86:0/2733634809 shutdown_connections 2024-04-16T05:45:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x55698090b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x556980908e20 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 --2- 172.21.15.86:0/2733634809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x556980900240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 -- 172.21.15.86:0/2733634809 >> 172.21.15.86:0/2733634809 conn(0x5569808fa590 msgr2=0x5569808fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 -- 172.21.15.86:0/2733634809 shutdown_connections 2024-04-16T05:45:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 -- 172.21.15.86:0/2733634809 wait complete. 2024-04-16T05:45:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 -- start start 2024-04-16T05:45:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 Processor -- start 2024-04-16T05:45:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x556980906360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x5569809068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x556980906de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55698090c000 con 0x556980900780 2024-04-16T05:45:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55698081e4d0 con 0x5569808ffe60 2024-04-16T05:45:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.924+0000 7f843d9ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55698090c180 con 0x556980909360 2024-04-16T05:45:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f843d9ce740 10 client.0 _ll_register_callbacks cb 0x5569808032c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8433fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x5569809068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8433fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x5569809068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42552/0 (socket says 172.21.15.86:42552) 2024-04-16T05:45:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8433fff640 1 -- 172.21.15.86:0/133036395 learned_addr learned my addr 172.21.15.86:0/133036395 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f843937f640 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x556980906de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8438b7e640 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x556980906360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8433fff640 1 -- 172.21.15.86:0/133036395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 msgr2=0x556980906de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8433fff640 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x556980906de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8433fff640 1 -- 172.21.15.86:0/133036395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 msgr2=0x556980906360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8433fff640 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x556980906360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8433fff640 1 -- 172.21.15.86:0/133036395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569805773f0 con 0x556980900780 2024-04-16T05:45:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8433fff640 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x5569809068a0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f841c007590 tx=0x7f841c0a0a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2aec42a75007de server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8413fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8413fff640 1 -- 172.21.15.86:0/133036395 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c01d040 con 0x556980900780 2024-04-16T05:45:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8413fff640 1 -- 172.21.15.86:0/133036395 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f841c0949a0 con 0x556980900780 2024-04-16T05:45:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f843d9ce740 1 -- 172.21.15.86:0/133036395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f841c09ccc0 con 0x556980900780 2024-04-16T05:45:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f8413fff640 1 -- 172.21.15.86:0/133036395 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f841c0ce1f0 con 0x556980900780 2024-04-16T05:45:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f843d9ce740 10 client.18564 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.925+0000 7f843d9ce740 1 -- 172.21.15.86:0/133036395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569806417d0 con 0x556980900780 2024-04-16T05:45:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.926+0000 7f8413fff640 1 -- 172.21.15.86:0/133036395 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c09ccc0 con 0x556980900780 2024-04-16T05:45:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.926+0000 7f8413fff640 1 -- 172.21.15.86:0/133036395 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f841c09b480 con 0x556980900780 2024-04-16T05:45:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.926+0000 7f843d9ce740 10 client.18564 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.926+0000 7f843d9ce740 10 client.18564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.926+0000 7f843d9ce740 20 client.18564 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.926+0000 7f843d9ce740 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556980919060 0x55698091b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.926+0000 7f843d9ce740 4 client.18564 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.926+0000 7f843d9ce740 1 -- 172.21.15.86:0/133036395 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556980872b00 con 0x556980919060 2024-04-16T05:45:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.926+0000 7f8438b7e640 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556980919060 0x55698091b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f8413fff640 10 client.18564 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f8438b7e640 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556980919060 0x55698091b450 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b5b4e13018c233 server_cookie=a357e6dfb28fcfd8 in_seq=0 out_seq=0 2024-04-16T05:45:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f8413fff640 1 -- 172.21.15.86:0/133036395 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556980872b00 con 0x556980919060 2024-04-16T05:45:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f8413fff640 10 client.18564 handle_command_reply: tid=0 2024-04-16T05:45:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f843d9ce740 1 client.18564 shutdown 2024-04-16T05:45:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f843d9ce740 1 -- 172.21.15.86:0/133036395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 msgr2=0x5569809068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f843d9ce740 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x5569809068a0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f841c007590 tx=0x7f841c0a0a60 comp rx=0 tx=0).stop 2024-04-16T05:45:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f843d9ce740 1 -- 172.21.15.86:0/133036395 shutdown_connections 2024-04-16T05:45:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f843d9ce740 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x556980919060 0x55698091b450 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f843d9ce740 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x556980906de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f8413fff640 0 client.18564 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f843d9ce740 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980900780 0x5569809068a0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f843d9ce740 1 --2- 172.21.15.86:0/133036395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x556980906360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.927+0000 7f843d9ce740 1 -- 172.21.15.86:0/133036395 >> 172.21.15.86:0/133036395 conn(0x5569808fa590 msgr2=0x5569808fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.928+0000 7f843d9ce740 1 -- 172.21.15.86:0/133036395 shutdown_connections 2024-04-16T05:45:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.928+0000 7f843d9ce740 1 -- 172.21.15.86:0/133036395 wait complete. 2024-04-16T05:45:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.928+0000 7f843d9ce740 20 client.18564 trim_cache size 0 max 16384 2024-04-16T05:45:10.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843d9ce740 1 Processor -- start 2024-04-16T05:45:10.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843d9ce740 1 -- start start 2024-04-16T05:45:10.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569808ffe60 0x556980908a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556980900780 0x5569809035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x556980903b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843d9ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84341af7a0 con 0x5569808ffe60 2024-04-16T05:45:10.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843d9ce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84341b12e0 con 0x556980900780 2024-04-16T05:45:10.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843d9ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84341afe70 con 0x556980909360 2024-04-16T05:45:10.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8438b7e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569808ffe60 0x556980908a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8438b7e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569808ffe60 0x556980908a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42562/0 (socket says 172.21.15.86:42562) 2024-04-16T05:45:10.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8438b7e640 1 -- 172.21.15.86:0/659691544 learned_addr learned my addr 172.21.15.86:0/659691544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843937f640 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x556980903b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8433fff640 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556980900780 0x5569809035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8438b7e640 1 -- 172.21.15.86:0/659691544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 msgr2=0x556980903b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8438b7e640 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x556980903b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8438b7e640 1 -- 172.21.15.86:0/659691544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556980900780 msgr2=0x5569809035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8438b7e640 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556980900780 0x5569809035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8438b7e640 1 -- 172.21.15.86:0/659691544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55698090c050 con 0x5569808ffe60 2024-04-16T05:45:10.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f843937f640 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x556980903b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8433fff640 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556980900780 0x5569809035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:10.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.941+0000 7f8438b7e640 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569808ffe60 0x556980908a90 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8424005cc0 tx=0x7f842402c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99974124cc112cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f842a7fc640 1 -- 172.21.15.86:0/659691544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424039040 con 0x5569808ffe60 2024-04-16T05:45:10.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f842a7fc640 1 -- 172.21.15.86:0/659691544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f842403a040 con 0x5569808ffe60 2024-04-16T05:45:10.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f842a7fc640 1 -- 172.21.15.86:0/659691544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84240453c0 con 0x5569808ffe60 2024-04-16T05:45:10.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 -- 172.21.15.86:0/659691544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569808ffe60 msgr2=0x556980908a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569808ffe60 0x556980908a90 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8424005cc0 tx=0x7f842402c810 comp rx=0 tx=0).stop 2024-04-16T05:45:10.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 -- 172.21.15.86:0/659691544 shutdown_connections 2024-04-16T05:45:10.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980909360 0x556980903b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556980900780 0x5569809035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 --2- 172.21.15.86:0/659691544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569808ffe60 0x556980908a90 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 -- 172.21.15.86:0/659691544 >> 172.21.15.86:0/659691544 conn(0x5569808fa590 msgr2=0x5569808f55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 -- 172.21.15.86:0/659691544 shutdown_connections 2024-04-16T05:45:10.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 -- 172.21.15.86:0/659691544 wait complete. 2024-04-16T05:45:10.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.942+0000 7f843d9ce740 1 -- start start 2024-04-16T05:45:10.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843d9ce740 1 Processor -- start 2024-04-16T05:45:10.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x55698091c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980900780 0x55698091c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843d9ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980909360 0x55698091cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843d9ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84341b0c10 con 0x556980909360 2024-04-16T05:45:10.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843d9ce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84341b0540 con 0x5569808ffe60 2024-04-16T05:45:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843d9ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84341ad780 con 0x556980900780 2024-04-16T05:45:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843d9ce740 10 client.0 _ll_register_callbacks cb 0x7f841c09c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843937f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980909360 0x55698091cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843937f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980909360 0x55698091cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42570/0 (socket says 172.21.15.86:42570) 2024-04-16T05:45:10.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843937f640 1 -- 172.21.15.86:0/3931161565 learned_addr learned my addr 172.21.15.86:0/3931161565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:10.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f8438b7e640 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x55698091c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f8433fff640 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980900780 0x55698091c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843937f640 1 -- 172.21.15.86:0/3931161565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980900780 msgr2=0x55698091c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843937f640 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980900780 0x55698091c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843937f640 1 -- 172.21.15.86:0/3931161565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 msgr2=0x55698091c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843937f640 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x55698091c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f843937f640 1 -- 172.21.15.86:0/3931161565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55698090b750 con 0x556980909360 2024-04-16T05:45:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f8433fff640 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980900780 0x55698091c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.943+0000 7f8438b7e640 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x55698091c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843937f640 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980909360 0x55698091cab0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f842c021660 tx=0x7f842c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6c512518d8c7ec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f8413fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f8413fff640 1 -- 172.21.15.86:0/3931161565 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c005020 con 0x556980909360 2024-04-16T05:45:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f8413fff640 1 -- 172.21.15.86:0/3931161565 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f842c0051c0 con 0x556980909360 2024-04-16T05:45:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843d9ce740 1 -- 172.21.15.86:0/3931161565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f842403a040 con 0x556980909360 2024-04-16T05:45:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f8413fff640 1 -- 172.21.15.86:0/3931161565 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f842c032050 con 0x556980909360 2024-04-16T05:45:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843d9ce740 10 client.18570 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843d9ce740 1 -- 172.21.15.86:0/3931161565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569808fa3c0 con 0x556980909360 2024-04-16T05:45:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f8413fff640 1 -- 172.21.15.86:0/3931161565 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f842403a040 con 0x556980909360 2024-04-16T05:45:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f8413fff640 1 -- 172.21.15.86:0/3931161565 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f842c00f040 con 0x556980909360 2024-04-16T05:45:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843d9ce740 10 client.18570 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843d9ce740 10 client.18570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843d9ce740 20 client.18570 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843d9ce740 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55698097fa30 0x556980981e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843d9ce740 4 client.18570 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.944+0000 7f843d9ce740 1 -- 172.21.15.86:0/3931161565 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55698086c970 con 0x55698097fa30 2024-04-16T05:45:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f8438b7e640 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55698097fa30 0x556980981e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f8413fff640 10 client.18570 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f8438b7e640 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55698097fa30 0x556980981e50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8fec7d3f9ea9df1 server_cookie=3f418482cfc01b0d in_seq=0 out_seq=0 2024-04-16T05:45:10.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f8413fff640 1 -- 172.21.15.86:0/3931161565 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55698086c970 con 0x55698097fa30 2024-04-16T05:45:10.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f8413fff640 10 client.18570 handle_command_reply: tid=0 2024-04-16T05:45:10.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 client.18570 shutdown 2024-04-16T05:45:10.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 -- 172.21.15.86:0/3931161565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980909360 msgr2=0x55698091cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980909360 0x55698091cab0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f842c021660 tx=0x7f842c004730 comp rx=0 tx=0).stop 2024-04-16T05:45:10.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 -- 172.21.15.86:0/3931161565 shutdown_connections 2024-04-16T05:45:10.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55698097fa30 0x556980981e50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556980909360 0x55698091cab0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556980900780 0x55698091c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f8413fff640 0 client.18570 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:10.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 --2- 172.21.15.86:0/3931161565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5569808ffe60 0x55698091c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 -- 172.21.15.86:0/3931161565 >> 172.21.15.86:0/3931161565 conn(0x5569808fa590 msgr2=0x556980906300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 -- 172.21.15.86:0/3931161565 shutdown_connections 2024-04-16T05:45:10.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 1 -- 172.21.15.86:0/3931161565 wait complete. 2024-04-16T05:45:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.945+0000 7f843d9ce740 20 client.18570 trim_cache size 0 max 16384 2024-04-16T05:45:10.971 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 -- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84200a3d20 msgr2=0x7f84200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84200a3d20 0x7f84200a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f842c009c00 tx=0x7f842c009680 comp rx=0 tx=0).stop 2024-04-16T05:45:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 -- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84341072f0 msgr2=0x7f843419ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84341072f0 0x7f843419ebb0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f841c00c970 tx=0x7f841c00ce30 comp rx=0 tx=0).stop 2024-04-16T05:45:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 -- 172.21.15.86:0/2152737502 shutdown_connections 2024-04-16T05:45:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84200a3d20 0x7f84200a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8434198730 0x7f8434198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84341072f0 0x7f843419ebb0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 --2- 172.21.15.86:0/2152737502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8434102220 0x7f843419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 -- 172.21.15.86:0/2152737502 >> 172.21.15.86:0/2152737502 conn(0x7f84340fdac0 msgr2=0x7f84340fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 -- 172.21.15.86:0/2152737502 shutdown_connections 2024-04-16T05:45:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:10.946+0000 7f843ade2640 1 -- 172.21.15.86:0/2152737502 wait complete. 2024-04-16T05:45:11.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:11 smithi194 ceph-mon[35865]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T05:45:11.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:11 smithi086 ceph-mon[32028]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T05:45:11.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:11 smithi196 ceph-mon[36373]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T05:45:11.975 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:45:11.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:45:11.976 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:45:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/4230445487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec06b220 msgr2=0x7fe2ec104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/4230445487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec06b220 0x7fe2ec104a40 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009900 tx=0x7fe2e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:12.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/4230445487 shutdown_connections 2024-04-16T05:45:12.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/4230445487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec06b220 0x7fe2ec104a40 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/4230445487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2ec1073d0 0x7fe2ec06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/4230445487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2ec106ff0 0x7fe2ec06a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/4230445487 >> 172.21.15.86:0/4230445487 conn(0x7fe2ec076d00 msgr2=0x7fe2ec077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/4230445487 shutdown_connections 2024-04-16T05:45:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/4230445487 wait complete. 2024-04-16T05:45:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 Processor -- start 2024-04-16T05:45:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 -- start start 2024-04-16T05:45:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec106ff0 0x7fe2ec19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2ec1073d0 0x7fe2ec19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2ec1a3840 0x7fe2ec1a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec118aa0 con 0x7fe2ec106ff0 2024-04-16T05:45:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec118920 con 0x7fe2ec1073d0 2024-04-16T05:45:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.098+0000 7fe2f3b9d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec118c20 con 0x7fe2ec1a3840 2024-04-16T05:45:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1939640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec106ff0 0x7fe2ec19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1939640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec106ff0 0x7fe2ec19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42578/0 (socket says 172.21.15.86:42578) 2024-04-16T05:45:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1939640 1 -- 172.21.15.86:0/2600039983 learned_addr learned my addr 172.21.15.86:0/2600039983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f213a640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2ec1a3840 0x7fe2ec1a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1138640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2ec1073d0 0x7fe2ec19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1939640 1 -- 172.21.15.86:0/2600039983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2ec1a3840 msgr2=0x7fe2ec1a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1939640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2ec1a3840 0x7fe2ec1a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1939640 1 -- 172.21.15.86:0/2600039983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2ec1073d0 msgr2=0x7fe2ec19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1939640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2ec1073d0 0x7fe2ec19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1939640 1 -- 172.21.15.86:0/2600039983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec1a4330 con 0x7fe2ec106ff0 2024-04-16T05:45:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1138640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2ec1073d0 0x7fe2ec19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f213a640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2ec1a3840 0x7fe2ec1a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f1939640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec106ff0 0x7fe2ec19ef80 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00ba50 tx=0x7fe2dc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2eaffd640 1 -- 172.21.15.86:0/2600039983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc019070 con 0x7fe2ec106ff0 2024-04-16T05:45:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2eaffd640 1 -- 172.21.15.86:0/2600039983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc00cbe0 con 0x7fe2ec106ff0 2024-04-16T05:45:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/2600039983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec1a4620 con 0x7fe2ec106ff0 2024-04-16T05:45:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2eaffd640 1 -- 172.21.15.86:0/2600039983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0076e0 con 0x7fe2ec106ff0 2024-04-16T05:45:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.099+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/2600039983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec1abe10 con 0x7fe2ec106ff0 2024-04-16T05:45:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.102+0000 7fe2eaffd640 1 -- 172.21.15.86:0/2600039983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe2dc01c070 con 0x7fe2ec106ff0 2024-04-16T05:45:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.102+0000 7fe2f6789740 1 Processor -- start 2024-04-16T05:45:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f6789740 1 -- start start 2024-04-16T05:45:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b531ab0 0x55e36b531e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b53aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 0x55e36b53d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f6789740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e36b496e90 con 0x55e36b5323d0 2024-04-16T05:45:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f6789740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e36b4344c0 con 0x55e36b531ab0 2024-04-16T05:45:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f6789740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e36b492e50 con 0x55e36b53afb0 2024-04-16T05:45:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1939640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b53aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1939640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b53aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42584/0 (socket says 172.21.15.86:42584) 2024-04-16T05:45:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1939640 1 -- 172.21.15.86:0/530068911 learned_addr learned my addr 172.21.15.86:0/530068911 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2eaffd640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe2d40a3d20 0x7fe2d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f213a640 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b531ab0 0x55e36b531e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1138640 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 0x55e36b53d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1939640 1 -- 172.21.15.86:0/530068911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 msgr2=0x55e36b53d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1939640 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 0x55e36b53d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1939640 1 -- 172.21.15.86:0/530068911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b531ab0 msgr2=0x55e36b531e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1939640 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b531ab0 0x55e36b531e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1939640 1 -- 172.21.15.86:0/530068911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36b2717e0 con 0x55e36b5323d0 2024-04-16T05:45:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.103+0000 7fe2f1138640 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 0x55e36b53d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.104+0000 7fe2f213a640 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b531ab0 0x55e36b531e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.104+0000 7fe2f213a640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe2d40a3d20 0x7fe2d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.104+0000 7fe2eaffd640 1 -- 172.21.15.86:0/2600039983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fe2dc0d0190 con 0x7fe2ec106ff0 2024-04-16T05:45:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.104+0000 7fe2f1939640 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b53aa70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0cd510 tx=0x7fe2dc0981f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f048d22d389d909d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.104+0000 7fe2d3fff640 1 -- 172.21.15.86:0/530068911 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc09bbf0 con 0x55e36b5323d0 2024-04-16T05:45:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.104+0000 7fe2f213a640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe2d40a3d20 0x7fe2d40a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4006010 tx=0x7fe2e4009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.104+0000 7fe2d3fff640 1 -- 172.21.15.86:0/530068911 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc09c520 con 0x55e36b5323d0 2024-04-16T05:45:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.104+0000 7fe2d3fff640 1 -- 172.21.15.86:0/530068911 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0d3d70 con 0x55e36b5323d0 2024-04-16T05:45:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 -- 172.21.15.86:0/530068911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 msgr2=0x55e36b53aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b53aa70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0cd510 tx=0x7fe2dc0981f0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 -- 172.21.15.86:0/530068911 shutdown_connections 2024-04-16T05:45:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 0x55e36b53d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b53aa70 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 --2- 172.21.15.86:0/530068911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b531ab0 0x55e36b531e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 -- 172.21.15.86:0/530068911 >> 172.21.15.86:0/530068911 conn(0x55e36b52c1e0 msgr2=0x55e36b52c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 -- 172.21.15.86:0/530068911 shutdown_connections 2024-04-16T05:45:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 -- 172.21.15.86:0/530068911 wait complete. 2024-04-16T05:45:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 -- start start 2024-04-16T05:45:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.105+0000 7fe2f6789740 1 Processor -- start 2024-04-16T05:45:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b531ab0 0x55e36b537fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b538510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 0x55e36b538a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f6789740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e36b53dd40 con 0x55e36b5323d0 2024-04-16T05:45:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f6789740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e36b41f410 con 0x55e36b53afb0 2024-04-16T05:45:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f6789740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e36b53dec0 con 0x55e36b531ab0 2024-04-16T05:45:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f6789740 10 client.0 _ll_register_callbacks cb 0x55e36b41ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1138640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b538510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1138640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b538510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42594/0 (socket says 172.21.15.86:42594) 2024-04-16T05:45:12.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1138640 1 -- 172.21.15.86:0/3802151475 learned_addr learned my addr 172.21.15.86:0/3802151475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f213a640 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 0x55e36b538a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1939640 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b531ab0 0x55e36b537fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1138640 1 -- 172.21.15.86:0/3802151475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b531ab0 msgr2=0x55e36b537fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1138640 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b531ab0 0x55e36b537fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1138640 1 -- 172.21.15.86:0/3802151475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 msgr2=0x55e36b538a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1138640 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 0x55e36b538a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1138640 1 -- 172.21.15.86:0/3802151475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36b2d1f70 con 0x55e36b5323d0 2024-04-16T05:45:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2f1138640 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b538510 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000d270 tx=0x7fe2e000d730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a792fe46a89e4cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.106+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3802151475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0016070 con 0x55e36b5323d0 2024-04-16T05:45:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3802151475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fe2e0012870 con 0x55e36b5323d0 2024-04-16T05:45:12.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f6789740 1 -- 172.21.15.86:0/3802151475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2dc09c520 con 0x55e36b5323d0 2024-04-16T05:45:12.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3802151475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2e0022050 con 0x55e36b5323d0 2024-04-16T05:45:12.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f6789740 10 client.18582 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:12.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f6789740 1 -- 172.21.15.86:0/3802151475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e36b169170 con 0x55e36b5323d0 2024-04-16T05:45:12.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3802151475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc09c520 con 0x55e36b5323d0 2024-04-16T05:45:12.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3802151475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe2e0011c30 con 0x55e36b5323d0 2024-04-16T05:45:12.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f6789740 10 client.18582 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:12.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f6789740 10 client.18582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f6789740 20 client.18582 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e36b54b2f0 0x55e36b54d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f6789740 4 client.18582 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f6789740 1 -- 172.21.15.86:0/3802151475 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e36b4a48b0 con 0x55e36b54b2f0 2024-04-16T05:45:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.107+0000 7fe2f1939640 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e36b54b2f0 0x55e36b54d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.108+0000 7fe2d17fa640 10 client.18582 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.108+0000 7fe2f1939640 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e36b54b2f0 0x55e36b54d710 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29d05051af85a8f server_cookie=c3f7072c8e1731dd in_seq=0 out_seq=0 2024-04-16T05:45:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3802151475 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e36b4a48b0 con 0x55e36b54b2f0 2024-04-16T05:45:12.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2d17fa640 10 client.18582 handle_command_reply: tid=0 2024-04-16T05:45:12.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 client.18582 shutdown 2024-04-16T05:45:12.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 -- 172.21.15.86:0/3802151475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 msgr2=0x55e36b538510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b538510 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000d270 tx=0x7fe2e000d730 comp rx=0 tx=0).stop 2024-04-16T05:45:12.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 -- 172.21.15.86:0/3802151475 shutdown_connections 2024-04-16T05:45:12.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e36b54b2f0 0x55e36b54d710 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 0x55e36b538a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b5323d0 0x55e36b538510 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2d17fa640 0 client.18582 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3802151475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b531ab0 0x55e36b537fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 -- 172.21.15.86:0/3802151475 >> 172.21.15.86:0/3802151475 conn(0x55e36b52c1e0 msgr2=0x55e36b52df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 -- 172.21.15.86:0/3802151475 shutdown_connections 2024-04-16T05:45:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 1 -- 172.21.15.86:0/3802151475 wait complete. 2024-04-16T05:45:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.109+0000 7fe2f6789740 20 client.18582 trim_cache size 0 max 16384 2024-04-16T05:45:12.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.126+0000 7fe2f6789740 1 Processor -- start 2024-04-16T05:45:12.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.126+0000 7fe2f6789740 1 -- start start 2024-04-16T05:45:12.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b546c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b5323d0 0x55e36b538870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 0x55e36b552fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f6789740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b5d60 con 0x55e36b531ab0 2024-04-16T05:45:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f6789740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b78a0 con 0x55e36b5323d0 2024-04-16T05:45:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f6789740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b6430 con 0x55e36b53afb0 2024-04-16T05:45:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1939640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b546c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1939640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b546c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42596/0 (socket says 172.21.15.86:42596) 2024-04-16T05:45:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1939640 1 -- 172.21.15.86:0/1780228991 learned_addr learned my addr 172.21.15.86:0/1780228991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1138640 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b5323d0 0x55e36b538870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f213a640 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 0x55e36b552fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1939640 1 -- 172.21.15.86:0/1780228991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 msgr2=0x55e36b552fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1939640 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 0x55e36b552fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1939640 1 -- 172.21.15.86:0/1780228991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b5323d0 msgr2=0x55e36b538870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1939640 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b5323d0 0x55e36b538870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1939640 1 -- 172.21.15.86:0/1780228991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36b549ea0 con 0x55e36b531ab0 2024-04-16T05:45:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1138640 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b5323d0 0x55e36b538870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2f1939640 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b546c30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc098280 tx=0x7fe2dc0981f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39e066488f6b01b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2d37fe640 1 -- 172.21.15.86:0/1780228991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc021120 con 0x55e36b531ab0 2024-04-16T05:45:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2d37fe640 1 -- 172.21.15.86:0/1780228991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc09c520 con 0x55e36b531ab0 2024-04-16T05:45:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.127+0000 7fe2d37fe640 1 -- 172.21.15.86:0/1780228991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc09b780 con 0x55e36b531ab0 2024-04-16T05:45:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 -- 172.21.15.86:0/1780228991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 msgr2=0x55e36b546c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b546c30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc098280 tx=0x7fe2dc0981f0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 -- 172.21.15.86:0/1780228991 shutdown_connections 2024-04-16T05:45:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b53afb0 0x55e36b552fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b5323d0 0x55e36b538870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 --2- 172.21.15.86:0/1780228991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b546c30 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 -- 172.21.15.86:0/1780228991 >> 172.21.15.86:0/1780228991 conn(0x55e36b52c1e0 msgr2=0x55e36b52b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 -- 172.21.15.86:0/1780228991 shutdown_connections 2024-04-16T05:45:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 -- 172.21.15.86:0/1780228991 wait complete. 2024-04-16T05:45:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 -- start start 2024-04-16T05:45:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.128+0000 7fe2f6789740 1 Processor -- start 2024-04-16T05:45:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b535090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b5323d0 0x55e36b5355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f6789740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 0x55e36b5b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f6789740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b71d0 con 0x55e36b531ab0 2024-04-16T05:45:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f6789740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b6b00 con 0x55e36b53afb0 2024-04-16T05:45:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f6789740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b3d40 con 0x55e36b5323d0 2024-04-16T05:45:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f6789740 10 client.0 _ll_register_callbacks cb 0x7fe2e000b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1939640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b535090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1939640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b535090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42598/0 (socket says 172.21.15.86:42598) 2024-04-16T05:45:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1939640 1 -- 172.21.15.86:0/3896143645 learned_addr learned my addr 172.21.15.86:0/3896143645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1138640 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b5323d0 0x55e36b5355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f213a640 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 0x55e36b5b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1939640 1 -- 172.21.15.86:0/3896143645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b5323d0 msgr2=0x55e36b5355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1939640 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b5323d0 0x55e36b5355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1939640 1 -- 172.21.15.86:0/3896143645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 msgr2=0x55e36b5b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1939640 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 0x55e36b5b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1939640 1 -- 172.21.15.86:0/3896143645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36b53d3a0 con 0x55e36b531ab0 2024-04-16T05:45:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f1939640 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b535090 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc000f80 tx=0x7fe2dc0a01d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b9bfeb20e90d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:12.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3896143645 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc09b960 con 0x55e36b531ab0 2024-04-16T05:45:12.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3896143645 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fe2dc098490 con 0x55e36b531ab0 2024-04-16T05:45:12.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.129+0000 7fe2f6789740 1 -- 172.21.15.86:0/3896143645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2dc09c520 con 0x55e36b531ab0 2024-04-16T05:45:12.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3896143645 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2dc097b70 con 0x55e36b531ab0 2024-04-16T05:45:12.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2f6789740 10 client.18588 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:12.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2f6789740 1 -- 172.21.15.86:0/3896143645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e36b5b1da0 con 0x55e36b531ab0 2024-04-16T05:45:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3896143645 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc09c520 con 0x55e36b531ab0 2024-04-16T05:45:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3896143645 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe2dc018200 con 0x55e36b531ab0 2024-04-16T05:45:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2f6789740 10 client.18588 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2f6789740 10 client.18588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2f6789740 20 client.18588 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e36b5b2020 0x55e36b5b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2f6789740 4 client.18588 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:45:12.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2f6789740 1 -- 172.21.15.86:0/3896143645 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e36b49e720 con 0x55e36b5b2020 2024-04-16T05:45:12.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.130+0000 7fe2f1138640 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e36b5b2020 0x55e36b5b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.131+0000 7fe2d17fa640 10 client.18588 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:12.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.131+0000 7fe2f1138640 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e36b5b2020 0x55e36b5b4410 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e030094ee2a99449 server_cookie=ddaf54214cbeea11 in_seq=0 out_seq=0 2024-04-16T05:45:12.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2d17fa640 1 -- 172.21.15.86:0/3896143645 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e36b49e720 con 0x55e36b5b2020 2024-04-16T05:45:12.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2d17fa640 10 client.18588 handle_command_reply: tid=0 2024-04-16T05:45:12.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 client.18588 shutdown 2024-04-16T05:45:12.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 -- 172.21.15.86:0/3896143645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 msgr2=0x55e36b535090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b535090 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc000f80 tx=0x7fe2dc0a01d0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 -- 172.21.15.86:0/3896143645 shutdown_connections 2024-04-16T05:45:12.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55e36b5b2020 0x55e36b5b4410 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e36b53afb0 0x55e36b5b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e36b5323d0 0x55e36b5355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2d17fa640 0 client.18588 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 --2- 172.21.15.86:0/3896143645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36b531ab0 0x55e36b535090 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 -- 172.21.15.86:0/3896143645 >> 172.21.15.86:0/3896143645 conn(0x55e36b52c1e0 msgr2=0x55e36b52d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 -- 172.21.15.86:0/3896143645 shutdown_connections 2024-04-16T05:45:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 1 -- 172.21.15.86:0/3896143645 wait complete. 2024-04-16T05:45:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.132+0000 7fe2f6789740 20 client.18588 trim_cache size 0 max 16384 2024-04-16T05:45:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:45:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-16T05:45:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "30eb5871-1089-4078-a59d-48ab867a01fc", 2024-04-16T05:45:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-16T05:45:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:45:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe2d40a3d20 msgr2=0x7fe2d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe2d40a3d20 0x7fe2d40a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4006010 tx=0x7fe2e4009260 comp rx=0 tx=0).stop 2024-04-16T05:45:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec106ff0 msgr2=0x7fe2ec19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec106ff0 0x7fe2ec19ef80 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00ba50 tx=0x7fe2dc00bf10 comp rx=0 tx=0).stop 2024-04-16T05:45:12.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/2600039983 shutdown_connections 2024-04-16T05:45:12.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2ec1a3840 0x7fe2ec1a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe2d40a3d20 0x7fe2d40a61e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2ec1073d0 0x7fe2ec19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 --2- 172.21.15.86:0/2600039983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2ec106ff0 0x7fe2ec19ef80 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/2600039983 >> 172.21.15.86:0/2600039983 conn(0x7fe2ec076d00 msgr2=0x7fe2ec0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/2600039983 shutdown_connections 2024-04-16T05:45:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.133+0000 7fe2f3b9d640 1 -- 172.21.15.86:0/2600039983 wait complete. 2024-04-16T05:45:12.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:45:12.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:45:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.264+0000 7f41c1167640 1 -- 172.21.15.86:0/583163071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 msgr2=0x7f41bc10c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.264+0000 7f41c1167640 1 --2- 172.21.15.86:0/583163071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 0x7f41bc10c080 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009900 tx=0x7f41b002bd60 comp rx=0 tx=0).stop 2024-04-16T05:45:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.264+0000 7f41c1167640 1 -- 172.21.15.86:0/583163071 shutdown_connections 2024-04-16T05:45:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.264+0000 7f41c1167640 1 --2- 172.21.15.86:0/583163071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc10c5c0 0x7f41bc10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.264+0000 7f41c1167640 1 --2- 172.21.15.86:0/583163071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 0x7f41bc10c080 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.264+0000 7f41c1167640 1 --2- 172.21.15.86:0/583163071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41bc103280 0x7f41bc103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.264+0000 7f41c1167640 1 -- 172.21.15.86:0/583163071 >> 172.21.15.86:0/583163071 conn(0x7f41bc078000 msgr2=0x7f41bc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.264+0000 7f41c1167640 1 -- 172.21.15.86:0/583163071 shutdown_connections 2024-04-16T05:45:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.264+0000 7f41c1167640 1 -- 172.21.15.86:0/583163071 wait complete. 2024-04-16T05:45:12.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.265+0000 7f41c1167640 1 Processor -- start 2024-04-16T05:45:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.265+0000 7f41c1167640 1 -- start start 2024-04-16T05:45:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41c1167640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41bc103280 0x7f41bc19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41c1167640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 0x7f41bc19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41c1167640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc10c5c0 0x7f41bc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41c1167640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc112450 con 0x7f41bc103ba0 2024-04-16T05:45:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41c1167640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1122d0 con 0x7f41bc103280 2024-04-16T05:45:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41c1167640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1125d0 con 0x7f41bc10c5c0 2024-04-16T05:45:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41ba59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 0x7f41bc19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41ba59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 0x7f41bc19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42602/0 (socket says 172.21.15.86:42602) 2024-04-16T05:45:12.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41ba59c640 1 -- 172.21.15.86:0/1286657995 learned_addr learned my addr 172.21.15.86:0/1286657995 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41bad9d640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41bc103280 0x7f41bc19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41bb59e640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc10c5c0 0x7f41bc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41ba59c640 1 -- 172.21.15.86:0/1286657995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc10c5c0 msgr2=0x7f41bc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41ba59c640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc10c5c0 0x7f41bc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41ba59c640 1 -- 172.21.15.86:0/1286657995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41bc103280 msgr2=0x7f41bc19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41ba59c640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41bc103280 0x7f41bc19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41ba59c640 1 -- 172.21.15.86:0/1286657995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc198ff0 con 0x7f41bc103ba0 2024-04-16T05:45:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41bb59e640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc10c5c0 0x7f41bc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41ba59c640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 0x7f41bc19ebb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009a20 tx=0x7f41b002dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41a3fff640 1 -- 172.21.15.86:0/1286657995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0043020 con 0x7f41bc103ba0 2024-04-16T05:45:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41c1167640 1 -- 172.21.15.86:0/1286657995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc199280 con 0x7f41bc103ba0 2024-04-16T05:45:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41a3fff640 1 -- 172.21.15.86:0/1286657995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b00054e0 con 0x7f41bc103ba0 2024-04-16T05:45:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.266+0000 7f41a3fff640 1 -- 172.21.15.86:0/1286657995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0003bd0 con 0x7f41bc103ba0 2024-04-16T05:45:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.267+0000 7f41c1167640 1 -- 172.21.15.86:0/1286657995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc1a55f0 con 0x7f41bc103ba0 2024-04-16T05:45:12.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.269+0000 7f41a3fff640 1 -- 172.21.15.86:0/1286657995 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41b0046020 con 0x7f41bc103ba0 2024-04-16T05:45:12.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.269+0000 7f41c3e2f740 1 Processor -- start 2024-04-16T05:45:12.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.269+0000 7f41c3e2f740 1 -- start start 2024-04-16T05:45:12.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e117230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e11fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e120350 0x55989e122740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55989e07c180 con 0x55989e117770 2024-04-16T05:45:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55989e019360 con 0x55989e116e50 2024-04-16T05:45:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55989e078140 con 0x55989e120350 2024-04-16T05:45:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e11fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e11fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42610/0 (socket says 172.21.15.86:42610) 2024-04-16T05:45:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bad9d640 1 -- 172.21.15.86:0/1617161054 learned_addr learned my addr 172.21.15.86:0/1617161054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bb59e640 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e117230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41a3fff640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f41a40a3d20 0x7f41a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41ba59c640 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e120350 0x55989e122740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bad9d640 1 -- 172.21.15.86:0/1617161054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e120350 msgr2=0x55989e122740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bad9d640 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e120350 0x55989e122740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bad9d640 1 -- 172.21.15.86:0/1617161054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 msgr2=0x55989e117230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bad9d640 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e117230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bad9d640 1 -- 172.21.15.86:0/1617161054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989de587d0 con 0x55989e117770 2024-04-16T05:45:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bb59e640 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e117230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bb59e640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f41a40a3d20 0x7f41a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41a3fff640 1 -- 172.21.15.86:0/1286657995 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f41b00f2ca0 con 0x7f41bc103ba0 2024-04-16T05:45:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41bad9d640 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e11fe10 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009cf0 tx=0x7f41ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9ad8c79fb3d38b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.270+0000 7f41a17fa640 1 -- 172.21.15.86:0/1617161054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac004230 con 0x55989e117770 2024-04-16T05:45:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41a17fa640 1 -- 172.21.15.86:0/1617161054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41ac0043d0 con 0x55989e117770 2024-04-16T05:45:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41a17fa640 1 -- 172.21.15.86:0/1617161054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0198c0 con 0x55989e117770 2024-04-16T05:45:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1617161054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 msgr2=0x55989e11fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e11fe10 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009cf0 tx=0x7f41ac011810 comp rx=0 tx=0).stop 2024-04-16T05:45:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41bb59e640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f41a40a3d20 0x7f41a40a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41b400a5a0 tx=0x7f41b400c040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1617161054 shutdown_connections 2024-04-16T05:45:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e120350 0x55989e122740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e11fe10 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1617161054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e117230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1617161054 >> 172.21.15.86:0/1617161054 conn(0x55989e111580 msgr2=0x55989e111960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1617161054 shutdown_connections 2024-04-16T05:45:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.271+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1617161054 wait complete. 2024-04-16T05:45:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41c3e2f740 1 -- start start 2024-04-16T05:45:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41c3e2f740 1 Processor -- start 2024-04-16T05:45:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e11d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e11dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55989e123140 con 0x55989e120350 2024-04-16T05:45:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55989e0352c0 con 0x55989e116e50 2024-04-16T05:45:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55989e1232c0 con 0x55989e117770 2024-04-16T05:45:12.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41c3e2f740 10 client.0 _ll_register_callbacks cb 0x55989e01c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:12.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e11dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e11dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42626/0 (socket says 172.21.15.86:42626) 2024-04-16T05:45:12.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bb59e640 1 -- 172.21.15.86:0/4281069621 learned_addr learned my addr 172.21.15.86:0/4281069621 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bad9d640 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41ba59c640 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e11d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bb59e640 1 -- 172.21.15.86:0/4281069621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 msgr2=0x55989e11d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bb59e640 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e11d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bb59e640 1 -- 172.21.15.86:0/4281069621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 msgr2=0x55989e11d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bb59e640 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bb59e640 1 -- 172.21.15.86:0/4281069621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989dd8e3f0 con 0x55989e120350 2024-04-16T05:45:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bad9d640 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41ba59c640 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e11d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.272+0000 7f41bb59e640 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e11dd50 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f41b4017db0 tx=0x7f41b40096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5db70f3bc74d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f419eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:12.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f419eff5640 1 -- 172.21.15.86:0/4281069621 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4027060 con 0x55989e120350 2024-04-16T05:45:12.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f419eff5640 1 -- 172.21.15.86:0/4281069621 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f41b4022600 con 0x55989e120350 2024-04-16T05:45:12.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41c3e2f740 1 -- 172.21.15.86:0/4281069621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41ac0043d0 con 0x55989e120350 2024-04-16T05:45:12.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f419eff5640 1 -- 172.21.15.86:0/4281069621 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41b402e050 con 0x55989e120350 2024-04-16T05:45:12.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41c3e2f740 10 client.18600 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:12.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41c3e2f740 1 -- 172.21.15.86:0/4281069621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55989dd4e0f0 con 0x55989e120350 2024-04-16T05:45:12.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f419eff5640 1 -- 172.21.15.86:0/4281069621 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41ac0043d0 con 0x55989e120350 2024-04-16T05:45:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f419eff5640 1 -- 172.21.15.86:0/4281069621 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f41b40164d0 con 0x55989e120350 2024-04-16T05:45:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41c3e2f740 10 client.18600 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41c3e2f740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41c3e2f740 20 client.18600 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55989e130670 0x55989e132ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41c3e2f740 4 client.18600 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41c3e2f740 1 -- 172.21.15.86:0/4281069621 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55989e089ba0 con 0x55989e130670 2024-04-16T05:45:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.273+0000 7f41bad9d640 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55989e130670 0x55989e132ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.274+0000 7f419eff5640 10 client.18600 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.274+0000 7f41bad9d640 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55989e130670 0x55989e132ac0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e85e9a31944f732 server_cookie=352f4dc5147903cc in_seq=0 out_seq=0 2024-04-16T05:45:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.274+0000 7f419eff5640 1 -- 172.21.15.86:0/4281069621 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55989e089ba0 con 0x55989e130670 2024-04-16T05:45:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.274+0000 7f419eff5640 10 client.18600 handle_command_reply: tid=0 2024-04-16T05:45:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.274+0000 7f41c3e2f740 1 client.18600 shutdown 2024-04-16T05:45:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 -- 172.21.15.86:0/4281069621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 msgr2=0x55989e11dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e11dd50 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f41b4017db0 tx=0x7f41b40096c0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 -- 172.21.15.86:0/4281069621 shutdown_connections 2024-04-16T05:45:12.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55989e130670 0x55989e132ac0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e11dd50 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f419eff5640 0 client.18600 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:12.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e11d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/4281069621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 -- 172.21.15.86:0/4281069621 >> 172.21.15.86:0/4281069621 conn(0x55989e111580 msgr2=0x55989e1132d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 -- 172.21.15.86:0/4281069621 shutdown_connections 2024-04-16T05:45:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 1 -- 172.21.15.86:0/4281069621 wait complete. 2024-04-16T05:45:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.275+0000 7f41c3e2f740 20 client.18600 trim_cache size 0 max 16384 2024-04-16T05:45:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41c3e2f740 1 Processor -- start 2024-04-16T05:45:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41c3e2f740 1 -- start start 2024-04-16T05:45:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e12c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e120350 0x55989e1105e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1af6e0 con 0x55989e117770 2024-04-16T05:45:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1b1220 con 0x55989e116e50 2024-04-16T05:45:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1afdb0 con 0x55989e120350 2024-04-16T05:45:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41ba59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e12c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41ba59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e12c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42632/0 (socket says 172.21.15.86:42632) 2024-04-16T05:45:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41ba59c640 1 -- 172.21.15.86:0/2106668486 learned_addr learned my addr 172.21.15.86:0/2106668486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41ba59c640 1 -- 172.21.15.86:0/2106668486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e120350 msgr2=0x55989e1105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41bad9d640 1 --2- 172.21.15.86:0/2106668486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41ba59c640 1 --2- 172.21.15.86:0/2106668486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e120350 0x55989e1105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41ba59c640 1 -- 172.21.15.86:0/2106668486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 msgr2=0x55989e11fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41ba59c640 1 --2- 172.21.15.86:0/2106668486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41ba59c640 1 -- 172.21.15.86:0/2106668486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989e1113b0 con 0x55989e117770 2024-04-16T05:45:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41bad9d640 1 --2- 172.21.15.86:0/2106668486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.292+0000 7f41ba59c640 1 --2- 172.21.15.86:0/2106668486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e12c000 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f41b00bad90 tx=0x7f41b00e9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f4d9f27c5078db server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41a0ff9640 1 -- 172.21.15.86:0/2106668486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0004c80 con 0x55989e117770 2024-04-16T05:45:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41a0ff9640 1 -- 172.21.15.86:0/2106668486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b0005690 con 0x55989e117770 2024-04-16T05:45:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41a0ff9640 1 -- 172.21.15.86:0/2106668486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00043c0 con 0x55989e117770 2024-04-16T05:45:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41c3e2f740 1 -- 172.21.15.86:0/2106668486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 msgr2=0x55989e12c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/2106668486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e12c000 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f41b00bad90 tx=0x7f41b00e9a40 comp rx=0 tx=0).stop 2024-04-16T05:45:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41c3e2f740 1 -- 172.21.15.86:0/2106668486 shutdown_connections 2024-04-16T05:45:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/2106668486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e120350 0x55989e1105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/2106668486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e117770 0x55989e12c000 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/2106668486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e11fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41c3e2f740 1 -- 172.21.15.86:0/2106668486 >> 172.21.15.86:0/2106668486 conn(0x55989e111580 msgr2=0x55989e10fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41c3e2f740 1 -- 172.21.15.86:0/2106668486 shutdown_connections 2024-04-16T05:45:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.293+0000 7f41c3e2f740 1 -- 172.21.15.86:0/2106668486 wait complete. 2024-04-16T05:45:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41c3e2f740 1 -- start start 2024-04-16T05:45:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41c3e2f740 1 Processor -- start 2024-04-16T05:45:12.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e128770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e125b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41c3e2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e126040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1b0b50 con 0x55989e120350 2024-04-16T05:45:12.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1b0480 con 0x55989e116e50 2024-04-16T05:45:12.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41c3e2f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1ad6c0 con 0x55989e117770 2024-04-16T05:45:12.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41c3e2f740 10 client.0 _ll_register_callbacks cb 0x7f41b40098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:12.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e126040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e126040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42636/0 (socket says 172.21.15.86:42636) 2024-04-16T05:45:12.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bb59e640 1 -- 172.21.15.86:0/1167540344 learned_addr learned my addr 172.21.15.86:0/1167540344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:12.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bad9d640 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e128770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41ba59c640 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e125b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bb59e640 1 -- 172.21.15.86:0/1167540344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 msgr2=0x55989e125b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bb59e640 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e125b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bb59e640 1 -- 172.21.15.86:0/1167540344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 msgr2=0x55989e128770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bb59e640 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e128770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bb59e640 1 -- 172.21.15.86:0/1167540344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989e1266e0 con 0x55989e120350 2024-04-16T05:45:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41ba59c640 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e125b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bad9d640 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e128770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f41bb59e640 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e126040 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f41b400ae30 tx=0x7f41b4016cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8049e58465d1334a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f419eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.294+0000 7f419eff5640 1 -- 172.21.15.86:0/1167540344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4037060 con 0x55989e120350 2024-04-16T05:45:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f419eff5640 1 -- 172.21.15.86:0/1167540344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7f41b40163b0 con 0x55989e120350 2024-04-16T05:45:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1167540344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55989e083a10 con 0x55989e120350 2024-04-16T05:45:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f419eff5640 1 -- 172.21.15.86:0/1167540344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41b4027020 con 0x55989e120350 2024-04-16T05:45:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41c3e2f740 10 client.18606 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1167540344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55989e196a20 con 0x55989e120350 2024-04-16T05:45:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f419eff5640 1 -- 172.21.15.86:0/1167540344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55989e083a10 con 0x55989e120350 2024-04-16T05:45:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f419eff5640 1 -- 172.21.15.86:0/1167540344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f41b40325b0 con 0x55989e120350 2024-04-16T05:45:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41c3e2f740 10 client.18606 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41c3e2f740 10 client.18606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41c3e2f740 20 client.18606 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:12.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55989e197170 0x55989e199590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:12.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41c3e2f740 4 client.18606 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:45:12.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1167540344 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55989dfff560 con 0x55989e197170 2024-04-16T05:45:12.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.295+0000 7f41bad9d640 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55989e197170 0x55989e199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:12.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f419eff5640 10 client.18606 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:12.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41bad9d640 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55989e197170 0x55989e199590 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c53398b46826a20 server_cookie=a9d9d2338f89d8cd in_seq=0 out_seq=0 2024-04-16T05:45:12.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f419eff5640 1 -- 172.21.15.86:0/1167540344 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55989dfff560 con 0x55989e197170 2024-04-16T05:45:12.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f419eff5640 10 client.18606 handle_command_reply: tid=0 2024-04-16T05:45:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 client.18606 shutdown 2024-04-16T05:45:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1167540344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 msgr2=0x55989e126040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e126040 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f41b400ae30 tx=0x7f41b4016cf0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1167540344 shutdown_connections 2024-04-16T05:45:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x55989e197170 0x55989e199590 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55989e120350 0x55989e126040 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f419eff5640 0 client.18606 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55989e117770 0x55989e125b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 --2- 172.21.15.86:0/1167540344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55989e116e50 0x55989e128770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1167540344 >> 172.21.15.86:0/1167540344 conn(0x55989e111580 msgr2=0x55989e110760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1167540344 shutdown_connections 2024-04-16T05:45:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 1 -- 172.21.15.86:0/1167540344 wait complete. 2024-04-16T05:45:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.296+0000 7f41c3e2f740 20 client.18606 trim_cache size 0 max 16384 2024-04-16T05:45:12.307 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:45:12.308 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:45:12.308 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-16T05:45:12.308 INFO:teuthology.orchestra.run.smithi086.stdout: "30eb5871-1089-4078-a59d-48ab867a01fc": { 2024-04-16T05:45:12.308 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-16T05:45:12.308 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "30eb5871-1089-4078-a59d-48ab867a01fc", 2024-04-16T05:45:12.308 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-16T05:45:12.308 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:12.309 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:12.309 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:45:12.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 -- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f41a40a3d20 msgr2=0x7f41a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f41a40a3d20 0x7f41a40a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41b400a5a0 tx=0x7f41b400c040 comp rx=0 tx=0).stop 2024-04-16T05:45:12.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 -- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 msgr2=0x7f41bc19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:12.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 0x7f41bc19ebb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009a20 tx=0x7f41b002dd30 comp rx=0 tx=0).stop 2024-04-16T05:45:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 -- 172.21.15.86:0/1286657995 shutdown_connections 2024-04-16T05:45:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc10c5c0 0x7f41bc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc103ba0 0x7f41bc19ebb0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f41a40a3d20 0x7f41a40a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 --2- 172.21.15.86:0/1286657995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41bc103280 0x7f41bc19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 -- 172.21.15.86:0/1286657995 >> 172.21.15.86:0/1286657995 conn(0x7f41bc078000 msgr2=0x7f41bc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 -- 172.21.15.86:0/1286657995 shutdown_connections 2024-04-16T05:45:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:12.297+0000 7f41c1167640 1 -- 172.21.15.86:0/1286657995 wait complete. 2024-04-16T05:45:12.312 INFO:tasks.cephfs.filesystem:scrub status for tag:30eb5871-1089-4078-a59d-48ab867a01fc - {'path': '/', 'tag': '30eb5871-1089-4078-a59d-48ab867a01fc', 'options': 'recursive,force'} 2024-04-16T05:45:12.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:12 smithi194 ceph-mon[35865]: pgmap v307: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-16T05:45:12.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:12 smithi086 ceph-mon[32028]: pgmap v307: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-16T05:45:12.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:12 smithi196 ceph-mon[36373]: pgmap v307: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-16T05:45:13.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:13 smithi194 ceph-mon[35865]: scrub queued for path: / 2024-04-16T05:45:13.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:13 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:45:13.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:13 smithi194 ceph-mon[35865]: scrub summary: active paths [/] 2024-04-16T05:45:13.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:13 smithi194 ceph-mon[35865]: scrub summary: idle+waiting paths [/] 2024-04-16T05:45:13.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:13 smithi086 ceph-mon[32028]: scrub queued for path: / 2024-04-16T05:45:13.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:13 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:45:13.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:13 smithi086 ceph-mon[32028]: scrub summary: active paths [/] 2024-04-16T05:45:13.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:13 smithi086 ceph-mon[32028]: scrub summary: idle+waiting paths [/] 2024-04-16T05:45:13.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:13 smithi196 ceph-mon[36373]: scrub queued for path: / 2024-04-16T05:45:13.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:13 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:45:13.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:13 smithi196 ceph-mon[36373]: scrub summary: active paths [/] 2024-04-16T05:45:13.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:13 smithi196 ceph-mon[36373]: scrub summary: idle+waiting paths [/] 2024-04-16T05:45:14.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:14 smithi194 ceph-mon[35865]: pgmap v308: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-16T05:45:14.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:14 smithi086 ceph-mon[32028]: pgmap v308: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-16T05:45:14.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:14 smithi196 ceph-mon[36373]: pgmap v308: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-16T05:45:15.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:15 smithi194 ceph-mon[35865]: scrub summary: idle 2024-04-16T05:45:15.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:15 smithi194 ceph-mon[35865]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T05:45:15.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:15 smithi086 ceph-mon[32028]: scrub summary: idle 2024-04-16T05:45:15.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:15 smithi086 ceph-mon[32028]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T05:45:15.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:15 smithi196 ceph-mon[36373]: scrub summary: idle 2024-04-16T05:45:15.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:15 smithi196 ceph-mon[36373]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T05:45:16.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:16 smithi086 ceph-mon[32028]: pgmap v310: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-16T05:45:16.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:16 smithi196 ceph-mon[36373]: pgmap v310: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-16T05:45:16.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:16 smithi194 ceph-mon[35865]: pgmap v310: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-16T05:45:18.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:18 smithi086 ceph-mon[32028]: pgmap v311: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 127 B/s wr, 8 op/s 2024-04-16T05:45:18.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:18 smithi196 ceph-mon[36373]: pgmap v311: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 127 B/s wr, 8 op/s 2024-04-16T05:45:18.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:18 smithi194 ceph-mon[35865]: pgmap v311: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 127 B/s wr, 8 op/s 2024-04-16T05:45:20.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:20 smithi086 ceph-mon[32028]: pgmap v312: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 234 B/s wr, 13 op/s 2024-04-16T05:45:20.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:20 smithi196 ceph-mon[36373]: pgmap v312: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 234 B/s wr, 13 op/s 2024-04-16T05:45:20.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:20 smithi194 ceph-mon[35865]: pgmap v312: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 234 B/s wr, 13 op/s 2024-04-16T05:45:22.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:22 smithi086 ceph-mon[32028]: pgmap v313: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-16T05:45:22.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:22 smithi196 ceph-mon[36373]: pgmap v313: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-16T05:45:22.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:22 smithi194 ceph-mon[35865]: pgmap v313: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-16T05:45:24.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:24 smithi086 ceph-mon[32028]: pgmap v314: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:45:24.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:24 smithi196 ceph-mon[36373]: pgmap v314: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:45:24.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:24 smithi194 ceph-mon[35865]: pgmap v314: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:45:26.492 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-16T05:45:26.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.610+0000 7f204121b640 1 -- 172.21.15.86:0/781547370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06a830 msgr2=0x7f203c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:26.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.610+0000 7f204121b640 1 --2- 172.21.15.86:0/781547370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06a830 0x7f203c06ac90 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f2024009900 tx=0x7f202402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.610+0000 7f204121b640 1 -- 172.21.15.86:0/781547370 shutdown_connections 2024-04-16T05:45:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.610+0000 7f204121b640 1 --2- 172.21.15.86:0/781547370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f203c06b1d0 0x7f203c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.610+0000 7f204121b640 1 --2- 172.21.15.86:0/781547370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06a830 0x7f203c06ac90 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.610+0000 7f204121b640 1 --2- 172.21.15.86:0/781547370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203c108190 0x7f203c108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.610+0000 7f204121b640 1 -- 172.21.15.86:0/781547370 >> 172.21.15.86:0/781547370 conn(0x7f203c076d00 msgr2=0x7f203c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.610+0000 7f204121b640 1 -- 172.21.15.86:0/781547370 shutdown_connections 2024-04-16T05:45:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.610+0000 7f204121b640 1 -- 172.21.15.86:0/781547370 wait complete. 2024-04-16T05:45:26.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f204121b640 1 Processor -- start 2024-04-16T05:45:26.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f204121b640 1 -- start start 2024-04-16T05:45:26.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f204121b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f203c06a830 0x7f203c101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:26.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f204121b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06b1d0 0x7f203c101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:26.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f204121b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203c108190 0x7f203c101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:26.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f204121b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f203c1123a0 con 0x7f203c06b1d0 2024-04-16T05:45:26.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f204121b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f203c112220 con 0x7f203c06a830 2024-04-16T05:45:26.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f204121b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f203c112520 con 0x7f203c108190 2024-04-16T05:45:26.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f2033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06b1d0 0x7f203c101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:26.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f2033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06b1d0 0x7f203c101860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57460/0 (socket says 172.21.15.86:57460) 2024-04-16T05:45:26.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.611+0000 7f2033fff640 1 -- 172.21.15.86:0/3835188726 learned_addr learned my addr 172.21.15.86:0/3835188726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:26.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f203b59e640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203c108190 0x7f203c101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:26.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f2033fff640 1 -- 172.21.15.86:0/3835188726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203c108190 msgr2=0x7f203c101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:26.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f2033fff640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203c108190 0x7f203c101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f2033fff640 1 -- 172.21.15.86:0/3835188726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f203c06a830 msgr2=0x7f203c101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:26.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f203ad9d640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f203c06a830 0x7f203c101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:26.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f2033fff640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f203c06a830 0x7f203c101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f2033fff640 1 -- 172.21.15.86:0/3835188726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203c1a6cc0 con 0x7f203c06b1d0 2024-04-16T05:45:26.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f203ad9d640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f203c06a830 0x7f203c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:26.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f2033fff640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06b1d0 0x7f203c101860 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f2024005940 tx=0x7f202402cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:26.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f2038d99640 1 -- 172.21.15.86:0/3835188726 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024041070 con 0x7f203c06b1d0 2024-04-16T05:45:26.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203c1a6f50 con 0x7f203c06b1d0 2024-04-16T05:45:26.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f2038d99640 1 -- 172.21.15.86:0/3835188726 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2024004450 con 0x7f203c06b1d0 2024-04-16T05:45:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f2038d99640 1 -- 172.21.15.86:0/3835188726 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024034440 con 0x7f203c06b1d0 2024-04-16T05:45:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.612+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203c1a73b0 con 0x7f203c06b1d0 2024-04-16T05:45:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.613+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203c103710 con 0x7f203c06b1d0 2024-04-16T05:45:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.614+0000 7f2038d99640 1 -- 172.21.15.86:0/3835188726 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2024044070 con 0x7f203c06b1d0 2024-04-16T05:45:26.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.615+0000 7f2038d99640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f20280a3dc0 0x7f20280a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:26.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.615+0000 7f203ad9d640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f20280a3dc0 0x7f20280a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:26.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.616+0000 7f2038d99640 1 -- 172.21.15.86:0/3835188726 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f20240f27b0 con 0x7f203c06b1d0 2024-04-16T05:45:26.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.616+0000 7f203ad9d640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f20280a3dc0 0x7f20280a6280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f202c0058c0 tx=0x7f202c005cb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:26.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.620+0000 7f2038d99640 1 -- 172.21.15.86:0/3835188726 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20240beb90 con 0x7f203c06b1d0 2024-04-16T05:45:26.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:26 smithi086 ceph-mon[32028]: pgmap v315: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:45:26.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:26 smithi196 ceph-mon[36373]: pgmap v315: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:45:26.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.819+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f203c100470 con 0x7f20280a3dc0 2024-04-16T05:45:26.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.827+0000 7f2038d99640 1 -- 172.21.15.86:0/3835188726 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f203c100470 con 0x7f20280a3dc0 2024-04-16T05:45:26.826 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-16T00:00:00", "created": "2024-04-16T05:35:25", "first": "2024-04-16T05:36:00", "last": "2024-04-16T05:45:00", "last_pruned": "2024-04-16T05:45:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-16T05:45:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f20280a3dc0 msgr2=0x7f20280a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f20280a3dc0 0x7f20280a6280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f202c0058c0 tx=0x7f202c005cb0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06b1d0 msgr2=0x7f203c101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06b1d0 0x7f203c101860 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f2024005940 tx=0x7f202402cd10 comp rx=0 tx=0).stop 2024-04-16T05:45:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 shutdown_connections 2024-04-16T05:45:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203c108190 0x7f203c101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f203c06b1d0 0x7f203c101860 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f20280a3dc0 0x7f20280a6280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 --2- 172.21.15.86:0/3835188726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f203c06a830 0x7f203c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 >> 172.21.15.86:0/3835188726 conn(0x7f203c076d00 msgr2=0x7f203c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:26.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 shutdown_connections 2024-04-16T05:45:26.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.829+0000 7f204121b640 1 -- 172.21.15.86:0/3835188726 wait complete. 2024-04-16T05:45:26.838 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-16T05:45:26.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:26 smithi194 ceph-mon[35865]: pgmap v315: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:45:26.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.945+0000 7fba72b1b640 1 -- 172.21.15.86:0/3917653895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c106220 msgr2=0x7fba6c10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.945+0000 7fba72b1b640 1 --2- 172.21.15.86:0/3917653895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c106220 0x7fba6c10ddf0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fba64009ca0 tx=0x7fba6402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.945+0000 7fba72b1b640 1 -- 172.21.15.86:0/3917653895 shutdown_connections 2024-04-16T05:45:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.945+0000 7fba72b1b640 1 --2- 172.21.15.86:0/3917653895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c106220 0x7fba6c10ddf0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.945+0000 7fba72b1b640 1 --2- 172.21.15.86:0/3917653895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba6c101b50 0x7fba6c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.945+0000 7fba72b1b640 1 --2- 172.21.15.86:0/3917653895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6c1011a0 0x7fba6c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.945+0000 7fba72b1b640 1 -- 172.21.15.86:0/3917653895 >> 172.21.15.86:0/3917653895 conn(0x7fba6c078080 msgr2=0x7fba6c078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.945+0000 7fba72b1b640 1 -- 172.21.15.86:0/3917653895 shutdown_connections 2024-04-16T05:45:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.945+0000 7fba72b1b640 1 -- 172.21.15.86:0/3917653895 wait complete. 2024-04-16T05:45:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba72b1b640 1 Processor -- start 2024-04-16T05:45:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba72b1b640 1 -- start start 2024-04-16T05:45:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba72b1b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba6c1011a0 0x7fba6c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba72b1b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6c106220 0x7fba6c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba72b1b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c19d000 0x7fba6c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba72b1b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c1123f0 con 0x7fba6c19d000 2024-04-16T05:45:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba72b1b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c112270 con 0x7fba6c1011a0 2024-04-16T05:45:26.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba72b1b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c112570 con 0x7fba6c106220 2024-04-16T05:45:26.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba710b8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c19d000 0x7fba6c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:26.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba710b8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c19d000 0x7fba6c19d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57470/0 (socket says 172.21.15.86:57470) 2024-04-16T05:45:26.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba710b8640 1 -- 172.21.15.86:0/672776369 learned_addr learned my addr 172.21.15.86:0/672776369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:26.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba708b7640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba6c1011a0 0x7fba6c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:26.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba710b8640 1 -- 172.21.15.86:0/672776369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6c106220 msgr2=0x7fba6c198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:26.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba710b8640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6c106220 0x7fba6c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba710b8640 1 -- 172.21.15.86:0/672776369 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba6c1011a0 msgr2=0x7fba6c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba710b8640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba6c1011a0 0x7fba6c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba710b8640 1 -- 172.21.15.86:0/672776369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6c19daf0 con 0x7fba6c19d000 2024-04-16T05:45:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba708b7640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba6c1011a0 0x7fba6c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba710b8640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c19d000 0x7fba6c19d4b0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fba6402c700 tx=0x7fba64032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.946+0000 7fba69ffb640 1 -- 172.21.15.86:0/672776369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64039040 con 0x7fba6c19d000 2024-04-16T05:45:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.947+0000 7fba69ffb640 1 -- 172.21.15.86:0/672776369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba64044070 con 0x7fba6c19d000 2024-04-16T05:45:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.947+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba6c19dd80 con 0x7fba6c19d000 2024-04-16T05:45:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.947+0000 7fba69ffb640 1 -- 172.21.15.86:0/672776369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba640042a0 con 0x7fba6c19d000 2024-04-16T05:45:26.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.947+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba6c1a5540 con 0x7fba6c19d000 2024-04-16T05:45:26.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.948+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba6c04fd90 con 0x7fba6c19d000 2024-04-16T05:45:26.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.949+0000 7fba69ffb640 1 -- 172.21.15.86:0/672776369 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba64004440 con 0x7fba6c19d000 2024-04-16T05:45:26.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.949+0000 7fba69ffb640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba5c0a3dc0 0x7fba5c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:26.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.949+0000 7fba708b7640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba5c0a3dc0 0x7fba5c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:26.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.950+0000 7fba69ffb640 1 -- 172.21.15.86:0/672776369 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fba640f2f30 con 0x7fba6c19d000 2024-04-16T05:45:26.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.950+0000 7fba708b7640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba5c0a3dc0 0x7fba5c0a6280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fba600058c0 tx=0x7fba60005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:26.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:26.954+0000 7fba69ffb640 1 -- 172.21.15.86:0/672776369 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba640333a0 con 0x7fba6c19d000 2024-04-16T05:45:27.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.155+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 --> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fba6c101d70 con 0x7fba5c0a3dc0 2024-04-16T05:45:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.162+0000 7fba69ffb640 1 -- 172.21.15.86:0/672776369 <== mgr.15528 v2:172.21.15.86:6800/2117030010 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fba6c101d70 con 0x7fba5c0a3dc0 2024-04-16T05:45:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-04-16T05:45:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba5c0a3dc0 msgr2=0x7fba5c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba5c0a3dc0 0x7fba5c0a6280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fba600058c0 tx=0x7fba60005850 comp rx=0 tx=0).stop 2024-04-16T05:45:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c19d000 msgr2=0x7fba6c19d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c19d000 0x7fba6c19d4b0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fba6402c700 tx=0x7fba64032f20 comp rx=0 tx=0).stop 2024-04-16T05:45:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 shutdown_connections 2024-04-16T05:45:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba6c19d000 0x7fba6c19d4b0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6c106220 0x7fba6c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fba5c0a3dc0 0x7fba5c0a6280 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 --2- 172.21.15.86:0/672776369 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba6c1011a0 0x7fba6c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 >> 172.21.15.86:0/672776369 conn(0x7fba6c078080 msgr2=0x7fba6c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 shutdown_connections 2024-04-16T05:45:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:27.164+0000 7fba72b1b640 1 -- 172.21.15.86:0/672776369 wait complete. 2024-04-16T05:45:27.173 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-16T05:45:27.184 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-16T05:45:27.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:27 smithi086 ceph-mon[32028]: from='client.18612 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:27.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:27 smithi196 ceph-mon[36373]: from='client.18612 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:27.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:27 smithi194 ceph-mon[35865]: from='client.18612 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:28.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:28 smithi086 ceph-mon[32028]: pgmap v316: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:45:28.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:28 smithi086 ceph-mon[32028]: from='client.18618 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:28.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:28 smithi196 ceph-mon[36373]: pgmap v316: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:45:28.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:28 smithi196 ceph-mon[36373]: from='client.18618 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:28.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:28 smithi194 ceph-mon[35865]: pgmap v316: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:45:28.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:28 smithi194 ceph-mon[35865]: from='client.18618 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:30.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:30 smithi086 ceph-mon[32028]: pgmap v317: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-04-16T05:45:30.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:30 smithi196 ceph-mon[36373]: pgmap v317: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-04-16T05:45:30.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:30 smithi194 ceph-mon[35865]: pgmap v317: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-04-16T05:45:31.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:31 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:45:31.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:31 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:45:31.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:31 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:45:32.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:32 smithi086 ceph-mon[32028]: pgmap v318: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-16T05:45:32.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:32 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:32 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:32 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:32 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:32 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:32 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:32 smithi196 ceph-mon[36373]: pgmap v318: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-16T05:45:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:32 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:32 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:32 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:32 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:32 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:32 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:32 smithi194 ceph-mon[35865]: pgmap v318: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-16T05:45:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:32 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:32 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:32 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:32 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:32 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:32.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:32 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:33.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-04-16T05:45:33.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: Unable to set osd_memory_target on smithi196 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-16T05:45:33.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-04-16T05:45:33.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-16T05:45:33.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:45:33.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-16T05:45:33.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:45:33.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:45:33.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:33 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:33.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-04-16T05:45:33.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: Unable to set osd_memory_target on smithi196 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-16T05:45:33.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-04-16T05:45:33.736 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-16T05:45:33.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:45:33.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-16T05:45:33.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:45:33.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:45:33.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:33 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:33.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-04-16T05:45:33.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: Unable to set osd_memory_target on smithi196 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-16T05:45:33.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.907 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.908 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:33.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-04-16T05:45:33.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-16T05:45:33.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:45:33.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-16T05:45:33.909 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:45:33.910 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:45:33.910 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:33 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' 2024-04-16T05:45:35.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:34 smithi194 ceph-mon[35865]: pgmap v319: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-16T05:45:35.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:34 smithi086 ceph-mon[32028]: pgmap v319: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-16T05:45:35.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:34 smithi196 ceph-mon[36373]: pgmap v319: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-16T05:45:37.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:36 smithi194 ceph-mon[35865]: pgmap v320: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-16T05:45:37.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:36 smithi086 ceph-mon[32028]: pgmap v320: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-16T05:45:37.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:36 smithi196 ceph-mon[36373]: pgmap v320: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-16T05:45:39.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:38 smithi194 ceph-mon[35865]: pgmap v321: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-16T05:45:39.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:38 smithi086 ceph-mon[32028]: pgmap v321: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-16T05:45:39.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:38 smithi196 ceph-mon[36373]: pgmap v321: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-16T05:45:41.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:40 smithi194 ceph-mon[35865]: pgmap v322: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-16T05:45:41.217 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:40 smithi086 ceph-mon[32028]: pgmap v322: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-16T05:45:41.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:40 smithi196 ceph-mon[36373]: pgmap v322: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-16T05:45:42.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:45:42.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:45:42.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.434+0000 7fdccf84a640 1 -- 172.21.15.86:0/2975031765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc806a620 msgr2=0x7fdcc806aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.434+0000 7fdccf84a640 1 --2- 172.21.15.86:0/2975031765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc806a620 0x7fdcc806aaa0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009930 tx=0x7fdcbc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 -- 172.21.15.86:0/2975031765 shutdown_connections 2024-04-16T05:45:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 --2- 172.21.15.86:0/2975031765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc806afe0 0x7fdcc810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 --2- 172.21.15.86:0/2975031765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc806a620 0x7fdcc806aaa0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 --2- 172.21.15.86:0/2975031765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcc8107040 0x7fdcc8107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 -- 172.21.15.86:0/2975031765 >> 172.21.15.86:0/2975031765 conn(0x7fdcc8077ee0 msgr2=0x7fdcc80782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 -- 172.21.15.86:0/2975031765 shutdown_connections 2024-04-16T05:45:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 -- 172.21.15.86:0/2975031765 wait complete. 2024-04-16T05:45:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 Processor -- start 2024-04-16T05:45:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 -- start start 2024-04-16T05:45:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc806a620 0x7fdcc81012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcc806afe0 0x7fdcc81017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc8107040 0x7fdcc8101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8112410 con 0x7fdcc8107040 2024-04-16T05:45:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8112290 con 0x7fdcc806afe0 2024-04-16T05:45:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.435+0000 7fdccf84a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8112590 con 0x7fdcc806a620 2024-04-16T05:45:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccdde7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc8107040 0x7fdcc8101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccdde7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc8107040 0x7fdcc8101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48428/0 (socket says 172.21.15.86:48428) 2024-04-16T05:45:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccdde7640 1 -- 172.21.15.86:0/297371055 learned_addr learned my addr 172.21.15.86:0/297371055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc806a620 0x7fdcc81012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdcccde5640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcc806afe0 0x7fdcc81017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccdde7640 1 -- 172.21.15.86:0/297371055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc806a620 msgr2=0x7fdcc81012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccdde7640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc806a620 0x7fdcc81012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccdde7640 1 -- 172.21.15.86:0/297371055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcc806afe0 msgr2=0x7fdcc81017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccdde7640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcc806afe0 0x7fdcc81017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccdde7640 1 -- 172.21.15.86:0/297371055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc81a6db0 con 0x7fdcc8107040 2024-04-16T05:45:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdcccde5640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcc806afe0 0x7fdcc81017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccdde7640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc8107040 0x7fdcc8101d30 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000cad0 tx=0x7fdcc000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdcba7fc640 1 -- 172.21.15.86:0/297371055 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000ae20 con 0x7fdcc8107040 2024-04-16T05:45:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdcba7fc640 1 -- 172.21.15.86:0/297371055 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc0005020 con 0x7fdcc8107040 2024-04-16T05:45:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccf84a640 1 -- 172.21.15.86:0/297371055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc81a70a0 con 0x7fdcc8107040 2024-04-16T05:45:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdcba7fc640 1 -- 172.21.15.86:0/297371055 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000f6f0 con 0x7fdcc8107040 2024-04-16T05:45:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.436+0000 7fdccf84a640 1 -- 172.21.15.86:0/297371055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc81a7640 con 0x7fdcc8107040 2024-04-16T05:45:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.439+0000 7fdcba7fc640 1 -- 172.21.15.86:0/297371055 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdcc00051d0 con 0x7fdcc8107040 2024-04-16T05:45:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.439+0000 7fdcd2438740 1 Processor -- start 2024-04-16T05:45:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.439+0000 7fdcd2438740 1 -- start start 2024-04-16T05:45:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.439+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 0x560b5b167230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.439+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b167770 0x560b5b16fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b170350 0x560b5b172740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcd2438740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b5b0cc180 con 0x560b5b167770 2024-04-16T05:45:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcd2438740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560b5b069360 con 0x560b5b170350 2024-04-16T05:45:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcd2438740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b5b0c8140 con 0x560b5b166e50 2024-04-16T05:45:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccd5e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b167770 0x560b5b16fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccd5e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b167770 0x560b5b16fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48430/0 (socket says 172.21.15.86:48430) 2024-04-16T05:45:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccd5e6640 1 -- 172.21.15.86:0/2333941783 learned_addr learned my addr 172.21.15.86:0/2333941783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcba7fc640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdcb40a3da0 0x7fdcb40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcccde5640 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 0x560b5b167230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccdde7640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdcb40a3da0 0x7fdcb40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcccde5640 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b170350 0x560b5b172740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccd5e6640 1 -- 172.21.15.86:0/2333941783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 msgr2=0x560b5b167230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 0x560b5b167230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccd5e6640 1 -- 172.21.15.86:0/2333941783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b170350 msgr2=0x560b5b172740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b170350 0x560b5b172740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccd5e6640 1 -- 172.21.15.86:0/2333941783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5aea87d0 con 0x560b5b167770 2024-04-16T05:45:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcccde5640 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b170350 0x560b5b172740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcba7fc640 1 -- 172.21.15.86:0/297371055 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fdcc00d0f10 con 0x7fdcc8107040 2024-04-16T05:45:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b167770 0x560b5b16fe10 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400bc90 tx=0x7fdcc40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7204fe9b87c4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdccdde7640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdcb40a3da0 0x7fdcb40a6260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0098ee0 tx=0x7fdcc006f570 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcb37fe640 1 -- 172.21.15.86:0/2333941783 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc401c480 con 0x560b5b167770 2024-04-16T05:45:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.440+0000 7fdcb37fe640 1 -- 172.21.15.86:0/2333941783 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc401cb00 con 0x560b5b167770 2024-04-16T05:45:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcb37fe640 1 -- 172.21.15.86:0/2333941783 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4011bb0 con 0x560b5b167770 2024-04-16T05:45:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcd2438740 1 -- 172.21.15.86:0/2333941783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b167770 msgr2=0x560b5b16fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcd2438740 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b167770 0x560b5b16fe10 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400bc90 tx=0x7fdcc40107a0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcd2438740 1 -- 172.21.15.86:0/2333941783 shutdown_connections 2024-04-16T05:45:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcd2438740 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b170350 0x560b5b172740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcd2438740 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b167770 0x560b5b16fe10 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcd2438740 1 --2- 172.21.15.86:0/2333941783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 0x560b5b167230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcd2438740 1 -- 172.21.15.86:0/2333941783 >> 172.21.15.86:0/2333941783 conn(0x560b5b161580 msgr2=0x560b5b161960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcd2438740 1 -- 172.21.15.86:0/2333941783 shutdown_connections 2024-04-16T05:45:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.441+0000 7fdcd2438740 1 -- 172.21.15.86:0/2333941783 wait complete. 2024-04-16T05:45:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcd2438740 1 -- start start 2024-04-16T05:45:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcd2438740 1 Processor -- start 2024-04-16T05:45:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 0x560b5b16d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b167770 0x560b5b16d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcd2438740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b5b173140 con 0x560b5b170350 2024-04-16T05:45:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcd2438740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560b5b0852c0 con 0x560b5b167770 2024-04-16T05:45:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcd2438740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b5b1732c0 con 0x560b5b166e50 2024-04-16T05:45:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcd2438740 10 client.0 _ll_register_callbacks cb 0x560b5b06c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcccde5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcccde5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48446/0 (socket says 172.21.15.86:48446) 2024-04-16T05:45:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcccde5640 1 -- 172.21.15.86:0/3567720580 learned_addr learned my addr 172.21.15.86:0/3567720580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcccde5640 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 0x560b5b16d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcccde5640 1 -- 172.21.15.86:0/3567720580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 msgr2=0x560b5b16d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcccde5640 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 0x560b5b16d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdcccde5640 1 -- 172.21.15.86:0/3567720580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b167770 msgr2=0x560b5b16d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.442+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b167770 0x560b5b16d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcccde5640 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b167770 0x560b5b16d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcccde5640 1 -- 172.21.15.86:0/3567720580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5adde3f0 con 0x560b5b170350 2024-04-16T05:45:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b167770 0x560b5b16d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcccde5640 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16de50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009930 tx=0x7fdcbc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14f2fde2666296b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/3567720580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc004490 con 0x560b5b170350 2024-04-16T05:45:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcd2438740 1 -- 172.21.15.86:0/3567720580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc401cb00 con 0x560b5b170350 2024-04-16T05:45:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/3567720580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fdcbc03a040 con 0x560b5b170350 2024-04-16T05:45:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/3567720580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc053050 con 0x560b5b170350 2024-04-16T05:45:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcd2438740 10 client.18630 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcd2438740 1 -- 172.21.15.86:0/3567720580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b5ad9e0f0 con 0x560b5b170350 2024-04-16T05:45:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/3567720580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc401cb00 con 0x560b5b170350 2024-04-16T05:45:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/3567720580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdcbc042720 con 0x560b5b170350 2024-04-16T05:45:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcd2438740 10 client.18630 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcd2438740 10 client.18630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcd2438740 20 client.18630 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcd2438740 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560b5b180770 0x560b5b182bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcd2438740 4 client.18630 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.443+0000 7fdcd2438740 1 -- 172.21.15.86:0/3567720580 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b5b0d9ba0 con 0x560b5b180770 2024-04-16T05:45:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.444+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560b5b180770 0x560b5b182bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.444+0000 7fdcb0ff9640 10 client.18630 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.444+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560b5b180770 0x560b5b182bd0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36bdf9d609c3754f server_cookie=28e4137c07e6c316 in_seq=0 out_seq=0 2024-04-16T05:45:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/3567720580 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b5b0d9ba0 con 0x560b5b180770 2024-04-16T05:45:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcb0ff9640 10 client.18630 handle_command_reply: tid=0 2024-04-16T05:45:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 client.18630 shutdown 2024-04-16T05:45:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 -- 172.21.15.86:0/3567720580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 msgr2=0x560b5b16de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16de50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009930 tx=0x7fdcbc009260 comp rx=0 tx=0).stop 2024-04-16T05:45:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 -- 172.21.15.86:0/3567720580 shutdown_connections 2024-04-16T05:45:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560b5b180770 0x560b5b182bd0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16de50 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b167770 0x560b5b16d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcb0ff9640 0 client.18630 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 --2- 172.21.15.86:0/3567720580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b166e50 0x560b5b16d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 -- 172.21.15.86:0/3567720580 >> 172.21.15.86:0/3567720580 conn(0x560b5b161580 msgr2=0x560b5b1632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 -- 172.21.15.86:0/3567720580 shutdown_connections 2024-04-16T05:45:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 1 -- 172.21.15.86:0/3567720580 wait complete. 2024-04-16T05:45:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.445+0000 7fdcd2438740 20 client.18630 trim_cache size 0 max 16384 2024-04-16T05:45:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcd2438740 1 Processor -- start 2024-04-16T05:45:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcd2438740 1 -- start start 2024-04-16T05:45:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b166e50 0x560b5b16fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 0x560b5b17c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcd2438740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81b1870 con 0x560b5b170350 2024-04-16T05:45:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcd2438740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81b33b0 con 0x560b5b166e50 2024-04-16T05:45:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcd2438740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81b1f40 con 0x560b5b167770 2024-04-16T05:45:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcccde5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcccde5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48448/0 (socket says 172.21.15.86:48448) 2024-04-16T05:45:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcccde5640 1 -- 172.21.15.86:0/2332027351 learned_addr learned my addr 172.21.15.86:0/2332027351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/2332027351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 0x560b5b17c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcccde5640 1 -- 172.21.15.86:0/2332027351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 msgr2=0x560b5b17c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcccde5640 1 --2- 172.21.15.86:0/2332027351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 0x560b5b17c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcccde5640 1 -- 172.21.15.86:0/2332027351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b166e50 msgr2=0x560b5b16fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcccde5640 1 --2- 172.21.15.86:0/2332027351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b166e50 0x560b5b16fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcccde5640 1 -- 172.21.15.86:0/2332027351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5b1660e0 con 0x560b5b170350 2024-04-16T05:45:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/2332027351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 0x560b5b17c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcccde5640 1 --2- 172.21.15.86:0/2332027351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16c530 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005ec0 tx=0x7fdcbc02c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674f88040f990f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.463+0000 7fdcb2ffd640 1 -- 172.21.15.86:0/2332027351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc004490 con 0x560b5b170350 2024-04-16T05:45:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcb2ffd640 1 -- 172.21.15.86:0/2332027351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc048020 con 0x560b5b170350 2024-04-16T05:45:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcb2ffd640 1 -- 172.21.15.86:0/2332027351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc003c00 con 0x560b5b170350 2024-04-16T05:45:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcd2438740 1 -- 172.21.15.86:0/2332027351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 msgr2=0x560b5b16c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcd2438740 1 --2- 172.21.15.86:0/2332027351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16c530 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005ec0 tx=0x7fdcbc02c990 comp rx=0 tx=0).stop 2024-04-16T05:45:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcd2438740 1 -- 172.21.15.86:0/2332027351 shutdown_connections 2024-04-16T05:45:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcd2438740 1 --2- 172.21.15.86:0/2332027351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b16c530 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcd2438740 1 --2- 172.21.15.86:0/2332027351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 0x560b5b17c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcd2438740 1 --2- 172.21.15.86:0/2332027351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b166e50 0x560b5b16fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcd2438740 1 -- 172.21.15.86:0/2332027351 >> 172.21.15.86:0/2332027351 conn(0x560b5b161580 msgr2=0x560b5b15fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcd2438740 1 -- 172.21.15.86:0/2332027351 shutdown_connections 2024-04-16T05:45:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.464+0000 7fdcd2438740 1 -- 172.21.15.86:0/2332027351 wait complete. 2024-04-16T05:45:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcd2438740 1 -- start start 2024-04-16T05:45:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcd2438740 1 Processor -- start 2024-04-16T05:45:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b166e50 0x560b5b178470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 0x560b5b1789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcd2438740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b1759c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcd2438740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81b2ce0 con 0x560b5b170350 2024-04-16T05:45:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcd2438740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81b2610 con 0x560b5b166e50 2024-04-16T05:45:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcd2438740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81af870 con 0x560b5b167770 2024-04-16T05:45:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcd2438740 10 client.0 _ll_register_callbacks cb 0x7fdcbc02cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcccde5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b1759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcccde5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b1759c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48456/0 (socket says 172.21.15.86:48456) 2024-04-16T05:45:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcccde5640 1 -- 172.21.15.86:0/1258221436 learned_addr learned my addr 172.21.15.86:0/1258221436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcccde5640 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b166e50 0x560b5b178470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcccde5640 1 -- 172.21.15.86:0/1258221436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 msgr2=0x560b5b1789b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcccde5640 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 0x560b5b1789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcccde5640 1 -- 172.21.15.86:0/1258221436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b166e50 msgr2=0x560b5b178470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcccde5640 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b166e50 0x560b5b178470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.465+0000 7fdcccde5640 1 -- 172.21.15.86:0/1258221436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5b1761c0 con 0x560b5b170350 2024-04-16T05:45:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcccde5640 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b1759c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc050040 tx=0x7fdcbc0037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbdbc23861f3df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/1258221436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0035f0 con 0x560b5b170350 2024-04-16T05:45:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/1258221436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fdcbc003880 con 0x560b5b170350 2024-04-16T05:45:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcd2438740 1 -- 172.21.15.86:0/1258221436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc048020 con 0x560b5b170350 2024-04-16T05:45:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/1258221436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc003a80 con 0x560b5b170350 2024-04-16T05:45:42.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcd2438740 10 client.18636 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:42.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcd2438740 1 -- 172.21.15.86:0/1258221436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b5b1e6860 con 0x560b5b170350 2024-04-16T05:45:42.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/1258221436 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc048020 con 0x560b5b170350 2024-04-16T05:45:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/1258221436 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdcbc058cc0 con 0x560b5b170350 2024-04-16T05:45:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcd2438740 10 client.18636 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcd2438740 10 client.18636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcd2438740 20 client.18636 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcd2438740 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560b5b1e6ec0 0x560b5b1e92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcd2438740 4 client.18636 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:45:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.466+0000 7fdcd2438740 1 -- 172.21.15.86:0/1258221436 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b5b0d3a10 con 0x560b5b1e6ec0 2024-04-16T05:45:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560b5b1e6ec0 0x560b5b1e92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcb0ff9640 10 client.18636 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdccd5e6640 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560b5b1e6ec0 0x560b5b1e92b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf344227a321811 server_cookie=13ad3cddd6398802 in_seq=0 out_seq=0 2024-04-16T05:45:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcb0ff9640 1 -- 172.21.15.86:0/1258221436 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560b5b0d3a10 con 0x560b5b1e6ec0 2024-04-16T05:45:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcb0ff9640 10 client.18636 handle_command_reply: tid=0 2024-04-16T05:45:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcd2438740 1 client.18636 shutdown 2024-04-16T05:45:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcd2438740 1 -- 172.21.15.86:0/1258221436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 msgr2=0x560b5b1759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcd2438740 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b1759c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc050040 tx=0x7fdcbc0037e0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcd2438740 1 -- 172.21.15.86:0/1258221436 shutdown_connections 2024-04-16T05:45:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcd2438740 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560b5b1e6ec0 0x560b5b1e92b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcd2438740 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b5b170350 0x560b5b1759c0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcd2438740 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b5b167770 0x560b5b1789b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcb0ff9640 0 client.18636 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcd2438740 1 --2- 172.21.15.86:0/1258221436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b5b166e50 0x560b5b178470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.467+0000 7fdcd2438740 1 -- 172.21.15.86:0/1258221436 >> 172.21.15.86:0/1258221436 conn(0x560b5b161580 msgr2=0x560b5b160760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdcd2438740 1 -- 172.21.15.86:0/1258221436 shutdown_connections 2024-04-16T05:45:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdcd2438740 1 -- 172.21.15.86:0/1258221436 wait complete. 2024-04-16T05:45:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdcd2438740 20 client.18636 trim_cache size 0 max 16384 2024-04-16T05:45:42.482 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:45:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-16T05:45:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-16T05:45:42.483 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:45:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 -- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdcb40a3da0 msgr2=0x7fdcb40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdcb40a3da0 0x7fdcb40a6260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0098ee0 tx=0x7fdcc006f570 comp rx=0 tx=0).stop 2024-04-16T05:45:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 -- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc8107040 msgr2=0x7fdcc8101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc8107040 0x7fdcc8101d30 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000cad0 tx=0x7fdcc000cf90 comp rx=0 tx=0).stop 2024-04-16T05:45:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 -- 172.21.15.86:0/297371055 shutdown_connections 2024-04-16T05:45:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcc8107040 0x7fdcc8101d30 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdcb40a3da0 0x7fdcb40a6260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcc806afe0 0x7fdcc81017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 --2- 172.21.15.86:0/297371055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc806a620 0x7fdcc81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 -- 172.21.15.86:0/297371055 >> 172.21.15.86:0/297371055 conn(0x7fdcc8077ee0 msgr2=0x7fdcc80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 -- 172.21.15.86:0/297371055 shutdown_connections 2024-04-16T05:45:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.468+0000 7fdccf84a640 1 -- 172.21.15.86:0/297371055 wait complete. 2024-04-16T05:45:42.489 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:45:42.489 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:45:42.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.596+0000 7fdaf45e1640 1 -- 172.21.15.86:0/1476868954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 msgr2=0x7fdaec100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.596+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/1476868954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 0x7fdaec100dd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009900 tx=0x7fdadc02bcd0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.596+0000 7fdaf45e1640 1 -- 172.21.15.86:0/1476868954 shutdown_connections 2024-04-16T05:45:42.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.596+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/1476868954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaec101310 0x7fdaec10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.596+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/1476868954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 0x7fdaec100dd0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.596+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/1476868954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaec106730 0x7fdaec106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.596+0000 7fdaf45e1640 1 -- 172.21.15.86:0/1476868954 >> 172.21.15.86:0/1476868954 conn(0x7fdaec0761e0 msgr2=0x7fdaec0765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.596+0000 7fdaf45e1640 1 -- 172.21.15.86:0/1476868954 shutdown_connections 2024-04-16T05:45:42.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.596+0000 7fdaf45e1640 1 -- 172.21.15.86:0/1476868954 wait complete. 2024-04-16T05:45:42.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf45e1640 1 Processor -- start 2024-04-16T05:45:42.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf45e1640 1 -- start start 2024-04-16T05:45:42.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf45e1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 0x7fdaec19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf45e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaec101310 0x7fdaec19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf45e1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaec106730 0x7fdaec1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf45e1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec1101e0 con 0x7fdaec100950 2024-04-16T05:45:42.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf45e1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec110060 con 0x7fdaec106730 2024-04-16T05:45:42.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf45e1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec110360 con 0x7fdaec101310 2024-04-16T05:45:42.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf237d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 0x7fdaec19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf237d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 0x7fdaec19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48470/0 (socket says 172.21.15.86:48470) 2024-04-16T05:45:42.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf237d640 1 -- 172.21.15.86:0/2204528993 learned_addr learned my addr 172.21.15.86:0/2204528993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.597+0000 7fdaf2b7e640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaec106730 0x7fdaec1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf1b7c640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaec101310 0x7fdaec19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf237d640 1 -- 172.21.15.86:0/2204528993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaec101310 msgr2=0x7fdaec19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf237d640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaec101310 0x7fdaec19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf237d640 1 -- 172.21.15.86:0/2204528993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaec106730 msgr2=0x7fdaec1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf237d640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaec106730 0x7fdaec1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf237d640 1 -- 172.21.15.86:0/2204528993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec198fb0 con 0x7fdaec100950 2024-04-16T05:45:42.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf1b7c640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaec101310 0x7fdaec19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:42.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf237d640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 0x7fdaec19e5e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d770 tx=0x7fdae800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdae37fe640 1 -- 172.21.15.86:0/2204528993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8017070 con 0x7fdaec100950 2024-04-16T05:45:42.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdae37fe640 1 -- 172.21.15.86:0/2204528993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae8005500 con 0x7fdaec100950 2024-04-16T05:45:42.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaec1992a0 con 0x7fdaec100950 2024-04-16T05:45:42.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdae37fe640 1 -- 172.21.15.86:0/2204528993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8004e00 con 0x7fdaec100950 2024-04-16T05:45:42.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.598+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaec1a5560 con 0x7fdaec100950 2024-04-16T05:45:42.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.599+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaec101dc0 con 0x7fdaec100950 2024-04-16T05:45:42.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.600+0000 7fdae37fe640 1 -- 172.21.15.86:0/2204528993 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdae8004fa0 con 0x7fdaec100950 2024-04-16T05:45:42.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.601+0000 7fdae37fe640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdad80a3d20 0x7fdad80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.601+0000 7fdaf1b7c640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdad80a3d20 0x7fdad80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.601+0000 7fdae37fe640 1 -- 172.21.15.86:0/2204528993 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fdae80d0a20 con 0x7fdaec100950 2024-04-16T05:45:42.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.601+0000 7fdaf1b7c640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdad80a3d20 0x7fdad80a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02efd0 tx=0x7fdadc000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.604+0000 7fdae37fe640 1 -- 172.21.15.86:0/2204528993 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdae809ccf0 con 0x7fdaec100950 2024-04-16T05:45:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.823+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaec1994d0 con 0x7fdaec100950 2024-04-16T05:45:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.827+0000 7fdae37fe640 1 -- 172.21.15.86:0/2204528993 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fdae809c5f0 con 0x7fdaec100950 2024-04-16T05:45:42.825 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:45:42.826 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:45:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:45:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdad80a3d20 msgr2=0x7fdad80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdad80a3d20 0x7fdad80a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02efd0 tx=0x7fdadc000bc0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 msgr2=0x7fdaec19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 0x7fdaec19e5e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d770 tx=0x7fdae800dc30 comp rx=0 tx=0).stop 2024-04-16T05:45:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 shutdown_connections 2024-04-16T05:45:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdad80a3d20 0x7fdad80a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaec106730 0x7fdaec1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaec101310 0x7fdaec19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 --2- 172.21.15.86:0/2204528993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100950 0x7fdaec19e5e0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 >> 172.21.15.86:0/2204528993 conn(0x7fdaec0761e0 msgr2=0x7fdaec0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 shutdown_connections 2024-04-16T05:45:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.829+0000 7fdaf45e1640 1 -- 172.21.15.86:0/2204528993 wait complete. 2024-04-16T05:45:42.839 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-16T05:45:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.949+0000 7f27cb782640 1 -- 172.21.15.86:0/3278406481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4101130 msgr2=0x7f27c4101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.949+0000 7f27cb782640 1 --2- 172.21.15.86:0/3278406481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4101130 0x7f27c4101510 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009900 tx=0x7f27c002bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.949+0000 7f27cb782640 1 -- 172.21.15.86:0/3278406481 shutdown_connections 2024-04-16T05:45:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.949+0000 7f27cb782640 1 --2- 172.21.15.86:0/3278406481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c4106200 0x7f27c410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.949+0000 7f27cb782640 1 --2- 172.21.15.86:0/3278406481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c4101ae0 0x7f27c4105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.949+0000 7f27cb782640 1 --2- 172.21.15.86:0/3278406481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4101130 0x7f27c4101510 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.949+0000 7f27cb782640 1 -- 172.21.15.86:0/3278406481 >> 172.21.15.86:0/3278406481 conn(0x7f27c40780b0 msgr2=0x7f27c40784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.949+0000 7f27cb782640 1 -- 172.21.15.86:0/3278406481 shutdown_connections 2024-04-16T05:45:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.949+0000 7f27cb782640 1 -- 172.21.15.86:0/3278406481 wait complete. 2024-04-16T05:45:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27cb782640 1 Processor -- start 2024-04-16T05:45:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27cb782640 1 -- start start 2024-04-16T05:45:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27cb782640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c4101130 0x7f27c41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27cb782640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4106200 0x7f27c4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27cb782640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c419bfa0 0x7f27c419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27cb782640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41112c0 con 0x7f27c4106200 2024-04-16T05:45:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27cb782640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4111140 con 0x7f27c419bfa0 2024-04-16T05:45:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27cb782640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4111440 con 0x7f27c4101130 2024-04-16T05:45:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27c8d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4106200 0x7f27c4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27c8d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4106200 0x7f27c4197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48476/0 (socket says 172.21.15.86:48476) 2024-04-16T05:45:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27c8d1d640 1 -- 172.21.15.86:0/2352512751 learned_addr learned my addr 172.21.15.86:0/2352512751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27c8d1d640 1 -- 172.21.15.86:0/2352512751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c4101130 msgr2=0x7f27c41976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27c951e640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c4101130 0x7f27c41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c4101130 0x7f27c41976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27c9d1f640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c419bfa0 0x7f27c419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27c8d1d640 1 -- 172.21.15.86:0/2352512751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c419bfa0 msgr2=0x7f27c419c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.950+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c419bfa0 0x7f27c419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.951+0000 7f27c8d1d640 1 -- 172.21.15.86:0/2352512751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c419ca90 con 0x7f27c4106200 2024-04-16T05:45:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.951+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4106200 0x7f27c4197c20 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f27b4007b50 tx=0x7f27b400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.951+0000 7f27ba7fc640 1 -- 172.21.15.86:0/2352512751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b401a070 con 0x7f27c4106200 2024-04-16T05:45:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.951+0000 7f27cb782640 1 -- 172.21.15.86:0/2352512751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c419cd80 con 0x7f27c4106200 2024-04-16T05:45:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.951+0000 7f27ba7fc640 1 -- 172.21.15.86:0/2352512751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b4007c40 con 0x7f27c4106200 2024-04-16T05:45:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.951+0000 7f27ba7fc640 1 -- 172.21.15.86:0/2352512751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4003920 con 0x7f27c4106200 2024-04-16T05:45:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.951+0000 7f27cb782640 1 -- 172.21.15.86:0/2352512751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c4069500 con 0x7f27c4106200 2024-04-16T05:45:42.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.953+0000 7f27ba7fc640 1 -- 172.21.15.86:0/2352512751 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27b4003ac0 con 0x7f27c4106200 2024-04-16T05:45:42.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27ce36c740 1 Processor -- start 2024-04-16T05:45:42.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27ce36c740 1 -- start start 2024-04-16T05:45:42.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 0x564640df8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27ba7fc640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27b00a3d20 0x7f27b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640e00e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640e03750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27ce36c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564640d5d0e0 con 0x564640df8780 2024-04-16T05:45:42.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27ce36c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564640cfa520 con 0x564640df7e60 2024-04-16T05:45:42.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27ce36c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564640d590a0 con 0x564640e01360 2024-04-16T05:45:42.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27c8d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640e00e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27c951e640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27b00a3d20 0x7f27b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27c8d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640e00e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48486/0 (socket says 172.21.15.86:48486) 2024-04-16T05:45:42.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27c8d1d640 1 -- 172.21.15.86:0/1241312400 learned_addr learned my addr 172.21.15.86:0/1241312400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27c9d1f640 1 --2- 172.21.15.86:0/1241312400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640e03750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.954+0000 7f27c9d1f640 1 --2- 172.21.15.86:0/1241312400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 0x564640df8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27ba7fc640 1 -- 172.21.15.86:0/2352512751 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f27b40d1ac0 con 0x7f27c4106200 2024-04-16T05:45:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27c8d1d640 1 -- 172.21.15.86:0/1241312400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 msgr2=0x564640e03750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/1241312400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640e03750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27c8d1d640 1 -- 172.21.15.86:0/1241312400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 msgr2=0x564640df8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/1241312400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 0x564640df8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27c8d1d640 1 -- 172.21.15.86:0/1241312400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564640d228d0 con 0x564640df8780 2024-04-16T05:45:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27c951e640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27b00a3d20 0x7f27b00a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f27c0036040 tx=0x7f27c0005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/1241312400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640e00e20 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f27b4099370 tx=0x7f27b400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665b1e8b36f9a66a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27af7fe640 1 -- 172.21.15.86:0/1241312400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b40a2bd0 con 0x564640df8780 2024-04-16T05:45:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27af7fe640 1 -- 172.21.15.86:0/1241312400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b409dd90 con 0x564640df8780 2024-04-16T05:45:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.955+0000 7f27af7fe640 1 -- 172.21.15.86:0/1241312400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b40a3fe0 con 0x564640df8780 2024-04-16T05:45:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 -- 172.21.15.86:0/1241312400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 msgr2=0x564640e00e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 --2- 172.21.15.86:0/1241312400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640e00e20 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f27b4099370 tx=0x7f27b400ed40 comp rx=0 tx=0).stop 2024-04-16T05:45:42.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 -- 172.21.15.86:0/1241312400 shutdown_connections 2024-04-16T05:45:42.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 --2- 172.21.15.86:0/1241312400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640e03750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 --2- 172.21.15.86:0/1241312400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640e00e20 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 --2- 172.21.15.86:0/1241312400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 0x564640df8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 -- 172.21.15.86:0/1241312400 >> 172.21.15.86:0/1241312400 conn(0x564640df2590 msgr2=0x564640df2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 -- 172.21.15.86:0/1241312400 shutdown_connections 2024-04-16T05:45:42.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 -- 172.21.15.86:0/1241312400 wait complete. 2024-04-16T05:45:42.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 -- start start 2024-04-16T05:45:42.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 Processor -- start 2024-04-16T05:45:42.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.956+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 0x564640dfe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640dfeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27ce36c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564640e03fa0 con 0x564640df8780 2024-04-16T05:45:42.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27ce36c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564640d164d0 con 0x564640df7e60 2024-04-16T05:45:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27ce36c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564640e04120 con 0x564640e01360 2024-04-16T05:45:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27ce36c740 10 client.0 _ll_register_callbacks cb 0x564640cfb2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c8d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c8d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfe990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48494/0 (socket says 172.21.15.86:48494) 2024-04-16T05:45:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c8d1d640 1 -- 172.21.15.86:0/2557834800 learned_addr learned my addr 172.21.15.86:0/2557834800 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c9d1f640 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640dfeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c951e640 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 0x564640dfe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c8d1d640 1 -- 172.21.15.86:0/2557834800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 msgr2=0x564640dfeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640dfeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c8d1d640 1 -- 172.21.15.86:0/2557834800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 msgr2=0x564640dfe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 0x564640dfe450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c8d1d640 1 -- 172.21.15.86:0/2557834800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564640a6f3f0 con 0x564640df8780 2024-04-16T05:45:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfe990 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ebe0 tx=0x7f27b40a2c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16eec772d115d96b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27acff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27acff9640 1 -- 172.21.15.86:0/2557834800 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b40a2f50 con 0x564640df8780 2024-04-16T05:45:42.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27acff9640 1 -- 172.21.15.86:0/2557834800 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f27b409b990 con 0x564640df8780 2024-04-16T05:45:42.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27ce36c740 1 -- 172.21.15.86:0/2557834800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b409dd90 con 0x564640df8780 2024-04-16T05:45:42.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27acff9640 1 -- 172.21.15.86:0/2557834800 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27b40a6300 con 0x564640df8780 2024-04-16T05:45:42.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27ce36c740 10 client.18654 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:42.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.957+0000 7f27ce36c740 1 -- 172.21.15.86:0/2557834800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564640b397d0 con 0x564640df8780 2024-04-16T05:45:42.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.958+0000 7f27acff9640 1 -- 172.21.15.86:0/2557834800 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b409dd90 con 0x564640df8780 2024-04-16T05:45:42.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.958+0000 7f27acff9640 1 -- 172.21.15.86:0/2557834800 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f27b4014030 con 0x564640df8780 2024-04-16T05:45:42.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.958+0000 7f27ce36c740 10 client.18654 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:42.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.958+0000 7f27ce36c740 10 client.18654 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:45:42.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.958+0000 7f27ce36c740 20 client.18654 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:42.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.958+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x564640e11100 0x564640e134f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.958+0000 7f27ce36c740 4 client.18654 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:42.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.958+0000 7f27ce36c740 1 -- 172.21.15.86:0/2557834800 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564640d6ab00 con 0x564640e11100 2024-04-16T05:45:42.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.958+0000 7f27c951e640 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x564640e11100 0x564640e134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27acff9640 10 client.18654 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:45:42.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27c951e640 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x564640e11100 0x564640e134f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46b389ce4f6b8eaa server_cookie=996d013fa128856c in_seq=0 out_seq=0 2024-04-16T05:45:42.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27acff9640 1 -- 172.21.15.86:0/2557834800 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564640d6ab00 con 0x564640e11100 2024-04-16T05:45:42.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27acff9640 10 client.18654 handle_command_reply: tid=0 2024-04-16T05:45:42.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27ce36c740 1 client.18654 shutdown 2024-04-16T05:45:42.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27ce36c740 1 -- 172.21.15.86:0/2557834800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 msgr2=0x564640dfe990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfe990 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ebe0 tx=0x7f27b40a2c40 comp rx=0 tx=0).stop 2024-04-16T05:45:42.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27ce36c740 1 -- 172.21.15.86:0/2557834800 shutdown_connections 2024-04-16T05:45:42.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x564640e11100 0x564640e134f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640dfeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27acff9640 0 client.18654 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:45:42.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.959+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfe990 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.960+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2557834800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df7e60 0x564640dfe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.960+0000 7f27ce36c740 1 -- 172.21.15.86:0/2557834800 >> 172.21.15.86:0/2557834800 conn(0x564640df2590 msgr2=0x564640df42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.960+0000 7f27ce36c740 1 -- 172.21.15.86:0/2557834800 shutdown_connections 2024-04-16T05:45:42.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.960+0000 7f27ce36c740 1 -- 172.21.15.86:0/2557834800 wait complete. 2024-04-16T05:45:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.960+0000 7f27ce36c740 20 client.18654 trim_cache size 0 max 16384 2024-04-16T05:45:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27ce36c740 1 Processor -- start 2024-04-16T05:45:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27ce36c740 1 -- start start 2024-04-16T05:45:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640df7e60 0x564640dfb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640e01360 0x564640e0c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27ce36c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ad270 con 0x564640df8780 2024-04-16T05:45:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27ce36c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41aedb0 con 0x564640e01360 2024-04-16T05:45:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27ce36c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ad940 con 0x564640df7e60 2024-04-16T05:45:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27c8d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27c8d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48508/0 (socket says 172.21.15.86:48508) 2024-04-16T05:45:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.973+0000 7f27c8d1d640 1 -- 172.21.15.86:0/1855446987 learned_addr learned my addr 172.21.15.86:0/1855446987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27c9d1f640 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640e01360 0x564640e0c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27c951e640 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640df7e60 0x564640dfb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27c8d1d640 1 -- 172.21.15.86:0/1855446987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640df7e60 msgr2=0x564640dfb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640df7e60 0x564640dfb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27c8d1d640 1 -- 172.21.15.86:0/1855446987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640e01360 msgr2=0x564640e0c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640e01360 0x564640e0c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27c8d1d640 1 -- 172.21.15.86:0/1855446987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564640dede60 con 0x564640df8780 2024-04-16T05:45:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27c9d1f640 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640e01360 0x564640e0c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfbc50 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f27b40a1c00 tx=0x7f27b400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccfeb0e596622bd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27aeffd640 1 -- 172.21.15.86:0/1855446987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4086e60 con 0x564640df8780 2024-04-16T05:45:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27aeffd640 1 -- 172.21.15.86:0/1855446987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b4086bd0 con 0x564640df8780 2024-04-16T05:45:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.974+0000 7f27aeffd640 1 -- 172.21.15.86:0/1855446987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b409cae0 con 0x564640df8780 2024-04-16T05:45:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 -- 172.21.15.86:0/1855446987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 msgr2=0x564640dfbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfbc50 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f27b40a1c00 tx=0x7f27b400ed40 comp rx=0 tx=0).stop 2024-04-16T05:45:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 -- 172.21.15.86:0/1855446987 shutdown_connections 2024-04-16T05:45:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640e01360 0x564640e0c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df8780 0x564640dfbc50 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 --2- 172.21.15.86:0/1855446987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640df7e60 0x564640dfb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 -- 172.21.15.86:0/1855446987 >> 172.21.15.86:0/1855446987 conn(0x564640df2590 msgr2=0x564640df2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 -- 172.21.15.86:0/1855446987 shutdown_connections 2024-04-16T05:45:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 -- 172.21.15.86:0/1855446987 wait complete. 2024-04-16T05:45:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 -- start start 2024-04-16T05:45:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.975+0000 7f27ce36c740 1 Processor -- start 2024-04-16T05:45:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df7e60 0x564640e13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df8780 0x564640e143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27ce36c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640e148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27ce36c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ae6e0 con 0x564640df7e60 2024-04-16T05:45:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27ce36c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ae010 con 0x564640df8780 2024-04-16T05:45:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27ce36c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ab270 con 0x564640e01360 2024-04-16T05:45:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27ce36c740 10 client.0 _ll_register_callbacks cb 0x7f27b4099270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c951e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df7e60 0x564640e13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c951e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df7e60 0x564640e13e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48520/0 (socket says 172.21.15.86:48520) 2024-04-16T05:45:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c951e640 1 -- 172.21.15.86:0/2493491929 learned_addr learned my addr 172.21.15.86:0/2493491929 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df8780 0x564640e143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c9d1f640 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640e148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c951e640 1 -- 172.21.15.86:0/2493491929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 msgr2=0x564640e148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c951e640 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640e148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c951e640 1 -- 172.21.15.86:0/2493491929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df8780 msgr2=0x564640e143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c951e640 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df8780 0x564640e143a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c951e640 1 -- 172.21.15.86:0/2493491929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564640e762e0 con 0x564640df7e60 2024-04-16T05:45:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c9d1f640 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640e148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27c951e640 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df7e60 0x564640e13e60 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f27c002caf0 tx=0x7f27c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7d3615f8e03a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27acff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27acff9640 1 -- 172.21.15.86:0/2493491929 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0051070 con 0x564640df7e60 2024-04-16T05:45:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27acff9640 1 -- 172.21.15.86:0/2493491929 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f27c0005310 con 0x564640df7e60 2024-04-16T05:45:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.976+0000 7f27ce36c740 1 -- 172.21.15.86:0/2493491929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564640d64970 con 0x564640df7e60 2024-04-16T05:45:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27acff9640 1 -- 172.21.15.86:0/2493491929 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27c005d050 con 0x564640df7e60 2024-04-16T05:45:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27ce36c740 10 client.18660 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27ce36c740 1 -- 172.21.15.86:0/2493491929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564640e774e0 con 0x564640df7e60 2024-04-16T05:45:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27acff9640 1 -- 172.21.15.86:0/2493491929 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564640d64970 con 0x564640df7e60 2024-04-16T05:45:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27acff9640 1 -- 172.21.15.86:0/2493491929 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f27c0048040 con 0x564640df7e60 2024-04-16T05:45:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27ce36c740 10 client.18660 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27ce36c740 10 client.18660 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-16T05:45:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27ce36c740 20 client.18660 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x564640e11100 0x564640e79c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27ce36c740 4 client.18660 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27ce36c740 1 -- 172.21.15.86:0/2493491929 --> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564640ce06b0 con 0x564640e11100 2024-04-16T05:45:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.977+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x564640e11100 0x564640e79c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27c8d1d640 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x564640e11100 0x564640e79c50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6f0d988a0f38a64 server_cookie=330df399e5b0f02f in_seq=0 out_seq=0 2024-04-16T05:45:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27acff9640 10 client.18660 ms_handle_connect on v2:172.21.15.196:6834/3736061748 2024-04-16T05:45:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27acff9640 1 -- 172.21.15.86:0/2493491929 <== mds.4 v2:172.21.15.196:6834/3736061748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564640ce06b0 con 0x564640e11100 2024-04-16T05:45:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27acff9640 10 client.18660 handle_command_reply: tid=0 2024-04-16T05:45:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 client.18660 shutdown 2024-04-16T05:45:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 -- 172.21.15.86:0/2493491929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df7e60 msgr2=0x564640e13e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df7e60 0x564640e13e60 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f27c002caf0 tx=0x7f27c0009260 comp rx=0 tx=0).stop 2024-04-16T05:45:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 -- 172.21.15.86:0/2493491929 shutdown_connections 2024-04-16T05:45:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.196:6834/3736061748,v1:172.21.15.196:6835/3736061748] conn(0x564640e11100 0x564640e79c50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564640e01360 0x564640e148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564640df8780 0x564640e143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 --2- 172.21.15.86:0/2493491929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564640df7e60 0x564640e13e60 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27acff9640 0 client.18660 ms_handle_reset on v2:172.21.15.196:6834/3736061748 2024-04-16T05:45:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 -- 172.21.15.86:0/2493491929 >> 172.21.15.86:0/2493491929 conn(0x564640df2590 msgr2=0x564640e0c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 -- 172.21.15.86:0/2493491929 shutdown_connections 2024-04-16T05:45:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 1 -- 172.21.15.86:0/2493491929 wait complete. 2024-04-16T05:45:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.978+0000 7f27ce36c740 20 client.18660 trim_cache size 0 max 16384 2024-04-16T05:45:42.992 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 -- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27b00a3d20 msgr2=0x7f27b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27b00a3d20 0x7f27b00a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f27c0036040 tx=0x7f27c0005fd0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 -- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4106200 msgr2=0x7f27c4197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4106200 0x7f27c4197c20 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f27b4007b50 tx=0x7f27b400e420 comp rx=0 tx=0).stop 2024-04-16T05:45:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 -- 172.21.15.86:0/2352512751 shutdown_connections 2024-04-16T05:45:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f27b00a3d20 0x7f27b00a61e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c419bfa0 0x7f27c419c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27c4106200 0x7f27c4197c20 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 --2- 172.21.15.86:0/2352512751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c4101130 0x7f27c41976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 -- 172.21.15.86:0/2352512751 >> 172.21.15.86:0/2352512751 conn(0x7f27c40780b0 msgr2=0x7f27c40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 -- 172.21.15.86:0/2352512751 shutdown_connections 2024-04-16T05:45:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:42.979+0000 7f27cb782640 1 -- 172.21.15.86:0/2352512751 wait complete. 2024-04-16T05:45:42.995 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:42 smithi086 ceph-mon[32028]: pgmap v323: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:42.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-16T05:45:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 -- 172.21.15.86:0/3230164912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 msgr2=0x7fe8fc10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 --2- 172.21.15.86:0/3230164912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 0x7fe8fc10ddd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009ca0 tx=0x7fe8f402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 -- 172.21.15.86:0/3230164912 shutdown_connections 2024-04-16T05:45:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 --2- 172.21.15.86:0/3230164912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 0x7fe8fc10ddd0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 --2- 172.21.15.86:0/3230164912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8fc101ae0 0x7fe8fc105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 --2- 172.21.15.86:0/3230164912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc101130 0x7fe8fc101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 -- 172.21.15.86:0/3230164912 >> 172.21.15.86:0/3230164912 conn(0x7fe8fc0780b0 msgr2=0x7fe8fc0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 -- 172.21.15.86:0/3230164912 shutdown_connections 2024-04-16T05:45:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 -- 172.21.15.86:0/3230164912 wait complete. 2024-04-16T05:45:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.108+0000 7fe903c47640 1 Processor -- start 2024-04-16T05:45:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe903c47640 1 -- start start 2024-04-16T05:45:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe903c47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc101130 0x7fe8fc19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe903c47640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 0x7fe8fc19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe903c47640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8fc1986a0 0x7fe8fc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe903c47640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1112c0 con 0x7fe8fc106200 2024-04-16T05:45:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe903c47640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc111140 con 0x7fe8fc1986a0 2024-04-16T05:45:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe903c47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc111440 con 0x7fe8fc101130 2024-04-16T05:45:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9011e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 0x7fe8fc19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9011e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 0x7fe8fc19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48526/0 (socket says 172.21.15.86:48526) 2024-04-16T05:45:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9011e2640 1 -- 172.21.15.86:0/1638943646 learned_addr learned my addr 172.21.15.86:0/1638943646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8fc1986a0 0x7fe8fc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc101130 0x7fe8fc19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9011e2640 1 -- 172.21.15.86:0/1638943646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc101130 msgr2=0x7fe8fc19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9011e2640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc101130 0x7fe8fc19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9011e2640 1 -- 172.21.15.86:0/1638943646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8fc1986a0 msgr2=0x7fe8fc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9011e2640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8fc1986a0 0x7fe8fc198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9011e2640 1 -- 172.21.15.86:0/1638943646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc1992a0 con 0x7fe8fc106200 2024-04-16T05:45:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc101130 0x7fe8fc19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8fc1986a0 0x7fe8fc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.109+0000 7fe9011e2640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 0x7fe8fc19eb20 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00d770 tx=0x7fe8ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.110+0000 7fe8f2ffd640 1 -- 172.21.15.86:0/1638943646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec017070 con 0x7fe8fc106200 2024-04-16T05:45:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.110+0000 7fe8f2ffd640 1 -- 172.21.15.86:0/1638943646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8ec009c30 con 0x7fe8fc106200 2024-04-16T05:45:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.110+0000 7fe903c47640 1 -- 172.21.15.86:0/1638943646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc1a5380 con 0x7fe8fc106200 2024-04-16T05:45:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.110+0000 7fe8f2ffd640 1 -- 172.21.15.86:0/1638943646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00f620 con 0x7fe8fc106200 2024-04-16T05:45:43.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.110+0000 7fe903c47640 1 -- 172.21.15.86:0/1638943646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc1a57e0 con 0x7fe8fc106200 2024-04-16T05:45:43.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.112+0000 7fe8f2ffd640 1 -- 172.21.15.86:0/1638943646 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe8ec01a070 con 0x7fe8fc106200 2024-04-16T05:45:43.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.112+0000 7fe90682f740 1 Processor -- start 2024-04-16T05:45:43.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe90682f740 1 -- start start 2024-04-16T05:45:43.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe90682f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 0x55c854d00240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe90682f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d08e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe8f2ffd640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe8e80a3da0 0x7fe8e80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe90682f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d09360 0x55c854d0b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe90682f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c854c650e0 con 0x55c854d09360 2024-04-16T05:45:43.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe90682f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c854c02520 con 0x55c854d00780 2024-04-16T05:45:43.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe90682f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c854c610a0 con 0x55c854cffe60 2024-04-16T05:45:43.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe9021e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d09360 0x55c854d0b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe8e80a3da0 0x7fe8e80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe9021e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d09360 0x55c854d0b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48530/0 (socket says 172.21.15.86:48530) 2024-04-16T05:45:43.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe9021e4640 1 -- 172.21.15.86:0/1443178628 learned_addr learned my addr 172.21.15.86:0/1443178628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe8f2ffd640 1 -- 172.21.15.86:0/1638943646 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe8ec0d0710 con 0x7fe8fc106200 2024-04-16T05:45:43.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe9011e2640 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d08e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.113+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 0x55c854d00240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe9021e4640 1 -- 172.21.15.86:0/1443178628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 msgr2=0x55c854d00240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 0x55c854d00240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe9021e4640 1 -- 172.21.15.86:0/1443178628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 msgr2=0x55c854d08e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d08e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe9021e4640 1 -- 172.21.15.86:0/1443178628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c854c2a8d0 con 0x55c854d09360 2024-04-16T05:45:43.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe9011e2640 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d08e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:45:43.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe8e80a3da0 0x7fe8e80a6260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009b50 tx=0x7fe8f8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d09360 0x55c854d0b750 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005fe0 tx=0x7fe8f402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e891ef05e782ea6e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe8e7fff640 1 -- 172.21.15.86:0/1443178628 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4048070 con 0x55c854d09360 2024-04-16T05:45:43.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe8e7fff640 1 -- 172.21.15.86:0/1443178628 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f4003970 con 0x55c854d09360 2024-04-16T05:45:43.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.114+0000 7fe8e7fff640 1 -- 172.21.15.86:0/1443178628 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4039070 con 0x55c854d09360 2024-04-16T05:45:43.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 -- 172.21.15.86:0/1443178628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d09360 msgr2=0x55c854d0b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d09360 0x55c854d0b750 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005fe0 tx=0x7fe8f402e040 comp rx=0 tx=0).stop 2024-04-16T05:45:43.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 -- 172.21.15.86:0/1443178628 shutdown_connections 2024-04-16T05:45:43.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d09360 0x55c854d0b750 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d08e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 --2- 172.21.15.86:0/1443178628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 0x55c854d00240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 -- 172.21.15.86:0/1443178628 >> 172.21.15.86:0/1443178628 conn(0x55c854cfa590 msgr2=0x55c854cfa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 -- 172.21.15.86:0/1443178628 shutdown_connections 2024-04-16T05:45:43.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 -- 172.21.15.86:0/1443178628 wait complete. 2024-04-16T05:45:43.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 -- start start 2024-04-16T05:45:43.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.115+0000 7fe90682f740 1 Processor -- start 2024-04-16T05:45:43.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe90682f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 0x55c854d06480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe90682f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d00780 0x55c854d069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe90682f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d09360 0x55c854d06f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe90682f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c854d0bfa0 con 0x55c854d00780 2024-04-16T05:45:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe90682f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c854c1e4d0 con 0x55c854d09360 2024-04-16T05:45:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe90682f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c854d0c120 con 0x55c854cffe60 2024-04-16T05:45:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe90682f740 10 client.0 _ll_register_callbacks cb 0x55c854c032c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9011e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d00780 0x55c854d069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9011e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d00780 0x55c854d069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48546/0 (socket says 172.21.15.86:48546) 2024-04-16T05:45:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9011e2640 1 -- 172.21.15.86:0/1848653072 learned_addr learned my addr 172.21.15.86:0/1848653072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d09360 0x55c854d06f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 0x55c854d06480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9011e2640 1 -- 172.21.15.86:0/1848653072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 msgr2=0x55c854d06480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9011e2640 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 0x55c854d06480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9011e2640 1 -- 172.21.15.86:0/1848653072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d09360 msgr2=0x55c854d06f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9011e2640 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d09360 0x55c854d06f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9011e2640 1 -- 172.21.15.86:0/1848653072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8549773f0 con 0x55c854d00780 2024-04-16T05:45:43.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d09360 0x55c854d06f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe9011e2640 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d00780 0x55c854d069c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec09bfe0 tx=0x7fe8ec09c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9928eff438a1e2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe8e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe8e57fa640 1 -- 172.21.15.86:0/1848653072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0b5060 con 0x55c854d00780 2024-04-16T05:45:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe8e57fa640 1 -- 172.21.15.86:0/1848653072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe8ec098950 con 0x55c854d00780 2024-04-16T05:45:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.116+0000 7fe90682f740 1 -- 172.21.15.86:0/1848653072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f4003970 con 0x55c854d00780 2024-04-16T05:45:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe8e57fa640 1 -- 172.21.15.86:0/1848653072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8ec021040 con 0x55c854d00780 2024-04-16T05:45:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe90682f740 10 client.18672 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe90682f740 1 -- 172.21.15.86:0/1848653072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c854a417d0 con 0x55c854d00780 2024-04-16T05:45:43.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe8e57fa640 1 -- 172.21.15.86:0/1848653072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f4003970 con 0x55c854d00780 2024-04-16T05:45:43.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe8e57fa640 1 -- 172.21.15.86:0/1848653072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe8ec0a0170 con 0x55c854d00780 2024-04-16T05:45:43.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe90682f740 10 client.18672 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:43.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe90682f740 10 client.18672 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:45:43.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe90682f740 20 client.18672 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:43.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe90682f740 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c854d19380 0x55c854d1b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe90682f740 4 client.18672 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:43.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe90682f740 1 -- 172.21.15.86:0/1848653072 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c854c72b00 con 0x55c854d19380 2024-04-16T05:45:43.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.117+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c854d19380 0x55c854d1b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.118+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c854d19380 0x55c854d1b7d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b89f0bf9c11d2ed5 server_cookie=bbbf9f9cd4144828 in_seq=0 out_seq=0 2024-04-16T05:45:43.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.118+0000 7fe8e57fa640 10 client.18672 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:45:43.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe8e57fa640 1 -- 172.21.15.86:0/1848653072 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c854c72b00 con 0x55c854d19380 2024-04-16T05:45:43.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe8e57fa640 10 client.18672 handle_command_reply: tid=0 2024-04-16T05:45:43.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 client.18672 shutdown 2024-04-16T05:45:43.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 -- 172.21.15.86:0/1848653072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d00780 msgr2=0x55c854d069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d00780 0x55c854d069c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec09bfe0 tx=0x7fe8ec09c9e0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 -- 172.21.15.86:0/1848653072 shutdown_connections 2024-04-16T05:45:43.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d09360 0x55c854d06f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854d00780 0x55c854d069c0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c854d19380 0x55c854d1b7d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 --2- 172.21.15.86:0/1848653072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854cffe60 0x55c854d06480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe8e57fa640 0 client.18672 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:45:43.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 -- 172.21.15.86:0/1848653072 >> 172.21.15.86:0/1848653072 conn(0x55c854cfa590 msgr2=0x55c854cfc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 -- 172.21.15.86:0/1848653072 shutdown_connections 2024-04-16T05:45:43.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 1 -- 172.21.15.86:0/1848653072 wait complete. 2024-04-16T05:45:43.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.119+0000 7fe90682f740 20 client.18672 trim_cache size 0 max 16384 2024-04-16T05:45:43.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe90682f740 1 Processor -- start 2024-04-16T05:45:43.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe90682f740 1 -- start start 2024-04-16T05:45:43.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe90682f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d08bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe90682f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d00b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe90682f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 0x55c854d21070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe90682f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1af5a0 con 0x55c854cffe60 2024-04-16T05:45:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe90682f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1b10e0 con 0x55c854d00780 2024-04-16T05:45:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe90682f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1afc70 con 0x55c854d09360 2024-04-16T05:45:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9019e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d08bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9019e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d08bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48548/0 (socket says 172.21.15.86:48548) 2024-04-16T05:45:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9019e3640 1 -- 172.21.15.86:0/1083745336 learned_addr learned my addr 172.21.15.86:0/1083745336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9011e2640 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d00b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 0x55c854d21070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9019e3640 1 -- 172.21.15.86:0/1083745336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 msgr2=0x55c854d21070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 0x55c854d21070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9019e3640 1 -- 172.21.15.86:0/1083745336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 msgr2=0x55c854d00b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d00b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9019e3640 1 -- 172.21.15.86:0/1083745336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c854d05430 con 0x55c854cffe60 2024-04-16T05:45:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.133+0000 7fe9021e4640 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 0x55c854d21070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.134+0000 7fe9019e3640 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d08bb0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8010800 tx=0x7fe8f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d85810db9b95f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.134+0000 7fe8e77fe640 1 -- 172.21.15.86:0/1083745336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8026b50 con 0x55c854cffe60 2024-04-16T05:45:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.134+0000 7fe8e77fe640 1 -- 172.21.15.86:0/1083745336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f8026cf0 con 0x55c854cffe60 2024-04-16T05:45:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.134+0000 7fe8e77fe640 1 -- 172.21.15.86:0/1083745336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f801fa00 con 0x55c854cffe60 2024-04-16T05:45:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 -- 172.21.15.86:0/1083745336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 msgr2=0x55c854d08bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d08bb0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8010800 tx=0x7fe8f8016000 comp rx=0 tx=0).stop 2024-04-16T05:45:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 -- 172.21.15.86:0/1083745336 shutdown_connections 2024-04-16T05:45:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 0x55c854d21070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d00b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 --2- 172.21.15.86:0/1083745336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d08bb0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 -- 172.21.15.86:0/1083745336 >> 172.21.15.86:0/1083745336 conn(0x55c854cfa590 msgr2=0x55c854cfc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 -- 172.21.15.86:0/1083745336 shutdown_connections 2024-04-16T05:45:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 -- 172.21.15.86:0/1083745336 wait complete. 2024-04-16T05:45:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 -- start start 2024-04-16T05:45:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.135+0000 7fe90682f740 1 Processor -- start 2024-04-16T05:45:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe90682f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d11490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe90682f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe90682f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 0x55c854d0e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe90682f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1b0a10 con 0x55c854cffe60 2024-04-16T05:45:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe90682f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1b0340 con 0x55c854d00780 2024-04-16T05:45:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe90682f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1ad5a0 con 0x55c854d09360 2024-04-16T05:45:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe90682f740 10 client.0 _ll_register_callbacks cb 0x7fe8ec00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9019e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d11490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9019e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d11490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48560/0 (socket says 172.21.15.86:48560) 2024-04-16T05:45:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9019e3640 1 -- 172.21.15.86:0/670594910 learned_addr learned my addr 172.21.15.86:0/670594910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9011e2640 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9021e4640 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 0x55c854d0e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9019e3640 1 -- 172.21.15.86:0/670594910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 msgr2=0x55c854d0e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9019e3640 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 0x55c854d0e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9019e3640 1 -- 172.21.15.86:0/670594910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 msgr2=0x55c854d119d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9019e3640 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d119d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9019e3640 1 -- 172.21.15.86:0/670594910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c854d0ed50 con 0x55c854cffe60 2024-04-16T05:45:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9011e2640 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d119d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe9019e3640 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d11490 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8026b60 tx=0x7fe8f8026b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1934dbf355d58a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe8e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe8e57fa640 1 -- 172.21.15.86:0/670594910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8026760 con 0x55c854cffe60 2024-04-16T05:45:43.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe8e57fa640 1 -- 172.21.15.86:0/670594910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe8f801f600 con 0x55c854cffe60 2024-04-16T05:45:43.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.136+0000 7fe90682f740 1 -- 172.21.15.86:0/670594910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f8026cf0 con 0x55c854cffe60 2024-04-16T05:45:43.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe8e57fa640 1 -- 172.21.15.86:0/670594910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8f8041050 con 0x55c854cffe60 2024-04-16T05:45:43.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe90682f740 10 client.18678 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:43.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe90682f740 1 -- 172.21.15.86:0/670594910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c854d80440 con 0x55c854cffe60 2024-04-16T05:45:43.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe8e57fa640 1 -- 172.21.15.86:0/670594910 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f8026cf0 con 0x55c854cffe60 2024-04-16T05:45:43.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe8e57fa640 1 -- 172.21.15.86:0/670594910 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe8f800f040 con 0x55c854cffe60 2024-04-16T05:45:43.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe90682f740 10 client.18678 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:43.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe90682f740 10 client.18678 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-16T05:45:43.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe90682f740 20 client.18678 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:43.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe90682f740 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c854d80aa0 0x55c854d82ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe90682f740 4 client.18678 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:43.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe90682f740 1 -- 172.21.15.86:0/670594910 --> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c854cf5c60 con 0x55c854d80aa0 2024-04-16T05:45:43.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.137+0000 7fe9011e2640 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c854d80aa0 0x55c854d82ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe9011e2640 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c854d80aa0 0x55c854d82ec0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=92f084880d5a4994 server_cookie=2ea29cafb12fcc5d in_seq=0 out_seq=0 2024-04-16T05:45:43.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe8e57fa640 10 client.18678 ms_handle_connect on v2:172.21.15.194:6836/3400155977 2024-04-16T05:45:43.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe8e57fa640 1 -- 172.21.15.86:0/670594910 <== mds.3 v2:172.21.15.194:6836/3400155977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c854cf5c60 con 0x55c854d80aa0 2024-04-16T05:45:43.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe8e57fa640 10 client.18678 handle_command_reply: tid=0 2024-04-16T05:45:43.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 client.18678 shutdown 2024-04-16T05:45:43.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 -- 172.21.15.86:0/670594910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 msgr2=0x55c854d11490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d11490 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8026b60 tx=0x7fe8f8026b90 comp rx=0 tx=0).stop 2024-04-16T05:45:43.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 -- 172.21.15.86:0/670594910 shutdown_connections 2024-04-16T05:45:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.194:6836/3400155977,v1:172.21.15.194:6837/3400155977] conn(0x55c854d80aa0 0x55c854d82ec0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c854d09360 0x55c854d0e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c854d00780 0x55c854d119d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 --2- 172.21.15.86:0/670594910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c854cffe60 0x55c854d11490 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 -- 172.21.15.86:0/670594910 >> 172.21.15.86:0/670594910 conn(0x55c854cfa590 msgr2=0x55c854d06440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe8e57fa640 0 client.18678 ms_handle_reset on v2:172.21.15.194:6836/3400155977 2024-04-16T05:45:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 -- 172.21.15.86:0/670594910 shutdown_connections 2024-04-16T05:45:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 1 -- 172.21.15.86:0/670594910 wait complete. 2024-04-16T05:45:43.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.138+0000 7fe90682f740 20 client.18678 trim_cache size 0 max 16384 2024-04-16T05:45:43.152 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 -- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe8e80a3da0 msgr2=0x7fe8e80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe8e80a3da0 0x7fe8e80a6260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009b50 tx=0x7fe8f8005f90 comp rx=0 tx=0).stop 2024-04-16T05:45:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 -- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 msgr2=0x7fe8fc19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 0x7fe8fc19eb20 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00d770 tx=0x7fe8ec00dc30 comp rx=0 tx=0).stop 2024-04-16T05:45:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 -- 172.21.15.86:0/1638943646 shutdown_connections 2024-04-16T05:45:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe8e80a3da0 0x7fe8e80a6260 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8fc1986a0 0x7fe8fc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc106200 0x7fe8fc19eb20 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 --2- 172.21.15.86:0/1638943646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc101130 0x7fe8fc19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 -- 172.21.15.86:0/1638943646 >> 172.21.15.86:0/1638943646 conn(0x7fe8fc0780b0 msgr2=0x7fe8fc069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 -- 172.21.15.86:0/1638943646 shutdown_connections 2024-04-16T05:45:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.139+0000 7fe903c47640 1 -- 172.21.15.86:0/1638943646 wait complete. 2024-04-16T05:45:43.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:45:43.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:42 smithi194 ceph-mon[35865]: pgmap v323: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:43.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:42 smithi196 ceph-mon[36373]: pgmap v323: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.266+0000 7fe6efd09640 1 -- 172.21.15.86:0/2715819630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8106200 msgr2=0x7fe6e810ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.266+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2715819630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8106200 0x7fe6e810ddd0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009ca0 tx=0x7fe6e002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.266+0000 7fe6efd09640 1 -- 172.21.15.86:0/2715819630 shutdown_connections 2024-04-16T05:45:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.266+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2715819630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8106200 0x7fe6e810ddd0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.266+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2715819630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6e8101ae0 0x7fe6e8105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.266+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2715819630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6e8101130 0x7fe6e8101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.266+0000 7fe6efd09640 1 -- 172.21.15.86:0/2715819630 >> 172.21.15.86:0/2715819630 conn(0x7fe6e80780b0 msgr2=0x7fe6e80784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.266+0000 7fe6efd09640 1 -- 172.21.15.86:0/2715819630 shutdown_connections 2024-04-16T05:45:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.266+0000 7fe6efd09640 1 -- 172.21.15.86:0/2715819630 wait complete. 2024-04-16T05:45:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6efd09640 1 Processor -- start 2024-04-16T05:45:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6efd09640 1 -- start start 2024-04-16T05:45:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6efd09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8101130 0x7fe6e80fecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6efd09640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6e8106200 0x7fe6e80ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6efd09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6e80ff740 0x7fe6e81a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6efd09640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81112c0 con 0x7fe6e8101130 2024-04-16T05:45:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6efd09640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8111140 con 0x7fe6e8106200 2024-04-16T05:45:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6efd09640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8111440 con 0x7fe6e80ff740 2024-04-16T05:45:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6edaa5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8101130 0x7fe6e80fecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6edaa5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8101130 0x7fe6e80fecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48578/0 (socket says 172.21.15.86:48578) 2024-04-16T05:45:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6edaa5640 1 -- 172.21.15.86:0/2981214425 learned_addr learned my addr 172.21.15.86:0/2981214425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6e8106200 0x7fe6e80ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6edaa5640 1 -- 172.21.15.86:0/2981214425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6e80ff740 msgr2=0x7fe6e81a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6e80ff740 0x7fe6e81a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6edaa5640 1 -- 172.21.15.86:0/2981214425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6e8106200 msgr2=0x7fe6e80ff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6e8106200 0x7fe6e80ff200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.267+0000 7fe6edaa5640 1 -- 172.21.15.86:0/2981214425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e81a7140 con 0x7fe6e8101130 2024-04-16T05:45:43.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.268+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8101130 0x7fe6e80fecc0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400c970 tx=0x7fe6e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.268+0000 7fe6deffd640 1 -- 172.21.15.86:0/2981214425 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4007bb0 con 0x7fe6e8101130 2024-04-16T05:45:43.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.268+0000 7fe6efd09640 1 -- 172.21.15.86:0/2981214425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e81a7430 con 0x7fe6e8101130 2024-04-16T05:45:43.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.268+0000 7fe6deffd640 1 -- 172.21.15.86:0/2981214425 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e4007d50 con 0x7fe6e8101130 2024-04-16T05:45:43.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.268+0000 7fe6deffd640 1 -- 172.21.15.86:0/2981214425 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400f450 con 0x7fe6e8101130 2024-04-16T05:45:43.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.268+0000 7fe6efd09640 1 -- 172.21.15.86:0/2981214425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e81a7980 con 0x7fe6e8101130 2024-04-16T05:45:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.270+0000 7fe6deffd640 1 -- 172.21.15.86:0/2981214425 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe6e4005020 con 0x7fe6e8101130 2024-04-16T05:45:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6f29c5740 1 Processor -- start 2024-04-16T05:45:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6f29c5740 1 -- start start 2024-04-16T05:45:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e4503b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 0x563e45043e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 0x563e45046750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e44fa00e0 con 0x563e4503ae60 2024-04-16T05:45:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563e44f3d520 con 0x563e45044360 2024-04-16T05:45:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e44f9c0a0 con 0x563e4503b780 2024-04-16T05:45:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ee2a6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e4503b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ee2a6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e4503b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48586/0 (socket says 172.21.15.86:48586) 2024-04-16T05:45:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ee2a6640 1 -- 172.21.15.86:0/2097133541 learned_addr learned my addr 172.21.15.86:0/2097133541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ee2a6640 1 -- 172.21.15.86:0/2097133541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 msgr2=0x563e45043e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 0x563e45043e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 0x563e45046750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6deffd640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6d40a3d50 0x7fe6d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ee2a6640 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 0x563e45043e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ee2a6640 1 -- 172.21.15.86:0/2097133541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 msgr2=0x563e45046750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ee2a6640 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 0x563e45046750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ee2a6640 1 -- 172.21.15.86:0/2097133541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e44f658d0 con 0x563e4503ae60 2024-04-16T05:45:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.271+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 0x563e45046750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.272+0000 7fe6deffd640 1 -- 172.21.15.86:0/2981214425 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe6e40d0a40 con 0x7fe6e8101130 2024-04-16T05:45:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.272+0000 7fe6ee2a6640 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e4503b240 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009dc0 tx=0x7fe6e00324a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea732da4b5aa21ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.272+0000 7fe6ee2a6640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6d40a3d50 0x7fe6d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.272+0000 7fe6d3fff640 1 -- 172.21.15.86:0/2097133541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0041020 con 0x563e4503ae60 2024-04-16T05:45:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.272+0000 7fe6d3fff640 1 -- 172.21.15.86:0/2097133541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e0003970 con 0x563e4503ae60 2024-04-16T05:45:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.272+0000 7fe6d3fff640 1 -- 172.21.15.86:0/2097133541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0039040 con 0x563e4503ae60 2024-04-16T05:45:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2097133541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 msgr2=0x563e4503b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6ee2a6640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6d40a3d50 0x7fe6d40a6210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0032530 tx=0x7fe6e0049040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e4503b240 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009dc0 tx=0x7fe6e00324a0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2097133541 shutdown_connections 2024-04-16T05:45:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 0x563e45046750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 0x563e45043e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/2097133541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e4503b240 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2097133541 >> 172.21.15.86:0/2097133541 conn(0x563e45035590 msgr2=0x563e45035970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2097133541 shutdown_connections 2024-04-16T05:45:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2097133541 wait complete. 2024-04-16T05:45:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 -- start start 2024-04-16T05:45:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.273+0000 7fe6f29c5740 1 Processor -- start 2024-04-16T05:45:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e45041400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503b780 0x563e45041940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e45044360 0x563e45041e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e45047000 con 0x563e4503ae60 2024-04-16T05:45:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563e44f594d0 con 0x563e4503b780 2024-04-16T05:45:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e45047180 con 0x563e45044360 2024-04-16T05:45:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6f29c5740 10 client.0 _ll_register_callbacks cb 0x563e44f3e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6edaa5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e45041400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6edaa5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e45041400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48598/0 (socket says 172.21.15.86:48598) 2024-04-16T05:45:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6edaa5640 1 -- 172.21.15.86:0/906464188 learned_addr learned my addr 172.21.15.86:0/906464188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503b780 0x563e45041940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6ee2a6640 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e45044360 0x563e45041e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6edaa5640 1 -- 172.21.15.86:0/906464188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e45044360 msgr2=0x563e45041e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e45044360 0x563e45041e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6edaa5640 1 -- 172.21.15.86:0/906464188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503b780 msgr2=0x563e45041940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503b780 0x563e45041940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6edaa5640 1 -- 172.21.15.86:0/906464188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e44cb23f0 con 0x563e4503ae60 2024-04-16T05:45:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503b780 0x563e45041940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e45041400 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40cde20 tx=0x7fe6e4098280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f1ce27126646c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6d17fa640 1 -- 172.21.15.86:0/906464188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e401d040 con 0x563e4503ae60 2024-04-16T05:45:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6d17fa640 1 -- 172.21.15.86:0/906464188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe6e4005310 con 0x563e4503ae60 2024-04-16T05:45:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.274+0000 7fe6f29c5740 1 -- 172.21.15.86:0/906464188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e0003970 con 0x563e4503ae60 2024-04-16T05:45:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6d17fa640 1 -- 172.21.15.86:0/906464188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6e409cd10 con 0x563e4503ae60 2024-04-16T05:45:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6f29c5740 10 client.18690 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6f29c5740 1 -- 172.21.15.86:0/906464188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e44d7c7d0 con 0x563e4503ae60 2024-04-16T05:45:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6d17fa640 1 -- 172.21.15.86:0/906464188 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e0003970 con 0x563e4503ae60 2024-04-16T05:45:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6d17fa640 1 -- 172.21.15.86:0/906464188 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe6e40bb2b0 con 0x563e4503ae60 2024-04-16T05:45:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6f29c5740 10 client.18690 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6f29c5740 10 client.18690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:45:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6f29c5740 20 client.18690 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x563e450541d0 0x563e45056640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6f29c5740 4 client.18690 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6f29c5740 1 -- 172.21.15.86:0/906464188 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e44fadb00 con 0x563e450541d0 2024-04-16T05:45:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.275+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x563e450541d0 0x563e45056640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.276+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x563e450541d0 0x563e45056640 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16ceab3c595dce09 server_cookie=4dcecaef4fa08e24 in_seq=0 out_seq=0 2024-04-16T05:45:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.276+0000 7fe6d17fa640 10 client.18690 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:45:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.276+0000 7fe6d17fa640 1 -- 172.21.15.86:0/906464188 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e44fadb00 con 0x563e450541d0 2024-04-16T05:45:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.276+0000 7fe6d17fa640 10 client.18690 handle_command_reply: tid=0 2024-04-16T05:45:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.276+0000 7fe6f29c5740 1 client.18690 shutdown 2024-04-16T05:45:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 -- 172.21.15.86:0/906464188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 msgr2=0x563e45041400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e45041400 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40cde20 tx=0x7fe6e4098280 comp rx=0 tx=0).stop 2024-04-16T05:45:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 -- 172.21.15.86:0/906464188 shutdown_connections 2024-04-16T05:45:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e45044360 0x563e45041e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503b780 0x563e45041940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x563e450541d0 0x563e45056640 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/906464188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503ae60 0x563e45041400 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6d17fa640 0 client.18690 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:45:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 -- 172.21.15.86:0/906464188 >> 172.21.15.86:0/906464188 conn(0x563e45035590 msgr2=0x563e450372e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 -- 172.21.15.86:0/906464188 shutdown_connections 2024-04-16T05:45:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 1 -- 172.21.15.86:0/906464188 wait complete. 2024-04-16T05:45:43.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.277+0000 7fe6f29c5740 20 client.18690 trim_cache size 0 max 16384 2024-04-16T05:45:43.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6f29c5740 1 Processor -- start 2024-04-16T05:45:43.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6f29c5740 1 -- start start 2024-04-16T05:45:43.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503ae60 0x563e45043b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503b780 0x563e4504ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 0x563e4505bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81b16b0 con 0x563e4503b780 2024-04-16T05:45:43.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81b31f0 con 0x563e45044360 2024-04-16T05:45:43.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81b1d80 con 0x563e4503ae60 2024-04-16T05:45:43.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6ed2a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503b780 0x563e4504ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6ed2a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503b780 0x563e4504ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48602/0 (socket says 172.21.15.86:48602) 2024-04-16T05:45:43.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6ed2a4640 1 -- 172.21.15.86:0/2836371772 learned_addr learned my addr 172.21.15.86:0/2836371772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/2836371772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503ae60 0x563e45043b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6ed2a4640 1 -- 172.21.15.86:0/2836371772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503ae60 msgr2=0x563e45043b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/2836371772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503ae60 0x563e45043b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6ed2a4640 1 -- 172.21.15.86:0/2836371772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 msgr2=0x563e4505bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/2836371772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 0x563e4505bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6ed2a4640 1 -- 172.21.15.86:0/2836371772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e450350c0 con 0x563e4503b780 2024-04-16T05:45:43.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.290+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/2836371772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503ae60 0x563e45043b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.291+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/2836371772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503b780 0x563e4504ffa0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800dc10 tx=0x7fe6d800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b427c6c26f5eec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.291+0000 7fe6d37fe640 1 -- 172.21.15.86:0/2836371772 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d801f070 con 0x563e4503b780 2024-04-16T05:45:43.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.291+0000 7fe6d37fe640 1 -- 172.21.15.86:0/2836371772 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d801b880 con 0x563e4503b780 2024-04-16T05:45:43.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.291+0000 7fe6d37fe640 1 -- 172.21.15.86:0/2836371772 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8005530 con 0x563e4503b780 2024-04-16T05:45:43.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.291+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2836371772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503b780 msgr2=0x563e4504ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.291+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/2836371772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503b780 0x563e4504ffa0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800dc10 tx=0x7fe6d800d340 comp rx=0 tx=0).stop 2024-04-16T05:45:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2836371772 shutdown_connections 2024-04-16T05:45:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/2836371772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e45044360 0x563e4505bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/2836371772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e4503b780 0x563e4504ffa0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/2836371772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503ae60 0x563e45043b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2836371772 >> 172.21.15.86:0/2836371772 conn(0x563e45035590 msgr2=0x563e45034730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2836371772 shutdown_connections 2024-04-16T05:45:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 -- 172.21.15.86:0/2836371772 wait complete. 2024-04-16T05:45:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 -- start start 2024-04-16T05:45:43.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 Processor -- start 2024-04-16T05:45:43.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503ae60 0x563e45048630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 0x563e45048b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e45044360 0x563e450b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81b2b20 con 0x563e45044360 2024-04-16T05:45:43.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81b2450 con 0x563e4503ae60 2024-04-16T05:45:43.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81af6b0 con 0x563e4503b780 2024-04-16T05:45:43.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6f29c5740 10 client.0 _ll_register_callbacks cb 0x7fe6e40058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:43.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6ee2a6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e45044360 0x563e450b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6ee2a6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e45044360 0x563e450b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48616/0 (socket says 172.21.15.86:48616) 2024-04-16T05:45:43.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.292+0000 7fe6ee2a6640 1 -- 172.21.15.86:0/3440515230 learned_addr learned my addr 172.21.15.86:0/3440515230 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 0x563e45048b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503ae60 0x563e45048630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6ee2a6640 1 -- 172.21.15.86:0/3440515230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 msgr2=0x563e45048b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6ee2a6640 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 0x563e45048b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6ee2a6640 1 -- 172.21.15.86:0/3440515230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503ae60 msgr2=0x563e45048630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6ee2a6640 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503ae60 0x563e45048630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6ee2a6640 1 -- 172.21.15.86:0/3440515230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e450b9cf0 con 0x563e45044360 2024-04-16T05:45:43.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503ae60 0x563e45048630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6ed2a4640 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 0x563e45048b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6ee2a6640 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e45044360 0x563e450b9780 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0000f80 tx=0x7fe6e0004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1d2fb2addb33f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:43.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6d17fa640 1 -- 172.21.15.86:0/3440515230 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0041050 con 0x563e45044360 2024-04-16T05:45:43.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6d17fa640 1 -- 172.21.15.86:0/3440515230 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe6e00327a0 con 0x563e45044360 2024-04-16T05:45:43.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6f29c5740 1 -- 172.21.15.86:0/3440515230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d801b880 con 0x563e45044360 2024-04-16T05:45:43.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6d17fa640 1 -- 172.21.15.86:0/3440515230 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6e00329c0 con 0x563e45044360 2024-04-16T05:45:43.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6f29c5740 10 client.18696 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:43.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6f29c5740 1 -- 172.21.15.86:0/3440515230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e450baf90 con 0x563e45044360 2024-04-16T05:45:43.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.293+0000 7fe6d17fa640 1 -- 172.21.15.86:0/3440515230 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d801b880 con 0x563e45044360 2024-04-16T05:45:43.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6d17fa640 1 -- 172.21.15.86:0/3440515230 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe6e003c030 con 0x563e45044360 2024-04-16T05:45:43.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6f29c5740 10 client.18696 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:43.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6f29c5740 10 client.18696 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-16T05:45:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6f29c5740 20 client.18696 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x563e450bb3d0 0x563e450bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6f29c5740 4 client.18696 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6f29c5740 1 -- 172.21.15.86:0/3440515230 --> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e44fa7970 con 0x563e450bb3d0 2024-04-16T05:45:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x563e450bb3d0 0x563e450bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6d17fa640 10 client.18696 ms_handle_connect on v2:172.21.15.196:6836/2445566344 2024-04-16T05:45:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6edaa5640 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x563e450bb3d0 0x563e450bd7c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebe16ee50aed0149 server_cookie=8211879e9179a047 in_seq=0 out_seq=0 2024-04-16T05:45:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6d17fa640 1 -- 172.21.15.86:0/3440515230 <== mds.2 v2:172.21.15.196:6836/2445566344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e44fa7970 con 0x563e450bb3d0 2024-04-16T05:45:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6d17fa640 10 client.18696 handle_command_reply: tid=0 2024-04-16T05:45:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.294+0000 7fe6f29c5740 1 client.18696 shutdown 2024-04-16T05:45:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 -- 172.21.15.86:0/3440515230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e45044360 msgr2=0x563e450b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e45044360 0x563e450b9780 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0000f80 tx=0x7fe6e0004810 comp rx=0 tx=0).stop 2024-04-16T05:45:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 -- 172.21.15.86:0/3440515230 shutdown_connections 2024-04-16T05:45:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.196:6836/2445566344,v1:172.21.15.196:6837/2445566344] conn(0x563e450bb3d0 0x563e450bd7c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e45044360 0x563e450b9780 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e4503b780 0x563e45048b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6d17fa640 0 client.18696 ms_handle_reset on v2:172.21.15.196:6836/2445566344 2024-04-16T05:45:43.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 --2- 172.21.15.86:0/3440515230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e4503ae60 0x563e45048630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 -- 172.21.15.86:0/3440515230 >> 172.21.15.86:0/3440515230 conn(0x563e45035590 msgr2=0x563e45033ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 -- 172.21.15.86:0/3440515230 shutdown_connections 2024-04-16T05:45:43.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 1 -- 172.21.15.86:0/3440515230 wait complete. 2024-04-16T05:45:43.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6f29c5740 20 client.18696 trim_cache size 0 max 16384 2024-04-16T05:45:43.305 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:43.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 -- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6d40a3d50 msgr2=0x7fe6d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6d40a3d50 0x7fe6d40a6210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0032530 tx=0x7fe6e0049040 comp rx=0 tx=0).stop 2024-04-16T05:45:43.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 -- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8101130 msgr2=0x7fe6e80fecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8101130 0x7fe6e80fecc0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400c970 tx=0x7fe6e400ce30 comp rx=0 tx=0).stop 2024-04-16T05:45:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 -- 172.21.15.86:0/2981214425 shutdown_connections 2024-04-16T05:45:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6e80ff740 0x7fe6e81a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fe6d40a3d50 0x7fe6d40a6210 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6e8106200 0x7fe6e80ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 --2- 172.21.15.86:0/2981214425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8101130 0x7fe6e80fecc0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 -- 172.21.15.86:0/2981214425 >> 172.21.15.86:0/2981214425 conn(0x7fe6e80780b0 msgr2=0x7fe6e80693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.295+0000 7fe6efd09640 1 -- 172.21.15.86:0/2981214425 shutdown_connections 2024-04-16T05:45:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.296+0000 7fe6efd09640 1 -- 172.21.15.86:0/2981214425 wait complete. 2024-04-16T05:45:43.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:45:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.420+0000 7fdc755b5640 1 -- 172.21.15.86:0/2468120148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc70102260 msgr2=0x7fdc70105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.420+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2468120148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc70102260 0x7fdc70105b70 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009900 tx=0x7fdc5802bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.420+0000 7fdc755b5640 1 -- 172.21.15.86:0/2468120148 shutdown_connections 2024-04-16T05:45:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.420+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2468120148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc701062a0 0x7fdc7010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.420+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2468120148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc70102260 0x7fdc70105b70 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.420+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2468120148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc701018b0 0x7fdc70101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.420+0000 7fdc755b5640 1 -- 172.21.15.86:0/2468120148 >> 172.21.15.86:0/2468120148 conn(0x7fdc70077410 msgr2=0x7fdc700fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.420+0000 7fdc755b5640 1 -- 172.21.15.86:0/2468120148 shutdown_connections 2024-04-16T05:45:43.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.420+0000 7fdc755b5640 1 -- 172.21.15.86:0/2468120148 wait complete. 2024-04-16T05:45:43.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc755b5640 1 Processor -- start 2024-04-16T05:45:43.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc755b5640 1 -- start start 2024-04-16T05:45:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc755b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc701018b0 0x7fdc70199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc755b5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc701062a0 0x7fdc70199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc755b5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc7019a250 0x7fdc701a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc755b5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701112c0 con 0x7fdc701018b0 2024-04-16T05:45:43.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc755b5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70111140 con 0x7fdc701062a0 2024-04-16T05:45:43.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc755b5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70111440 con 0x7fdc7019a250 2024-04-16T05:45:43.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc701018b0 0x7fdc70199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc701018b0 0x7fdc70199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48646/0 (socket says 172.21.15.86:48646) 2024-04-16T05:45:43.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6effd640 1 -- 172.21.15.86:0/2864602635 learned_addr learned my addr 172.21.15.86:0/2864602635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc67fff640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc701062a0 0x7fdc70199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc7019a250 0x7fdc701a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6effd640 1 -- 172.21.15.86:0/2864602635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc7019a250 msgr2=0x7fdc701a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6effd640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc7019a250 0x7fdc701a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6effd640 1 -- 172.21.15.86:0/2864602635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc701062a0 msgr2=0x7fdc70199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6effd640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc701062a0 0x7fdc70199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6effd640 1 -- 172.21.15.86:0/2864602635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc701a4fd0 con 0x7fdc701018b0 2024-04-16T05:45:43.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc67fff640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc701062a0 0x7fdc70199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:43.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.421+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc7019a250 0x7fdc701a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.422+0000 7fdc6effd640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc701018b0 0x7fdc70199780 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e9f0 tx=0x7fdc6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.422+0000 7fdc6cff9640 1 -- 172.21.15.86:0/2864602635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000cd80 con 0x7fdc701018b0 2024-04-16T05:45:43.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.422+0000 7fdc755b5640 1 -- 172.21.15.86:0/2864602635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc701a52c0 con 0x7fdc701018b0 2024-04-16T05:45:43.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.422+0000 7fdc6cff9640 1 -- 172.21.15.86:0/2864602635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc60005500 con 0x7fdc701018b0 2024-04-16T05:45:43.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.422+0000 7fdc6cff9640 1 -- 172.21.15.86:0/2864602635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60010690 con 0x7fdc701018b0 2024-04-16T05:45:43.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.422+0000 7fdc755b5640 1 -- 172.21.15.86:0/2864602635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc701a57e0 con 0x7fdc701018b0 2024-04-16T05:45:43.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.424+0000 7fdc6cff9640 1 -- 172.21.15.86:0/2864602635 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc600046c0 con 0x7fdc701018b0 2024-04-16T05:45:43.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.424+0000 7fdc78195740 1 Processor -- start 2024-04-16T05:45:43.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.424+0000 7fdc78195740 1 -- start start 2024-04-16T05:45:43.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc78195740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc0cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc78195740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 0x55e4fdc1a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc78195740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 0x55e4fdc1cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc78195740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e4fdb720e0 con 0x55e4fdc0cb10 2024-04-16T05:45:43.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc78195740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e4fdb0f520 con 0x55e4fdc0d430 2024-04-16T05:45:43.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc78195740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4fdb6e0a0 con 0x55e4fdc1a770 2024-04-16T05:45:43.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc0cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc0cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48648/0 (socket says 172.21.15.86:48648) 2024-04-16T05:45:43.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/2239087408 learned_addr learned my addr 172.21.15.86:0/2239087408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6effd640 1 --2- 172.21.15.86:0/2239087408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 0x55e4fdc1a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc67fff640 1 --2- 172.21.15.86:0/2239087408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 0x55e4fdc1cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6cff9640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc5c0a3d20 0x7fdc5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/2239087408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 msgr2=0x55e4fdc1cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/2239087408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 0x55e4fdc1cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/2239087408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 msgr2=0x55e4fdc1a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/2239087408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 0x55e4fdc1a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/2239087408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4fdb378d0 con 0x55e4fdc0cb10 2024-04-16T05:45:43.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.425+0000 7fdc6cff9640 1 -- 172.21.15.86:0/2864602635 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fdc600d1140 con 0x7fdc701018b0 2024-04-16T05:45:43.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.426+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc5c0a3d20 0x7fdc5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.426+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/2239087408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc0cef0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009fa0 tx=0x7fdc6800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f54be036bdf002 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.426+0000 7fdc657fa640 1 -- 172.21.15.86:0/2239087408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6801a610 con 0x55e4fdc0cb10 2024-04-16T05:45:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.426+0000 7fdc657fa640 1 -- 172.21.15.86:0/2239087408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6801ac90 con 0x55e4fdc0cb10 2024-04-16T05:45:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.426+0000 7fdc657fa640 1 -- 172.21.15.86:0/2239087408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68010910 con 0x55e4fdc0cb10 2024-04-16T05:45:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 -- 172.21.15.86:0/2239087408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 msgr2=0x55e4fdc0cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 --2- 172.21.15.86:0/2239087408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc0cef0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009fa0 tx=0x7fdc6800f7f0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc5c0a3d20 0x7fdc5c0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdc68020000 tx=0x7fdc6801d000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 -- 172.21.15.86:0/2239087408 shutdown_connections 2024-04-16T05:45:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 --2- 172.21.15.86:0/2239087408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 0x55e4fdc1cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 --2- 172.21.15.86:0/2239087408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 0x55e4fdc1a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 --2- 172.21.15.86:0/2239087408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc0cef0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 -- 172.21.15.86:0/2239087408 >> 172.21.15.86:0/2239087408 conn(0x55e4fdc07590 msgr2=0x55e4fdc07970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 -- 172.21.15.86:0/2239087408 shutdown_connections 2024-04-16T05:45:43.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 -- 172.21.15.86:0/2239087408 wait complete. 2024-04-16T05:45:43.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 -- start start 2024-04-16T05:45:43.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 Processor -- start 2024-04-16T05:45:43.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 0x55e4fdc17ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 0x55e4fdc18220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e4fdc1d5b0 con 0x55e4fdc0cb10 2024-04-16T05:45:43.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e4fdb2b4d0 con 0x55e4fdc0d430 2024-04-16T05:45:43.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4fdc1d730 con 0x55e4fdc1a770 2024-04-16T05:45:43.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.427+0000 7fdc78195740 10 client.0 _ll_register_callbacks cb 0x55e4fdb102c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:43.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc177a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48654/0 (socket says 172.21.15.86:48654) 2024-04-16T05:45:43.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6effd640 1 -- 172.21.15.86:0/22767605 learned_addr learned my addr 172.21.15.86:0/22767605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 0x55e4fdc18220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc67fff640 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 0x55e4fdc17ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6effd640 1 -- 172.21.15.86:0/22767605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 msgr2=0x55e4fdc18220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6effd640 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 0x55e4fdc18220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6effd640 1 -- 172.21.15.86:0/22767605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 msgr2=0x55e4fdc17ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6effd640 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 0x55e4fdc17ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6effd640 1 -- 172.21.15.86:0/22767605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4fd8843f0 con 0x55e4fdc0cb10 2024-04-16T05:45:43.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc6effd640 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc177a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c6a0 tx=0x7fdc600056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958b50e684adbb11 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc567fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc567fc640 1 -- 172.21.15.86:0/22767605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6009c850 con 0x55e4fdc0cb10 2024-04-16T05:45:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc567fc640 1 -- 172.21.15.86:0/22767605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fdc60099380 con 0x55e4fdc0cb10 2024-04-16T05:45:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc78195740 1 -- 172.21.15.86:0/22767605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc6801ac90 con 0x55e4fdc0cb10 2024-04-16T05:45:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc567fc640 1 -- 172.21.15.86:0/22767605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc6009c670 con 0x55e4fdc0cb10 2024-04-16T05:45:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc78195740 10 client.18708 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc78195740 1 -- 172.21.15.86:0/22767605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4fd94e7d0 con 0x55e4fdc0cb10 2024-04-16T05:45:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.428+0000 7fdc567fc640 1 -- 172.21.15.86:0/22767605 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6801ac90 con 0x55e4fdc0cb10 2024-04-16T05:45:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc567fc640 1 -- 172.21.15.86:0/22767605 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc600b60d0 con 0x55e4fdc0cb10 2024-04-16T05:45:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc78195740 10 client.18708 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc78195740 10 client.18708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:45:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc78195740 20 client.18708 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc78195740 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e4fdc2a820 0x55e4fdc2cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc78195740 4 client.18708 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc78195740 1 -- 172.21.15.86:0/22767605 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4fdb7fb00 con 0x55e4fdc2a820 2024-04-16T05:45:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc67fff640 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e4fdc2a820 0x55e4fdc2cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc67fff640 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e4fdc2a820 0x55e4fdc2cc10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5834386937f7ac12 server_cookie=c9a13358c5e28064 in_seq=0 out_seq=0 2024-04-16T05:45:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.429+0000 7fdc567fc640 10 client.18708 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:45:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc567fc640 1 -- 172.21.15.86:0/22767605 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4fdb7fb00 con 0x55e4fdc2a820 2024-04-16T05:45:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc567fc640 10 client.18708 handle_command_reply: tid=0 2024-04-16T05:45:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 client.18708 shutdown 2024-04-16T05:45:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 -- 172.21.15.86:0/22767605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 msgr2=0x55e4fdc177a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc177a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c6a0 tx=0x7fdc600056b0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 -- 172.21.15.86:0/22767605 shutdown_connections 2024-04-16T05:45:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e4fdc2a820 0x55e4fdc2cc10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc567fc640 0 client.18708 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:45:43.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc1a770 0x55e4fdc18220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0d430 0x55e4fdc17ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 --2- 172.21.15.86:0/22767605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc177a0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 -- 172.21.15.86:0/22767605 >> 172.21.15.86:0/22767605 conn(0x55e4fdc07590 msgr2=0x55e4fdc092e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 -- 172.21.15.86:0/22767605 shutdown_connections 2024-04-16T05:45:43.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.430+0000 7fdc78195740 1 -- 172.21.15.86:0/22767605 wait complete. 2024-04-16T05:45:43.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.431+0000 7fdc78195740 20 client.18708 trim_cache size 0 max 16384 2024-04-16T05:45:43.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc78195740 1 Processor -- start 2024-04-16T05:45:43.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc78195740 1 -- start start 2024-04-16T05:45:43.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc78195740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc19e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc78195740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 0x55e4fdc264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc78195740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc78195740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ad400 con 0x55e4fdc1a770 2024-04-16T05:45:43.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc78195740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701aef40 con 0x55e4fdc0cb10 2024-04-16T05:45:43.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc78195740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701adad0 con 0x55e4fdc0d430 2024-04-16T05:45:43.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48656/0 (socket says 172.21.15.86:48656) 2024-04-16T05:45:43.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/3626595356 learned_addr learned my addr 172.21.15.86:0/3626595356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6effd640 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc19e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc67fff640 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 0x55e4fdc264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/3626595356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 msgr2=0x55e4fdc264e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 0x55e4fdc264e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/3626595356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 msgr2=0x55e4fdc19e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc19e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/3626595356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4fdc0c5e0 con 0x55e4fdc1a770 2024-04-16T05:45:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc6effd640 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc19e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.444+0000 7fdc67fff640 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 0x55e4fdc264e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.445+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc090f0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdc68005ba0 tx=0x7fdc680105f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bace4673901b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.445+0000 7fdc64ff9640 1 -- 172.21.15.86:0/3626595356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68032070 con 0x55e4fdc1a770 2024-04-16T05:45:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.445+0000 7fdc64ff9640 1 -- 172.21.15.86:0/3626595356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc68029040 con 0x55e4fdc1a770 2024-04-16T05:45:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.445+0000 7fdc64ff9640 1 -- 172.21.15.86:0/3626595356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc680108c0 con 0x55e4fdc1a770 2024-04-16T05:45:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 -- 172.21.15.86:0/3626595356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 msgr2=0x55e4fdc090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc090f0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdc68005ba0 tx=0x7fdc680105f0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 -- 172.21.15.86:0/3626595356 shutdown_connections 2024-04-16T05:45:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc090f0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 0x55e4fdc264e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 --2- 172.21.15.86:0/3626595356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc19e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 -- 172.21.15.86:0/3626595356 >> 172.21.15.86:0/3626595356 conn(0x55e4fdc07590 msgr2=0x55e4fdc089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 -- 172.21.15.86:0/3626595356 shutdown_connections 2024-04-16T05:45:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 -- 172.21.15.86:0/3626595356 wait complete. 2024-04-16T05:45:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 -- start start 2024-04-16T05:45:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 Processor -- start 2024-04-16T05:45:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc1e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 0x55e4fdc1ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc8fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ae870 con 0x55e4fdc1a770 2024-04-16T05:45:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ae1a0 con 0x55e4fdc0cb10 2024-04-16T05:45:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ab400 con 0x55e4fdc0d430 2024-04-16T05:45:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc78195740 10 client.0 _ll_register_callbacks cb 0x7fdc60009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.446+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc8fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc8fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48666/0 (socket says 172.21.15.86:48666) 2024-04-16T05:45:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/3047214747 learned_addr learned my addr 172.21.15.86:0/3047214747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc67fff640 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 0x55e4fdc1ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6effd640 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc1e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/3047214747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 msgr2=0x55e4fdc1ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 0x55e4fdc1ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/3047214747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 msgr2=0x55e4fdc1e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc1e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6f7fe640 1 -- 172.21.15.86:0/3047214747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4fdc90260 con 0x55e4fdc1a770 2024-04-16T05:45:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6effd640 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc1e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc6f7fe640 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc8fd20 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fdc6801c010 tx=0x7fdc68010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c912b082afc3953 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc567fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:43.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc567fc640 1 -- 172.21.15.86:0/3047214747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6803b060 con 0x55e4fdc1a770 2024-04-16T05:45:43.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc567fc640 1 -- 172.21.15.86:0/3047214747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fdc68010870 con 0x55e4fdc1a770 2024-04-16T05:45:43.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc78195740 1 -- 172.21.15.86:0/3047214747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc68029040 con 0x55e4fdc1a770 2024-04-16T05:45:43.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc567fc640 1 -- 172.21.15.86:0/3047214747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc68032030 con 0x55e4fdc1a770 2024-04-16T05:45:43.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc78195740 10 client.18714 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:43.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.447+0000 7fdc78195740 1 -- 172.21.15.86:0/3047214747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4fdc91420 con 0x55e4fdc1a770 2024-04-16T05:45:43.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc567fc640 1 -- 172.21.15.86:0/3047214747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc68029040 con 0x55e4fdc1a770 2024-04-16T05:45:43.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc567fc640 1 -- 172.21.15.86:0/3047214747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc68025440 con 0x55e4fdc1a770 2024-04-16T05:45:43.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc78195740 10 client.18714 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:43.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc78195740 10 client.18714 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-16T05:45:43.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc78195740 20 client.18714 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:43.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc78195740 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e4fdc2a820 0x55e4fdc939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc78195740 4 client.18714 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:43.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc78195740 1 -- 172.21.15.86:0/3047214747 --> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4fdb79970 con 0x55e4fdc2a820 2024-04-16T05:45:43.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc6effd640 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e4fdc2a820 0x55e4fdc939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc567fc640 10 client.18714 ms_handle_connect on v2:172.21.15.196:6838/3455974815 2024-04-16T05:45:43.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.448+0000 7fdc6effd640 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e4fdc2a820 0x55e4fdc939c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5b9e22186763ff6 server_cookie=85d103972343a374 in_seq=0 out_seq=0 2024-04-16T05:45:43.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc567fc640 1 -- 172.21.15.86:0/3047214747 <== mds.1 v2:172.21.15.196:6838/3455974815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4fdb79970 con 0x55e4fdc2a820 2024-04-16T05:45:43.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc567fc640 10 client.18714 handle_command_reply: tid=0 2024-04-16T05:45:43.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 client.18714 shutdown 2024-04-16T05:45:43.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 -- 172.21.15.86:0/3047214747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 msgr2=0x55e4fdc8fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc8fd20 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fdc6801c010 tx=0x7fdc68010e80 comp rx=0 tx=0).stop 2024-04-16T05:45:43.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 -- 172.21.15.86:0/3047214747 shutdown_connections 2024-04-16T05:45:43.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.196:6838/3455974815,v1:172.21.15.196:6839/3455974815] conn(0x55e4fdc2a820 0x55e4fdc939c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4fdc1a770 0x55e4fdc8fd20 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc567fc640 0 client.18714 ms_handle_reset on v2:172.21.15.196:6838/3455974815 2024-04-16T05:45:43.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4fdc0d430 0x55e4fdc1ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 --2- 172.21.15.86:0/3047214747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4fdc0cb10 0x55e4fdc1e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 -- 172.21.15.86:0/3047214747 >> 172.21.15.86:0/3047214747 conn(0x55e4fdc07590 msgr2=0x55e4fdc17ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 -- 172.21.15.86:0/3047214747 shutdown_connections 2024-04-16T05:45:43.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 1 -- 172.21.15.86:0/3047214747 wait complete. 2024-04-16T05:45:43.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc78195740 20 client.18714 trim_cache size 0 max 16384 2024-04-16T05:45:43.460 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:43.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 -- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc5c0a3d20 msgr2=0x7fdc5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc5c0a3d20 0x7fdc5c0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdc68020000 tx=0x7fdc6801d000 comp rx=0 tx=0).stop 2024-04-16T05:45:43.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 -- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc701018b0 msgr2=0x7fdc70199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc701018b0 0x7fdc70199780 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e9f0 tx=0x7fdc6000edf0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 -- 172.21.15.86:0/2864602635 shutdown_connections 2024-04-16T05:45:43.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc7019a250 0x7fdc701a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fdc5c0a3d20 0x7fdc5c0a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc701062a0 0x7fdc70199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 --2- 172.21.15.86:0/2864602635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc701018b0 0x7fdc70199780 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.449+0000 7fdc755b5640 1 -- 172.21.15.86:0/2864602635 >> 172.21.15.86:0/2864602635 conn(0x7fdc70077410 msgr2=0x7fdc701044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.450+0000 7fdc755b5640 1 -- 172.21.15.86:0/2864602635 shutdown_connections 2024-04-16T05:45:43.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.450+0000 7fdc755b5640 1 -- 172.21.15.86:0/2864602635 wait complete. 2024-04-16T05:45:43.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:45:43.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.574+0000 7fb88c674640 1 -- 172.21.15.86:0/2965310418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 msgr2=0x7fb88410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.574+0000 7fb88c674640 1 --2- 172.21.15.86:0/2965310418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 0x7fb88410ddd0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009ca0 tx=0x7fb87c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.574+0000 7fb88c674640 1 -- 172.21.15.86:0/2965310418 shutdown_connections 2024-04-16T05:45:43.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.574+0000 7fb88c674640 1 --2- 172.21.15.86:0/2965310418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 0x7fb88410ddd0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.574+0000 7fb88c674640 1 --2- 172.21.15.86:0/2965310418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb884101ae0 0x7fb884105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.574+0000 7fb88c674640 1 --2- 172.21.15.86:0/2965310418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb884101130 0x7fb884101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.574+0000 7fb88c674640 1 -- 172.21.15.86:0/2965310418 >> 172.21.15.86:0/2965310418 conn(0x7fb8840780b0 msgr2=0x7fb8840784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.574+0000 7fb88c674640 1 -- 172.21.15.86:0/2965310418 shutdown_connections 2024-04-16T05:45:43.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.574+0000 7fb88c674640 1 -- 172.21.15.86:0/2965310418 wait complete. 2024-04-16T05:45:43.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb88c674640 1 Processor -- start 2024-04-16T05:45:43.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb88c674640 1 -- start start 2024-04-16T05:45:43.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb88c674640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb884101130 0x7fb884197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb88c674640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 0x7fb884197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb88c674640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88419c020 0x7fb88419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb88c674640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841112c0 con 0x7fb884106200 2024-04-16T05:45:43.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb88c674640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb884111140 con 0x7fb884101130 2024-04-16T05:45:43.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb88c674640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb884111440 con 0x7fb88419c020 2024-04-16T05:45:43.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb889c0f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 0x7fb884197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb889c0f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 0x7fb884197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48676/0 (socket says 172.21.15.86:48676) 2024-04-16T05:45:43.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.575+0000 7fb889c0f640 1 -- 172.21.15.86:0/3675404322 learned_addr learned my addr 172.21.15.86:0/3675404322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb88ac11640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88419c020 0x7fb88419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb889c0f640 1 -- 172.21.15.86:0/3675404322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88419c020 msgr2=0x7fb88419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb889c0f640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88419c020 0x7fb88419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb889c0f640 1 -- 172.21.15.86:0/3675404322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb884101130 msgr2=0x7fb884197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb889c0f640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb884101130 0x7fb884197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb889c0f640 1 -- 172.21.15.86:0/3675404322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88419cb10 con 0x7fb884106200 2024-04-16T05:45:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb88ac11640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88419c020 0x7fb88419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb889c0f640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 0x7fb884197ca0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fb87400d770 tx=0x7fb87400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb87b7fe640 1 -- 172.21.15.86:0/3675404322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874017070 con 0x7fb884106200 2024-04-16T05:45:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb87b7fe640 1 -- 172.21.15.86:0/3675404322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb874009c30 con 0x7fb884106200 2024-04-16T05:45:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb88c674640 1 -- 172.21.15.86:0/3675404322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88419ce00 con 0x7fb884106200 2024-04-16T05:45:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb87b7fe640 1 -- 172.21.15.86:0/3675404322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87400f620 con 0x7fb884106200 2024-04-16T05:45:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.576+0000 7fb88c674640 1 -- 172.21.15.86:0/3675404322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8841a45f0 con 0x7fb884106200 2024-04-16T05:45:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88f267740 1 Processor -- start 2024-04-16T05:45:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88f267740 1 -- start start 2024-04-16T05:45:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88f267740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 0x560ebe761450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88f267740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe76a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88f267740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe76a570 0x560ebe76c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88f267740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ebe6c40e0 con 0x560ebe761990 2024-04-16T05:45:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88f267740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ebe661520 con 0x560ebe76a570 2024-04-16T05:45:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88f267740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560ebe6c00a0 con 0x560ebe761070 2024-04-16T05:45:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88a410640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe76a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb87b7fe640 1 -- 172.21.15.86:0/3675404322 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb87401a070 con 0x7fb884106200 2024-04-16T05:45:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88a410640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe76a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48692/0 (socket says 172.21.15.86:48692) 2024-04-16T05:45:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88a410640 1 -- 172.21.15.86:0/1018235422 learned_addr learned my addr 172.21.15.86:0/1018235422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88a410640 1 -- 172.21.15.86:0/1018235422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 msgr2=0x560ebe761450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.579+0000 7fb88ac11640 1 --2- 172.21.15.86:0/1018235422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 0x560ebe761450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb88a410640 1 --2- 172.21.15.86:0/1018235422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 0x560ebe761450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb88a410640 1 -- 172.21.15.86:0/1018235422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe76a570 msgr2=0x560ebe76c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb88a410640 1 --2- 172.21.15.86:0/1018235422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe76a570 0x560ebe76c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb88a410640 1 -- 172.21.15.86:0/1018235422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ebe6898d0 con 0x560ebe761990 2024-04-16T05:45:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb88ac11640 1 --2- 172.21.15.86:0/1018235422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 0x560ebe761450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb88a410640 1 --2- 172.21.15.86:0/1018235422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe76a030 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fb88000bb10 tx=0x7fb88000d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9563c855412fad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb878ff9640 1 -- 172.21.15.86:0/1018235422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880016070 con 0x560ebe761990 2024-04-16T05:45:43.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb87b7fe640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb8700a3d20 0x7fb8700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb878ff9640 1 -- 172.21.15.86:0/1018235422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880005500 con 0x560ebe761990 2024-04-16T05:45:43.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb878ff9640 1 -- 172.21.15.86:0/1018235422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800106c0 con 0x560ebe761990 2024-04-16T05:45:43.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb88ac11640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb8700a3d20 0x7fb8700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.580+0000 7fb87b7fe640 1 -- 172.21.15.86:0/3675404322 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fb8740d0710 con 0x7fb884106200 2024-04-16T05:45:43.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88ac11640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb8700a3d20 0x7fb8700a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb87c005cc0 tx=0x7fb87c02e040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 -- 172.21.15.86:0/1018235422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 msgr2=0x560ebe76a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 --2- 172.21.15.86:0/1018235422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe76a030 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fb88000bb10 tx=0x7fb88000d2e0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 -- 172.21.15.86:0/1018235422 shutdown_connections 2024-04-16T05:45:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 --2- 172.21.15.86:0/1018235422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe76a570 0x560ebe76c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 --2- 172.21.15.86:0/1018235422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe76a030 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 --2- 172.21.15.86:0/1018235422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 0x560ebe761450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 -- 172.21.15.86:0/1018235422 >> 172.21.15.86:0/1018235422 conn(0x560ebe75b7a0 msgr2=0x560ebe75bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 -- 172.21.15.86:0/1018235422 shutdown_connections 2024-04-16T05:45:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 -- 172.21.15.86:0/1018235422 wait complete. 2024-04-16T05:45:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 -- start start 2024-04-16T05:45:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.581+0000 7fb88f267740 1 Processor -- start 2024-04-16T05:45:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88f267740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe7674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88f267740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761990 0x560ebe767a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88f267740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe767f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88f267740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ebe76d360 con 0x560ebe76a570 2024-04-16T05:45:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88f267740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ebe67d4d0 con 0x560ebe761070 2024-04-16T05:45:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88f267740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560ebe76d4e0 con 0x560ebe761990 2024-04-16T05:45:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88f267740 10 client.0 _ll_register_callbacks cb 0x560ebe6622c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88ac11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe767f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88ac11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe767f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48694/0 (socket says 172.21.15.86:48694) 2024-04-16T05:45:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88ac11640 1 -- 172.21.15.86:0/3192195673 learned_addr learned my addr 172.21.15.86:0/3192195673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88a410640 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe7674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb889c0f640 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761990 0x560ebe767a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88ac11640 1 -- 172.21.15.86:0/3192195673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761990 msgr2=0x560ebe767a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88ac11640 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761990 0x560ebe767a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88ac11640 1 -- 172.21.15.86:0/3192195673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 msgr2=0x560ebe7674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88ac11640 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe7674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88ac11640 1 -- 172.21.15.86:0/3192195673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ebe3d63f0 con 0x560ebe76a570 2024-04-16T05:45:43.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88a410640 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe7674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:45:43.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb88ac11640 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe767f70 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009ca0 tx=0x7fb87c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc399ff8859fa2a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.582+0000 7fb86dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:43.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb86dffb640 1 -- 172.21.15.86:0/3192195673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c051060 con 0x560ebe76a570 2024-04-16T05:45:43.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb86dffb640 1 -- 172.21.15.86:0/3192195673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fb87c003b70 con 0x560ebe76a570 2024-04-16T05:45:43.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb88f267740 1 -- 172.21.15.86:0/3192195673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb880005500 con 0x560ebe76a570 2024-04-16T05:45:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb86dffb640 1 -- 172.21.15.86:0/3192195673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb87c039050 con 0x560ebe76a570 2024-04-16T05:45:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb88f267740 10 client.18726 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb88f267740 1 -- 172.21.15.86:0/3192195673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ebe4a07d0 con 0x560ebe76a570 2024-04-16T05:45:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb86dffb640 1 -- 172.21.15.86:0/3192195673 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880005500 con 0x560ebe76a570 2024-04-16T05:45:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb86dffb640 1 -- 172.21.15.86:0/3192195673 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb87c040640 con 0x560ebe76a570 2024-04-16T05:45:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb88f267740 10 client.18726 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb88f267740 10 client.18726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb88f267740 20 client.18726 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb88f267740 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560ebe77a470 0x560ebe77c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb88f267740 4 client.18726 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.583+0000 7fb88f267740 1 -- 172.21.15.86:0/3192195673 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ebe6d1b00 con 0x560ebe77a470 2024-04-16T05:45:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.584+0000 7fb88a410640 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560ebe77a470 0x560ebe77c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.584+0000 7fb86dffb640 10 client.18726 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.584+0000 7fb88a410640 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560ebe77a470 0x560ebe77c910 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad2e531cc3f4408 server_cookie=54a9c07dd6937bdd in_seq=0 out_seq=0 2024-04-16T05:45:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb86dffb640 1 -- 172.21.15.86:0/3192195673 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ebe6d1b00 con 0x560ebe77a470 2024-04-16T05:45:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb86dffb640 10 client.18726 handle_command_reply: tid=0 2024-04-16T05:45:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 client.18726 shutdown 2024-04-16T05:45:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 -- 172.21.15.86:0/3192195673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 msgr2=0x560ebe767f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe767f70 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009ca0 tx=0x7fb87c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 -- 172.21.15.86:0/3192195673 shutdown_connections 2024-04-16T05:45:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560ebe77a470 0x560ebe77c910 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe767f70 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761990 0x560ebe767a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb86dffb640 0 client.18726 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 --2- 172.21.15.86:0/3192195673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe7674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 -- 172.21.15.86:0/3192195673 >> 172.21.15.86:0/3192195673 conn(0x560ebe75b7a0 msgr2=0x560ebe758730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 -- 172.21.15.86:0/3192195673 shutdown_connections 2024-04-16T05:45:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 1 -- 172.21.15.86:0/3192195673 wait complete. 2024-04-16T05:45:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.585+0000 7fb88f267740 20 client.18726 trim_cache size 0 max 16384 2024-04-16T05:45:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.610+0000 7fb88f267740 1 Processor -- start 2024-04-16T05:45:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb88f267740 1 -- start start 2024-04-16T05:45:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb88f267740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe769c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb88f267740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe776280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb88f267740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe76a570 0x560ebe766650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb88f267740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841ad320 con 0x560ebe761990 2024-04-16T05:45:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb88f267740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841aee60 con 0x560ebe761070 2024-04-16T05:45:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb88f267740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841ad9f0 con 0x560ebe76a570 2024-04-16T05:45:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb889c0f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe776280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb889c0f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe776280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48698/0 (socket says 172.21.15.86:48698) 2024-04-16T05:45:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb889c0f640 1 -- 172.21.15.86:0/1444560927 learned_addr learned my addr 172.21.15.86:0/1444560927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb88a410640 1 --2- 172.21.15.86:0/1444560927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe769c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb889c0f640 1 -- 172.21.15.86:0/1444560927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe76a570 msgr2=0x560ebe766650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb889c0f640 1 --2- 172.21.15.86:0/1444560927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe76a570 0x560ebe766650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb889c0f640 1 -- 172.21.15.86:0/1444560927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 msgr2=0x560ebe769c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb889c0f640 1 --2- 172.21.15.86:0/1444560927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe769c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb889c0f640 1 -- 172.21.15.86:0/1444560927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ebe75b5d0 con 0x560ebe761990 2024-04-16T05:45:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb88a410640 1 --2- 172.21.15.86:0/1444560927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe769c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb889c0f640 1 --2- 172.21.15.86:0/1444560927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe776280 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb874010fd0 tx=0x7fb87409bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2426ba0247c1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb86ffff640 1 -- 172.21.15.86:0/1444560927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874021040 con 0x560ebe761990 2024-04-16T05:45:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb86ffff640 1 -- 172.21.15.86:0/1444560927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb874096e50 con 0x560ebe761990 2024-04-16T05:45:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.611+0000 7fb86ffff640 1 -- 172.21.15.86:0/1444560927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740d02e0 con 0x560ebe761990 2024-04-16T05:45:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 -- 172.21.15.86:0/1444560927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 msgr2=0x560ebe776280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 --2- 172.21.15.86:0/1444560927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe776280 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb874010fd0 tx=0x7fb87409bd60 comp rx=0 tx=0).stop 2024-04-16T05:45:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 -- 172.21.15.86:0/1444560927 shutdown_connections 2024-04-16T05:45:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 --2- 172.21.15.86:0/1444560927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe76a570 0x560ebe766650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 --2- 172.21.15.86:0/1444560927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe761990 0x560ebe776280 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 --2- 172.21.15.86:0/1444560927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761070 0x560ebe769c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 -- 172.21.15.86:0/1444560927 >> 172.21.15.86:0/1444560927 conn(0x560ebe75b7a0 msgr2=0x560ebe7545c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 -- 172.21.15.86:0/1444560927 shutdown_connections 2024-04-16T05:45:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 -- 172.21.15.86:0/1444560927 wait complete. 2024-04-16T05:45:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.612+0000 7fb88f267740 1 -- start start 2024-04-16T05:45:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88f267740 1 Processor -- start 2024-04-16T05:45:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88f267740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 0x560ebe7728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88f267740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761990 0x560ebe76dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88f267740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe76e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88f267740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841ae790 con 0x560ebe76a570 2024-04-16T05:45:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88f267740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841ae0c0 con 0x560ebe761990 2024-04-16T05:45:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88f267740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841ab320 con 0x560ebe761070 2024-04-16T05:45:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88f267740 10 client.0 _ll_register_callbacks cb 0x7fb87c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88ac11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe76e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88ac11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe76e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48712/0 (socket says 172.21.15.86:48712) 2024-04-16T05:45:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88ac11640 1 -- 172.21.15.86:0/4028127370 learned_addr learned my addr 172.21.15.86:0/4028127370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb889c0f640 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761990 0x560ebe76dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88a410640 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 0x560ebe7728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88ac11640 1 -- 172.21.15.86:0/4028127370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 msgr2=0x560ebe7728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88ac11640 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 0x560ebe7728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88ac11640 1 -- 172.21.15.86:0/4028127370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761990 msgr2=0x560ebe76dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88ac11640 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761990 0x560ebe76dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.613+0000 7fb88ac11640 1 -- 172.21.15.86:0/4028127370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ebe76e7d0 con 0x560ebe76a570 2024-04-16T05:45:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88ac11640 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe76e290 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0409e0 tx=0x7fb87c040730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d2a25157f35a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb86dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-16T05:45:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb86dffb640 1 -- 172.21.15.86:0/4028127370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c061060 con 0x560ebe76a570 2024-04-16T05:45:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb86dffb640 1 -- 172.21.15.86:0/4028127370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fb87c003d80 con 0x560ebe76a570 2024-04-16T05:45:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88f267740 1 -- 172.21.15.86:0/4028127370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb874096e50 con 0x560ebe76a570 2024-04-16T05:45:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb86dffb640 1 -- 172.21.15.86:0/4028127370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb87c068050 con 0x560ebe76a570 2024-04-16T05:45:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88f267740 10 client.18732 fetch_fsmap learned FSMap version 29 2024-04-16T05:45:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88f267740 1 -- 172.21.15.86:0/4028127370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ebe7e0710 con 0x560ebe76a570 2024-04-16T05:45:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb86dffb640 1 -- 172.21.15.86:0/4028127370 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb874096e50 con 0x560ebe76a570 2024-04-16T05:45:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb86dffb640 1 -- 172.21.15.86:0/4028127370 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb87c04c470 con 0x560ebe76a570 2024-04-16T05:45:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88f267740 10 client.18732 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T05:45:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88f267740 10 client.18732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88f267740 20 client.18732 populate_metadata read hostname 'smithi086' 2024-04-16T05:45:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88f267740 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560ebe7e0e30 0x560ebe7e3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88f267740 4 client.18732 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.614+0000 7fb88f267740 1 -- 172.21.15.86:0/4028127370 --> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ebe6cb970 con 0x560ebe7e0e30 2024-04-16T05:45:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88a410640 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560ebe7e0e30 0x560ebe7e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88a410640 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560ebe7e0e30 0x560ebe7e3250 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a63755075c83e0b5 server_cookie=acb2f9e0019c9916 in_seq=0 out_seq=0 2024-04-16T05:45:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb86dffb640 10 client.18732 ms_handle_connect on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb86dffb640 1 -- 172.21.15.86:0/4028127370 <== mds.0 v2:172.21.15.194:6838/601781370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ebe6cb970 con 0x560ebe7e0e30 2024-04-16T05:45:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb86dffb640 10 client.18732 handle_command_reply: tid=0 2024-04-16T05:45:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88f267740 1 client.18732 shutdown 2024-04-16T05:45:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88f267740 1 -- 172.21.15.86:0/4028127370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 msgr2=0x560ebe76e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88f267740 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe76e290 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0409e0 tx=0x7fb87c040730 comp rx=0 tx=0).stop 2024-04-16T05:45:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88f267740 1 -- 172.21.15.86:0/4028127370 shutdown_connections 2024-04-16T05:45:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88f267740 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.194:6838/601781370,v1:172.21.15.194:6839/601781370] conn(0x560ebe7e0e30 0x560ebe7e3250 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88f267740 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ebe76a570 0x560ebe76e290 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb86dffb640 0 client.18732 ms_handle_reset on v2:172.21.15.194:6838/601781370 2024-04-16T05:45:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88f267740 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ebe761990 0x560ebe76dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88f267740 1 --2- 172.21.15.86:0/4028127370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ebe761070 0x560ebe7728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.615+0000 7fb88f267740 1 -- 172.21.15.86:0/4028127370 >> 172.21.15.86:0/4028127370 conn(0x560ebe75b7a0 msgr2=0x560ebe766650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88f267740 1 -- 172.21.15.86:0/4028127370 shutdown_connections 2024-04-16T05:45:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88f267740 1 -- 172.21.15.86:0/4028127370 wait complete. 2024-04-16T05:45:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88f267740 20 client.18732 trim_cache size 0 max 16384 2024-04-16T05:45:43.626 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-16T05:45:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 -- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb8700a3d20 msgr2=0x7fb8700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb8700a3d20 0x7fb8700a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb87c005cc0 tx=0x7fb87c02e040 comp rx=0 tx=0).stop 2024-04-16T05:45:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 -- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 msgr2=0x7fb884197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 0x7fb884197ca0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fb87400d770 tx=0x7fb87400dc30 comp rx=0 tx=0).stop 2024-04-16T05:45:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 -- 172.21.15.86:0/3675404322 shutdown_connections 2024-04-16T05:45:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88419c020 0x7fb88419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb884106200 0x7fb884197ca0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fb8700a3d20 0x7fb8700a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 --2- 172.21.15.86:0/3675404322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb884101130 0x7fb884197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 -- 172.21.15.86:0/3675404322 >> 172.21.15.86:0/3675404322 conn(0x7fb8840780b0 msgr2=0x7fb8840fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 -- 172.21.15.86:0/3675404322 shutdown_connections 2024-04-16T05:45:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:43.616+0000 7fb88c674640 1 -- 172.21.15.86:0/3675404322 wait complete. 2024-04-16T05:45:43.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:43 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2204528993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:45:44.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:43 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2204528993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:45:44.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:43 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2204528993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:45:44.630 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-16T05:45:44.630 INFO:tasks.fwd_scrub:done joining 2024-04-16T05:45:44.631 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-16T05:45:44.644 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:45:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.763+0000 7fc034dca640 1 -- 172.21.15.86:0/4140312178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030107390 msgr2=0x7fc03010ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.763+0000 7fc034dca640 1 --2- 172.21.15.86:0/4140312178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030107390 0x7fc03010ef10 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc028009ca0 tx=0x7fc02802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.763+0000 7fc034dca640 1 -- 172.21.15.86:0/4140312178 shutdown_connections 2024-04-16T05:45:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.763+0000 7fc034dca640 1 --2- 172.21.15.86:0/4140312178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030107390 0x7fc03010ef10 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.763+0000 7fc034dca640 1 --2- 172.21.15.86:0/4140312178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc030101130 0x7fc030106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.763+0000 7fc034dca640 1 --2- 172.21.15.86:0/4140312178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc030100780 0x7fc030100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.763+0000 7fc034dca640 1 -- 172.21.15.86:0/4140312178 >> 172.21.15.86:0/4140312178 conn(0x7fc0300774d0 msgr2=0x7fc0300fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:44.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.763+0000 7fc034dca640 1 -- 172.21.15.86:0/4140312178 shutdown_connections 2024-04-16T05:45:44.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.763+0000 7fc034dca640 1 -- 172.21.15.86:0/4140312178 wait complete. 2024-04-16T05:45:44.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc034dca640 1 Processor -- start 2024-04-16T05:45:44.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc034dca640 1 -- start start 2024-04-16T05:45:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc034dca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc030100780 0x7fc0301016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc034dca640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc030107390 0x7fc030101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc034dca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030102170 0x7fc0301a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc034dca640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc030112400 con 0x7fc030102170 2024-04-16T05:45:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc034dca640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc030112280 con 0x7fc030107390 2024-04-16T05:45:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc034dca640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc030112580 con 0x7fc030100780 2024-04-16T05:45:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc02ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030102170 0x7fc0301a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc02ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030102170 0x7fc0301a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48730/0 (socket says 172.21.15.86:48730) 2024-04-16T05:45:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc02ed9d640 1 -- 172.21.15.86:0/335060297 learned_addr learned my addr 172.21.15.86:0/335060297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc027fff640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc030107390 0x7fc030101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc02e59c640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc030100780 0x7fc0301016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc02ed9d640 1 -- 172.21.15.86:0/335060297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc030100780 msgr2=0x7fc0301016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc02ed9d640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc030100780 0x7fc0301016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc02ed9d640 1 -- 172.21.15.86:0/335060297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc030107390 msgr2=0x7fc030101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc02ed9d640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc030107390 0x7fc030101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc02ed9d640 1 -- 172.21.15.86:0/335060297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0301a7190 con 0x7fc030102170 2024-04-16T05:45:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.764+0000 7fc027fff640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc030107390 0x7fc030101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.765+0000 7fc02ed9d640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030102170 0x7fc0301a6a90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc02802c700 tx=0x7fc028032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.765+0000 7fc0277fe640 1 -- 172.21.15.86:0/335060297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028039040 con 0x7fc030102170 2024-04-16T05:45:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.765+0000 7fc0277fe640 1 -- 172.21.15.86:0/335060297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc028044070 con 0x7fc030102170 2024-04-16T05:45:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.765+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0301a7420 con 0x7fc030102170 2024-04-16T05:45:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.765+0000 7fc0277fe640 1 -- 172.21.15.86:0/335060297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0280042a0 con 0x7fc030102170 2024-04-16T05:45:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.765+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0301a7910 con 0x7fc030102170 2024-04-16T05:45:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.766+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc030102c20 con 0x7fc030102170 2024-04-16T05:45:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.767+0000 7fc0277fe640 1 -- 172.21.15.86:0/335060297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc028004440 con 0x7fc030102170 2024-04-16T05:45:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.767+0000 7fc0277fe640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc01c0a3d50 0x7fc01c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.768+0000 7fc02e59c640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc01c0a3d50 0x7fc01c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.768+0000 7fc0277fe640 1 -- 172.21.15.86:0/335060297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fc0280f2ec0 con 0x7fc030102170 2024-04-16T05:45:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.768+0000 7fc02e59c640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc01c0a3d50 0x7fc01c0a6210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc020005fd0 tx=0x7fc020005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:44.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.772+0000 7fc0277fe640 1 -- 172.21.15.86:0/335060297 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0280333a0 con 0x7fc030102170 2024-04-16T05:45:44.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.954+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc03010a4b0 con 0x7fc030102170 2024-04-16T05:45:44.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.956+0000 7fc0277fe640 1 -- 172.21.15.86:0/335060297 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25263 (secure 0 0 0) 0x7fc0280bf2a0 con 0x7fc030102170 2024-04-16T05:45:44.956 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:45:44.956 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":103,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:45:15.000498+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2330974702":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/3900523320":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2641972995":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/501397249":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:45:44.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc01c0a3d50 msgr2=0x7fc01c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:44.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc01c0a3d50 0x7fc01c0a6210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc020005fd0 tx=0x7fc020005850 comp rx=0 tx=0).stop 2024-04-16T05:45:44.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030102170 msgr2=0x7fc0301a6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:44.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030102170 0x7fc0301a6a90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc02802c700 tx=0x7fc028032f20 comp rx=0 tx=0).stop 2024-04-16T05:45:44.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 shutdown_connections 2024-04-16T05:45:44.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030102170 0x7fc0301a6a90 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc01c0a3d50 0x7fc01c0a6210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc030107390 0x7fc030101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 --2- 172.21.15.86:0/335060297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc030100780 0x7fc0301016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:44.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 >> 172.21.15.86:0/335060297 conn(0x7fc0300774d0 msgr2=0x7fc030105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:44.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 shutdown_connections 2024-04-16T05:45:44.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:44.958+0000 7fc034dca640 1 -- 172.21.15.86:0/335060297 wait complete. 2024-04-16T05:45:44.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:44 smithi086 ceph-mon[32028]: pgmap v324: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:44.968 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:45:44.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:45:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.077+0000 7fc781d7c640 1 -- 172.21.15.86:0/40966401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101250 msgr2=0x7fc77c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.077+0000 7fc781d7c640 1 --2- 172.21.15.86:0/40966401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101250 0x7fc77c1016b0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc770009930 tx=0x7fc77002bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.077+0000 7fc781d7c640 1 -- 172.21.15.86:0/40966401 shutdown_connections 2024-04-16T05:45:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.077+0000 7fc781d7c640 1 --2- 172.21.15.86:0/40966401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c101bf0 0x7fc77c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.077+0000 7fc781d7c640 1 --2- 172.21.15.86:0/40966401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101250 0x7fc77c1016b0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.077+0000 7fc781d7c640 1 --2- 172.21.15.86:0/40966401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc77c1070e0 0x7fc77c1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.077+0000 7fc781d7c640 1 -- 172.21.15.86:0/40966401 >> 172.21.15.86:0/40966401 conn(0x7fc77c076d00 msgr2=0x7fc77c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.077+0000 7fc781d7c640 1 -- 172.21.15.86:0/40966401 shutdown_connections 2024-04-16T05:45:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.077+0000 7fc781d7c640 1 -- 172.21.15.86:0/40966401 wait complete. 2024-04-16T05:45:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc781d7c640 1 Processor -- start 2024-04-16T05:45:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc781d7c640 1 -- start start 2024-04-16T05:45:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc781d7c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc77c101250 0x7fc77c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc781d7c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101bf0 0x7fc77c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc781d7c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c1070e0 0x7fc77c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc781d7c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c1113c0 con 0x7fc77c101bf0 2024-04-16T05:45:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc781d7c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c111240 con 0x7fc77c101250 2024-04-16T05:45:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc781d7c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c111540 con 0x7fc77c1070e0 2024-04-16T05:45:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc77affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101bf0 0x7fc77c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc77affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101bf0 0x7fc77c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48746/0 (socket says 172.21.15.86:48746) 2024-04-16T05:45:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc77affd640 1 -- 172.21.15.86:0/151875876 learned_addr learned my addr 172.21.15.86:0/151875876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc77b7fe640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc77c101250 0x7fc77c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.078+0000 7fc77bfff640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c1070e0 0x7fc77c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc77affd640 1 -- 172.21.15.86:0/151875876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c1070e0 msgr2=0x7fc77c197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc77affd640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c1070e0 0x7fc77c197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc77affd640 1 -- 172.21.15.86:0/151875876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc77c101250 msgr2=0x7fc77c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc77affd640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc77c101250 0x7fc77c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc77affd640 1 -- 172.21.15.86:0/151875876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c197fc0 con 0x7fc77c101bf0 2024-04-16T05:45:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc77affd640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101bf0 0x7fc77c19db80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc770004f50 tx=0x7fc77002ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc778ff9640 1 -- 172.21.15.86:0/151875876 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770041070 con 0x7fc77c101bf0 2024-04-16T05:45:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc778ff9640 1 -- 172.21.15.86:0/151875876 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7700044d0 con 0x7fc77c101bf0 2024-04-16T05:45:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc778ff9640 1 -- 172.21.15.86:0/151875876 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770034440 con 0x7fc77c101bf0 2024-04-16T05:45:45.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77c198250 con 0x7fc77c101bf0 2024-04-16T05:45:45.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.079+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77c1a43e0 con 0x7fc77c101bf0 2024-04-16T05:45:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.080+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77c069e50 con 0x7fc77c101bf0 2024-04-16T05:45:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.081+0000 7fc778ff9640 1 -- 172.21.15.86:0/151875876 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc770044070 con 0x7fc77c101bf0 2024-04-16T05:45:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.082+0000 7fc778ff9640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc7640a3d20 0x7fc7640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.082+0000 7fc77b7fe640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc7640a3d20 0x7fc7640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.082+0000 7fc778ff9640 1 -- 172.21.15.86:0/151875876 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fc7700f27a0 con 0x7fc77c101bf0 2024-04-16T05:45:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.082+0000 7fc77b7fe640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc7640a3d20 0x7fc7640a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009990 tx=0x7fc76c009110 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:45.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.087+0000 7fc778ff9640 1 -- 172.21.15.86:0/151875876 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7700beb80 con 0x7fc77c101bf0 2024-04-16T05:45:45.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:44 smithi194 ceph-mon[35865]: pgmap v324: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:45.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:44 smithi196 ceph-mon[36373]: pgmap v324: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.263+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc77c069270 con 0x7fc77c101bf0 2024-04-16T05:45:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.263+0000 7fc778ff9640 1 -- 172.21.15.86:0/151875876 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7fc7700be480 con 0x7fc77c101bf0 2024-04-16T05:45:45.263 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:45:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc7640a3d20 msgr2=0x7fc7640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc7640a3d20 0x7fc7640a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009990 tx=0x7fc76c009110 comp rx=0 tx=0).stop 2024-04-16T05:45:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101bf0 msgr2=0x7fc77c19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101bf0 0x7fc77c19db80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc770004f50 tx=0x7fc77002ce20 comp rx=0 tx=0).stop 2024-04-16T05:45:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 shutdown_connections 2024-04-16T05:45:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c1070e0 0x7fc77c197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc77c101bf0 0x7fc77c19db80 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc7640a3d20 0x7fc7640a61e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 --2- 172.21.15.86:0/151875876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc77c101250 0x7fc77c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 >> 172.21.15.86:0/151875876 conn(0x7fc77c076d00 msgr2=0x7fc77c0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 shutdown_connections 2024-04-16T05:45:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.265+0000 7fc781d7c640 1 -- 172.21.15.86:0/151875876 wait complete. 2024-04-16T05:45:45.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:45:45.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.383+0000 7fd6f75bd640 1 -- 172.21.15.86:0/2956084663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0107310 msgr2=0x7fd6f010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.383+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/2956084663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0107310 0x7fd6f010eee0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009ca0 tx=0x7fd6e802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.383+0000 7fd6f75bd640 1 -- 172.21.15.86:0/2956084663 shutdown_connections 2024-04-16T05:45:45.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.383+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/2956084663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0107310 0x7fd6f010eee0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.383+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/2956084663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f0102bf0 0x7fd6f0106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.383+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/2956084663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f0102240 0x7fd6f0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.383+0000 7fd6f75bd640 1 -- 172.21.15.86:0/2956084663 >> 172.21.15.86:0/2956084663 conn(0x7fd6f0079450 msgr2=0x7fd6f0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:45.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.383+0000 7fd6f75bd640 1 -- 172.21.15.86:0/2956084663 shutdown_connections 2024-04-16T05:45:45.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.383+0000 7fd6f75bd640 1 -- 172.21.15.86:0/2956084663 wait complete. 2024-04-16T05:45:45.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f75bd640 1 Processor -- start 2024-04-16T05:45:45.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f75bd640 1 -- start start 2024-04-16T05:45:45.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f75bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f0102240 0x7fd6f019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f75bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f0107310 0x7fd6f019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f75bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0198750 0x7fd6f0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f75bd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01123d0 con 0x7fd6f0198750 2024-04-16T05:45:45.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f75bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0112250 con 0x7fd6f0102240 2024-04-16T05:45:45.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f75bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0112550 con 0x7fd6f0107310 2024-04-16T05:45:45.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5b5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0198750 0x7fd6f0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5b5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0198750 0x7fd6f0198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48760/0 (socket says 172.21.15.86:48760) 2024-04-16T05:45:45.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5b5a640 1 -- 172.21.15.86:0/1575442502 learned_addr learned my addr 172.21.15.86:0/1575442502 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:45.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5359640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f0102240 0x7fd6f019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5b5a640 1 -- 172.21.15.86:0/1575442502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f0107310 msgr2=0x7fd6f019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5b5a640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f0107310 0x7fd6f019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5b5a640 1 -- 172.21.15.86:0/1575442502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f0102240 msgr2=0x7fd6f019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5b5a640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f0102240 0x7fd6f019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5b5a640 1 -- 172.21.15.86:0/1575442502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f0199350 con 0x7fd6f0198750 2024-04-16T05:45:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.384+0000 7fd6f5359640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f0102240 0x7fd6f019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.385+0000 7fd6f5b5a640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0198750 0x7fd6f0198bd0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd6e802c700 tx=0x7fd6e8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.385+0000 7fd6e67fc640 1 -- 172.21.15.86:0/1575442502 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8039040 con 0x7fd6f0198750 2024-04-16T05:45:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.385+0000 7fd6e67fc640 1 -- 172.21.15.86:0/1575442502 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6e8044070 con 0x7fd6f0198750 2024-04-16T05:45:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.385+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f01995e0 con 0x7fd6f0198750 2024-04-16T05:45:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.385+0000 7fd6e67fc640 1 -- 172.21.15.86:0/1575442502 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80042a0 con 0x7fd6f0198750 2024-04-16T05:45:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.385+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a56d0 con 0x7fd6f0198750 2024-04-16T05:45:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.386+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f0103710 con 0x7fd6f0198750 2024-04-16T05:45:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.387+0000 7fd6e67fc640 1 -- 172.21.15.86:0/1575442502 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6e8004440 con 0x7fd6f0198750 2024-04-16T05:45:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.387+0000 7fd6e67fc640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6dc0a3d20 0x7fd6dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.387+0000 7fd6f5359640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6dc0a3d20 0x7fd6dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.388+0000 7fd6e67fc640 1 -- 172.21.15.86:0/1575442502 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fd6e80f2f00 con 0x7fd6f0198750 2024-04-16T05:45:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.388+0000 7fd6f5359640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6dc0a3d20 0x7fd6dc0a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec005fd0 tx=0x7fd6ec005ea0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:45.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.392+0000 7fd6e67fc640 1 -- 172.21.15.86:0/1575442502 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6e80bf2e0 con 0x7fd6f0198750 2024-04-16T05:45:45.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.569+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6f0102bf0 con 0x7fd6f0198750 2024-04-16T05:45:45.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.569+0000 7fd6e67fc640 1 -- 172.21.15.86:0/1575442502 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7fd6e80bebe0 con 0x7fd6f0198750 2024-04-16T05:45:45.569 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:45:45.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6dc0a3d20 msgr2=0x7fd6dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6dc0a3d20 0x7fd6dc0a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec005fd0 tx=0x7fd6ec005ea0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0198750 msgr2=0x7fd6f0198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0198750 0x7fd6f0198bd0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd6e802c700 tx=0x7fd6e8032f20 comp rx=0 tx=0).stop 2024-04-16T05:45:45.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 shutdown_connections 2024-04-16T05:45:45.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f0198750 0x7fd6f0198bd0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f0107310 0x7fd6f019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd6dc0a3d20 0x7fd6dc0a61e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 --2- 172.21.15.86:0/1575442502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f0102240 0x7fd6f019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 >> 172.21.15.86:0/1575442502 conn(0x7fd6f0079450 msgr2=0x7fd6f00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:45.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.571+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 shutdown_connections 2024-04-16T05:45:45.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.572+0000 7fd6f75bd640 1 -- 172.21.15.86:0/1575442502 wait complete. 2024-04-16T05:45:45.582 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:45:45.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.690+0000 7feb64aa5640 1 -- 172.21.15.86:0/3003627309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60101af0 msgr2=0x7feb60105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.690+0000 7feb64aa5640 1 --2- 172.21.15.86:0/3003627309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60101af0 0x7feb60105b30 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7feb54009900 tx=0x7feb5402bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.690+0000 7feb64aa5640 1 -- 172.21.15.86:0/3003627309 shutdown_connections 2024-04-16T05:45:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.690+0000 7feb64aa5640 1 --2- 172.21.15.86:0/3003627309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb60106210 0x7feb6010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.690+0000 7feb64aa5640 1 --2- 172.21.15.86:0/3003627309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60101af0 0x7feb60105b30 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.690+0000 7feb64aa5640 1 --2- 172.21.15.86:0/3003627309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb60101140 0x7feb60101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.690+0000 7feb64aa5640 1 -- 172.21.15.86:0/3003627309 >> 172.21.15.86:0/3003627309 conn(0x7feb60078060 msgr2=0x7feb60078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.690+0000 7feb64aa5640 1 -- 172.21.15.86:0/3003627309 shutdown_connections 2024-04-16T05:45:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.690+0000 7feb64aa5640 1 -- 172.21.15.86:0/3003627309 wait complete. 2024-04-16T05:45:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb64aa5640 1 Processor -- start 2024-04-16T05:45:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb64aa5640 1 -- start start 2024-04-16T05:45:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb64aa5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb60101140 0x7feb6019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb64aa5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60106210 0x7feb6019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb64aa5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb601976a0 0x7feb60197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb64aa5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb60111300 con 0x7feb60106210 2024-04-16T05:45:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb64aa5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb60111180 con 0x7feb601976a0 2024-04-16T05:45:45.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb64aa5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb60111480 con 0x7feb60101140 2024-04-16T05:45:45.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60106210 0x7feb6019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60106210 0x7feb6019db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48784/0 (socket says 172.21.15.86:48784) 2024-04-16T05:45:45.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5dd9b640 1 -- 172.21.15.86:0/2655719577 learned_addr learned my addr 172.21.15.86:0/2655719577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:45.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5ed9d640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb601976a0 0x7feb60197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5dd9b640 1 -- 172.21.15.86:0/2655719577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb60101140 msgr2=0x7feb6019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5dd9b640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb60101140 0x7feb6019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5dd9b640 1 -- 172.21.15.86:0/2655719577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb601976a0 msgr2=0x7feb60197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5dd9b640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb601976a0 0x7feb60197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5dd9b640 1 -- 172.21.15.86:0/2655719577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb601982a0 con 0x7feb60106210 2024-04-16T05:45:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.691+0000 7feb5ed9d640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb601976a0 0x7feb60197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.692+0000 7feb5dd9b640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60106210 0x7feb6019db20 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7feb54036040 tx=0x7feb5402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.692+0000 7feb4f7fe640 1 -- 172.21.15.86:0/2655719577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54041070 con 0x7feb60106210 2024-04-16T05:45:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.692+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb60198530 con 0x7feb60106210 2024-04-16T05:45:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.692+0000 7feb4f7fe640 1 -- 172.21.15.86:0/2655719577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb540054e0 con 0x7feb60106210 2024-04-16T05:45:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.692+0000 7feb4f7fe640 1 -- 172.21.15.86:0/2655719577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54034440 con 0x7feb60106210 2024-04-16T05:45:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.692+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb601a4670 con 0x7feb60106210 2024-04-16T05:45:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.693+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb60102610 con 0x7feb60106210 2024-04-16T05:45:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.694+0000 7feb4f7fe640 1 -- 172.21.15.86:0/2655719577 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb54044070 con 0x7feb60106210 2024-04-16T05:45:45.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.695+0000 7feb4f7fe640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb480a3d20 0x7feb480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:45.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.695+0000 7feb5e59c640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb480a3d20 0x7feb480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:45.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.695+0000 7feb4f7fe640 1 -- 172.21.15.86:0/2655719577 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7feb540f2c70 con 0x7feb60106210 2024-04-16T05:45:45.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.695+0000 7feb5e59c640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb480a3d20 0x7feb480a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb58005c40 tx=0x7feb58005bd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:45.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.699+0000 7feb4f7fe640 1 -- 172.21.15.86:0/2655719577 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb540bf050 con 0x7feb60106210 2024-04-16T05:45:45.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.878+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb60101af0 con 0x7feb60106210 2024-04-16T05:45:45.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.878+0000 7feb4f7fe640 1 -- 172.21.15.86:0/2655719577 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7feb540be950 con 0x7feb60106210 2024-04-16T05:45:45.878 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:45:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb480a3d20 msgr2=0x7feb480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb480a3d20 0x7feb480a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb58005c40 tx=0x7feb58005bd0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60106210 msgr2=0x7feb6019db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60106210 0x7feb6019db20 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7feb54036040 tx=0x7feb5402cbf0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 shutdown_connections 2024-04-16T05:45:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7feb480a3d20 0x7feb480a61e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb601976a0 0x7feb60197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb60106210 0x7feb6019db20 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 --2- 172.21.15.86:0/2655719577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb60101140 0x7feb6019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 >> 172.21.15.86:0/2655719577 conn(0x7feb60078060 msgr2=0x7feb601073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 shutdown_connections 2024-04-16T05:45:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.880+0000 7feb64aa5640 1 -- 172.21.15.86:0/2655719577 wait complete. 2024-04-16T05:45:45.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:45:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 -- 172.21.15.86:0/4014346686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d6006b2a0 msgr2=0x7f0d60106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 --2- 172.21.15.86:0/4014346686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d6006b2a0 0x7f0d60106c40 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009900 tx=0x7f0d5c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 -- 172.21.15.86:0/4014346686 shutdown_connections 2024-04-16T05:45:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 --2- 172.21.15.86:0/4014346686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d60107320 0x7f0d6010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 --2- 172.21.15.86:0/4014346686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d6006b2a0 0x7f0d60106c40 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 --2- 172.21.15.86:0/4014346686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d6006a8f0 0x7f0d6006acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 -- 172.21.15.86:0/4014346686 >> 172.21.15.86:0/4014346686 conn(0x7f0d60077f60 msgr2=0x7f0d60078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 -- 172.21.15.86:0/4014346686 shutdown_connections 2024-04-16T05:45:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 -- 172.21.15.86:0/4014346686 wait complete. 2024-04-16T05:45:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 Processor -- start 2024-04-16T05:45:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:45.999+0000 7f0d68722640 1 -- start start 2024-04-16T05:45:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d68722640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d6006a8f0 0x7f0d6019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d68722640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d60107320 0x7f0d6019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d68722640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d601987d0 0x7f0d60198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d68722640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112490 con 0x7f0d601987d0 2024-04-16T05:45:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d68722640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112310 con 0x7f0d6006a8f0 2024-04-16T05:45:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d68722640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112610 con 0x7f0d60107320 2024-04-16T05:45:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d66cbf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d601987d0 0x7f0d60198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d66cbf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d601987d0 0x7f0d60198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48798/0 (socket says 172.21.15.86:48798) 2024-04-16T05:45:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d66cbf640 1 -- 172.21.15.86:0/3744167910 learned_addr learned my addr 172.21.15.86:0/3744167910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d664be640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d6006a8f0 0x7f0d6019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d66cbf640 1 -- 172.21.15.86:0/3744167910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d60107320 msgr2=0x7f0d6019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d65cbd640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d60107320 0x7f0d6019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d66cbf640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d60107320 0x7f0d6019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d66cbf640 1 -- 172.21.15.86:0/3744167910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d6006a8f0 msgr2=0x7f0d6019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:46.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d66cbf640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d6006a8f0 0x7f0d6019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d66cbf640 1 -- 172.21.15.86:0/3744167910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d60199360 con 0x7f0d601987d0 2024-04-16T05:45:46.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d664be640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d6006a8f0 0x7f0d6019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:46.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d66cbf640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d601987d0 0x7f0d60198c80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b970 tx=0x7f0d5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:46.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d577fe640 1 -- 172.21.15.86:0/3744167910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58019070 con 0x7f0d601987d0 2024-04-16T05:45:46.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d577fe640 1 -- 172.21.15.86:0/3744167910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d5800cb40 con 0x7f0d601987d0 2024-04-16T05:45:46.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d60199650 con 0x7f0d601987d0 2024-04-16T05:45:46.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d577fe640 1 -- 172.21.15.86:0/3744167910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58007650 con 0x7f0d601987d0 2024-04-16T05:45:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.000+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d601a5880 con 0x7f0d601987d0 2024-04-16T05:45:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.001+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d60103720 con 0x7f0d601987d0 2024-04-16T05:45:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.002+0000 7f0d577fe640 1 -- 172.21.15.86:0/3744167910 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d5801c070 con 0x7f0d601987d0 2024-04-16T05:45:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.003+0000 7f0d577fe640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d4c0a3da0 0x7f0d4c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.003+0000 7f0d664be640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d4c0a3da0 0x7f0d4c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.003+0000 7f0d577fe640 1 -- 172.21.15.86:0/3744167910 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f0d580d0060 con 0x7f0d601987d0 2024-04-16T05:45:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.004+0000 7f0d664be640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d4c0a3da0 0x7f0d4c0a6260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009930 tx=0x7f0d50008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.008+0000 7f0d577fe640 1 -- 172.21.15.86:0/3744167910 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d60199650 con 0x7f0d601987d0 2024-04-16T05:45:46.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:45 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/335060297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:45:46.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:45 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/151875876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:45:46.156 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:45 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1575442502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:45:46.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.187+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0d6006b2a0 con 0x7f0d601987d0 2024-04-16T05:45:46.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.187+0000 7f0d577fe640 1 -- 172.21.15.86:0/3744167910 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) ==== 103+0+11 (secure 0 0 0) 0x7f0d5809c440 con 0x7f0d601987d0 2024-04-16T05:45:46.187 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:45:46.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d4c0a3da0 msgr2=0x7f0d4c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:46.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d4c0a3da0 0x7f0d4c0a6260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009930 tx=0x7f0d50008040 comp rx=0 tx=0).stop 2024-04-16T05:45:46.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d601987d0 msgr2=0x7f0d60198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:46.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d601987d0 0x7f0d60198c80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b970 tx=0x7f0d5800be30 comp rx=0 tx=0).stop 2024-04-16T05:45:46.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 shutdown_connections 2024-04-16T05:45:46.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d601987d0 0x7f0d60198c80 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d60107320 0x7f0d6019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f0d4c0a3da0 0x7f0d4c0a6260 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 --2- 172.21.15.86:0/3744167910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d6006a8f0 0x7f0d6019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 >> 172.21.15.86:0/3744167910 conn(0x7f0d60077f60 msgr2=0x7f0d601005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:46.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 shutdown_connections 2024-04-16T05:45:46.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.189+0000 7f0d68722640 1 -- 172.21.15.86:0/3744167910 wait complete. 2024-04-16T05:45:46.198 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:45 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/335060297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:45:46.198 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:45 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/151875876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:45:46.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:45 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1575442502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:45:46.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-16T05:45:46.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:45 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/335060297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:45:46.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:45 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/151875876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:45:46.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:45 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1575442502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:45:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.305+0000 7f3b01f82640 1 -- 172.21.15.86:0/4284598221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101b00 msgr2=0x7f3afc105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.305+0000 7f3b01f82640 1 --2- 172.21.15.86:0/4284598221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101b00 0x7f3afc105b40 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009900 tx=0x7f3ae402bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.305+0000 7f3b01f82640 1 -- 172.21.15.86:0/4284598221 shutdown_connections 2024-04-16T05:45:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.305+0000 7f3b01f82640 1 --2- 172.21.15.86:0/4284598221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3afc106220 0x7f3afc10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.305+0000 7f3b01f82640 1 --2- 172.21.15.86:0/4284598221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101b00 0x7f3afc105b40 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.305+0000 7f3b01f82640 1 --2- 172.21.15.86:0/4284598221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3afc101150 0x7f3afc101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.305+0000 7f3b01f82640 1 -- 172.21.15.86:0/4284598221 >> 172.21.15.86:0/4284598221 conn(0x7f3afc078000 msgr2=0x7f3afc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.305+0000 7f3b01f82640 1 -- 172.21.15.86:0/4284598221 shutdown_connections 2024-04-16T05:45:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.306+0000 7f3b01f82640 1 -- 172.21.15.86:0/4284598221 wait complete. 2024-04-16T05:45:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.306+0000 7f3b01f82640 1 Processor -- start 2024-04-16T05:45:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.306+0000 7f3b01f82640 1 -- start start 2024-04-16T05:45:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.306+0000 7f3b01f82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101150 0x7f3afc19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.306+0000 7f3b01f82640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3afc106220 0x7f3afc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.306+0000 7f3b01f82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3afc197610 0x7f3afc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.306+0000 7f3b01f82640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1112b0 con 0x7f3afc101150 2024-04-16T05:45:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.306+0000 7f3b01f82640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111130 con 0x7f3afc106220 2024-04-16T05:45:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.306+0000 7f3b01f82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111430 con 0x7f3afc197610 2024-04-16T05:45:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101150 0x7f3afc19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101150 0x7f3afc19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48808/0 (socket says 172.21.15.86:48808) 2024-04-16T05:45:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afb7fe640 1 -- 172.21.15.86:0/403098392 learned_addr learned my addr 172.21.15.86:0/403098392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:45:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afbfff640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3afc197610 0x7f3afc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:46.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afb7fe640 1 -- 172.21.15.86:0/403098392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3afc197610 msgr2=0x7f3afc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:46.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afb7fe640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3afc197610 0x7f3afc197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afb7fe640 1 -- 172.21.15.86:0/403098392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3afc106220 msgr2=0x7f3afc19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:46.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afb7fe640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3afc106220 0x7f3afc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:46.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afb7fe640 1 -- 172.21.15.86:0/403098392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc198130 con 0x7f3afc101150 2024-04-16T05:45:46.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3afb7fe640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101150 0x7f3afc19d460 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3aec007910 tx=0x7f3aec007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:46.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3af97fa640 1 -- 172.21.15.86:0/403098392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec019070 con 0x7f3afc101150 2024-04-16T05:45:46.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3af97fa640 1 -- 172.21.15.86:0/403098392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aec005020 con 0x7f3afc101150 2024-04-16T05:45:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3af97fa640 1 -- 172.21.15.86:0/403098392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004a20 con 0x7f3afc101150 2024-04-16T05:45:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3afc198420 con 0x7f3afc101150 2024-04-16T05:45:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.307+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3afc0694e0 con 0x7f3afc101150 2024-04-16T05:45:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.308+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3afc102620 con 0x7f3afc101150 2024-04-16T05:45:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.309+0000 7f3af97fa640 1 -- 172.21.15.86:0/403098392 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3aec01c070 con 0x7f3afc101150 2024-04-16T05:45:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.310+0000 7f3af97fa640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3ae80a3d20 0x7f3ae80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.310+0000 7f3af2ffd640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3ae80a3d20 0x7f3ae80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.310+0000 7f3af97fa640 1 -- 172.21.15.86:0/403098392 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f3aec0d1d90 con 0x7f3afc101150 2024-04-16T05:45:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.311+0000 7f3af2ffd640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3ae80a3d20 0x7f3ae80a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ae402c170 tx=0x7f3ae4000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.314+0000 7f3af97fa640 1 -- 172.21.15.86:0/403098392 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3aec09e060 con 0x7f3afc101150 2024-04-16T05:45:46.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.543+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f3afc069fc0 con 0x7f3afc101150 2024-04-16T05:45:46.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.551+0000 7f3af97fa640 1 -- 172.21.15.86:0/403098392 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746061 (secure 0 0 0) 0x7f3aec09d960 con 0x7f3afc101150 2024-04-16T05:45:46.561 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:45:46.561 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:45:46.561 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 26, 2024-04-16T05:45:46.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.561 INFO:teuthology.orchestra.run.smithi086.stdout: "active_gid": 15528, 2024-04-16T05:45:46.561 INFO:teuthology.orchestra.run.smithi086.stdout: "active_name": "x", 2024-04-16T05:45:46.562 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addrs": { 2024-04-16T05:45:46.562 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-16T05:45:46.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-16T05:45:46.562 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6800", 2024-04-16T05:45:46.562 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2117030010 2024-04-16T05:45:46.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-16T05:45:46.563 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6801", 2024-04-16T05:45:46.563 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2117030010 2024-04-16T05:45:46.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.563 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:46.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.563 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addr": "172.21.15.86:6801/2117030010", 2024-04-16T05:45:46.563 INFO:teuthology.orchestra.run.smithi086.stdout: "active_change": "2024-04-16T05:35:16.930041+0000", 2024-04-16T05:45:46.564 INFO:teuthology.orchestra.run.smithi086.stdout: "active_mgr_features": 4540701547738038271, 2024-04-16T05:45:46.564 INFO:teuthology.orchestra.run.smithi086.stdout: "available": true, 2024-04-16T05:45:46.564 INFO:teuthology.orchestra.run.smithi086.stdout: "standbys": [ 2024-04-16T05:45:46.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.564 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 25024, 2024-04-16T05:45:46.564 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "z", 2024-04-16T05:45:46.564 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-04-16T05:45:46.564 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-16T05:45:46.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-16T05:45:46.565 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.565 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.565 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.565 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:46.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:46.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.566 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.566 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:46.566 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.566 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.566 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.566 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T05:45:46.566 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.567 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.567 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.569 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.569 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.569 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.569 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.569 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.570 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.570 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.570 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.570 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.570 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.571 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.571 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.571 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.571 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.571 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.571 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.571 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.572 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.572 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.572 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.572 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.572 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.572 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.572 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.572 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.573 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.573 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.573 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.573 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.573 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.573 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.573 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.574 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.574 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.574 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.574 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.575 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.575 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-16T05:45:46.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-16T05:45:46.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.575 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.576 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-16T05:45:46.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.577 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-16T05:45:46.577 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-16T05:45:46.577 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.577 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-16T05:45:46.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.578 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.578 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.578 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-16T05:45:46.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.578 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.578 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-16T05:45:46.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-16T05:45:46.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.579 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.579 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.579 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.579 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.580 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-16T05:45:46.580 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.580 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-16T05:45:46.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-16T05:45:46.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.580 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-16T05:45:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.582 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-16T05:45:46.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-16T05:45:46.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.582 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.582 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.582 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-16T05:45:46.582 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.583 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-16T05:45:46.583 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.583 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.583 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-16T05:45:46.584 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-16T05:45:46.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.584 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.584 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.584 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.584 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.584 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-16T05:45:46.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.585 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-16T05:45:46.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-16T05:45:46.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.586 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.586 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.586 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.586 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.586 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.586 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T05:45:46.586 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.587 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.587 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.587 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-16T05:45:46.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-16T05:45:46.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.587 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.587 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.588 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.588 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.588 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.588 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-16T05:45:46.588 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.588 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-16T05:45:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-16T05:45:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-16T05:45:46.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.590 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.590 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.590 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T05:45:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-16T05:45:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-16T05:45:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-16T05:45:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T05:45:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-16T05:45:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-16T05:45:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-16T05:45:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-16T05:45:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T05:45:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-16T05:45:46.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-16T05:45:46.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:46.595 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.595 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-16T05:45:46.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:46.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-16T05:45:46.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.595 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T05:45:46.596 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.596 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.596 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-16T05:45:46.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-16T05:45:46.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.596 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.597 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T05:45:46.597 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.597 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.597 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T05:45:46.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T05:45:46.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-16T05:45:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-16T05:45:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-16T05:45:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-16T05:45:46.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-16T05:45:46.599 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.599 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-16T05:45:46.599 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T05:45:46.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.599 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-16T05:45:46.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-16T05:45:46.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.600 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.600 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-16T05:45:46.600 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.600 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.600 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.601 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T05:45:46.601 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:46.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.601 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.601 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-16T05:45:46.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-16T05:45:46.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:46.601 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.602 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:46.602 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-16T05:45:46.602 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-16T05:45:46.602 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.602 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T05:45:46.602 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:46.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.603 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.603 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.604 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.604 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.604 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.604 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.604 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.604 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.604 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.605 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.605 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.606 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.606 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.606 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.607 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.607 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.607 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.608 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.608 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.608 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.608 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.608 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.609 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.609 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.609 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.609 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.610 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.610 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.611 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-16T05:45:46.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-16T05:45:46.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:46.612 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:46.612 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T05:45:46.612 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-16T05:45:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-16T05:45:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-16T05:45:46.614 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.614 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.614 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.614 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-16T05:45:46.614 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-16T05:45:46.614 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-16T05:45:46.614 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-16T05:45:46.614 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-16T05:45:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-16T05:45:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-16T05:45:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-16T05:45:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.616 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.616 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.616 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.616 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T05:45:46.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.617 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.617 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:46.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:46.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.617 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:46.618 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.618 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.618 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.618 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T05:45:46.618 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.618 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.618 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.619 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-16T05:45:46.619 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-16T05:45:46.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.619 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.619 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:46.619 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:46.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.620 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.620 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T05:45:46.620 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T05:45:46.620 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.620 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.620 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-16T05:45:46.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-16T05:45:46.621 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:46.621 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.621 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:46.621 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.621 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.621 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T05:45:46.622 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.622 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.622 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.622 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.622 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-16T05:45:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-16T05:45:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-16T05:45:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-16T05:45:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T05:45:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.625 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-16T05:45:46.625 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-16T05:45:46.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-16T05:45:46.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.626 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.626 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T05:45:46.626 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.626 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.626 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.626 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-16T05:45:46.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-16T05:45:46.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.627 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.627 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.627 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-16T05:45:46.627 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.627 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.627 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.627 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T05:45:46.628 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.628 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.628 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-16T05:45:46.628 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-16T05:45:46.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.628 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.629 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.629 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.629 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T05:45:46.629 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T05:45:46.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-16T05:45:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-16T05:45:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T05:45:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-16T05:45:46.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T05:45:46.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:46.632 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-16T05:45:46.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.633 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T05:45:46.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.633 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-16T05:45:46.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-16T05:45:46.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.634 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.634 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.634 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.634 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.634 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-16T05:45:46.634 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-16T05:45:46.635 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-16T05:45:46.635 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.635 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T05:45:46.635 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.635 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-16T05:45:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-16T05:45:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T05:45:46.637 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.637 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.637 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.637 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-16T05:45:46.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-16T05:45:46.637 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.637 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.638 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.638 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.638 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.638 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T05:45:46.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.639 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.639 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-16T05:45:46.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-16T05:45:46.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.639 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.640 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.640 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.640 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T05:45:46.640 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.640 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-16T05:45:46.641 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-16T05:45:46.641 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.641 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.641 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T05:45:46.641 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.642 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:46.642 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.642 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.642 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.642 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-16T05:45:46.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-16T05:45:46.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.643 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T05:45:46.643 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.643 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.643 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.643 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-16T05:45:46.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.644 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-16T05:45:46.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-16T05:45:46.644 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.644 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T05:45:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-16T05:45:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-16T05:45:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-16T05:45:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T05:45:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.647 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:46.647 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.647 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.647 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-16T05:45:46.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-16T05:45:46.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.648 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T05:45:46.648 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.648 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.648 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-16T05:45:46.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.649 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.649 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.649 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-16T05:45:46.649 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-16T05:45:46.649 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.649 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.650 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T05:45:46.650 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.650 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.650 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.650 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-16T05:45:46.650 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.651 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-16T05:45:46.651 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-16T05:45:46.651 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.651 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.651 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.651 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T05:45:46.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.652 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.652 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-16T05:45:46.652 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.652 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.652 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.652 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-16T05:45:46.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-16T05:45:46.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.653 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T05:45:46.653 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.653 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.653 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-16T05:45:46.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.654 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.654 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-16T05:45:46.654 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-16T05:45:46.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.654 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.655 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T05:45:46.655 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.655 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.655 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.655 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-16T05:45:46.655 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.655 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.656 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-16T05:45:46.656 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-16T05:45:46.656 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.656 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.656 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.656 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T05:45:46.656 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.657 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.657 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-16T05:45:46.657 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.657 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.657 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.657 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-16T05:45:46.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-16T05:45:46.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.658 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.658 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T05:45:46.658 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.658 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.658 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:46.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.659 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.659 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-16T05:45:46.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-16T05:45:46.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.659 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T05:45:46.660 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.660 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-16T05:45:46.660 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-16T05:45:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-16T05:45:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T05:45:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.662 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.662 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.662 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:46.662 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.662 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.662 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.662 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-16T05:45:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-16T05:45:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T05:45:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-16T05:45:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-16T05:45:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-16T05:45:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.665 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T05:45:46.665 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.665 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.665 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.665 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-16T05:45:46.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.666 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.666 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-16T05:45:46.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-16T05:45:46.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.666 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.666 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T05:45:46.667 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.667 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.667 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.667 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-16T05:45:46.667 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.667 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.667 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.667 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-16T05:45:46.668 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-16T05:45:46.668 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.668 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.668 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.668 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.668 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.669 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.669 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T05:45:46.669 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.669 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.669 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-16T05:45:46.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-16T05:45:46.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.670 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.670 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:46.670 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.670 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.670 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.670 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T05:45:46.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.671 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-16T05:45:46.671 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T05:45:46.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.671 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-16T05:45:46.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T05:45:46.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.672 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-16T05:45:46.673 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-16T05:45:46.673 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.673 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-16T05:45:46.673 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.674 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.674 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.674 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T05:45:46.674 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.674 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.674 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.674 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-16T05:45:46.675 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-16T05:45:46.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.675 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-16T05:45:46.675 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.675 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.675 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-16T05:45:46.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.676 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-16T05:45:46.676 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-16T05:45:46.676 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.677 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.677 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T05:45:46.677 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.677 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.678 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.678 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-16T05:45:46.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-16T05:45:46.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.678 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.678 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:46.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-16T05:45:46.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-16T05:45:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-16T05:45:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T05:45:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.681 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T05:45:46.681 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.681 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.681 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.681 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-16T05:45:46.681 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-16T05:45:46.681 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.682 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.682 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:46.682 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.682 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.682 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.682 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-16T05:45:46.682 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.683 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.683 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-16T05:45:46.683 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-16T05:45:46.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.684 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.684 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.684 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.684 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T05:45:46.684 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.685 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-16T05:45:46.685 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-16T05:45:46.685 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.685 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.685 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.685 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.685 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.685 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.686 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.686 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T05:45:46.686 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.686 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.686 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.686 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.687 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.687 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.688 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.688 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.688 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.688 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.688 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.688 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.688 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.688 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.689 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.689 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-16T05:45:46.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-16T05:45:46.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.689 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.689 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.690 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.690 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T05:45:46.690 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.690 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.690 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.691 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.691 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.691 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.691 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.691 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.691 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.691 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.692 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.692 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T05:45:46.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.692 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.692 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.693 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.693 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.693 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.694 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.694 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.694 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.694 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.694 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.694 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.694 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.694 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.695 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.695 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.695 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.695 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.695 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.695 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.696 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.696 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.696 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.696 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.696 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.696 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.697 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T05:45:46.697 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T05:45:46.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.697 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.697 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.697 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.697 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T05:45:46.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.698 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.698 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T05:45:46.698 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T05:45:46.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.699 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-16T05:45:46.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.699 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.699 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.699 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T05:45:46.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.700 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.700 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-16T05:45:46.700 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-16T05:45:46.700 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.700 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:46.701 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.701 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.701 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.701 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-16T05:45:46.701 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.701 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.701 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.702 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-16T05:45:46.702 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-16T05:45:46.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.702 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.702 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:46.702 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.703 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T05:45:46.703 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.703 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.703 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.703 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-16T05:45:46.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-16T05:45:46.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.704 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.704 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.704 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.704 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-16T05:45:46.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.705 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-16T05:45:46.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-16T05:45:46.705 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.705 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-16T05:45:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-16T05:45:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-16T05:45:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T05:45:46.707 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.707 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.707 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.707 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-16T05:45:46.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-16T05:45:46.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.707 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.708 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-16T05:45:46.708 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.708 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.708 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.708 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T05:45:46.708 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.709 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.709 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-16T05:45:46.709 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-16T05:45:46.709 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.709 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.709 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.709 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T05:45:46.710 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.710 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.710 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.710 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T05:45:46.710 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.710 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.710 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-16T05:45:46.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-16T05:45:46.711 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.711 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.711 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.711 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.711 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.712 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.712 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T05:45:46.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.712 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.712 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.712 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-16T05:45:46.712 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-16T05:45:46.712 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.713 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.713 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T05:45:46.713 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.714 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-16T05:45:46.714 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-16T05:45:46.714 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.714 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.714 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T05:45:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.716 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-16T05:45:46.716 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-16T05:45:46.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.716 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.716 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.716 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.716 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.717 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.717 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T05:45:46.717 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.717 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.717 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-16T05:45:46.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-16T05:45:46.718 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.718 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.718 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.718 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.718 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.718 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.718 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.718 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T05:45:46.718 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.719 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-16T05:45:46.719 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-16T05:45:46.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.719 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.719 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-16T05:45:46.720 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.720 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.720 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.720 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-16T05:45:46.720 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.720 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.720 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.721 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-16T05:45:46.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-16T05:45:46.721 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.721 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.721 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.721 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.721 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.721 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.722 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T05:45:46.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.722 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.722 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-16T05:45:46.722 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-16T05:45:46.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.723 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.723 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.723 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.723 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T05:45:46.724 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.724 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.724 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.724 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-16T05:45:46.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-16T05:45:46.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.725 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.725 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.725 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T05:45:46.725 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.725 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.725 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-16T05:45:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-16T05:45:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T05:45:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-16T05:45:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-16T05:45:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T05:45:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-16T05:45:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-16T05:45:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T05:45:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-16T05:45:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-16T05:45:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-16T05:45:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-16T05:45:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T05:45:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-16T05:45:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-16T05:45:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-16T05:45:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T05:45:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-16T05:45:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T05:45:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T05:45:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T05:45:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T05:45:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T05:45:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T05:45:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-16T05:45:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T05:45:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T05:45:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T05:45:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T05:45:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T05:45:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T05:45:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T05:45:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T05:45:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T05:45:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T05:45:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T05:45:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T05:45:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T05:45:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T05:45:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T05:45:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T05:45:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T05:45:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T05:45:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T05:45:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T05:45:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T05:45:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T05:45:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T05:45:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-16T05:45:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T05:45:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T05:45:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-16T05:45:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-16T05:45:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T05:45:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T05:45:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-16T05:45:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T05:45:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T05:45:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T05:45:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T05:45:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T05:45:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T05:45:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T05:45:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T05:45:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T05:45:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T05:45:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-16T05:45:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T05:45:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T05:45:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-16T05:45:46.785 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.785 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.785 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.785 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.785 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.785 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.786 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T05:45:46.786 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T05:45:46.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.786 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.786 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.786 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.786 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.787 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.787 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.787 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.787 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.787 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.787 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.787 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T05:45:46.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T05:45:46.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.788 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.788 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.788 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.788 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.789 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.789 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.789 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.789 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.789 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T05:45:46.789 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T05:45:46.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.789 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.790 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.790 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.790 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.790 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.790 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.790 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.791 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.791 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T05:45:46.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T05:45:46.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.791 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.792 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.792 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.792 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.792 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.792 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.793 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T05:45:46.793 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T05:45:46.793 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.793 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.794 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.794 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.794 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.794 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T05:45:46.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T05:45:46.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.795 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.795 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.795 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.795 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.796 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T05:45:46.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T05:45:46.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.797 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.797 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.798 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T05:45:46.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T05:45:46.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.799 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T05:45:46.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T05:45:46.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.801 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.801 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.801 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.801 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T05:45:46.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T05:45:46.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.802 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.802 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.802 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.802 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.803 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.803 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T05:45:46.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T05:45:46.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T05:45:46.804 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.804 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.804 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.805 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T05:45:46.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T05:45:46.805 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.805 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.805 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:46.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.806 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T05:45:46.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T05:45:46.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.807 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-16T05:45:46.807 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.807 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.808 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T05:45:46.808 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T05:45:46.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.808 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-16T05:45:46.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.809 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.809 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.809 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.809 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.810 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T05:45:46.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T05:45:46.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.810 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.810 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.810 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.811 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.811 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.811 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T05:45:46.811 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T05:45:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-16T05:45:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T05:45:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T05:45:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.814 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.814 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.814 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.814 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.814 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.815 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T05:45:46.815 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T05:45:46.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.815 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.815 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.816 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.816 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.816 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.816 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.816 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T05:45:46.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T05:45:46.817 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.817 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.817 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.817 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.817 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.817 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.817 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.818 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.818 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.818 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.818 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T05:45:46.818 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T05:45:46.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T05:45:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T05:45:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.821 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.821 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.821 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.821 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.821 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.821 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T05:45:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T05:45:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-16T05:45:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-16T05:45:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.824 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.824 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.824 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.824 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.824 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.825 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-16T05:45:46.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-16T05:45:46.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.825 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.825 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.826 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.826 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.826 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.826 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.827 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-16T05:45:46.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-16T05:45:46.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.827 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.827 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.828 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.828 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-16T05:45:46.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.828 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-16T05:45:46.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-16T05:45:46.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.829 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.829 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.829 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-16T05:45:46.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.830 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.830 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.830 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.830 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-16T05:45:46.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-16T05:45:46.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.831 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.831 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.831 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.831 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.831 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.831 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.831 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.831 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.832 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.832 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.832 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.832 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.832 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.832 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.832 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.833 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.833 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.833 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.833 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.833 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.833 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.833 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.833 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.834 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.834 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.834 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.834 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.835 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.835 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.835 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.835 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.835 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.835 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.836 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.836 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.836 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.836 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.838 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.838 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.838 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.838 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.838 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.838 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.838 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-16T05:45:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-16T05:45:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-16T05:45:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-16T05:45:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T05:45:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-16T05:45:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-16T05:45:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-16T05:45:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-16T05:45:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-16T05:45:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-16T05:45:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-16T05:45:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-16T05:45:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-16T05:45:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-16T05:45:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-16T05:45:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-16T05:45:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-16T05:45:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-16T05:45:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-16T05:45:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-16T05:45:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-16T05:45:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-16T05:45:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-16T05:45:46.854 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.854 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.854 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-16T05:45:46.854 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-16T05:45:46.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.855 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.855 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.855 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.855 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.856 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.856 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.856 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.856 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.856 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-16T05:45:46.856 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.857 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.857 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.857 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-16T05:45:46.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-16T05:45:46.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.857 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.858 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.858 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.858 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.858 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-16T05:45:46.858 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.858 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.859 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.859 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.859 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.859 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.859 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.859 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.859 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.859 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.860 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.860 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.860 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.860 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.860 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.860 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.860 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.860 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.861 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.861 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.861 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.861 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.862 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.862 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.862 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.864 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.864 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.864 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.864 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.864 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.864 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.864 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.864 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.865 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.865 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.865 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.865 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.865 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.866 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.866 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.866 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.866 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.866 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.866 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.866 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.867 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.867 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.867 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.867 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-16T05:45:46.867 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-16T05:45:46.867 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.867 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-16T05:45:46.868 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.868 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.868 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T05:45:46.868 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-16T05:45:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-16T05:45:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-16T05:45:46.870 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.870 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.870 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.870 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T05:45:46.870 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.870 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.871 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-16T05:45:46.871 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-16T05:45:46.871 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.871 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.871 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-16T05:45:46.871 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.872 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-16T05:45:46.872 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.872 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.872 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-16T05:45:46.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-16T05:45:46.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.873 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.873 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.873 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-16T05:45:46.873 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.873 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T05:45:46.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.874 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.874 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-16T05:45:46.874 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-16T05:45:46.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.875 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:46.875 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.875 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.875 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.875 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T05:45:46.875 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.875 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.876 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:46.876 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:46.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.876 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.876 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:46.876 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.877 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.877 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.877 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T05:45:46.877 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.877 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.877 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-16T05:45:46.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-16T05:45:46.878 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.878 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.878 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.878 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-16T05:45:46.878 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.878 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.878 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.878 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T05:45:46.879 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.879 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.879 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.879 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-16T05:45:46.880 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.880 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.880 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.880 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.880 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.880 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.880 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.880 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.880 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.881 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.881 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.881 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.881 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.881 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.881 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.881 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.881 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.882 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.882 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.882 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.882 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.882 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.882 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.883 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.883 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.883 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.883 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.883 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.883 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.883 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.884 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.884 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.884 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.884 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.885 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.885 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.885 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.885 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.885 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.885 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.885 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.886 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.886 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.886 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.886 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.886 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.886 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.887 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.887 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.887 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.887 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.887 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.887 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.888 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.888 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.888 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.888 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.888 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-16T05:45:46.888 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-16T05:45:46.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.889 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.889 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.889 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-16T05:45:46.889 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.889 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.889 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.889 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.890 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.890 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.890 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-16T05:45:46.890 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-16T05:45:46.890 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.890 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.890 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.891 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-16T05:45:46.891 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.891 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.891 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.891 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.891 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.892 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:46.892 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:46.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.892 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.892 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:46.892 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.893 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.893 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.893 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.893 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.893 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.893 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.893 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-16T05:45:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-16T05:45:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-16T05:45:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-16T05:45:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-16T05:45:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-16T05:45:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T05:45:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-16T05:45:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-16T05:45:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-16T05:45:46.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T05:45:46.897 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.897 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.897 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.898 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-16T05:45:46.898 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-16T05:45:46.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.898 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.898 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.898 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.899 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.899 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-16T05:45:46.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.899 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:46.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:46.900 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:46.900 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.900 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-16T05:45:46.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-16T05:45:46.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.900 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T05:45:46.901 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.901 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.901 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.902 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.902 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.902 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.902 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.902 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.902 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.902 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.903 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.903 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.903 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.903 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.903 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.903 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.904 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.904 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.904 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.904 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.904 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.904 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.904 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.905 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.905 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.905 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.905 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.905 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.905 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.905 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.907 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.907 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.907 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.907 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.907 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.907 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.907 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.908 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.908 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.908 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.908 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.908 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.909 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.909 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.909 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.909 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.909 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-16T05:45:46.910 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-16T05:45:46.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.910 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.910 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-16T05:45:46.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.911 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-16T05:45:46.911 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-16T05:45:46.911 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.912 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.912 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-16T05:45:46.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.912 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.912 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-16T05:45:46.912 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.912 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.913 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.913 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-16T05:45:46.913 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-16T05:45:46.913 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.913 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.913 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.914 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-16T05:45:46.914 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.914 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.914 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:46.914 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.914 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.915 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-16T05:45:46.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-16T05:45:46.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.915 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.915 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:46.915 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:46.915 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-16T05:45:46.916 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.916 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T05:45:46.916 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.916 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.916 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-16T05:45:46.916 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-16T05:45:46.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.917 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.917 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.917 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.917 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-16T05:45:46.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.918 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-16T05:45:46.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-16T05:45:46.918 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.918 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.918 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-16T05:45:46.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.919 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:46.919 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.919 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.919 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.919 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-16T05:45:46.920 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.920 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.920 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.920 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.921 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.921 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.921 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.922 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.922 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.922 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.922 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.922 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.922 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.922 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.923 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.923 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.924 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.924 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.925 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.925 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.925 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.925 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.926 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.926 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.926 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.926 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.926 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.926 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.926 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.926 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.927 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.927 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.927 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.929 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.929 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-16T05:45:46.929 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.929 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.929 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.930 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.930 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.930 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.930 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.930 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.931 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.931 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.931 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.931 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.931 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.931 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.931 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.931 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.933 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.933 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.933 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.933 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.933 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.933 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.934 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.934 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.934 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.934 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.934 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.934 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.934 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.934 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.935 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.935 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.935 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.935 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.935 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.935 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.935 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.935 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.935 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.936 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.936 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.936 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.936 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.936 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.936 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.937 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.937 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.937 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.938 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.938 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.938 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-16T05:45:46.938 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.939 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.939 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.939 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-16T05:45:46.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-16T05:45:46.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.939 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.939 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-16T05:45:46.940 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.940 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.940 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T05:45:46.940 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.940 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-16T05:45:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-16T05:45:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-16T05:45:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T05:45:46.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.942 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.942 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.942 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.943 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.943 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.943 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.943 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.943 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.943 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.943 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.944 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.944 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.944 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.944 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.944 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.944 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.944 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.944 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.946 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.946 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.946 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.946 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.947 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.947 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.947 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.947 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.948 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.948 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.948 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.948 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.948 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.948 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.948 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.948 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.949 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.950 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.950 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.950 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.950 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.950 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.950 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.950 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-16T05:45:46.951 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.951 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.951 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.951 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-16T05:45:46.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-16T05:45:46.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.952 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-16T05:45:46.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.952 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-16T05:45:46.952 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.954 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.954 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.954 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.954 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.954 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.955 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.955 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.955 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.955 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.956 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.956 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.956 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.956 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.957 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.958 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.958 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.958 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.958 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.958 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.961 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.961 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-16T05:45:46.961 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-16T05:45:46.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.962 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.962 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-16T05:45:46.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T05:45:46.963 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.963 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.963 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-16T05:45:46.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-16T05:45:46.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.964 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-16T05:45:46.964 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.964 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-16T05:45:46.964 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.964 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.965 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-16T05:45:46.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-16T05:45:46.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:46.965 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-16T05:45:46.965 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.965 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.966 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.966 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T05:45:46.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.966 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.966 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-16T05:45:46.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-16T05:45:46.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.967 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.967 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-16T05:45:46.967 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-16T05:45:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-16T05:45:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-16T05:45:46.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.969 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T05:45:46.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T05:45:46.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.969 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-16T05:45:46.970 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.970 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.970 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.970 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.971 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.971 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.972 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.972 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.972 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.972 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.972 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.972 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.972 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.975 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.975 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.975 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.975 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.975 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.975 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.975 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.975 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.975 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.978 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.978 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.978 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.978 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.978 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.978 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-16T05:45:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.980 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.980 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.980 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.980 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.980 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.980 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.980 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.981 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.981 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.981 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.981 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.981 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.981 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.981 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.982 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.982 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.982 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.983 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.983 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.983 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.983 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.983 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.984 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.984 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.984 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.985 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.985 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.985 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.985 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.985 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.985 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.985 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.985 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.986 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.986 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.986 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.986 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.986 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.986 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.987 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.987 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.987 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.987 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.987 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.987 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.988 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.988 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.988 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.988 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.988 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.988 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-16T05:45:46.988 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.989 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.989 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.989 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:46.989 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:46.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.989 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.989 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:46.989 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.990 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.990 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.990 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.990 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.990 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.990 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.990 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.990 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.991 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.991 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:46.991 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:46.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.992 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.992 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.992 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.992 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.993 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:46.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:46.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:46.993 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.994 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:46.994 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.994 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.994 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:46.994 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:46.994 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:46.994 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:46.994 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:46.994 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:46.995 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:46.995 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:46.995 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.995 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.995 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.995 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.995 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:46.996 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:46.996 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.996 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.996 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:46.996 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.996 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.996 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:46.997 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.997 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.997 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.997 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:46.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:46.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:46.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-16T05:45:46.997 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:46.998 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:46.998 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:46.998 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-16T05:45:46.998 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-16T05:45:46.998 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:46.998 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:46.998 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:46.998 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T05:45:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.005 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.005 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.005 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.005 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.005 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.005 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.005 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.005 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.006 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.006 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.006 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.006 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.006 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.008 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.008 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-16T05:45:47.008 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-16T05:45:47.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.008 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-16T05:45:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-16T05:45:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-16T05:45:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-16T05:45:47.010 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-16T05:45:47.011 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.011 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.011 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.011 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.011 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.011 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.011 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.012 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.012 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.012 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.012 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.012 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.012 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.012 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.012 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.012 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.019 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.019 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.019 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.019 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.019 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-16T05:45:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-16T05:45:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-16T05:45:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-16T05:45:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-16T05:45:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-16T05:45:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-16T05:45:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T05:45:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-16T05:45:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-16T05:45:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-16T05:45:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T05:45:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-16T05:45:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-16T05:45:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-16T05:45:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-16T05:45:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-16T05:45:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-16T05:45:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-16T05:45:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-16T05:45:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-16T05:45:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-16T05:45:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-16T05:45:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-16T05:45:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T05:45:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T05:45:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.062 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.062 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.062 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.062 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.062 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.062 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.062 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.063 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.063 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.063 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.063 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.064 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.064 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.064 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.064 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.064 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.064 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.064 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.065 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.065 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.065 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.065 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.065 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.065 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.066 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.066 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.066 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.066 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.066 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.066 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.067 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.067 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.067 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.067 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.067 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.067 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.067 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.067 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.067 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.068 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.068 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.068 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.069 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.069 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.069 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.069 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.069 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.069 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.069 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-16T05:45:47.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-16T05:45:47.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.070 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.070 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.070 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.070 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.071 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.071 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T05:45:47.071 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T05:45:47.071 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-16T05:45:47.072 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.072 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.072 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.073 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.073 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-16T05:45:47.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-16T05:45:47.073 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.073 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.073 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-16T05:45:47.074 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.074 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.074 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.074 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.074 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.074 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.074 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.075 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-16T05:45:47.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-16T05:45:47.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-16T05:45:47.075 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.075 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T05:45:47.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.076 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.076 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-16T05:45:47.076 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-16T05:45:47.076 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.077 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.077 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-16T05:45:47.077 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.077 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.077 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.077 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T05:45:47.077 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.078 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.078 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-16T05:45:47.078 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-16T05:45:47.078 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.078 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.078 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.079 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-16T05:45:47.079 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.079 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.079 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.079 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.080 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-16T05:45:47.080 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-16T05:45:47.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.080 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-16T05:45:47.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.081 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.081 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.081 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-16T05:45:47.081 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-16T05:45:47.081 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.081 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.081 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.082 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.082 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-16T05:45:47.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:47.082 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.082 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.082 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.082 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-16T05:45:47.082 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-16T05:45:47.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-16T05:45:47.083 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.083 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.083 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.083 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.083 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-16T05:45:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.085 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.085 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.085 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.086 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.086 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.086 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.086 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.086 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.086 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.086 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.086 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.087 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.087 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.087 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.087 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.087 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.088 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.089 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.089 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.090 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.090 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.090 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.090 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.090 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.090 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.090 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.090 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.091 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.091 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.091 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.091 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.091 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.091 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.092 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.092 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.092 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.092 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.092 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.092 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.093 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-16T05:45:47.093 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-16T05:45:47.093 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.093 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.093 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.093 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:47.093 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.094 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.094 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.094 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.094 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.094 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.094 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.094 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-16T05:45:47.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T05:45:47.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.095 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.095 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.095 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.095 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.095 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.095 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.095 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.096 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.096 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.096 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-16T05:45:47.096 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-16T05:45:47.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.097 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.097 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.098 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-16T05:45:47.099 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.099 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.099 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.099 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-16T05:45:47.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-16T05:45:47.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.099 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.100 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.100 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.100 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.100 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.100 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.100 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.100 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.101 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-16T05:45:47.101 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-16T05:45:47.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.101 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.101 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.101 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.102 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.102 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.102 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.102 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.102 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.103 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.103 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.103 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.103 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.103 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.103 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.104 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.104 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.104 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.104 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.104 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.104 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.104 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.104 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.105 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.105 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.105 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.105 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.105 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.106 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.106 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.106 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.107 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.107 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.107 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.107 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.107 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.107 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.108 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.108 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.108 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.108 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.108 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.108 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.108 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.108 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.109 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.109 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.109 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.109 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.110 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.110 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.110 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.110 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.110 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.110 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.111 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.111 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-16T05:45:47.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-16T05:45:47.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.111 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.111 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.111 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.112 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.112 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.112 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.112 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.112 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.112 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.112 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-16T05:45:47.113 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-16T05:45:47.113 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.113 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.113 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.113 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.113 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.113 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.114 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.114 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.114 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.114 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.114 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.114 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-16T05:45:47.115 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.115 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.115 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.115 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.115 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.116 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.116 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.116 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.116 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.116 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.116 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.116 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.116 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.116 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.117 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.117 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.117 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.117 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.117 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.117 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.118 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.118 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.118 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.118 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.118 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.118 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.119 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.119 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.119 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.119 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.119 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.119 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.120 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.120 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.120 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.120 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.120 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.120 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.121 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.121 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.121 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.121 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.121 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.121 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.121 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.121 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.122 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.122 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.122 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.122 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.122 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.122 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.122 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.123 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.123 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.123 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.123 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-16T05:45:47.124 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.124 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.124 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.124 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.125 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.125 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.125 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.125 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.125 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.125 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.125 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.125 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.126 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.126 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.126 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.126 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.126 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.126 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.126 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.127 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.127 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.127 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.127 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.127 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.128 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.128 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.128 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.128 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.128 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.128 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.129 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.129 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.129 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.129 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.129 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.129 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.130 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.130 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.130 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.130 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.130 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.130 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.130 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.130 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.131 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.131 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.131 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.131 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.131 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.131 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.131 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.132 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.132 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.132 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.132 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.132 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.132 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.132 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.132 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.132 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.133 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-16T05:45:47.133 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-16T05:45:47.133 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.133 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.133 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.133 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-16T05:45:47.133 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.134 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.134 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.134 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T05:45:47.134 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.134 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.134 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.134 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-16T05:45:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.136 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.136 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.136 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.136 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.136 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.136 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.136 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.136 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.136 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.137 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.137 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.137 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.137 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.137 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.137 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.137 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.137 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.138 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.138 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.138 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.138 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.138 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.138 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.139 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.139 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.139 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.139 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.139 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.139 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.139 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.139 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.140 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.140 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.140 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.140 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.140 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.140 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.141 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.141 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.141 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.141 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.141 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.141 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.142 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.142 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.142 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.142 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.143 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.143 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.143 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.143 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.143 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.143 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.144 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-16T05:45:47.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-16T05:45:47.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.144 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.144 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.144 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.144 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.145 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.145 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.145 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.145 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.145 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.145 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.145 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-16T05:45:47.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-16T05:45:47.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-16T05:45:47.146 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.146 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.146 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.146 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.147 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.147 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.147 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-16T05:45:47.147 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-16T05:45:47.147 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.147 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.147 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.148 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.148 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.148 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.148 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.148 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.148 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.148 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.148 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.149 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-16T05:45:47.149 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-16T05:45:47.149 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.149 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.149 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.149 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.150 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.150 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.150 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.150 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.150 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.150 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.150 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-16T05:45:47.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-16T05:45:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-16T05:45:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-16T05:45:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.154 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-16T05:45:47.154 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-16T05:45:47.154 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.154 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.154 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.155 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.155 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.155 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.155 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-16T05:45:47.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-16T05:45:47.156 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.156 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.156 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.156 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.156 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.156 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.157 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.157 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.157 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.157 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-16T05:45:47.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-16T05:45:47.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.158 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.158 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:47.158 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-16T05:45:47.158 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.159 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-16T05:45:47.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-16T05:45:47.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.159 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.160 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.160 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.160 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.160 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-16T05:45:47.161 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-16T05:45:47.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.161 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.161 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.161 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.161 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.162 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.162 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.162 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.162 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-16T05:45:47.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-16T05:45:47.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.163 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.163 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.163 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.164 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.164 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-16T05:45:47.164 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.164 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.164 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.165 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-16T05:45:47.165 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-16T05:45:47.165 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.165 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.165 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.165 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.165 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.166 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.166 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T05:45:47.166 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.166 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.166 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.166 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-16T05:45:47.166 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-16T05:45:47.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.167 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.167 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.167 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.167 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.167 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.167 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.167 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-16T05:45:47.168 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.168 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.168 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.168 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.169 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.169 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.169 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.169 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.169 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.169 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.169 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.170 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.170 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.170 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.170 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.170 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.170 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.170 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.171 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.171 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.172 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.172 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.172 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.173 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.173 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.173 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.173 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.173 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.173 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.173 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.173 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.174 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.174 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.174 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.174 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.174 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.174 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.174 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.176 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.176 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.176 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.176 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.176 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-16T05:45:47.177 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.177 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.177 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.177 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.178 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.178 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.179 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.179 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.179 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.179 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.179 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.179 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.179 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.181 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.181 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.181 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.181 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.181 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.181 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.182 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.182 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.183 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.183 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.183 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.183 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.183 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.183 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.183 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.183 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.184 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.184 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.184 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.184 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.185 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.185 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.185 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.185 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.185 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.185 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.185 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.185 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.185 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.186 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.186 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.186 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-16T05:45:47.186 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.186 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.186 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.187 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.187 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.187 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.187 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.187 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.187 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.187 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.188 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.188 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.188 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.188 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.188 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.188 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.188 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.188 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.188 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.189 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.189 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.189 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.191 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.191 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.191 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.191 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.191 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.192 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.192 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.192 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.192 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.192 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.192 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.192 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.192 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.192 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.193 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.193 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.193 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.193 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.193 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.194 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.194 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.194 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.195 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-16T05:45:47.195 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.196 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.196 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.196 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-16T05:45:47.196 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-16T05:45:47.196 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.196 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.196 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.196 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T05:45:47.197 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.197 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.197 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.197 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.197 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.198 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:47.198 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:47.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.198 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-16T05:45:47.198 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.198 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.199 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.199 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.199 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.199 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.199 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.200 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.200 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.200 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.200 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.200 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.200 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.200 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.201 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.201 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.201 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.201 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.201 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.201 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.201 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.202 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.202 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.202 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.203 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.203 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.203 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.203 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.203 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.203 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.204 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.204 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.204 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.205 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.205 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.205 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.205 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.205 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.205 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.205 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.205 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.206 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.206 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.207 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.207 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.207 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.207 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.207 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.208 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.208 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.208 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-16T05:45:47.208 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.208 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.208 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.209 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-16T05:45:47.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-16T05:45:47.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.209 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.209 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.209 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.209 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.210 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T05:45:47.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.210 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.210 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-16T05:45:47.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-16T05:45:47.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.211 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.211 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T05:45:47.211 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.212 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.212 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-16T05:45:47.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-16T05:45:47.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.212 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.213 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.213 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.213 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.213 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T05:45:47.213 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.213 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.214 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-16T05:45:47.214 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-16T05:45:47.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.214 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.214 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.214 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.215 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.215 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T05:45:47.215 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.215 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.215 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.215 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-16T05:45:47.215 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-16T05:45:47.216 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.216 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.216 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.216 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.216 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.216 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T05:45:47.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.217 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.217 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-16T05:45:47.217 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-16T05:45:47.217 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.217 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.217 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.218 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.218 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.218 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.218 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.219 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-16T05:45:47.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-16T05:45:47.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.219 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.219 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.219 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.219 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.220 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.220 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.220 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.220 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.220 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.220 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.220 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-16T05:45:47.220 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-16T05:45:47.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.221 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.221 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T05:45:47.221 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.221 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.221 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.221 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.221 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.222 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-16T05:45:47.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-16T05:45:47.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.222 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.223 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.223 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.223 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.223 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.223 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.223 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.224 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:47.224 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:47.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.224 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.224 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.224 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-16T05:45:47.224 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-16T05:45:47.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.225 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.225 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.225 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.225 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-16T05:45:47.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-16T05:45:47.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.226 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.226 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.226 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-16T05:45:47.226 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.226 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.226 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.227 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.227 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.227 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-16T05:45:47.227 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-16T05:45:47.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.228 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.228 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.228 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.228 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.228 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.228 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.228 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.229 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.229 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-16T05:45:47.229 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-16T05:45:47.229 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.229 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.229 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.229 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.230 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.231 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.231 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.232 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.232 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.232 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.232 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.232 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.232 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.232 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.232 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.233 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.233 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.233 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.233 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.234 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.234 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.234 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.234 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.234 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.234 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.235 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.235 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.235 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.235 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.235 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.236 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.236 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.236 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.236 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.236 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.236 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.236 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.236 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.237 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.237 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.237 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.237 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.237 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.237 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.238 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.238 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.238 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.238 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.239 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.239 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-16T05:45:47.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-16T05:45:47.239 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.239 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.240 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.240 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.240 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.240 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.240 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.240 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.240 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.240 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.241 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-16T05:45:47.241 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-16T05:45:47.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.241 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.242 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.242 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.242 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-16T05:45:47.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-16T05:45:47.243 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T05:45:47.243 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.243 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.243 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.243 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.244 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.244 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-16T05:45:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.246 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.246 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.246 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.246 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.246 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.246 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.246 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.247 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.247 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.247 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.247 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.247 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.247 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.247 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.248 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.248 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.248 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.248 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.248 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.248 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.248 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.251 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.251 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.251 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.251 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.251 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.251 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.251 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.252 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.252 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.252 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.253 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.253 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.253 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.253 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-16T05:45:47.254 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.254 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.254 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.254 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.255 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.255 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.255 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.255 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.255 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.255 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.256 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.256 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.256 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.256 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.256 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.256 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.256 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.256 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.257 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.258 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.258 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.258 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.258 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.258 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.258 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.259 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.259 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.259 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.259 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.260 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.260 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.260 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.260 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.260 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.260 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.260 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.260 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.261 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.261 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.261 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.261 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.261 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.261 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.262 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.262 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.262 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.262 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.262 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.265 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-16T05:45:47.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-16T05:45:47.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.266 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.266 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-16T05:45:47.266 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.266 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.266 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.266 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T05:45:47.266 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.267 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-16T05:45:47.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-16T05:45:47.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.267 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.267 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.268 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.268 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-16T05:45:47.268 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.268 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.268 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.269 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-16T05:45:47.269 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-16T05:45:47.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.269 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.269 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.269 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:47.269 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.269 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.269 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.270 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T05:45:47.270 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.270 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.270 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-16T05:45:47.270 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T05:45:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T05:45:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.272 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-16T05:45:47.272 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-16T05:45:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-16T05:45:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-16T05:45:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.274 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.274 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.274 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.274 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.274 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-16T05:45:47.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-16T05:45:47.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.275 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.275 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.275 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.275 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.276 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.276 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.276 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.276 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:47.276 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:47.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.276 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.277 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.277 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:47.277 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.277 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.277 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.277 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.278 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.278 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.278 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.278 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.278 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.280 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.280 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.280 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.281 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.281 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.281 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.281 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.281 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.282 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.282 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.282 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.282 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.283 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.283 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.283 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.283 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.283 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.283 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-16T05:45:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-16T05:45:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.288 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-16T05:45:47.288 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-16T05:45:47.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.288 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.288 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-16T05:45:47.288 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.289 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.289 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.289 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.289 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.289 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.289 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-16T05:45:47.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-16T05:45:47.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.290 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T05:45:47.290 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.290 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.290 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.290 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.290 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.291 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.291 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.291 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.291 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.291 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:47.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 25037, 2024-04-16T05:45:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "y", 2024-04-16T05:45:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-04-16T05:45:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-16T05:45:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-16T05:45:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.293 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:47.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:47.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.293 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.293 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.293 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:47.293 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.293 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.293 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.294 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T05:45:47.294 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.294 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.294 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.298 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.299 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.299 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.299 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.299 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.299 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.300 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.300 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.300 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.300 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.300 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.300 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.300 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.301 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.301 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.301 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.301 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.302 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.302 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.302 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.302 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.302 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.302 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.302 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.302 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.303 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.304 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.304 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.304 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.304 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.305 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.305 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.305 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.305 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.305 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.305 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.305 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.305 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.306 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.306 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.306 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.306 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.307 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.307 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.307 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.307 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.307 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.307 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.308 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.308 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.308 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-16T05:45:47.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-16T05:45:47.308 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.308 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-16T05:45:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.310 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-16T05:45:47.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-16T05:45:47.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.310 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.310 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-16T05:45:47.310 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.311 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-16T05:45:47.311 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.311 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.311 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.311 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-16T05:45:47.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-16T05:45:47.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.312 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.312 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.312 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.312 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.313 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-16T05:45:47.313 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.313 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.313 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-16T05:45:47.313 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-16T05:45:47.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.313 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.314 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.314 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.314 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.314 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.314 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-16T05:45:47.314 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.315 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.315 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.315 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-16T05:45:47.315 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-16T05:45:47.315 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.315 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.315 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.316 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-16T05:45:47.316 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.316 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.316 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.316 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-16T05:45:47.316 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.316 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.316 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.317 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-16T05:45:47.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-16T05:45:47.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.317 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.317 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.317 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.317 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.318 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.318 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-16T05:45:47.318 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.318 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.318 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.318 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-16T05:45:47.318 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-16T05:45:47.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.319 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.319 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.319 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.319 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.319 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T05:45:47.320 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.320 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.320 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.320 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-16T05:45:47.320 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-16T05:45:47.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.320 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.321 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.321 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.321 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.321 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-16T05:45:47.321 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.321 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.322 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.322 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.322 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-16T05:45:47.322 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.322 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.323 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.323 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-16T05:45:47.323 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-16T05:45:47.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.323 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.323 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.323 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.323 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.324 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.324 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.324 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T05:45:47.324 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.324 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.324 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.324 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-16T05:45:47.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-16T05:45:47.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.325 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.325 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-16T05:45:47.325 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.325 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.325 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.326 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T05:45:47.326 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:47.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.326 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.326 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-16T05:45:47.326 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-16T05:45:47.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:47.327 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.327 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.327 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:47.327 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-16T05:45:47.327 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-16T05:45:47.327 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.327 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T05:45:47.327 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:47.328 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.328 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.328 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-16T05:45:47.328 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-16T05:45:47.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:47.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.329 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-16T05:45:47.329 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:47.329 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-16T05:45:47.329 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T05:45:47.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.330 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-16T05:45:47.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-16T05:45:47.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.331 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T05:45:47.331 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.331 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.331 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T05:45:47.331 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T05:45:47.331 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.332 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.332 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-16T05:45:47.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-16T05:45:47.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.332 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.332 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-16T05:45:47.332 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-16T05:45:47.333 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-16T05:45:47.333 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.333 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-16T05:45:47.333 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T05:45:47.333 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.333 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-16T05:45:47.334 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-16T05:45:47.334 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.334 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.334 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.334 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-16T05:45:47.334 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.334 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.334 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T05:45:47.335 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:47.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.335 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.335 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-16T05:45:47.335 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-16T05:45:47.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:47.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.336 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:47.336 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-16T05:45:47.336 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-16T05:45:47.336 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.336 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T05:45:47.336 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:47.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.337 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.337 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.337 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.337 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.337 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.338 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.338 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.338 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.338 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.338 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.338 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.338 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.338 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.339 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.339 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.339 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.339 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.339 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.339 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.339 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.339 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.340 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.340 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.340 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.340 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.340 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.340 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.340 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.341 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.341 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.342 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.342 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.343 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.343 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.343 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.343 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.343 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.343 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.344 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.344 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.344 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.344 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.344 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-16T05:45:47.346 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-16T05:45:47.346 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.346 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.346 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.346 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:47.346 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.346 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.346 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.347 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T05:45:47.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.347 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-16T05:45:47.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-16T05:45:47.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.348 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.348 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-16T05:45:47.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.348 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-16T05:45:47.348 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-16T05:45:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-16T05:45:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-16T05:45:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-16T05:45:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-16T05:45:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-16T05:45:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-16T05:45:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T05:45:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:47.351 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T05:45:47.353 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.353 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.353 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.353 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-16T05:45:47.353 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-16T05:45:47.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.353 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.354 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.354 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:47.354 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:47.354 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.354 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.354 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T05:45:47.354 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T05:45:47.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.355 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.355 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-16T05:45:47.355 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-16T05:45:47.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:47.355 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.355 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.355 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:47.356 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.356 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.356 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.356 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T05:45:47.356 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.356 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.357 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.357 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-16T05:45:47.357 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.357 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.357 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.357 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-16T05:45:47.358 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-16T05:45:47.358 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.358 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.358 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.358 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-16T05:45:47.358 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.358 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.358 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.359 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T05:45:47.359 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.359 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.359 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.359 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-16T05:45:47.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-16T05:45:47.360 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.360 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.360 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.360 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-16T05:45:47.360 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.360 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.360 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.361 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T05:45:47.361 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.361 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.361 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.361 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-16T05:45:47.361 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-16T05:45:47.361 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.361 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.362 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-16T05:45:47.362 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.362 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.362 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.362 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T05:45:47.362 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.363 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-16T05:45:47.363 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-16T05:45:47.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.363 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.363 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.364 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.364 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T05:45:47.364 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T05:45:47.364 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.364 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.365 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-16T05:45:47.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-16T05:45:47.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.365 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.365 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.365 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:47.365 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.365 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.366 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.366 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T05:45:47.366 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.366 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.366 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.366 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-16T05:45:47.366 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T05:45:47.367 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.367 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.367 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.367 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-16T05:45:47.367 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.367 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.367 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.367 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T05:45:47.368 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.368 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.368 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-16T05:45:47.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-16T05:45:47.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.368 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.369 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.369 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.369 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.369 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.369 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-16T05:45:47.369 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-16T05:45:47.369 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-16T05:45:47.370 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.370 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T05:45:47.370 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.370 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.370 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.370 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-16T05:45:47.370 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-16T05:45:47.371 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.371 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.371 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.371 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.371 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.371 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.371 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.371 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T05:45:47.371 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.372 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.372 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.372 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-16T05:45:47.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-16T05:45:47.372 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.372 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.372 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.373 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.373 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.373 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.373 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.373 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T05:45:47.373 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.373 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.373 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.374 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-16T05:45:47.374 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-16T05:45:47.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.374 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.374 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.374 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.375 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.375 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.375 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T05:45:47.375 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.375 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.375 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.375 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-16T05:45:47.376 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-16T05:45:47.376 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.376 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.376 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.376 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T05:45:47.376 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.376 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.376 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-16T05:45:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-16T05:45:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.378 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.378 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T05:45:47.378 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.378 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.378 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.378 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-16T05:45:47.378 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.378 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.379 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.379 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-16T05:45:47.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-16T05:45:47.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.379 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.379 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.379 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T05:45:47.379 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.380 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.380 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.380 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-16T05:45:47.380 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.380 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.380 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.380 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-16T05:45:47.381 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-16T05:45:47.381 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.381 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.381 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.381 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T05:45:47.381 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.381 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.381 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.382 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:47.382 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.382 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.382 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.382 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-16T05:45:47.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-16T05:45:47.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.383 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.383 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.383 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T05:45:47.383 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.383 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.383 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-16T05:45:47.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.383 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.384 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.384 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-16T05:45:47.384 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-16T05:45:47.384 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.384 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.384 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.385 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T05:45:47.385 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.385 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.385 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.385 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-16T05:45:47.385 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.385 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.385 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.386 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-16T05:45:47.386 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-16T05:45:47.386 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.386 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.386 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.386 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T05:45:47.387 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.387 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.387 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.387 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-16T05:45:47.387 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.387 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.387 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-16T05:45:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-16T05:45:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T05:45:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.389 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.389 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-16T05:45:47.389 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.389 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.389 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.389 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-16T05:45:47.389 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-16T05:45:47.390 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.390 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.390 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.390 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T05:45:47.390 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.390 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.390 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.390 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-16T05:45:47.391 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.391 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.391 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.391 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-16T05:45:47.391 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-16T05:45:47.391 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.391 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.392 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.392 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T05:45:47.392 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.392 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.392 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.392 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-16T05:45:47.392 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.392 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.392 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.393 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-16T05:45:47.393 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-16T05:45:47.393 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.393 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.393 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.393 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T05:45:47.393 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.394 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.394 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.394 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:47.394 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.394 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.394 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.394 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-16T05:45:47.395 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-16T05:45:47.395 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.395 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.395 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.395 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T05:45:47.395 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.395 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.395 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.396 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-16T05:45:47.396 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.396 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.396 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.396 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-16T05:45:47.396 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-16T05:45:47.396 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.396 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.397 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.397 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T05:45:47.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.397 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.397 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.397 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:47.397 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.398 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.398 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-16T05:45:47.398 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-16T05:45:47.398 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.398 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.398 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.398 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T05:45:47.398 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.399 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.399 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.399 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-16T05:45:47.399 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.399 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.399 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.399 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-16T05:45:47.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-16T05:45:47.400 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.400 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.400 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.400 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T05:45:47.400 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.400 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.400 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.401 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-16T05:45:47.401 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.401 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.401 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.401 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-16T05:45:47.401 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-16T05:45:47.401 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.402 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.402 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.402 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T05:45:47.402 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.402 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.402 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.402 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-16T05:45:47.402 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.403 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.403 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.403 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-16T05:45:47.403 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-16T05:45:47.403 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.403 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.403 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.404 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.404 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.404 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.404 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.404 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T05:45:47.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.404 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.404 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.405 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-16T05:45:47.405 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-16T05:45:47.405 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.405 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.405 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:47.405 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.406 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.406 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.406 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T05:45:47.406 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.406 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-16T05:45:47.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T05:45:47.407 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.407 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.407 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.407 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-16T05:45:47.407 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.407 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.407 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.408 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T05:45:47.408 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.408 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.408 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.408 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-16T05:45:47.408 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-16T05:45:47.408 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.409 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.409 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.409 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-16T05:45:47.409 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.409 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.409 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.409 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T05:45:47.409 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.410 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.410 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-16T05:45:47.410 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-16T05:45:47.410 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.410 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.410 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.411 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-16T05:45:47.411 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.411 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.411 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.411 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-16T05:45:47.411 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.411 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.411 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.412 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-16T05:45:47.412 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-16T05:45:47.412 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.412 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.412 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.412 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.412 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.412 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.413 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.413 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T05:45:47.413 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.413 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.413 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.413 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-16T05:45:47.413 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-16T05:45:47.414 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.414 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.414 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.414 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:47.414 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.414 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.414 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.414 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-16T05:45:47.415 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.415 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.415 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.415 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-16T05:45:47.415 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-16T05:45:47.415 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.415 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.415 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.416 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T05:45:47.416 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.416 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.416 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.416 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T05:45:47.416 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.416 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.416 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.417 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-16T05:45:47.417 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-16T05:45:47.417 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.417 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.417 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.417 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:47.417 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.417 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.418 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.418 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-16T05:45:47.418 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.418 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.418 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.418 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-16T05:45:47.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-16T05:45:47.419 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.419 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.419 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.419 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.419 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.419 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.419 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T05:45:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-16T05:45:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-16T05:45:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.421 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.421 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.421 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.421 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.421 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T05:45:47.421 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.421 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.421 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.422 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.422 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.422 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.422 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.422 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.423 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.423 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.423 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.423 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.423 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.423 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.423 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.424 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.424 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.424 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.424 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.424 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.424 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-16T05:45:47.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-16T05:45:47.425 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.425 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.425 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.425 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.425 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.425 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.425 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.425 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T05:45:47.426 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.426 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.426 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.426 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.426 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.426 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.427 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.427 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.427 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.427 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T05:45:47.427 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.427 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.427 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.428 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.428 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.428 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.428 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.428 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.428 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.428 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.429 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.429 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.429 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.429 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.429 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.429 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.429 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.429 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.430 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.430 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.430 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.430 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.430 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T05:45:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T05:45:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T05:45:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T05:45:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T05:45:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-16T05:45:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T05:45:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-16T05:45:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-16T05:45:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-16T05:45:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.437 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.437 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.437 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-16T05:45:47.437 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-16T05:45:47.437 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.437 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.437 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T05:45:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-16T05:45:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-16T05:45:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-16T05:45:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-16T05:45:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-16T05:45:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-16T05:45:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-16T05:45:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-16T05:45:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T05:45:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-16T05:45:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-16T05:45:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-16T05:45:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T05:45:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-16T05:45:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-16T05:45:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T05:45:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T05:45:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-16T05:45:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-16T05:45:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T05:45:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-16T05:45:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-16T05:45:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T05:45:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-16T05:45:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-16T05:45:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T05:45:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-16T05:45:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-16T05:45:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T05:45:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-16T05:45:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-16T05:45:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T05:45:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-16T05:45:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-16T05:45:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-16T05:45:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-16T05:45:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-16T05:45:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-16T05:45:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T05:45:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-16T05:45:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-16T05:45:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T05:45:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-16T05:45:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-16T05:45:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T05:45:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-16T05:45:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-16T05:45:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T05:45:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-16T05:45:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-16T05:45:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T05:45:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-16T05:45:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-16T05:45:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T05:45:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-16T05:45:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-16T05:45:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-16T05:45:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-16T05:45:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T05:45:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-16T05:45:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-16T05:45:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-16T05:45:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T05:45:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-16T05:45:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T05:45:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T05:45:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T05:45:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T05:45:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T05:45:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T05:45:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-16T05:45:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T05:45:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T05:45:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T05:45:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T05:45:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T05:45:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T05:45:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T05:45:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T05:45:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T05:45:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T05:45:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T05:45:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T05:45:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T05:45:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T05:45:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T05:45:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T05:45:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T05:45:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T05:45:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T05:45:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T05:45:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T05:45:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T05:45:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T05:45:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-16T05:45:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T05:45:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T05:45:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-16T05:45:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-16T05:45:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T05:45:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T05:45:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-16T05:45:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T05:45:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T05:45:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T05:45:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T05:45:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T05:45:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T05:45:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T05:45:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T05:45:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T05:45:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T05:45:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-16T05:45:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T05:45:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T05:45:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-16T05:45:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T05:45:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T05:45:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T05:45:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T05:45:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T05:45:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T05:45:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T05:45:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T05:45:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T05:45:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T05:45:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T05:45:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T05:45:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T05:45:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T05:45:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T05:45:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T05:45:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T05:45:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T05:45:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T05:45:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T05:45:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T05:45:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T05:45:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T05:45:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T05:45:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T05:45:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T05:45:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T05:45:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-16T05:45:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T05:45:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T05:45:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-16T05:45:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T05:45:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T05:45:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T05:45:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T05:45:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-16T05:45:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.549 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.549 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.549 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.549 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.549 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T05:45:47.549 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T05:45:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T05:45:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T05:45:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.553 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T05:45:47.553 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T05:45:47.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.553 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.553 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.553 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.554 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.554 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.554 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.554 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.554 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T05:45:47.554 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T05:45:47.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.555 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.555 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:47.555 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.555 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.555 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.555 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.556 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.556 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.556 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T05:45:47.556 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T05:45:47.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.556 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.557 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:47.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.557 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.557 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.557 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.557 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.557 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.558 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T05:45:47.558 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T05:45:47.558 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.558 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.558 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:47.559 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.559 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.559 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.559 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.559 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.559 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.560 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-16T05:45:47.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-16T05:45:47.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.560 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.560 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.560 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.561 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.561 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.561 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.561 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-16T05:45:47.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-16T05:45:47.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.562 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.562 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.562 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.562 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.562 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.562 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.562 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.563 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.563 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.563 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.563 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-16T05:45:47.563 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-16T05:45:47.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.564 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.564 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.564 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.564 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.564 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-16T05:45:47.564 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.565 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.565 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-16T05:45:47.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-16T05:45:47.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-16T05:45:47.566 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.566 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.566 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.566 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.566 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.566 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.567 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-16T05:45:47.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-16T05:45:47.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.568 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.568 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.568 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.568 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.568 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.570 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.570 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.570 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.570 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.570 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.570 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.570 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-16T05:45:47.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-16T05:45:47.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-16T05:45:47.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.648 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-16T05:45:47.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-16T05:45:47.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.649 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-16T05:45:47.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.649 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-16T05:45:47.649 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.649 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.650 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-16T05:45:47.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-16T05:45:47.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.650 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.650 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-16T05:45:47.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.651 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:47.651 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.651 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.651 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.652 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-16T05:45:47.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-16T05:45:47.652 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.652 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.652 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.652 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:47.652 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:47.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-16T05:45:47.653 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T05:45:47.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.653 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.653 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-16T05:45:47.653 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-16T05:45:47.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.654 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.654 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-16T05:45:47.654 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.655 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.655 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-16T05:45:47.655 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-16T05:45:47.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.655 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.656 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-16T05:45:47.656 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.656 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.656 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:47.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.656 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.657 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-16T05:45:47.657 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.657 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.657 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.657 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.658 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.658 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.658 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.658 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.658 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.658 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.659 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.659 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.659 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.659 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.659 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.659 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.659 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.660 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.660 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.660 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.660 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.661 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.661 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.661 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.661 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.661 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.661 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.661 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.661 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.662 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.662 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.662 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.662 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.662 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.662 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.663 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.663 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.663 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.663 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.663 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.663 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.663 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.663 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.663 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.664 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.664 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.664 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.664 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.664 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.664 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.665 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.665 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.665 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.665 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.665 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.665 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.666 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.666 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.666 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-16T05:45:47.666 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.666 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.667 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.667 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.667 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.667 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.667 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.667 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.667 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.667 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.668 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.668 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.668 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.668 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.668 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.668 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.668 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.668 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.668 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.669 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.669 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.669 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.669 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.670 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.670 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.670 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.670 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.670 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.670 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.670 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.671 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.671 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.671 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.671 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.672 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.672 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.672 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.672 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.672 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.673 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.673 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.673 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.673 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.673 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.673 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.673 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.674 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.674 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.674 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.674 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.674 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.674 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.674 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.675 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-16T05:45:47.676 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.676 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.676 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.676 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-16T05:45:47.676 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-16T05:45:47.676 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.676 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.676 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-16T05:45:47.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.677 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.677 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T05:45:47.677 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.677 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.677 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.678 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-16T05:45:47.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-16T05:45:47.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.678 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.678 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:47.678 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-16T05:45:47.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T05:45:47.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.679 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.680 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.680 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.680 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.680 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.680 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.680 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.680 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.681 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.681 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.681 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.681 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.681 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.681 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.681 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.681 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.682 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.682 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.682 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.682 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.683 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.683 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.683 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.683 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.684 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.684 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.684 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.684 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.684 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.684 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.685 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.685 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.685 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.685 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.685 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.685 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.685 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.685 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.686 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.686 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.686 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.686 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.686 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.687 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.687 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.687 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.688 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.688 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-16T05:45:47.688 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.688 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.689 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.689 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-16T05:45:47.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-16T05:45:47.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.689 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.689 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-16T05:45:47.689 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.690 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-16T05:45:47.690 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T05:45:47.690 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.690 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.690 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.691 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.691 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.691 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.691 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.691 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.691 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.692 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.692 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.692 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.692 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.692 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.692 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.692 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.692 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.693 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.693 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.693 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.693 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.693 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.693 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.694 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.694 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.694 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.694 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.694 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.695 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.695 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.695 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.695 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.695 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.696 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.696 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.696 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.696 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.696 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.698 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.698 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.698 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.699 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.699 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.700 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.700 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.700 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.700 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-16T05:45:47.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-16T05:45:47.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.701 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.701 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.701 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.702 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.702 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-16T05:45:47.702 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T05:45:47.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.702 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.702 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-16T05:45:47.702 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-16T05:45:47.703 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.703 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.703 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-16T05:45:47.703 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.703 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-16T05:45:47.704 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.704 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.704 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.704 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-16T05:45:47.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-16T05:45:47.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.705 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.705 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-16T05:45:47.705 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.705 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.705 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.705 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T05:45:47.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.706 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-16T05:45:47.706 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-16T05:45:47.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.706 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.707 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.707 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.707 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-16T05:45:47.707 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.707 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.707 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.707 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-16T05:45:47.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-16T05:45:47.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.708 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.708 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-16T05:45:47.708 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.708 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.708 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.708 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T05:45:47.709 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T05:45:47.709 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.709 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.709 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.709 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.709 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-16T05:45:47.710 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.710 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.710 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.710 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.710 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.710 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.711 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.711 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.711 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.711 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.711 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.711 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.711 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.712 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.712 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.712 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.712 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.712 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.712 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.713 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.713 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.714 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.714 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.714 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.715 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.715 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.716 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.716 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.716 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.716 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.716 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.716 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.716 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.716 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.717 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.717 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.717 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.717 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.717 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.718 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.718 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.718 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.718 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.718 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.718 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.719 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.719 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.719 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-16T05:45:47.719 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.720 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.720 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.720 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.720 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.720 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.720 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.721 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.721 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.721 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.721 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.721 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.721 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.721 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.721 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.721 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.722 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.722 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.722 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.723 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.723 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.723 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.724 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.724 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.724 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.724 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.724 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.725 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.725 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.725 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.725 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.725 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.726 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.726 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.726 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.726 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.726 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.726 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.726 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.727 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.727 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.727 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.727 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.727 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.728 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.728 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.728 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.728 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.729 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.729 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-16T05:45:47.729 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.729 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.729 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.729 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.730 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.730 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.730 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.730 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.730 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.730 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.730 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.730 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.730 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.731 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.731 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.731 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.731 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.731 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.731 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.731 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.731 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.731 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.732 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.732 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.732 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.732 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.732 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.732 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.733 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.733 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.733 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.733 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.733 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.733 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.734 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.734 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.734 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.734 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.734 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.735 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.735 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.735 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.735 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.735 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.735 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.735 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.735 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.736 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.736 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.736 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.736 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.736 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.737 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.737 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.737 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.738 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.738 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.738 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.738 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.738 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.738 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-16T05:45:47.739 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.739 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.739 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.739 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-16T05:45:47.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-16T05:45:47.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.739 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.740 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.740 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.740 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T05:45:47.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.741 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.741 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.741 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.741 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.742 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.742 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.742 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.742 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.742 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.742 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.742 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.742 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.743 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.743 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.743 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.743 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.744 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.744 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.744 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.744 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.744 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.744 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.745 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.745 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.745 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.745 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.746 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.746 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.746 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.746 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.746 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.746 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.746 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.746 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.747 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.747 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.747 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.748 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.748 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.748 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.748 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.748 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.748 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.749 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-16T05:45:47.749 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-16T05:45:47.749 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.749 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.750 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.750 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.750 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-16T05:45:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-16T05:45:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-16T05:45:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-16T05:45:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-16T05:45:47.752 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.752 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.752 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.753 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.753 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.753 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.753 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.753 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.753 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.753 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.753 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.754 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.754 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.754 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.754 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.754 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.754 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.754 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.754 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.756 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.756 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.756 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.756 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.756 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.757 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.757 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.757 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.757 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.757 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.758 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.758 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.758 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.758 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.758 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.758 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.758 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.759 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.759 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.759 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.759 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.759 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.759 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.759 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.759 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.761 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.761 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.761 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-16T05:45:47.761 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.761 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.762 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.762 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.762 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.762 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.762 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.763 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.763 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.763 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.763 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.763 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.763 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.763 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.764 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.764 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.764 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.764 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.764 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.764 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.764 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.765 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.765 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.765 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.765 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.765 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.765 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.765 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.766 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.766 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.766 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.766 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.767 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.767 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.767 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.767 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.767 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.767 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.767 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.767 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.768 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.768 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.768 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.768 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.768 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.768 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.768 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.768 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.769 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.769 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.769 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.769 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.769 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.770 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.770 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.770 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.770 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.770 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.771 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.771 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-16T05:45:47.771 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.771 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.771 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.771 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.771 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.772 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.772 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.772 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.772 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.772 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.773 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.773 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.773 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.773 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.773 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.773 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.773 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.773 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.774 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.774 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.774 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.774 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.775 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.775 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.775 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.775 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.775 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.775 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.776 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.776 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.776 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.776 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.777 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.777 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.777 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.777 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.777 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.778 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.778 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.778 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.778 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.779 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.780 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.780 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.780 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.780 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.780 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.780 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:47.780 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:47.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.781 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.781 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.781 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:47.781 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.781 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.781 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.781 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.782 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.782 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.782 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.782 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-16T05:45:47.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-16T05:45:47.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.782 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.783 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.783 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-16T05:45:47.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-16T05:45:47.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.783 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-16T05:45:47.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T05:45:47.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.783 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-16T05:45:47.784 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.784 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.784 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.785 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-16T05:45:47.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-16T05:45:47.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.785 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.785 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.785 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.785 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T05:45:47.786 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.786 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.786 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-16T05:45:47.786 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-16T05:45:47.787 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.787 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.787 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.787 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.787 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.787 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.787 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.789 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.789 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.789 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.789 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.789 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.789 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.789 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.791 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.791 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.791 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.791 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.792 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.792 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.792 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.792 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.793 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.793 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.793 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.793 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.794 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.794 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.794 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.794 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.794 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.794 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.794 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.795 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.795 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.796 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.796 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.796 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.796 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-16T05:45:47.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-16T05:45:47.797 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.797 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.797 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.797 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-16T05:45:47.797 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-16T05:45:47.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.798 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.798 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.798 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:47.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:47.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.799 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.799 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:47.799 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.799 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-16T05:45:47.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.800 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.800 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.800 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.800 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-16T05:45:47.800 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.800 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.801 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.801 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-16T05:45:47.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-16T05:45:47.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.801 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.801 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.802 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-16T05:45:47.802 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-16T05:45:47.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.803 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.803 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T05:45:47.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T05:45:47.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.804 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.805 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.805 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.805 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.805 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.806 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.806 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.806 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.807 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.807 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.807 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.807 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.807 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.807 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.808 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.808 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.809 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.809 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.809 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.809 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.809 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.809 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.810 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.810 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.810 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.810 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.810 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.810 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.811 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.811 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.811 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.811 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.811 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.811 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.812 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-16T05:45:47.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-16T05:45:47.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.813 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.814 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.814 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.814 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T05:45:47.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T05:45:47.814 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.814 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.814 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.815 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-16T05:45:47.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.815 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.815 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.816 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-16T05:45:47.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-16T05:45:47.816 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.816 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.816 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.816 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-16T05:45:47.816 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.817 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.817 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-16T05:45:47.817 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-16T05:45:47.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.818 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-16T05:45:47.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.818 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.818 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T05:45:47.818 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.819 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-16T05:45:47.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-16T05:45:47.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.819 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.819 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-16T05:45:47.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.820 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T05:45:47.820 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.820 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.820 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.820 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-16T05:45:47.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-16T05:45:47.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-16T05:45:47.821 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.821 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.821 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.821 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.822 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.822 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-16T05:45:47.822 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-16T05:45:47.822 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.822 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.823 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.823 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-16T05:45:47.823 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.823 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.823 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.823 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-16T05:45:47.823 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-16T05:45:47.823 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.823 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.824 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.824 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.824 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.824 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-16T05:45:47.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:47.824 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-16T05:45:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-16T05:45:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-16T05:45:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.826 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.826 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.826 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-16T05:45:47.826 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.826 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.826 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.827 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.827 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.827 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.827 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.828 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.828 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.828 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.828 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.828 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.828 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.830 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.830 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.830 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.830 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.830 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.831 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.831 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.831 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.831 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.831 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.831 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.831 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.832 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.832 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.832 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.832 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.832 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.832 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.832 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.833 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.833 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.833 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.833 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.833 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.833 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.833 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.834 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.834 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.834 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.834 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.834 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.834 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.835 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-16T05:45:47.835 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-16T05:45:47.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.835 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.835 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.835 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:47.836 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.836 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.836 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.836 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.836 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.836 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-16T05:45:47.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T05:45:47.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.837 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.837 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.837 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.837 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.837 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-16T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-16T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.839 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.840 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.840 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.840 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-16T05:45:47.840 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.840 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.840 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-16T05:45:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-16T05:45:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.841 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.842 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.842 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.842 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.842 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-16T05:45:47.842 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-16T05:45:47.842 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.843 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.843 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.843 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.843 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.843 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.843 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.844 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.844 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.844 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.844 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.844 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.845 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.845 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.845 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.845 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.845 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.845 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.845 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.846 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.846 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.846 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.846 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.848 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.848 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.848 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.848 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.848 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.849 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.849 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.849 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.849 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.849 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.849 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.849 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.849 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.850 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.850 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.850 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.850 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.851 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.851 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.851 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.851 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.851 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.852 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-16T05:45:47.852 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-16T05:45:47.852 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.853 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.853 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.853 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.853 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.854 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-16T05:45:47.854 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-16T05:45:47.854 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.854 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.854 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.854 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.855 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.855 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.855 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.855 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.855 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.855 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.856 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.856 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-16T05:45:47.856 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.856 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.856 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.856 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.856 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.857 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.857 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.857 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.857 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.858 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.858 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.858 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.858 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.858 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.858 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.858 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.858 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.859 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.859 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.859 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.859 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.859 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.859 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.860 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.860 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.860 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.860 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.860 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.860 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.862 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.862 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.862 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.862 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.862 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.862 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.862 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.862 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.863 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.863 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.863 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.864 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.864 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.864 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.864 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.864 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.864 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.864 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-16T05:45:47.865 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.865 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.865 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.866 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.866 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.866 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.866 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.866 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.867 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.867 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.867 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.867 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.867 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.867 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.867 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.867 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.867 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.868 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.868 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.868 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.868 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.868 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.870 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.871 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.871 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.871 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.871 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.871 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.871 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.872 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.872 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.872 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.872 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.872 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.873 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.873 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.873 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.873 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.873 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.873 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.874 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.874 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-16T05:45:47.874 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-16T05:45:47.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.874 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-16T05:45:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T05:45:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.876 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-16T05:45:47.876 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.876 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.876 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.876 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.876 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.877 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.877 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.877 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.877 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.877 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.877 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.877 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.877 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.878 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.878 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.878 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.878 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.878 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.878 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.878 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.878 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.879 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.879 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.879 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.879 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.879 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.879 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.880 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.880 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.880 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.880 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.880 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.880 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.881 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.881 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.881 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.881 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.881 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.882 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.882 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.882 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.882 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.882 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.882 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.882 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.882 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.883 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.883 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.883 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.883 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.883 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.884 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.884 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.884 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.884 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.884 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.884 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.885 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.885 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-16T05:45:47.885 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-16T05:45:47.885 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.885 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.886 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.886 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.886 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.886 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.886 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.886 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.887 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-16T05:45:47.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-16T05:45:47.887 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.887 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.887 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.887 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-16T05:45:47.887 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.888 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.889 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.889 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.890 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-16T05:45:47.890 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-16T05:45:47.890 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.890 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.890 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.890 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.890 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-16T05:45:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-16T05:45:47.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.892 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.892 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.892 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.892 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.893 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.893 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.893 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-16T05:45:47.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-16T05:45:47.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:47.893 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.893 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.894 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.894 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.894 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.894 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.894 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.894 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.894 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.895 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-16T05:45:47.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-16T05:45:47.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.895 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.896 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.896 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.896 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.896 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-16T05:45:47.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-16T05:45:47.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.898 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.898 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.898 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-16T05:45:47.898 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-16T05:45:47.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.898 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.899 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.899 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.899 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.899 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.899 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.900 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.900 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-16T05:45:47.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-16T05:45:47.900 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.900 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.900 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.901 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:47.901 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-16T05:45:47.901 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.901 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.901 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.902 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-16T05:45:47.902 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-16T05:45:47.902 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.902 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.902 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.902 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.902 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.903 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.903 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.903 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.903 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-16T05:45:47.903 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-16T05:45:47.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.904 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.904 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.904 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.904 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.904 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.904 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.904 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.904 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.905 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.905 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.905 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-16T05:45:47.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-16T05:45:47.905 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.905 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.905 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.906 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.906 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.906 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.906 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.906 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.906 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.906 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.907 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.907 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.907 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-16T05:45:47.907 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.907 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.907 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.908 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-16T05:45:47.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-16T05:45:47.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.908 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.908 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.908 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.908 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.909 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T05:45:47.909 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.909 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.909 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-16T05:45:47.909 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-16T05:45:47.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.910 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.910 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.910 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-16T05:45:47.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.911 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.911 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.911 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.911 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.912 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.912 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.912 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.912 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.912 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.913 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.913 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.913 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.913 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.913 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.913 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.913 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.913 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.914 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.914 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.914 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.914 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.914 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.917 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.917 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.917 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.917 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.917 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.917 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.918 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.918 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.918 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.918 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.919 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.919 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.919 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.919 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.919 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-16T05:45:47.920 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.920 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.920 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.920 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.921 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.921 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.922 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.922 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.922 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.922 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.922 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.922 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.922 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.922 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.922 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.923 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.923 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.924 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.924 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.924 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.925 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.925 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.925 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.925 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.926 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.926 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.926 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.926 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.926 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.926 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.926 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.926 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.927 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.927 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.928 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.929 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-16T05:45:47.929 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.929 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.929 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.930 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.930 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.930 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.930 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.930 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.931 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.931 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.931 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.931 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.931 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.931 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.931 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.931 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.932 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.932 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.932 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.932 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.932 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.932 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.933 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.933 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.933 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.933 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.933 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.933 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.933 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.934 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.934 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.934 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.934 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.934 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.934 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.934 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.935 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.935 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.935 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.935 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.935 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.935 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.935 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.935 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.936 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.936 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.936 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.936 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.936 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.936 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.936 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.937 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.937 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.937 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.937 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-16T05:45:47.939 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.939 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.939 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.939 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-16T05:45:47.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-16T05:45:47.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.939 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.940 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T05:45:47.940 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.940 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.940 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.940 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.940 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.941 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:47.941 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:47.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:47.941 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-16T05:45:47.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.941 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.942 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.942 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.942 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.943 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.943 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.943 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.943 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.943 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.944 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.944 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.944 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.944 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.944 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.944 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.944 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.944 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.944 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.945 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.945 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.945 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.945 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.945 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.945 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.946 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.946 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.946 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.946 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.947 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.947 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.948 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.948 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.948 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.948 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.948 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.948 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.948 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.948 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.948 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.949 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.949 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.949 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.950 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.950 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.950 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.950 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.950 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.950 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.950 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.951 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.951 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-16T05:45:47.951 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.951 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.951 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.952 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-16T05:45:47.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-16T05:45:47.952 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.952 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T05:45:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-16T05:45:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-16T05:45:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.954 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.954 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.954 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T05:45:47.954 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.955 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.955 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-16T05:45:47.955 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-16T05:45:47.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.955 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:47.956 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.956 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T05:45:47.956 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.957 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-16T05:45:47.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-16T05:45:47.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.957 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.957 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.957 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T05:45:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-16T05:45:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-16T05:45:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.959 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.959 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.959 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.959 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.959 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T05:45:47.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-16T05:45:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-16T05:45:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.961 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.961 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.961 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.962 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-16T05:45:47.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-16T05:45:47.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.962 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.962 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.963 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.963 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.963 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.963 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.963 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-16T05:45:47.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-16T05:45:47.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.964 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.964 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T05:45:47.964 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.964 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.964 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.965 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-16T05:45:47.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-16T05:45:47.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.965 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.966 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.966 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.966 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.966 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.967 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:47.967 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:47.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-16T05:45:47.967 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-16T05:45:47.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.968 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.968 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-16T05:45:47.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-16T05:45:47.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:47.969 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-16T05:45:47.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.969 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.970 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-16T05:45:47.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-16T05:45:47.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.970 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.970 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.970 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.971 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.971 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.971 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.971 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.971 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.971 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-16T05:45:47.972 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-16T05:45:47.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.972 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.972 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.972 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.973 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.973 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.973 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.973 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.974 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.974 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.974 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.974 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.974 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.974 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.974 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.974 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.975 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.975 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.975 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.975 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.975 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.976 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.976 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.976 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.976 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.977 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.977 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.977 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.977 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.977 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.977 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.978 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.978 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.978 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.978 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.978 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.978 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.978 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.979 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.979 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.979 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.979 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.979 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.979 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.979 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.980 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.980 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.980 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.980 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.980 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-16T05:45:47.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-16T05:45:47.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.982 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.982 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.982 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.983 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.983 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.983 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-16T05:45:47.983 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-16T05:45:47.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.983 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.984 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.984 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.984 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.984 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.985 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.985 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-16T05:45:47.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-16T05:45:47.985 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.985 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:47.985 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T05:45:47.986 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.986 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.986 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.986 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.986 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.986 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.986 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.987 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-16T05:45:47.987 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.987 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.987 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.987 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.987 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.988 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.988 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.988 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.988 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.988 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.988 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.989 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.989 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.989 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.989 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.989 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.989 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.989 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.989 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.990 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.990 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.990 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.990 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.991 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.991 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.991 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.992 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:47.992 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:47.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.992 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:47.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.992 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.993 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.993 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.993 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.993 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.993 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.993 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.993 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.993 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.994 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.994 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.994 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.994 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:47.994 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:47.994 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.994 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.994 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.995 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:47.995 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.995 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.995 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:47.995 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.995 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.995 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.995 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:47.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:47.996 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-16T05:45:47.996 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:47.996 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:47.996 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:47.997 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:47.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:47.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:47.997 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:47.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:47.997 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:47.997 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:47.997 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:47.997 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:47.998 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:47.998 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:47.998 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:47.998 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:47.998 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:47.998 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:47.998 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:47.998 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:47.999 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:47.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:47.999 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:47.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:47.999 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:47.999 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:47.999 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:47.999 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.000 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.000 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.000 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.000 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.000 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.000 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.000 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.000 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.000 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.001 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.001 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.001 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.001 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.001 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.002 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.002 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.002 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.002 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.002 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.002 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.002 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.002 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.003 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.003 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.003 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.003 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.004 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.004 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.004 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.005 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.005 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.005 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-16T05:45:48.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-16T05:45:48.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.005 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.006 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-16T05:45:48.006 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.006 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.006 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T05:45:48.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.006 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.007 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.007 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-16T05:45:48.007 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-16T05:45:48.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.008 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.008 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.008 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.008 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-16T05:45:48.008 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.008 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.008 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-16T05:45:48.009 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-16T05:45:48.009 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.009 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.009 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:48.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.009 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.009 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.010 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T05:45:48.010 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.010 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-16T05:45:48.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T05:45:48.010 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.011 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.011 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.011 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.011 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.011 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.011 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T05:45:48.011 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.012 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.012 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.012 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-16T05:45:48.012 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.013 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.013 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.013 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-16T05:45:48.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-16T05:45:48.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:48.013 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.013 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.013 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-16T05:45:48.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.014 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.014 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-16T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-16T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.016 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.016 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.016 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.016 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:48.016 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:48.016 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:48.017 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.017 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.017 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.017 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.017 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.018 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.018 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.018 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.018 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.018 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.018 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.019 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.019 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.019 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.019 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.019 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.019 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.020 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.020 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.020 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.020 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.020 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.020 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.020 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.020 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.021 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.021 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.023 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.023 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.023 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.023 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.023 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.023 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.024 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.024 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.024 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.024 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.024 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.025 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.025 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.026 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.026 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.026 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.026 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.027 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-16T05:45:48.027 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-16T05:45:48.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.028 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.028 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-16T05:45:48.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-16T05:45:48.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.029 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.029 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-16T05:45:48.029 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.029 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.029 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.029 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.030 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.030 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.030 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-16T05:45:48.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-16T05:45:48.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.030 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.031 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.031 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T05:45:48.031 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.031 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.031 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.031 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.031 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.031 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.032 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.032 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.032 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.032 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.032 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:48.032 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.032 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.032 INFO:teuthology.orchestra.run.smithi086.stdout: "modules": [ 2024-04-16T05:45:48.033 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-16T05:45:48.033 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard", 2024-04-16T05:45:48.033 INFO:teuthology.orchestra.run.smithi086.stdout: "iostat", 2024-04-16T05:45:48.033 INFO:teuthology.orchestra.run.smithi086.stdout: "nfs", 2024-04-16T05:45:48.033 INFO:teuthology.orchestra.run.smithi086.stdout: "restful", 2024-04-16T05:45:48.033 INFO:teuthology.orchestra.run.smithi086.stdout: "snap_schedule" 2024-04-16T05:45:48.033 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.033 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-16T05:45:48.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-16T05:45:48.034 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.034 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.034 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.034 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:48.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:48.034 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.034 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.035 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:48.035 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.035 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.035 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T05:45:48.035 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.036 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.036 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.036 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.036 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.036 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.037 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.037 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.037 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.037 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.037 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.038 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.038 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.038 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.038 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.038 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.038 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.039 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.039 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.039 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.039 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.039 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.039 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.039 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.040 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.040 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.040 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.040 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.040 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.040 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.041 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.041 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.041 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.041 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.041 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.041 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.041 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-16T05:45:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-16T05:45:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-16T05:45:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-16T05:45:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-16T05:45:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.047 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.047 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-16T05:45:48.047 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.047 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.047 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.047 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-16T05:45:48.047 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.048 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.048 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-16T05:45:48.048 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-16T05:45:48.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.048 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.048 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.048 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-16T05:45:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-16T05:45:48.050 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-16T05:45:48.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.050 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.050 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.050 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.050 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-16T05:45:48.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.051 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.051 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.051 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-16T05:45:48.051 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-16T05:45:48.051 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.052 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.052 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.052 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-16T05:45:48.052 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.052 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.052 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.052 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-16T05:45:48.052 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.053 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.053 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.053 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-16T05:45:48.053 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-16T05:45:48.053 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.053 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.053 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.054 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.054 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.054 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-16T05:45:48.054 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.054 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.054 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.055 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-16T05:45:48.055 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-16T05:45:48.055 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.055 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.055 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.055 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.055 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.055 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T05:45:48.056 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.056 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.056 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-16T05:45:48.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-16T05:45:48.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.057 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.057 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.057 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.057 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.057 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.057 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-16T05:45:48.058 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.058 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.058 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.058 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.058 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-16T05:45:48.059 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.059 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.059 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.059 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-16T05:45:48.059 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-16T05:45:48.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.059 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.060 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.060 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.060 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.060 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T05:45:48.060 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.060 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.061 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-16T05:45:48.061 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-16T05:45:48.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.061 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.061 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-16T05:45:48.061 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.062 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.062 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.062 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T05:45:48.062 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:48.062 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.062 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.062 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-16T05:45:48.063 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-16T05:45:48.063 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:48.063 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.063 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.063 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:48.063 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-16T05:45:48.063 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-16T05:45:48.063 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T05:45:48.064 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:48.064 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.064 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.064 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-16T05:45:48.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-16T05:45:48.064 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:48.064 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.065 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.065 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-16T05:45:48.065 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:48.065 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-16T05:45:48.065 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.065 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T05:45:48.065 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.065 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.066 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.066 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-16T05:45:48.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-16T05:45:48.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.066 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.066 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T05:45:48.066 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.067 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.067 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.067 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T05:45:48.067 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T05:45:48.067 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.067 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.067 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-16T05:45:48.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-16T05:45:48.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:48.068 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.068 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-16T05:45:48.068 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-16T05:45:48.068 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-16T05:45:48.068 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-16T05:45:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T05:45:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-16T05:45:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-16T05:45:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.070 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-16T05:45:48.070 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.070 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.070 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.070 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T05:45:48.070 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:48.070 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.071 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-16T05:45:48.071 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-16T05:45:48.071 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:48.071 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.071 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.071 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:48.071 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-16T05:45:48.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-16T05:45:48.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.072 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T05:45:48.072 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:48.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.072 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.072 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.072 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.073 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.073 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.073 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.073 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.073 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.073 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.074 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.074 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.074 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.074 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.074 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.074 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.074 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.074 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.075 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.076 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.077 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.077 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.077 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.077 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.077 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.078 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.078 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.078 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.078 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.078 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.078 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.078 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.078 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.079 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.079 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.079 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.079 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.079 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.079 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.080 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.080 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.080 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.080 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.080 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.081 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-16T05:45:48.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-16T05:45:48.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:48.081 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.081 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.081 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:48.082 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.082 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.082 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.082 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T05:45:48.082 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.082 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-16T05:45:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-16T05:45:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-16T05:45:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.084 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.084 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.084 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-16T05:45:48.084 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-16T05:45:48.084 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-16T05:45:48.084 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-16T05:45:48.084 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-16T05:45:48.084 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.084 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-16T05:45:48.085 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.085 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.085 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.085 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-16T05:45:48.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-16T05:45:48.085 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.085 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.086 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.086 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.086 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.086 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.086 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T05:45:48.086 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.086 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.087 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:48.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:48.087 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.087 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.087 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.087 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:48.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.088 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T05:45:48.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.089 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-16T05:45:48.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-16T05:45:48.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.089 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:48.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:48.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.090 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.090 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T05:45:48.090 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T05:45:48.090 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.090 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.090 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-16T05:45:48.091 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-16T05:45:48.091 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:48.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.091 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.091 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:48.091 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T05:45:48.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.094 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.094 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.094 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-16T05:45:48.094 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.094 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.095 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.095 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-16T05:45:48.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-16T05:45:48.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:48.095 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.095 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.095 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-16T05:45:48.096 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.096 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.096 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.096 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T05:45:48.096 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.096 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.097 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-16T05:45:48.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-16T05:45:48.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-16T05:45:48.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.097 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T05:45:48.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.098 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-16T05:45:48.098 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-16T05:45:48.098 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.099 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.099 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-16T05:45:48.099 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.099 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.099 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.099 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T05:45:48.099 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.100 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.100 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-16T05:45:48.100 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-16T05:45:48.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.100 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.101 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.101 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.101 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T05:45:48.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T05:45:48.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.101 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.102 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-16T05:45:48.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-16T05:45:48.102 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.102 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.102 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.102 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:48.102 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.103 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.103 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T05:45:48.103 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.103 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.103 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.103 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-16T05:45:48.103 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T05:45:48.104 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:48.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-16T05:45:48.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.104 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.104 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T05:45:48.105 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.105 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.105 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-16T05:45:48.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-16T05:45:48.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.105 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.106 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-16T05:45:48.106 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-16T05:45:48.106 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-16T05:45:48.107 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.107 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T05:45:48.107 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.107 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.107 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-16T05:45:48.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-16T05:45:48.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.108 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.108 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.108 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.108 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.108 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T05:45:48.108 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.109 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-16T05:45:48.109 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-16T05:45:48.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.109 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.109 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.109 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.110 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.110 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.110 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T05:45:48.110 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.110 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.111 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-16T05:45:48.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-16T05:45:48.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.111 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.111 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.111 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.111 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.111 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.112 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.112 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T05:45:48.112 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.112 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.112 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-16T05:45:48.112 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-16T05:45:48.113 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.113 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.113 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T05:45:48.113 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.113 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.113 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.113 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:48.113 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.114 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.114 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.114 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-16T05:45:48.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-16T05:45:48.114 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.114 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T05:45:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-16T05:45:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.116 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-16T05:45:48.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-16T05:45:48.116 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.116 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.116 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.116 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T05:45:48.116 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.117 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.117 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.117 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-16T05:45:48.117 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.117 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-16T05:45:48.117 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-16T05:45:48.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.118 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.118 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T05:45:48.118 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.118 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.118 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.118 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:48.118 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.119 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.119 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.119 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-16T05:45:48.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-16T05:45:48.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.119 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.119 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-16T05:45:48.120 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-16T05:45:48.121 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.121 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.121 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.121 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T05:45:48.121 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.121 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.121 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.121 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-16T05:45:48.121 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.122 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.122 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.122 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-16T05:45:48.122 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-16T05:45:48.122 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.122 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.122 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.122 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T05:45:48.123 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.123 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.123 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.123 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-16T05:45:48.123 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.123 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-16T05:45:48.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-16T05:45:48.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.124 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.124 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.124 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T05:45:48.124 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.124 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.124 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.124 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-16T05:45:48.125 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.125 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.125 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.125 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-16T05:45:48.125 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-16T05:45:48.125 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.125 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.126 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T05:45:48.126 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.126 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.126 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-16T05:45:48.126 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.127 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.127 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-16T05:45:48.127 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-16T05:45:48.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.127 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.127 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T05:45:48.127 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.128 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.128 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-16T05:45:48.128 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.128 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.128 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.128 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-16T05:45:48.129 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-16T05:45:48.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.129 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.129 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T05:45:48.129 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.129 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.129 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.129 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:48.130 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.130 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.130 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.130 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-16T05:45:48.130 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-16T05:45:48.130 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.130 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.131 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.131 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T05:45:48.131 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.131 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.131 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.131 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-16T05:45:48.131 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.131 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.131 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-16T05:45:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-16T05:45:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T05:45:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.133 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.133 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:48.133 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.133 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.133 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.133 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-16T05:45:48.133 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-16T05:45:48.134 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.134 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.134 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T05:45:48.134 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.134 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.134 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.134 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-16T05:45:48.134 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.135 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.135 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.135 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-16T05:45:48.135 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-16T05:45:48.135 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.135 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.135 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.136 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T05:45:48.136 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.136 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.136 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.136 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-16T05:45:48.136 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.136 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.136 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.137 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-16T05:45:48.137 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-16T05:45:48.137 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.137 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.137 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T05:45:48.137 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.138 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.138 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.138 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-16T05:45:48.138 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.138 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.138 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.138 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-16T05:45:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-16T05:45:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T05:45:48.140 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.140 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.140 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-16T05:45:48.140 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-16T05:45:48.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.140 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.141 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.141 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:48.141 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.141 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.141 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T05:45:48.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.141 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.142 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-16T05:45:48.142 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T05:45:48.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.142 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.142 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-16T05:45:48.142 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.143 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.143 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T05:45:48.143 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.143 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.143 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-16T05:45:48.143 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-16T05:45:48.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.144 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.144 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.144 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-16T05:45:48.144 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.144 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.144 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.144 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T05:45:48.144 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.145 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.145 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.145 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-16T05:45:48.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-16T05:45:48.145 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.145 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.145 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-16T05:45:48.146 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.146 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.146 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.146 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-16T05:45:48.146 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.146 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.147 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.148 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T05:45:48.148 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.148 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.148 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.148 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-16T05:45:48.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-16T05:45:48.148 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-16T05:45:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.150 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.150 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-16T05:45:48.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-16T05:45:48.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.150 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.150 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T05:45:48.150 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.151 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.151 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.151 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T05:45:48.151 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.151 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.151 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-16T05:45:48.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-16T05:45:48.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.152 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:48.152 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.152 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.152 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.152 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.154 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T05:45:48.154 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.154 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.154 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-16T05:45:48.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-16T05:45:48.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.155 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.155 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.155 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.155 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.155 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T05:45:48.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.156 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.156 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.156 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.157 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.157 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.157 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.157 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.158 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.158 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.158 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.158 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.158 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.159 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-16T05:45:48.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-16T05:45:48.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.159 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.159 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.160 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.160 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T05:45:48.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.160 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T05:45:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.162 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.162 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.162 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.163 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.163 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.163 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.163 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.163 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.163 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.163 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.164 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.164 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.164 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.164 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.164 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T05:45:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T05:45:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.167 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.167 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.167 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.167 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T05:45:48.167 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.168 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T05:45:48.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T05:45:48.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-16T05:45:48.168 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.169 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.169 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T05:45:48.169 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.169 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.169 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-16T05:45:48.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-16T05:45:48.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.170 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.170 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:48.170 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.170 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.170 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.170 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-16T05:45:48.171 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.171 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.171 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-16T05:45:48.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-16T05:45:48.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.171 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T05:45:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.173 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-16T05:45:48.173 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-16T05:45:48.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.173 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.173 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.173 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.173 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.174 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.174 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.174 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-16T05:45:48.174 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.174 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.174 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-16T05:45:48.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-16T05:45:48.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-16T05:45:48.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.175 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.175 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.175 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-16T05:45:48.176 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-16T05:45:48.176 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.176 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T05:45:48.176 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.176 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.176 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.176 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-16T05:45:48.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-16T05:45:48.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.177 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.177 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-16T05:45:48.177 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.177 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.177 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.178 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T05:45:48.178 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.178 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.178 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-16T05:45:48.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-16T05:45:48.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.179 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T05:45:48.179 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.179 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.179 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.179 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T05:45:48.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.180 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-16T05:45:48.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-16T05:45:48.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.180 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.180 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.180 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.180 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.181 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.181 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.181 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T05:45:48.181 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.181 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.181 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-16T05:45:48.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-16T05:45:48.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.182 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T05:45:48.183 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.183 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.183 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.183 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-16T05:45:48.183 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-16T05:45:48.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.183 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.183 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.184 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.184 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.184 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.184 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.184 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T05:45:48.184 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.184 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.185 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-16T05:45:48.185 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-16T05:45:48.185 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.185 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.185 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.185 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.185 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.186 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.186 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.186 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T05:45:48.186 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.186 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.186 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-16T05:45:48.187 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-16T05:45:48.187 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.187 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.187 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.187 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.187 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.187 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.187 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T05:45:48.188 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.188 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.188 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.188 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-16T05:45:48.188 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-16T05:45:48.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.188 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.189 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-16T05:45:48.189 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.189 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.189 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.189 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-16T05:45:48.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.189 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.190 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-16T05:45:48.190 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-16T05:45:48.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.190 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.191 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.191 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T05:45:48.191 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.191 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.191 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.191 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-16T05:45:48.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-16T05:45:48.192 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.192 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.192 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.192 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.192 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.192 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.192 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.192 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T05:45:48.192 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.193 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.193 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-16T05:45:48.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-16T05:45:48.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.193 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.193 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.194 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.194 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.194 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T05:45:48.194 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.194 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.194 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.194 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-16T05:45:48.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-16T05:45:48.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.195 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.195 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.195 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.195 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.195 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.196 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T05:45:48.196 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.196 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.196 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.196 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-16T05:45:48.196 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-16T05:45:48.196 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.197 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.197 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.197 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.197 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.197 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.197 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.197 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T05:45:48.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.198 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.198 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.198 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-16T05:45:48.198 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-16T05:45:48.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.199 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.199 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.199 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.199 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.199 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T05:45:48.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.199 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.200 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-16T05:45:48.200 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.200 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.200 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.200 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.201 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.201 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.201 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.201 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.201 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.201 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.201 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.203 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.203 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.203 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.203 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.203 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.204 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.204 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.205 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.205 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.206 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.206 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.206 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.206 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.206 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.206 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.206 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.206 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.207 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.207 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.207 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.207 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.207 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.207 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.208 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.208 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.208 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.208 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.208 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.208 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.208 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.208 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.209 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.209 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-16T05:45:48.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-16T05:45:48.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.209 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.209 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-16T05:45:48.209 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.210 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.210 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.210 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T05:45:48.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.210 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.210 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-16T05:45:48.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-16T05:45:48.211 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.211 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.211 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-16T05:45:48.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T05:45:48.211 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.212 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.212 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-16T05:45:48.212 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.213 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.214 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.214 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.214 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.214 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.214 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T05:45:48.214 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T05:45:48.215 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.215 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.215 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.215 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.215 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.215 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.215 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.215 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.216 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.216 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T05:45:48.216 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T05:45:48.216 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.216 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.216 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.217 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.217 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.217 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.217 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.217 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.218 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-16T05:45:48.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T05:45:48.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.218 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.218 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T05:45:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T05:45:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.220 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.220 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.220 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.220 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.220 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.220 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.220 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.220 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.221 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.221 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T05:45:48.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T05:45:48.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.221 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.221 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.222 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.222 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.222 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.222 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.222 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.222 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T05:45:48.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T05:45:48.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.223 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.223 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.223 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.223 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.223 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.224 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T05:45:48.224 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T05:45:48.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.224 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.225 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.225 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.225 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.225 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.226 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T05:45:48.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T05:45:48.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.226 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.226 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.226 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.226 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.226 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.227 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.227 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.227 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.227 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T05:45:48.227 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T05:45:48.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.228 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.228 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.228 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.228 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.228 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.229 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.230 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.231 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T05:45:48.231 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T05:45:48.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.232 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.232 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T05:45:48.232 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T05:45:48.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.233 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.233 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.233 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.233 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.233 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.233 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.233 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.233 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.234 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.234 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T05:45:48.234 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T05:45:48.234 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.234 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.235 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.235 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.235 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.235 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.235 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.235 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.235 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.236 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T05:45:48.236 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T05:45:48.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.236 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.236 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-16T05:45:48.236 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.236 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.237 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.237 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.237 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.237 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.237 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T05:45:48.237 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T05:45:48.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.238 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.238 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.238 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.239 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.239 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.239 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-16T05:45:48.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-16T05:45:48.239 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.239 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.240 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.240 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.240 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.240 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.240 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.240 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.240 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.240 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.241 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T05:45:48.241 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T05:45:48.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.241 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-16T05:45:48.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.241 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.242 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T05:45:48.242 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T05:45:48.242 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.243 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.243 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.243 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.243 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.243 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.243 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.244 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T05:45:48.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T05:45:48.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.244 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.244 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.244 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.245 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.245 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.245 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.245 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.245 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T05:45:48.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T05:45:48.246 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.246 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.246 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.246 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.246 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.246 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.247 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.247 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.247 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.247 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.247 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T05:45:48.247 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T05:45:48.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.247 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.248 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.248 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.248 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.248 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.248 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.248 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.248 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.249 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T05:45:48.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T05:45:48.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.249 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-16T05:45:48.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T05:45:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T05:45:48.251 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.251 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-16T05:45:48.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.251 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.251 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.252 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.252 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T05:45:48.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T05:45:48.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.254 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T05:45:48.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T05:45:48.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.254 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.254 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.254 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.254 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.254 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.255 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.255 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.255 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.255 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.255 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T05:45:48.255 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T05:45:48.255 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.256 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.256 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.256 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.256 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.256 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.256 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.258 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.258 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.258 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.258 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.258 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T05:45:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T05:45:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.260 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.260 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.260 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.260 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.260 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T05:45:48.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T05:45:48.260 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.260 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.262 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T05:45:48.262 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T05:45:48.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.263 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.263 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.263 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.263 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.263 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T05:45:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T05:45:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T05:45:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T05:45:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.266 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.266 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.266 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.266 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.266 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.266 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.267 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T05:45:48.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T05:45:48.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.267 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.269 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T05:45:48.269 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T05:45:48.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.269 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.269 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.269 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T05:45:48.269 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.269 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.270 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.270 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.270 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.270 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.270 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T05:45:48.270 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T05:45:48.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.271 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.271 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.271 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:48.271 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.271 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.271 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.271 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.271 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.272 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T05:45:48.272 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T05:45:48.272 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.272 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.272 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-16T05:45:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.274 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T05:45:48.274 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T05:45:48.274 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.274 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.274 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.274 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-16T05:45:48.274 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.275 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.275 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.275 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.275 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T05:45:48.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T05:45:48.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.276 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.276 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.276 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.276 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.276 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.276 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.276 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.277 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T05:45:48.277 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T05:45:48.277 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.277 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.277 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.277 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-16T05:45:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T05:45:48.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T05:45:48.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.279 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.279 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.279 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.279 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.279 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.280 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T05:45:48.280 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T05:45:48.280 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.280 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.281 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.281 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.281 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.281 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T05:45:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T05:45:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.283 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.283 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.283 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.283 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T05:45:48.283 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T05:45:48.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.284 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.284 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.284 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:48.284 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.284 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.284 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.285 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.285 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T05:45:48.285 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T05:45:48.285 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.285 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.285 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:48.286 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.286 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.286 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.286 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.286 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.286 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.286 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T05:45:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T05:45:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.288 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.288 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.288 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.288 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-16T05:45:48.288 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-16T05:45:48.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.289 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.289 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.289 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.289 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.289 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.289 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.289 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.289 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.290 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.290 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.290 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-16T05:45:48.290 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-16T05:45:48.290 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.290 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.291 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.291 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.291 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.291 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.291 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.291 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.292 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-16T05:45:48.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-16T05:45:48.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.292 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.292 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.292 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.292 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.293 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-16T05:45:48.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.293 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.293 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-16T05:45:48.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-16T05:45:48.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.293 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.294 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-16T05:45:48.294 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.294 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.294 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.294 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.294 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.294 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-16T05:45:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-16T05:45:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.296 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.296 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.296 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.296 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.297 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.297 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.297 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.297 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.297 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.297 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.297 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.297 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.298 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.298 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.298 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.298 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.298 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.298 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.298 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.298 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.299 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.299 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.299 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.299 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.299 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.299 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.300 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.300 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.300 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.300 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.300 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.300 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.300 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.300 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.301 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.301 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.301 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.301 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.301 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.302 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.302 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.302 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.302 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.302 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.302 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.302 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.302 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.303 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.303 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.303 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.303 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.303 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.304 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.304 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.304 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.305 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.305 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.305 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-16T05:45:48.305 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-16T05:45:48.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.306 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.306 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.306 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.306 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.306 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-16T05:45:48.306 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.306 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.306 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.307 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-16T05:45:48.307 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T05:45:48.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.307 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.307 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.308 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.308 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.308 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.308 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.308 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.308 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.308 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-16T05:45:48.309 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-16T05:45:48.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.309 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.309 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.309 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-16T05:45:48.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.310 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.310 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.310 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-16T05:45:48.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-16T05:45:48.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.310 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.311 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.311 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-16T05:45:48.311 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.311 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.311 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.311 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.312 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.312 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-16T05:45:48.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-16T05:45:48.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.312 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.313 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.313 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.313 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.313 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.313 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.313 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.314 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-16T05:45:48.314 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-16T05:45:48.314 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.314 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.314 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-16T05:45:48.314 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.314 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.315 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.315 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.315 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.315 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.315 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.315 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-16T05:45:48.315 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-16T05:45:48.315 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.316 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.316 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.316 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-16T05:45:48.316 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.316 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.316 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.316 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.316 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-16T05:45:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-16T05:45:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.318 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.318 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.318 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-16T05:45:48.318 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-16T05:45:48.318 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-16T05:45:48.318 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.318 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.318 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.319 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.319 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.319 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-16T05:45:48.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-16T05:45:48.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.319 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.320 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.320 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.320 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.320 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.320 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.320 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.320 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.320 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.320 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.321 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.321 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-16T05:45:48.321 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.321 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.321 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.321 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-16T05:45:48.322 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-16T05:45:48.322 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.322 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.322 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.322 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.322 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.322 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-16T05:45:48.323 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.323 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.323 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.323 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.323 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.324 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.324 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.324 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.324 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.324 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.324 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.324 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.324 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.324 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.325 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.325 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.325 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.325 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.325 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.325 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.325 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.325 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.326 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.326 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.326 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.326 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.326 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.327 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.327 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.327 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.327 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.327 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.328 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.328 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.328 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.328 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.328 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.328 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.329 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.329 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.329 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.329 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-16T05:45:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-16T05:45:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-16T05:45:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T05:45:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-16T05:45:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-16T05:45:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-16T05:45:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T05:45:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-16T05:45:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-16T05:45:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-16T05:45:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-16T05:45:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-16T05:45:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-16T05:45:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-16T05:45:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T05:45:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-16T05:45:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-16T05:45:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.339 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.339 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.339 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.339 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.339 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.339 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T05:45:48.339 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.339 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.341 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.341 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T05:45:48.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.341 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-16T05:45:48.342 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-16T05:45:48.342 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-16T05:45:48.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T05:45:48.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.343 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.343 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.343 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-16T05:45:48.344 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.344 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.344 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.344 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.344 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.345 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.345 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.345 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.345 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.345 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.346 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.346 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.346 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.346 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.346 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.346 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.346 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.347 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.347 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.347 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.347 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.348 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.348 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.348 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.348 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.349 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.349 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.349 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.349 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.349 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.349 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.349 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.349 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.350 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.350 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.350 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.350 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.350 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.350 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.350 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.350 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.351 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.351 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.351 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.351 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.351 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.352 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.352 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.352 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.353 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-16T05:45:48.353 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-16T05:45:48.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.353 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.353 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.353 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-16T05:45:48.353 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.354 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.354 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.354 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.354 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.354 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.354 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-16T05:45:48.355 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-16T05:45:48.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.355 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.355 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.355 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-16T05:45:48.355 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.355 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.355 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.356 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.356 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.356 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:48.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:48.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.357 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.357 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.357 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-16T05:45:48.357 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.357 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.357 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.357 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.357 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.358 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.358 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-16T05:45:48.358 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-16T05:45:48.358 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-16T05:45:48.358 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.359 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-16T05:45:48.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-16T05:45:48.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.359 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.359 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.359 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-16T05:45:48.359 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.359 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.360 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.360 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T05:45:48.360 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.360 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.360 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.360 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-16T05:45:48.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-16T05:45:48.360 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-16T05:45:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T05:45:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.362 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.362 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-16T05:45:48.362 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-16T05:45:48.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.362 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.362 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.363 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.363 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.363 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-16T05:45:48.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.363 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:48.364 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:48.364 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.364 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.364 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.364 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-16T05:45:48.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-16T05:45:48.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.364 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T05:45:48.365 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.365 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.365 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.365 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.365 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.366 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.366 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.366 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.366 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.366 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.366 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.366 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.366 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.367 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.367 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.367 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.367 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.367 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.367 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.367 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.367 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.368 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.368 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.368 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.368 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.368 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.368 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.369 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.369 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.369 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.369 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.369 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.370 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.370 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.370 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.370 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.370 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.370 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.370 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.370 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.371 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.371 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.371 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.371 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.371 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.371 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.371 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.371 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.371 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.373 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.373 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.373 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.373 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.373 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.373 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.373 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.373 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-16T05:45:48.374 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-16T05:45:48.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.374 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.374 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.374 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.374 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.374 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.375 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-16T05:45:48.375 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.375 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.375 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.375 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-16T05:45:48.375 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-16T05:45:48.375 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.375 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.376 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.376 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-16T05:45:48.376 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.376 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.376 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.376 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-16T05:45:48.376 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.376 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.377 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.377 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-16T05:45:48.377 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-16T05:45:48.377 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.377 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.377 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.378 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-16T05:45:48.378 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.378 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.378 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.378 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:48.378 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.378 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.378 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.379 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-16T05:45:48.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-16T05:45:48.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.379 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.379 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.379 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:48.379 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:48.379 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-16T05:45:48.380 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.380 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T05:45:48.380 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.380 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.380 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.380 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-16T05:45:48.380 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-16T05:45:48.381 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.381 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.381 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.381 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.381 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.381 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.381 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.381 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-16T05:45:48.382 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.382 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.382 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.382 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-16T05:45:48.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-16T05:45:48.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.382 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.383 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.383 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-16T05:45:48.383 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.383 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.383 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:48.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.383 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.384 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.384 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.384 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.384 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.384 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-16T05:45:48.384 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.384 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.384 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.385 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.385 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.385 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.385 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.385 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.385 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.386 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.386 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.386 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.386 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.386 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.386 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.386 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.386 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.387 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.387 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.387 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.387 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.387 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.387 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.387 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.388 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.388 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.388 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.388 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.388 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.388 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.388 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.390 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.390 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.390 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.390 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.390 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.390 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.390 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.390 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.391 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.391 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.391 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.391 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.391 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.391 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.391 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.391 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.392 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.392 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.392 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.392 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.392 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.392 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.392 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.392 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-16T05:45:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.394 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.394 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.394 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.394 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.394 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.394 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.394 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.394 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.395 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.395 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.395 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.395 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.395 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.395 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.395 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.395 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.395 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.396 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.396 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.396 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.396 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.396 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.396 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.397 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.397 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.397 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.397 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.397 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.397 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.397 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.397 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.398 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.398 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.398 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.398 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.398 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.398 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.398 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.399 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.399 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.399 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.399 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.399 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.399 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.399 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.399 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.399 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.401 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.401 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.401 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.401 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.401 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.401 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.401 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.401 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.402 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.402 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.402 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.402 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.402 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.402 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.402 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.402 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-16T05:45:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-16T05:45:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-16T05:45:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-16T05:45:48.404 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.404 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.404 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.404 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T05:45:48.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.404 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.404 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.405 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-16T05:45:48.405 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-16T05:45:48.405 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.405 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.405 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:48.405 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-16T05:45:48.405 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.406 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.406 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T05:45:48.406 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.406 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-16T05:45:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-16T05:45:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-16T05:45:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-16T05:45:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-16T05:45:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T05:45:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-16T05:45:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-16T05:45:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-16T05:45:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T05:45:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-16T05:45:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-16T05:45:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-16T05:45:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-16T05:45:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-16T05:45:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-16T05:45:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-16T05:45:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T05:45:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-16T05:45:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-16T05:45:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-16T05:45:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-16T05:45:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-16T05:45:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-16T05:45:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T05:45:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T05:45:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-16T05:45:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-16T05:45:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-16T05:45:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.455 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.455 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.455 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.455 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.455 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.455 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.455 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.455 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.455 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.456 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.456 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.456 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.456 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.457 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.457 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.457 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.457 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.457 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.457 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.458 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.458 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.458 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.458 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.459 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.459 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.459 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.459 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.459 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.459 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.459 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.460 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.460 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.460 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.460 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.460 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.460 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.461 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.461 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.461 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.461 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.461 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.461 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.462 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.462 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.462 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.462 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.462 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.462 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.462 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.462 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.463 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.463 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.463 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.463 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-16T05:45:48.463 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.463 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.464 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.464 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-16T05:45:48.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-16T05:45:48.464 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.464 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.464 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.464 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.464 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.464 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.465 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.465 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T05:45:48.465 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.465 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.465 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.465 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.465 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.467 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.467 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.467 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.467 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.467 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.467 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.467 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.467 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.468 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.468 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.468 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.468 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.468 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.468 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.468 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.469 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.469 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.469 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.469 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.469 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.469 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.469 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.470 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.470 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.470 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.470 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.470 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.471 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.471 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.471 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.471 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.471 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.471 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.471 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.471 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.471 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.472 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.472 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.472 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.472 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.472 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.472 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.472 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.473 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.473 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.473 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.473 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.473 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.473 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.473 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.473 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.474 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.474 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.474 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.474 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-16T05:45:48.474 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-16T05:45:48.474 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.474 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.475 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.475 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.475 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.475 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.475 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.475 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-16T05:45:48.475 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-16T05:45:48.475 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-16T05:45:48.475 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.476 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-16T05:45:48.476 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.476 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.477 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-16T05:45:48.477 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.477 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.477 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.477 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.477 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.477 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.478 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.478 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.478 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.478 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.478 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.478 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.478 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.479 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.479 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.479 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.479 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.479 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.479 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.479 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.479 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.479 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.480 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.480 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.480 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.480 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.480 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.480 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.481 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.481 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.481 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.481 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.481 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.481 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.481 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.482 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.482 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.482 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.482 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.482 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.482 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.482 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.482 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.483 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.483 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.483 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.483 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.483 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.483 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.483 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.483 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.483 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.484 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.484 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.484 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.484 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.484 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.485 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.485 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.485 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.485 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.485 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.485 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.485 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.485 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.486 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.486 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.486 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-16T05:45:48.486 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.486 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.486 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.487 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.487 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.487 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.487 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.487 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.487 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.487 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.487 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.488 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.488 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.488 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.488 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.488 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.488 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.488 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.488 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.488 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.489 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.489 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.489 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.489 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.489 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.489 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.490 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.490 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.490 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.490 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.490 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.490 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.490 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.490 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.491 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.491 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.491 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.491 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.491 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.492 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.492 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.492 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.492 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.492 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.492 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.492 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.492 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.493 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.493 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.493 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.493 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.493 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.493 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.494 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.494 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.494 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.494 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.494 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.494 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.494 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.495 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.495 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-16T05:45:48.496 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.496 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.496 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.496 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.496 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.496 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.497 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.497 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.497 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.497 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.497 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.497 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.497 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.497 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.498 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.498 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.498 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.498 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.498 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.498 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.499 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.499 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.499 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.499 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.499 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.500 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.500 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.500 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.500 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.500 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.500 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.501 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.501 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.501 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.501 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.501 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.501 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.501 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.502 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.502 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.502 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.502 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.502 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.502 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.502 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.503 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.503 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.503 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.503 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.503 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.503 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.504 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.504 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.504 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.504 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.504 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.504 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.504 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.504 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.505 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.505 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.505 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:48.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:48.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.506 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:48.506 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.506 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.507 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.507 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.507 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.507 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.507 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.507 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-16T05:45:48.507 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-16T05:45:48.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:48.508 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.508 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-16T05:45:48.508 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-16T05:45:48.508 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.508 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.508 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-16T05:45:48.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T05:45:48.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.509 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.510 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-16T05:45:48.510 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.510 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.510 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.510 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-16T05:45:48.510 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-16T05:45:48.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.510 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.511 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.511 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.511 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.511 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.511 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T05:45:48.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.512 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.512 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-16T05:45:48.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-16T05:45:48.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.513 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.513 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.513 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.513 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.513 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.514 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.514 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.514 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.514 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.514 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.514 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.516 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.516 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.516 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.516 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.516 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.516 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.517 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.517 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.517 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.517 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.517 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.517 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.518 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.518 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.518 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.518 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.518 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.519 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.519 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.519 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.519 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.519 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.519 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.519 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.519 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.520 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.520 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.520 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.520 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.520 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.520 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.521 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.521 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.521 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.521 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.521 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.521 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.522 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-16T05:45:48.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-16T05:45:48.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.522 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.522 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.522 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-16T05:45:48.522 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-16T05:45:48.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.523 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.523 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-16T05:45:48.524 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-16T05:45:48.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-16T05:45:48.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.524 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.525 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-16T05:45:48.525 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.525 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-16T05:45:48.526 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.526 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.526 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.526 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-16T05:45:48.526 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-16T05:45:48.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.526 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.526 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.528 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-16T05:45:48.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-16T05:45:48.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.528 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.529 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T05:45:48.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T05:45:48.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.529 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.529 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.530 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.530 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.530 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.530 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.530 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.531 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.531 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.531 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.531 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.531 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.531 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.531 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.531 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.533 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.533 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.533 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.533 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.534 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.534 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.534 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.534 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.534 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.535 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.535 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.535 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.535 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.535 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.535 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.536 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.536 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.536 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.536 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.537 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.537 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.537 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.537 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-16T05:45:48.538 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-16T05:45:48.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.538 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.538 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.538 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.539 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T05:45:48.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T05:45:48.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.539 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-16T05:45:48.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.540 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.541 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.541 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-16T05:45:48.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-16T05:45:48.541 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:48.541 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.541 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.541 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-16T05:45:48.541 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.542 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.542 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.542 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.542 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.542 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.542 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-16T05:45:48.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-16T05:45:48.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.543 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.543 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-16T05:45:48.543 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.543 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.544 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T05:45:48.544 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.544 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.544 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-16T05:45:48.544 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-16T05:45:48.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.545 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.545 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-16T05:45:48.545 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.545 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.545 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.545 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T05:45:48.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.546 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.546 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-16T05:45:48.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-16T05:45:48.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.546 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.547 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-16T05:45:48.547 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.547 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.547 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.547 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.547 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.547 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.547 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.548 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-16T05:45:48.548 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-16T05:45:48.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.548 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-16T05:45:48.548 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-16T05:45:48.549 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-16T05:45:48.549 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.549 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.549 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.549 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.549 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.550 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-16T05:45:48.550 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:48.550 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.550 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.550 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.550 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-16T05:45:48.550 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-16T05:45:48.551 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-16T05:45:48.551 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.551 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.551 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.551 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.551 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.551 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.552 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-16T05:45:48.552 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.552 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.552 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.552 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.552 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.552 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.553 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.553 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.553 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.553 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.553 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.553 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.553 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.554 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.554 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.554 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.554 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.554 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.554 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.554 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.555 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.555 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.555 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.555 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.555 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.556 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.556 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.556 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.556 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.556 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.556 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.557 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.557 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.557 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.557 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.557 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.557 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-16T05:45:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-16T05:45:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-16T05:45:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-16T05:45:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T05:45:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-16T05:45:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-16T05:45:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.565 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.565 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.565 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.565 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.565 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.565 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.565 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.565 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.566 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-16T05:45:48.566 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.566 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.566 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.566 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-16T05:45:48.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-16T05:45:48.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.567 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.567 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.568 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.568 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-16T05:45:48.568 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-16T05:45:48.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.568 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.570 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.570 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.570 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.570 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.571 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.571 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.571 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.571 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.571 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.571 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.571 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.571 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.571 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.572 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.572 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.572 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.572 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.572 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.573 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.573 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.573 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.573 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.573 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.573 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.574 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.574 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.574 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.575 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.575 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.575 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.575 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.575 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.575 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.575 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.575 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.575 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.576 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.576 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.577 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.577 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.577 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.578 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.578 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-16T05:45:48.578 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-16T05:45:48.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.578 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-16T05:45:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-16T05:45:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-16T05:45:48.582 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.582 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.582 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.582 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.582 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.583 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.583 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.583 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.583 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.583 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.584 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.584 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.584 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.584 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.584 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.584 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.584 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.585 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.585 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.585 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.585 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.586 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.586 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.586 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.586 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.586 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.586 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.587 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.587 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.587 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.587 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.587 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.588 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.588 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.588 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.588 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.588 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.588 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.588 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.589 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.589 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.589 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.590 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.590 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.590 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.590 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-16T05:45:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.592 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.592 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.592 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.592 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.593 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.593 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.593 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.593 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.593 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.593 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.593 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.593 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.594 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.594 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.594 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.595 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.595 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.596 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.596 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.596 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.596 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.598 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.598 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.598 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.598 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.598 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.598 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.599 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.599 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.599 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.599 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.599 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.599 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.599 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.600 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.600 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.600 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.600 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.600 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.600 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-16T05:45:48.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-16T05:45:48.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.601 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.601 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.601 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-16T05:45:48.601 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.601 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.601 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T05:45:48.601 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.602 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.602 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.602 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.602 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-16T05:45:48.602 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.603 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.603 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.603 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.603 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.604 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.604 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.604 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.604 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.604 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.604 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.604 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.605 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.605 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.606 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.606 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.607 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.607 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.607 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.608 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.608 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.608 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.608 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.608 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.609 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.609 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.609 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.609 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.610 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.610 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-16T05:45:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-16T05:45:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.612 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.612 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.613 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-16T05:45:48.613 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-16T05:45:48.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.613 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.613 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-16T05:45:48.613 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.613 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.614 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.614 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.614 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.614 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.614 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-16T05:45:48.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-16T05:45:48.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.615 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.615 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.615 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.615 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.616 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.616 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-16T05:45:48.616 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-16T05:45:48.616 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.617 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.617 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.617 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.617 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.617 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.617 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.618 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-16T05:45:48.618 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-16T05:45:48.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-16T05:45:48.618 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.618 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.619 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.619 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.619 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.619 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-16T05:45:48.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-16T05:45:48.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.620 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.620 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.620 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.620 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.620 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.620 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.620 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.621 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.621 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.621 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-16T05:45:48.621 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-16T05:45:48.621 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.621 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.622 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.622 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.622 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.622 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.622 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.623 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-16T05:45:48.623 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-16T05:45:48.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.623 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.623 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.623 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.623 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.624 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.624 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.624 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.624 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-16T05:45:48.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-16T05:45:48.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-16T05:45:48.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-16T05:45:48.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.625 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.625 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.626 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.626 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.626 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-16T05:45:48.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-16T05:45:48.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.627 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.627 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.627 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.627 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.627 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.627 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.629 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.629 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.629 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.629 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-16T05:45:48.629 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-16T05:45:48.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.630 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.630 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.630 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.630 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.630 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.631 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-16T05:45:48.631 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.632 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.632 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.632 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-16T05:45:48.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-16T05:45:48.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.632 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-16T05:45:48.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.634 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.634 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.634 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.634 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.634 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-16T05:45:48.634 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.635 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.635 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.635 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.635 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.636 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.636 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.636 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.636 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.636 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.636 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.636 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.637 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.637 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.637 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.637 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.637 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.637 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.637 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.638 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.638 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.638 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.638 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.639 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.639 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.639 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.640 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.640 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.640 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.640 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.640 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.640 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.640 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.640 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.640 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.641 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.641 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.641 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.641 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.641 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.641 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.642 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.642 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.642 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.642 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.642 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.643 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.643 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.643 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.643 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.643 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-16T05:45:48.644 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.644 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.644 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.644 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.644 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.644 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.645 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.645 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.645 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.645 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.645 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.645 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.648 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.649 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.649 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.649 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.649 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.649 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.649 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.651 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.652 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.652 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.652 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.652 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-16T05:45:48.652 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.652 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.654 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.654 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.654 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.654 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.654 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.654 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.654 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.654 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.655 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.655 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.655 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.655 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.655 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.656 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.656 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.656 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.656 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.657 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.657 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.657 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.657 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.657 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.658 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.658 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.658 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.658 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.658 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.658 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.658 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.658 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.659 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.659 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.659 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.660 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.660 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.660 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.660 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.661 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.661 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-16T05:45:48.661 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.661 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.661 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.661 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-16T05:45:48.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-16T05:45:48.662 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.662 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.662 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.662 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T05:45:48.662 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.662 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.662 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.663 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.663 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.663 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:48.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:48.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.665 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.665 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.665 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.665 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.665 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.665 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.665 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.666 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.666 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.666 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.666 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.666 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.666 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.666 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.666 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.667 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.667 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.667 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.667 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.667 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.667 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.667 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.667 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.668 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.668 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.668 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.668 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.668 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.668 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.668 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.668 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.669 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.669 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.669 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.669 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.669 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.669 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.673 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.673 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-16T05:45:48.673 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.673 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.673 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.673 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-16T05:45:48.673 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-16T05:45:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T05:45:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.675 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-16T05:45:48.675 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-16T05:45:48.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.675 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.676 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.676 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.676 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T05:45:48.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.676 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-16T05:45:48.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-16T05:45:48.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.677 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.678 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T05:45:48.678 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.678 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.678 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-16T05:45:48.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-16T05:45:48.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.679 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T05:45:48.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.680 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-16T05:45:48.680 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-16T05:45:48.680 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.680 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.680 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T05:45:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-16T05:45:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-16T05:45:48.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.682 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.682 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.682 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.682 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.682 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.682 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.682 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-16T05:45:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-16T05:45:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.684 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.684 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.684 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.684 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.684 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-16T05:45:48.685 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-16T05:45:48.685 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.685 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.685 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.685 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T05:45:48.685 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.685 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.685 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.686 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.686 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.686 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.686 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.686 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-16T05:45:48.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-16T05:45:48.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.687 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.687 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.687 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.688 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.688 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:48.688 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:48.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.688 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.688 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-16T05:45:48.689 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-16T05:45:48.689 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.689 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.689 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.689 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.689 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.689 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.689 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-16T05:45:48.690 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-16T05:45:48.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.690 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.690 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.690 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-16T05:45:48.690 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.690 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.691 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.691 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.691 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.691 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-16T05:45:48.691 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-16T05:45:48.691 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.691 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.692 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.692 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.692 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.692 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.692 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-16T05:45:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-16T05:45:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.694 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.694 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.694 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.694 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.694 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.694 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.695 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.695 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.695 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.695 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.695 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.695 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.695 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.696 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.696 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.696 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.696 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.696 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.696 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.696 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.696 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.696 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.697 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.697 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.697 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.697 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.697 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.697 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.698 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.698 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.699 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.699 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.699 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.699 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.699 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.700 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.700 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.700 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.700 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.700 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.700 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.700 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.702 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.702 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.702 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.702 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.702 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.702 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-16T05:45:48.703 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-16T05:45:48.703 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.703 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.703 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.703 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.703 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.705 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.705 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.705 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.705 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.706 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-16T05:45:48.706 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-16T05:45:48.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.706 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T05:45:48.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.707 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.707 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.707 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.707 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.707 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-16T05:45:48.708 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.708 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.708 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.708 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.708 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.709 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.709 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.709 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.709 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.709 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.709 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.709 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.709 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.709 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.710 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.710 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.710 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.710 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.710 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.710 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.710 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.711 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.711 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.711 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.711 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.711 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.711 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.711 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.712 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.713 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.713 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.713 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.713 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.713 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.713 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.715 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.715 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.716 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.716 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.716 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.716 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-16T05:45:48.716 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.718 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.718 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.718 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.718 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.718 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.718 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.718 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.718 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.718 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.719 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.720 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.720 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.720 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.720 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.720 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.720 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.720 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.721 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.721 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.721 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.721 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.721 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.721 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.721 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.721 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.722 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.722 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.722 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.722 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.722 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.722 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.722 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.724 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.724 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.724 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.724 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.724 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.724 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.724 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.724 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-16T05:45:48.725 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-16T05:45:48.725 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.725 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.725 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-16T05:45:48.725 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.725 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T05:45:48.726 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.726 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.726 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.726 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-16T05:45:48.726 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-16T05:45:48.726 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.726 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.727 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.727 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.727 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.727 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-16T05:45:48.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.729 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T05:45:48.729 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.729 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.729 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.729 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-16T05:45:48.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T05:45:48.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.729 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.730 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.730 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-16T05:45:48.730 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.730 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.730 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T05:45:48.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.730 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.731 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-16T05:45:48.731 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-16T05:45:48.731 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-16T05:45:48.731 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-16T05:45:48.731 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-16T05:45:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-16T05:45:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-16T05:45:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-16T05:45:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.733 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.733 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.733 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.733 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-16T05:45:48.733 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-16T05:45:48.733 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.733 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.733 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.734 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.734 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.734 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.734 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.734 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.734 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.735 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.735 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-16T05:45:48.735 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-16T05:45:48.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-16T05:45:48.735 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.735 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.735 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-16T05:45:48.735 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.736 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.736 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.736 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.736 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.736 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-16T05:45:48.737 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-16T05:45:48.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.737 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.738 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.738 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.738 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.738 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.738 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.738 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.738 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.738 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.739 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.739 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.739 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.739 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.739 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-16T05:45:48.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-16T05:45:48.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.740 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.740 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.740 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.740 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.740 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.741 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-16T05:45:48.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:48.741 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-16T05:45:48.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.742 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-16T05:45:48.742 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-16T05:45:48.742 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-16T05:45:48.742 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-16T05:45:48.742 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-16T05:45:48.742 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-16T05:45:48.742 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-16T05:45:48.742 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.743 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.743 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.743 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.743 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-16T05:45:48.743 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-16T05:45:48.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-16T05:45:48.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-16T05:45:48.744 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-16T05:45:48.744 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.744 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.744 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.744 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.744 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.745 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-16T05:45:48.745 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-16T05:45:48.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.745 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.745 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-16T05:45:48.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.746 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.746 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.746 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.746 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.746 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.746 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-16T05:45:48.746 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-16T05:45:48.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-16T05:45:48.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-16T05:45:48.747 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.747 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.748 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.748 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.748 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-16T05:45:48.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-16T05:45:48.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-16T05:45:48.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-16T05:45:48.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-16T05:45:48.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T05:45:48.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-16T05:45:48.749 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-16T05:45:48.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-16T05:45:48.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-16T05:45:48.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-16T05:45:48.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-16T05:45:48.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-16T05:45:48.750 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.750 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.750 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.750 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.750 INFO:teuthology.orchestra.run.smithi086.stdout: "services": { 2024-04-16T05:45:48.750 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard": "https://172.21.15.86:8443/" 2024-04-16T05:45:48.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.750 INFO:teuthology.orchestra.run.smithi086.stdout: "always_on_modules": { 2024-04-16T05:45:48.750 INFO:teuthology.orchestra.run.smithi086.stdout: "octopus": [ 2024-04-16T05:45:48.751 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-16T05:45:48.751 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-16T05:45:48.751 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-16T05:45:48.751 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-16T05:45:48.751 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-16T05:45:48.751 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-16T05:45:48.751 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-16T05:45:48.751 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-16T05:45:48.752 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-16T05:45:48.752 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-16T05:45:48.752 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.752 INFO:teuthology.orchestra.run.smithi086.stdout: "pacific": [ 2024-04-16T05:45:48.752 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-16T05:45:48.752 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-16T05:45:48.752 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-16T05:45:48.752 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-16T05:45:48.753 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-16T05:45:48.753 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-16T05:45:48.753 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-16T05:45:48.753 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-16T05:45:48.753 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-16T05:45:48.753 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-16T05:45:48.753 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.754 INFO:teuthology.orchestra.run.smithi086.stdout: "quincy": [ 2024-04-16T05:45:48.754 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-16T05:45:48.754 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-16T05:45:48.754 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-16T05:45:48.754 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-16T05:45:48.754 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-16T05:45:48.755 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-16T05:45:48.755 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-16T05:45:48.755 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-16T05:45:48.755 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-16T05:45:48.755 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-16T05:45:48.755 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.755 INFO:teuthology.orchestra.run.smithi086.stdout: "reef": [ 2024-04-16T05:45:48.756 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-16T05:45:48.756 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-16T05:45:48.756 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-16T05:45:48.756 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-16T05:45:48.756 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-16T05:45:48.756 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-16T05:45:48.756 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-16T05:45:48.756 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-16T05:45:48.757 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-16T05:45:48.757 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-16T05:45:48.757 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-16T05:45:48.757 INFO:teuthology.orchestra.run.smithi086.stdout: "squid": [ 2024-04-16T05:45:48.757 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-16T05:45:48.757 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-16T05:45:48.757 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-16T05:45:48.757 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-16T05:45:48.757 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-16T05:45:48.758 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-16T05:45:48.758 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-16T05:45:48.758 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-16T05:45:48.758 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-16T05:45:48.758 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-16T05:45:48.758 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:48.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.758 INFO:teuthology.orchestra.run.smithi086.stdout: "last_failure_osd_epoch": 95, 2024-04-16T05:45:48.759 INFO:teuthology.orchestra.run.smithi086.stdout: "active_clients": [ 2024-04-16T05:45:48.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.759 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-16T05:45:48.759 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-16T05:45:48.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.759 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-16T05:45:48.759 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-16T05:45:48.759 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4275327133 2024-04-16T05:45:48.760 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.760 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:48.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "libcephsqlite", 2024-04-16T05:45:48.760 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-16T05:45:48.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-16T05:45:48.761 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-16T05:45:48.761 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 294980895 2024-04-16T05:45:48.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.761 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:48.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-16T05:45:48.762 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-16T05:45:48.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-16T05:45:48.762 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-16T05:45:48.762 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1206756949 2024-04-16T05:45:48.762 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.762 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:48.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-16T05:45:48.763 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-16T05:45:48.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.763 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-16T05:45:48.763 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-16T05:45:48.763 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 249311087 2024-04-16T05:45:48.764 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.764 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:48.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.764 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-16T05:45:48.764 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-16T05:45:48.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-16T05:45:48.765 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-16T05:45:48.765 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3420024136 2024-04-16T05:45:48.765 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.765 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:48.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:48.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-16T05:45:48.765 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-16T05:45:48.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-16T05:45:48.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-16T05:45:48.766 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-16T05:45:48.766 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3825654224 2024-04-16T05:45:48.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.766 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:48.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:48.766 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-16T05:45:48.766 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:45:48.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:46 smithi196 ceph-mon[36373]: pgmap v325: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:48.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:46 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2655719577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:45:48.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:46 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3744167910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:45:48.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:46 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/403098392' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T05:45:48.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:47 smithi196 ceph-mon[36373]: pgmap v326: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:48.769 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:46 smithi194 ceph-mon[35865]: pgmap v325: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:48.769 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:46 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2655719577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:45:48.769 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:46 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3744167910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:45:48.769 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:46 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/403098392' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T05:45:48.769 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:47 smithi194 ceph-mon[35865]: pgmap v326: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3ae80a3d20 msgr2=0x7f3ae80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3ae80a3d20 0x7f3ae80a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ae402c170 tx=0x7f3ae4000bc0 comp rx=0 tx=0).stop 2024-04-16T05:45:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101150 msgr2=0x7f3afc19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101150 0x7f3afc19d460 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3aec007910 tx=0x7f3aec007dd0 comp rx=0 tx=0).stop 2024-04-16T05:45:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 shutdown_connections 2024-04-16T05:45:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3afc197610 0x7f3afc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3ae80a3d20 0x7f3ae80a61e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3afc106220 0x7f3afc19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 --2- 172.21.15.86:0/403098392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc101150 0x7f3afc19d460 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 >> 172.21.15.86:0/403098392 conn(0x7f3afc078000 msgr2=0x7f3afc0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 shutdown_connections 2024-04-16T05:45:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:45:46.557+0000 7f3b01f82640 1 -- 172.21.15.86:0/403098392 wait complete. 2024-04-16T05:45:48.779 DEBUG:tasks.check_counter:Getting stats from a 2024-04-16T05:45:48.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-16T05:45:48.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:46 smithi086 ceph-mon[32028]: pgmap v325: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:48.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:46 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2655719577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:45:48.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:46 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3744167910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:45:48.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:46 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/403098392' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T05:45:48.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:47 smithi086 ceph-mon[32028]: pgmap v326: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:45:50.448 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:50 smithi086 ceph-mon[32028]: pgmap v327: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:50.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:50 smithi196 ceph-mon[36373]: pgmap v327: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:50.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:50 smithi194 ceph-mon[35865]: pgmap v327: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:50.846 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:45:50.846 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:50.846 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1511, 2024-04-16T05:45:50.846 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1513, 2024-04-16T05:45:50.846 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 453804, 2024-04-16T05:45:50.847 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 480499, 2024-04-16T05:45:50.847 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-16T05:45:50.847 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-16T05:45:50.847 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.126960762, 2024-04-16T05:45:50.847 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.045754816, 2024-04-16T05:45:50.847 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.030217728, 2024-04-16T05:45:50.847 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.038414634, 2024-04-16T05:45:50.847 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:50.848 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1511, 2024-04-16T05:45:50.848 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.046744540, 2024-04-16T05:45:50.848 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000030936 2024-04-16T05:45:50.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.848 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:50.848 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.848 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.848 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.849 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:45:50.849 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:45:50.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.849 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:50.849 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 938, 2024-04-16T05:45:50.849 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 924, 2024-04-16T05:45:50.849 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1797156, 2024-04-16T05:45:50.849 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 353676, 2024-04-16T05:45:50.850 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:50.850 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:50.850 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.100638906, 2024-04-16T05:45:50.850 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.031719609, 2024-04-16T05:45:50.850 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.045669722, 2024-04-16T05:45:50.850 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.027224927, 2024-04-16T05:45:50.850 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:50.850 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 924, 2024-04-16T05:45:50.851 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.029599019, 2024-04-16T05:45:50.851 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000032033 2024-04-16T05:45:50.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.851 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:50.851 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.851 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.851 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1626272, 2024-04-16T05:45:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 133552 2024-04-16T05:45:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 3, 2024-04-16T05:45:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 162, 2024-04-16T05:45:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 588, 2024-04-16T05:45:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 444302, 2024-04-16T05:45:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-16T05:45:50.853 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-04-16T05:45:50.853 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.013606656, 2024-04-16T05:45:50.853 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.008817737, 2024-04-16T05:45:50.853 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002609015, 2024-04-16T05:45:50.853 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000015552, 2024-04-16T05:45:50.853 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:50.853 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 159, 2024-04-16T05:45:50.853 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.011735681, 2024-04-16T05:45:50.853 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000073809 2024-04-16T05:45:50.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.854 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:50.854 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.854 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.854 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.854 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:45:50.854 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 443984 2024-04-16T05:45:50.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.855 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-16T05:45:50.855 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-04-16T05:45:50.855 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-16T05:45:50.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.855 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-04-16T05:45:50.855 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-16T05:45:50.855 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-16T05:45:50.855 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3641, 2024-04-16T05:45:50.856 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.088575156, 2024-04-16T05:45:50.856 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000024327 2024-04-16T05:45:50.856 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:50.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.856 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-04-16T05:45:50.856 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-16T05:45:50.856 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-16T05:45:50.856 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.857 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.857 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.857 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:50.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.857 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-04-16T05:45:50.857 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-04-16T05:45:50.857 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-04-16T05:45:50.857 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-04-16T05:45:50.858 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.858 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.858 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.858 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-04-16T05:45:50.858 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-04-16T05:45:50.858 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-04-16T05:45:50.858 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-04-16T05:45:50.859 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-04-16T05:45:50.859 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-04-16T05:45:50.859 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-04-16T05:45:50.859 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 0, 2024-04-16T05:45:50.859 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-04-16T05:45:50.859 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 0, 2024-04-16T05:45:50.859 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-04-16T05:45:50.859 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 0, 2024-04-16T05:45:50.860 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-04-16T05:45:50.860 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-04-16T05:45:50.860 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-04-16T05:45:50.860 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-04-16T05:45:50.860 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-04-16T05:45:50.860 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-04-16T05:45:50.860 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-04-16T05:45:50.860 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-04-16T05:45:50.861 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:45:50.861 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-04-16T05:45:50.861 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-04-16T05:45:50.861 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-04-16T05:45:50.861 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-04-16T05:45:50.861 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-04-16T05:45:50.861 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-04-16T05:45:50.861 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-04-16T05:45:50.862 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-04-16T05:45:50.862 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-04-16T05:45:50.862 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:45:50.862 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-04-16T05:45:50.862 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-04-16T05:45:50.862 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-04-16T05:45:50.862 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-04-16T05:45:50.862 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:45:50.863 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-04-16T05:45:50.863 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:45:50.863 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:45:50.863 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:45:50.863 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:45:50.863 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:45:50.863 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-04-16T05:45:50.863 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:45:50.863 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:45:50.864 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:45:50.864 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:45:50.864 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:45:50.864 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:45:50.864 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-04-16T05:45:50.864 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:45:50.864 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-04-16T05:45:50.864 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-04-16T05:45:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-04-16T05:45:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-04-16T05:45:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-04-16T05:45:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-04-16T05:45:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-04-16T05:45:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-04-16T05:45:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-04-16T05:45:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-04-16T05:45:50.866 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:45:50.866 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-04-16T05:45:50.866 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-04-16T05:45:50.866 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-04-16T05:45:50.866 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-04-16T05:45:50.866 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-04-16T05:45:50.866 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:45:50.867 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:45:50.867 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-04-16T05:45:50.867 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-04-16T05:45:50.867 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:45:50.867 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:45:50.867 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:45:50.867 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-04-16T05:45:50.867 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-04-16T05:45:50.868 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:45:50.868 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-04-16T05:45:50.868 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-04-16T05:45:50.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.868 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-04-16T05:45:50.868 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-04-16T05:45:50.868 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-04-16T05:45:50.868 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-04-16T05:45:50.868 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-04-16T05:45:50.869 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 8, 2024-04-16T05:45:50.869 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-04-16T05:45:50.869 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-04-16T05:45:50.869 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-04-16T05:45:50.869 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-04-16T05:45:50.869 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-04-16T05:45:50.869 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-04-16T05:45:50.869 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-04-16T05:45:50.869 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-04-16T05:45:50.870 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-04-16T05:45:50.870 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-04-16T05:45:50.870 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4196655, 2024-04-16T05:45:50.870 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 4196514, 2024-04-16T05:45:50.870 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-04-16T05:45:50.870 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.870 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.870 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.871 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 8 2024-04-16T05:45:50.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.871 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-04-16T05:45:50.871 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3, 2024-04-16T05:45:50.871 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3, 2024-04-16T05:45:50.871 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-04-16T05:45:50.871 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1, 2024-04-16T05:45:50.871 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 1, 2024-04-16T05:45:50.872 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 0, 2024-04-16T05:45:50.872 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 0, 2024-04-16T05:45:50.872 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 0, 2024-04-16T05:45:50.872 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-04-16T05:45:50.872 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-04-16T05:45:50.872 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-04-16T05:45:50.872 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-04-16T05:45:50.872 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 53444, 2024-04-16T05:45:50.873 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215416 2024-04-16T05:45:50.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.873 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-04-16T05:45:50.873 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-04-16T05:45:50.873 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-04-16T05:45:50.873 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-04-16T05:45:50.873 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-04-16T05:45:50.873 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-04-16T05:45:50.874 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-04-16T05:45:50.874 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.874 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.874 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.874 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-04-16T05:45:50.874 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.874 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.874 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.875 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-04-16T05:45:50.875 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.875 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.875 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.875 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-04-16T05:45:50.875 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.876 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.876 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.876 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-04-16T05:45:50.876 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.876 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.876 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.877 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-04-16T05:45:50.877 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.877 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.877 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.877 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-04-16T05:45:50.877 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.877 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.878 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.878 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-04-16T05:45:50.878 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.878 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.878 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.878 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-04-16T05:45:50.878 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.879 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.879 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.879 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-04-16T05:45:50.879 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.879 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.879 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.880 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-04-16T05:45:50.880 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.880 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.880 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.880 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-04-16T05:45:50.880 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.881 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.881 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.881 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-04-16T05:45:50.882 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.882 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.882 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.882 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-04-16T05:45:50.882 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.882 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.882 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.883 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-04-16T05:45:50.883 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.883 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.883 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.883 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-04-16T05:45:50.883 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.884 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.884 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.884 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-04-16T05:45:50.884 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.884 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.884 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.885 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-04-16T05:45:50.885 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.885 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.885 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.885 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-04-16T05:45:50.885 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.885 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.885 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.886 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-04-16T05:45:50.886 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.886 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.886 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.886 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-04-16T05:45:50.886 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.887 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.887 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.887 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-04-16T05:45:50.887 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.887 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.887 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.888 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-04-16T05:45:50.888 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.888 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.888 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.888 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-04-16T05:45:50.888 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.888 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.888 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-04-16T05:45:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-04-16T05:45:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.890 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.890 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-04-16T05:45:50.890 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.890 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.890 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.890 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-04-16T05:45:50.891 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.891 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.891 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.891 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-04-16T05:45:50.891 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.891 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.891 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.892 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:45:50.892 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:45:50.892 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-04-16T05:45:50.892 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.892 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.892 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.892 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:50.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.892 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-04-16T05:45:50.893 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-04-16T05:45:50.893 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 4, 2024-04-16T05:45:50.893 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 2, 2024-04-16T05:45:50.893 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 3, 2024-04-16T05:45:50.893 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-04-16T05:45:50.893 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-04-16T05:45:50.893 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-04-16T05:45:50.893 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 2579, 2024-04-16T05:45:50.894 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-04-16T05:45:50.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.894 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-16T05:45:50.894 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:50.894 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:50.894 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:50.894 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:50.894 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:50.895 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:50.895 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:50.895 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:50.895 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:50.895 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:50.895 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:50.895 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:50.895 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:50.896 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:50.896 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:50.896 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:50.896 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:50.896 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:50.896 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:50.896 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:50.896 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:50.898 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:50.898 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-16T05:45:50.898 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:50.898 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:50.898 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:50.898 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:50.898 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 189473, 2024-04-16T05:45:50.898 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 53, 2024-04-16T05:45:50.898 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:50.899 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:50.899 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-16T05:45:50.899 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-16T05:45:50.899 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:50.899 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:50.899 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:50.899 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:50.899 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9512, 2024-04-16T05:45:50.899 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 182, 2024-04-16T05:45:50.900 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:50.900 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:50.900 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:50.900 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-16T05:45:50.900 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 8512, 2024-04-16T05:45:50.900 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 13, 2024-04-16T05:45:50.900 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:50.900 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-16T05:45:50.901 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:50.901 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-16T05:45:50.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.901 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-04-16T05:45:50.901 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-04-16T05:45:50.901 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-04-16T05:45:50.901 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 2204, 2024-04-16T05:45:50.901 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-04-16T05:45:50.902 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-04-16T05:45:50.902 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 2204, 2024-04-16T05:45:50.902 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-04-16T05:45:50.902 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2204, 2024-04-16T05:45:50.902 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.116975861, 2024-04-16T05:45:50.902 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000506794 2024-04-16T05:45:50.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.902 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-04-16T05:45:50.903 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-04-16T05:45:50.903 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2204, 2024-04-16T05:45:50.903 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2205 2024-04-16T05:45:50.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.903 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 2204, 2024-04-16T05:45:50.903 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 2204, 2024-04-16T05:45:50.903 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-04-16T05:45:50.903 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-04-16T05:45:50.904 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-04-16T05:45:50.904 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 1441, 2024-04-16T05:45:50.904 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-04-16T05:45:50.904 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 761, 2024-04-16T05:45:50.904 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-04-16T05:45:50.904 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-04-16T05:45:50.904 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-04-16T05:45:50.904 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-04-16T05:45:50.905 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-04-16T05:45:50.905 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-04-16T05:45:50.905 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-04-16T05:45:50.905 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-04-16T05:45:50.905 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-04-16T05:45:50.905 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-04-16T05:45:50.905 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-04-16T05:45:50.905 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-04-16T05:45:50.905 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:45:50.906 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-04-16T05:45:50.906 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:45:50.906 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-04-16T05:45:50.906 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-04-16T05:45:50.906 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-04-16T05:45:50.906 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:45:50.906 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-04-16T05:45:50.906 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:45:50.907 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-04-16T05:45:50.907 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-04-16T05:45:50.907 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-04-16T05:45:50.907 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-04-16T05:45:50.907 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-04-16T05:45:50.907 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-04-16T05:45:50.907 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-04-16T05:45:50.907 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-04-16T05:45:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-04-16T05:45:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-04-16T05:45:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-04-16T05:45:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-04-16T05:45:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-04-16T05:45:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-04-16T05:45:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-04-16T05:45:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-04-16T05:45:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-04-16T05:45:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 103, 2024-04-16T05:45:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-04-16T05:45:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 13, 2024-04-16T05:45:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 5, 2024-04-16T05:45:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 5, 2024-04-16T05:45:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-04-16T05:45:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-04-16T05:45:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-04-16T05:45:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-04-16T05:45:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-04-16T05:45:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-04-16T05:45:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-04-16T05:45:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:45:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-04-16T05:45:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-04-16T05:45:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-04-16T05:45:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-04-16T05:45:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-04-16T05:45:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-04-16T05:45:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:45:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-04-16T05:45:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-04-16T05:45:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-04-16T05:45:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-04-16T05:45:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-04-16T05:45:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-16T05:45:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 2448, 2024-04-16T05:45:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2015791, 2024-04-16T05:45:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 2448, 2024-04-16T05:45:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-16T05:45:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-16T05:45:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2448, 2024-04-16T05:45:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2015791, 2024-04-16T05:45:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-04-16T05:45:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-16T05:45:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-16T05:45:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-16T05:45:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 2204, 2024-04-16T05:45:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 2356, 2024-04-16T05:45:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2204, 2024-04-16T05:45:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2356, 2024-04-16T05:45:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-04-16T05:45:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-04-16T05:45:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-16T05:45:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-16T05:45:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 2204, 2024-04-16T05:45:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 2204, 2024-04-16T05:45:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2204, 2024-04-16T05:45:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2204, 2024-04-16T05:45:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:45:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-16T05:45:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-16T05:45:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-16T05:45:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-16T05:45:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-16T05:45:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-16T05:45:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-16T05:45:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x561be9d306e0": { 2024-04-16T05:45:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-16T05:45:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-16T05:45:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-16T05:45:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-16T05:45:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-16T05:45:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-16T05:45:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-16T05:45:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:50.923 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:45:51.658 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-16T05:45:51.658 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-16T05:45:51.658 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-16T05:45:51.658 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:45:51.658 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:45:51.659 DEBUG:tasks.check_counter:Getting stats from d 2024-04-16T05:45:51.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-16T05:45:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:45:52.285 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:52 smithi086 ceph-mon[32028]: pgmap v328: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:52.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:52 smithi196 ceph-mon[36373]: pgmap v328: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:52.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:52 smithi194 ceph-mon[35865]: pgmap v328: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:53.612 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:45:53.613 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:53.613 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 228, 2024-04-16T05:45:53.613 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 217, 2024-04-16T05:45:53.613 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1591600, 2024-04-16T05:45:53.613 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 133184, 2024-04-16T05:45:53.614 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:53.614 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-04-16T05:45:53.614 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.046461692, 2024-04-16T05:45:53.614 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.014934880, 2024-04-16T05:45:53.614 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.034671921, 2024-04-16T05:45:53.614 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.004541515, 2024-04-16T05:45:53.614 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:53.614 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 216, 2024-04-16T05:45:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.011470955, 2024-04-16T05:45:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000053106 2024-04-16T05:45:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1591600, 2024-04-16T05:45:53.616 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 133184 2024-04-16T05:45:53.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.616 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:53.616 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2, 2024-04-16T05:45:53.616 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 166, 2024-04-16T05:45:53.616 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 320, 2024-04-16T05:45:53.616 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 185520, 2024-04-16T05:45:53.617 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-04-16T05:45:53.617 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:53.617 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.014494246, 2024-04-16T05:45:53.617 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.010126511, 2024-04-16T05:45:53.617 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.001816170, 2024-04-16T05:45:53.617 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:53.617 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:53.617 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 163, 2024-04-16T05:45:53.617 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.016421670, 2024-04-16T05:45:53.618 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000100746 2024-04-16T05:45:53.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.618 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:53.618 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:53.618 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.618 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.618 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:45:53.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 185520 2024-04-16T05:45:53.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.619 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:53.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 4, 2024-04-16T05:45:53.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 4, 2024-04-16T05:45:53.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3024, 2024-04-16T05:45:53.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 448, 2024-04-16T05:45:53.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-04-16T05:45:53.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-04-16T05:45:53.620 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.000521011, 2024-04-16T05:45:53.620 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000036233, 2024-04-16T05:45:53.620 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000462423, 2024-04-16T05:45:53.620 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:53.620 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:53.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-04-16T05:45:53.620 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000648775, 2024-04-16T05:45:53.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000216258 2024-04-16T05:45:53.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.621 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:53.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:53.621 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.621 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-16T05:45:53.621 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 448 2024-04-16T05:45:53.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.622 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-16T05:45:53.622 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-04-16T05:45:53.622 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-16T05:45:53.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.622 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-16T05:45:53.622 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:53.622 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:53.622 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:53.622 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:53.623 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:53.623 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:53.623 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:53.623 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:53.623 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:53.623 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:53.623 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:53.623 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:53.624 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:53.624 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:53.624 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:53.624 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:53.624 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:53.624 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:53.624 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:53.624 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:53.624 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:53.625 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:53.625 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:53.625 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:53.625 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:53.625 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:53.625 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:53.625 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:53.625 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:53.626 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:53.626 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:53.626 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-16T05:45:53.626 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:53.626 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:53.626 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:53.626 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:53.626 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148995, 2024-04-16T05:45:53.626 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 31, 2024-04-16T05:45:53.627 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:53.627 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:53.627 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-16T05:45:53.627 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-16T05:45:53.627 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:53.627 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:53.627 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:53.627 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:53.627 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-04-16T05:45:53.628 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-04-16T05:45:53.628 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:53.628 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:53.628 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:53.628 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-16T05:45:53.628 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-04-16T05:45:53.628 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-04-16T05:45:53.628 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:53.628 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-16T05:45:53.629 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:53.629 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-16T05:45:53.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:53.629 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:53.629 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:53.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-16T05:45:53.629 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:53.629 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 230, 2024-04-16T05:45:53.629 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1559443, 2024-04-16T05:45:53.630 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:53.630 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 230, 2024-04-16T05:45:53.630 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-16T05:45:53.630 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-16T05:45:53.630 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 230, 2024-04-16T05:45:53.630 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1559443, 2024-04-16T05:45:53.630 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:53.630 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:53.630 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.631 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:53.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:53.631 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:45:54.314 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:54 smithi086 ceph-mon[32028]: pgmap v329: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:54.342 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-16T05:45:54.342 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-16T05:45:54.342 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-16T05:45:54.343 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-16T05:45:54.343 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-16T05:45:54.343 DEBUG:tasks.check_counter:Getting stats from g 2024-04-16T05:45:54.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-16T05:45:54.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:54 smithi196 ceph-mon[36373]: pgmap v329: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:54.586 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:45:54.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:54 smithi194 ceph-mon[35865]: pgmap v329: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:45:56.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:56 smithi086 ceph-mon[32028]: pgmap v330: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:56.304 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:45:56.304 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:56.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 762, 2024-04-16T05:45:56.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 762, 2024-04-16T05:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 257657, 2024-04-16T05:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 241493, 2024-04-16T05:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-16T05:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-16T05:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.087940223, 2024-04-16T05:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.037637199, 2024-04-16T05:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.018920391, 2024-04-16T05:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.022345891, 2024-04-16T05:45:56.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:56.306 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 761, 2024-04-16T05:45:56.306 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.038814404, 2024-04-16T05:45:56.306 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000051004 2024-04-16T05:45:56.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:56.306 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.306 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.307 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.307 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-16T05:45:56.307 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 448 2024-04-16T05:45:56.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.307 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:56.307 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1598, 2024-04-16T05:45:56.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1586, 2024-04-16T05:45:56.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1970031, 2024-04-16T05:45:56.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 565013, 2024-04-16T05:45:56.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:56.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:56.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.129856488, 2024-04-16T05:45:56.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.039868134, 2024-04-16T05:45:56.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.055670097, 2024-04-16T05:45:56.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.036868660, 2024-04-16T05:45:56.309 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:56.309 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1585, 2024-04-16T05:45:56.309 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.049801327, 2024-04-16T05:45:56.309 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000031420 2024-04-16T05:45:56.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.309 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:56.309 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.310 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.310 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.310 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1623792, 2024-04-16T05:45:56.310 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 134384 2024-04-16T05:45:56.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.310 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:56.311 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 5, 2024-04-16T05:45:56.311 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 166, 2024-04-16T05:45:56.311 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1668, 2024-04-16T05:45:56.311 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 448533, 2024-04-16T05:45:56.311 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-04-16T05:45:56.311 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-04-16T05:45:56.311 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.014034620, 2024-04-16T05:45:56.311 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.009361848, 2024-04-16T05:45:56.312 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002384429, 2024-04-16T05:45:56.312 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000054520, 2024-04-16T05:45:56.312 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:56.312 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 163, 2024-04-16T05:45:56.312 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.012582650, 2024-04-16T05:45:56.312 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000077194 2024-04-16T05:45:56.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.312 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:56.312 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.313 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.313 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.313 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:45:56.313 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 447568 2024-04-16T05:45:56.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.313 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-16T05:45:56.313 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-04-16T05:45:56.313 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-16T05:45:56.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.314 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-04-16T05:45:56.314 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-16T05:45:56.314 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-16T05:45:56.314 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3619, 2024-04-16T05:45:56.314 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.090133853, 2024-04-16T05:45:56.314 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000024905 2024-04-16T05:45:56.314 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:56.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.315 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-04-16T05:45:56.315 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-16T05:45:56.315 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-16T05:45:56.315 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.315 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.315 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.315 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:56.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.316 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-04-16T05:45:56.316 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-04-16T05:45:56.316 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-04-16T05:45:56.316 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-04-16T05:45:56.316 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.316 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.316 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-04-16T05:45:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-04-16T05:45:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-04-16T05:45:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-04-16T05:45:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-04-16T05:45:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-04-16T05:45:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-04-16T05:45:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 0, 2024-04-16T05:45:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-04-16T05:45:56.318 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 0, 2024-04-16T05:45:56.318 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-04-16T05:45:56.318 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 0, 2024-04-16T05:45:56.318 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-04-16T05:45:56.318 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-04-16T05:45:56.318 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-04-16T05:45:56.318 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-04-16T05:45:56.318 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-04-16T05:45:56.319 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-04-16T05:45:56.319 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-04-16T05:45:56.319 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-04-16T05:45:56.319 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:45:56.319 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-04-16T05:45:56.319 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-04-16T05:45:56.319 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-04-16T05:45:56.319 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-04-16T05:45:56.320 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-04-16T05:45:56.320 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-04-16T05:45:56.320 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-04-16T05:45:56.320 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-04-16T05:45:56.320 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-04-16T05:45:56.320 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:45:56.320 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-04-16T05:45:56.320 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-04-16T05:45:56.321 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-04-16T05:45:56.321 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-04-16T05:45:56.321 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:45:56.321 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-04-16T05:45:56.321 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:45:56.321 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:45:56.321 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:45:56.321 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:45:56.322 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:45:56.322 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-04-16T05:45:56.322 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:45:56.322 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:45:56.322 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:45:56.322 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:45:56.322 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:45:56.322 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:45:56.323 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-04-16T05:45:56.323 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:45:56.323 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-04-16T05:45:56.323 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-04-16T05:45:56.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.323 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-04-16T05:45:56.323 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-04-16T05:45:56.323 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-04-16T05:45:56.324 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-04-16T05:45:56.324 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-04-16T05:45:56.324 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-04-16T05:45:56.324 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-04-16T05:45:56.324 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-04-16T05:45:56.324 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:45:56.324 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-04-16T05:45:56.324 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-04-16T05:45:56.324 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-04-16T05:45:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-04-16T05:45:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-04-16T05:45:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:45:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:45:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-04-16T05:45:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-04-16T05:45:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:45:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:45:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:45:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-04-16T05:45:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-04-16T05:45:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:45:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-04-16T05:45:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-04-16T05:45:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-04-16T05:45:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-04-16T05:45:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-04-16T05:45:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-04-16T05:45:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-04-16T05:45:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 3, 2024-04-16T05:45:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-04-16T05:45:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-04-16T05:45:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-04-16T05:45:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-04-16T05:45:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-04-16T05:45:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-04-16T05:45:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-04-16T05:45:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-04-16T05:45:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-04-16T05:45:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-04-16T05:45:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4195283, 2024-04-16T05:45:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 4194892, 2024-04-16T05:45:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-04-16T05:45:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 3 2024-04-16T05:45:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-04-16T05:45:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3, 2024-04-16T05:45:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3, 2024-04-16T05:45:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-04-16T05:45:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1, 2024-04-16T05:45:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 1, 2024-04-16T05:45:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 0, 2024-04-16T05:45:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 0, 2024-04-16T05:45:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 0, 2024-04-16T05:45:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-04-16T05:45:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-04-16T05:45:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-04-16T05:45:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-04-16T05:45:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 50608, 2024-04-16T05:45:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215416 2024-04-16T05:45:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-04-16T05:45:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-04-16T05:45:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-04-16T05:45:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-04-16T05:45:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-04-16T05:45:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-04-16T05:45:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-04-16T05:45:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-04-16T05:45:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-04-16T05:45:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-04-16T05:45:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-04-16T05:45:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-04-16T05:45:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-04-16T05:45:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-04-16T05:45:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-04-16T05:45:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-04-16T05:45:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-04-16T05:45:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-04-16T05:45:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-04-16T05:45:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-04-16T05:45:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-04-16T05:45:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-04-16T05:45:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-04-16T05:45:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-04-16T05:45:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-04-16T05:45:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-04-16T05:45:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-04-16T05:45:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-04-16T05:45:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-04-16T05:45:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-04-16T05:45:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-04-16T05:45:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-04-16T05:45:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-04-16T05:45:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-04-16T05:45:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-04-16T05:45:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:45:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:45:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-04-16T05:45:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-04-16T05:45:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-04-16T05:45:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-04-16T05:45:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-04-16T05:45:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-04-16T05:45:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-04-16T05:45:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-04-16T05:45:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-04-16T05:45:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 1935, 2024-04-16T05:45:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-04-16T05:45:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-16T05:45:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-16T05:45:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 197490, 2024-04-16T05:45:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 56, 2024-04-16T05:45:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-16T05:45:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-16T05:45:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9512, 2024-04-16T05:45:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 182, 2024-04-16T05:45:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-16T05:45:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 8512, 2024-04-16T05:45:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 13, 2024-04-16T05:45:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-16T05:45:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-16T05:45:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-04-16T05:45:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-04-16T05:45:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-04-16T05:45:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 2115, 2024-04-16T05:45:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-04-16T05:45:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-04-16T05:45:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 2115, 2024-04-16T05:45:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-04-16T05:45:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2115, 2024-04-16T05:45:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.033977635, 2024-04-16T05:45:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000488878 2024-04-16T05:45:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-04-16T05:45:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-04-16T05:45:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2115, 2024-04-16T05:45:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2116 2024-04-16T05:45:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 2115, 2024-04-16T05:45:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 2115, 2024-04-16T05:45:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-04-16T05:45:56.364 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-04-16T05:45:56.364 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-04-16T05:45:56.364 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 1351, 2024-04-16T05:45:56.364 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-04-16T05:45:56.364 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 762, 2024-04-16T05:45:56.364 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-04-16T05:45:56.364 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-04-16T05:45:56.364 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-04-16T05:45:56.364 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-04-16T05:45:56.365 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-04-16T05:45:56.365 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-04-16T05:45:56.365 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-04-16T05:45:56.365 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-04-16T05:45:56.365 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-04-16T05:45:56.365 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-04-16T05:45:56.365 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-04-16T05:45:56.365 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-04-16T05:45:56.366 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:45:56.366 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-04-16T05:45:56.366 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:45:56.366 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-04-16T05:45:56.366 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-04-16T05:45:56.366 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-04-16T05:45:56.366 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:45:56.366 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-04-16T05:45:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:45:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-04-16T05:45:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-04-16T05:45:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-04-16T05:45:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-04-16T05:45:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-04-16T05:45:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-04-16T05:45:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-04-16T05:45:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-04-16T05:45:56.368 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-04-16T05:45:56.368 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-04-16T05:45:56.368 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-04-16T05:45:56.368 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-04-16T05:45:56.368 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-04-16T05:45:56.368 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-04-16T05:45:56.368 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-04-16T05:45:56.368 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-04-16T05:45:56.369 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-04-16T05:45:56.369 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 103, 2024-04-16T05:45:56.369 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-04-16T05:45:56.369 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 13, 2024-04-16T05:45:56.369 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 6, 2024-04-16T05:45:56.369 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 6, 2024-04-16T05:45:56.369 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-04-16T05:45:56.369 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-04-16T05:45:56.369 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-04-16T05:45:56.370 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-04-16T05:45:56.370 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-04-16T05:45:56.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.370 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-04-16T05:45:56.370 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-04-16T05:45:56.370 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:45:56.370 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-04-16T05:45:56.370 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-04-16T05:45:56.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.371 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-04-16T05:45:56.371 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-04-16T05:45:56.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.371 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-04-16T05:45:56.371 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-04-16T05:45:56.371 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:45:56.371 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-04-16T05:45:56.372 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-04-16T05:45:56.372 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-04-16T05:45:56.372 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-04-16T05:45:56.372 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-04-16T05:45:56.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.372 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:56.372 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:56.372 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-16T05:45:56.372 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:56.373 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 2361, 2024-04-16T05:45:56.373 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2001325, 2024-04-16T05:45:56.373 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:56.373 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 2361, 2024-04-16T05:45:56.373 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-16T05:45:56.373 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-16T05:45:56.373 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2361, 2024-04-16T05:45:56.373 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2001325, 2024-04-16T05:45:56.374 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:56.374 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.374 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.374 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.374 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:56.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.374 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-04-16T05:45:56.374 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:56.375 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-16T05:45:56.375 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:56.375 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-16T05:45:56.375 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-16T05:45:56.375 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:56.375 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:56.375 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 2115, 2024-04-16T05:45:56.375 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 984, 2024-04-16T05:45:56.375 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2115, 2024-04-16T05:45:56.376 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 984, 2024-04-16T05:45:56.376 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:56.376 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.376 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.376 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.376 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:56.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.376 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-04-16T05:45:56.377 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:56.377 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-04-16T05:45:56.377 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:56.377 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-16T05:45:56.377 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-16T05:45:56.377 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:56.377 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:56.377 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 2115, 2024-04-16T05:45:56.377 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 2115, 2024-04-16T05:45:56.378 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2115, 2024-04-16T05:45:56.378 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2115, 2024-04-16T05:45:56.378 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:56.378 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.378 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.378 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.378 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:56.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.379 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:45:56.379 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:56.379 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-16T05:45:56.379 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:56.379 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-16T05:45:56.379 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-16T05:45:56.379 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:56.379 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:56.379 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-16T05:45:56.380 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-16T05:45:56.380 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-16T05:45:56.380 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-16T05:45:56.380 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:56.380 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.380 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.380 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.380 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:56.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:56.381 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x5592ac88a6e0": { 2024-04-16T05:45:56.381 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:56.381 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-16T05:45:56.381 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:56.381 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-16T05:45:56.381 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-16T05:45:56.381 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:56.382 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:56.382 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-16T05:45:56.382 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-16T05:45:56.382 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-16T05:45:56.382 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-16T05:45:56.382 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:56.382 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:56.382 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:56.383 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:56.383 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:56.383 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:56.383 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:45:56.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:56 smithi196 ceph-mon[36373]: pgmap v330: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:56.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:56 smithi194 ceph-mon[35865]: pgmap v330: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:57.133 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-16T05:45:57.133 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-16T05:45:57.134 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-16T05:45:57.134 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:45:57.134 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:45:57.134 DEBUG:tasks.check_counter:Getting stats from j 2024-04-16T05:45:57.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-16T05:45:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:45:58.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:45:58 smithi086 ceph-mon[32028]: pgmap v331: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:58.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:45:58 smithi196 ceph-mon[36373]: pgmap v331: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:58.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:45:58 smithi194 ceph-mon[35865]: pgmap v331: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:45:59.221 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-16T05:45:59.221 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 4, 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 4, 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3024, 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 448, 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.001416739, 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000043364, 2024-04-16T05:45:59.223 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000579987, 2024-04-16T05:45:59.223 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:59.223 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:59.223 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-04-16T05:45:59.223 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000784151, 2024-04-16T05:45:59.223 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000261383 2024-04-16T05:45:59.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.223 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 448 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 230, 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 219, 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1592144, 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 134464, 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.048693066, 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.015373246, 2024-04-16T05:45:59.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.039287447, 2024-04-16T05:45:59.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.004477430, 2024-04-16T05:45:59.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:59.226 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 218, 2024-04-16T05:45:59.226 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.010864487, 2024-04-16T05:45:59.226 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000049837 2024-04-16T05:45:59.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:59.227 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:59.227 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:59.227 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:59.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.227 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1592144, 2024-04-16T05:45:59.227 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 134464 2024-04-16T05:45:59.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.227 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:59.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2, 2024-04-16T05:45:59.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 168, 2024-04-16T05:45:59.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 320, 2024-04-16T05:45:59.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 185872, 2024-04-16T05:45:59.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-04-16T05:45:59.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:59.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.015619546, 2024-04-16T05:45:59.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.011005168, 2024-04-16T05:45:59.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.001890622, 2024-04-16T05:45:59.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:59.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:59.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 165, 2024-04-16T05:45:59.229 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.018287547, 2024-04-16T05:45:59.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000110833 2024-04-16T05:45:59.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:59.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:59.230 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:59.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:59.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:45:59.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 185872 2024-04-16T05:45:59.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.230 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-16T05:45:59.230 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-04-16T05:45:59.231 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-16T05:45:59.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.231 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-16T05:45:59.231 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:59.231 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:59.231 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:59.231 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:59.231 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:59.232 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:59.232 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:59.232 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:59.232 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:59.232 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:59.232 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:59.232 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:59.232 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:59.233 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:59.233 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:59.233 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:59.233 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:59.233 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:59.233 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:59.233 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:59.234 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:59.234 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:59.234 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:59.234 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:59.234 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:59.234 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:59.234 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:59.234 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:59.235 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:59.235 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:59.235 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:59.235 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-16T05:45:59.235 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:59.235 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:59.235 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:59.235 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:59.236 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148977, 2024-04-16T05:45:59.236 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 30, 2024-04-16T05:45:59.236 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:59.236 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:59.236 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-16T05:45:59.236 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-16T05:45:59.236 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:59.236 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:59.237 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:59.237 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:59.237 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-04-16T05:45:59.237 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-04-16T05:45:59.237 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:59.237 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:59.237 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:59.237 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-16T05:45:59.238 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-04-16T05:45:59.238 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-04-16T05:45:59.238 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:59.238 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-16T05:45:59.238 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:59.238 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-16T05:45:59.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-16T05:45:59.239 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:59.239 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-16T05:45:59.239 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-16T05:45:59.239 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-16T05:45:59.239 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 232, 2024-04-16T05:45:59.239 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1559703, 2024-04-16T05:45:59.239 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:59.239 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 232, 2024-04-16T05:45:59.239 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-16T05:45:59.240 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-16T05:45:59.240 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 232, 2024-04-16T05:45:59.240 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1559703, 2024-04-16T05:45:59.240 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-16T05:45:59.240 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-16T05:45:59.240 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-16T05:45:59.240 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-16T05:45:59.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:59.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-16T05:45:59.241 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-16T05:45:59.947 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-16T05:45:59.947 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-16T05:45:59.947 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-16T05:45:59.947 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-16T05:45:59.947 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-16T05:45:59.947 DEBUG:tasks.check_counter:Getting stats from b 2024-04-16T05:45:59.948 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-16T05:46:00.198 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:46:00.320 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:00 smithi194 ceph-mon[35865]: pgmap v332: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:46:00.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:00 smithi086 ceph-mon[32028]: pgmap v332: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:46:00.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:00 smithi196 ceph-mon[36373]: pgmap v332: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-16T05:46:02.034 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-16T05:46:02.034 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:46:02.035 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1062, 2024-04-16T05:46:02.035 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 811, 2024-04-16T05:46:02.035 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 509665, 2024-04-16T05:46:02.035 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 394348, 2024-04-16T05:46:02.035 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 13, 2024-04-16T05:46:02.035 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-04-16T05:46:02.035 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.117477617, 2024-04-16T05:46:02.035 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.027576120, 2024-04-16T05:46:02.036 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.037078226, 2024-04-16T05:46:02.036 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.037138687, 2024-04-16T05:46:02.036 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:02.036 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 807, 2024-04-16T05:46:02.036 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.716819068, 2024-04-16T05:46:02.036 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000888251 2024-04-16T05:46:02.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.036 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:02.037 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 952, 2024-04-16T05:46:02.037 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000217234, 2024-04-16T05:46:02.037 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000228 2024-04-16T05:46:02.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.037 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 46496, 2024-04-16T05:46:02.037 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 4864 2024-04-16T05:46:02.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.037 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:46:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2632, 2024-04-16T05:46:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1217, 2024-04-16T05:46:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 2568179, 2024-04-16T05:46:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 734758, 2024-04-16T05:46:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 31, 2024-04-16T05:46:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 8, 2024-04-16T05:46:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.325309776, 2024-04-16T05:46:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.077984747, 2024-04-16T05:46:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.142730708, 2024-04-16T05:46:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.110654530, 2024-04-16T05:46:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1216, 2024-04-16T05:46:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.259925302, 2024-04-16T05:46:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000213754 2024-04-16T05:46:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2602, 2024-04-16T05:46:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000535073, 2024-04-16T05:46:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000205 2024-04-16T05:46:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 2036032, 2024-04-16T05:46:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 290768 2024-04-16T05:46:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:46:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1556, 2024-04-16T05:46:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1295, 2024-04-16T05:46:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 707047, 2024-04-16T05:46:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 2538482, 2024-04-16T05:46:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 162, 2024-04-16T05:46:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-04-16T05:46:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.208833862, 2024-04-16T05:46:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.056103229, 2024-04-16T05:46:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.095565656, 2024-04-16T05:46:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.047946508, 2024-04-16T05:46:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1295, 2024-04-16T05:46:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.284323129, 2024-04-16T05:46:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000219554 2024-04-16T05:46:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1988, 2024-04-16T05:46:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000374280, 2024-04-16T05:46:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000188 2024-04-16T05:46:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-04-16T05:46:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 459776 2024-04-16T05:46:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-04-16T05:46:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-04-16T05:46:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-04-16T05:46:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 484, 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.115396169, 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000238421 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-04-16T05:46:02.044 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-16T05:46:02.045 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-16T05:46:02.045 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-04-16T05:46:02.045 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000008529, 2024-04-16T05:46:02.045 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000004264 2024-04-16T05:46:02.045 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:02.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.045 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-04-16T05:46:02.045 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 175, 2024-04-16T05:46:02.045 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 175, 2024-04-16T05:46:02.046 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-04-16T05:46:02.046 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 175, 2024-04-16T05:46:02.046 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.142750934, 2024-04-16T05:46:02.046 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000815719 2024-04-16T05:46:02.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.046 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-04-16T05:46:02.046 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-04-16T05:46:02.046 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-04-16T05:46:02.047 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-04-16T05:46:02.047 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 25, 2024-04-16T05:46:02.047 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-04-16T05:46:02.047 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-04-16T05:46:02.047 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 27, 2024-04-16T05:46:02.047 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 7, 2024-04-16T05:46:02.047 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 20, 2024-04-16T05:46:02.047 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-04-16T05:46:02.047 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 17, 2024-04-16T05:46:02.048 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-04-16T05:46:02.048 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 4, 2024-04-16T05:46:02.048 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 4, 2024-04-16T05:46:02.048 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 2, 2024-04-16T05:46:02.048 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 243, 2024-04-16T05:46:02.048 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 236, 2024-04-16T05:46:02.048 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-04-16T05:46:02.048 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-04-16T05:46:02.048 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:46:02.049 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-04-16T05:46:02.049 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-04-16T05:46:02.049 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-04-16T05:46:02.049 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-04-16T05:46:02.049 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-04-16T05:46:02.049 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-04-16T05:46:02.049 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-04-16T05:46:02.049 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-04-16T05:46:02.049 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-04-16T05:46:02.050 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 243, 2024-04-16T05:46:02.050 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-04-16T05:46:02.050 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 4, 2024-04-16T05:46:02.050 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 234, 2024-04-16T05:46:02.050 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 6, 2024-04-16T05:46:02.050 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 243, 2024-04-16T05:46:02.050 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-04-16T05:46:02.050 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:46:02.050 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:46:02.051 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 167, 2024-04-16T05:46:02.051 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 19, 2024-04-16T05:46:02.051 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 148, 2024-04-16T05:46:02.051 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 76, 2024-04-16T05:46:02.051 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 1, 2024-04-16T05:46:02.051 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 8, 2024-04-16T05:46:02.051 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 6, 2024-04-16T05:46:02.051 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:46:02.051 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:46:02.052 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:46:02.052 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 16, 2024-04-16T05:46:02.052 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 3, 2024-04-16T05:46:02.052 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-04-16T05:46:02.052 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 17 2024-04-16T05:46:02.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.052 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-04-16T05:46:02.052 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-04-16T05:46:02.052 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 9, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-04-16T05:46:02.053 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-04-16T05:46:02.054 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-04-16T05:46:02.054 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:46:02.054 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:46:02.054 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-04-16T05:46:02.054 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-04-16T05:46:02.054 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:46:02.054 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:46:02.054 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 19, 2024-04-16T05:46:02.055 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-04-16T05:46:02.055 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-04-16T05:46:02.055 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:46:02.055 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-04-16T05:46:02.055 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-04-16T05:46:02.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.055 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-04-16T05:46:02.055 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-04-16T05:46:02.055 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 120, 2024-04-16T05:46:02.056 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 53, 2024-04-16T05:46:02.056 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 53, 2024-04-16T05:46:02.056 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 67, 2024-04-16T05:46:02.056 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-04-16T05:46:02.056 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 1, 2024-04-16T05:46:02.056 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 8, 2024-04-16T05:46:02.056 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 7, 2024-04-16T05:46:02.056 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 7, 2024-04-16T05:46:02.057 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 1, 2024-04-16T05:46:02.057 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 1, 2024-04-16T05:46:02.057 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-04-16T05:46:02.057 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 1, 2024-04-16T05:46:02.057 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4284449, 2024-04-16T05:46:02.057 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4322409, 2024-04-16T05:46:02.057 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2024-04-16T05:46:02.057 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-04-16T05:46:02.057 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 96, 2024-04-16T05:46:02.058 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.170109709, 2024-04-16T05:46:02.058 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001771976 2024-04-16T05:46:02.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.058 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2024-04-16T05:46:02.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.058 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-04-16T05:46:02.058 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 25, 2024-04-16T05:46:02.058 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 44, 2024-04-16T05:46:02.059 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 19, 2024-04-16T05:46:02.059 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 20, 2024-04-16T05:46:02.059 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 20, 2024-04-16T05:46:02.059 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 0, 2024-04-16T05:46:02.059 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 27, 2024-04-16T05:46:02.059 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 27, 2024-04-16T05:46:02.059 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 0, 2024-04-16T05:46:02.059 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 4, 2024-04-16T05:46:02.060 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 19, 2024-04-16T05:46:02.060 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 15, 2024-04-16T05:46:02.060 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 51812, 2024-04-16T05:46:02.060 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 231800 2024-04-16T05:46:02.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.060 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-04-16T05:46:02.060 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 220, 2024-04-16T05:46:02.060 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 2, 2024-04-16T05:46:02.061 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 175, 2024-04-16T05:46:02.061 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 36, 2024-04-16T05:46:02.061 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 4, 2024-04-16T05:46:02.061 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-04-16T05:46:02.061 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 4, 2024-04-16T05:46:02.061 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.002519868, 2024-04-16T05:46:02.061 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000629967 2024-04-16T05:46:02.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.061 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-04-16T05:46:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 120, 2024-04-16T05:46:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.022529661, 2024-04-16T05:46:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000187747 2024-04-16T05:46:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-04-16T05:46:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.063 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-04-16T05:46:02.063 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.063 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.063 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.063 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-04-16T05:46:02.063 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-04-16T05:46:02.064 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000480445, 2024-04-16T05:46:02.064 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000240222 2024-04-16T05:46:02.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.064 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-04-16T05:46:02.064 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.064 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.064 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.065 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-04-16T05:46:02.065 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.065 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.065 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.065 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-04-16T05:46:02.066 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.066 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.066 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.066 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-04-16T05:46:02.066 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.066 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.067 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.067 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-04-16T05:46:02.067 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.067 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.067 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.067 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-04-16T05:46:02.067 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 11, 2024-04-16T05:46:02.068 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001792590, 2024-04-16T05:46:02.068 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000162962 2024-04-16T05:46:02.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.068 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-04-16T05:46:02.069 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 8, 2024-04-16T05:46:02.069 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.016639360, 2024-04-16T05:46:02.069 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002079920 2024-04-16T05:46:02.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.069 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-04-16T05:46:02.069 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.069 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.070 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.070 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-04-16T05:46:02.070 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 11, 2024-04-16T05:46:02.070 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.053586882, 2024-04-16T05:46:02.070 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.004871534 2024-04-16T05:46:02.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.070 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-04-16T05:46:02.071 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1, 2024-04-16T05:46:02.071 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000226754, 2024-04-16T05:46:02.071 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000226754 2024-04-16T05:46:02.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.071 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-04-16T05:46:02.071 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.071 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.071 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.072 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-04-16T05:46:02.072 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-04-16T05:46:02.072 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.002021717, 2024-04-16T05:46:02.072 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001010858 2024-04-16T05:46:02.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.072 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-04-16T05:46:02.072 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.072 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.072 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.073 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-04-16T05:46:02.073 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.073 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.073 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.073 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-04-16T05:46:02.073 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.074 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.074 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.074 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-04-16T05:46:02.074 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5, 2024-04-16T05:46:02.074 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.021181906, 2024-04-16T05:46:02.074 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.004236381 2024-04-16T05:46:02.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.074 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-04-16T05:46:02.075 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.075 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.075 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.075 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-04-16T05:46:02.075 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-04-16T05:46:02.075 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001360734, 2024-04-16T05:46:02.075 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000680367 2024-04-16T05:46:02.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.076 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-04-16T05:46:02.076 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.076 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.076 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.076 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-04-16T05:46:02.076 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.076 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.077 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.077 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-04-16T05:46:02.077 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.077 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.077 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.077 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-04-16T05:46:02.078 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 9, 2024-04-16T05:46:02.078 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.020411017, 2024-04-16T05:46:02.078 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002267890 2024-04-16T05:46:02.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.078 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-04-16T05:46:02.078 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.078 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.078 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.079 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-04-16T05:46:02.079 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.079 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.079 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.079 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:46:02.079 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:46:02.079 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-04-16T05:46:02.079 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.080 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.080 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.080 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:02.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.080 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-04-16T05:46:02.080 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-04-16T05:46:02.080 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 6, 2024-04-16T05:46:02.080 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 4, 2024-04-16T05:46:02.081 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 2, 2024-04-16T05:46:02.081 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-04-16T05:46:02.081 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 27, 2024-04-16T05:46:02.081 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 13, 2024-04-16T05:46:02.081 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 621, 2024-04-16T05:46:02.081 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-04-16T05:46:02.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.081 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-04-16T05:46:02.082 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:46:02.082 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-04-16T05:46:02.082 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:46:02.082 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:46:02.082 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:46:02.082 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:46:02.082 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:46:02.082 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:46:02.082 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:46:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:46:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:46:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:46:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:46:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:46:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:46:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-04-16T05:46:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:46:02.084 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-04-16T05:46:02.084 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:46:02.084 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:46:02.084 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:46:02.084 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:46:02.084 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:46:02.084 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:46:02.084 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:46:02.085 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-04-16T05:46:02.085 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:46:02.085 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:46:02.085 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:46:02.085 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-04-16T05:46:02.085 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-04-16T05:46:02.085 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-04-16T05:46:02.085 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:46:02.086 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:46:02.086 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:46:02.086 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:46:02.086 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 349973, 2024-04-16T05:46:02.086 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 143, 2024-04-16T05:46:02.086 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 968, 2024-04-16T05:46:02.086 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 11, 2024-04-16T05:46:02.086 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-04-16T05:46:02.087 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-04-16T05:46:02.087 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:46:02.087 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-04-16T05:46:02.087 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:46:02.087 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-04-16T05:46:02.087 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9512, 2024-04-16T05:46:02.087 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 182, 2024-04-16T05:46:02.087 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:46:02.088 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:46:02.088 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-04-16T05:46:02.088 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-04-16T05:46:02.088 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 109925, 2024-04-16T05:46:02.088 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 733, 2024-04-16T05:46:02.088 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-04-16T05:46:02.088 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-04-16T05:46:02.088 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-04-16T05:46:02.088 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-04-16T05:46:02.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.089 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-04-16T05:46:02.089 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-04-16T05:46:02.089 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-04-16T05:46:02.089 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 997, 2024-04-16T05:46:02.089 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 163857, 2024-04-16T05:46:02.089 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 37, 2024-04-16T05:46:02.089 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 997, 2024-04-16T05:46:02.090 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-04-16T05:46:02.090 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 997, 2024-04-16T05:46:02.090 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 20.193589710, 2024-04-16T05:46:02.090 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.020254352 2024-04-16T05:46:02.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.090 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-04-16T05:46:02.090 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-04-16T05:46:02.090 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 997, 2024-04-16T05:46:02.091 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1084 2024-04-16T05:46:02.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.091 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 997, 2024-04-16T05:46:02.091 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 281, 2024-04-16T05:46:02.091 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 716, 2024-04-16T05:46:02.091 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-04-16T05:46:02.091 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-04-16T05:46:02.091 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 6, 2024-04-16T05:46:02.091 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 27, 2024-04-16T05:46:02.092 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 19, 2024-04-16T05:46:02.092 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 96, 2024-04-16T05:46:02.092 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 39, 2024-04-16T05:46:02.092 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-04-16T05:46:02.092 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-04-16T05:46:02.092 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-04-16T05:46:02.092 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-04-16T05:46:02.092 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 506, 2024-04-16T05:46:02.093 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-04-16T05:46:02.093 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-04-16T05:46:02.093 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-04-16T05:46:02.093 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 262, 2024-04-16T05:46:02.093 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 50, 2024-04-16T05:46:02.093 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:46:02.093 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-04-16T05:46:02.093 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:46:02.093 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-04-16T05:46:02.094 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-04-16T05:46:02.094 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-04-16T05:46:02.094 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:46:02.094 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-04-16T05:46:02.094 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:46:02.094 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-04-16T05:46:02.094 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-04-16T05:46:02.094 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-04-16T05:46:02.095 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-04-16T05:46:02.095 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-04-16T05:46:02.095 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-04-16T05:46:02.095 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-04-16T05:46:02.095 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-04-16T05:46:02.095 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-04-16T05:46:02.095 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-04-16T05:46:02.095 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-04-16T05:46:02.096 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-04-16T05:46:02.096 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-04-16T05:46:02.096 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-04-16T05:46:02.096 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-04-16T05:46:02.096 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-04-16T05:46:02.096 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-04-16T05:46:02.096 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 103, 2024-04-16T05:46:02.096 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-04-16T05:46:02.097 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 19, 2024-04-16T05:46:02.097 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 12, 2024-04-16T05:46:02.097 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 12, 2024-04-16T05:46:02.097 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-04-16T05:46:02.097 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-04-16T05:46:02.097 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 72, 2024-04-16T05:46:02.097 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 0, 2024-04-16T05:46:02.097 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 7 2024-04-16T05:46:02.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.098 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-04-16T05:46:02.098 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2024-04-16T05:46:02.098 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:46:02.098 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-04-16T05:46:02.098 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-04-16T05:46:02.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.098 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-04-16T05:46:02.099 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-04-16T05:46:02.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.099 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-04-16T05:46:02.099 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-04-16T05:46:02.099 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:46:02.099 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-04-16T05:46:02.099 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-04-16T05:46:02.099 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-04-16T05:46:02.100 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-04-16T05:46:02.100 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-04-16T05:46:02.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.100 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:46:02.100 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:02.100 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-16T05:46:02.100 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:02.100 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 5246, 2024-04-16T05:46:02.101 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 3245800, 2024-04-16T05:46:02.101 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:02.101 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 5246, 2024-04-16T05:46:02.101 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:02.101 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:02.101 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 5246, 2024-04-16T05:46:02.101 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3245800, 2024-04-16T05:46:02.101 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:02.102 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.102 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.102 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.102 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:02.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.102 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-04-16T05:46:02.102 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:02.102 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-16T05:46:02.103 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:02.103 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:02.103 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:02.103 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:02.103 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:02.103 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 997, 2024-04-16T05:46:02.103 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 163857, 2024-04-16T05:46:02.103 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 997, 2024-04-16T05:46:02.103 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 163857, 2024-04-16T05:46:02.104 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:02.104 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.104 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.104 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.104 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:02.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.104 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-04-16T05:46:02.104 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:02.104 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 997, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 997, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 997, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 997, 2024-04-16T05:46:02.105 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:02.106 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.106 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.106 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.106 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:02.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.106 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:46:02.106 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:02.106 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-16T05:46:02.107 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:02.107 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:02.107 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:02.107 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:02.107 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:02.107 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:02.107 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:02.107 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-16T05:46:02.108 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-16T05:46:02.108 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:02.108 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.108 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.108 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.108 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:02.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:02.108 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55db38b9e500": { 2024-04-16T05:46:02.108 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 84, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 120, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 128105, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 120, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 96, 2024-04-16T05:46:02.109 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 128021, 2024-04-16T05:46:02.110 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:02.110 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:02.110 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:02.110 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:02.110 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:02.110 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:02.110 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-16T05:46:02.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:02 smithi086 ceph-mon[32028]: pgmap v333: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:46:02.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:02 smithi196 ceph-mon[36373]: pgmap v333: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:46:02.487 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:02 smithi194 ceph-mon[35865]: pgmap v333: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-16T05:46:02.774 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-16T05:46:02.774 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-16T05:46:02.774 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-16T05:46:02.774 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-16T05:46:02.774 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-16T05:46:02.774 DEBUG:tasks.check_counter:Getting stats from e 2024-04-16T05:46:02.775 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-16T05:46:03.025 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:46:04.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:04 smithi086 ceph-mon[32028]: pgmap v334: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:46:04.480 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:04 smithi194 ceph-mon[35865]: pgmap v334: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:46:04.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:04 smithi196 ceph-mon[36373]: pgmap v334: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:46:04.734 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-16T05:46:04.735 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:46:04.735 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1599, 2024-04-16T05:46:04.735 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1599, 2024-04-16T05:46:04.735 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 605316, 2024-04-16T05:46:04.735 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 507668, 2024-04-16T05:46:04.735 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 6, 2024-04-16T05:46:04.736 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-04-16T05:46:04.736 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.155065619, 2024-04-16T05:46:04.736 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.058670615, 2024-04-16T05:46:04.736 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.035856079, 2024-04-16T05:46:04.736 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.046445198, 2024-04-16T05:46:04.736 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:04.737 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1598, 2024-04-16T05:46:04.737 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.057939434, 2024-04-16T05:46:04.737 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000036257 2024-04-16T05:46:04.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.737 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:04.737 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.738 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.738 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.738 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-16T05:46:04.738 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 448 2024-04-16T05:46:04.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.738 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:46:04.738 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 6, 2024-04-16T05:46:04.739 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 172, 2024-04-16T05:46:04.739 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1976, 2024-04-16T05:46:04.739 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 453018, 2024-04-16T05:46:04.739 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 7, 2024-04-16T05:46:04.739 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-04-16T05:46:04.739 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.010564593, 2024-04-16T05:46:04.739 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.006696297, 2024-04-16T05:46:04.739 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.001914343, 2024-04-16T05:46:04.740 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000070395, 2024-04-16T05:46:04.740 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:04.740 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 169, 2024-04-16T05:46:04.740 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.016393545, 2024-04-16T05:46:04.740 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000097003 2024-04-16T05:46:04.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.740 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:04.741 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.741 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.741 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.741 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:46:04.741 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 451680 2024-04-16T05:46:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:46:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1017, 2024-04-16T05:46:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1005, 2024-04-16T05:46:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1814065, 2024-04-16T05:46:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 381651, 2024-04-16T05:46:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 3, 2024-04-16T05:46:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-04-16T05:46:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.116906596, 2024-04-16T05:46:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.034997438, 2024-04-16T05:46:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.054231281, 2024-04-16T05:46:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.031243353, 2024-04-16T05:46:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1004, 2024-04-16T05:46:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.035173641, 2024-04-16T05:46:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000035033 2024-04-16T05:46:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1625312, 2024-04-16T05:46:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 137744 2024-04-16T05:46:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-04-16T05:46:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-04-16T05:46:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-04-16T05:46:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-04-16T05:46:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-16T05:46:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-16T05:46:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3929, 2024-04-16T05:46:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.108132644, 2024-04-16T05:46:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000027521 2024-04-16T05:46:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-04-16T05:46:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-16T05:46:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-16T05:46:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.748 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.748 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.748 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:04.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.748 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-04-16T05:46:04.748 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-04-16T05:46:04.748 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-04-16T05:46:04.748 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-04-16T05:46:04.749 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.749 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.749 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.749 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-04-16T05:46:04.749 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-04-16T05:46:04.749 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-04-16T05:46:04.750 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-04-16T05:46:04.750 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-04-16T05:46:04.750 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-04-16T05:46:04.750 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-04-16T05:46:04.750 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 14, 2024-04-16T05:46:04.750 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 5, 2024-04-16T05:46:04.750 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 9, 2024-04-16T05:46:04.751 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-04-16T05:46:04.751 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 7, 2024-04-16T05:46:04.751 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-04-16T05:46:04.751 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-04-16T05:46:04.751 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-04-16T05:46:04.751 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 2, 2024-04-16T05:46:04.751 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-04-16T05:46:04.752 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-04-16T05:46:04.752 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-04-16T05:46:04.752 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-04-16T05:46:04.752 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:46:04.752 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-04-16T05:46:04.752 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-04-16T05:46:04.752 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-04-16T05:46:04.753 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-04-16T05:46:04.753 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-04-16T05:46:04.753 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-04-16T05:46:04.753 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-04-16T05:46:04.753 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-04-16T05:46:04.753 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-04-16T05:46:04.753 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:46:04.754 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-04-16T05:46:04.754 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 4, 2024-04-16T05:46:04.754 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 234, 2024-04-16T05:46:04.754 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 6, 2024-04-16T05:46:04.754 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:46:04.754 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-04-16T05:46:04.754 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:46:04.755 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:46:04.755 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:46:04.755 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:46:04.755 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:46:04.755 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-04-16T05:46:04.755 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:46:04.755 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:46:04.755 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:46:04.756 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:46:04.756 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:46:04.756 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:46:04.756 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-04-16T05:46:04.756 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:46:04.756 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-04-16T05:46:04.757 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-04-16T05:46:04.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.757 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-04-16T05:46:04.757 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-04-16T05:46:04.757 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-04-16T05:46:04.757 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-04-16T05:46:04.757 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-04-16T05:46:04.757 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-04-16T05:46:04.758 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-04-16T05:46:04.758 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-04-16T05:46:04.758 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:46:04.758 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-04-16T05:46:04.758 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-04-16T05:46:04.758 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-04-16T05:46:04.758 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-04-16T05:46:04.758 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-04-16T05:46:04.759 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:46:04.759 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:46:04.759 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-04-16T05:46:04.759 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-04-16T05:46:04.759 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:46:04.759 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:46:04.759 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:46:04.759 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-04-16T05:46:04.760 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-04-16T05:46:04.760 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:46:04.760 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-04-16T05:46:04.760 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-04-16T05:46:04.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.760 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-04-16T05:46:04.760 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-04-16T05:46:04.760 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-04-16T05:46:04.761 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-04-16T05:46:04.761 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-04-16T05:46:04.761 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 120, 2024-04-16T05:46:04.761 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-04-16T05:46:04.761 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-04-16T05:46:04.761 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-04-16T05:46:04.761 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-04-16T05:46:04.761 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-04-16T05:46:04.762 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-04-16T05:46:04.762 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 8, 2024-04-16T05:46:04.762 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-04-16T05:46:04.762 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-04-16T05:46:04.762 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4284449, 2024-04-16T05:46:04.762 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4322409, 2024-04-16T05:46:04.762 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 4322325, 2024-04-16T05:46:04.762 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-04-16T05:46:04.763 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.763 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.763 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.763 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 120 2024-04-16T05:46:04.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.763 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-04-16T05:46:04.763 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 15, 2024-04-16T05:46:04.764 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 15, 2024-04-16T05:46:04.764 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 0, 2024-04-16T05:46:04.764 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 10, 2024-04-16T05:46:04.764 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 10, 2024-04-16T05:46:04.764 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 0, 2024-04-16T05:46:04.764 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 14, 2024-04-16T05:46:04.764 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 14, 2024-04-16T05:46:04.764 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 0, 2024-04-16T05:46:04.765 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-04-16T05:46:04.765 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-04-16T05:46:04.765 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-04-16T05:46:04.765 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 48272, 2024-04-16T05:46:04.765 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215416 2024-04-16T05:46:04.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.765 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-04-16T05:46:04.765 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-04-16T05:46:04.766 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-04-16T05:46:04.766 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-04-16T05:46:04.766 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-04-16T05:46:04.766 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-04-16T05:46:04.766 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-04-16T05:46:04.766 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.766 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.767 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.767 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-04-16T05:46:04.767 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.767 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.767 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.767 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-04-16T05:46:04.768 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.768 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.768 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.768 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-04-16T05:46:04.768 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.768 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.769 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.769 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-04-16T05:46:04.769 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.769 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.769 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.769 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-04-16T05:46:04.770 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.770 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.770 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.770 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-04-16T05:46:04.770 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.770 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.771 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.771 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-04-16T05:46:04.771 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.771 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.771 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.771 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-04-16T05:46:04.772 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.772 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.772 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.772 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-04-16T05:46:04.772 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.772 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.772 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.773 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-04-16T05:46:04.773 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.773 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.773 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.773 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-04-16T05:46:04.773 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.774 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.774 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.775 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-04-16T05:46:04.775 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.775 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.775 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.775 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-04-16T05:46:04.775 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.776 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.776 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.776 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-04-16T05:46:04.776 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.776 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.776 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.777 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-04-16T05:46:04.777 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.777 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.777 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.777 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-04-16T05:46:04.777 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.777 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.778 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.778 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-04-16T05:46:04.778 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.778 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.778 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.779 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-04-16T05:46:04.779 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.779 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.779 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.779 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-04-16T05:46:04.779 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.779 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.780 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.780 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-04-16T05:46:04.780 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.780 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.780 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.781 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-04-16T05:46:04.781 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.781 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.781 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.781 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-04-16T05:46:04.782 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.782 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.782 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.782 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-04-16T05:46:04.782 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.782 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.782 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.783 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-04-16T05:46:04.783 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.783 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.783 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.783 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-04-16T05:46:04.784 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.784 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.784 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.784 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-04-16T05:46:04.784 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.784 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.784 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.785 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-04-16T05:46:04.785 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.785 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.785 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.785 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-04-16T05:46:04.785 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.785 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.786 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.786 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:46:04.786 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:46:04.786 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-04-16T05:46:04.786 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.787 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.787 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.787 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:04.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.787 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-04-16T05:46:04.787 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-04-16T05:46:04.787 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 6, 2024-04-16T05:46:04.787 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 4, 2024-04-16T05:46:04.788 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 4, 2024-04-16T05:46:04.788 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-04-16T05:46:04.788 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-04-16T05:46:04.788 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-04-16T05:46:04.788 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 3663, 2024-04-16T05:46:04.788 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-04-16T05:46:04.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.788 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-04-16T05:46:04.789 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:46:04.789 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-04-16T05:46:04.789 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:46:04.789 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:46:04.789 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:46:04.789 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:46:04.789 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:46:04.789 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:46:04.790 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:46:04.790 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:46:04.790 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:46:04.790 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:46:04.790 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:46:04.790 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:46:04.791 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:46:04.791 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-04-16T05:46:04.791 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:46:04.791 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-04-16T05:46:04.791 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:46:04.791 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:46:04.791 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:46:04.792 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:46:04.792 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:46:04.792 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:46:04.792 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:46:04.792 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-04-16T05:46:04.792 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:46:04.792 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:46:04.793 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:46:04.793 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-04-16T05:46:04.793 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-04-16T05:46:04.793 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-04-16T05:46:04.793 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:46:04.793 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:46:04.793 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:46:04.793 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:46:04.794 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 189521, 2024-04-16T05:46:04.794 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 55, 2024-04-16T05:46:04.794 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:46:04.794 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-04-16T05:46:04.794 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-04-16T05:46:04.794 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-04-16T05:46:04.794 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:46:04.795 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-04-16T05:46:04.795 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:46:04.795 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-04-16T05:46:04.795 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9512, 2024-04-16T05:46:04.795 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 182, 2024-04-16T05:46:04.795 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:46:04.795 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:46:04.795 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-04-16T05:46:04.796 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-04-16T05:46:04.796 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 61037, 2024-04-16T05:46:04.796 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 318, 2024-04-16T05:46:04.796 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-04-16T05:46:04.796 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-04-16T05:46:04.796 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-04-16T05:46:04.796 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-04-16T05:46:04.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.797 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-04-16T05:46:04.797 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-04-16T05:46:04.797 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-04-16T05:46:04.797 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 2366, 2024-04-16T05:46:04.797 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 26, 2024-04-16T05:46:04.797 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 4, 2024-04-16T05:46:04.797 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 2366, 2024-04-16T05:46:04.798 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-04-16T05:46:04.798 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2366, 2024-04-16T05:46:04.798 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.018979273, 2024-04-16T05:46:04.798 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000430675 2024-04-16T05:46:04.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.798 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-04-16T05:46:04.798 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-04-16T05:46:04.798 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2366, 2024-04-16T05:46:04.799 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 2367 2024-04-16T05:46:04.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.799 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 2366, 2024-04-16T05:46:04.799 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 2366, 2024-04-16T05:46:04.799 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-04-16T05:46:04.799 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-04-16T05:46:04.799 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-04-16T05:46:04.799 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 1532, 2024-04-16T05:46:04.800 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-04-16T05:46:04.800 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 831, 2024-04-16T05:46:04.800 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-04-16T05:46:04.800 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-04-16T05:46:04.800 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-04-16T05:46:04.800 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-04-16T05:46:04.800 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-04-16T05:46:04.800 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-04-16T05:46:04.801 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-04-16T05:46:04.801 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-04-16T05:46:04.801 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-04-16T05:46:04.801 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-04-16T05:46:04.801 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 2, 2024-04-16T05:46:04.801 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-04-16T05:46:04.801 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:46:04.801 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-04-16T05:46:04.802 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:46:04.802 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-04-16T05:46:04.802 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-04-16T05:46:04.802 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-04-16T05:46:04.802 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:46:04.802 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-04-16T05:46:04.802 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:46:04.803 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-04-16T05:46:04.803 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-04-16T05:46:04.803 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-04-16T05:46:04.803 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-04-16T05:46:04.803 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-04-16T05:46:04.803 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-04-16T05:46:04.803 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-04-16T05:46:04.803 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-04-16T05:46:04.804 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-04-16T05:46:04.804 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-04-16T05:46:04.804 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-04-16T05:46:04.804 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-04-16T05:46:04.804 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-04-16T05:46:04.804 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-04-16T05:46:04.804 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-04-16T05:46:04.805 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-04-16T05:46:04.805 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-04-16T05:46:04.805 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 103, 2024-04-16T05:46:04.805 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-04-16T05:46:04.805 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 13, 2024-04-16T05:46:04.805 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 5, 2024-04-16T05:46:04.805 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 5, 2024-04-16T05:46:04.805 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-04-16T05:46:04.806 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-04-16T05:46:04.806 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-04-16T05:46:04.806 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-04-16T05:46:04.806 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-04-16T05:46:04.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.806 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-04-16T05:46:04.806 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-04-16T05:46:04.807 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:46:04.807 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-04-16T05:46:04.807 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-04-16T05:46:04.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.807 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-04-16T05:46:04.807 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-04-16T05:46:04.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.807 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-04-16T05:46:04.808 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-04-16T05:46:04.808 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:46:04.808 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-04-16T05:46:04.808 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-04-16T05:46:04.808 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-04-16T05:46:04.808 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-04-16T05:46:04.808 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-04-16T05:46:04.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.809 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:46:04.809 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:04.809 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-16T05:46:04.809 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:04.809 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 2618, 2024-04-16T05:46:04.809 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2169890, 2024-04-16T05:46:04.809 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:04.809 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 2618, 2024-04-16T05:46:04.810 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:04.810 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:04.810 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 2618, 2024-04-16T05:46:04.810 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2169890, 2024-04-16T05:46:04.810 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:04.810 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.810 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.811 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.811 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:04.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.811 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-04-16T05:46:04.811 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:04.811 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-16T05:46:04.811 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:04.811 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:04.812 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:04.812 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:04.812 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:04.812 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 2366, 2024-04-16T05:46:04.812 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 128115, 2024-04-16T05:46:04.812 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 2366, 2024-04-16T05:46:04.812 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 128115, 2024-04-16T05:46:04.813 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:04.813 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.813 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.813 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.813 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:04.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.813 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-04-16T05:46:04.813 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:04.814 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-04-16T05:46:04.814 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:04.814 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:04.814 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:04.814 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:04.814 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:04.814 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 2366, 2024-04-16T05:46:04.815 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 2366, 2024-04-16T05:46:04.815 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 2366, 2024-04-16T05:46:04.815 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2366, 2024-04-16T05:46:04.815 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:04.815 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.815 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.815 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.815 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:04.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.816 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:46:04.816 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:04.816 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-16T05:46:04.816 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:04.816 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:04.816 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:04.817 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:04.817 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:04.817 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:04.817 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:04.817 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-16T05:46:04.817 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-16T05:46:04.817 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:04.817 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.818 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.818 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.818 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:04.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:04.818 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x562c5f1ec6e0": { 2024-04-16T05:46:04.818 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:04.818 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-16T05:46:04.818 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:04.819 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:04.819 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:04.819 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:04.819 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:04.819 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:04.819 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:04.819 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-16T05:46:04.819 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-16T05:46:04.819 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:04.820 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:04.820 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:04.820 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:04.820 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:04.820 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:04.820 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-16T05:46:05.542 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-16T05:46:05.542 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-16T05:46:05.542 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-16T05:46:05.543 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:46:05.543 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:46:05.543 DEBUG:tasks.check_counter:Getting stats from h 2024-04-16T05:46:05.543 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-16T05:46:05.783 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:46:06.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:06 smithi086 ceph-mon[32028]: pgmap v335: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:46:06.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:06 smithi196 ceph-mon[36373]: pgmap v335: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:46:06.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:06 smithi194 ceph-mon[35865]: pgmap v335: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:46:07.605 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-16T05:46:07.605 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:46:07.605 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 339, 2024-04-16T05:46:07.605 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 387, 2024-04-16T05:46:07.606 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1659191, 2024-04-16T05:46:07.606 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 245657, 2024-04-16T05:46:07.606 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 15, 2024-04-16T05:46:07.606 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-04-16T05:46:07.606 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.065852042, 2024-04-16T05:46:07.606 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.019789435, 2024-04-16T05:46:07.606 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.045389389, 2024-04-16T05:46:07.607 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.005901619, 2024-04-16T05:46:07.607 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:07.607 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 387, 2024-04-16T05:46:07.607 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.028556518, 2024-04-16T05:46:07.607 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000073789 2024-04-16T05:46:07.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.607 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:07.607 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 123, 2024-04-16T05:46:07.608 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000025113, 2024-04-16T05:46:07.608 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000204 2024-04-16T05:46:07.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.608 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1651232, 2024-04-16T05:46:07.608 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 155168 2024-04-16T05:46:07.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.608 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:46:07.608 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 18, 2024-04-16T05:46:07.609 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 182, 2024-04-16T05:46:07.609 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 4484, 2024-04-16T05:46:07.609 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 481629, 2024-04-16T05:46:07.609 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 12, 2024-04-16T05:46:07.609 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-04-16T05:46:07.609 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.012890593, 2024-04-16T05:46:07.609 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.007115145, 2024-04-16T05:46:07.609 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.003295674, 2024-04-16T05:46:07.610 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000321751, 2024-04-16T05:46:07.610 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:07.610 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 179, 2024-04-16T05:46:07.610 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.021702403, 2024-04-16T05:46:07.610 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000121242 2024-04-16T05:46:07.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.610 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:07.610 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-04-16T05:46:07.611 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000199, 2024-04-16T05:46:07.611 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000099 2024-04-16T05:46:07.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.611 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:46:07.611 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 451328 2024-04-16T05:46:07.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.611 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:46:07.611 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 404, 2024-04-16T05:46:07.612 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 563, 2024-04-16T05:46:07.612 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 115190, 2024-04-16T05:46:07.612 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 727598, 2024-04-16T05:46:07.612 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 41, 2024-04-16T05:46:07.612 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 7, 2024-04-16T05:46:07.612 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.068871971, 2024-04-16T05:46:07.612 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.029166096, 2024-04-16T05:46:07.612 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.032723969, 2024-04-16T05:46:07.613 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.003911997, 2024-04-16T05:46:07.613 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:07.613 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 561, 2024-04-16T05:46:07.613 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.040543892, 2024-04-16T05:46:07.613 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000072270 2024-04-16T05:46:07.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.613 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:07.613 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 890, 2024-04-16T05:46:07.614 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000188230, 2024-04-16T05:46:07.614 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000211 2024-04-16T05:46:07.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.614 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-16T05:46:07.614 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 976 2024-04-16T05:46:07.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.614 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-04-16T05:46:07.614 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-04-16T05:46:07.615 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-04-16T05:46:07.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.615 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-04-16T05:46:07.615 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-16T05:46:07.615 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-16T05:46:07.615 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 64, 2024-04-16T05:46:07.615 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001339108, 2024-04-16T05:46:07.615 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000020923 2024-04-16T05:46:07.616 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:07.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.616 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-04-16T05:46:07.616 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-16T05:46:07.616 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-16T05:46:07.616 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-04-16T05:46:07.616 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000004516, 2024-04-16T05:46:07.616 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000002258 2024-04-16T05:46:07.617 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:07.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.617 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-04-16T05:46:07.617 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 11, 2024-04-16T05:46:07.617 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 6, 2024-04-16T05:46:07.617 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-04-16T05:46:07.617 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 6, 2024-04-16T05:46:07.617 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001578182, 2024-04-16T05:46:07.618 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000263030 2024-04-16T05:46:07.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.618 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-04-16T05:46:07.618 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 5, 2024-04-16T05:46:07.618 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-04-16T05:46:07.618 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-04-16T05:46:07.618 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 11, 2024-04-16T05:46:07.618 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-04-16T05:46:07.619 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-04-16T05:46:07.619 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 14, 2024-04-16T05:46:07.619 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 0, 2024-04-16T05:46:07.619 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 14, 2024-04-16T05:46:07.619 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-04-16T05:46:07.619 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 14, 2024-04-16T05:46:07.619 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-04-16T05:46:07.619 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 3, 2024-04-16T05:46:07.620 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 3, 2024-04-16T05:46:07.620 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 2, 2024-04-16T05:46:07.620 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 14, 2024-04-16T05:46:07.620 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 9, 2024-04-16T05:46:07.620 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-04-16T05:46:07.620 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 1, 2024-04-16T05:46:07.620 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:46:07.620 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-04-16T05:46:07.621 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-04-16T05:46:07.621 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-04-16T05:46:07.621 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-04-16T05:46:07.621 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-04-16T05:46:07.621 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-04-16T05:46:07.621 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-04-16T05:46:07.621 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-04-16T05:46:07.621 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-04-16T05:46:07.622 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:46:07.622 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-04-16T05:46:07.622 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 4, 2024-04-16T05:46:07.622 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 234, 2024-04-16T05:46:07.622 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 6, 2024-04-16T05:46:07.622 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:46:07.622 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-04-16T05:46:07.623 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:46:07.623 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:46:07.623 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:46:07.623 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:46:07.623 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:46:07.623 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-04-16T05:46:07.623 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:46:07.623 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 1, 2024-04-16T05:46:07.624 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 2, 2024-04-16T05:46:07.624 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:46:07.624 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:46:07.624 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:46:07.624 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 1, 2024-04-16T05:46:07.624 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:46:07.624 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-04-16T05:46:07.624 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 3 2024-04-16T05:46:07.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.625 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-04-16T05:46:07.625 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-04-16T05:46:07.625 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-04-16T05:46:07.625 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-04-16T05:46:07.625 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 2, 2024-04-16T05:46:07.625 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-04-16T05:46:07.625 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-04-16T05:46:07.626 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-04-16T05:46:07.626 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:46:07.626 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-04-16T05:46:07.626 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-04-16T05:46:07.626 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-04-16T05:46:07.626 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-04-16T05:46:07.626 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-04-16T05:46:07.627 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:46:07.627 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:46:07.627 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-04-16T05:46:07.627 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-04-16T05:46:07.627 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:46:07.627 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:46:07.627 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:46:07.627 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-04-16T05:46:07.628 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-04-16T05:46:07.628 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:46:07.628 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-04-16T05:46:07.628 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-04-16T05:46:07.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.628 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-04-16T05:46:07.628 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-04-16T05:46:07.628 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 8, 2024-04-16T05:46:07.629 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-04-16T05:46:07.629 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-04-16T05:46:07.629 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 8, 2024-04-16T05:46:07.629 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-04-16T05:46:07.629 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-04-16T05:46:07.629 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 2, 2024-04-16T05:46:07.629 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-04-16T05:46:07.629 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-04-16T05:46:07.630 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-04-16T05:46:07.630 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-04-16T05:46:07.630 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-04-16T05:46:07.630 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-04-16T05:46:07.630 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-04-16T05:46:07.630 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4196655, 2024-04-16T05:46:07.630 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2024-04-16T05:46:07.630 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-04-16T05:46:07.631 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 7, 2024-04-16T05:46:07.631 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.013029519, 2024-04-16T05:46:07.631 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001861359 2024-04-16T05:46:07.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.631 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2024-04-16T05:46:07.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.631 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-04-16T05:46:07.631 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 17, 2024-04-16T05:46:07.632 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 17, 2024-04-16T05:46:07.632 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 0, 2024-04-16T05:46:07.632 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 15, 2024-04-16T05:46:07.632 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 16, 2024-04-16T05:46:07.632 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 1, 2024-04-16T05:46:07.632 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 14, 2024-04-16T05:46:07.632 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 14, 2024-04-16T05:46:07.632 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 0, 2024-04-16T05:46:07.632 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 3, 2024-04-16T05:46:07.633 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 4, 2024-04-16T05:46:07.633 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 1, 2024-04-16T05:46:07.633 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 50620, 2024-04-16T05:46:07.633 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 223608 2024-04-16T05:46:07.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.633 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-04-16T05:46:07.633 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 12, 2024-04-16T05:46:07.634 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-04-16T05:46:07.634 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 11, 2024-04-16T05:46:07.634 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 26, 2024-04-16T05:46:07.634 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-04-16T05:46:07.634 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-04-16T05:46:07.634 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.634 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.634 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.635 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-04-16T05:46:07.635 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5, 2024-04-16T05:46:07.635 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001408115, 2024-04-16T05:46:07.635 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000281623 2024-04-16T05:46:07.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.635 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-04-16T05:46:07.635 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.636 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.636 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.636 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-04-16T05:46:07.636 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.636 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.636 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.636 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-04-16T05:46:07.637 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.637 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.637 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.637 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-04-16T05:46:07.637 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.637 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.637 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.638 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-04-16T05:46:07.638 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1, 2024-04-16T05:46:07.638 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000170067, 2024-04-16T05:46:07.638 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000170067 2024-04-16T05:46:07.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.638 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-04-16T05:46:07.638 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.639 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.639 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.639 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-04-16T05:46:07.639 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.639 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.639 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.640 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-04-16T05:46:07.640 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.640 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.640 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.640 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-04-16T05:46:07.640 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.640 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.641 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.641 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-04-16T05:46:07.641 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.642 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.642 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.642 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-04-16T05:46:07.642 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.642 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.642 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.643 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-04-16T05:46:07.643 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.643 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.643 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.643 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-04-16T05:46:07.643 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.644 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.644 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.644 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-04-16T05:46:07.644 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.644 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.644 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.645 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-04-16T05:46:07.645 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.645 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.645 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.645 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-04-16T05:46:07.645 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.645 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.646 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.646 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-04-16T05:46:07.646 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.646 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.646 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.646 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-04-16T05:46:07.647 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.647 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.647 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.647 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-04-16T05:46:07.647 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.647 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.647 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.648 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-04-16T05:46:07.648 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.648 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.648 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.648 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-04-16T05:46:07.648 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.649 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.649 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.649 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-04-16T05:46:07.649 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.649 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.649 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.650 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-04-16T05:46:07.650 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.650 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.650 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.650 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-04-16T05:46:07.650 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.650 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.651 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.651 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-04-16T05:46:07.651 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.651 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.651 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.651 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-04-16T05:46:07.652 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.652 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.652 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.652 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-04-16T05:46:07.652 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.652 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.652 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.653 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:46:07.653 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:46:07.653 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-04-16T05:46:07.653 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.653 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.653 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.653 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:07.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.654 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-04-16T05:46:07.654 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-04-16T05:46:07.654 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 4, 2024-04-16T05:46:07.654 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 2, 2024-04-16T05:46:07.654 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 2, 2024-04-16T05:46:07.654 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-04-16T05:46:07.654 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 4, 2024-04-16T05:46:07.655 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 1, 2024-04-16T05:46:07.655 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 625, 2024-04-16T05:46:07.655 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-04-16T05:46:07.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.655 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-04-16T05:46:07.655 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:46:07.655 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-04-16T05:46:07.655 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:46:07.656 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:46:07.656 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:46:07.656 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:46:07.656 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:46:07.656 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:46:07.656 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:46:07.656 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:46:07.656 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:46:07.656 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:46:07.657 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:46:07.657 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:46:07.657 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:46:07.657 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-04-16T05:46:07.657 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:46:07.657 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-04-16T05:46:07.657 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:46:07.657 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:46:07.658 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:46:07.658 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:46:07.658 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:46:07.658 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:46:07.658 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:46:07.658 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-04-16T05:46:07.658 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:46:07.658 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:46:07.659 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:46:07.659 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-04-16T05:46:07.659 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-04-16T05:46:07.659 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-04-16T05:46:07.659 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:46:07.659 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:46:07.659 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:46:07.659 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:46:07.660 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 272919, 2024-04-16T05:46:07.660 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 105, 2024-04-16T05:46:07.660 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 88, 2024-04-16T05:46:07.660 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 1, 2024-04-16T05:46:07.660 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-04-16T05:46:07.660 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-04-16T05:46:07.660 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:46:07.660 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-04-16T05:46:07.661 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:46:07.661 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-04-16T05:46:07.661 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9512, 2024-04-16T05:46:07.661 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 182, 2024-04-16T05:46:07.661 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:46:07.661 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:46:07.661 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-04-16T05:46:07.661 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-04-16T05:46:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 68781, 2024-04-16T05:46:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 293, 2024-04-16T05:46:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-04-16T05:46:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-04-16T05:46:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-04-16T05:46:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-04-16T05:46:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-04-16T05:46:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-04-16T05:46:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-04-16T05:46:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 47, 2024-04-16T05:46:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 13298, 2024-04-16T05:46:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 28, 2024-04-16T05:46:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 47, 2024-04-16T05:46:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-04-16T05:46:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 47, 2024-04-16T05:46:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.209995743, 2024-04-16T05:46:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.004467994 2024-04-16T05:46:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-04-16T05:46:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-04-16T05:46:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 47, 2024-04-16T05:46:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 71 2024-04-16T05:46:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 47, 2024-04-16T05:46:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 0, 2024-04-16T05:46:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 47, 2024-04-16T05:46:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-04-16T05:46:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-04-16T05:46:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 0, 2024-04-16T05:46:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 10, 2024-04-16T05:46:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 0, 2024-04-16T05:46:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 7, 2024-04-16T05:46:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 9, 2024-04-16T05:46:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-04-16T05:46:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-04-16T05:46:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-04-16T05:46:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-04-16T05:46:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 6, 2024-04-16T05:46:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-04-16T05:46:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-04-16T05:46:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-04-16T05:46:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 0, 2024-04-16T05:46:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 20, 2024-04-16T05:46:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:46:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-04-16T05:46:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:46:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-04-16T05:46:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-04-16T05:46:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-04-16T05:46:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:46:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-04-16T05:46:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:46:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-04-16T05:46:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-04-16T05:46:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-04-16T05:46:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-04-16T05:46:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-04-16T05:46:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-04-16T05:46:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-04-16T05:46:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-04-16T05:46:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-04-16T05:46:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-04-16T05:46:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-04-16T05:46:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-04-16T05:46:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-04-16T05:46:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-04-16T05:46:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-04-16T05:46:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-04-16T05:46:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-04-16T05:46:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 103, 2024-04-16T05:46:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-04-16T05:46:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 13, 2024-04-16T05:46:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 9, 2024-04-16T05:46:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 9, 2024-04-16T05:46:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-04-16T05:46:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-04-16T05:46:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 18, 2024-04-16T05:46:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 0, 2024-04-16T05:46:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 1 2024-04-16T05:46:07.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.672 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-04-16T05:46:07.672 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2024-04-16T05:46:07.672 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:46:07.672 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-04-16T05:46:07.672 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-04-16T05:46:07.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.672 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-04-16T05:46:07.672 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-04-16T05:46:07.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.673 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-04-16T05:46:07.673 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-04-16T05:46:07.673 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:46:07.673 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-04-16T05:46:07.673 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-04-16T05:46:07.673 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-04-16T05:46:07.673 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-04-16T05:46:07.674 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-04-16T05:46:07.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.674 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:46:07.674 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:07.674 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-16T05:46:07.674 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:07.674 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 757, 2024-04-16T05:46:07.675 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 1692566, 2024-04-16T05:46:07.675 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:07.675 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 757, 2024-04-16T05:46:07.675 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:07.675 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:07.675 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 757, 2024-04-16T05:46:07.675 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1692566, 2024-04-16T05:46:07.675 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:07.676 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.676 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.676 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.676 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:07.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.676 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-04-16T05:46:07.676 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:07.676 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-16T05:46:07.677 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:07.677 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:07.677 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:07.677 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:07.677 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:07.677 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 47, 2024-04-16T05:46:07.677 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 13298, 2024-04-16T05:46:07.677 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 47, 2024-04-16T05:46:07.677 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 13298, 2024-04-16T05:46:07.678 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:07.678 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.678 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.678 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.678 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:07.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.678 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-04-16T05:46:07.679 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:07.679 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-04-16T05:46:07.679 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:07.679 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:07.679 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:07.679 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:07.679 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:07.679 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 47, 2024-04-16T05:46:07.680 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 47, 2024-04-16T05:46:07.680 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 47, 2024-04-16T05:46:07.680 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 47, 2024-04-16T05:46:07.680 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:07.680 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.680 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.680 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.680 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:07.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.681 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:46:07.681 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:07.681 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-16T05:46:07.681 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:07.681 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:07.681 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:07.681 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:07.681 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:07.682 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:07.682 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:07.682 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-16T05:46:07.682 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-16T05:46:07.682 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:07.682 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.682 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.682 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.683 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:07.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:07.683 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55a76c27a500": { 2024-04-16T05:46:07.683 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:07.683 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-16T05:46:07.683 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:07.683 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 8, 2024-04-16T05:46:07.683 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2351, 2024-04-16T05:46:07.684 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:07.684 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 8, 2024-04-16T05:46:07.684 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:07.684 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:07.684 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 7, 2024-04-16T05:46:07.684 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2351, 2024-04-16T05:46:07.684 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:07.684 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:07.684 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:07.685 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:07.685 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:07.685 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:07.685 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-16T05:46:08.390 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:08 smithi194 ceph-mon[35865]: pgmap v336: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:46:08.418 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-16T05:46:08.419 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-16T05:46:08.419 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-16T05:46:08.419 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:46:08.419 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:46:08.419 DEBUG:tasks.check_counter:Getting stats from k 2024-04-16T05:46:08.419 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-16T05:46:08.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:08 smithi086 ceph-mon[32028]: pgmap v336: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:46:08.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:08 smithi196 ceph-mon[36373]: pgmap v336: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-16T05:46:08.659 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:46:10.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:10 smithi086 ceph-mon[32028]: pgmap v337: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:46:10.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:10 smithi196 ceph-mon[36373]: pgmap v337: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:46:10.495 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:10 smithi194 ceph-mon[35865]: pgmap v337: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-16T05:46:10.519 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-16T05:46:10.519 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:46:10.519 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 781, 2024-04-16T05:46:10.519 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 783, 2024-04-16T05:46:10.519 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 208032, 2024-04-16T05:46:10.519 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 248422, 2024-04-16T05:46:10.520 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 6, 2024-04-16T05:46:10.520 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-04-16T05:46:10.520 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.071904082, 2024-04-16T05:46:10.520 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.028551562, 2024-04-16T05:46:10.520 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.017294576, 2024-04-16T05:46:10.520 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.019312454, 2024-04-16T05:46:10.520 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:10.520 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 781, 2024-04-16T05:46:10.521 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.026422204, 2024-04-16T05:46:10.521 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000033831 2024-04-16T05:46:10.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.521 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:10.521 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.521 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.521 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.521 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:46:10.522 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:46:10.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.522 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:46:10.522 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 964, 2024-04-16T05:46:10.522 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 952, 2024-04-16T05:46:10.522 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1802234, 2024-04-16T05:46:10.522 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 366105, 2024-04-16T05:46:10.522 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 4, 2024-04-16T05:46:10.522 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-04-16T05:46:10.523 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.118173976, 2024-04-16T05:46:10.523 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.033647466, 2024-04-16T05:46:10.523 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.056236705, 2024-04-16T05:46:10.523 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.033088041, 2024-04-16T05:46:10.523 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:10.523 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 951, 2024-04-16T05:46:10.523 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.034511077, 2024-04-16T05:46:10.523 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000036289 2024-04-16T05:46:10.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.524 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:10.524 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.524 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.524 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.524 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1627552, 2024-04-16T05:46:10.524 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 140320 2024-04-16T05:46:10.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.524 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:46:10.525 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 777, 2024-04-16T05:46:10.525 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 943, 2024-04-16T05:46:10.525 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 262400, 2024-04-16T05:46:10.525 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 697658, 2024-04-16T05:46:10.525 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 6, 2024-04-16T05:46:10.525 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-04-16T05:46:10.525 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.095304647, 2024-04-16T05:46:10.525 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.035737869, 2024-04-16T05:46:10.525 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.021907604, 2024-04-16T05:46:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.026698257, 2024-04-16T05:46:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 941, 2024-04-16T05:46:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.056551470, 2024-04-16T05:46:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000060097 2024-04-16T05:46:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-16T05:46:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 452480 2024-04-16T05:46:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-04-16T05:46:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-04-16T05:46:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-04-16T05:46:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-04-16T05:46:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-16T05:46:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-16T05:46:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3746, 2024-04-16T05:46:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.106925008, 2024-04-16T05:46:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000028543 2024-04-16T05:46:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-04-16T05:46:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-16T05:46:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-16T05:46:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-04-16T05:46:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-04-16T05:46:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-04-16T05:46:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-04-16T05:46:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-04-16T05:46:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-04-16T05:46:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-04-16T05:46:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-04-16T05:46:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-04-16T05:46:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-04-16T05:46:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-04-16T05:46:10.532 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 0, 2024-04-16T05:46:10.532 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 0, 2024-04-16T05:46:10.532 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 0, 2024-04-16T05:46:10.532 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-04-16T05:46:10.532 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 0, 2024-04-16T05:46:10.532 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-04-16T05:46:10.532 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-04-16T05:46:10.532 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-04-16T05:46:10.532 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 1, 2024-04-16T05:46:10.533 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-04-16T05:46:10.533 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-04-16T05:46:10.533 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-04-16T05:46:10.533 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-04-16T05:46:10.533 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:46:10.533 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-04-16T05:46:10.533 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-04-16T05:46:10.533 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-04-16T05:46:10.534 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-04-16T05:46:10.534 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-04-16T05:46:10.534 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-04-16T05:46:10.534 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-04-16T05:46:10.534 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-04-16T05:46:10.534 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-04-16T05:46:10.534 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:46:10.534 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-04-16T05:46:10.534 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 0, 2024-04-16T05:46:10.535 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 0, 2024-04-16T05:46:10.535 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-04-16T05:46:10.535 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:46:10.535 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-04-16T05:46:10.535 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:46:10.535 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:46:10.535 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:46:10.535 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:46:10.536 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:46:10.536 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-04-16T05:46:10.536 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:46:10.536 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:46:10.536 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:46:10.536 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:46:10.536 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:46:10.536 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:46:10.536 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-04-16T05:46:10.537 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:46:10.537 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-04-16T05:46:10.537 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-04-16T05:46:10.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.537 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-04-16T05:46:10.537 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-04-16T05:46:10.537 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-04-16T05:46:10.537 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-04-16T05:46:10.537 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-04-16T05:46:10.538 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-04-16T05:46:10.538 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-04-16T05:46:10.538 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-04-16T05:46:10.538 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:46:10.538 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-04-16T05:46:10.538 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-04-16T05:46:10.538 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-04-16T05:46:10.538 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-04-16T05:46:10.539 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-04-16T05:46:10.539 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:46:10.539 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:46:10.539 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-04-16T05:46:10.539 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-04-16T05:46:10.539 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:46:10.539 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:46:10.539 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:46:10.539 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-04-16T05:46:10.540 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-04-16T05:46:10.540 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:46:10.540 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-04-16T05:46:10.540 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-04-16T05:46:10.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.540 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-04-16T05:46:10.540 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-04-16T05:46:10.540 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-04-16T05:46:10.541 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-04-16T05:46:10.541 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-04-16T05:46:10.541 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 10, 2024-04-16T05:46:10.541 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-04-16T05:46:10.541 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-04-16T05:46:10.541 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-04-16T05:46:10.541 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-04-16T05:46:10.541 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-04-16T05:46:10.542 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-04-16T05:46:10.542 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-04-16T05:46:10.542 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-04-16T05:46:10.542 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-04-16T05:46:10.542 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-04-16T05:46:10.542 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4197187, 2024-04-16T05:46:10.542 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 4197046, 2024-04-16T05:46:10.542 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-04-16T05:46:10.542 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.543 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.543 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.543 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 10 2024-04-16T05:46:10.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.543 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-04-16T05:46:10.543 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 3, 2024-04-16T05:46:10.544 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 3, 2024-04-16T05:46:10.544 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 0, 2024-04-16T05:46:10.544 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 1, 2024-04-16T05:46:10.544 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 1, 2024-04-16T05:46:10.544 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 0, 2024-04-16T05:46:10.544 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 0, 2024-04-16T05:46:10.544 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 0, 2024-04-16T05:46:10.544 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 0, 2024-04-16T05:46:10.544 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-04-16T05:46:10.545 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-04-16T05:46:10.545 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-04-16T05:46:10.545 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 48200, 2024-04-16T05:46:10.545 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215416 2024-04-16T05:46:10.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.545 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-04-16T05:46:10.545 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-04-16T05:46:10.545 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-04-16T05:46:10.545 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-04-16T05:46:10.546 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-04-16T05:46:10.546 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-04-16T05:46:10.546 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-04-16T05:46:10.546 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.546 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.546 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.547 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-04-16T05:46:10.547 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.547 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.547 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.547 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-04-16T05:46:10.547 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.547 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.548 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.548 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-04-16T05:46:10.548 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.548 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.548 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.548 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-04-16T05:46:10.548 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.549 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.549 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.549 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-04-16T05:46:10.549 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.549 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.549 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.550 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-04-16T05:46:10.550 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.550 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.550 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.550 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-04-16T05:46:10.550 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.550 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.551 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.551 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-04-16T05:46:10.551 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.551 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.551 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.551 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-04-16T05:46:10.552 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.552 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.552 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.552 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-04-16T05:46:10.552 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.552 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.552 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.553 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-04-16T05:46:10.553 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.553 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.553 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.554 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-04-16T05:46:10.554 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.554 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.554 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.554 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-04-16T05:46:10.555 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.555 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.555 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.555 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-04-16T05:46:10.555 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.555 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.555 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.556 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-04-16T05:46:10.556 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.556 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.556 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.556 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-04-16T05:46:10.556 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.557 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.557 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.557 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-04-16T05:46:10.557 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.557 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.557 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.558 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-04-16T05:46:10.558 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.558 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.558 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.558 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-04-16T05:46:10.558 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.558 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.559 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.559 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-04-16T05:46:10.559 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.559 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.559 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.559 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-04-16T05:46:10.560 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.560 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.560 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.560 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-04-16T05:46:10.560 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.560 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.560 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.561 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-04-16T05:46:10.561 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.561 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.561 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.561 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-04-16T05:46:10.561 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.562 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.562 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.562 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-04-16T05:46:10.562 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.562 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.562 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.563 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-04-16T05:46:10.563 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.563 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.563 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.563 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-04-16T05:46:10.563 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.563 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.564 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.564 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-04-16T05:46:10.564 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.564 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.564 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.564 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:46:10.565 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:46:10.565 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-04-16T05:46:10.565 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.565 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.565 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.565 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:10.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.565 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-04-16T05:46:10.566 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-04-16T05:46:10.566 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 5, 2024-04-16T05:46:10.566 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 3, 2024-04-16T05:46:10.566 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 4, 2024-04-16T05:46:10.566 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-04-16T05:46:10.566 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-04-16T05:46:10.566 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-04-16T05:46:10.566 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 3669, 2024-04-16T05:46:10.566 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-04-16T05:46:10.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.567 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-04-16T05:46:10.567 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:46:10.567 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-04-16T05:46:10.567 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:46:10.567 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:46:10.567 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:46:10.567 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:46:10.568 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:46:10.568 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:46:10.568 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:46:10.568 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:46:10.568 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:46:10.568 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:46:10.568 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:46:10.568 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:46:10.569 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:46:10.569 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-04-16T05:46:10.569 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:46:10.569 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-04-16T05:46:10.569 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:46:10.569 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:46:10.569 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:46:10.569 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:46:10.569 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:46:10.570 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:46:10.570 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:46:10.570 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-04-16T05:46:10.570 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:46:10.570 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:46:10.570 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:46:10.570 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-04-16T05:46:10.570 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-04-16T05:46:10.571 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-04-16T05:46:10.571 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:46:10.571 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:46:10.571 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:46:10.571 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:46:10.571 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 189393, 2024-04-16T05:46:10.571 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 51, 2024-04-16T05:46:10.571 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:46:10.571 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-04-16T05:46:10.572 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-04-16T05:46:10.572 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-04-16T05:46:10.572 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:46:10.572 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-04-16T05:46:10.572 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:46:10.572 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-04-16T05:46:10.572 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9752, 2024-04-16T05:46:10.572 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 188, 2024-04-16T05:46:10.573 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:46:10.573 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:46:10.573 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-04-16T05:46:10.573 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-04-16T05:46:10.573 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 8512, 2024-04-16T05:46:10.573 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 13, 2024-04-16T05:46:10.573 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-04-16T05:46:10.573 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-04-16T05:46:10.573 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-04-16T05:46:10.574 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-04-16T05:46:10.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.574 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-04-16T05:46:10.574 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-04-16T05:46:10.574 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-04-16T05:46:10.574 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 2262, 2024-04-16T05:46:10.574 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 21, 2024-04-16T05:46:10.574 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 3, 2024-04-16T05:46:10.575 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 2262, 2024-04-16T05:46:10.575 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-04-16T05:46:10.575 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2262, 2024-04-16T05:46:10.575 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.056978506, 2024-04-16T05:46:10.575 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000467276 2024-04-16T05:46:10.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.575 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-04-16T05:46:10.575 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-04-16T05:46:10.576 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2262, 2024-04-16T05:46:10.576 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 2263 2024-04-16T05:46:10.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.576 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 2262, 2024-04-16T05:46:10.576 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 2262, 2024-04-16T05:46:10.576 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-04-16T05:46:10.576 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-04-16T05:46:10.576 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-04-16T05:46:10.576 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 1479, 2024-04-16T05:46:10.577 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-04-16T05:46:10.577 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 781, 2024-04-16T05:46:10.577 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-04-16T05:46:10.577 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-04-16T05:46:10.577 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-04-16T05:46:10.577 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-04-16T05:46:10.577 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-04-16T05:46:10.577 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-04-16T05:46:10.578 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-04-16T05:46:10.578 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-04-16T05:46:10.578 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-04-16T05:46:10.578 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-04-16T05:46:10.578 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 1, 2024-04-16T05:46:10.578 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-04-16T05:46:10.578 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:46:10.578 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-04-16T05:46:10.579 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:46:10.579 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-04-16T05:46:10.579 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-04-16T05:46:10.579 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-04-16T05:46:10.579 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:46:10.579 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-04-16T05:46:10.579 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:46:10.579 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-04-16T05:46:10.579 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-04-16T05:46:10.580 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-04-16T05:46:10.580 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-04-16T05:46:10.580 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-04-16T05:46:10.580 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-04-16T05:46:10.580 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-04-16T05:46:10.580 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-04-16T05:46:10.580 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-04-16T05:46:10.580 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-04-16T05:46:10.581 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-04-16T05:46:10.581 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-04-16T05:46:10.581 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-04-16T05:46:10.581 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-04-16T05:46:10.581 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-04-16T05:46:10.581 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-04-16T05:46:10.581 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-04-16T05:46:10.581 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 104, 2024-04-16T05:46:10.581 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-04-16T05:46:10.582 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 14, 2024-04-16T05:46:10.582 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 5, 2024-04-16T05:46:10.582 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 5, 2024-04-16T05:46:10.582 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-04-16T05:46:10.582 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-04-16T05:46:10.582 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-04-16T05:46:10.582 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-04-16T05:46:10.582 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-04-16T05:46:10.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.583 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-04-16T05:46:10.583 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-04-16T05:46:10.583 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:46:10.583 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-04-16T05:46:10.583 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-04-16T05:46:10.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.583 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-04-16T05:46:10.584 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-04-16T05:46:10.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.584 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-04-16T05:46:10.584 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-04-16T05:46:10.584 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:46:10.584 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-04-16T05:46:10.584 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-04-16T05:46:10.584 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-04-16T05:46:10.585 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-04-16T05:46:10.585 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-04-16T05:46:10.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.585 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:46:10.585 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:10.585 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-16T05:46:10.585 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:10.585 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 2518, 2024-04-16T05:46:10.585 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2029987, 2024-04-16T05:46:10.586 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:10.586 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 2518, 2024-04-16T05:46:10.586 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:10.586 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:10.586 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 2518, 2024-04-16T05:46:10.586 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2029987, 2024-04-16T05:46:10.586 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:10.586 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.587 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.587 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.587 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:10.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.587 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-04-16T05:46:10.587 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:10.587 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-16T05:46:10.587 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:10.587 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:10.588 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:10.588 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:10.588 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:10.588 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 2262, 2024-04-16T05:46:10.588 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 2888, 2024-04-16T05:46:10.588 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 2262, 2024-04-16T05:46:10.588 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2888, 2024-04-16T05:46:10.588 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:10.589 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.589 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.589 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.589 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:10.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.589 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-04-16T05:46:10.589 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:10.589 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-04-16T05:46:10.589 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:10.590 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:10.590 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:10.590 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:10.590 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:10.590 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 2262, 2024-04-16T05:46:10.590 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 2262, 2024-04-16T05:46:10.590 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 2262, 2024-04-16T05:46:10.590 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2262, 2024-04-16T05:46:10.591 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:10.591 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.591 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.591 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.591 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:10.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.591 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:46:10.591 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:10.591 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-16T05:46:10.592 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:10.592 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:10.592 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:10.592 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:10.592 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:10.592 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:10.592 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:10.592 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-16T05:46:10.592 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-16T05:46:10.593 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:10.593 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.593 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.593 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.593 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:10.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-16T05:46:10.593 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x5598b62e66e0": { 2024-04-16T05:46:10.593 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-16T05:46:10.593 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-16T05:46:10.594 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-16T05:46:10.594 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-16T05:46:10.594 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-16T05:46:10.594 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:10.594 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:10.594 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-16T05:46:10.594 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-16T05:46:10.594 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-16T05:46:10.595 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-16T05:46:10.595 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-16T05:46:10.595 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-16T05:46:10.595 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-16T05:46:10.595 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-16T05:46:10.595 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:10.595 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-16T05:46:10.595 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-16T05:46:11.308 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:11 smithi194 ceph-mon[35865]: osdmap e104: 12 total, 12 up, 12 in 2024-04-16T05:46:11.335 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-16T05:46:11.335 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-16T05:46:11.335 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-16T05:46:11.336 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:46:11.336 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:46:11.336 DEBUG:tasks.check_counter:Getting stats from c 2024-04-16T05:46:11.336 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-16T05:46:11.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:11 smithi196 ceph-mon[36373]: osdmap e104: 12 total, 12 up, 12 in 2024-04-16T05:46:11.591 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:46:11.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:11 smithi086 ceph-mon[32028]: osdmap e104: 12 total, 12 up, 12 in 2024-04-16T05:46:12.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:12 smithi196 ceph-mon[36373]: pgmap v339: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-16T05:46:12.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:12 smithi086 ceph-mon[32028]: pgmap v339: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-16T05:46:12.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:12 smithi194 ceph-mon[35865]: pgmap v339: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-16T05:46:13.449 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-16T05:46:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:46:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 48, 2024-04-16T05:46:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 82, 2024-04-16T05:46:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 10877, 2024-04-16T05:46:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 71909, 2024-04-16T05:46:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 14, 2024-04-16T05:46:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-16T05:46:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.017748243, 2024-04-16T05:46:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.003894276, 2024-04-16T05:46:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.008089263, 2024-04-16T05:46:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000759521, 2024-04-16T05:46:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 80, 2024-04-16T05:46:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.016358700, 2024-04-16T05:46:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000204483 2024-04-16T05:46:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 58, 2024-04-16T05:46:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000014239, 2024-04-16T05:46:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000245 2024-04-16T05:46:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-16T05:46:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 736 2024-04-16T05:46:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:46:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 314, 2024-04-16T05:46:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 444, 2024-04-16T05:46:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1657087, 2024-04-16T05:46:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 509677, 2024-04-16T05:46:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 9, 2024-04-16T05:46:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-04-16T05:46:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.062132722, 2024-04-16T05:46:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.022520490, 2024-04-16T05:46:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.042494694, 2024-04-16T05:46:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.006726995, 2024-04-16T05:46:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 443, 2024-04-16T05:46:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.037108751, 2024-04-16T05:46:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000083766 2024-04-16T05:46:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 36, 2024-04-16T05:46:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000009566, 2024-04-16T05:46:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000265 2024-04-16T05:46:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1653008, 2024-04-16T05:46:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 474128 2024-04-16T05:46:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:46:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 401, 2024-04-16T05:46:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 608, 2024-04-16T05:46:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 112055, 2024-04-16T05:46:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 890148, 2024-04-16T05:46:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 45, 2024-04-16T05:46:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-04-16T05:46:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.071451625, 2024-04-16T05:46:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.026154625, 2024-04-16T05:46:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.036395305, 2024-04-16T05:46:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.004780424, 2024-04-16T05:46:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 606, 2024-04-16T05:46:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.033078500, 2024-04-16T05:46:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000054584 2024-04-16T05:46:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 911, 2024-04-16T05:46:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000277633, 2024-04-16T05:46:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000304 2024-04-16T05:46:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 5968, 2024-04-16T05:46:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 133184 2024-04-16T05:46:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-16T05:46:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-16T05:46:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-16T05:46:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-16T05:46:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-16T05:46:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-16T05:46:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 60, 2024-04-16T05:46:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002191627, 2024-04-16T05:46:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000036527 2024-04-16T05:46:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-16T05:46:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-16T05:46:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-16T05:46:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-16T05:46:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000007244, 2024-04-16T05:46:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000003622 2024-04-16T05:46:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-16T05:46:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 4, 2024-04-16T05:46:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 4, 2024-04-16T05:46:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-16T05:46:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4, 2024-04-16T05:46:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002429531, 2024-04-16T05:46:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000607382 2024-04-16T05:46:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-16T05:46:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-16T05:46:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-16T05:46:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-16T05:46:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-04-16T05:46:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-16T05:46:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-16T05:46:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 11, 2024-04-16T05:46:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-16T05:46:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 11, 2024-04-16T05:46:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-16T05:46:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 10, 2024-04-16T05:46:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 1, 2024-04-16T05:46:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 1, 2024-04-16T05:46:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 1, 2024-04-16T05:46:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 2, 2024-04-16T05:46:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 8, 2024-04-16T05:46:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 4, 2024-04-16T05:46:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-16T05:46:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 2, 2024-04-16T05:46:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:46:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-16T05:46:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-04-16T05:46:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-16T05:46:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-16T05:46:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-16T05:46:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-16T05:46:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-16T05:46:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-16T05:46:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-16T05:46:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:46:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-16T05:46:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-04-16T05:46:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-04-16T05:46:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-04-16T05:46:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:46:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-16T05:46:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:46:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:46:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:46:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:46:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:46:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-16T05:46:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:46:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:46:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:46:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:46:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:46:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:46:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-16T05:46:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:46:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-16T05:46:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-16T05:46:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-16T05:46:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-16T05:46:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-04-16T05:46:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-04-16T05:46:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 3, 2024-04-16T05:46:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-16T05:46:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-16T05:46:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-16T05:46:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:46:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-16T05:46:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-16T05:46:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-16T05:46:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-16T05:46:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-16T05:46:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:46:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:46:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-16T05:46:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-16T05:46:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:46:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:46:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:46:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-16T05:46:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-16T05:46:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:46:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-16T05:46:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-16T05:46:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-16T05:46:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-16T05:46:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 10, 2024-04-16T05:46:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-16T05:46:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-16T05:46:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 10, 2024-04-16T05:46:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-16T05:46:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-16T05:46:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-04-16T05:46:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-16T05:46:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-16T05:46:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-16T05:46:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-16T05:46:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-16T05:46:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-16T05:46:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-16T05:46:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4197187, 2024-04-16T05:46:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-04-16T05:46:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-16T05:46:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 9, 2024-04-16T05:46:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.016742478, 2024-04-16T05:46:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001860275 2024-04-16T05:46:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-04-16T05:46:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-16T05:46:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 14, 2024-04-16T05:46:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 14, 2024-04-16T05:46:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-04-16T05:46:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 13, 2024-04-16T05:46:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 14, 2024-04-16T05:46:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 1, 2024-04-16T05:46:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 11, 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 12, 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 1, 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 1, 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 3, 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 2, 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50192, 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223608 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-16T05:46:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 9, 2024-04-16T05:46:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-16T05:46:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 4, 2024-04-16T05:46:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 30, 2024-04-16T05:46:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 1, 2024-04-16T05:46:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-16T05:46:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-16T05:46:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-04-16T05:46:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001583654, 2024-04-16T05:46:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001583654 2024-04-16T05:46:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-16T05:46:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-16T05:46:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-16T05:46:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-16T05:46:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000694225, 2024-04-16T05:46:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000347112 2024-04-16T05:46:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-16T05:46:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-16T05:46:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-04-16T05:46:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000151652, 2024-04-16T05:46:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000151652 2024-04-16T05:46:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-16T05:46:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-16T05:46:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-16T05:46:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-16T05:46:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-16T05:46:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-16T05:46:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-16T05:46:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-16T05:46:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-16T05:46:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-16T05:46:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-16T05:46:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-16T05:46:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-16T05:46:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-16T05:46:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-16T05:46:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-16T05:46:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-16T05:46:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.491 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-16T05:46:13.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.491 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.491 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-16T05:46:13.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.491 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.493 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-16T05:46:13.493 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.493 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.493 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.493 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:46:13.493 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:46:13.493 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-16T05:46:13.494 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.494 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.494 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.494 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:13.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.494 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-16T05:46:13.494 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-04-16T05:46:13.494 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 5, 2024-04-16T05:46:13.495 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 3, 2024-04-16T05:46:13.495 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-04-16T05:46:13.495 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-16T05:46:13.495 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 3, 2024-04-16T05:46:13.495 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 1, 2024-04-16T05:46:13.495 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 626, 2024-04-16T05:46:13.495 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-16T05:46:13.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.495 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-16T05:46:13.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:46:13.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-16T05:46:13.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:46:13.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:46:13.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:46:13.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:46:13.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:46:13.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:46:13.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:46:13.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:46:13.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:46:13.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:46:13.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:46:13.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:46:13.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:46:13.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-16T05:46:13.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:46:13.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-16T05:46:13.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:46:13.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:46:13.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:46:13.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:46:13.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:46:13.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:46:13.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:46:13.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:46:13.499 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:46:13.500 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 247559, 2024-04-16T05:46:13.500 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 91, 2024-04-16T05:46:13.500 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:46:13.500 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-16T05:46:13.500 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-16T05:46:13.500 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-16T05:46:13.500 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:46:13.500 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-16T05:46:13.500 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:46:13.501 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-16T05:46:13.501 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9512, 2024-04-16T05:46:13.501 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 182, 2024-04-16T05:46:13.501 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:46:13.501 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:46:13.501 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-16T05:46:13.501 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-16T05:46:13.501 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 56928, 2024-04-16T05:46:13.501 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 228, 2024-04-16T05:46:13.502 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-16T05:46:13.502 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-16T05:46:13.502 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-16T05:46:13.502 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-16T05:46:13.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.502 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-16T05:46:13.502 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-16T05:46:13.502 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-16T05:46:13.503 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 49, 2024-04-16T05:46:13.503 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 13510, 2024-04-16T05:46:13.503 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-04-16T05:46:13.503 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 49, 2024-04-16T05:46:13.503 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-16T05:46:13.503 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 49, 2024-04-16T05:46:13.503 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.229995041, 2024-04-16T05:46:13.503 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004693776 2024-04-16T05:46:13.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.504 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-16T05:46:13.504 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-16T05:46:13.504 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 49, 2024-04-16T05:46:13.504 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 73 2024-04-16T05:46:13.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.504 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 49, 2024-04-16T05:46:13.504 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-04-16T05:46:13.504 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 49, 2024-04-16T05:46:13.504 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-16T05:46:13.505 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-16T05:46:13.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-04-16T05:46:13.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-04-16T05:46:13.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-04-16T05:46:13.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 9, 2024-04-16T05:46:13.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 9, 2024-04-16T05:46:13.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-16T05:46:13.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-16T05:46:13.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-16T05:46:13.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-16T05:46:13.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-04-16T05:46:13.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-16T05:46:13.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-16T05:46:13.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-16T05:46:13.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-04-16T05:46:13.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-04-16T05:46:13.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:46:13.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-16T05:46:13.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:46:13.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-16T05:46:13.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-16T05:46:13.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-16T05:46:13.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:46:13.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-16T05:46:13.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:46:13.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-16T05:46:13.507 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-16T05:46:13.508 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-16T05:46:13.508 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-16T05:46:13.508 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-16T05:46:13.508 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-16T05:46:13.508 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-16T05:46:13.508 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-16T05:46:13.508 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-16T05:46:13.508 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-16T05:46:13.508 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-16T05:46:13.509 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-16T05:46:13.509 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-16T05:46:13.509 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-16T05:46:13.509 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-16T05:46:13.509 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-16T05:46:13.509 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-16T05:46:13.509 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 104, 2024-04-16T05:46:13.509 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-16T05:46:13.510 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 14, 2024-04-16T05:46:13.510 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 9, 2024-04-16T05:46:13.510 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 9, 2024-04-16T05:46:13.510 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-04-16T05:46:13.510 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-16T05:46:13.510 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 17, 2024-04-16T05:46:13.510 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-04-16T05:46:13.510 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 2 2024-04-16T05:46:13.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.511 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-16T05:46:13.511 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-04-16T05:46:13.511 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:46:13.511 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-16T05:46:13.511 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-16T05:46:13.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.511 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-16T05:46:13.511 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-04-16T05:46:13.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.512 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-16T05:46:13.512 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-16T05:46:13.512 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:46:13.512 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-16T05:46:13.512 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-16T05:46:13.512 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-16T05:46:13.512 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-16T05:46:13.512 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-16T05:46:13.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.513 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:46:13.513 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:13.513 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-16T05:46:13.513 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:13.513 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 759, 2024-04-16T05:46:13.513 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1693064, 2024-04-16T05:46:13.513 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:13.513 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 759, 2024-04-16T05:46:13.513 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:13.514 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:13.514 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 759, 2024-04-16T05:46:13.514 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1693064, 2024-04-16T05:46:13.514 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:13.514 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.514 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.514 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.514 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:13.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.515 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-16T05:46:13.515 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:13.515 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-16T05:46:13.515 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:13.515 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:13.515 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:13.515 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:13.515 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:13.515 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 49, 2024-04-16T05:46:13.516 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 13510, 2024-04-16T05:46:13.516 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 49, 2024-04-16T05:46:13.516 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 13510, 2024-04-16T05:46:13.516 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:13.516 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.516 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.516 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.516 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:13.517 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 49, 2024-04-16T05:46:13.518 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 49, 2024-04-16T05:46:13.518 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 49, 2024-04-16T05:46:13.518 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 49, 2024-04-16T05:46:13.518 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:13.518 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.518 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.518 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.518 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:13.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:46:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-16T05:46:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-16T05:46:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-16T05:46:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55943dae0500": { 2024-04-16T05:46:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-16T05:46:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 10, 2024-04-16T05:46:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2883, 2024-04-16T05:46:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 10, 2024-04-16T05:46:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 9, 2024-04-16T05:46:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2883, 2024-04-16T05:46:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:13.523 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-16T05:46:14.413 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-16T05:46:14.413 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-16T05:46:14.413 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-16T05:46:14.413 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:46:14.413 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:46:14.413 DEBUG:tasks.check_counter:Getting stats from f 2024-04-16T05:46:14.414 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-16T05:46:14.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:14 smithi196 ceph-mon[36373]: pgmap v340: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:46:14.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:14 smithi194 ceph-mon[35865]: pgmap v340: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:46:14.657 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:46:14.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:14 smithi086 ceph-mon[32028]: pgmap v340: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:46:16.056 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:16 smithi196 ceph-mon[36373]: osdmap e105: 12 total, 12 up, 12 in 2024-04-16T05:46:16.056 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:16 smithi196 ceph-mon[36373]: pgmap v342: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:46:16.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:16 smithi194 ceph-mon[35865]: osdmap e105: 12 total, 12 up, 12 in 2024-04-16T05:46:16.405 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:16 smithi194 ceph-mon[35865]: pgmap v342: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:46:16.418 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-16T05:46:16.418 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:46:16.418 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 346, 2024-04-16T05:46:16.418 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 512, 2024-04-16T05:46:16.418 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 97098, 2024-04-16T05:46:16.418 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 185255, 2024-04-16T05:46:16.419 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-04-16T05:46:16.419 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-04-16T05:46:16.419 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.057169699, 2024-04-16T05:46:16.419 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.021958218, 2024-04-16T05:46:16.419 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.024121089, 2024-04-16T05:46:16.419 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.004888327, 2024-04-16T05:46:16.419 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:16.419 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 510, 2024-04-16T05:46:16.420 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.041157527, 2024-04-16T05:46:16.420 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000080701 2024-04-16T05:46:16.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.420 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:16.420 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 823, 2024-04-16T05:46:16.420 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000208503, 2024-04-16T05:46:16.420 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000253 2024-04-16T05:46:16.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.421 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:46:16.421 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:46:16.421 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.421 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:46:16.421 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 310, 2024-04-16T05:46:16.421 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 312, 2024-04-16T05:46:16.421 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1645354, 2024-04-16T05:46:16.421 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 231303, 2024-04-16T05:46:16.422 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 13, 2024-04-16T05:46:16.422 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-16T05:46:16.422 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.058995454, 2024-04-16T05:46:16.422 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.017344122, 2024-04-16T05:46:16.422 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.046170569, 2024-04-16T05:46:16.422 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.006937552, 2024-04-16T05:46:16.422 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:16.422 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 311, 2024-04-16T05:46:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.026439239, 2024-04-16T05:46:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000085013 2024-04-16T05:46:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 40, 2024-04-16T05:46:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000008465, 2024-04-16T05:46:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000211 2024-04-16T05:46:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1641984, 2024-04-16T05:46:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 150528 2024-04-16T05:46:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:46:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 57, 2024-04-16T05:46:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 222, 2024-04-16T05:46:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 12627, 2024-04-16T05:46:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 883295, 2024-04-16T05:46:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 44, 2024-04-16T05:46:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-04-16T05:46:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.025000627, 2024-04-16T05:46:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.007619758, 2024-04-16T05:46:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.013554771, 2024-04-16T05:46:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.001009386, 2024-04-16T05:46:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 220, 2024-04-16T05:46:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.035197271, 2024-04-16T05:46:16.426 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000159987 2024-04-16T05:46:16.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.426 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:16.426 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 36, 2024-04-16T05:46:16.426 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000002981, 2024-04-16T05:46:16.426 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000082 2024-04-16T05:46:16.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.426 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-16T05:46:16.427 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 453920 2024-04-16T05:46:16.427 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.427 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-16T05:46:16.427 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-16T05:46:16.427 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-16T05:46:16.427 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.427 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-16T05:46:16.428 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-16T05:46:16.428 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-16T05:46:16.428 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 38, 2024-04-16T05:46:16.428 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001783683, 2024-04-16T05:46:16.428 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000046939 2024-04-16T05:46:16.428 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:16.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.428 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-16T05:46:16.428 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-16T05:46:16.429 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-16T05:46:16.429 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-16T05:46:16.429 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000006529, 2024-04-16T05:46:16.429 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000003264 2024-04-16T05:46:16.429 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:16.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.429 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-16T05:46:16.429 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-04-16T05:46:16.430 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-04-16T05:46:16.430 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-16T05:46:16.430 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.430 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.430 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.430 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-16T05:46:16.431 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-16T05:46:16.431 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-16T05:46:16.431 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-16T05:46:16.431 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-04-16T05:46:16.431 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-16T05:46:16.431 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-16T05:46:16.431 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 10, 2024-04-16T05:46:16.431 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-16T05:46:16.432 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 10, 2024-04-16T05:46:16.432 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-16T05:46:16.432 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 10, 2024-04-16T05:46:16.432 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-04-16T05:46:16.432 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-04-16T05:46:16.432 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-04-16T05:46:16.432 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-04-16T05:46:16.432 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-04-16T05:46:16.432 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-04-16T05:46:16.433 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-16T05:46:16.433 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-04-16T05:46:16.433 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:46:16.433 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-16T05:46:16.433 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-04-16T05:46:16.433 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-16T05:46:16.433 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-16T05:46:16.433 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-16T05:46:16.434 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-16T05:46:16.434 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-16T05:46:16.434 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-16T05:46:16.434 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-16T05:46:16.434 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:46:16.434 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-16T05:46:16.434 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-04-16T05:46:16.434 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-04-16T05:46:16.434 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-04-16T05:46:16.435 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:46:16.435 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-16T05:46:16.435 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:46:16.435 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:46:16.435 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:46:16.435 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:46:16.435 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:46:16.435 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-16T05:46:16.436 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:46:16.436 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:46:16.436 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:46:16.436 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:46:16.436 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:46:16.436 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:46:16.436 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-16T05:46:16.436 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:46:16.437 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-16T05:46:16.437 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-16T05:46:16.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.437 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-16T05:46:16.437 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-16T05:46:16.437 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-04-16T05:46:16.437 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-04-16T05:46:16.437 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 1, 2024-04-16T05:46:16.437 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-16T05:46:16.438 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-16T05:46:16.438 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-16T05:46:16.438 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:46:16.438 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-16T05:46:16.438 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-16T05:46:16.438 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-16T05:46:16.438 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-16T05:46:16.438 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-16T05:46:16.439 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:46:16.439 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:46:16.439 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-16T05:46:16.439 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-16T05:46:16.439 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:46:16.439 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:46:16.439 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:46:16.439 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-16T05:46:16.439 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-16T05:46:16.440 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:46:16.440 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-16T05:46:16.440 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-16T05:46:16.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.440 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-16T05:46:16.440 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-16T05:46:16.440 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 3, 2024-04-16T05:46:16.440 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-16T05:46:16.441 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-16T05:46:16.441 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 3, 2024-04-16T05:46:16.441 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-16T05:46:16.441 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-16T05:46:16.441 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-04-16T05:46:16.441 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-16T05:46:16.441 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-16T05:46:16.441 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-16T05:46:16.441 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-16T05:46:16.442 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-16T05:46:16.442 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-16T05:46:16.442 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-16T05:46:16.442 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195283, 2024-04-16T05:46:16.442 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-04-16T05:46:16.442 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-16T05:46:16.442 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-16T05:46:16.442 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002985479, 2024-04-16T05:46:16.443 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001492739 2024-04-16T05:46:16.443 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.443 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-04-16T05:46:16.443 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.443 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-16T05:46:16.443 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 13, 2024-04-16T05:46:16.443 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 13, 2024-04-16T05:46:16.443 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-04-16T05:46:16.443 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 11, 2024-04-16T05:46:16.444 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 12, 2024-04-16T05:46:16.444 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 1, 2024-04-16T05:46:16.444 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 10, 2024-04-16T05:46:16.444 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 10, 2024-04-16T05:46:16.444 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 0, 2024-04-16T05:46:16.444 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-04-16T05:46:16.444 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-04-16T05:46:16.444 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-04-16T05:46:16.444 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 49588, 2024-04-16T05:46:16.445 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223608 2024-04-16T05:46:16.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.445 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-16T05:46:16.445 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-04-16T05:46:16.445 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-16T05:46:16.445 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-04-16T05:46:16.445 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 9, 2024-04-16T05:46:16.445 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-04-16T05:46:16.446 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-16T05:46:16.446 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.446 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.446 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.446 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-16T05:46:16.446 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.446 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.447 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.447 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.447 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-16T05:46:16.447 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.447 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.447 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.447 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.447 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-16T05:46:16.448 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.448 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.448 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.448 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.448 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-16T05:46:16.448 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.448 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.448 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.448 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.449 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-16T05:46:16.449 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.449 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.449 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.449 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-16T05:46:16.449 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.449 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.450 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.450 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-16T05:46:16.450 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.450 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.450 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.451 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-16T05:46:16.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.451 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.451 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-16T05:46:16.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.451 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.452 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.452 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.452 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-16T05:46:16.452 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.452 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.452 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.452 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.452 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-16T05:46:16.453 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.453 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.454 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.454 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-16T05:46:16.454 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.454 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.454 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.455 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-16T05:46:16.455 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.455 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.455 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.455 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.455 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-16T05:46:16.455 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.455 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.456 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.456 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.456 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-16T05:46:16.456 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.456 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.456 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.456 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.457 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-16T05:46:16.457 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.457 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.457 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.457 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.457 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-16T05:46:16.457 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.458 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.458 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.458 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.458 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-16T05:46:16.458 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.458 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.458 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.459 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.459 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-16T05:46:16.459 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.459 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.459 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.459 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.459 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-16T05:46:16.459 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.460 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.460 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.460 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.460 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-16T05:46:16.460 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.460 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.460 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.460 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.461 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-16T05:46:16.461 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.461 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.461 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.461 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.461 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-16T05:46:16.461 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.461 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.462 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.462 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-16T05:46:16.462 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.462 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.462 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.462 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-16T05:46:16.463 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.463 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.463 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.463 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.463 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-16T05:46:16.463 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.463 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.463 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.464 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-16T05:46:16.464 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.464 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.464 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.464 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-16T05:46:16.464 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.464 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.465 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.465 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:46:16.465 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:46:16.465 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-16T05:46:16.465 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.465 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.465 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.466 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:16.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.466 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-16T05:46:16.466 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-04-16T05:46:16.466 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 1, 2024-04-16T05:46:16.466 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 0, 2024-04-16T05:46:16.466 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-04-16T05:46:16.466 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-16T05:46:16.467 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-04-16T05:46:16.467 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-04-16T05:46:16.467 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 611, 2024-04-16T05:46:16.467 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-16T05:46:16.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.467 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-16T05:46:16.467 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:46:16.468 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-16T05:46:16.468 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:46:16.468 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:46:16.468 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:46:16.468 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:46:16.468 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:46:16.468 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:46:16.468 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:46:16.469 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:46:16.469 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:46:16.469 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:46:16.469 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:46:16.469 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:46:16.469 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:46:16.469 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-16T05:46:16.469 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:46:16.470 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-16T05:46:16.470 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:46:16.470 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:46:16.470 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:46:16.473 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:46:16.473 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:46:16.473 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:46:16.473 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:46:16.474 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-16T05:46:16.474 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:46:16.474 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:46:16.474 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:46:16.474 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-16T05:46:16.474 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-16T05:46:16.474 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-16T05:46:16.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:46:16.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:46:16.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:46:16.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:46:16.475 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 246135, 2024-04-16T05:46:16.475 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 78, 2024-04-16T05:46:16.475 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:46:16.475 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-16T05:46:16.476 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-16T05:46:16.476 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-16T05:46:16.476 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:46:16.476 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-16T05:46:16.476 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:46:16.476 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-16T05:46:16.476 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9752, 2024-04-16T05:46:16.476 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 188, 2024-04-16T05:46:16.477 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:46:16.477 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:46:16.477 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-16T05:46:16.477 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-16T05:46:16.477 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 50872, 2024-04-16T05:46:16.477 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 198, 2024-04-16T05:46:16.477 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-16T05:46:16.477 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-16T05:46:16.478 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-16T05:46:16.478 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-16T05:46:16.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.478 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-16T05:46:16.478 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-16T05:46:16.478 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-16T05:46:16.478 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 37, 2024-04-16T05:46:16.478 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 11191, 2024-04-16T05:46:16.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-04-16T05:46:16.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 37, 2024-04-16T05:46:16.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-16T05:46:16.479 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 37, 2024-04-16T05:46:16.479 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.218995260, 2024-04-16T05:46:16.479 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005918790 2024-04-16T05:46:16.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-16T05:46:16.480 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-16T05:46:16.480 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 37, 2024-04-16T05:46:16.480 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 59 2024-04-16T05:46:16.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.480 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 37, 2024-04-16T05:46:16.481 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-04-16T05:46:16.481 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 37, 2024-04-16T05:46:16.481 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-16T05:46:16.482 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-16T05:46:16.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-04-16T05:46:16.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-04-16T05:46:16.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-04-16T05:46:16.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 2, 2024-04-16T05:46:16.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 6, 2024-04-16T05:46:16.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-16T05:46:16.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-16T05:46:16.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-16T05:46:16.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-16T05:46:16.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-04-16T05:46:16.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-16T05:46:16.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-16T05:46:16.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-16T05:46:16.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-04-16T05:46:16.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-04-16T05:46:16.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:46:16.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-16T05:46:16.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:46:16.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-16T05:46:16.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-16T05:46:16.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-16T05:46:16.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:46:16.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-16T05:46:16.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:46:16.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-16T05:46:16.485 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-16T05:46:16.485 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-16T05:46:16.485 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-16T05:46:16.485 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-16T05:46:16.485 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-16T05:46:16.485 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-16T05:46:16.485 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-16T05:46:16.485 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-16T05:46:16.485 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-16T05:46:16.486 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-16T05:46:16.486 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-16T05:46:16.486 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-16T05:46:16.486 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-16T05:46:16.486 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-16T05:46:16.486 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-16T05:46:16.486 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-16T05:46:16.486 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 105, 2024-04-16T05:46:16.487 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-16T05:46:16.487 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 15, 2024-04-16T05:46:16.487 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 9, 2024-04-16T05:46:16.487 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 9, 2024-04-16T05:46:16.487 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-04-16T05:46:16.487 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-16T05:46:16.487 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 14, 2024-04-16T05:46:16.487 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-04-16T05:46:16.487 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 1 2024-04-16T05:46:16.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.488 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-16T05:46:16.488 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-04-16T05:46:16.488 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:46:16.488 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-16T05:46:16.488 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-16T05:46:16.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.488 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-16T05:46:16.488 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-04-16T05:46:16.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.489 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-16T05:46:16.489 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-16T05:46:16.489 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:46:16.489 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-16T05:46:16.489 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-16T05:46:16.489 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-16T05:46:16.489 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-16T05:46:16.490 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-16T05:46:16.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.490 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:46:16.490 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:16.490 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-16T05:46:16.490 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:16.490 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 709, 2024-04-16T05:46:16.490 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1673362, 2024-04-16T05:46:16.490 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:16.491 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 709, 2024-04-16T05:46:16.491 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:16.491 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:16.491 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 709, 2024-04-16T05:46:16.491 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1673362, 2024-04-16T05:46:16.491 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:16.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.491 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.492 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:16.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.492 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-16T05:46:16.492 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:16.492 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-16T05:46:16.492 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:16.492 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:16.492 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:16.492 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:16.493 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:16.493 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 37, 2024-04-16T05:46:16.493 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 11191, 2024-04-16T05:46:16.493 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 37, 2024-04-16T05:46:16.493 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 11191, 2024-04-16T05:46:16.493 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:16.493 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.493 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.493 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.494 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:16.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.494 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-16T05:46:16.494 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:16.494 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-16T05:46:16.494 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:16.494 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:16.494 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:16.495 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:16.495 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:16.495 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 37, 2024-04-16T05:46:16.495 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 37, 2024-04-16T05:46:16.495 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 37, 2024-04-16T05:46:16.495 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 37, 2024-04-16T05:46:16.495 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:16.495 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.496 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.496 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.496 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:16.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.496 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:46:16.496 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:16.496 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-16T05:46:16.496 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:16.496 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:16.497 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:16.497 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:16.497 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:16.497 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:16.497 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:16.497 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-16T05:46:16.497 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-16T05:46:16.497 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:16.498 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.498 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.498 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.498 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:16.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:16.498 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55731707c500": { 2024-04-16T05:46:16.498 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:16.498 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-16T05:46:16.499 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:16.499 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3, 2024-04-16T05:46:16.499 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 979, 2024-04-16T05:46:16.499 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:16.499 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3, 2024-04-16T05:46:16.499 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:16.499 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:16.499 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2, 2024-04-16T05:46:16.499 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 979, 2024-04-16T05:46:16.500 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:16.500 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:16.500 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:16.500 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:16.500 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:16.500 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:16.500 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-16T05:46:16.501 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:16 smithi086 ceph-mon[32028]: osdmap e105: 12 total, 12 up, 12 in 2024-04-16T05:46:16.501 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:16 smithi086 ceph-mon[32028]: pgmap v342: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-16T05:46:17.282 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-16T05:46:17.282 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-16T05:46:17.282 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-16T05:46:17.282 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:46:17.283 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:46:17.283 DEBUG:tasks.check_counter:Getting stats from i 2024-04-16T05:46:17.283 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-16T05:46:17.518 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:46:18.235 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:18 smithi196 ceph-mon[36373]: pgmap v343: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:46:18.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:18 smithi086 ceph-mon[32028]: pgmap v343: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:46:18.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:18 smithi194 ceph-mon[35865]: pgmap v343: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-16T05:46:19.263 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-16T05:46:19.264 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:46:19.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 52, 2024-04-16T05:46:19.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 192, 2024-04-16T05:46:19.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 14536, 2024-04-16T05:46:19.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 416881, 2024-04-16T05:46:19.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 18, 2024-04-16T05:46:19.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-16T05:46:19.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.019442868, 2024-04-16T05:46:19.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.006514860, 2024-04-16T05:46:19.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.005983330, 2024-04-16T05:46:19.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000987649, 2024-04-16T05:46:19.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:19.265 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 186, 2024-04-16T05:46:19.265 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.025553166, 2024-04-16T05:46:19.265 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000137382 2024-04-16T05:46:19.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:19.266 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 27, 2024-04-16T05:46:19.266 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000003642, 2024-04-16T05:46:19.266 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000134 2024-04-16T05:46:19.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 5952, 2024-04-16T05:46:19.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 330272 2024-04-16T05:46:19.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.267 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:46:19.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 399, 2024-04-16T05:46:19.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 607, 2024-04-16T05:46:19.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 108834, 2024-04-16T05:46:19.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 779463, 2024-04-16T05:46:19.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 43, 2024-04-16T05:46:19.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-04-16T05:46:19.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.070963319, 2024-04-16T05:46:19.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.026249742, 2024-04-16T05:46:19.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.035927917, 2024-04-16T05:46:19.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.004946398, 2024-04-16T05:46:19.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:19.268 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 609, 2024-04-16T05:46:19.268 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.041189485, 2024-04-16T05:46:19.268 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000067634 2024-04-16T05:46:19.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:19.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 907, 2024-04-16T05:46:19.269 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000265420, 2024-04-16T05:46:19.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000292 2024-04-16T05:46:19.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-04-16T05:46:19.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 132272 2024-04-16T05:46:19.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.269 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:46:19.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 312, 2024-04-16T05:46:19.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 308, 2024-04-16T05:46:19.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1660444, 2024-04-16T05:46:19.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 183257, 2024-04-16T05:46:19.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 9, 2024-04-16T05:46:19.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-04-16T05:46:19.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.054820808, 2024-04-16T05:46:19.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.016988177, 2024-04-16T05:46:19.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.041689889, 2024-04-16T05:46:19.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.006217590, 2024-04-16T05:46:19.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:19.271 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 307, 2024-04-16T05:46:19.271 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.023202896, 2024-04-16T05:46:19.271 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000075579 2024-04-16T05:46:19.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:19.271 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 36, 2024-04-16T05:46:19.271 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000008976, 2024-04-16T05:46:19.272 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000249 2024-04-16T05:46:19.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.272 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1657344, 2024-04-16T05:46:19.272 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 149888 2024-04-16T05:46:19.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.272 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-16T05:46:19.272 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-16T05:46:19.272 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-16T05:46:19.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.273 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-16T05:46:19.273 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-16T05:46:19.273 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-16T05:46:19.273 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 63, 2024-04-16T05:46:19.273 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002287425, 2024-04-16T05:46:19.273 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000036308 2024-04-16T05:46:19.273 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:19.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.273 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-16T05:46:19.274 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-16T05:46:19.274 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-16T05:46:19.274 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-16T05:46:19.274 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000009238, 2024-04-16T05:46:19.274 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000004619 2024-04-16T05:46:19.274 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:19.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.274 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-16T05:46:19.274 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 3, 2024-04-16T05:46:19.275 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 3, 2024-04-16T05:46:19.275 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-16T05:46:19.275 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3, 2024-04-16T05:46:19.275 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001699669, 2024-04-16T05:46:19.275 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000566556 2024-04-16T05:46:19.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.275 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-16T05:46:19.275 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-16T05:46:19.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-16T05:46:19.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-16T05:46:19.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-04-16T05:46:19.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-16T05:46:19.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-16T05:46:19.276 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 11, 2024-04-16T05:46:19.276 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-16T05:46:19.276 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 11, 2024-04-16T05:46:19.276 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-16T05:46:19.277 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 10, 2024-04-16T05:46:19.277 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 1, 2024-04-16T05:46:19.277 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-04-16T05:46:19.277 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-04-16T05:46:19.277 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 2, 2024-04-16T05:46:19.277 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 8, 2024-04-16T05:46:19.277 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 4, 2024-04-16T05:46:19.277 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-16T05:46:19.277 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 2, 2024-04-16T05:46:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:46:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-16T05:46:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-04-16T05:46:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-16T05:46:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-16T05:46:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-16T05:46:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-16T05:46:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-16T05:46:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-16T05:46:19.279 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-16T05:46:19.279 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:46:19.279 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-16T05:46:19.279 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-04-16T05:46:19.279 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-04-16T05:46:19.279 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-04-16T05:46:19.279 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:46:19.279 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-16T05:46:19.280 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:46:19.280 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:46:19.280 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:46:19.280 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:46:19.280 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:46:19.280 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-16T05:46:19.280 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:46:19.280 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:46:19.280 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-16T05:46:19.281 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-16T05:46:19.282 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-04-16T05:46:19.282 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-04-16T05:46:19.282 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 3, 2024-04-16T05:46:19.282 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-16T05:46:19.282 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-16T05:46:19.282 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-16T05:46:19.282 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:46:19.282 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-16T05:46:19.282 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-16T05:46:19.283 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-16T05:46:19.283 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-16T05:46:19.283 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-16T05:46:19.283 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:46:19.283 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:46:19.283 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-16T05:46:19.283 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-16T05:46:19.283 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-16T05:46:19.284 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-16T05:46:19.285 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 11, 2024-04-16T05:46:19.285 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-16T05:46:19.285 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-16T05:46:19.285 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 11, 2024-04-16T05:46:19.285 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-16T05:46:19.285 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-16T05:46:19.285 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-04-16T05:46:19.285 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-16T05:46:19.285 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-16T05:46:19.286 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-16T05:46:19.286 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-16T05:46:19.286 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-16T05:46:19.286 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-16T05:46:19.286 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-16T05:46:19.286 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4197411, 2024-04-16T05:46:19.286 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-04-16T05:46:19.286 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-16T05:46:19.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 10, 2024-04-16T05:46:19.287 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.016024853, 2024-04-16T05:46:19.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001602485 2024-04-16T05:46:19.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.287 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-04-16T05:46:19.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.287 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-16T05:46:19.287 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 14, 2024-04-16T05:46:19.287 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 14, 2024-04-16T05:46:19.287 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-04-16T05:46:19.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 13, 2024-04-16T05:46:19.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 14, 2024-04-16T05:46:19.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 1, 2024-04-16T05:46:19.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 11, 2024-04-16T05:46:19.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 12, 2024-04-16T05:46:19.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 1, 2024-04-16T05:46:19.288 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-04-16T05:46:19.288 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 2, 2024-04-16T05:46:19.288 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 2, 2024-04-16T05:46:19.289 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 52148, 2024-04-16T05:46:19.289 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223608 2024-04-16T05:46:19.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.289 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-16T05:46:19.289 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 7, 2024-04-16T05:46:19.289 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-16T05:46:19.289 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 3, 2024-04-16T05:46:19.289 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 25, 2024-04-16T05:46:19.289 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 1, 2024-04-16T05:46:19.290 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-16T05:46:19.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.290 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.290 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-16T05:46:19.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-04-16T05:46:19.290 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000866353, 2024-04-16T05:46:19.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000866353 2024-04-16T05:46:19.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.291 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-16T05:46:19.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.291 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.291 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-16T05:46:19.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.291 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000833316, 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000416658 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.292 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-16T05:46:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-16T05:46:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.294 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.294 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.294 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-16T05:46:19.294 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.294 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.295 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.295 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-16T05:46:19.295 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.295 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.295 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.296 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-16T05:46:19.296 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.296 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.296 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.296 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-16T05:46:19.296 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.297 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.297 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.297 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-16T05:46:19.297 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.298 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.298 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.298 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-16T05:46:19.298 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.298 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.298 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.299 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-16T05:46:19.299 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.299 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.299 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.299 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-16T05:46:19.299 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.299 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.300 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.300 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-16T05:46:19.300 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.300 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.300 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.300 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-16T05:46:19.300 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.301 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.301 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-16T05:46:19.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.301 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.302 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-16T05:46:19.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.302 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.302 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-16T05:46:19.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.302 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.303 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-16T05:46:19.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.303 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.303 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-16T05:46:19.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.304 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.304 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-16T05:46:19.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.304 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.305 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-16T05:46:19.305 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.305 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.305 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.305 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-16T05:46:19.305 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.305 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.306 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.306 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-16T05:46:19.306 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.306 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.306 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.306 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-16T05:46:19.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.307 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.307 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-16T05:46:19.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.307 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.308 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:46:19.308 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:46:19.308 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-16T05:46:19.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.308 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.308 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:19.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.309 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-16T05:46:19.309 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-04-16T05:46:19.309 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 5, 2024-04-16T05:46:19.309 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 4, 2024-04-16T05:46:19.309 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-04-16T05:46:19.309 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-16T05:46:19.309 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 2, 2024-04-16T05:46:19.309 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 1, 2024-04-16T05:46:19.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 614, 2024-04-16T05:46:19.310 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-16T05:46:19.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.310 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-16T05:46:19.310 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:46:19.310 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-16T05:46:19.310 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:46:19.310 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:46:19.310 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:46:19.310 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:46:19.311 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:46:19.311 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:46:19.311 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:46:19.311 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:46:19.311 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:46:19.311 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:46:19.311 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:46:19.311 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:46:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:46:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-16T05:46:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:46:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-16T05:46:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:46:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:46:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:46:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:46:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:46:19.313 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:46:19.313 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:46:19.313 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-16T05:46:19.313 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:46:19.313 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:46:19.313 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:46:19.313 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-16T05:46:19.313 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-16T05:46:19.313 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-16T05:46:19.314 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:46:19.314 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:46:19.314 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:46:19.314 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:46:19.314 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 254759, 2024-04-16T05:46:19.314 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 83, 2024-04-16T05:46:19.314 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:46:19.314 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-16T05:46:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-16T05:46:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-16T05:46:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:46:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-16T05:46:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:46:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-16T05:46:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9512, 2024-04-16T05:46:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 182, 2024-04-16T05:46:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:46:19.316 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:46:19.316 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-16T05:46:19.316 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-16T05:46:19.316 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 56568, 2024-04-16T05:46:19.316 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 227, 2024-04-16T05:46:19.316 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-16T05:46:19.316 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-16T05:46:19.316 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-16T05:46:19.317 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-16T05:46:19.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.317 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-16T05:46:19.317 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-16T05:46:19.317 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-16T05:46:19.317 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 50, 2024-04-16T05:46:19.317 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 13734, 2024-04-16T05:46:19.317 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-04-16T05:46:19.317 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 50, 2024-04-16T05:46:19.318 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-16T05:46:19.318 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 50, 2024-04-16T05:46:19.318 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.227995059, 2024-04-16T05:46:19.318 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004559901 2024-04-16T05:46:19.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.318 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-16T05:46:19.318 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-16T05:46:19.318 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 50, 2024-04-16T05:46:19.318 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 74 2024-04-16T05:46:19.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.319 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 50, 2024-04-16T05:46:19.319 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-04-16T05:46:19.319 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 50, 2024-04-16T05:46:19.319 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-16T05:46:19.319 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-16T05:46:19.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-04-16T05:46:19.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-04-16T05:46:19.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-04-16T05:46:19.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 10, 2024-04-16T05:46:19.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 9, 2024-04-16T05:46:19.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-16T05:46:19.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-16T05:46:19.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-16T05:46:19.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-16T05:46:19.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-04-16T05:46:19.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-16T05:46:19.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-16T05:46:19.321 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-16T05:46:19.321 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-04-16T05:46:19.321 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-04-16T05:46:19.321 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:46:19.321 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-16T05:46:19.321 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:46:19.321 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-16T05:46:19.321 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-16T05:46:19.321 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-16T05:46:19.322 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:46:19.322 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-16T05:46:19.322 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:46:19.322 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-16T05:46:19.322 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-16T05:46:19.322 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-16T05:46:19.322 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-16T05:46:19.322 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-16T05:46:19.322 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-16T05:46:19.323 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-16T05:46:19.323 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-16T05:46:19.323 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-16T05:46:19.323 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-16T05:46:19.323 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-16T05:46:19.323 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-16T05:46:19.323 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-16T05:46:19.323 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-16T05:46:19.323 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-16T05:46:19.324 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-16T05:46:19.324 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-16T05:46:19.324 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 105, 2024-04-16T05:46:19.324 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-16T05:46:19.324 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 15, 2024-04-16T05:46:19.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 10, 2024-04-16T05:46:19.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 10, 2024-04-16T05:46:19.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-04-16T05:46:19.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-16T05:46:19.325 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 17, 2024-04-16T05:46:19.325 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-04-16T05:46:19.325 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 2 2024-04-16T05:46:19.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.325 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-16T05:46:19.325 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-04-16T05:46:19.325 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:46:19.325 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-16T05:46:19.325 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-16T05:46:19.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.326 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-16T05:46:19.326 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-04-16T05:46:19.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.326 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-16T05:46:19.326 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-16T05:46:19.326 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:46:19.327 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-16T05:46:19.327 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-16T05:46:19.327 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-16T05:46:19.327 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-16T05:46:19.327 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-16T05:46:19.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.327 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:46:19.327 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:19.327 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-16T05:46:19.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:19.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 759, 2024-04-16T05:46:19.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1696564, 2024-04-16T05:46:19.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:19.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 759, 2024-04-16T05:46:19.328 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:19.328 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:19.328 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 759, 2024-04-16T05:46:19.328 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1696564, 2024-04-16T05:46:19.329 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:19.329 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.329 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.329 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:19.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.329 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-16T05:46:19.329 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:19.329 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-16T05:46:19.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:19.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:19.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:19.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:19.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:19.330 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 50, 2024-04-16T05:46:19.330 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 13734, 2024-04-16T05:46:19.330 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 50, 2024-04-16T05:46:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 13734, 2024-04-16T05:46:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-16T05:46:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:19.332 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-16T05:46:19.332 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:19.332 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:19.332 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:19.332 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:19.332 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:19.332 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 50, 2024-04-16T05:46:19.332 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 50, 2024-04-16T05:46:19.332 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 50, 2024-04-16T05:46:19.333 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 50, 2024-04-16T05:46:19.333 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:19.333 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.333 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.333 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.333 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:19.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.333 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:46:19.333 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:19.334 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-16T05:46:19.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:19.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:19.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:19.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:19.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:19.334 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:19.334 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:19.335 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-16T05:46:19.335 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-16T05:46:19.335 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:19.335 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.335 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.335 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.335 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:19.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:19.336 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x5560a9700500": { 2024-04-16T05:46:19.336 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:19.336 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-16T05:46:19.336 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:19.336 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 11, 2024-04-16T05:46:19.336 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 3107, 2024-04-16T05:46:19.336 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:19.336 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 11, 2024-04-16T05:46:19.337 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:19.337 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:19.337 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 10, 2024-04-16T05:46:19.337 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3107, 2024-04-16T05:46:19.337 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:19.337 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:19.337 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:19.337 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:19.337 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:19.338 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:19.338 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-16T05:46:20.037 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-16T05:46:20.037 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-16T05:46:20.037 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-16T05:46:20.037 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:46:20.037 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:46:20.037 DEBUG:tasks.check_counter:Getting stats from l 2024-04-16T05:46:20.038 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-16T05:46:20.276 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:46:20.397 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:20 smithi196 ceph-mon[36373]: pgmap v344: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-16T05:46:20.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:20 smithi086 ceph-mon[32028]: pgmap v344: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-16T05:46:20.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:20 smithi194 ceph-mon[35865]: pgmap v344: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-16T05:46:21.962 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-16T05:46:21.962 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:46:21.962 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2294, 2024-04-16T05:46:21.963 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 2296, 2024-04-16T05:46:21.963 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 651052, 2024-04-16T05:46:21.963 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 729499, 2024-04-16T05:46:21.963 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-04-16T05:46:21.963 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-04-16T05:46:21.963 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.203826566, 2024-04-16T05:46:21.963 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.067668875, 2024-04-16T05:46:21.964 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.048021441, 2024-04-16T05:46:21.964 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.071675314, 2024-04-16T05:46:21.964 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:21.964 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2294, 2024-04-16T05:46:21.964 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.098790755, 2024-04-16T05:46:21.964 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000043064 2024-04-16T05:46:21.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.965 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:21.965 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.965 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.965 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.965 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:46:21.965 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:46:21.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.966 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:46:21.966 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 261, 2024-04-16T05:46:21.966 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 249, 2024-04-16T05:46:21.966 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1629708, 2024-04-16T05:46:21.966 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 143754, 2024-04-16T05:46:21.966 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 3, 2024-04-16T05:46:21.966 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-04-16T05:46:21.966 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.050633588, 2024-04-16T05:46:21.967 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.015435844, 2024-04-16T05:46:21.967 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.039350450, 2024-04-16T05:46:21.967 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.006256276, 2024-04-16T05:46:21.967 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:21.967 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 248, 2024-04-16T05:46:21.967 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.013976554, 2024-04-16T05:46:21.967 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000056357 2024-04-16T05:46:21.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.968 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:21.968 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.968 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.968 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.968 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1629152, 2024-04-16T05:46:21.968 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 143056 2024-04-16T05:46:21.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:46:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 7, 2024-04-16T05:46:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 176, 2024-04-16T05:46:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 3612, 2024-04-16T05:46:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 456558, 2024-04-16T05:46:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-04-16T05:46:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-16T05:46:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.011524653, 2024-04-16T05:46:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.007213631, 2024-04-16T05:46:21.970 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.002192768, 2024-04-16T05:46:21.970 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000212207, 2024-04-16T05:46:21.970 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:46:21.970 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 174, 2024-04-16T05:46:21.970 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.017265358, 2024-04-16T05:46:21.970 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000099226 2024-04-16T05:46:21.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.970 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:46:21.971 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.971 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.971 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.971 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-16T05:46:21.971 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 456240 2024-04-16T05:46:21.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.971 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-16T05:46:21.972 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-16T05:46:21.972 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-16T05:46:21.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.972 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-16T05:46:21.972 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-16T05:46:21.972 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-16T05:46:21.972 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3867, 2024-04-16T05:46:21.972 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.111374429, 2024-04-16T05:46:21.972 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000028801 2024-04-16T05:46:21.973 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:21.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.973 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-16T05:46:21.973 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-16T05:46:21.973 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-16T05:46:21.973 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.973 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.973 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.973 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:21.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.974 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-16T05:46:21.974 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-04-16T05:46:21.974 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-04-16T05:46:21.974 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-16T05:46:21.974 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.974 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.974 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.975 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-16T05:46:21.975 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-16T05:46:21.975 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-16T05:46:21.975 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-16T05:46:21.975 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-04-16T05:46:21.975 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-16T05:46:21.975 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-16T05:46:21.975 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 0, 2024-04-16T05:46:21.976 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-16T05:46:21.976 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 0, 2024-04-16T05:46:21.976 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-16T05:46:21.976 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 0, 2024-04-16T05:46:21.976 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-04-16T05:46:21.976 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-04-16T05:46:21.976 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-04-16T05:46:21.976 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-04-16T05:46:21.977 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-04-16T05:46:21.977 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-04-16T05:46:21.977 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-16T05:46:21.977 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-04-16T05:46:21.977 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:46:21.977 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-16T05:46:21.977 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-04-16T05:46:21.977 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-16T05:46:21.978 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-16T05:46:21.978 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-16T05:46:21.978 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-16T05:46:21.978 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-16T05:46:21.978 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-16T05:46:21.978 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-16T05:46:21.978 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:46:21.978 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-16T05:46:21.979 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-04-16T05:46:21.979 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-04-16T05:46:21.979 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-04-16T05:46:21.979 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:46:21.979 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-16T05:46:21.979 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:46:21.979 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:46:21.979 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:46:21.979 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:46:21.980 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:46:21.980 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-16T05:46:21.980 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:46:21.980 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:46:21.980 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:46:21.980 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:46:21.980 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:46:21.980 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:46:21.981 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-16T05:46:21.981 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:46:21.981 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-16T05:46:21.981 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-16T05:46:21.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.981 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-16T05:46:21.981 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-16T05:46:21.981 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-04-16T05:46:21.981 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-04-16T05:46:21.982 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-04-16T05:46:21.982 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-16T05:46:21.982 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-16T05:46:21.982 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-16T05:46:21.982 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:46:21.982 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-16T05:46:21.982 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-16T05:46:21.982 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-16T05:46:21.983 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-16T05:46:21.983 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-16T05:46:21.983 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:46:21.983 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:46:21.983 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-16T05:46:21.983 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-16T05:46:21.983 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:46:21.983 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:46:21.983 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:46:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-16T05:46:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-16T05:46:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:46:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-16T05:46:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-16T05:46:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-16T05:46:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-16T05:46:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-04-16T05:46:21.985 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-16T05:46:21.985 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-16T05:46:21.985 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 11, 2024-04-16T05:46:21.985 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-16T05:46:21.985 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-16T05:46:21.985 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-04-16T05:46:21.985 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-16T05:46:21.985 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-16T05:46:21.986 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-16T05:46:21.986 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-16T05:46:21.986 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-16T05:46:21.986 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-16T05:46:21.986 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-16T05:46:21.986 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4197411, 2024-04-16T05:46:21.986 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 4197270, 2024-04-16T05:46:21.986 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-16T05:46:21.987 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.987 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.987 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.987 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 11 2024-04-16T05:46:21.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.987 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-16T05:46:21.987 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 3, 2024-04-16T05:46:21.987 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 3, 2024-04-16T05:46:21.988 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-04-16T05:46:21.988 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 1, 2024-04-16T05:46:21.988 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 1, 2024-04-16T05:46:21.988 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 0, 2024-04-16T05:46:21.988 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 0, 2024-04-16T05:46:21.988 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 0, 2024-04-16T05:46:21.988 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 0, 2024-04-16T05:46:21.988 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-04-16T05:46:21.989 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-04-16T05:46:21.989 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-04-16T05:46:21.989 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50748, 2024-04-16T05:46:21.989 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215416 2024-04-16T05:46:21.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.989 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-16T05:46:21.989 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-04-16T05:46:21.989 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-16T05:46:21.990 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-04-16T05:46:21.990 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-04-16T05:46:21.990 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-04-16T05:46:21.990 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-16T05:46:21.990 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.990 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.990 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.991 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-16T05:46:21.991 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.991 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.991 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.991 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-16T05:46:21.991 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.991 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.991 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.992 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-16T05:46:21.992 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.992 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.992 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.992 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-16T05:46:21.992 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.992 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.993 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.993 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-16T05:46:21.993 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.993 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.993 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.993 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-16T05:46:21.994 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.994 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.994 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.994 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-16T05:46:21.994 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.994 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.994 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.995 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-16T05:46:21.995 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.995 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.995 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.995 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-16T05:46:21.995 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.995 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.996 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.996 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-16T05:46:21.996 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.996 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.996 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.996 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-16T05:46:21.997 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.997 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.997 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.998 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-16T05:46:21.998 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.998 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.998 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.998 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-16T05:46:21.998 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.998 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.999 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.999 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-16T05:46:21.999 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:21.999 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:21.999 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:21.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:21.999 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-16T05:46:21.999 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.000 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.000 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.000 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-16T05:46:22.000 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.000 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.000 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.001 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-16T05:46:22.001 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.001 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.001 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.001 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-16T05:46:22.001 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.001 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.001 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.002 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-16T05:46:22.002 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.002 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.002 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.002 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-16T05:46:22.002 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.003 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.003 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.003 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-16T05:46:22.003 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.003 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.003 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.003 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-16T05:46:22.004 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.004 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.004 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.004 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-16T05:46:22.004 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.004 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.004 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.005 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-16T05:46:22.005 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.005 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.005 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.005 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-16T05:46:22.005 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.005 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.005 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.006 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-16T05:46:22.006 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.006 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.006 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.006 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-16T05:46:22.006 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.007 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.007 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.007 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-16T05:46:22.007 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.007 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.007 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.007 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:46:22.008 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:46:22.008 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-16T05:46:22.008 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.008 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.008 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.008 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:22.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.008 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-16T05:46:22.008 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-04-16T05:46:22.009 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 5, 2024-04-16T05:46:22.009 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 4, 2024-04-16T05:46:22.009 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 4, 2024-04-16T05:46:22.009 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-16T05:46:22.009 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-04-16T05:46:22.009 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-04-16T05:46:22.009 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 6693, 2024-04-16T05:46:22.009 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-16T05:46:22.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.010 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-16T05:46:22.010 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:46:22.010 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-16T05:46:22.010 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:46:22.010 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:46:22.010 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:46:22.010 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:46:22.010 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:46:22.011 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:46:22.011 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:46:22.011 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:46:22.011 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:46:22.011 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:46:22.011 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:46:22.011 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:46:22.011 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:46:22.012 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-16T05:46:22.012 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:46:22.012 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-16T05:46:22.012 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:46:22.012 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:46:22.012 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:46:22.012 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:46:22.012 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:46:22.012 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:46:22.013 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:46:22.013 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-16T05:46:22.013 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:46:22.013 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:46:22.013 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:46:22.013 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-16T05:46:22.013 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-16T05:46:22.013 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-16T05:46:22.013 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:46:22.014 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:46:22.014 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:46:22.014 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:46:22.014 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 182138, 2024-04-16T05:46:22.014 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 50, 2024-04-16T05:46:22.014 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:46:22.014 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-16T05:46:22.014 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-16T05:46:22.014 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-16T05:46:22.015 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:46:22.015 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-16T05:46:22.015 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:46:22.015 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-16T05:46:22.015 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9512, 2024-04-16T05:46:22.015 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 182, 2024-04-16T05:46:22.015 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:46:22.015 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:46:22.016 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-16T05:46:22.016 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-16T05:46:22.016 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 8512, 2024-04-16T05:46:22.016 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 13, 2024-04-16T05:46:22.016 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-16T05:46:22.016 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-16T05:46:22.016 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-16T05:46:22.016 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-16T05:46:22.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.017 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-16T05:46:22.017 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-16T05:46:22.017 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-16T05:46:22.017 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 2297, 2024-04-16T05:46:22.017 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-04-16T05:46:22.017 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-04-16T05:46:22.017 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 2297, 2024-04-16T05:46:22.017 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-16T05:46:22.017 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2297, 2024-04-16T05:46:22.018 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.956979228, 2024-04-16T05:46:22.018 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000416621 2024-04-16T05:46:22.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.018 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-16T05:46:22.018 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-16T05:46:22.018 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2297, 2024-04-16T05:46:22.018 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2298 2024-04-16T05:46:22.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.019 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 2297, 2024-04-16T05:46:22.019 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 2297, 2024-04-16T05:46:22.019 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-04-16T05:46:22.019 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-16T05:46:22.019 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-16T05:46:22.019 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 1503, 2024-04-16T05:46:22.019 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-04-16T05:46:22.019 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 792, 2024-04-16T05:46:22.020 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-04-16T05:46:22.020 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-04-16T05:46:22.020 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-16T05:46:22.020 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-16T05:46:22.020 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-16T05:46:22.020 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-16T05:46:22.020 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-04-16T05:46:22.020 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-16T05:46:22.020 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-16T05:46:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-16T05:46:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-04-16T05:46:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-04-16T05:46:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:46:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-16T05:46:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:46:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-16T05:46:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-16T05:46:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-16T05:46:22.022 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:46:22.022 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-16T05:46:22.022 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:46:22.022 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-16T05:46:22.022 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-16T05:46:22.022 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-16T05:46:22.022 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-16T05:46:22.022 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-16T05:46:22.022 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-16T05:46:22.023 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-16T05:46:22.023 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-16T05:46:22.023 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-16T05:46:22.023 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-16T05:46:22.023 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-16T05:46:22.023 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-16T05:46:22.023 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-16T05:46:22.023 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-16T05:46:22.023 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-16T05:46:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-16T05:46:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-16T05:46:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 105, 2024-04-16T05:46:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-16T05:46:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 15, 2024-04-16T05:46:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 4, 2024-04-16T05:46:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 4, 2024-04-16T05:46:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-04-16T05:46:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-16T05:46:22.025 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-04-16T05:46:22.025 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-04-16T05:46:22.025 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-04-16T05:46:22.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.025 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-16T05:46:22.025 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-04-16T05:46:22.025 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:46:22.025 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-16T05:46:22.025 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-16T05:46:22.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.026 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-16T05:46:22.026 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-04-16T05:46:22.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.026 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-16T05:46:22.026 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-16T05:46:22.026 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:46:22.026 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-16T05:46:22.027 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-16T05:46:22.027 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-16T05:46:22.027 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-16T05:46:22.027 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-16T05:46:22.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.027 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:46:22.027 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:22.027 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-16T05:46:22.027 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:22.028 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 2558, 2024-04-16T05:46:22.028 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2037839, 2024-04-16T05:46:22.028 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:22.028 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 2558, 2024-04-16T05:46:22.028 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:22.028 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:22.028 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2558, 2024-04-16T05:46:22.028 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2037839, 2024-04-16T05:46:22.028 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:22.029 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.029 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.029 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.029 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:22.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.029 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-16T05:46:22.029 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:22.029 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-16T05:46:22.030 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:22.030 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:22.030 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:22.030 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:22.030 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:22.030 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2297, 2024-04-16T05:46:22.030 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3112, 2024-04-16T05:46:22.030 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2297, 2024-04-16T05:46:22.030 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3112, 2024-04-16T05:46:22.031 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:22.031 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.031 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.031 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.031 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:22.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.031 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-16T05:46:22.031 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:22.031 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-16T05:46:22.032 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:22.032 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:22.032 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:22.032 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:22.032 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:22.032 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2297, 2024-04-16T05:46:22.032 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2297, 2024-04-16T05:46:22.032 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2297, 2024-04-16T05:46:22.032 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2297, 2024-04-16T05:46:22.033 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:22.033 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.033 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.033 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.033 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:22.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.033 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:46:22.033 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:22.033 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-16T05:46:22.034 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:22.034 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:22.034 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:22.034 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:22.034 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:22.034 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:22.034 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:22.034 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-16T05:46:22.034 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-16T05:46:22.035 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:22.035 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.035 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.035 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.035 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:22.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-16T05:46:22.035 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55ce5657c6e0": { 2024-04-16T05:46:22.035 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-16T05:46:22.035 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-16T05:46:22.036 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-16T05:46:22.036 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-16T05:46:22.036 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-16T05:46:22.036 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-16T05:46:22.036 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-16T05:46:22.036 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-16T05:46:22.036 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-16T05:46:22.036 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-16T05:46:22.037 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-16T05:46:22.037 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-16T05:46:22.037 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-16T05:46:22.037 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-16T05:46:22.037 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-16T05:46:22.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:22.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-16T05:46:22.037 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-16T05:46:22.234 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:22 smithi196 ceph-mon[36373]: pgmap v345: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:46:22.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:22 smithi086 ceph-mon[32028]: pgmap v345: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:46:22.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:22 smithi194 ceph-mon[35865]: pgmap v345: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-16T05:46:22.782 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-16T05:46:22.782 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-16T05:46:22.782 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-16T05:46:22.782 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:46:22.782 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:46:22.783 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} 2024-04-16T05:46:22.792 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-16T05:46:22.803 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-16T05:46:22.804 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:46:22.804 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:46:22.832 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:46:22.832 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:46:22.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:46:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.052+0000 7ff1e8982640 1 -- 172.21.15.86:0/414425444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e410e990 msgr2=0x7ff1e4110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.052+0000 7ff1e8982640 1 --2- 172.21.15.86:0/414425444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e410e990 0x7ff1e4110d80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009ca0 tx=0x7ff1dc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.053+0000 7ff1e8982640 1 -- 172.21.15.86:0/414425444 shutdown_connections 2024-04-16T05:46:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.053+0000 7ff1e8982640 1 --2- 172.21.15.86:0/414425444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e410e990 0x7ff1e4110d80 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.053+0000 7ff1e8982640 1 --2- 172.21.15.86:0/414425444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e4101700 0x7ff1e410e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.053+0000 7ff1e8982640 1 --2- 172.21.15.86:0/414425444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1e4100de0 0x7ff1e41011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.053+0000 7ff1e8982640 1 -- 172.21.15.86:0/414425444 >> 172.21.15.86:0/414425444 conn(0x7ff1e4077280 msgr2=0x7ff1e4077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.053+0000 7ff1e8982640 1 -- 172.21.15.86:0/414425444 shutdown_connections 2024-04-16T05:46:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.053+0000 7ff1e8982640 1 -- 172.21.15.86:0/414425444 wait complete. 2024-04-16T05:46:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.053+0000 7ff1e8982640 1 Processor -- start 2024-04-16T05:46:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.053+0000 7ff1e8982640 1 -- start start 2024-04-16T05:46:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e8982640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4100de0 0x7ff1e41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e8982640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e4101700 0x7ff1e41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e8982640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1e410e990 0x7ff1e419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e8982640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41145c0 con 0x7ff1e4100de0 2024-04-16T05:46:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e8982640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e4114440 con 0x7ff1e410e990 2024-04-16T05:46:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e8982640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e4114740 con 0x7ff1e4101700 2024-04-16T05:46:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4100de0 0x7ff1e41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4100de0 0x7ff1e41a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42638/0 (socket says 172.21.15.86:42638) 2024-04-16T05:46:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e259c640 1 -- 172.21.15.86:0/1252812932 learned_addr learned my addr 172.21.15.86:0/1252812932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1dbfff640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e4101700 0x7ff1e41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e2d9d640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1e410e990 0x7ff1e419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e259c640 1 -- 172.21.15.86:0/1252812932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e4101700 msgr2=0x7ff1e41a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e259c640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e4101700 0x7ff1e41a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e259c640 1 -- 172.21.15.86:0/1252812932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1e410e990 msgr2=0x7ff1e419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e259c640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1e410e990 0x7ff1e419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e259c640 1 -- 172.21.15.86:0/1252812932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e419b0e0 con 0x7ff1e4100de0 2024-04-16T05:46:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e259c640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4100de0 0x7ff1e41a07b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400d950 tx=0x7ff1d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1db7fe640 1 -- 172.21.15.86:0/1252812932 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4017070 con 0x7ff1e4100de0 2024-04-16T05:46:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1db7fe640 1 -- 172.21.15.86:0/1252812932 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d4009c30 con 0x7ff1e4100de0 2024-04-16T05:46:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e419b3d0 con 0x7ff1e4100de0 2024-04-16T05:46:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1db7fe640 1 -- 172.21.15.86:0/1252812932 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400f6b0 con 0x7ff1e4100de0 2024-04-16T05:46:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.054+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e41a7670 con 0x7ff1e4100de0 2024-04-16T05:46:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.055+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e406ae80 con 0x7ff1e4100de0 2024-04-16T05:46:23.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.056+0000 7ff1db7fe640 1 -- 172.21.15.86:0/1252812932 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1d40109d0 con 0x7ff1e4100de0 2024-04-16T05:46:23.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.057+0000 7ff1db7fe640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff1d00a3d20 0x7ff1d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.057+0000 7ff1dbfff640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff1d00a3d20 0x7ff1d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.057+0000 7ff1db7fe640 1 -- 172.21.15.86:0/1252812932 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7ff1d40d0750 con 0x7ff1e4100de0 2024-04-16T05:46:23.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.058+0000 7ff1dbfff640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff1d00a3d20 0x7ff1d00a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff1e410f710 tx=0x7ff1cc005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.062+0000 7ff1db7fe640 1 -- 172.21.15.86:0/1252812932 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1d4009de0 con 0x7ff1e4100de0 2024-04-16T05:46:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.241+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1e4109b20 con 0x7ff1e4100de0 2024-04-16T05:46:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.243+0000 7ff1db7fe640 1 -- 172.21.15.86:0/1252812932 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+25263 (secure 0 0 0) 0x7ff1d409c3d0 con 0x7ff1e4100de0 2024-04-16T05:46:23.241 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:46:23.242 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":105,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:46:15.013279+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2330974702":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/3900523320":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2641972995":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/501397249":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:46:23.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff1d00a3d20 msgr2=0x7ff1d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff1d00a3d20 0x7ff1d00a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff1e410f710 tx=0x7ff1cc005f90 comp rx=0 tx=0).stop 2024-04-16T05:46:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4100de0 msgr2=0x7ff1e41a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4100de0 0x7ff1e41a07b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400d950 tx=0x7ff1d400de10 comp rx=0 tx=0).stop 2024-04-16T05:46:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 shutdown_connections 2024-04-16T05:46:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7ff1d00a3d20 0x7ff1d00a61e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1e410e990 0x7ff1e419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e4101700 0x7ff1e41a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 --2- 172.21.15.86:0/1252812932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4100de0 0x7ff1e41a07b0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 >> 172.21.15.86:0/1252812932 conn(0x7ff1e4077280 msgr2=0x7ff1e410ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 shutdown_connections 2024-04-16T05:46:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.245+0000 7ff1e8982640 1 -- 172.21.15.86:0/1252812932 wait complete. 2024-04-16T05:46:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:28:15.487997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:52.794474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:53.806139+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:57.504528+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:46:23.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:46:23.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 -- 172.21.15.86:0/3606497546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 msgr2=0x7f3f9010dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3606497546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 0x7f3f9010dde0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009900 tx=0x7f3f8802bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:23.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 -- 172.21.15.86:0/3606497546 shutdown_connections 2024-04-16T05:46:23.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3606497546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 0x7f3f9010dde0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3606497546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f90101af0 0x7f3f90105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3606497546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f90101140 0x7f3f90101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 -- 172.21.15.86:0/3606497546 >> 172.21.15.86:0/3606497546 conn(0x7f3f90078000 msgr2=0x7f3f90078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:23.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 -- 172.21.15.86:0/3606497546 shutdown_connections 2024-04-16T05:46:23.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 -- 172.21.15.86:0/3606497546 wait complete. 2024-04-16T05:46:23.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 Processor -- start 2024-04-16T05:46:23.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 -- start start 2024-04-16T05:46:23.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f90101140 0x7f3f9019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.364+0000 7f3f97f40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 0x7f3f9019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f97f40640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f901986d0 0x7f3f90198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f97f40640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f901112d0 con 0x7f3f90106210 2024-04-16T05:46:23.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f97f40640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90111150 con 0x7f3f901986d0 2024-04-16T05:46:23.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f97f40640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90111450 con 0x7f3f90101140 2024-04-16T05:46:23.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f954db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 0x7f3f9019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f954db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 0x7f3f9019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42652/0 (socket says 172.21.15.86:42652) 2024-04-16T05:46:23.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f954db640 1 -- 172.21.15.86:0/3689132048 learned_addr learned my addr 172.21.15.86:0/3689132048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:23.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f95cdc640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f90101140 0x7f3f9019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f964dd640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f901986d0 0x7f3f90198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f954db640 1 -- 172.21.15.86:0/3689132048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f90101140 msgr2=0x7f3f9019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f954db640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f90101140 0x7f3f9019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f954db640 1 -- 172.21.15.86:0/3689132048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f901986d0 msgr2=0x7f3f90198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f954db640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f901986d0 0x7f3f90198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f954db640 1 -- 172.21.15.86:0/3689132048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f901992d0 con 0x7f3f90106210 2024-04-16T05:46:23.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f954db640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 0x7f3f9019eb50 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400d740 tx=0x7f3f8400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:23.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f8effd640 1 -- 172.21.15.86:0/3689132048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84017070 con 0x7f3f90106210 2024-04-16T05:46:23.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f8effd640 1 -- 172.21.15.86:0/3689132048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f8400de50 con 0x7f3f90106210 2024-04-16T05:46:23.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f901a53b0 con 0x7f3f90106210 2024-04-16T05:46:23.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f8effd640 1 -- 172.21.15.86:0/3689132048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84004e00 con 0x7f3f90106210 2024-04-16T05:46:23.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.365+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f901a5760 con 0x7f3f90106210 2024-04-16T05:46:23.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.366+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f90102610 con 0x7f3f90106210 2024-04-16T05:46:23.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.368+0000 7f3f8effd640 1 -- 172.21.15.86:0/3689132048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3f84004fa0 con 0x7f3f90106210 2024-04-16T05:46:23.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.368+0000 7f3f8effd640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3f780a3d20 0x7f3f780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.368+0000 7f3f95cdc640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3f780a3d20 0x7f3f780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.368+0000 7f3f8effd640 1 -- 172.21.15.86:0/3689132048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f3f840d04f0 con 0x7f3f90106210 2024-04-16T05:46:23.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.369+0000 7f3f95cdc640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3f780a3d20 0x7f3f780a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3f80005fd0 tx=0x7f3f80009680 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:23.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.373+0000 7f3f8effd640 1 -- 172.21.15.86:0/3689132048 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f8409c7c0 con 0x7f3f90106210 2024-04-16T05:46:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.553+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f90068e20 con 0x7f3f90106210 2024-04-16T05:46:23.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.553+0000 7f3f8effd640 1 -- 172.21.15.86:0/3689132048 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f3f8409c0c0 con 0x7f3f90106210 2024-04-16T05:46:23.552 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-16T05:46:23.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3f780a3d20 msgr2=0x7f3f780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3f780a3d20 0x7f3f780a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3f80005fd0 tx=0x7f3f80009680 comp rx=0 tx=0).stop 2024-04-16T05:46:23.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 msgr2=0x7f3f9019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 0x7f3f9019eb50 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400d740 tx=0x7f3f8400dc00 comp rx=0 tx=0).stop 2024-04-16T05:46:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 shutdown_connections 2024-04-16T05:46:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f3f780a3d20 0x7f3f780a61e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f901986d0 0x7f3f90198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f90106210 0x7f3f9019eb50 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 --2- 172.21.15.86:0/3689132048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f90101140 0x7f3f9019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.555+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 >> 172.21.15.86:0/3689132048 conn(0x7f3f90078000 msgr2=0x7f3f900694b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.556+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 shutdown_connections 2024-04-16T05:46:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.556+0000 7f3f97f40640 1 -- 172.21.15.86:0/3689132048 wait complete. 2024-04-16T05:46:23.565 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:46:23.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.674+0000 7fef1889a640 1 -- 172.21.15.86:0/3244735193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 msgr2=0x7fef1410ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.674+0000 7fef1889a640 1 --2- 172.21.15.86:0/3244735193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 0x7fef1410ea60 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009900 tx=0x7fef0c02bd60 comp rx=0 tx=0).stop 2024-04-16T05:46:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.674+0000 7fef1889a640 1 -- 172.21.15.86:0/3244735193 shutdown_connections 2024-04-16T05:46:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.674+0000 7fef1889a640 1 --2- 172.21.15.86:0/3244735193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 0x7fef1410ea60 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.675+0000 7fef1889a640 1 --2- 172.21.15.86:0/3244735193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef14103ba0 0x7fef1410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.675+0000 7fef1889a640 1 --2- 172.21.15.86:0/3244735193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef14103280 0x7fef14103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.675+0000 7fef1889a640 1 -- 172.21.15.86:0/3244735193 >> 172.21.15.86:0/3244735193 conn(0x7fef14078000 msgr2=0x7fef14078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.675+0000 7fef1889a640 1 -- 172.21.15.86:0/3244735193 shutdown_connections 2024-04-16T05:46:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.675+0000 7fef1889a640 1 -- 172.21.15.86:0/3244735193 wait complete. 2024-04-16T05:46:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.675+0000 7fef1889a640 1 Processor -- start 2024-04-16T05:46:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.675+0000 7fef1889a640 1 -- start start 2024-04-16T05:46:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.675+0000 7fef1889a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef14103280 0x7fef1419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef1889a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef14103ba0 0x7fef1419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef1889a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 0x7fef14198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef1889a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef14112450 con 0x7fef1410c5c0 2024-04-16T05:46:23.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef1889a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fef141122d0 con 0x7fef14103ba0 2024-04-16T05:46:23.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef1889a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef141125d0 con 0x7fef14103280 2024-04-16T05:46:23.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef12d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 0x7fef14198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef12d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 0x7fef14198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42668/0 (socket says 172.21.15.86:42668) 2024-04-16T05:46:23.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef12d9d640 1 -- 172.21.15.86:0/1454946713 learned_addr learned my addr 172.21.15.86:0/1454946713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:23.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef1259c640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef14103280 0x7fef1419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef11d9b640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef14103ba0 0x7fef1419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef12d9d640 1 -- 172.21.15.86:0/1454946713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef14103280 msgr2=0x7fef1419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef12d9d640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef14103280 0x7fef1419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef12d9d640 1 -- 172.21.15.86:0/1454946713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef14103ba0 msgr2=0x7fef1419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef12d9d640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef14103ba0 0x7fef1419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef12d9d640 1 -- 172.21.15.86:0/1454946713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef14198ff0 con 0x7fef1410c5c0 2024-04-16T05:46:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef12d9d640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 0x7fef14198730 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fef0c02c370 tx=0x7fef0c02dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef0b7fe640 1 -- 172.21.15.86:0/1454946713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c043020 con 0x7fef1410c5c0 2024-04-16T05:46:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef0b7fe640 1 -- 172.21.15.86:0/1454946713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0c0054e0 con 0x7fef1410c5c0 2024-04-16T05:46:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef14199280 con 0x7fef1410c5c0 2024-04-16T05:46:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef0b7fe640 1 -- 172.21.15.86:0/1454946713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c003c20 con 0x7fef1410c5c0 2024-04-16T05:46:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.676+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef141a55a0 con 0x7fef1410c5c0 2024-04-16T05:46:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.677+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef141995f0 con 0x7fef1410c5c0 2024-04-16T05:46:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.678+0000 7fef0b7fe640 1 -- 172.21.15.86:0/1454946713 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef0c046020 con 0x7fef1410c5c0 2024-04-16T05:46:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.679+0000 7fef0b7fe640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fef000a3d20 0x7fef000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.679+0000 7fef1259c640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fef000a3d20 0x7fef000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.679+0000 7fef0b7fe640 1 -- 172.21.15.86:0/1454946713 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7fef0c0f2e10 con 0x7fef1410c5c0 2024-04-16T05:46:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.679+0000 7fef1259c640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fef000a3d20 0x7fef000a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef04009f20 tx=0x7fef040094f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.683+0000 7fef0b7fe640 1 -- 172.21.15.86:0/1454946713 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef0c04d020 con 0x7fef1410c5c0 2024-04-16T05:46:23.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.860+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fef141a5770 con 0x7fef1410c5c0 2024-04-16T05:46:23.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.860+0000 7fef0b7fe640 1 -- 172.21.15.86:0/1454946713 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7fef0c0bf1f0 con 0x7fef1410c5c0 2024-04-16T05:46:23.859 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:46:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fef000a3d20 msgr2=0x7fef000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fef000a3d20 0x7fef000a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef04009f20 tx=0x7fef040094f0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 msgr2=0x7fef14198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 0x7fef14198730 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fef0c02c370 tx=0x7fef0c02dd40 comp rx=0 tx=0).stop 2024-04-16T05:46:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 shutdown_connections 2024-04-16T05:46:23.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1410c5c0 0x7fef14198730 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fef000a3d20 0x7fef000a61e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef14103ba0 0x7fef1419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 --2- 172.21.15.86:0/1454946713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef14103280 0x7fef1419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 >> 172.21.15.86:0/1454946713 conn(0x7fef14078000 msgr2=0x7fef140fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:23.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 shutdown_connections 2024-04-16T05:46:23.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.862+0000 7fef1889a640 1 -- 172.21.15.86:0/1454946713 wait complete. 2024-04-16T05:46:23.872 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:46:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 -- 172.21.15.86:0/19103882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 msgr2=0x7f4b5006aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 --2- 172.21.15.86:0/19103882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 0x7f4b5006aa80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009930 tx=0x7f4b4002bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 -- 172.21.15.86:0/19103882 shutdown_connections 2024-04-16T05:46:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 --2- 172.21.15.86:0/19103882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b5006afc0 0x7f4b5010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 --2- 172.21.15.86:0/19103882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 0x7f4b5006aa80 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 --2- 172.21.15.86:0/19103882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b501080b0 0x7f4b50108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 -- 172.21.15.86:0/19103882 >> 172.21.15.86:0/19103882 conn(0x7f4b50076d00 msgr2=0x7f4b50077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 -- 172.21.15.86:0/19103882 shutdown_connections 2024-04-16T05:46:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 -- 172.21.15.86:0/19103882 wait complete. 2024-04-16T05:46:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 Processor -- start 2024-04-16T05:46:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 -- start start 2024-04-16T05:46:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 0x7f4b5019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b5006afc0 0x7f4b5019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b501080b0 0x7f4b50198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501123f0 con 0x7f4b5006a600 2024-04-16T05:46:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.982+0000 7f4b58488640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50112270 con 0x7f4b5006afc0 2024-04-16T05:46:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b58488640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50112570 con 0x7f4b501080b0 2024-04-16T05:46:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56224640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 0x7f4b5019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56224640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 0x7f4b5019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42678/0 (socket says 172.21.15.86:42678) 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56224640 1 -- 172.21.15.86:0/2439024502 learned_addr learned my addr 172.21.15.86:0/2439024502 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56a25640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b501080b0 0x7f4b50198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b55a23640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b5006afc0 0x7f4b5019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56224640 1 -- 172.21.15.86:0/2439024502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b501080b0 msgr2=0x7f4b50198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56224640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b501080b0 0x7f4b50198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56224640 1 -- 172.21.15.86:0/2439024502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b5006afc0 msgr2=0x7f4b5019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56224640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b5006afc0 0x7f4b5019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56224640 1 -- 172.21.15.86:0/2439024502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b50198f60 con 0x7f4b5006a600 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b55a23640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b5006afc0 0x7f4b5019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:46:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b56224640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 0x7f4b5019e620 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00cce0 tx=0x7f4b4c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:23.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b477fe640 1 -- 172.21.15.86:0/2439024502 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004b20 con 0x7f4b5006a600 2024-04-16T05:46:23.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b50199250 con 0x7f4b5006a600 2024-04-16T05:46:23.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b477fe640 1 -- 172.21.15.86:0/2439024502 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b4c004cc0 con 0x7f4b5006a600 2024-04-16T05:46:23.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b477fe640 1 -- 172.21.15.86:0/2439024502 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00f7d0 con 0x7f4b5006a600 2024-04-16T05:46:23.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.983+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501a5470 con 0x7f4b5006a600 2024-04-16T05:46:23.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.984+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b50103710 con 0x7f4b5006a600 2024-04-16T05:46:23.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.985+0000 7f4b477fe640 1 -- 172.21.15.86:0/2439024502 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b4c00f970 con 0x7f4b5006a600 2024-04-16T05:46:23.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.986+0000 7f4b477fe640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4b3c0a3d20 0x7f4b3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:23.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.986+0000 7f4b55a23640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4b3c0a3d20 0x7f4b3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:23.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.986+0000 7f4b477fe640 1 -- 172.21.15.86:0/2439024502 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f4b4c0d0d60 con 0x7f4b5006a600 2024-04-16T05:46:23.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.987+0000 7f4b55a23640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4b3c0a3d20 0x7f4b3c0a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4b40006010 tx=0x7f4b40000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:23.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:23.991+0000 7f4b477fe640 1 -- 172.21.15.86:0/2439024502 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b4c014030 con 0x7f4b5006a600 2024-04-16T05:46:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.168+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4b5002d620 con 0x7f4b5006a600 2024-04-16T05:46:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.168+0000 7f4b477fe640 1 -- 172.21.15.86:0/2439024502 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f4b4c09d030 con 0x7f4b5006a600 2024-04-16T05:46:24.166 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-16T05:46:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4b3c0a3d20 msgr2=0x7f4b3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4b3c0a3d20 0x7f4b3c0a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4b40006010 tx=0x7f4b40000bc0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 msgr2=0x7f4b5019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 0x7f4b5019e620 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00cce0 tx=0x7f4b4c00a620 comp rx=0 tx=0).stop 2024-04-16T05:46:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 shutdown_connections 2024-04-16T05:46:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b501080b0 0x7f4b50198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f4b3c0a3d20 0x7f4b3c0a61e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b5006afc0 0x7f4b5019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 --2- 172.21.15.86:0/2439024502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5006a600 0x7f4b5019e620 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 >> 172.21.15.86:0/2439024502 conn(0x7f4b50076d00 msgr2=0x7f4b501000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 shutdown_connections 2024-04-16T05:46:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.170+0000 7f4b58488640 1 -- 172.21.15.86:0/2439024502 wait complete. 2024-04-16T05:46:24.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:46:24.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.288+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2958474656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c1065b0 msgr2=0x7f8b5c100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.288+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2958474656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c1065b0 0x7f8b5c100930 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c009900 tx=0x7f8b4c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:24.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.288+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2958474656 shutdown_connections 2024-04-16T05:46:24.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.288+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2958474656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b5c100e70 0x7f8b5c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.288+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2958474656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c1065b0 0x7f8b5c100930 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.288+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2958474656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b5c105c90 0x7f8b5c106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.288+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2958474656 >> 172.21.15.86:0/2958474656 conn(0x7f8b5c076190 msgr2=0x7f8b5c0765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:24.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.288+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2958474656 shutdown_connections 2024-04-16T05:46:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.288+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2958474656 wait complete. 2024-04-16T05:46:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b63c7b640 1 Processor -- start 2024-04-16T05:46:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b63c7b640 1 -- start start 2024-04-16T05:46:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b63c7b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c100e70 0x7f8b5c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b63c7b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b5c105c90 0x7f8b5c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b63c7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b5c1065b0 0x7f8b5c1a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b63c7b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c118a30 con 0x7f8b5c100e70 2024-04-16T05:46:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b63c7b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1188b0 con 0x7f8b5c105c90 2024-04-16T05:46:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b63c7b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c118bb0 con 0x7f8b5c1065b0 2024-04-16T05:46:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b61a17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c100e70 0x7f8b5c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b61a17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c100e70 0x7f8b5c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42710/0 (socket says 172.21.15.86:42710) 2024-04-16T05:46:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b61a17640 1 -- 172.21.15.86:0/2928057108 learned_addr learned my addr 172.21.15.86:0/2928057108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b61216640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b5c105c90 0x7f8b5c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.289+0000 7f8b62218640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b5c1065b0 0x7f8b5c1a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b61a17640 1 -- 172.21.15.86:0/2928057108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b5c1065b0 msgr2=0x7f8b5c1a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b61a17640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b5c1065b0 0x7f8b5c1a3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b61a17640 1 -- 172.21.15.86:0/2928057108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b5c105c90 msgr2=0x7f8b5c19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b61a17640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b5c105c90 0x7f8b5c19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b61a17640 1 -- 172.21.15.86:0/2928057108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5c1a3e80 con 0x7f8b5c100e70 2024-04-16T05:46:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b61216640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b5c105c90 0x7f8b5c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:46:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b62218640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b5c1065b0 0x7f8b5c1a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:46:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b61a17640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c100e70 0x7f8b5c19ee90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800d950 tx=0x7f8b5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b52ffd640 1 -- 172.21.15.86:0/2928057108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58017030 con 0x7f8b5c100e70 2024-04-16T05:46:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b52ffd640 1 -- 172.21.15.86:0/2928057108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b58004020 con 0x7f8b5c100e70 2024-04-16T05:46:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b52ffd640 1 -- 172.21.15.86:0/2928057108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b580047a0 con 0x7f8b5c100e70 2024-04-16T05:46:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5c1a4170 con 0x7f8b5c100e70 2024-04-16T05:46:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.290+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5c1aba00 con 0x7f8b5c100e70 2024-04-16T05:46:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.291+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b5c069270 con 0x7f8b5c100e70 2024-04-16T05:46:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.292+0000 7f8b52ffd640 1 -- 172.21.15.86:0/2928057108 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b580041d0 con 0x7f8b5c100e70 2024-04-16T05:46:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.293+0000 7f8b52ffd640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b480a3d20 0x7f8b480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.293+0000 7f8b61216640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b480a3d20 0x7f8b480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.293+0000 7f8b52ffd640 1 -- 172.21.15.86:0/2928057108 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f8b580d0620 con 0x7f8b5c100e70 2024-04-16T05:46:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.294+0000 7f8b61216640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b480a3d20 0x7f8b480a61e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c1047b0 tx=0x7f8b4c02c710 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.298+0000 7f8b52ffd640 1 -- 172.21.15.86:0/2928057108 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b5809c8f0 con 0x7f8b5c100e70 2024-04-16T05:46:24.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:24 smithi086 ceph-mon[32028]: pgmap v346: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:46:24.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:24 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1252812932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:46:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:24 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3689132048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:46:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:24 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1454946713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:46:24.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:24 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2439024502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:46:24.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.480+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8b5c1abc60 con 0x7f8b5c100e70 2024-04-16T05:46:24.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.480+0000 7f8b52ffd640 1 -- 172.21.15.86:0/2928057108 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) ==== 103+0+11 (secure 0 0 0) 0x7f8b5809c1f0 con 0x7f8b5c100e70 2024-04-16T05:46:24.479 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-16T05:46:24.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b480a3d20 msgr2=0x7f8b480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b480a3d20 0x7f8b480a61e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c1047b0 tx=0x7f8b4c02c710 comp rx=0 tx=0).stop 2024-04-16T05:46:24.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c100e70 msgr2=0x7f8b5c19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c100e70 0x7f8b5c19ee90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800d950 tx=0x7f8b5800de10 comp rx=0 tx=0).stop 2024-04-16T05:46:24.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 shutdown_connections 2024-04-16T05:46:24.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b5c1065b0 0x7f8b5c1a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f8b480a3d20 0x7f8b480a61e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b5c105c90 0x7f8b5c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 --2- 172.21.15.86:0/2928057108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b5c100e70 0x7f8b5c19ee90 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 >> 172.21.15.86:0/2928057108 conn(0x7f8b5c076190 msgr2=0x7f8b5c102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 shutdown_connections 2024-04-16T05:46:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.482+0000 7f8b63c7b640 1 -- 172.21.15.86:0/2928057108 wait complete. 2024-04-16T05:46:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:24 smithi196 ceph-mon[36373]: pgmap v346: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:46:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:24 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1252812932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:46:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:24 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3689132048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:46:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:24 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1454946713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:46:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:24 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2439024502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:46:24.492 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T05:46:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 -- 172.21.15.86:0/1602085467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 msgr2=0x7f8904102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 --2- 172.21.15.86:0/1602085467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 0x7f8904102620 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009900 tx=0x7f88f402bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 -- 172.21.15.86:0/1602085467 shutdown_connections 2024-04-16T05:46:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 --2- 172.21.15.86:0/1602085467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8904107310 0x7f890410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 --2- 172.21.15.86:0/1602085467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904102bf0 0x7f8904106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 --2- 172.21.15.86:0/1602085467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 0x7f8904102620 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 -- 172.21.15.86:0/1602085467 >> 172.21.15.86:0/1602085467 conn(0x7f8904079450 msgr2=0x7f8904079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 -- 172.21.15.86:0/1602085467 shutdown_connections 2024-04-16T05:46:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 -- 172.21.15.86:0/1602085467 wait complete. 2024-04-16T05:46:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.599+0000 7f890b532640 1 Processor -- start 2024-04-16T05:46:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f890b532640 1 -- start start 2024-04-16T05:46:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f890b532640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 0x7f890419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f890b532640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904107310 0x7f890419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f890b532640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89041987a0 0x7f8904198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:24.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f890b532640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8904112420 con 0x7f8904102240 2024-04-16T05:46:24.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f890b532640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89041122a0 con 0x7f8904107310 2024-04-16T05:46:24.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f890b532640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89041125a0 con 0x7f89041987a0 2024-04-16T05:46:24.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f89092ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 0x7f890419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:24.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f89092ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 0x7f890419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42728/0 (socket says 172.21.15.86:42728) 2024-04-16T05:46:24.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f89092ce640 1 -- 172.21.15.86:0/1976677213 learned_addr learned my addr 172.21.15.86:0/1976677213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:24.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f8908acd640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904107310 0x7f890419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:24.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f89092ce640 1 -- 172.21.15.86:0/1976677213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89041987a0 msgr2=0x7f8904198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f89092ce640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89041987a0 0x7f8904198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f89092ce640 1 -- 172.21.15.86:0/1976677213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904107310 msgr2=0x7f890419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f89092ce640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904107310 0x7f890419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f89092ce640 1 -- 172.21.15.86:0/1976677213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8904199330 con 0x7f8904102240 2024-04-16T05:46:24.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f8908acd640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904107310 0x7f890419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:46:24.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f89092ce640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 0x7f890419e630 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f88f4006010 tx=0x7f88f402cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:24.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f88fa7fc640 1 -- 172.21.15.86:0/1976677213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4041070 con 0x7f8904102240 2024-04-16T05:46:24.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f88fa7fc640 1 -- 172.21.15.86:0/1976677213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f40054e0 con 0x7f8904102240 2024-04-16T05:46:24.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f88fa7fc640 1 -- 172.21.15.86:0/1976677213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4034440 con 0x7f8904102240 2024-04-16T05:46:24.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.600+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89041995c0 con 0x7f8904102240 2024-04-16T05:46:24.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.601+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89041a5770 con 0x7f8904102240 2024-04-16T05:46:24.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.601+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8904103710 con 0x7f8904102240 2024-04-16T05:46:24.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.603+0000 7f88fa7fc640 1 -- 172.21.15.86:0/1976677213 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f88f4044070 con 0x7f8904102240 2024-04-16T05:46:24.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.603+0000 7f88fa7fc640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f88f00a3d20 0x7f88f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.603+0000 7f8908acd640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f88f00a3d20 0x7f88f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.603+0000 7f88fa7fc640 1 -- 172.21.15.86:0/1976677213 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f88f40f29d0 con 0x7f8904102240 2024-04-16T05:46:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.604+0000 7f8908acd640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f88f00a3d20 0x7f88f00a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8904101940 tx=0x7f8900005d30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.607+0000 7f88fa7fc640 1 -- 172.21.15.86:0/1976677213 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88f40bedb0 con 0x7f8904102240 2024-04-16T05:46:24.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:24 smithi194 ceph-mon[35865]: pgmap v346: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:46:24.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:24 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1252812932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:46:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:24 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3689132048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:46:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:24 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1454946713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:46:24.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:24 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2439024502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:46:24.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.798+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8904069270 con 0x7f8904102240 2024-04-16T05:46:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.799+0000 7f88fa7fc640 1 -- 172.21.15.86:0/1976677213 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v105) ==== 81+0+943 (secure 0 0 0) 0x7f88f40be6b0 con 0x7f8904102240 2024-04-16T05:46:24.797 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/3194681946 2024-04-17T05:35:16.929800+0000 2024-04-16T05:46:24.797 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2330974702 2024-04-17T05:35:16.929800+0000 2024-04-16T05:46:24.797 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3900523320 2024-04-17T05:35:16.929800+0000 2024-04-16T05:46:24.798 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2641972995 2024-04-17T05:35:16.929800+0000 2024-04-16T05:46:24.798 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/501397249 2024-04-17T05:35:16.929800+0000 2024-04-16T05:46:24.798 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1016266433 2024-04-17T05:25:51.394705+0000 2024-04-16T05:46:24.798 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/908662426 2024-04-17T05:25:51.394705+0000 2024-04-16T05:46:24.798 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1420225001 2024-04-17T05:25:51.394705+0000 2024-04-16T05:46:24.798 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3914995291 2024-04-17T05:25:24.654093+0000 2024-04-16T05:46:24.798 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/3194681946 2024-04-17T05:35:16.929800+0000 2024-04-16T05:46:24.799 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2966713555 2024-04-17T05:25:24.654093+0000 2024-04-16T05:46:24.799 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3037484112 2024-04-17T05:25:24.654093+0000 2024-04-16T05:46:24.799 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1420225001 2024-04-17T05:25:51.394705+0000 2024-04-16T05:46:24.799 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1904754808 2024-04-17T05:25:24.654093+0000 2024-04-16T05:46:24.799 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1904754808 2024-04-17T05:25:24.654093+0000 2024-04-16T05:46:24.799 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/661459728 2024-04-17T05:25:51.394705+0000 2024-04-16T05:46:24.800 INFO:teuthology.orchestra.run.smithi086.stderr:listed 16 entries 2024-04-16T05:46:24.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f88f00a3d20 msgr2=0x7f88f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f88f00a3d20 0x7f88f00a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8904101940 tx=0x7f8900005d30 comp rx=0 tx=0).stop 2024-04-16T05:46:24.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 msgr2=0x7f890419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:24.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 0x7f890419e630 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f88f4006010 tx=0x7f88f402cd70 comp rx=0 tx=0).stop 2024-04-16T05:46:24.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 shutdown_connections 2024-04-16T05:46:24.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89041987a0 0x7f8904198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f88f00a3d20 0x7f88f00a61e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904107310 0x7f890419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 --2- 172.21.15.86:0/1976677213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904102240 0x7f890419e630 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:24.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 >> 172.21.15.86:0/1976677213 conn(0x7f8904079450 msgr2=0x7f89040fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:24.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 shutdown_connections 2024-04-16T05:46:24.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:24.801+0000 7f890b532640 1 -- 172.21.15.86:0/1976677213 wait complete. 2024-04-16T05:46:24.810 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-16T05:46:24.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:46:24.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-16T05:46:24.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:46:24.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:46:24.844 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-16T05:46:24.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:46:24.900 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-16T05:46:24.957 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-16T05:46:24.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:46:24.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T05:46:25.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:46:25.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-16T05:46:25.124 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-16T05:46:25.125 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-16T05:46:25.125 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:46:25.141 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16701 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/b6c50a6da260f0cb308a6348521cbeed1d8d12e29579cee745793e589f38d8c0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/b6c50a6da260f0cb308a6348521cbeed1d8d12e29579cee745793e589f38d8c0/diff,workdir=/var/lib/containers/storage/overlay/b6c50a6da260f0cb308a6348521cbeed1d8d12e29579cee745793e589f38d8c0/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/edf9593d40cbdf892f93c670fcda7c7ce5141042081e3834ce6a663807a5c2f9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/edf9593d40cbdf892f93c670fcda7c7ce5141042081e3834ce6a663807a5c2f9/diff,workdir=/var/lib/containers/storage/overlay/edf9593d40cbdf892f93c670fcda7c7ce5141042081e3834ce6a663807a5c2f9/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/f5747f57fb823e996ecf42ce5431d6998086effe07cf3e06d545be1ff2893e57/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/f5747f57fb823e996ecf42ce5431d6998086effe07cf3e06d545be1ff2893e57/diff,workdir=/var/lib/containers/storage/overlay/f5747f57fb823e996ecf42ce5431d6998086effe07cf3e06d545be1ff2893e57/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/56fe7a21c83e3e13470bc9ebea9a946c75d1a7c230829a06c1adf0963298605a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/56fe7a21c83e3e13470bc9ebea9a946c75d1a7c230829a06c1adf0963298605a/diff,workdir=/var/lib/containers/storage/overlay/56fe7a21c83e3e13470bc9ebea9a946c75d1a7c230829a06c1adf0963298605a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ea385a6136a039499d148c1bb785febb2577b27a5dfd3407800db1940d50bb8b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/ea385a6136a039499d148c1bb785febb2577b27a5dfd3407800db1940d50bb8b/diff,workdir=/var/lib/containers/storage/overlay/ea385a6136a039499d148c1bb785febb2577b27a5dfd3407800db1940d50bb8b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4433118abfd891a00d57c636d1b84838a0e83611941ef55b1fee789a2168e1f7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/4433118abfd891a00d57c636d1b84838a0e83611941ef55b1fee789a2168e1f7/diff,workdir=/var/lib/containers/storage/overlay/4433118abfd891a00d57c636d1b84838a0e83611941ef55b1fee789a2168e1f7/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/156e17e4b4de05df087f6253b2c2a9c543faf058d2081a10fddaca26c021dd7a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/156e17e4b4de05df087f6253b2c2a9c543faf058d2081a10fddaca26c021dd7a/diff,workdir=/var/lib/containers/storage/overlay/156e17e4b4de05df087f6253b2c2a9c543faf058d2081a10fddaca26c021dd7a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3242fbac9f218213651c2deb6c361a8cfd0e55a3c8b085bafab3922dac2ea1a8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/3242fbac9f218213651c2deb6c361a8cfd0e55a3c8b085bafab3922dac2ea1a8/diff,workdir=/var/lib/containers/storage/overlay/3242fbac9f218213651c2deb6c361a8cfd0e55a3c8b085bafab3922dac2ea1a8/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/05fb00820607f20def66ba0ae16c957480fb9632f219d32a62ce6d44681f1781/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/05fb00820607f20def66ba0ae16c957480fb9632f219d32a62ce6d44681f1781/diff,workdir=/var/lib/containers/storage/overlay/05fb00820607f20def66ba0ae16c957480fb9632f219d32a62ce6d44681f1781/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a0eb818cbbe5f4a03ece36aa8b0b586862d209d9356886aa468ab00112b9fba4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XRFIMIZC2SJVEW5XSL672OG7YI:/var/lib/containers/storage/overlay/l/GEEOJU6RBTE53T5HWVZYR3AGZJ,upperdir=/var/lib/containers/storage/overlay/a0eb818cbbe5f4a03ece36aa8b0b586862d209d9356886aa468ab00112b9fba4/diff,workdir=/var/lib/containers/storage/overlay/a0eb818cbbe5f4a03ece36aa8b0b586862d209d9356886aa468ab00112b9fba4/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-16T05:46:25.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:46:25.141 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-16T05:46:25.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:46:25.198 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-16T05:46:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-16T05:46:25.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:46:25.271 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-16T05:46:25.282 INFO:tasks.cephadm:Teardown begin 2024-04-16T05:46:25.283 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-16T05:46:25.283 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:46:25.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.418+0000 7fd14b559640 1 -- 172.21.15.86:0/844989217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 msgr2=0x7fd144101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.418+0000 7fd14b559640 1 --2- 172.21.15.86:0/844989217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 0x7fd144101530 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fd134009900 tx=0x7fd13402bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:25.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.418+0000 7fd14b559640 1 -- 172.21.15.86:0/844989217 shutdown_connections 2024-04-16T05:46:25.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.418+0000 7fd14b559640 1 --2- 172.21.15.86:0/844989217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd144106220 0x7fd14410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.418+0000 7fd14b559640 1 --2- 172.21.15.86:0/844989217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd144101b00 0x7fd144105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.418+0000 7fd14b559640 1 --2- 172.21.15.86:0/844989217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 0x7fd144101530 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.418+0000 7fd14b559640 1 -- 172.21.15.86:0/844989217 >> 172.21.15.86:0/844989217 conn(0x7fd144078000 msgr2=0x7fd144078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:25.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.418+0000 7fd14b559640 1 -- 172.21.15.86:0/844989217 shutdown_connections 2024-04-16T05:46:25.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.418+0000 7fd14b559640 1 -- 172.21.15.86:0/844989217 wait complete. 2024-04-16T05:46:25.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd14b559640 1 Processor -- start 2024-04-16T05:46:25.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd14b559640 1 -- start start 2024-04-16T05:46:25.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd14b559640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 0x7fd144199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd14b559640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd144106220 0x7fd144199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd14b559640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd14419a200 0x7fd1441a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd14b559640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1441112d0 con 0x7fd144101150 2024-04-16T05:46:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd14b559640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd144111150 con 0x7fd14419a200 2024-04-16T05:46:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd14b559640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd144111450 con 0x7fd144106220 2024-04-16T05:46:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd1492f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 0x7fd144199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd1492f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 0x7fd144199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42744/0 (socket says 172.21.15.86:42744) 2024-04-16T05:46:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd1492f5640 1 -- 172.21.15.86:0/1261772395 learned_addr learned my addr 172.21.15.86:0/1261772395 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd149af6640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd14419a200 0x7fd1441a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd148af4640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd144106220 0x7fd144199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd1492f5640 1 -- 172.21.15.86:0/1261772395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd144106220 msgr2=0x7fd144199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd1492f5640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd144106220 0x7fd144199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd1492f5640 1 -- 172.21.15.86:0/1261772395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd14419a200 msgr2=0x7fd1441a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd1492f5640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd14419a200 0x7fd1441a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd1492f5640 1 -- 172.21.15.86:0/1261772395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1441a4f80 con 0x7fd144101150 2024-04-16T05:46:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd1492f5640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 0x7fd144199730 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd134006010 tx=0x7fd13402cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.419+0000 7fd13a7fc640 1 -- 172.21.15.86:0/1261772395 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134041070 con 0x7fd144101150 2024-04-16T05:46:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.420+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1441a5210 con 0x7fd144101150 2024-04-16T05:46:25.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.420+0000 7fd13a7fc640 1 -- 172.21.15.86:0/1261772395 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1340054e0 con 0x7fd144101150 2024-04-16T05:46:25.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.420+0000 7fd13a7fc640 1 -- 172.21.15.86:0/1261772395 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134034440 con 0x7fd144101150 2024-04-16T05:46:25.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.420+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1441a5750 con 0x7fd144101150 2024-04-16T05:46:25.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.420+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd144068e20 con 0x7fd144101150 2024-04-16T05:46:25.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.422+0000 7fd13a7fc640 1 -- 172.21.15.86:0/1261772395 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd134044070 con 0x7fd144101150 2024-04-16T05:46:25.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.422+0000 7fd13a7fc640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1300a3d20 0x7fd1300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:25.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.422+0000 7fd148af4640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1300a3d20 0x7fd1300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:25.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.422+0000 7fd13a7fc640 1 -- 172.21.15.86:0/1261772395 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7fd1340f27b0 con 0x7fd144101150 2024-04-16T05:46:25.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.423+0000 7fd148af4640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1300a3d20 0x7fd1300a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd140005fd0 tx=0x7fd140005e20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:25.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.427+0000 7fd13a7fc640 1 -- 172.21.15.86:0/1261772395 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1340beb90 con 0x7fd144101150 2024-04-16T05:46:25.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:25 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2928057108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:46:25.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:25 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1976677213' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:46:25.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:25 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2928057108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:46:25.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:25 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1976677213' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.642+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd144109450 con 0x7fd144101150 2024-04-16T05:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.642+0000 7fd13a7fc640 1 -- 172.21.15.86:0/1261772395 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fd1340be490 con 0x7fd144101150 2024-04-16T05:46:25.641 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:46:25.641 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.644+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1300a3d20 msgr2=0x7fd1300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.644+0000 7fd14b559640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1300a3d20 0x7fd1300a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd140005fd0 tx=0x7fd140005e20 comp rx=0 tx=0).stop 2024-04-16T05:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.644+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 msgr2=0x7fd144199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.644+0000 7fd14b559640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 0x7fd144199730 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd134006010 tx=0x7fd13402cd70 comp rx=0 tx=0).stop 2024-04-16T05:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.644+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 shutdown_connections 2024-04-16T05:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.644+0000 7fd14b559640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd1300a3d20 0x7fd1300a61e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.644+0000 7fd14b559640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd14419a200 0x7fd1441a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.645+0000 7fd14b559640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd144106220 0x7fd144199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.645+0000 7fd14b559640 1 --2- 172.21.15.86:0/1261772395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd144101150 0x7fd144199730 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.645+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 >> 172.21.15.86:0/1261772395 conn(0x7fd144078000 msgr2=0x7fd1440ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.645+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 shutdown_connections 2024-04-16T05:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.645+0000 7fd14b559640 1 -- 172.21.15.86:0/1261772395 wait complete. 2024-04-16T05:46:25.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:46:25.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:25 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2928057108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:46:25.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:25 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1976677213' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:46:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.761+0000 7fc910ea7640 1 -- 172.21.15.86:0/3572931910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c100bc0 msgr2=0x7fc90c100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.761+0000 7fc910ea7640 1 --2- 172.21.15.86:0/3572931910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c100bc0 0x7fc90c100fa0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009900 tx=0x7fc8fc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.761+0000 7fc910ea7640 1 -- 172.21.15.86:0/3572931910 shutdown_connections 2024-04-16T05:46:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.761+0000 7fc910ea7640 1 --2- 172.21.15.86:0/3572931910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c107380 0x7fc90c10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.761+0000 7fc910ea7640 1 --2- 172.21.15.86:0/3572931910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc90c101570 0x7fc90c106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.761+0000 7fc910ea7640 1 --2- 172.21.15.86:0/3572931910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c100bc0 0x7fc90c100fa0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.761+0000 7fc910ea7640 1 -- 172.21.15.86:0/3572931910 >> 172.21.15.86:0/3572931910 conn(0x7fc90c0761e0 msgr2=0x7fc90c0765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.761+0000 7fc910ea7640 1 -- 172.21.15.86:0/3572931910 shutdown_connections 2024-04-16T05:46:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.761+0000 7fc910ea7640 1 -- 172.21.15.86:0/3572931910 wait complete. 2024-04-16T05:46:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc910ea7640 1 Processor -- start 2024-04-16T05:46:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc910ea7640 1 -- start start 2024-04-16T05:46:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc910ea7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c100bc0 0x7fc90c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc910ea7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c107380 0x7fc90c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc910ea7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc90c1986f0 0x7fc90c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc910ea7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c112450 con 0x7fc90c107380 2024-04-16T05:46:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc910ea7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1122d0 con 0x7fc90c100bc0 2024-04-16T05:46:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc910ea7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1125d0 con 0x7fc90c1986f0 2024-04-16T05:46:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc903fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c107380 0x7fc90c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc903fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c107380 0x7fc90c19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42758/0 (socket says 172.21.15.86:42758) 2024-04-16T05:46:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc903fff640 1 -- 172.21.15.86:0/1771272128 learned_addr learned my addr 172.21.15.86:0/1771272128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc90a59c640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c100bc0 0x7fc90c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc90ad9d640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc90c1986f0 0x7fc90c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc903fff640 1 -- 172.21.15.86:0/1771272128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc90c1986f0 msgr2=0x7fc90c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc903fff640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc90c1986f0 0x7fc90c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc903fff640 1 -- 172.21.15.86:0/1771272128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c100bc0 msgr2=0x7fc90c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc903fff640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c100bc0 0x7fc90c19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.762+0000 7fc903fff640 1 -- 172.21.15.86:0/1771272128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c1992f0 con 0x7fc90c107380 2024-04-16T05:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.763+0000 7fc903fff640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c107380 0x7fc90c19eb70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400d740 tx=0x7fc8f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.763+0000 7fc9037fe640 1 -- 172.21.15.86:0/1771272128 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4017070 con 0x7fc90c107380 2024-04-16T05:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.763+0000 7fc9037fe640 1 -- 172.21.15.86:0/1771272128 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f400de50 con 0x7fc90c107380 2024-04-16T05:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.763+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c1a53d0 con 0x7fc90c107380 2024-04-16T05:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.763+0000 7fc9037fe640 1 -- 172.21.15.86:0/1771272128 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4004e00 con 0x7fc90c107380 2024-04-16T05:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.763+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c1a5780 con 0x7fc90c107380 2024-04-16T05:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.764+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c06a170 con 0x7fc90c107380 2024-04-16T05:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.765+0000 7fc9037fe640 1 -- 172.21.15.86:0/1771272128 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc8f4004fa0 con 0x7fc90c107380 2024-04-16T05:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.765+0000 7fc9037fe640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc8f80a3d20 0x7fc8f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.765+0000 7fc90a59c640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc8f80a3d20 0x7fc8f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.765+0000 7fc9037fe640 1 -- 172.21.15.86:0/1771272128 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7fc8f40d0590 con 0x7fc90c107380 2024-04-16T05:46:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.766+0000 7fc90a59c640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc8f80a3d20 0x7fc8f80a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006010 tx=0x7fc8fc005dd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.770+0000 7fc9037fe640 1 -- 172.21.15.86:0/1771272128 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8f409c860 con 0x7fc90c107380 2024-04-16T05:46:25.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.984+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc90c069270 con 0x7fc90c107380 2024-04-16T05:46:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.985+0000 7fc9037fe640 1 -- 172.21.15.86:0/1771272128 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11947 (secure 0 0 0) 0x7fc8f409c160 con 0x7fc90c107380 2024-04-16T05:46:25.983 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:46:25.984 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2004229798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2004229798},{"type":"v1","addr":"172.21.15.86:6835","nonce":2004229798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2865823959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2865823959},{"type":"v1","addr":"172.21.15.86:6837","nonce":2865823959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:54.852973+0000","modified":"2024-04-16T05:35:15.512846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24463,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/1379378484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1379378484},{"type":"v1","addr":"172.21.15.86:6839","nonce":1379378484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14631":{"gid":14631,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1989684988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1989684988},{"type":"v1","addr":"172.21.15.86:6841","nonce":1989684988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3318048466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3318048466},{"type":"v1","addr":"172.21.15.194:6833","nonce":3318048466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1220860047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1220860047},{"type":"v1","addr":"172.21.15.196:6833","nonce":1220860047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2618844409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2618844409},{"type":"v1","addr":"172.21.15.194:6835","nonce":2618844409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6835/3736061748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3736061748},{"type":"v1","addr":"172.21.15.196:6835","nonce":3736061748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.194:6837/3400155977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3400155977},{"type":"v1","addr":"172.21.15.194:6837","nonce":3400155977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6837/2445566344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":2445566344},{"type":"v1","addr":"172.21.15.196:6837","nonce":2445566344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6839/3455974815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":3455974815},{"type":"v1","addr":"172.21.15.196:6839","nonce":3455974815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6839/601781370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":601781370},{"type":"v1","addr":"172.21.15.194:6839","nonce":601781370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24455,24463,24481,24482]},"id":1}]} 2024-04-16T05:46:25.984 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-16T05:46:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc8f80a3d20 msgr2=0x7fc8f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc8f80a3d20 0x7fc8f80a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006010 tx=0x7fc8fc005dd0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c107380 msgr2=0x7fc90c19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:25.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c107380 0x7fc90c19eb70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400d740 tx=0x7fc8f400dc00 comp rx=0 tx=0).stop 2024-04-16T05:46:25.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 shutdown_connections 2024-04-16T05:46:25.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc90c1986f0 0x7fc90c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc90c107380 0x7fc90c19eb70 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc8f80a3d20 0x7fc8f80a61e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 --2- 172.21.15.86:0/1771272128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c100bc0 0x7fc90c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:25.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 >> 172.21.15.86:0/1771272128 conn(0x7fc90c0761e0 msgr2=0x7fc90c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:25.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 shutdown_connections 2024-04-16T05:46:25.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:25.987+0000 7fc910ea7640 1 -- 172.21.15.86:0/1771272128 wait complete. 2024-04-16T05:46:25.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:46:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.103+0000 7f046e814640 1 -- 172.21.15.86:0/2252106057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 msgr2=0x7f0468106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.103+0000 7f046e814640 1 --2- 172.21.15.86:0/2252106057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 0x7f0468106a40 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f045800aa40 tx=0x7f045802cf50 comp rx=0 tx=0).stop 2024-04-16T05:46:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.103+0000 7f046e814640 1 -- 172.21.15.86:0/2252106057 shutdown_connections 2024-04-16T05:46:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.103+0000 7f046e814640 1 --2- 172.21.15.86:0/2252106057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0468101060 0x7f046810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.103+0000 7f046e814640 1 --2- 172.21.15.86:0/2252106057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04681006a0 0x7f0468100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.103+0000 7f046e814640 1 --2- 172.21.15.86:0/2252106057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 0x7f0468106a40 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.103+0000 7f046e814640 1 -- 172.21.15.86:0/2252106057 >> 172.21.15.86:0/2252106057 conn(0x7f0468077530 msgr2=0x7f04680fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.103+0000 7f046e814640 1 -- 172.21.15.86:0/2252106057 shutdown_connections 2024-04-16T05:46:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.103+0000 7f046e814640 1 -- 172.21.15.86:0/2252106057 wait complete. 2024-04-16T05:46:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046e814640 1 Processor -- start 2024-04-16T05:46:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046e814640 1 -- start start 2024-04-16T05:46:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046e814640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04681006a0 0x7f04681966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046e814640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0468101060 0x7f0468196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046e814640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 0x7f046819afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046e814640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0468110190 con 0x7f0468106660 2024-04-16T05:46:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046e814640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0468110010 con 0x7f04681006a0 2024-04-16T05:46:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046e814640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0468110310 con 0x7f0468101060 2024-04-16T05:46:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046cdb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 0x7f046819afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046cdb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 0x7f046819afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42778/0 (socket says 172.21.15.86:42778) 2024-04-16T05:46:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046cdb1640 1 -- 172.21.15.86:0/656400334 learned_addr learned my addr 172.21.15.86:0/656400334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f0467fff640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04681006a0 0x7f04681966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f04677fe640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0468101060 0x7f0468196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.104+0000 7f046cdb1640 1 -- 172.21.15.86:0/656400334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0468101060 msgr2=0x7f0468196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f046cdb1640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0468101060 0x7f0468196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f046cdb1640 1 -- 172.21.15.86:0/656400334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04681006a0 msgr2=0x7f04681966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f046cdb1640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04681006a0 0x7f04681966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f046cdb1640 1 -- 172.21.15.86:0/656400334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046819b6e0 con 0x7f0468106660 2024-04-16T05:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f04677fe640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0468101060 0x7f0468196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f046cdb1640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 0x7f046819afb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0460009fd0 tx=0x7f046000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f04657fa640 1 -- 172.21.15.86:0/656400334 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460017070 con 0x7f0468106660 2024-04-16T05:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046819b9d0 con 0x7f0468106660 2024-04-16T05:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f04657fa640 1 -- 172.21.15.86:0/656400334 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0460003e50 con 0x7f0468106660 2024-04-16T05:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f04657fa640 1 -- 172.21.15.86:0/656400334 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600106c0 con 0x7f0468106660 2024-04-16T05:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.105+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681a3260 con 0x7f0468106660 2024-04-16T05:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.106+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0468069270 con 0x7f0468106660 2024-04-16T05:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.107+0000 7f04657fa640 1 -- 172.21.15.86:0/656400334 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0460003970 con 0x7f0468106660 2024-04-16T05:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.107+0000 7f04657fa640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f04500a3d20 0x7f04500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.108+0000 7f04657fa640 1 -- 172.21.15.86:0/656400334 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f04600d08f0 con 0x7f0468106660 2024-04-16T05:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.108+0000 7f0467fff640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f04500a3d20 0x7f04500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.108+0000 7f0467fff640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f04500a3d20 0x7f04500a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0458005ab0 tx=0x7f0458005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.112+0000 7f04657fa640 1 -- 172.21.15.86:0/656400334 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f046009cbc0 con 0x7f0468106660 2024-04-16T05:46:26.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.291+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0468108630 con 0x7f0468106660 2024-04-16T05:46:26.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.293+0000 7f04657fa640 1 -- 172.21.15.86:0/656400334 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+25263 (secure 0 0 0) 0x7f046009c4c0 con 0x7f0468106660 2024-04-16T05:46:26.291 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-16T05:46:26.292 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":105,"fsid":"6a4a23f8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:54.956200+0000","modified":"2024-04-16T05:46:15.013279+0000","last_up_change":"2024-04-16T05:32:05.109661+0000","last_in_change":"2024-04-16T05:31:44.704355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:28:15.487997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:52.794474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:53.806139+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:57.504528+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"249ee95d-f444-4c1b-aeae-e88da2c920fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6803","nonce":571788251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6805","nonce":571788251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6809","nonce":571788251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":571788251},{"type":"v1","addr":"172.21.15.86:6807","nonce":571788251}]},"public_addr":"172.21.15.86:6803/571788251","cluster_addr":"172.21.15.86:6805/571788251","heartbeat_back_addr":"172.21.15.86:6809/571788251","heartbeat_front_addr":"172.21.15.86:6807/571788251","state":["exists","up"]},{"osd":1,"uuid":"dfd22f89-9efb-4095-82f4-20ceb80af165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6801","nonce":2032185507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6803","nonce":2032185507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6807","nonce":2032185507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":2032185507},{"type":"v1","addr":"172.21.15.194:6805","nonce":2032185507}]},"public_addr":"172.21.15.194:6801/2032185507","cluster_addr":"172.21.15.194:6803/2032185507","heartbeat_back_addr":"172.21.15.194:6807/2032185507","heartbeat_front_addr":"172.21.15.194:6805/2032185507","state":["exists","up"]},{"osd":2,"uuid":"913b98ff-30fe-4214-877a-7a8cc4765ad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6801","nonce":4091288332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6803","nonce":4091288332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6807","nonce":4091288332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4091288332},{"type":"v1","addr":"172.21.15.196:6805","nonce":4091288332}]},"public_addr":"172.21.15.196:6801/4091288332","cluster_addr":"172.21.15.196:6803/4091288332","heartbeat_back_addr":"172.21.15.196:6807/4091288332","heartbeat_front_addr":"172.21.15.196:6805/4091288332","state":["exists","up"]},{"osd":3,"uuid":"63b377ce-0b9e-48a5-85c1-6903db7c60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6811","nonce":439908484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6813","nonce":439908484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6817","nonce":439908484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":439908484},{"type":"v1","addr":"172.21.15.86:6815","nonce":439908484}]},"public_addr":"172.21.15.86:6811/439908484","cluster_addr":"172.21.15.86:6813/439908484","heartbeat_back_addr":"172.21.15.86:6817/439908484","heartbeat_front_addr":"172.21.15.86:6815/439908484","state":["exists","up"]},{"osd":4,"uuid":"48c2fbf7-a6c1-4405-92e9-494e58e34298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6809","nonce":3924919713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6811","nonce":3924919713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6815","nonce":3924919713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3924919713},{"type":"v1","addr":"172.21.15.194:6813","nonce":3924919713}]},"public_addr":"172.21.15.194:6809/3924919713","cluster_addr":"172.21.15.194:6811/3924919713","heartbeat_back_addr":"172.21.15.194:6815/3924919713","heartbeat_front_addr":"172.21.15.194:6813/3924919713","state":["exists","up"]},{"osd":5,"uuid":"4e22bf80-acb0-4744-b7f5-94ae9032b5d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6809","nonce":90234472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6811","nonce":90234472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6815","nonce":90234472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":90234472},{"type":"v1","addr":"172.21.15.196:6813","nonce":90234472}]},"public_addr":"172.21.15.196:6809/90234472","cluster_addr":"172.21.15.196:6811/90234472","heartbeat_back_addr":"172.21.15.196:6815/90234472","heartbeat_front_addr":"172.21.15.196:6813/90234472","state":["exists","up"]},{"osd":6,"uuid":"5af57df8-4de1-4092-ae4e-9fb9a9e8b60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6819","nonce":983213207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6821","nonce":983213207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6825","nonce":983213207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":983213207},{"type":"v1","addr":"172.21.15.86:6823","nonce":983213207}]},"public_addr":"172.21.15.86:6819/983213207","cluster_addr":"172.21.15.86:6821/983213207","heartbeat_back_addr":"172.21.15.86:6825/983213207","heartbeat_front_addr":"172.21.15.86:6823/983213207","state":["exists","up"]},{"osd":7,"uuid":"b5a76013-993e-4ff1-976d-002d09209edf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6817","nonce":1878006717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6819","nonce":1878006717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6823","nonce":1878006717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1878006717},{"type":"v1","addr":"172.21.15.194:6821","nonce":1878006717}]},"public_addr":"172.21.15.194:6817/1878006717","cluster_addr":"172.21.15.194:6819/1878006717","heartbeat_back_addr":"172.21.15.194:6823/1878006717","heartbeat_front_addr":"172.21.15.194:6821/1878006717","state":["exists","up"]},{"osd":8,"uuid":"4b3010ed-edc0-471d-b44b-e700733fe43b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6817","nonce":3684108941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6819","nonce":3684108941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6823","nonce":3684108941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3684108941},{"type":"v1","addr":"172.21.15.196:6821","nonce":3684108941}]},"public_addr":"172.21.15.196:6817/3684108941","cluster_addr":"172.21.15.196:6819/3684108941","heartbeat_back_addr":"172.21.15.196:6823/3684108941","heartbeat_front_addr":"172.21.15.196:6821/3684108941","state":["exists","up"]},{"osd":9,"uuid":"70f95451-1619-47ff-b12f-b8651c020a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6827","nonce":423267579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6829","nonce":423267579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6833","nonce":423267579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":423267579},{"type":"v1","addr":"172.21.15.86:6831","nonce":423267579}]},"public_addr":"172.21.15.86:6827/423267579","cluster_addr":"172.21.15.86:6829/423267579","heartbeat_back_addr":"172.21.15.86:6833/423267579","heartbeat_front_addr":"172.21.15.86:6831/423267579","state":["exists","up"]},{"osd":10,"uuid":"d78b9e33-ca3b-4e0c-9f1d-3e51cf32034d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6825","nonce":656449752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6827","nonce":656449752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6831","nonce":656449752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":656449752},{"type":"v1","addr":"172.21.15.194:6829","nonce":656449752}]},"public_addr":"172.21.15.194:6825/656449752","cluster_addr":"172.21.15.194:6827/656449752","heartbeat_back_addr":"172.21.15.194:6831/656449752","heartbeat_front_addr":"172.21.15.194:6829/656449752","state":["exists","up"]},{"osd":11,"uuid":"66ee3110-b716-48c3-b625-d3a5f40beb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6825","nonce":2166568999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6827","nonce":2166568999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6831","nonce":2166568999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2166568999},{"type":"v1","addr":"172.21.15.196:6829","nonce":2166568999}]},"public_addr":"172.21.15.196:6825/2166568999","cluster_addr":"172.21.15.196:6827/2166568999","heartbeat_back_addr":"172.21.15.196:6831/2166568999","heartbeat_front_addr":"172.21.15.196:6829/2166568999","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:25.442230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:48.867878+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:12.988372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:37.966412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:03.169621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:27.923969+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:53.810624+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:19.320066+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:44.869380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:11.188797+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:37.026730+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:32:03.106349+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2330974702":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/3900523320":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2641972995":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/501397249":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/1016266433":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/908662426":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6801/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:0/3914995291":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/3194681946":"2024-04-17T05:35:16.929800+0000","172.21.15.86:0/2966713555":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/3037484112":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6800/1420225001":"2024-04-17T05:25:51.394705+0000","172.21.15.86:6800/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:6801/1904754808":"2024-04-17T05:25:24.654093+0000","172.21.15.86:0/661459728":"2024-04-17T05:25:51.394705+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:46:26.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f04500a3d20 msgr2=0x7f04500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f04500a3d20 0x7f04500a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0458005ab0 tx=0x7f0458005fd0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 msgr2=0x7f046819afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 0x7f046819afb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0460009fd0 tx=0x7f046000b260 comp rx=0 tx=0).stop 2024-04-16T05:46:26.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 shutdown_connections 2024-04-16T05:46:26.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0468106660 0x7f046819afb0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0468101060 0x7f0468196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f04500a3d20 0x7f04500a61e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 --2- 172.21.15.86:0/656400334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04681006a0 0x7f04681966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 >> 172.21.15.86:0/656400334 conn(0x7f0468077530 msgr2=0x7f04681035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:26.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 shutdown_connections 2024-04-16T05:46:26.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.295+0000 7f046e814640 1 -- 172.21.15.86:0/656400334 wait complete. 2024-04-16T05:46:26.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-mon[32028]: pgmap v347: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:46:26.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1261772395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:26.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/1771272128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:26.305 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-16T05:46:26.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.411+0000 7f49ecd1c640 1 -- 172.21.15.86:0/1892141039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8107000 msgr2=0x7f49e81073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.411+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/1892141039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8107000 0x7f49e81073e0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009930 tx=0x7f49d802bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:26.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.411+0000 7f49ecd1c640 1 -- 172.21.15.86:0/1892141039 shutdown_connections 2024-04-16T05:46:26.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.411+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/1892141039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49e8100f30 0x7f49e810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.411+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/1892141039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e8100570 0x7f49e81009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.411+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/1892141039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8107000 0x7f49e81073e0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.411+0000 7f49ecd1c640 1 -- 172.21.15.86:0/1892141039 >> 172.21.15.86:0/1892141039 conn(0x7f49e8077500 msgr2=0x7f49e80fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:26.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.411+0000 7f49ecd1c640 1 -- 172.21.15.86:0/1892141039 shutdown_connections 2024-04-16T05:46:26.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.411+0000 7f49ecd1c640 1 -- 172.21.15.86:0/1892141039 wait complete. 2024-04-16T05:46:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49ecd1c640 1 Processor -- start 2024-04-16T05:46:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49ecd1c640 1 -- start start 2024-04-16T05:46:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49ecd1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8100570 0x7f49e819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49ecd1c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e8100f30 0x7f49e819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49ecd1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49e8107000 0x7f49e81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49ecd1c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8112480 con 0x7f49e8100570 2024-04-16T05:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49ecd1c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8112300 con 0x7f49e8100f30 2024-04-16T05:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49ecd1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8112600 con 0x7f49e8107000 2024-04-16T05:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8100570 0x7f49e819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8100570 0x7f49e819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42786/0 (socket says 172.21.15.86:42786) 2024-04-16T05:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49e659c640 1 -- 172.21.15.86:0/3999259190 learned_addr learned my addr 172.21.15.86:0/3999259190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49e659c640 1 -- 172.21.15.86:0/3999259190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49e8107000 msgr2=0x7f49e81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49e659c640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49e8107000 0x7f49e81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49e659c640 1 -- 172.21.15.86:0/3999259190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e8100f30 msgr2=0x7f49e819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49e659c640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e8100f30 0x7f49e819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.412+0000 7f49e659c640 1 -- 172.21.15.86:0/3999259190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e8198f90 con 0x7f49e8100570 2024-04-16T05:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.413+0000 7f49e659c640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8100570 0x7f49e819e650 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009230 tx=0x7f49d80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.413+0000 7f49dffff640 1 -- 172.21.15.86:0/3999259190 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8041070 con 0x7f49e8100570 2024-04-16T05:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.413+0000 7f49dffff640 1 -- 172.21.15.86:0/3999259190 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d802ccc0 con 0x7f49e8100570 2024-04-16T05:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.413+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e8199220 con 0x7f49e8100570 2024-04-16T05:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.413+0000 7f49dffff640 1 -- 172.21.15.86:0/3999259190 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8003e20 con 0x7f49e8100570 2024-04-16T05:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.413+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e81a54a0 con 0x7f49e8100570 2024-04-16T05:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.414+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e8103890 con 0x7f49e8100570 2024-04-16T05:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.415+0000 7f49dffff640 1 -- 172.21.15.86:0/3999259190 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49d8044070 con 0x7f49e8100570 2024-04-16T05:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.416+0000 7f49dffff640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f49d40a3d20 0x7f49d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.416+0000 7f49ddd9b640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f49d40a3d20 0x7f49d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.416+0000 7f49dffff640 1 -- 172.21.15.86:0/3999259190 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f49d80f26d0 con 0x7f49e8100570 2024-04-16T05:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.416+0000 7f49ddd9b640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f49d40a3d20 0x7f49d40a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49d0005fd0 tx=0x7f49d0005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.420+0000 7f49dffff640 1 -- 172.21.15.86:0/3999259190 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49d80beab0 con 0x7f49e8100570 2024-04-16T05:46:26.484 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-mon[36373]: pgmap v347: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:46:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1261772395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:26.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/1771272128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.637+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f49e802d620 con 0x7f49e8100570 2024-04-16T05:46:26.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.640+0000 7f49dffff640 1 -- 172.21.15.86:0/3999259190 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f49d80be3b0 con 0x7f49e8100570 2024-04-16T05:46:26.638 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-16T05:46:26.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.643+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f49d40a3d20 msgr2=0x7f49d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.643+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f49d40a3d20 0x7f49d40a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49d0005fd0 tx=0x7f49d0005850 comp rx=0 tx=0).stop 2024-04-16T05:46:26.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.643+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8100570 msgr2=0x7f49e819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.643+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8100570 0x7f49e819e650 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009230 tx=0x7f49d80043b0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.644+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 shutdown_connections 2024-04-16T05:46:26.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.644+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49e8107000 0x7f49e81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.644+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f49d40a3d20 0x7f49d40a61e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.644+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e8100f30 0x7f49e819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.644+0000 7f49ecd1c640 1 --2- 172.21.15.86:0/3999259190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e8100570 0x7f49e819e650 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.644+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 >> 172.21.15.86:0/3999259190 conn(0x7f49e8077500 msgr2=0x7f49e8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:26.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.644+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 shutdown_connections 2024-04-16T05:46:26.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.644+0000 7f49ecd1c640 1 -- 172.21.15.86:0/3999259190 wait complete. 2024-04-16T05:46:26.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-mon[35865]: pgmap v347: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-16T05:46:26.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1261772395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:26.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/1771272128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:26.658 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-16T05:46:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 -- 172.21.15.86:0/2409396743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101af0 msgr2=0x7f846c105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 --2- 172.21.15.86:0/2409396743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101af0 0x7f846c105b30 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f845c009900 tx=0x7f845c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:26.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 -- 172.21.15.86:0/2409396743 shutdown_connections 2024-04-16T05:46:26.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 --2- 172.21.15.86:0/2409396743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f846c106210 0x7f846c10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 --2- 172.21.15.86:0/2409396743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101af0 0x7f846c105b30 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 --2- 172.21.15.86:0/2409396743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c101140 0x7f846c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 -- 172.21.15.86:0/2409396743 >> 172.21.15.86:0/2409396743 conn(0x7f846c078000 msgr2=0x7f846c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:26.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 -- 172.21.15.86:0/2409396743 shutdown_connections 2024-04-16T05:46:26.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 -- 172.21.15.86:0/2409396743 wait complete. 2024-04-16T05:46:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 Processor -- start 2024-04-16T05:46:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.769+0000 7f8473938640 1 -- start start 2024-04-16T05:46:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f8473938640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101140 0x7f846c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f8473938640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f846c106210 0x7f846c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f8473938640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c19a210 0x7f846c1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f8473938640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1112a0 con 0x7f846c101140 2024-04-16T05:46:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f8473938640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f846c111120 con 0x7f846c106210 2024-04-16T05:46:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f8473938640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f846c111420 con 0x7f846c19a210 2024-04-16T05:46:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84716d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101140 0x7f846c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84716d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101140 0x7f846c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42850/0 (socket says 172.21.15.86:42850) 2024-04-16T05:46:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84716d4640 1 -- 172.21.15.86:0/2149505319 learned_addr learned my addr 172.21.15.86:0/2149505319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f8470ed3640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f846c106210 0x7f846c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84716d4640 1 -- 172.21.15.86:0/2149505319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c19a210 msgr2=0x7f846c1a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84716d4640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c19a210 0x7f846c1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84716d4640 1 -- 172.21.15.86:0/2149505319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f846c106210 msgr2=0x7f846c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84716d4640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f846c106210 0x7f846c199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84716d4640 1 -- 172.21.15.86:0/2149505319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c1a4fc0 con 0x7f846c101140 2024-04-16T05:46:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84716d4640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101140 0x7f846c199740 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f846800e9f0 tx=0x7f846800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84627fc640 1 -- 172.21.15.86:0/2149505319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846800cd80 con 0x7f846c101140 2024-04-16T05:46:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84627fc640 1 -- 172.21.15.86:0/2149505319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8468005500 con 0x7f846c101140 2024-04-16T05:46:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c1a52b0 con 0x7f846c101140 2024-04-16T05:46:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f84627fc640 1 -- 172.21.15.86:0/2149505319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468010690 con 0x7f846c101140 2024-04-16T05:46:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.770+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1a5780 con 0x7f846c101140 2024-04-16T05:46:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.772+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c068e20 con 0x7f846c101140 2024-04-16T05:46:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.773+0000 7f84627fc640 1 -- 172.21.15.86:0/2149505319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84680046c0 con 0x7f846c101140 2024-04-16T05:46:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.773+0000 7f84627fc640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84580a3d20 0x7f84580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.773+0000 7f8470ed3640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84580a3d20 0x7f84580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.773+0000 7f84627fc640 1 -- 172.21.15.86:0/2149505319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9838+0+0 (secure 0 0 0) 0x7f84680d11e0 con 0x7f846c101140 2024-04-16T05:46:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.774+0000 7f8470ed3640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84580a3d20 0x7f84580a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f845c009230 tx=0x7f845c005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:26.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.778+0000 7f84627fc640 1 -- 172.21.15.86:0/2149505319 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f846809d360 con 0x7f846c101140 2024-04-16T05:46:26.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a[32024]: 2024-04-16T05:46:26.638+0000 7f473163f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T05:46:26.968 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g[70802]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:26.968 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g[70802]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:26.968 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g[70802]: starting mds.g at 2024-04-16T05:46:26.968 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a[71159]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:26.968 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a[71159]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:26.968 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-mds[71193]: -- 172.21.15.86:0/476311433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559384b9a40 con 0x55593848c800 2024-04-16T05:46:26.969 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:46:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a[71159]: starting mds.a at 2024-04-16T05:46:26.984 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f[66419]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:26.985 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f[66419]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:26.985 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-mds[66423]: -- 172.21.15.196:0/4098575028 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56323c235a40 con 0x56323c208800 2024-04-16T05:46:26.985 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f[66419]: starting mds.f at 2024-04-16T05:46:26.985 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i[67032]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:26.985 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i[67032]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:26.985 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-mds[67036]: -- 172.21.15.196:0/1989239281 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e49711a40 con 0x555e496e5000 2024-04-16T05:46:26.986 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i[67032]: starting mds.i at 2024-04-16T05:46:26.986 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c[66724]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:26.986 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c[66724]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:26.986 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-mds[66728]: -- 172.21.15.196:0/2326279317 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7d7585a40 con 0x55c7d7558800 2024-04-16T05:46:26.986 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c[66724]: starting mds.c at 2024-04-16T05:46:26.987 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l[66113]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:26.987 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l[66113]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:26.987 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-mds[66117]: -- 172.21.15.196:0/3966017049 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e375d7a40 con 0x555e375aac00 2024-04-16T05:46:26.987 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:46:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l[66113]: starting mds.l at 2024-04-16T05:46:26.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:26.996+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f846c101af0 con 0x7f846c101140 2024-04-16T05:46:27.155 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e[66340]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:27.155 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e[66340]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:27.156 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-mds[66344]: -- 172.21.15.194:0/637922514 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ef9c89a40 con 0x564ef9c5c800 2024-04-16T05:46:27.156 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e[66340]: starting mds.e at 2024-04-16T05:46:27.156 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k[66019]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:27.156 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k[66019]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:27.156 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-mds[66023]: -- 172.21.15.194:0/1852250598 <== mon.2 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3b4cc5a40 con 0x55d3b4c98c00 2024-04-16T05:46:27.156 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k[66019]: starting mds.k at 2024-04-16T05:46:27.157 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h[66663]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:27.157 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h[66663]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:27.157 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-mds[66667]: -- 172.21.15.194:0/394968153 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec45b4ba40 con 0x55ec45b1ec00 2024-04-16T05:46:27.157 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h[66663]: starting mds.h at 2024-04-16T05:46:27.157 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b[66970]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:27.157 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b[66970]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:27.158 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-mds[66974]: -- 172.21.15.194:0/778777908 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea30d97a40 con 0x55ea30d6ac00 2024-04-16T05:46:27.158 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:46:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b[66970]: starting mds.b at 2024-04-16T05:46:27.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:27 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/656400334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:46:27.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:27 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3999259190' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T05:46:27.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:27 smithi086 ceph-mon[32028]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T05:46:27.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:27 smithi086 ceph-mon[32028]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T05:46:27.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:27 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3999259190' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T05:46:27.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:27 smithi086 ceph-mon[32028]: osdmap e106: 12 total, 12 up, 12 in 2024-04-16T05:46:27.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:27 smithi086 ceph-mon[32028]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-16T05:46:27.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:27 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2149505319' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:46:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:27 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/656400334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:46:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:27 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3999259190' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T05:46:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:27 smithi196 ceph-mon[36373]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T05:46:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:27 smithi196 ceph-mon[36373]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T05:46:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:27 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3999259190' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T05:46:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:27 smithi196 ceph-mon[36373]: osdmap e106: 12 total, 12 up, 12 in 2024-04-16T05:46:27.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:27 smithi196 ceph-mon[36373]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-16T05:46:27.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:27 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2149505319' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:46:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.646+0000 7f84627fc640 1 -- 172.21.15.86:0/2149505319 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f84680bbe20 con 0x7f846c101140 2024-04-16T05:46:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84580a3d20 msgr2=0x7f84580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84580a3d20 0x7f84580a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f845c009230 tx=0x7f845c005fd0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101140 msgr2=0x7f846c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101140 0x7f846c199740 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f846800e9f0 tx=0x7f846800edf0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 shutdown_connections 2024-04-16T05:46:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c19a210 0x7f846c1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f84580a3d20 0x7f84580a61e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f846c106210 0x7f846c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 --2- 172.21.15.86:0/2149505319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846c101140 0x7f846c199740 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 >> 172.21.15.86:0/2149505319 conn(0x7f846c078000 msgr2=0x7f846c0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 shutdown_connections 2024-04-16T05:46:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.649+0000 7f8473938640 1 -- 172.21.15.86:0/2149505319 wait complete. 2024-04-16T05:46:27.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:27 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/656400334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:46:27.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:27 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3999259190' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T05:46:27.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:27 smithi194 ceph-mon[35865]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T05:46:27.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:27 smithi194 ceph-mon[35865]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T05:46:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:27 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3999259190' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T05:46:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:27 smithi194 ceph-mon[35865]: osdmap e106: 12 total, 12 up, 12 in 2024-04-16T05:46:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:27 smithi194 ceph-mon[35865]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-16T05:46:27.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:27 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2149505319' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:46:27.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-16T05:46:27.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 -- 172.21.15.86:0/3530545928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c105ec0 msgr2=0x7fc43c1062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:27.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 --2- 172.21.15.86:0/3530545928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c105ec0 0x7fc43c1062a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009900 tx=0x7fc42c02bc80 comp rx=0 tx=0).stop 2024-04-16T05:46:27.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 -- 172.21.15.86:0/3530545928 shutdown_connections 2024-04-16T05:46:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 --2- 172.21.15.86:0/3530545928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc43c102000 0x7fc43c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 --2- 172.21.15.86:0/3530545928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43c101660 0x7fc43c101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 --2- 172.21.15.86:0/3530545928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c105ec0 0x7fc43c1062a0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 -- 172.21.15.86:0/3530545928 >> 172.21.15.86:0/3530545928 conn(0x7fc43c0774a0 msgr2=0x7fc43c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 -- 172.21.15.86:0/3530545928 shutdown_connections 2024-04-16T05:46:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 -- 172.21.15.86:0/3530545928 wait complete. 2024-04-16T05:46:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 Processor -- start 2024-04-16T05:46:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.782+0000 7fc442855640 1 -- start start 2024-04-16T05:46:27.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc442855640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43c101660 0x7fc43c0fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc442855640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c102000 0x7fc43c0fce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc442855640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc43c105ec0 0x7fc43c0fd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc442855640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c1113e0 con 0x7fc43c102000 2024-04-16T05:46:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc442855640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c111260 con 0x7fc43c101660 2024-04-16T05:46:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc442855640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c111560 con 0x7fc43c105ec0 2024-04-16T05:46:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc433fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c102000 0x7fc43c0fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc433fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c102000 0x7fc43c0fce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42868/0 (socket says 172.21.15.86:42868) 2024-04-16T05:46:27.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc433fff640 1 -- 172.21.15.86:0/4068943273 learned_addr learned my addr 172.21.15.86:0/4068943273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc43bfff640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43c101660 0x7fc43c0fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc440df2640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc43c105ec0 0x7fc43c0fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc433fff640 1 -- 172.21.15.86:0/4068943273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc43c105ec0 msgr2=0x7fc43c0fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc433fff640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc43c105ec0 0x7fc43c0fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc433fff640 1 -- 172.21.15.86:0/4068943273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43c101660 msgr2=0x7fc43c0fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc433fff640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43c101660 0x7fc43c0fc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc433fff640 1 -- 172.21.15.86:0/4068943273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c1a6ec0 con 0x7fc43c102000 2024-04-16T05:46:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc43bfff640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43c101660 0x7fc43c0fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:46:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc433fff640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c102000 0x7fc43c0fce20 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fc42400c890 tx=0x7fc42400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc439ffb640 1 -- 172.21.15.86:0/4068943273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424004220 con 0x7fc43c102000 2024-04-16T05:46:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc439ffb640 1 -- 172.21.15.86:0/4068943273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4240043c0 con 0x7fc43c102000 2024-04-16T05:46:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43c1a71b0 con 0x7fc43c102000 2024-04-16T05:46:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc439ffb640 1 -- 172.21.15.86:0/4068943273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240049e0 con 0x7fc43c102000 2024-04-16T05:46:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.783+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43c1a7700 con 0x7fc43c102000 2024-04-16T05:46:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.784+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc43c068590 con 0x7fc43c102000 2024-04-16T05:46:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.786+0000 7fc439ffb640 1 -- 172.21.15.86:0/4068943273 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc4240079b0 con 0x7fc43c102000 2024-04-16T05:46:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.786+0000 7fc439ffb640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc4280a3d20 0x7fc4280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:27.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.786+0000 7fc43bfff640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc4280a3d20 0x7fc4280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.786+0000 7fc439ffb640 1 -- 172.21.15.86:0/4068943273 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9838+0+0 (secure 0 0 0) 0x7fc4240d0e60 con 0x7fc43c102000 2024-04-16T05:46:27.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.787+0000 7fc43bfff640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc4280a3d20 0x7fc4280a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc42c005ab0 tx=0x7fc42c005a20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:27.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.795+0000 7fc439ffb640 1 -- 172.21.15.86:0/4068943273 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc424014030 con 0x7fc43c102000 2024-04-16T05:46:27.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:27.981+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc43c0fde10 con 0x7fc43c102000 2024-04-16T05:46:28.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: pgmap v349: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:46:28.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T05:46:28.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T05:46:28.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: Cluster is now healthy 2024-04-16T05:46:28.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/2149505319' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:46:28.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.86:6838/1042569892,v1:172.21.15.86:6839/1042569892] up:boot 2024-04-16T05:46:28.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.86:6840/2219038959,v1:172.21.15.86:6841/2219038959] up:boot 2024-04-16T05:46:28.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6838/2751728886,v1:172.21.15.196:6839/2751728886] up:boot 2024-04-16T05:46:28.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6833/2990061232,v1:172.21.15.194:6835/2990061232] up:boot 2024-04-16T05:46:28.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6832/1876522571,v1:172.21.15.196:6834/1876522571] up:boot 2024-04-16T05:46:28.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6833/3336725703,v1:172.21.15.196:6835/3336725703] up:boot 2024-04-16T05:46:28.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6836/1188961063,v1:172.21.15.194:6837/1188961063] up:boot 2024-04-16T05:46:28.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6838/4135769329,v1:172.21.15.194:6839/4135769329] up:boot 2024-04-16T05:46:28.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.194:6832/3396252478,v1:172.21.15.194:6834/3396252478] up:boot 2024-04-16T05:46:28.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: mds.? [v2:172.21.15.196:6836/3280527708,v1:172.21.15.196:6837/3280527708] up:boot 2024-04-16T05:46:28.469 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: fsmap 12 up:standby 2024-04-16T05:46:28.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:46:28.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:46:28.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:46:28.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:46:28.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:46:28.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:46:28.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:46:28.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:46:28.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:46:28.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:46:28.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:28 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4068943273' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: pgmap v349: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:46:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T05:46:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T05:46:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: Cluster is now healthy 2024-04-16T05:46:28.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/2149505319' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:46:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.86:6838/1042569892,v1:172.21.15.86:6839/1042569892] up:boot 2024-04-16T05:46:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.86:6840/2219038959,v1:172.21.15.86:6841/2219038959] up:boot 2024-04-16T05:46:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6838/2751728886,v1:172.21.15.196:6839/2751728886] up:boot 2024-04-16T05:46:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6833/2990061232,v1:172.21.15.194:6835/2990061232] up:boot 2024-04-16T05:46:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6832/1876522571,v1:172.21.15.196:6834/1876522571] up:boot 2024-04-16T05:46:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6833/3336725703,v1:172.21.15.196:6835/3336725703] up:boot 2024-04-16T05:46:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6836/1188961063,v1:172.21.15.194:6837/1188961063] up:boot 2024-04-16T05:46:28.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6838/4135769329,v1:172.21.15.194:6839/4135769329] up:boot 2024-04-16T05:46:28.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.194:6832/3396252478,v1:172.21.15.194:6834/3396252478] up:boot 2024-04-16T05:46:28.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: mds.? [v2:172.21.15.196:6836/3280527708,v1:172.21.15.196:6837/3280527708] up:boot 2024-04-16T05:46:28.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: fsmap 12 up:standby 2024-04-16T05:46:28.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:46:28.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:46:28.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:46:28.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:46:28.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:46:28.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:46:28.488 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:46:28.488 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:46:28.488 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:46:28.488 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:46:28.488 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:28 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4068943273' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.647+0000 7fc439ffb640 1 -- 172.21.15.86:0/4068943273 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) ==== 181+0+0 (secure 0 0 0) 0x7fc43c0fde10 con 0x7fc43c102000 2024-04-16T05:46:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-16T05:46:28.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc4280a3d20 msgr2=0x7fc4280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:28.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc4280a3d20 0x7fc4280a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc42c005ab0 tx=0x7fc42c005a20 comp rx=0 tx=0).stop 2024-04-16T05:46:28.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c102000 msgr2=0x7fc43c0fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:28.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c102000 0x7fc43c0fce20 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fc42400c890 tx=0x7fc42400cd50 comp rx=0 tx=0).stop 2024-04-16T05:46:28.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 shutdown_connections 2024-04-16T05:46:28.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc43c105ec0 0x7fc43c0fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:28.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc43c102000 0x7fc43c0fce20 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:28.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fc4280a3d20 0x7fc4280a61e0 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc42c005ab0 tx=0x7fc42c005a20 comp rx=0 tx=0).stop 2024-04-16T05:46:28.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 --2- 172.21.15.86:0/4068943273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43c101660 0x7fc43c0fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:28.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.650+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 >> 172.21.15.86:0/4068943273 conn(0x7fc43c0774a0 msgr2=0x7fc43c1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:28.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.651+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 shutdown_connections 2024-04-16T05:46:28.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.651+0000 7fc442855640 1 -- 172.21.15.86:0/4068943273 wait complete. 2024-04-16T05:46:28.655 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: pgmap v349: 129 pgs: 129 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-16T05:46:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T05:46:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T05:46:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: Cluster is now healthy 2024-04-16T05:46:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/2149505319' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:46:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.86:6838/1042569892,v1:172.21.15.86:6839/1042569892] up:boot 2024-04-16T05:46:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.86:6840/2219038959,v1:172.21.15.86:6841/2219038959] up:boot 2024-04-16T05:46:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6838/2751728886,v1:172.21.15.196:6839/2751728886] up:boot 2024-04-16T05:46:28.656 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6833/2990061232,v1:172.21.15.194:6835/2990061232] up:boot 2024-04-16T05:46:28.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6832/1876522571,v1:172.21.15.196:6834/1876522571] up:boot 2024-04-16T05:46:28.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6833/3336725703,v1:172.21.15.196:6835/3336725703] up:boot 2024-04-16T05:46:28.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6836/1188961063,v1:172.21.15.194:6837/1188961063] up:boot 2024-04-16T05:46:28.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6838/4135769329,v1:172.21.15.194:6839/4135769329] up:boot 2024-04-16T05:46:28.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.194:6832/3396252478,v1:172.21.15.194:6834/3396252478] up:boot 2024-04-16T05:46:28.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: mds.? [v2:172.21.15.196:6836/3280527708,v1:172.21.15.196:6837/3280527708] up:boot 2024-04-16T05:46:28.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: fsmap 12 up:standby 2024-04-16T05:46:28.657 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:46:28.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:46:28.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:46:28.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:46:28.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:46:28.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:46:28.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:46:28.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:46:28.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:46:28.658 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='mgr.15528 172.21.15.86:0/1370567683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:46:28.659 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:28 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4068943273' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:28.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-16T05:46:28.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.776+0000 7fd5399cd640 1 -- 172.21.15.86:0/2721436497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd534101bf0 msgr2=0x7fd53410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:28.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.776+0000 7fd5399cd640 1 --2- 172.21.15.86:0/2721436497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd534101bf0 0x7fd53410dd30 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009cd0 tx=0x7fd52c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:46:28.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.776+0000 7fd5399cd640 1 -- 172.21.15.86:0/2721436497 shutdown_connections 2024-04-16T05:46:28.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.776+0000 7fd5399cd640 1 --2- 172.21.15.86:0/2721436497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd534101bf0 0x7fd53410dd30 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:28.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.776+0000 7fd5399cd640 1 --2- 172.21.15.86:0/2721436497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd534101250 0x7fd5341016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:28.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.776+0000 7fd5399cd640 1 --2- 172.21.15.86:0/2721436497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5341070e0 0x7fd5341074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:28.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.776+0000 7fd5399cd640 1 -- 172.21.15.86:0/2721436497 >> 172.21.15.86:0/2721436497 conn(0x7fd534076d00 msgr2=0x7fd534077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:28.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.776+0000 7fd5399cd640 1 -- 172.21.15.86:0/2721436497 shutdown_connections 2024-04-16T05:46:28.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.776+0000 7fd5399cd640 1 -- 172.21.15.86:0/2721436497 wait complete. 2024-04-16T05:46:28.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5399cd640 1 Processor -- start 2024-04-16T05:46:28.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5399cd640 1 -- start start 2024-04-16T05:46:28.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5399cd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd534101250 0x7fd53419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:28.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5399cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd534101bf0 0x7fd53419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:28.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5399cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5341070e0 0x7fd534198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:28.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5399cd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd534112460 con 0x7fd5341070e0 2024-04-16T05:46:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5399cd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd5341122e0 con 0x7fd534101250 2024-04-16T05:46:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5399cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5341125e0 con 0x7fd534101bf0 2024-04-16T05:46:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5341070e0 0x7fd534198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5341070e0 0x7fd534198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42890/0 (socket says 172.21.15.86:42890) 2024-04-16T05:46:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5337fe640 1 -- 172.21.15.86:0/3420843437 learned_addr learned my addr 172.21.15.86:0/3420843437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5327fc640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd534101bf0 0x7fd53419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd532ffd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd534101250 0x7fd53419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5337fe640 1 -- 172.21.15.86:0/3420843437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd534101bf0 msgr2=0x7fd53419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5337fe640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd534101bf0 0x7fd53419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5337fe640 1 -- 172.21.15.86:0/3420843437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd534101250 msgr2=0x7fd53419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5337fe640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd534101250 0x7fd53419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.777+0000 7fd5337fe640 1 -- 172.21.15.86:0/3420843437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd534198d80 con 0x7fd5341070e0 2024-04-16T05:46:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.778+0000 7fd532ffd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd534101250 0x7fd53419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:46:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.778+0000 7fd5337fe640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5341070e0 0x7fd534198600 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0348f0 tx=0x7fd52c032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.778+0000 7fd5389cb640 1 -- 172.21.15.86:0/3420843437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c041020 con 0x7fd5341070e0 2024-04-16T05:46:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.778+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd534199010 con 0x7fd5341070e0 2024-04-16T05:46:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.778+0000 7fd5389cb640 1 -- 172.21.15.86:0/3420843437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c034cd0 con 0x7fd5341070e0 2024-04-16T05:46:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.778+0000 7fd5389cb640 1 -- 172.21.15.86:0/3420843437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c039040 con 0x7fd5341070e0 2024-04-16T05:46:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.778+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5341a5310 con 0x7fd5341070e0 2024-04-16T05:46:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.779+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd534102640 con 0x7fd5341070e0 2024-04-16T05:46:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.780+0000 7fd5389cb640 1 -- 172.21.15.86:0/3420843437 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd52c044020 con 0x7fd5341070e0 2024-04-16T05:46:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.780+0000 7fd5389cb640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd51c0a3d20 0x7fd51c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.781+0000 7fd532ffd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd51c0a3d20 0x7fd51c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.781+0000 7fd5389cb640 1 -- 172.21.15.86:0/3420843437 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9342+0+0 (secure 0 0 0) 0x7fd52c0f2990 con 0x7fd5341070e0 2024-04-16T05:46:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.781+0000 7fd532ffd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd51c0a3d20 0x7fd51c0a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd524009b40 tx=0x7fd524005e50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.785+0000 7fd5389cb640 1 -- 172.21.15.86:0/3420843437 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd52c04b020 con 0x7fd5341070e0 2024-04-16T05:46:28.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:28.970+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd5341a5540 con 0x7fd5341070e0 2024-04-16T05:46:29.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.653+0000 7fd5389cb640 1 -- 172.21.15.86:0/3420843437 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7fd52c0bee10 con 0x7fd5341070e0 2024-04-16T05:46:29.651 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-16T05:46:29.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.655+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd51c0a3d20 msgr2=0x7fd51c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:29.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.655+0000 7fd5399cd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd51c0a3d20 0x7fd51c0a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd524009b40 tx=0x7fd524005e50 comp rx=0 tx=0).stop 2024-04-16T05:46:29.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5341070e0 msgr2=0x7fd534198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5341070e0 0x7fd534198600 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0348f0 tx=0x7fd52c032ec0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 shutdown_connections 2024-04-16T05:46:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5341070e0 0x7fd534198600 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd534101bf0 0x7fd53419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7fd51c0a3d20 0x7fd51c0a61e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 --2- 172.21.15.86:0/3420843437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd534101250 0x7fd53419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 >> 172.21.15.86:0/3420843437 conn(0x7fd534076d00 msgr2=0x7fd5340ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:29.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 shutdown_connections 2024-04-16T05:46:29.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.656+0000 7fd5399cd640 1 -- 172.21.15.86:0/3420843437 wait complete. 2024-04-16T05:46:29.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-16T05:46:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.781+0000 7f44b8e93640 1 -- 172.21.15.86:0/1801197970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101af0 msgr2=0x7f44b4105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.781+0000 7f44b8e93640 1 --2- 172.21.15.86:0/1801197970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101af0 0x7f44b4105b30 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f449c009900 tx=0x7f449c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.781+0000 7f44b8e93640 1 -- 172.21.15.86:0/1801197970 shutdown_connections 2024-04-16T05:46:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.781+0000 7f44b8e93640 1 --2- 172.21.15.86:0/1801197970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b4106210 0x7f44b410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.781+0000 7f44b8e93640 1 --2- 172.21.15.86:0/1801197970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101af0 0x7f44b4105b30 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.781+0000 7f44b8e93640 1 --2- 172.21.15.86:0/1801197970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b4101140 0x7f44b4101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.781+0000 7f44b8e93640 1 -- 172.21.15.86:0/1801197970 >> 172.21.15.86:0/1801197970 conn(0x7f44b4078000 msgr2=0x7f44b4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.781+0000 7f44b8e93640 1 -- 172.21.15.86:0/1801197970 shutdown_connections 2024-04-16T05:46:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.781+0000 7f44b8e93640 1 -- 172.21.15.86:0/1801197970 wait complete. 2024-04-16T05:46:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b8e93640 1 Processor -- start 2024-04-16T05:46:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b8e93640 1 -- start start 2024-04-16T05:46:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b8e93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101140 0x7f44b419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b8e93640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b4106210 0x7f44b419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b8e93640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b4198720 0x7f44b4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b8e93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41112a0 con 0x7f44b4101140 2024-04-16T05:46:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b8e93640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4111120 con 0x7f44b4106210 2024-04-16T05:46:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b8e93640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4111420 con 0x7f44b4198720 2024-04-16T05:46:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101140 0x7f44b419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101140 0x7f44b419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42908/0 (socket says 172.21.15.86:42908) 2024-04-16T05:46:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b259c640 1 -- 172.21.15.86:0/316176708 learned_addr learned my addr 172.21.15.86:0/316176708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-16T05:46:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44abfff640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b4106210 0x7f44b419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b2d9d640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b4198720 0x7f44b4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b259c640 1 -- 172.21.15.86:0/316176708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b4198720 msgr2=0x7f44b4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b259c640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b4198720 0x7f44b4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b259c640 1 -- 172.21.15.86:0/316176708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b4106210 msgr2=0x7f44b419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b259c640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b4106210 0x7f44b419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.782+0000 7f44b259c640 1 -- 172.21.15.86:0/316176708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b4199320 con 0x7f44b4101140 2024-04-16T05:46:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.783+0000 7f44b259c640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101140 0x7f44b419e660 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f44a400e9f0 tx=0x7f44a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.783+0000 7f44ab7fe640 1 -- 172.21.15.86:0/316176708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a400cd80 con 0x7f44b4101140 2024-04-16T05:46:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.783+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b41a5400 con 0x7f44b4101140 2024-04-16T05:46:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.783+0000 7f44ab7fe640 1 -- 172.21.15.86:0/316176708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a4005500 con 0x7f44b4101140 2024-04-16T05:46:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.783+0000 7f44ab7fe640 1 -- 172.21.15.86:0/316176708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4010690 con 0x7f44b4101140 2024-04-16T05:46:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.783+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b41a57b0 con 0x7f44b4101140 2024-04-16T05:46:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.784+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b4102610 con 0x7f44b4101140 2024-04-16T05:46:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.785+0000 7f44ab7fe640 1 -- 172.21.15.86:0/316176708 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f44a40046c0 con 0x7f44b4101140 2024-04-16T05:46:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.786+0000 7f44ab7fe640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44a00a3d20 0x7f44a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.786+0000 7f44abfff640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44a00a3d20 0x7f44a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.786+0000 7f44ab7fe640 1 -- 172.21.15.86:0/316176708 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8754+0+0 (secure 0 0 0) 0x7f44a40d0ce0 con 0x7f44b4101140 2024-04-16T05:46:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.786+0000 7f44abfff640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44a00a3d20 0x7f44a00a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f449c009230 tx=0x7f449c005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.790+0000 7f44ab7fe640 1 -- 172.21.15.86:0/316176708 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44a409d290 con 0x7f44b4101140 2024-04-16T05:46:29.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:29 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/4068943273' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:29.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:29 smithi194 ceph-mon[35865]: osdmap e107: 12 total, 12 up, 12 in 2024-04-16T05:46:29.906 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:29 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3420843437' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:29.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:29.968+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f44b410a450 con 0x7f44b4101140 2024-04-16T05:46:29.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:29 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/4068943273' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:29.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:29 smithi086 ceph-mon[32028]: osdmap e107: 12 total, 12 up, 12 in 2024-04-16T05:46:29.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:29 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3420843437' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:29.984 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:29 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/4068943273' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:29.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:29 smithi196 ceph-mon[36373]: osdmap e107: 12 total, 12 up, 12 in 2024-04-16T05:46:29.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:29 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3420843437' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:30.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.662+0000 7f44ab7fe640 1 -- 172.21.15.86:0/316176708 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v109) ==== 178+0+0 (secure 0 0 0) 0x7f44a409cb90 con 0x7f44b4101140 2024-04-16T05:46:30.662 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' removed 2024-04-16T05:46:30.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.665+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44a00a3d20 msgr2=0x7f44a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:30.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.665+0000 7f44b8e93640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44a00a3d20 0x7f44a00a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f449c009230 tx=0x7f449c005fd0 comp rx=0 tx=0).stop 2024-04-16T05:46:30.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.665+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101140 msgr2=0x7f44b419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:30.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.665+0000 7f44b8e93640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101140 0x7f44b419e660 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f44a400e9f0 tx=0x7f44a400edf0 comp rx=0 tx=0).stop 2024-04-16T05:46:30.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.665+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 shutdown_connections 2024-04-16T05:46:30.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.666+0000 7f44b8e93640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44b4198720 0x7f44b4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:30.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.666+0000 7f44b8e93640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:6800/2117030010,v1:172.21.15.86:6801/2117030010] conn(0x7f44a00a3d20 0x7f44a00a61e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:30.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.666+0000 7f44b8e93640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b4106210 0x7f44b419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:30.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.666+0000 7f44b8e93640 1 --2- 172.21.15.86:0/316176708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b4101140 0x7f44b419e660 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:30.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.666+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 >> 172.21.15.86:0/316176708 conn(0x7f44b4078000 msgr2=0x7f44b40695b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:30.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.666+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 shutdown_connections 2024-04-16T05:46:30.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-16T05:46:30.666+0000 7f44b8e93640 1 -- 172.21.15.86:0/316176708 wait complete. 2024-04-16T05:46:30.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:46:30.714 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:46:30.749 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:46:30.786 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-16T05:46:30.786 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T05:46:30.802 DEBUG:teuthology.orchestra.run.smithi194:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T05:46:30.819 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T05:46:30.843 INFO:tasks.cephadm:Stopping all daemons... 2024-04-16T05:46:30.843 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-16T05:46:30.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.a 2024-04-16T05:46:30.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:30 smithi194 ceph-mon[35865]: pgmap v351: 97 pgs: 97 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:46:30.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:30 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/3420843437' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:30.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:30 smithi194 ceph-mon[35865]: osdmap e108: 12 total, 12 up, 12 in 2024-04-16T05:46:30.905 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:30 smithi194 ceph-mon[35865]: from='client.? 172.21.15.86:0/316176708' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:30.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:30 smithi086 ceph-mon[32028]: pgmap v351: 97 pgs: 97 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:46:30.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:30 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/3420843437' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:30.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:30 smithi086 ceph-mon[32028]: osdmap e108: 12 total, 12 up, 12 in 2024-04-16T05:46:30.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:30 smithi086 ceph-mon[32028]: from='client.? 172.21.15.86:0/316176708' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:30.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:30 smithi086 systemd[1]: Stopping Ceph mon.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:30.984 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:30 smithi196 ceph-mon[36373]: pgmap v351: 97 pgs: 97 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:46:30.984 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:30 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/3420843437' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:30 smithi196 ceph-mon[36373]: osdmap e108: 12 total, 12 up, 12 in 2024-04-16T05:46:30.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:30 smithi196 ceph-mon[36373]: from='client.? 172.21.15.86:0/316176708' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:31.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:31 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a[32024]: 2024-04-16T05:46:31.159+0000 7f4736e4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:31.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:31 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a[32024]: 2024-04-16T05:46:31.159+0000 7f4736e4a700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-16T05:46:31.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:31 smithi086 podman[101267]: 2024-04-16 05:46:31.334427178 +0000 UTC m=+0.289365799 container died 29c249e4574d08a74b86554571b36f204251270b0b54e05dc5ac05a644e48123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-16T05:46:31.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:31 smithi086 podman[101267]: 2024-04-16 05:46:31.692752314 +0000 UTC m=+0.647690941 container cleanup 29c249e4574d08a74b86554571b36f204251270b0b54e05dc5ac05a644e48123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-16T05:46:31.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 16 05:46:31 smithi086 bash[101267]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-a 2024-04-16T05:46:32.116 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.a.service' 2024-04-16T05:46:32.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:32.170 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-16T05:46:32.170 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-16T05:46:32.170 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.b 2024-04-16T05:46:32.470 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:32 smithi194 systemd[1]: Stopping Ceph mon.b for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:32.887 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-b[35861]: 2024-04-16T05:46:32.471+0000 7fcf06f63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:32.888 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-b[35861]: 2024-04-16T05:46:32.471+0000 7fcf06f63700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-16T05:46:32.888 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:32 smithi194 podman[72484]: 2024-04-16 05:46:32.625269858 +0000 UTC m=+0.275155901 container died b922bb74c42a4dfdb9840bbb73b6980d01202a0cf7b7297d509abc21342046ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:33.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:32 smithi194 podman[72484]: 2024-04-16 05:46:32.989060538 +0000 UTC m=+0.638946581 container cleanup b922bb74c42a4dfdb9840bbb73b6980d01202a0cf7b7297d509abc21342046ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-16T05:46:33.155 INFO:journalctl@ceph.mon.b.smithi194.stdout:Apr 16 05:46:32 smithi194 bash[72484]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-b 2024-04-16T05:46:33.446 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.b.service' 2024-04-16T05:46:33.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:33.497 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-16T05:46:33.497 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-16T05:46:33.497 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.c 2024-04-16T05:46:33.834 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:33 smithi196 systemd[1]: Stopping Ceph mon.c for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:34.088 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:33 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-c[36369]: 2024-04-16T05:46:33.835+0000 7f921c273700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:34.088 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:33 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-c[36369]: 2024-04-16T05:46:33.835+0000 7f921c273700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-16T05:46:34.392 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:34 smithi196 podman[72403]: 2024-04-16 05:46:34.09050343 +0000 UTC m=+0.397497672 container died 2a64957c51d304e5f4f946e56fd1499e2e453bda7e90c3351a68f87040fc7f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD) 2024-04-16T05:46:34.734 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:34 smithi196 podman[72403]: 2024-04-16 05:46:34.494463933 +0000 UTC m=+0.801458162 container cleanup 2a64957c51d304e5f4f946e56fd1499e2e453bda7e90c3351a68f87040fc7f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:34.735 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 16 05:46:34 smithi196 bash[72403]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mon-c 2024-04-16T05:46:34.913 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mon.c.service' 2024-04-16T05:46:34.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:34.958 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-16T05:46:34.958 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-16T05:46:34.959 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.x 2024-04-16T05:46:35.308 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:46:35 smithi086 systemd[1]: Stopping Ceph mgr.x for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:35.603 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:46:35 smithi086 podman[101477]: 2024-04-16 05:46:35.311464686 +0000 UTC m=+0.156037613 container died 3de8902fe01512238291b91705b3da5a0ca03707565220b688c14cc7ce76d42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-16T05:46:35.967 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:46:35 smithi086 podman[101477]: 2024-04-16 05:46:35.705766639 +0000 UTC m=+0.550339563 container cleanup 3de8902fe01512238291b91705b3da5a0ca03707565220b688c14cc7ce76d42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-16T05:46:35.967 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 16 05:46:35 smithi086 bash[101477]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-x 2024-04-16T05:46:36.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service' 2024-04-16T05:46:36.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:36.169 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-16T05:46:36.169 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-16T05:46:36.169 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.y 2024-04-16T05:46:36.504 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:46:36 smithi194 systemd[1]: Stopping Ceph mgr.y for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:36.771 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:46:36 smithi194 podman[72607]: 2024-04-16 05:46:36.506106586 +0000 UTC m=+0.145407600 container died 28612d49764f63fe771079b87205e4e117e3070b29b116438a80c87b4f71a89f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T05:46:37.156 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:46:36 smithi194 podman[72607]: 2024-04-16 05:46:36.885946063 +0000 UTC m=+0.525247077 container cleanup 28612d49764f63fe771079b87205e4e117e3070b29b116438a80c87b4f71a89f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240409) 2024-04-16T05:46:37.156 INFO:journalctl@ceph.mgr.y.smithi194.stdout:Apr 16 05:46:36 smithi194 bash[72607]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-y 2024-04-16T05:46:37.328 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.y.service' 2024-04-16T05:46:37.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:37.367 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-16T05:46:37.367 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-16T05:46:37.367 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.z 2024-04-16T05:46:37.684 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:46:37 smithi196 systemd[1]: Stopping Ceph mgr.z for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:37.981 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:46:37 smithi196 podman[72524]: 2024-04-16 05:46:37.686454594 +0000 UTC m=+0.125719818 container died d3858bd2fb3031435adfb77641533fcae48a917ac72d3cb9ddd94cd4024d6093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:46:38.234 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:46:38 smithi196 podman[72524]: 2024-04-16 05:46:38.083514763 +0000 UTC m=+0.522779980 container cleanup d3858bd2fb3031435adfb77641533fcae48a917ac72d3cb9ddd94cd4024d6093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-16T05:46:38.235 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 16 05:46:38 smithi196 bash[72524]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mgr-z 2024-04-16T05:46:38.448 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mgr.z.service' 2024-04-16T05:46:38.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:38.485 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-16T05:46:38.485 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-16T05:46:38.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.0 2024-04-16T05:46:38.717 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:38 smithi086 systemd[1]: Stopping Ceph osd.0 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:39.217 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:38 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0[44192]: 2024-04-16T05:46:38.801+0000 7fd24e395700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:39.218 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:38 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0[44192]: 2024-04-16T05:46:38.801+0000 7fd24e395700 -1 osd.0 109 *** Got signal Terminated *** 2024-04-16T05:46:39.218 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:38 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0[44192]: 2024-04-16T05:46:38.801+0000 7fd24e395700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:46:44.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:44 smithi086 podman[101601]: 2024-04-16 05:46:44.265654963 +0000 UTC m=+5.583610871 container died f2d410462f5dfc48d6ce91065123d06f49360f54b3b40eadffe9137738239270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:44.967 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:44 smithi086 podman[101601]: 2024-04-16 05:46:44.718757695 +0000 UTC m=+6.036713608 container cleanup f2d410462f5dfc48d6ce91065123d06f49360f54b3b40eadffe9137738239270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:44.967 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:44 smithi086 bash[101601]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0 2024-04-16T05:46:45.389 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:45 smithi086 podman[101620]: 2024-04-16 05:46:45.036878108 +0000 UTC m=+0.766456563 container remove f2d410462f5dfc48d6ce91065123d06f49360f54b3b40eadffe9137738239270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:45.674 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:45 smithi086 podman[101708]: 2024-04-16 05:46:45.294969489 +0000 UTC m=+0.018745152 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:46:45.703 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:45 smithi086 podman[101708]: 2024-04-16 05:46:45.421508315 +0000 UTC m=+0.145283951 container create 189900668bcb03a90f75bd9278d50c5d876222827fd012e12efb841a0fcbd327 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, ceph=True) 2024-04-16T05:46:45.703 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:45 smithi086 podman[101708]: 2024-04-16 05:46:45.674039734 +0000 UTC m=+0.397815372 container init 189900668bcb03a90f75bd9278d50c5d876222827fd012e12efb841a0fcbd327 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:46:45.967 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:45 smithi086 podman[101708]: 2024-04-16 05:46:45.677825196 +0000 UTC m=+0.401600836 container start 189900668bcb03a90f75bd9278d50c5d876222827fd012e12efb841a0fcbd327 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:46:45.967 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:45 smithi086 podman[101708]: 2024-04-16 05:46:45.801853349 +0000 UTC m=+0.525629010 container attach 189900668bcb03a90f75bd9278d50c5d876222827fd012e12efb841a0fcbd327 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True) 2024-04-16T05:46:46.283 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 16 05:46:45 smithi086 podman[101708]: 2024-04-16 05:46:45.972701922 +0000 UTC m=+0.696477578 container died 189900668bcb03a90f75bd9278d50c5d876222827fd012e12efb841a0fcbd327 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:46.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.0.service' 2024-04-16T05:46:46.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:46.656 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-16T05:46:46.657 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-16T05:46:46.657 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.1 2024-04-16T05:46:46.972 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:46 smithi194 systemd[1]: Stopping Ceph osd.1 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:47.405 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:46 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1[40479]: 2024-04-16T05:46:46.973+0000 7fb2fe624700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:47.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:46 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1[40479]: 2024-04-16T05:46:46.973+0000 7fb2fe624700 -1 osd.1 109 *** Got signal Terminated *** 2024-04-16T05:46:47.406 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:46 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1[40479]: 2024-04-16T05:46:46.973+0000 7fb2fe624700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:46:52.615 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:52 smithi194 podman[72743]: 2024-04-16 05:46:52.324500233 +0000 UTC m=+5.473698988 container died 27921751cb2c9162268ef6643c79b6b890a3019167b77d032cf2e5fe5cfbd737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-16T05:46:52.905 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:52 smithi194 podman[72743]: 2024-04-16 05:46:52.698851769 +0000 UTC m=+5.848050526 container cleanup 27921751cb2c9162268ef6643c79b6b890a3019167b77d032cf2e5fe5cfbd737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:52.906 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:52 smithi194 bash[72743]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1 2024-04-16T05:46:53.307 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:52 smithi194 podman[72760]: 2024-04-16 05:46:52.97520204 +0000 UTC m=+0.648901714 container remove 27921751cb2c9162268ef6643c79b6b890a3019167b77d032cf2e5fe5cfbd737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:53.593 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:53 smithi194 podman[72849]: 2024-04-16 05:46:53.212541811 +0000 UTC m=+0.018150646 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:46:53.593 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:53 smithi194 podman[72849]: 2024-04-16 05:46:53.334534538 +0000 UTC m=+0.140143345 container create 25a4e438002c16d836b477a37f3532b184869cdece77a6717ef6480789724cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:53.889 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:53 smithi194 podman[72849]: 2024-04-16 05:46:53.59552038 +0000 UTC m=+0.401129196 container init 25a4e438002c16d836b477a37f3532b184869cdece77a6717ef6480789724cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-16T05:46:53.889 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:53 smithi194 podman[72849]: 2024-04-16 05:46:53.598733712 +0000 UTC m=+0.404342520 container start 25a4e438002c16d836b477a37f3532b184869cdece77a6717ef6480789724cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-16T05:46:53.889 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:53 smithi194 podman[72849]: 2024-04-16 05:46:53.712569839 +0000 UTC m=+0.518178669 container attach 25a4e438002c16d836b477a37f3532b184869cdece77a6717ef6480789724cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:46:54.155 INFO:journalctl@ceph.osd.1.smithi194.stdout:Apr 16 05:46:53 smithi194 podman[72849]: 2024-04-16 05:46:53.892005127 +0000 UTC m=+0.697613950 container died 25a4e438002c16d836b477a37f3532b184869cdece77a6717ef6480789724cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-16T05:46:54.481 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.1.service' 2024-04-16T05:46:54.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:54.522 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-16T05:46:54.523 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-16T05:46:54.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.2 2024-04-16T05:46:54.919 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:46:54 smithi196 systemd[1]: Stopping Ceph osd.2 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:55.183 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:46:54 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2[41391]: 2024-04-16T05:46:54.920+0000 7f8ae5f70700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:55.183 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:46:54 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2[41391]: 2024-04-16T05:46:54.920+0000 7f8ae5f70700 -1 osd.2 109 *** Got signal Terminated *** 2024-04-16T05:46:55.184 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:46:54 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2[41391]: 2024-04-16T05:46:54.920+0000 7f8ae5f70700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:00.671 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:00 smithi196 podman[72649]: 2024-04-16 05:47:00.373188294 +0000 UTC m=+5.654175455 container died e8eea4c52f6b065c52e3fa32c25ca9f4acac9ac7708ae4b567bf8ca65d68242a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-16T05:47:00.947 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:00 smithi196 podman[72649]: 2024-04-16 05:47:00.764678627 +0000 UTC m=+6.045665782 container cleanup e8eea4c52f6b065c52e3fa32c25ca9f4acac9ac7708ae4b567bf8ca65d68242a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:00.948 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:00 smithi196 bash[72649]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2 2024-04-16T05:47:01.234 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:00 smithi196 podman[72678]: 2024-04-16 05:47:00.949584967 +0000 UTC m=+0.565290462 container remove e8eea4c52f6b065c52e3fa32c25ca9f4acac9ac7708ae4b567bf8ca65d68242a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409) 2024-04-16T05:47:01.642 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:01 smithi196 podman[72767]: 2024-04-16 05:47:01.273815865 +0000 UTC m=+0.017818121 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:01.642 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:01 smithi196 podman[72767]: 2024-04-16 05:47:01.383824931 +0000 UTC m=+0.127827171 container create 09c5bf1b806b14cfddadaf6be97a58bf2aa895cd75ba5ddac261b9bae789fbc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:01.943 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:01 smithi196 podman[72767]: 2024-04-16 05:47:01.644968646 +0000 UTC m=+0.388970901 container init 09c5bf1b806b14cfddadaf6be97a58bf2aa895cd75ba5ddac261b9bae789fbc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:01.943 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:01 smithi196 podman[72767]: 2024-04-16 05:47:01.648401393 +0000 UTC m=+0.392403632 container start 09c5bf1b806b14cfddadaf6be97a58bf2aa895cd75ba5ddac261b9bae789fbc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:01.943 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:01 smithi196 podman[72767]: 2024-04-16 05:47:01.745162476 +0000 UTC m=+0.489164736 container attach 09c5bf1b806b14cfddadaf6be97a58bf2aa895cd75ba5ddac261b9bae789fbc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-16T05:47:02.234 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 16 05:47:01 smithi196 podman[72767]: 2024-04-16 05:47:01.945484031 +0000 UTC m=+0.689486285 container died 09c5bf1b806b14cfddadaf6be97a58bf2aa895cd75ba5ddac261b9bae789fbc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:47:02.611 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.2.service' 2024-04-16T05:47:02.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:02.655 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-16T05:47:02.655 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-16T05:47:02.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.3 2024-04-16T05:47:02.962 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:02 smithi086 systemd[1]: Stopping Ceph osd.3 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:03.217 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:02 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3[51156]: 2024-04-16T05:47:02.963+0000 7fa48f816700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:03.217 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:02 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3[51156]: 2024-04-16T05:47:02.963+0000 7fa48f816700 -1 osd.3 109 *** Got signal Terminated *** 2024-04-16T05:47:03.218 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:02 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3[51156]: 2024-04-16T05:47:02.963+0000 7fa48f816700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:05.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:05.096+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:05.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:04.948+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:06.005 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:05 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:05.745+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:06.234 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:06.128+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:06.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:05.946+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:06.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:05.905+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:06.467 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:06 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3[51156]: 2024-04-16T05:47:06.007+0000 7fa48be0a700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:41.262152+0000 front 2024-04-16T05:46:41.262236+0000 (oldest deadline 2024-04-16T05:47:05.962271+0000) 2024-04-16T05:47:07.038 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:06 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:06.721+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:07.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:06.872+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:07.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:07.110+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:07.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:06.921+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:07.467 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:07 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3[51156]: 2024-04-16T05:47:07.039+0000 7fa48be0a700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:41.262152+0000 front 2024-04-16T05:46:41.262236+0000 (oldest deadline 2024-04-16T05:47:05.962271+0000) 2024-04-16T05:47:08.150 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:07.913+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:08.151 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:07.842+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:08.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:07 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:07.771+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:08.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:08.152+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:08.678 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:08 smithi086 podman[101830]: 2024-04-16 05:47:08.39498913 +0000 UTC m=+5.542405981 container died 08ac20212f46bc85be3a14d389f751cdca4c3df6796c99dd30f7b32def2d2142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-04-16T05:47:08.967 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:08 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:08.741+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:08.968 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:08 smithi086 podman[101830]: 2024-04-16 05:47:08.779829495 +0000 UTC m=+5.927246335 container cleanup 08ac20212f46bc85be3a14d389f751cdca4c3df6796c99dd30f7b32def2d2142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-16T05:47:08.968 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:08 smithi086 bash[101830]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3 2024-04-16T05:47:09.110 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:08.816+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:09.111 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:08.902+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:09.393 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:09 smithi086 podman[101849]: 2024-04-16 05:47:09.047766674 +0000 UTC m=+0.649245142 container remove 08ac20212f46bc85be3a14d389f751cdca4c3df6796c99dd30f7b32def2d2142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-16T05:47:09.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:09.112+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:09.717 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:09 smithi086 podman[101942]: 2024-04-16 05:47:09.299266873 +0000 UTC m=+0.017816882 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:09.717 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:09 smithi086 podman[101942]: 2024-04-16 05:47:09.398626795 +0000 UTC m=+0.117176776 container create 88b953bfc2eeef4f636923c24778207efac02df950cfe53eacfbc716604d81d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-16T05:47:10.084 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:09.924+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:10.084 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:09.832+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:10.106 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:09 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:09.760+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:10.106 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:09 smithi086 podman[101942]: 2024-04-16 05:47:09.809702908 +0000 UTC m=+0.528252901 container init 88b953bfc2eeef4f636923c24778207efac02df950cfe53eacfbc716604d81d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:10.106 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:09 smithi086 podman[101942]: 2024-04-16 05:47:09.813039004 +0000 UTC m=+0.531588987 container start 88b953bfc2eeef4f636923c24778207efac02df950cfe53eacfbc716604d81d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:47:10.107 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:09 smithi086 podman[101942]: 2024-04-16 05:47:09.893014772 +0000 UTC m=+0.611564768 container attach 88b953bfc2eeef4f636923c24778207efac02df950cfe53eacfbc716604d81d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-16T05:47:10.379 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 16 05:47:10 smithi086 podman[101942]: 2024-04-16 05:47:10.108834469 +0000 UTC m=+0.827384461 container died 88b953bfc2eeef4f636923c24778207efac02df950cfe53eacfbc716604d81d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-16T05:47:10.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:10 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:10.068+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:10.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:10.086+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:10.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.3.service' 2024-04-16T05:47:10.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:10.839 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-16T05:47:10.839 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-16T05:47:10.840 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.4 2024-04-16T05:47:11.155 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:10 smithi194 systemd[1]: Stopping Ceph osd.4 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:11.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:11.022+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:11.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:10 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:10.762+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:11.234 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:10.855+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:11.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:11.064+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:11.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:10.973+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:11.655 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4[47467]: 2024-04-16T05:47:11.159+0000 7f588851c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:11.655 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4[47467]: 2024-04-16T05:47:11.159+0000 7f588851c700 -1 osd.4 109 *** Got signal Terminated *** 2024-04-16T05:47:11.655 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4[47467]: 2024-04-16T05:47:11.159+0000 7f588851c700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:12.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:11 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:11.787+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:12.234 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:11.818+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:12.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:12 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:12.054+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:12.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:11.971+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:12.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:11 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:11.993+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:12.984 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:12 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:12.827+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:13.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:12 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:12.834+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:13.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:13 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:13.009+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:13.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:13.016+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:13.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:13.004+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:13.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:13 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:13.477+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:14.118 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:13 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:13.834+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:14.234 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:13 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:13.831+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:14.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:14.033+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:14.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:14.026+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:14.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:14 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:14.048+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:14.905 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:14 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4[47467]: 2024-04-16T05:47:14.470+0000 7f5880b08700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.158292+0000 front 2024-04-16T05:46:49.158317+0000 (oldest deadline 2024-04-16T05:47:14.458407+0000) 2024-04-16T05:47:14.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:14 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:14.501+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:15.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:14 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:14.788+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:15.234 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:14.838+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:15.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:14.985+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:15.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:14.997+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:15.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:14 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:14.997+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:15.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:15 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:15.033+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:15.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:15 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:15.033+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:15.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:15 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:15.534+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:15.906 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:15 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4[47467]: 2024-04-16T05:47:15.450+0000 7f5880b08700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.158292+0000 front 2024-04-16T05:46:49.158317+0000 (oldest deadline 2024-04-16T05:47:14.458407+0000) 2024-04-16T05:47:16.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:15 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:15.836+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:16.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:15 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:15.777+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:16.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:15 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:15.777+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:16.234 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:16 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:16.009+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:16.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:15 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:15.880+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:16.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:15 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:15.880+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.129554+0000 front 2024-04-16T05:46:50.129640+0000 (oldest deadline 2024-04-16T05:47:15.429440+0000) 2024-04-16T05:47:16.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:16 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:16.012+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:16.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:16 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:16.012+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:16.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:16 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:16.022+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:16.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:16 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:16.022+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:16.855 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:16 smithi194 podman[72979]: 2024-04-16 05:47:16.65304505 +0000 UTC m=+5.612642109 container died 8c3fd860cfd097f2ea95098223c13fdbe9b9f714da34c93f5195de8b2adc5748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:16.855 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:16 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:16.576+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:17.155 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:17 smithi194 podman[72979]: 2024-04-16 05:47:17.035236449 +0000 UTC m=+5.994833491 container cleanup 8c3fd860cfd097f2ea95098223c13fdbe9b9f714da34c93f5195de8b2adc5748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240409, ceph=True) 2024-04-16T05:47:17.155 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:17 smithi194 bash[72979]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4 2024-04-16T05:47:17.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:17 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:17.067+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:17.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:17 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:17.067+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:17.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:16.837+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:17.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:16.798+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:17.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:16 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:16.798+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:17.234 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:17 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:17.028+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:17.234 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:16 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:16.904+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:17.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:16 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:16.904+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.129554+0000 front 2024-04-16T05:46:50.129640+0000 (oldest deadline 2024-04-16T05:47:15.429440+0000) 2024-04-16T05:47:17.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:17 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:17.000+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:17.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:17 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:17.000+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:17.643 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:17 smithi194 podman[72996]: 2024-04-16 05:47:17.303263205 +0000 UTC m=+0.645089211 container remove 8c3fd860cfd097f2ea95098223c13fdbe9b9f714da34c93f5195de8b2adc5748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4, org.label-schema.build-date=20240409, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:17.643 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:17 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:17.543+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:17.905 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:17 smithi194 podman[73088]: 2024-04-16 05:47:17.548054049 +0000 UTC m=+0.017758200 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:17.905 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:17 smithi194 podman[73088]: 2024-04-16 05:47:17.645773217 +0000 UTC m=+0.115477361 container create 52fd3e2ced6d7ab2e7c78c7682b414fe9f32cb214de09dc0b4851ef91552640e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:47:18.210 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:17 smithi194 podman[73088]: 2024-04-16 05:47:17.915334229 +0000 UTC m=+0.385038384 container init 52fd3e2ced6d7ab2e7c78c7682b414fe9f32cb214de09dc0b4851ef91552640e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-16T05:47:18.210 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:17 smithi194 podman[73088]: 2024-04-16 05:47:17.91863692 +0000 UTC m=+0.388341067 container start 52fd3e2ced6d7ab2e7c78c7682b414fe9f32cb214de09dc0b4851ef91552640e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-16T05:47:18.210 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:17 smithi194 podman[73088]: 2024-04-16 05:47:17.998170724 +0000 UTC m=+0.467874876 container attach 52fd3e2ced6d7ab2e7c78c7682b414fe9f32cb214de09dc0b4851ef91552640e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-16T05:47:18.210 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:18 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:18.083+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:18.211 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:18 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:18.083+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:18.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:17.839+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:18.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:17.847+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:18.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:17 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:17.847+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:18.234 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:18 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:18.063+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:18.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:17 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:17.938+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:18.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:17 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:17.938+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.129554+0000 front 2024-04-16T05:46:50.129640+0000 (oldest deadline 2024-04-16T05:47:15.429440+0000) 2024-04-16T05:47:18.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:18 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:18.050+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:18.236 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:18 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:18.050+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:18.557 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 16 05:47:18 smithi194 podman[73088]: 2024-04-16 05:47:18.212061707 +0000 UTC m=+0.681765852 container died 52fd3e2ced6d7ab2e7c78c7682b414fe9f32cb214de09dc0b4851ef91552640e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:18.768 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.4.service' 2024-04-16T05:47:18.808 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:18 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:18.558+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:18.810 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:18.810 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-16T05:47:18.811 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-16T05:47:18.811 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.5 2024-04-16T05:47:19.134 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:19 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:19.025+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:19.134 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:19 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:19.025+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:19.134 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:19 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:19.075+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:19.134 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:18 smithi196 systemd[1]: Stopping Ceph osd.5 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:19.135 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:18 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:18.950+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:19.135 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:18 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:18.950+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.129554+0000 front 2024-04-16T05:46:50.129640+0000 (oldest deadline 2024-04-16T05:47:15.429440+0000) 2024-04-16T05:47:19.135 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:19 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:19.135+0000 7f7152453700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:19.135 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:19 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:19.135+0000 7f7152453700 -1 osd.5 109 *** Got signal Terminated *** 2024-04-16T05:47:19.135 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:19 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:19.135+0000 7f7152453700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:19.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:19 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:19.044+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:19.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:19 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:19.044+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:19.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:18 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:18.841+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:19.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:18 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:18.870+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:19.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:18 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:18.870+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:19.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:19 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:19.559+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:20.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:19.865+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:20.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:19.917+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:20.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:19.917+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:20.234 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:20 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:20.031+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:20.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:19 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:19.934+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:20.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:19 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:19.934+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.129554+0000 front 2024-04-16T05:46:50.129640+0000 (oldest deadline 2024-04-16T05:47:15.429440+0000) 2024-04-16T05:47:20.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:20 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:20.010+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:20.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:20 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:20.010+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:20.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:20 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:20.034+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:20.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:20 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:20.034+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:20.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:20 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:20.576+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:21.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:20 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:20.877+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:21.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:20 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:20.944+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:21.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:20 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:20.944+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:21.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:20 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:20.944+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:21.234 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:21 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:21.013+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:21.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:20 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:20.921+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:21.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:20 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:20.921+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.129554+0000 front 2024-04-16T05:46:50.129640+0000 (oldest deadline 2024-04-16T05:47:15.429440+0000) 2024-04-16T05:47:21.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:21 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:21.017+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:21.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:21 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:21.017+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:21.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:21 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:21.017+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:21.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:21 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:21.033+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:21.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:21 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:21.033+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:21.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:21 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:21.545+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:21.906 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:21 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:21.545+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:22.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:21 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:21.876+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:22.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:21 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:21.986+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:22.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:21 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:21.986+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:22.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:21 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:21.986+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:22.234 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:22 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:22.006+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:22.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:21 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:21.945+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:22.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:21 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:21.945+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.129554+0000 front 2024-04-16T05:46:50.129640+0000 (oldest deadline 2024-04-16T05:47:15.429440+0000) 2024-04-16T05:47:22.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:21 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:21.945+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:56.530408+0000 front 2024-04-16T05:46:56.530386+0000 (oldest deadline 2024-04-16T05:47:21.830511+0000) 2024-04-16T05:47:22.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:22 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:22.064+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:22.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:22 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:22.064+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:22.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:22 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:22.064+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:22.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:22 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:22.077+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:22.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:22 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:22.077+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:22.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:22 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:22.585+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:22.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:22 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:22.585+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:23.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:22 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:22.830+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:23.217 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:22 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:22.969+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:23.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:22 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:22.969+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:23.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:22 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:22.969+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:23.234 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:23 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:23.056+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:23.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:23 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:23.056+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:23.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:22 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:22.941+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:23.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:22 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:22.941+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.129554+0000 front 2024-04-16T05:46:50.129640+0000 (oldest deadline 2024-04-16T05:47:15.429440+0000) 2024-04-16T05:47:23.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:22 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:22.941+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:56.530408+0000 front 2024-04-16T05:46:56.530386+0000 (oldest deadline 2024-04-16T05:47:21.830511+0000) 2024-04-16T05:47:23.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:23 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:23.091+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:23.236 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:23 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:23.091+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:23.236 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:23 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:23.091+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:23.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:23 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:23.085+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:23.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:23 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:23.085+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:23.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:23 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:23.562+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:23.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:23 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:23.562+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:24.170 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:23 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:23.874+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:24.171 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:23 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:23.961+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:24.171 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:23 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:23.961+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:24.171 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:23 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:23.961+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:24.234 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:23 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:23.939+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.228069+0000 front 2024-04-16T05:46:43.228087+0000 (oldest deadline 2024-04-16T05:47:05.527956+0000) 2024-04-16T05:47:24.234 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:23 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:23.939+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.129554+0000 front 2024-04-16T05:46:50.129640+0000 (oldest deadline 2024-04-16T05:47:15.429440+0000) 2024-04-16T05:47:24.235 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:23 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5[48086]: 2024-04-16T05:47:23.939+0000 7f714aa3f700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:56.530408+0000 front 2024-04-16T05:46:56.530386+0000 (oldest deadline 2024-04-16T05:47:21.830511+0000) 2024-04-16T05:47:24.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:24 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:24.039+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:24.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:24 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:24.039+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:24.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:24 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:24.095+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:24.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:24 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:24.095+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:24.236 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:24 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:24.095+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:24.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:24 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:24.132+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:24.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:24 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:24.132+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:24.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:24 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:24.132+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:24.734 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:24 smithi196 podman[72872]: 2024-04-16 05:47:24.566790832 +0000 UTC m=+5.555764271 container died 8e0b4421f552ba4c355c9132885b45b0a9f6ae4a8f0bd2cef65301e83922c30d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T05:47:24.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:24 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:24.515+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:24.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:24 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:24.515+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:25.060 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:24 smithi196 podman[72872]: 2024-04-16 05:47:24.894236019 +0000 UTC m=+5.883209451 container cleanup 8e0b4421f552ba4c355c9132885b45b0a9f6ae4a8f0bd2cef65301e83922c30d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:47:25.061 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:24 smithi196 bash[72872]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5 2024-04-16T05:47:25.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:24 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:24.852+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:25.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:24 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:24.852+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.544693+0000 front 2024-04-16T05:46:59.544665+0000 (oldest deadline 2024-04-16T05:47:24.844873+0000) 2024-04-16T05:47:25.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:25 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:25.009+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:25.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:25 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:25.009+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:25.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:25 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:25.009+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:25.377 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:25 smithi196 podman[72894]: 2024-04-16 05:47:25.112439658 +0000 UTC m=+0.548432233 container remove 8e0b4421f552ba4c355c9132885b45b0a9f6ae4a8f0bd2cef65301e83922c30d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:25.377 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:25 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:25.077+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:25.377 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:25 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:25.077+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:25.377 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:25 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:25.062+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:25.378 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:25 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:25.062+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:25.378 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:25 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:25.062+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:25.556 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:25 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:25.163+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:25.556 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:25 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:25.163+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:25.556 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:25 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:25.163+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:25.711 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:25 smithi196 podman[72990]: 2024-04-16 05:47:25.465816712 +0000 UTC m=+0.017807099 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:25.711 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:25 smithi196 podman[72990]: 2024-04-16 05:47:25.588580742 +0000 UTC m=+0.140571121 container create 044ffd675061a9417aef7bebfc433081de443cb0484387541e8a98d3598f4dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-16T05:47:25.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:25 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:25.558+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:25.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:25 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:25.558+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:25.984 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:25 smithi196 podman[72990]: 2024-04-16 05:47:25.841742706 +0000 UTC m=+0.393733103 container init 044ffd675061a9417aef7bebfc433081de443cb0484387541e8a98d3598f4dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-16T05:47:25.985 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:25 smithi196 podman[72990]: 2024-04-16 05:47:25.845265534 +0000 UTC m=+0.397255922 container start 044ffd675061a9417aef7bebfc433081de443cb0484387541e8a98d3598f4dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:25.985 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:25 smithi196 podman[72990]: 2024-04-16 05:47:25.933978325 +0000 UTC m=+0.485968734 container attach 044ffd675061a9417aef7bebfc433081de443cb0484387541e8a98d3598f4dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD) 2024-04-16T05:47:26.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:25 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:25.870+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:26.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:25 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:25.870+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.544693+0000 front 2024-04-16T05:46:59.544665+0000 (oldest deadline 2024-04-16T05:47:24.844873+0000) 2024-04-16T05:47:26.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:25 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:25.996+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:26.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:25 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:25.996+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:26.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:25 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:25.996+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:26.434 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:26.095+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:26.434 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:26.096+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:26.434 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 16 05:47:26 smithi196 podman[72990]: 2024-04-16 05:47:26.134032797 +0000 UTC m=+0.686023180 container died 044ffd675061a9417aef7bebfc433081de443cb0484387541e8a98d3598f4dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS) 2024-04-16T05:47:26.435 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:26.097+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:26.435 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:26.097+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:26.435 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:26 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:26.097+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:26.559 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:26.169+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:26.559 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:26.169+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:26.559 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:26.169+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:26.747 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.5.service' 2024-04-16T05:47:26.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:26.789 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-16T05:47:26.789 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-16T05:47:26.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.6 2024-04-16T05:47:26.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:26.561+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:26.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:26 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:26.561+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:27.114 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:26 smithi086 systemd[1]: Stopping Ceph osd.6 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:27.114 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:26.894+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:27.114 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:26.894+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.544693+0000 front 2024-04-16T05:46:59.544665+0000 (oldest deadline 2024-04-16T05:47:24.844873+0000) 2024-04-16T05:47:27.115 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:27 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:27.115+0000 7f3311f48700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:27.115 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:27 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:27.115+0000 7f3311f48700 -1 osd.6 109 *** Got signal Terminated *** 2024-04-16T05:47:27.115 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:26.971+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:27.115 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:26.971+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:27.115 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:26 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:26.971+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:27.467 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:27 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:27.115+0000 7f3311f48700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:27.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:27 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:27.126+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:27.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:27 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:27.126+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:27.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:27 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:27.142+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:27.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:27 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:27.142+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:27.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:27 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:27.142+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:27.550 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:27 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:27.160+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:27.550 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:27 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:27.160+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:27.551 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:27 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:27.160+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:27.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:27 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:27.552+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:27.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:27 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:27.552+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:28.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:27 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:27.849+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:28.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:27 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:27.849+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.544693+0000 front 2024-04-16T05:46:59.544665+0000 (oldest deadline 2024-04-16T05:47:24.844873+0000) 2024-04-16T05:47:28.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:28 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:28.010+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:28.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:28 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:28.010+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:28.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:28 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:28.010+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:28.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:28 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:28.139+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:28.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:28 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:28.139+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:28.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:28 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:28.172+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:28.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:28 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:28.172+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:28.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:28 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:28.172+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:28.524 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:28 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:28.206+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:28.524 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:28 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:28.206+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:28.524 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:28 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:28.206+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:28.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:28 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:28.526+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:28.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:28 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:28.526+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:29.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:28 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:28.837+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:29.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:28 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:28.837+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.544693+0000 front 2024-04-16T05:46:59.544665+0000 (oldest deadline 2024-04-16T05:47:24.844873+0000) 2024-04-16T05:47:29.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:29.039+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:29.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:29.039+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:29.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:29.039+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:29.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:29 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:29.166+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:29.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:29 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:29.166+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:29.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:29 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:29.178+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:29.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:29 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:29.178+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:29.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:29 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:29.178+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:29.546 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:29 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:29.206+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:29.547 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:29 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:29.206+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:29.547 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:29 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:29.206+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:29.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:29 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:29.548+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:29.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:29 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:29.548+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:30.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:29.839+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:30.218 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:29.839+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.544693+0000 front 2024-04-16T05:46:59.544665+0000 (oldest deadline 2024-04-16T05:47:24.844873+0000) 2024-04-16T05:47:30.218 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:29 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:29.839+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:04.845332+0000 front 2024-04-16T05:47:04.845356+0000 (oldest deadline 2024-04-16T05:47:28.945507+0000) 2024-04-16T05:47:30.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:30 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:30.066+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:30.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:30 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:30.066+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:30.219 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:30 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:30.066+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:30.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:30 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:30.191+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:30.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:30 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:30.191+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:30.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:30 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:30.148+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:30.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:30 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:30.148+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:30.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:30 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:30.148+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:30.513 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:30 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:30.215+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:30.513 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:30 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:30.215+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:30.513 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:30 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:30.215+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:30.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:30 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:30.514+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:30.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:30 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:30.514+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:31.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:30 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:30.887+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:31.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:30 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:30.887+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.544693+0000 front 2024-04-16T05:46:59.544665+0000 (oldest deadline 2024-04-16T05:47:24.844873+0000) 2024-04-16T05:47:31.218 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:30 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:30.887+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:04.845332+0000 front 2024-04-16T05:47:04.845356+0000 (oldest deadline 2024-04-16T05:47:28.945507+0000) 2024-04-16T05:47:31.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:31 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:31.048+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:31.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:31 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:31.048+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:31.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:31 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:31.048+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:31.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:31 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:31.185+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:31.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:31 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:31.185+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:31.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:31 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:31.176+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:31.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:31 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:31.176+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:31.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:31 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:31.176+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:31.531 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:31 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:31.228+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:31.531 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:31 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:31.228+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:31.531 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:31 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:31.228+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:31.531 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:31 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:31.228+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:31.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:31 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:31.533+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:31.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:31 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:31.533+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:32.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:31 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:31.884+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:50.743617+0000 front 2024-04-16T05:46:50.743641+0000 (oldest deadline 2024-04-16T05:47:15.443490+0000) 2024-04-16T05:47:32.217 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:31 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:31.884+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.544693+0000 front 2024-04-16T05:46:59.544665+0000 (oldest deadline 2024-04-16T05:47:24.844873+0000) 2024-04-16T05:47:32.218 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:31 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6[57732]: 2024-04-16T05:47:31.884+0000 7f330e53c700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:04.845332+0000 front 2024-04-16T05:47:04.845356+0000 (oldest deadline 2024-04-16T05:47:28.945507+0000) 2024-04-16T05:47:32.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:32 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:32.016+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:32.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:32 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:32.016+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:32.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:32 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:32.016+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:32.218 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:32 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:32.016+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:32.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:32 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:32.136+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:32.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:32 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:32.136+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:32.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:32 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:32.136+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:32.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:32 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:32.128+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:32.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:32 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:32.128+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:32.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:32 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:32.128+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:32.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:32 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:32.128+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:32.488 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:32.192+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:32.488 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:32.192+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:32.489 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:32.192+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:32.489 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:32.192+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:32.781 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:32 smithi086 podman[102060]: 2024-04-16 05:47:32.500671627 +0000 UTC m=+5.510886088 container died ddc58f48ce652d944f038e188c133bc2c86cd5b5084c90e6485339a25cb6e066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:32.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:32.491+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:32.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:32.491+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:32.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:32.491+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:03.620320+0000 front 2024-04-16T05:47:03.620286+0000 (oldest deadline 2024-04-16T05:47:31.820711+0000) 2024-04-16T05:47:33.057 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:32 smithi086 podman[102060]: 2024-04-16 05:47:32.882845381 +0000 UTC m=+5.893059834 container cleanup ddc58f48ce652d944f038e188c133bc2c86cd5b5084c90e6485339a25cb6e066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-16T05:47:33.057 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:32 smithi086 bash[102060]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6 2024-04-16T05:47:33.437 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:33 smithi086 podman[102080]: 2024-04-16 05:47:33.142456511 +0000 UTC m=+0.634442296 container remove ddc58f48ce652d944f038e188c133bc2c86cd5b5084c90e6485339a25cb6e066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:47:33.437 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:33 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:33.058+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:33.437 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:33 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:33.058+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:33.437 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:33 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:33.058+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:33.438 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:33 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:33.058+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:33.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:33 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:33.098+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:33.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:33 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:33.098+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:33.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:33 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:33.098+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:33.486 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:33 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:33.128+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:33.486 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:33 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:33.128+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:33.486 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:33 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:33.128+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:33.486 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:33 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:33.128+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:33.536 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:33 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:33.168+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:33.536 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:33 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:33.168+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:33.536 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:33 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:33.168+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:33.536 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:33 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:33.168+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:33.710 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:33 smithi086 podman[102171]: 2024-04-16 05:47:33.342454989 +0000 UTC m=+0.017078274 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:33.710 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:33 smithi086 podman[102171]: 2024-04-16 05:47:33.484886475 +0000 UTC m=+0.159509759 container create acd664ba1da5adef55b8769ea690adf5caaa5af478aea427ec0dae1a2a8b0f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:47:33.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:33 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:33.538+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:33.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:33 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:33.538+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:33.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:33 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:33.538+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:03.620320+0000 front 2024-04-16T05:47:03.620286+0000 (oldest deadline 2024-04-16T05:47:31.820711+0000) 2024-04-16T05:47:33.967 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:33 smithi086 podman[102171]: 2024-04-16 05:47:33.712699382 +0000 UTC m=+0.387322677 container init acd664ba1da5adef55b8769ea690adf5caaa5af478aea427ec0dae1a2a8b0f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-16T05:47:33.968 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:33 smithi086 podman[102171]: 2024-04-16 05:47:33.716470685 +0000 UTC m=+0.391093980 container start acd664ba1da5adef55b8769ea690adf5caaa5af478aea427ec0dae1a2a8b0f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:33.968 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:33 smithi086 podman[102171]: 2024-04-16 05:47:33.812484077 +0000 UTC m=+0.487107382 container attach acd664ba1da5adef55b8769ea690adf5caaa5af478aea427ec0dae1a2a8b0f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-16T05:47:34.355 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 16 05:47:34 smithi086 podman[102171]: 2024-04-16 05:47:34.013246425 +0000 UTC m=+0.687869713 container died acd664ba1da5adef55b8769ea690adf5caaa5af478aea427ec0dae1a2a8b0f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409) 2024-04-16T05:47:34.355 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:34 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:34.098+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:34.355 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:34 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:34.098+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:34.356 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:34 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:34.098+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:34.356 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:34 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:34.098+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:34.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:34 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:34.079+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:34.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:34 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:34.079+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:34.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:34 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:34.079+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:34.486 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:34 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:34.113+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:34.486 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:34 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:34.113+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:34.486 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:34 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:34.113+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:34.486 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:34 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:34.113+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:34.574 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:34 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:34.203+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:34.574 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:34 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:34.203+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:34.574 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:34 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:34.203+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:34.575 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:34 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:34.203+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:34.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.6.service' 2024-04-16T05:47:34.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:34.719 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-16T05:47:34.719 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-16T05:47:34.719 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.7 2024-04-16T05:47:34.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:34 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:34.576+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:34.906 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:34 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:34.576+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:34.906 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:34 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:34.576+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:03.620320+0000 front 2024-04-16T05:47:03.620286+0000 (oldest deadline 2024-04-16T05:47:31.820711+0000) 2024-04-16T05:47:34.906 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:34 smithi194 systemd[1]: Stopping Ceph osd.7 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:35.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:35.158+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:35.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:35.158+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:35.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:35.158+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:35.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:35.158+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:35.406 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:35.028+0000 7f4766265700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:35.406 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:35.028+0000 7f4766265700 -1 osd.7 109 *** Got signal Terminated *** 2024-04-16T05:47:35.407 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:35.028+0000 7f4766265700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:35.447 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:35 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:35.115+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:35.447 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:35 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:35.115+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:35.447 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:35 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:35.115+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:35.447 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:35 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:35.105+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:35.447 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:35 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:35.105+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:35.447 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:35 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:35.105+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:35.448 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:35 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:35.105+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:35.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:35 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:35.050+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:35.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:35 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:35.050+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:35.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:35 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:35.050+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:35.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:35 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:35.050+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:35.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:35.604+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:35.906 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:35.604+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:35.906 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:35.604+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:03.620320+0000 front 2024-04-16T05:47:03.620286+0000 (oldest deadline 2024-04-16T05:47:31.820711+0000) 2024-04-16T05:47:36.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:36 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:36.109+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:36.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:36 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:36.109+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:36.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:36 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:36.109+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:36.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:36 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:36.109+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:36.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:36 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:36.060+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:36.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:36 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:36.060+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:36.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:36 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:36.060+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:36.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:36 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:36.060+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:36.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:36 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:36.125+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:36.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:36 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:36.125+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:36.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:36 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:36.125+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:36.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:36 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:36.125+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:36.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:36 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:36.101+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:36.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:36 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:36.101+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:36.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:36 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:36.101+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:36.677 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:36 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:36.566+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:36.677 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:36 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:36.566+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:36.678 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:36 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:36.566+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:03.620320+0000 front 2024-04-16T05:47:03.620286+0000 (oldest deadline 2024-04-16T05:47:31.820711+0000) 2024-04-16T05:47:37.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:37.063+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:37.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:37.063+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:37.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:37.063+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:37.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:37.064+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:37.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:37 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:37.021+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:37.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:37 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:37.021+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:37.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:37 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:37.021+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:37.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:37 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:37.021+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:37.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:37 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:37.062+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:37.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:37 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:37.062+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:37.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:37 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:37.062+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:37.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:37 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:37.121+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:37.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:37 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:37.121+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:37.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:37 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:37.121+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:37.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:37 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:37.121+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:37.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:37.582+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:37.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:37.582+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:37.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:37 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:37.582+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:03.620320+0000 front 2024-04-16T05:47:03.620286+0000 (oldest deadline 2024-04-16T05:47:31.820711+0000) 2024-04-16T05:47:38.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:38.032+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:38.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:38.032+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:38.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:38.032+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:38.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:38.032+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:38.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:38 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:38.018+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:38.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:38 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:38.018+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:38.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:38 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:38.018+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:38.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:38 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:38.018+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:38.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:38 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:38.127+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:38.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:38 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:38.127+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:38.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:38 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:38.127+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:38.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:38 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:38.127+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:38.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:38 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:38.017+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:38.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:38 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:38.017+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:38.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:38 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:38.017+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:38.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:38 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:38.017+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:38.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:38.613+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:38.905 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:38.613+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:38.906 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:38.613+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:03.620320+0000 front 2024-04-16T05:47:03.620286+0000 (oldest deadline 2024-04-16T05:47:31.820711+0000) 2024-04-16T05:47:39.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:38.986+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:39.405 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:38.986+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:39.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:38.986+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:39.406 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:38 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:38.986+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:39.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:39 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:39.052+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:39.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:39 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:39.052+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:39.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:39 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:39.052+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:39.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:39 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:39.052+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:39.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:39 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:39.052+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:39.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:39 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:39.152+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:39.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:39 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:39.152+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:39.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:39 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:39.152+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:39.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:39 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:39.152+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:39.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:39 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:39.000+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:39.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:39 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:39.000+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:39.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:39 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:39.000+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:39.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:39 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:39.000+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:39.906 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:39 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:39.642+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.517324+0000 front 2024-04-16T05:46:51.517384+0000 (oldest deadline 2024-04-16T05:47:13.217475+0000) 2024-04-16T05:47:39.907 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:39 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:39.642+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:59.118608+0000 front 2024-04-16T05:46:59.118565+0000 (oldest deadline 2024-04-16T05:47:21.418511+0000) 2024-04-16T05:47:39.907 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:39 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7[54484]: 2024-04-16T05:47:39.642+0000 7f4762859700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:03.620320+0000 front 2024-04-16T05:47:03.620286+0000 (oldest deadline 2024-04-16T05:47:31.820711+0000) 2024-04-16T05:47:40.376 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:39 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:39.942+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:40.376 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:39 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:39.942+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:40.376 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:39 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:39.942+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:40.376 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:39 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:39.942+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:40.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:40 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:40.068+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:40.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:40 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:40.068+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:40.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:40 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:40.068+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:40.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:40 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:40.068+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:40.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:40 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:40.068+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:40.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:40 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:40.103+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:40.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:40 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:40.103+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:40.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:40 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:40.103+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:40.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:40 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:40.103+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:40.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:40 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:40.006+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:40.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:40 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:40.006+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:40.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:40 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:40.006+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:40.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:40 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:40.006+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:40.655 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:40 smithi194 podman[73191]: 2024-04-16 05:47:40.378409684 +0000 UTC m=+5.466793136 container died 640253ab46abfa50c71e7afaf24d262f3198c34170d10d96aea80608a8125ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2) 2024-04-16T05:47:41.052 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:40 smithi194 podman[73191]: 2024-04-16 05:47:40.761915545 +0000 UTC m=+5.850298958 container cleanup 640253ab46abfa50c71e7afaf24d262f3198c34170d10d96aea80608a8125ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409) 2024-04-16T05:47:41.053 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:40 smithi194 bash[73191]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7 2024-04-16T05:47:41.053 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:40 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:40.914+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:41.053 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:40 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:40.914+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:41.053 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:40 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:40.914+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:41.053 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:40 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:40.914+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:41.370 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:41 smithi194 podman[73220]: 2024-04-16 05:47:41.054966321 +0000 UTC m=+0.669422352 container remove 640253ab46abfa50c71e7afaf24d262f3198c34170d10d96aea80608a8125ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-16T05:47:41.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:41 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:41.065+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:41.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:41 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:41.065+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:41.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:41 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:41.065+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:41.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:41 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:41.065+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:41.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:41 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:41.065+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:41.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:41 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:41.092+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:41.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:41 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:41.092+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:41.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:41 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:41.092+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:41.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:41 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:41.092+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:41.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:41 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:41.019+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:41.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:41 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:41.019+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:41.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:41 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:41.019+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:41.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:41 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:41.019+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:41.622 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:41 smithi194 podman[73307]: 2024-04-16 05:47:41.275582462 +0000 UTC m=+0.017389850 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:41.622 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:41 smithi194 podman[73307]: 2024-04-16 05:47:41.389345418 +0000 UTC m=+0.131152800 container create 71ad7b0d128fdb24a584f364924514f71ecfc85544c15b20daeb94c55a95bfe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:41.623 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:41 smithi194 podman[73307]: 2024-04-16 05:47:41.624929426 +0000 UTC m=+0.366736813 container init 71ad7b0d128fdb24a584f364924514f71ecfc85544c15b20daeb94c55a95bfe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-16T05:47:41.882 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:41 smithi194 podman[73307]: 2024-04-16 05:47:41.628445752 +0000 UTC m=+0.370253147 container start 71ad7b0d128fdb24a584f364924514f71ecfc85544c15b20daeb94c55a95bfe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:41.883 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:41 smithi194 podman[73307]: 2024-04-16 05:47:41.699982619 +0000 UTC m=+0.441790015 container attach 71ad7b0d128fdb24a584f364924514f71ecfc85544c15b20daeb94c55a95bfe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-16T05:47:42.155 INFO:journalctl@ceph.osd.7.smithi194.stdout:Apr 16 05:47:41 smithi194 podman[73307]: 2024-04-16 05:47:41.917334125 +0000 UTC m=+0.659141524 container died 71ad7b0d128fdb24a584f364924514f71ecfc85544c15b20daeb94c55a95bfe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-16T05:47:42.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:41 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:41.884+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:42.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:41 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:41.884+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:42.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:41 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:41.884+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:42.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:41 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:41.884+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:42.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:41 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:41.884+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:42.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:42 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:42.059+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:42.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:42 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:42.059+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:42.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:42 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:42.059+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:42.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:42 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:42.059+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:42.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:42 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:42.059+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:42.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:42.099+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:42.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:42.099+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:42.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:42.099+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:42.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:42.099+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:42.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:42.006+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:42.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:42.006+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:42.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:42.006+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:42.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:42.006+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:42.558 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.7.service' 2024-04-16T05:47:42.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:42.601 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-16T05:47:42.602 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-16T05:47:42.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.8 2024-04-16T05:47:42.912 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:42 smithi196 systemd[1]: Stopping Ceph osd.8 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:43.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:42 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:42.865+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:43.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:42 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:42.865+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:43.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:42 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:42.865+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:43.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:42 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:42.865+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:43.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:42 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:42.865+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:43.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:43 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:43.052+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:43.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:43 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:43.052+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:43.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:43 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:43.052+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:43.235 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:43 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:43.052+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:43.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:42.914+0000 7f3ab3b65700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:43.235 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:42.914+0000 7f3ab3b65700 -1 osd.8 109 *** Got signal Terminated *** 2024-04-16T05:47:43.236 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:42 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:42.914+0000 7f3ab3b65700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:43.236 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:43 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:43.144+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:43.236 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:43 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:43.144+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:43.236 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:43 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:43.144+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:43.236 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:43 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:43.144+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:43.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:43 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:43.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:43.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:43 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:43.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:43.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:43 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:43.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:43.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:43 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:43.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:43.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:43 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:43.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:44.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:43 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:43.855+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:44.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:43 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:43.855+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:44.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:43 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:43.855+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:44.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:43 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:43.855+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:44.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:43 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:43.855+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:44.369 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:44 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:44.028+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:44.369 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:44 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:44.028+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:44.369 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:44 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:44.028+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:44.369 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:44 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:44.028+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:44.369 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:44 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:44.028+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:44.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:44 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:44.134+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:44.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:44 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:44.134+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:44.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:44 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:44.134+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:44.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:44 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:44.134+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:44.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:44 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:44.084+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:44.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:44 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:44.084+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:44.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:44 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:44.084+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:44.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:44 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:44.084+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:45.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:44 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:44.877+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:45.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:44 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:44.877+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:45.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:44 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:44.877+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:45.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:44 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:44.877+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:45.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:44 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:44.877+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:45.382 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:45 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:45.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:45.382 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:45 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:45.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:45.383 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:45 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:45.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:45.383 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:45 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:45.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:45.383 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:45 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:45.043+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:45.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:45.179+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:45.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:45.179+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:45.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:45.179+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:45.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:45.179+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:45.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:45.082+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:45.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:45.082+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:45.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:45.082+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:45.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:45 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:45.083+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:46.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:45 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:45.864+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:46.157 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:45 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:45.864+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:46.157 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:45 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:45.864+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:46.157 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:45 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:45.864+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:46.157 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:45 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:45.864+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:46.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:46.089+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:46.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:46.089+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:46.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:46.089+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:46.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:46.089+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:46.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:46 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:46.089+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:46.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:46 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:46.206+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:46.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:46 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:46.206+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:46.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:46 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:46.206+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:46.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:46 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:46.206+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:46.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:46 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:46.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:46.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:46 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:46.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:46.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:46 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:46.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:46.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:46 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:46.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:46.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:46 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:46.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:47.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:46 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:46.821+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:47.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:46 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:46.821+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:47.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:46 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:46.821+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:47.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:46 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:46.821+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:47.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:46 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:46.821+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:47.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:46 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:46.821+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:47.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:47 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:47.138+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:47.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:47 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:47.138+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:47.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:47 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:47.138+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:47.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:47 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:47.138+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:47.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:47 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:47.138+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:47.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:47.197+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.695654+0000 front 2024-04-16T05:46:39.695752+0000 (oldest deadline 2024-04-16T05:47:04.395557+0000) 2024-04-16T05:47:47.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:47.197+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.696642+0000 front 2024-04-16T05:46:49.696783+0000 (oldest deadline 2024-04-16T05:47:14.396723+0000) 2024-04-16T05:47:47.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:47.197+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:54.897700+0000 front 2024-04-16T05:46:54.897775+0000 (oldest deadline 2024-04-16T05:47:20.797907+0000) 2024-04-16T05:47:47.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:47.197+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.599630+0000 front 2024-04-16T05:47:06.599583+0000 (oldest deadline 2024-04-16T05:47:31.899465+0000) 2024-04-16T05:47:47.485 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8[54681]: 2024-04-16T05:47:47.197+0000 7f3aac151700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:23.601016+0000 front 2024-04-16T05:47:23.601072+0000 (oldest deadline 2024-04-16T05:47:47.101151+0000) 2024-04-16T05:47:47.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:47.048+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:47.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:47.048+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:47.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:47.048+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:47.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:47.048+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:47.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:47.048+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:48.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:47 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:47.778+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:48.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:47 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:47.778+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:48.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:47 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:47.778+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:48.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:47 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:47.778+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:48.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:47 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:47.778+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:48.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:47 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:47.778+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:48.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:48.172+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:48.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:48.172+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:48.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:48.172+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:48.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:48.172+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:48.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:48 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:48.172+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:48.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:48 smithi196 podman[73094]: 2024-04-16 05:47:48.28811258 +0000 UTC m=+5.494331768 container died 0fedd8cc998bdc062b2832e64a35da330d5d15cf0a1da9964356f2abd79d538e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-04-16T05:47:48.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:48 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:48.068+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:48.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:48 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:48.068+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:48.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:48 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:48.068+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:48.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:48 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:48.068+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:48.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:48 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:48.068+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:48.912 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:48 smithi196 podman[73094]: 2024-04-16 05:47:48.664195608 +0000 UTC m=+5.870414760 container cleanup 0fedd8cc998bdc062b2832e64a35da330d5d15cf0a1da9964356f2abd79d538e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409) 2024-04-16T05:47:48.913 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:48 smithi196 bash[73094]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8 2024-04-16T05:47:49.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:48 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:48.782+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:49.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:48 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:48.782+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:49.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:48 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:48.782+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:49.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:48 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:48.782+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:49.157 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:48 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:48.782+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:49.157 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:48 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:48.782+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:49.221 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:48 smithi196 podman[73111]: 2024-04-16 05:47:48.915383425 +0000 UTC m=+0.623167983 container remove 0fedd8cc998bdc062b2832e64a35da330d5d15cf0a1da9964356f2abd79d538e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:49.221 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:49 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:49.035+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:49.221 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:49 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:49.035+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:49.221 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:49 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:49.035+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:49.222 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:49 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:49.035+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:49.222 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:49 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:49.035+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:49.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:49 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:49.199+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:49.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:49 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:49.199+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:49.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:49 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:49.199+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:49.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:49 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:49.199+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:49.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:49 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:49.199+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:49.468 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:49 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:49.199+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.665919+0000 front 2024-04-16T05:47:20.665801+0000 (oldest deadline 2024-04-16T05:47:48.866042+0000) 2024-04-16T05:47:49.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:49 smithi196 podman[73204]: 2024-04-16 05:47:49.12676579 +0000 UTC m=+0.017356344 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:49.484 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:49 smithi196 podman[73204]: 2024-04-16 05:47:49.224466204 +0000 UTC m=+0.115056766 container create dd6c68ab74f89f9bdd20d0868d38a50e3616b4f1ddf57b4d5b69b2a3ced17f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T05:47:49.782 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:49 smithi196 podman[73204]: 2024-04-16 05:47:49.493641249 +0000 UTC m=+0.384231808 container init dd6c68ab74f89f9bdd20d0868d38a50e3616b4f1ddf57b4d5b69b2a3ced17f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-16T05:47:49.782 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:49 smithi196 podman[73204]: 2024-04-16 05:47:49.497023342 +0000 UTC m=+0.387613912 container start dd6c68ab74f89f9bdd20d0868d38a50e3616b4f1ddf57b4d5b69b2a3ced17f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True) 2024-04-16T05:47:49.782 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:49 smithi196 podman[73204]: 2024-04-16 05:47:49.576968815 +0000 UTC m=+0.467559386 container attach dd6c68ab74f89f9bdd20d0868d38a50e3616b4f1ddf57b4d5b69b2a3ced17f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-16T05:47:50.082 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 16 05:47:49 smithi196 podman[73204]: 2024-04-16 05:47:49.784813635 +0000 UTC m=+0.675404209 container died dd6c68ab74f89f9bdd20d0868d38a50e3616b4f1ddf57b4d5b69b2a3ced17f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:50.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:49 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:49.808+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:50.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:49 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:49.808+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:50.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:49 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:49.808+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:50.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:49 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:49.808+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:50.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:49 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:49.808+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:50.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:49 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:49.808+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:50.370 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:50 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:50.083+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:50.370 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:50 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:50.083+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:50.370 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:50 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:50.083+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:50.370 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:50 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:50.083+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:50.370 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:50 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:50.083+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:50.390 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.8.service' 2024-04-16T05:47:50.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:50.432 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-16T05:47:50.432 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-16T05:47:50.432 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.9 2024-04-16T05:47:50.464 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:50.179+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:50.464 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:50.179+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:50.464 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:50.179+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:50.465 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:50.179+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:50.465 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:50.179+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:50.465 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:50.179+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.665919+0000 front 2024-04-16T05:47:20.665801+0000 (oldest deadline 2024-04-16T05:47:48.866042+0000) 2024-04-16T05:47:50.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 systemd[1]: Stopping Ceph osd.9 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:51.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:50 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:50.795+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:51.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:50 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:50.795+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:51.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:50 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:50.795+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:51.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:50 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:50.795+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:51.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:50 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:50.795+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:51.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:50 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:50.795+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:51.198 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:50.752+0000 7f14097e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:51.198 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:50.752+0000 7f14097e5700 -1 osd.9 109 *** Got signal Terminated *** 2024-04-16T05:47:51.198 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:50 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:50.752+0000 7f14097e5700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:51.198 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:51 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:51.200+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:51.198 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:51 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:51.200+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:51.199 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:51 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:51.200+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:51.199 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:51 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:51.200+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:51.199 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:51 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:51.200+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:51.199 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:51 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:51.200+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.665919+0000 front 2024-04-16T05:47:20.665801+0000 (oldest deadline 2024-04-16T05:47:48.866042+0000) 2024-04-16T05:47:51.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:51 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:51.061+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:51.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:51 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:51.061+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:51.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:51 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:51.061+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:51.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:51 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:51.061+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:51.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:51 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:51.061+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:52.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:51 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:51.814+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:52.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:51 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:51.814+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:52.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:51 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:51.814+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:52.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:51 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:51.814+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:52.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:51 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:51.814+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:52.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:51 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:51.814+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:52.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:52 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:52.074+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:52.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:52 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:52.074+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:52.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:52 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:52.074+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:52.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:52 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:52.074+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:52.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:52 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:52.074+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:52.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:52 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:52.240+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:52.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:52 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:52.240+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:52.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:52 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:52.240+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:52.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:52 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:52.240+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:52.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:52 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:52.240+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:52.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:52 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:52.240+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.665919+0000 front 2024-04-16T05:47:20.665801+0000 (oldest deadline 2024-04-16T05:47:48.866042+0000) 2024-04-16T05:47:52.905 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:52 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:52.798+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:52.905 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:52 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:52.798+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:52.905 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:52 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:52.798+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:52.906 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:52 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:52.798+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:52.906 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:52 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:52.798+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:52.906 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:52 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:52.798+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:53.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:53 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:53.112+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:53.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:53 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:53.112+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:53.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:53 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:53.112+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:53.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:53 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:53.112+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:53.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:53 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:53.112+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:53.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:53 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:53.254+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:53.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:53 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:53.254+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:53.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:53 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:53.254+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:53.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:53 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:53.254+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:53.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:53 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:53.254+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:53.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:53 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:53.254+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.665919+0000 front 2024-04-16T05:47:20.665801+0000 (oldest deadline 2024-04-16T05:47:48.866042+0000) 2024-04-16T05:47:54.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:53 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:53.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:54.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:53 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:53.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:54.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:53 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:53.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:54.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:53 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:53.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:54.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:53 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:53.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:54.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:53 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:53.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:54.386 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:54 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:54.102+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:54.386 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:54 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:54.102+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:54.386 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:54 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:54.102+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:54.386 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:54 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:54.102+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:54.386 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:54 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:54.102+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:54.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:54 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:54.275+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:54.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:54 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:54.275+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:54.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:54 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:54.275+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:54.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:54 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:54.275+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:54.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:54 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:54.275+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:54.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:54 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:54.275+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.665919+0000 front 2024-04-16T05:47:20.665801+0000 (oldest deadline 2024-04-16T05:47:48.866042+0000) 2024-04-16T05:47:55.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:54 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:54.800+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:55.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:54 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:54.800+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:55.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:54 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:54.800+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:55.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:54 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:54.800+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:55.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:54 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:54.800+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:55.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:54 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:54.801+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:55.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:55 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:55.114+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:55.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:55 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:55.114+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:55.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:55 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:55.114+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:55.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:55 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:55.114+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:55.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:55 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:55.114+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:55.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:55 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:55.114+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:47:55.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:55 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:55.301+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.257526+0000 front 2024-04-16T05:46:43.257546+0000 (oldest deadline 2024-04-16T05:47:04.957754+0000) 2024-04-16T05:47:55.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:55 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:55.301+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:51.160332+0000 front 2024-04-16T05:46:51.160346+0000 (oldest deadline 2024-04-16T05:47:15.160745+0000) 2024-04-16T05:47:55.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:55 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:55.301+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.662179+0000 front 2024-04-16T05:46:58.662122+0000 (oldest deadline 2024-04-16T05:47:20.862425+0000) 2024-04-16T05:47:55.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:55 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:55.301+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:05.463693+0000 front 2024-04-16T05:47:05.463746+0000 (oldest deadline 2024-04-16T05:47:31.363847+0000) 2024-04-16T05:47:55.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:55 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:55.301+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:14.864967+0000 front 2024-04-16T05:47:14.864903+0000 (oldest deadline 2024-04-16T05:47:38.364635+0000) 2024-04-16T05:47:55.718 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:55 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9[64324]: 2024-04-16T05:47:55.301+0000 7f1401dd1700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.665919+0000 front 2024-04-16T05:47:20.665801+0000 (oldest deadline 2024-04-16T05:47:48.866042+0000) 2024-04-16T05:47:56.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:55 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:55.825+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:56.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:55 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:55.825+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:56.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:55 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:55.825+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:56.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:55 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:55.825+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:56.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:55 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:55.825+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:56.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:55 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:55.825+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:56.400 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:56 smithi086 podman[102288]: 2024-04-16 05:47:56.144477887 +0000 UTC m=+5.511709978 container died 183b558bb29655b8e786461dd0bfba3c75c141ee0aa7afc3b340c4fad6d5a4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-16T05:47:56.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:56 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:56.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:56.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:56 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:56.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:56.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:56 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:56.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:56.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:56 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:56.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:56.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:56 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:56.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:56.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:56 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:56.092+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:47:56.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:56 smithi086 podman[102288]: 2024-04-16 05:47:56.501448539 +0000 UTC m=+5.868680634 container cleanup 183b558bb29655b8e786461dd0bfba3c75c141ee0aa7afc3b340c4fad6d5a4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True) 2024-04-16T05:47:56.717 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:56 smithi086 bash[102288]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9 2024-04-16T05:47:57.117 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:56 smithi086 podman[102306]: 2024-04-16 05:47:56.769384144 +0000 UTC m=+0.619262788 container remove 183b558bb29655b8e786461dd0bfba3c75c141ee0aa7afc3b340c4fad6d5a4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:57.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:56 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:56.871+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:57.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:56 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:56.871+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:57.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:56 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:56.871+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:57.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:56 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:56.871+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:57.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:56 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:56.871+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:57.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:56 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:56.871+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:57.370 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:57 smithi086 podman[102398]: 2024-04-16 05:47:57.022411387 +0000 UTC m=+0.017325730 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:57.370 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:57 smithi086 podman[102398]: 2024-04-16 05:47:57.128330936 +0000 UTC m=+0.123245279 container create 8e40edae378072920883d58ec9ac751eb12a664a9fd19f4dad7e2073d52298c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-16T05:47:57.371 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:57 smithi086 podman[102398]: 2024-04-16 05:47:57.372945727 +0000 UTC m=+0.367860079 container init 8e40edae378072920883d58ec9ac751eb12a664a9fd19f4dad7e2073d52298c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-16T05:47:57.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:57 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:57.125+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:57.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:57 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:57.125+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:57.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:57 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:57.125+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:57.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:57 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:57.125+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:57.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:57 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:57.125+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:57.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:57 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:57.125+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:47:57.668 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:57 smithi086 podman[102398]: 2024-04-16 05:47:57.376861117 +0000 UTC m=+0.371775466 container start 8e40edae378072920883d58ec9ac751eb12a664a9fd19f4dad7e2073d52298c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:47:57.668 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:57 smithi086 podman[102398]: 2024-04-16 05:47:57.472676555 +0000 UTC m=+0.467590904 container attach 8e40edae378072920883d58ec9ac751eb12a664a9fd19f4dad7e2073d52298c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, ceph=True) 2024-04-16T05:47:57.964 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 16 05:47:57 smithi086 podman[102398]: 2024-04-16 05:47:57.671325153 +0000 UTC m=+0.666239495 container died 8e40edae378072920883d58ec9ac751eb12a664a9fd19f4dad7e2073d52298c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-16T05:47:58.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:57 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:57.830+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:58.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:57 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:57.830+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:58.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:57 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:57.830+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:58.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:57 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:57.830+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:58.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:57 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:57.830+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:58.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:57 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:57.830+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:58.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:57 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:57.830+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:31.158576+0000 front 2024-04-16T05:47:31.158699+0000 (oldest deadline 2024-04-16T05:47:57.058433+0000) 2024-04-16T05:47:58.268 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.9.service' 2024-04-16T05:47:58.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:58.352 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-16T05:47:58.352 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-16T05:47:58.353 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.10 2024-04-16T05:47:58.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:58 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:58.141+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:58.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:58 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:58.141+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:58.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:58 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:58.141+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:58.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:58 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:58.141+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:58.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:58 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:58.141+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:58.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:58 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:58.141+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:47:58.655 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 systemd[1]: Stopping Ceph osd.10 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:59.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.688+0000 7f0a290f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:59.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.688+0000 7f0a290f1700 -1 osd.10 109 *** Got signal Terminated *** 2024-04-16T05:47:59.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.688+0000 7f0a290f1700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:59.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.820+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:47:59.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.820+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:47:59.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.820+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:47:59.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.820+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:47:59.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.820+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:47:59.157 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.820+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:47:59.157 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:58 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:58.820+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:31.158576+0000 front 2024-04-16T05:47:31.158699+0000 (oldest deadline 2024-04-16T05:47:57.058433+0000) 2024-04-16T05:47:59.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:59 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:59.091+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:47:59.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:59 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:59.091+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:47:59.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:59 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:59.091+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:47:59.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:59 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:59.091+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:47:59.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:59 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:59.091+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:47:59.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:47:59 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:47:59.091+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:00.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:59 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:59.791+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:48:00.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:59 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:59.791+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:48:00.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:59 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:59.791+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:48:00.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:59 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:59.791+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:48:00.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:59 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:59.791+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:48:00.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:59 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:59.791+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:48:00.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:47:59 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:47:59.791+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:31.158576+0000 front 2024-04-16T05:47:31.158699+0000 (oldest deadline 2024-04-16T05:47:57.058433+0000) 2024-04-16T05:48:00.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:00 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:00.139+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:00.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:00 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:00.139+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:00.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:00 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:00.139+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:00.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:00 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:00.139+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:00.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:00 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:00.139+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:00.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:00 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:00.139+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:01.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:00 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:00.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:48:01.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:00 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:00.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:48:01.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:00 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:00.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:48:01.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:00 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:00.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:48:01.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:00 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:00.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:48:01.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:00 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:00.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:48:01.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:00 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:00.792+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:31.158576+0000 front 2024-04-16T05:47:31.158699+0000 (oldest deadline 2024-04-16T05:47:57.058433+0000) 2024-04-16T05:48:01.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:01 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:01.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:01.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:01 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:01.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:01.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:01 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:01.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:01.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:01 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:01.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:01.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:01 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:01.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:01.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:01 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:01.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:02.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:01 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:01.768+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:48:02.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:01 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:01.768+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:48:02.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:01 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:01.768+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:48:02.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:01 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:01.768+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:48:02.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:01 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:01.768+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:48:02.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:01 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:01.768+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:48:02.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:01 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:01.768+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:31.158576+0000 front 2024-04-16T05:47:31.158699+0000 (oldest deadline 2024-04-16T05:47:57.058433+0000) 2024-04-16T05:48:02.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:02.159+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:02.493 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:02.159+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:02.494 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:02.159+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:02.494 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:02.159+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:02.494 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:02.159+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:02.494 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:02.159+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:02.494 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:02 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:02.159+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:03.155 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:02 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:02.776+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:43.752343+0000 front 2024-04-16T05:46:43.752244+0000 (oldest deadline 2024-04-16T05:47:09.052394+0000) 2024-04-16T05:48:03.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:02 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:02.776+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6804 osd.1 since back 2024-04-16T05:46:49.052846+0000 front 2024-04-16T05:46:49.052903+0000 (oldest deadline 2024-04-16T05:47:14.952959+0000) 2024-04-16T05:48:03.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:02 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:02.776+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:58.354649+0000 front 2024-04-16T05:46:58.354635+0000 (oldest deadline 2024-04-16T05:47:23.654431+0000) 2024-04-16T05:48:03.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:02 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:02.776+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:06.456086+0000 front 2024-04-16T05:47:06.456002+0000 (oldest deadline 2024-04-16T05:47:31.155973+0000) 2024-04-16T05:48:03.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:02 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:02.776+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:15.856950+0000 front 2024-04-16T05:47:15.856969+0000 (oldest deadline 2024-04-16T05:47:41.756967+0000) 2024-04-16T05:48:03.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:02 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:02.776+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:21.757683+0000 front 2024-04-16T05:47:21.757546+0000 (oldest deadline 2024-04-16T05:47:46.457336+0000) 2024-04-16T05:48:03.156 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:02 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10[61547]: 2024-04-16T05:48:02.776+0000 7f0a256e5700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:31.158576+0000 front 2024-04-16T05:47:31.158699+0000 (oldest deadline 2024-04-16T05:47:57.058433+0000) 2024-04-16T05:48:03.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:03.175+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:03.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:03.175+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:03.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:03.175+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:03.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:03.175+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:03.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:03.175+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:03.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:03.175+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:03.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:03 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:03.175+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:04.311 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:04 smithi194 podman[73413]: 2024-04-16 05:48:04.054679357 +0000 UTC m=+5.500877424 container died 785b0be0941679c7853c092f269cd04309224cc4ba63db3bf021d3c06724168c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-16T05:48:04.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:04.144+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:04.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:04.144+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:04.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:04.144+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:04.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:04.144+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:04.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:04.144+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:04.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:04.144+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:04.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:04 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:04.144+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:04.620 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:04 smithi194 podman[73413]: 2024-04-16 05:48:04.388081012 +0000 UTC m=+5.834279066 container cleanup 785b0be0941679c7853c092f269cd04309224cc4ba63db3bf021d3c06724168c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:48:04.620 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:04 smithi194 bash[73413]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10 2024-04-16T05:48:04.905 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:04 smithi194 podman[73430]: 2024-04-16 05:48:04.62253803 +0000 UTC m=+0.561064389 container remove 785b0be0941679c7853c092f269cd04309224cc4ba63db3bf021d3c06724168c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-16T05:48:05.356 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:05 smithi194 podman[73523]: 2024-04-16 05:48:05.031459793 +0000 UTC m=+0.102323461 container create a158518694dbd9e845929294e9a23c2fa13bf67e8ef427d25ab5fb049d66b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-16T05:48:05.356 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:05 smithi194 podman[73523]: 2024-04-16 05:48:04.946839276 +0000 UTC m=+0.017702964 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:48:05.356 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:05 smithi194 podman[73523]: 2024-04-16 05:48:05.275347893 +0000 UTC m=+0.346211572 container init a158518694dbd9e845929294e9a23c2fa13bf67e8ef427d25ab5fb049d66b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-16T05:48:05.356 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:05 smithi194 podman[73523]: 2024-04-16 05:48:05.278465266 +0000 UTC m=+0.349328935 container start a158518694dbd9e845929294e9a23c2fa13bf67e8ef427d25ab5fb049d66b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_CLEAN=True) 2024-04-16T05:48:05.357 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:05 smithi194 podman[73523]: 2024-04-16 05:48:05.358514349 +0000 UTC m=+0.429378030 container attach a158518694dbd9e845929294e9a23c2fa13bf67e8ef427d25ab5fb049d66b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:48:05.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:05.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:05.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:05.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:05.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:05.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:05.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:05.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:05.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:05.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:05.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:05.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:05.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:05 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:05.117+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:05.655 INFO:journalctl@ceph.osd.10.smithi194.stdout:Apr 16 05:48:05 smithi194 podman[73523]: 2024-04-16 05:48:05.567431129 +0000 UTC m=+0.638294828 container died a158518694dbd9e845929294e9a23c2fa13bf67e8ef427d25ab5fb049d66b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:48:06.161 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.10.service' 2024-04-16T05:48:06.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:06.201 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-16T05:48:06.201 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-16T05:48:06.202 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.11 2024-04-16T05:48:06.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.151+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:06.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.151+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:06.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.151+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:06.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.151+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:06.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.151+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:06.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.151+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:06.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.151+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:06.486 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 systemd[1]: Stopping Ceph osd.11 for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:06.985 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.490+0000 7fe85cf6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:06.985 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.490+0000 7fe85cf6b700 -1 osd.11 109 *** Got signal Terminated *** 2024-04-16T05:48:06.985 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:06 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:06.490+0000 7fe85cf6b700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:48:07.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:07.191+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:07.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:07.191+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:07.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:07.191+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:07.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:07.191+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:07.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:07.191+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:07.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:07.191+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:07.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:07 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:07.191+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:08.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:08.198+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:08.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:08.198+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:08.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:08.198+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:08.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:08.198+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:08.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:08.198+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:08.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:08.198+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:08.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:08 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:08.198+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:09.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:09.209+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:09.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:09.209+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:09.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:09.209+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:09.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:09.209+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:09.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:09.209+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:09.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:09.209+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:09.485 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:09 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:09.209+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:10.734 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:10.257+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:10.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:10.257+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:10.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:10.257+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:10.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:10.257+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:10.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:10.257+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:10.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:10.257+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:10.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:10 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:10.257+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:11.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:11.291+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-16T05:46:39.879781+0000 front 2024-04-16T05:46:39.879914+0000 (oldest deadline 2024-04-16T05:47:04.579583+0000) 2024-04-16T05:48:11.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:11.291+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-16T05:46:57.382235+0000 front 2024-04-16T05:46:57.382210+0000 (oldest deadline 2024-04-16T05:47:22.082338+0000) 2024-04-16T05:48:11.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:11.291+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-16T05:47:07.284146+0000 front 2024-04-16T05:47:07.284196+0000 (oldest deadline 2024-04-16T05:47:31.984005+0000) 2024-04-16T05:48:11.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:11.291+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6812 osd.4 since back 2024-04-16T05:47:11.984455+0000 front 2024-04-16T05:47:11.984511+0000 (oldest deadline 2024-04-16T05:47:37.284517+0000) 2024-04-16T05:48:11.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:11.291+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-16T05:47:20.085725+0000 front 2024-04-16T05:47:20.085764+0000 (oldest deadline 2024-04-16T05:47:45.985751+0000) 2024-04-16T05:48:11.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:11.291+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-16T05:47:30.687172+0000 front 2024-04-16T05:47:30.687081+0000 (oldest deadline 2024-04-16T05:47:54.786805+0000) 2024-04-16T05:48:11.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:11 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11[61306]: 2024-04-16T05:48:11.291+0000 7fe85955f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.194:6820 osd.7 since back 2024-04-16T05:47:37.687644+0000 front 2024-04-16T05:47:37.687733+0000 (oldest deadline 2024-04-16T05:48:01.187540+0000) 2024-04-16T05:48:12.113 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:11 smithi196 podman[73324]: 2024-04-16 05:48:11.827019544 +0000 UTC m=+5.424278474 container died fc8b6cf50cae8ab05eec0d22105feb00133d864ebbd1e9c0c0ca76217c3a8d17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-16T05:48:12.463 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:12 smithi196 podman[73324]: 2024-04-16 05:48:12.214861178 +0000 UTC m=+5.812120107 container cleanup fc8b6cf50cae8ab05eec0d22105feb00133d864ebbd1e9c0c0ca76217c3a8d17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:48:12.464 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:12 smithi196 bash[73324]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11 2024-04-16T05:48:12.734 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:12 smithi196 podman[73341]: 2024-04-16 05:48:12.465883813 +0000 UTC m=+0.630778250 container remove fc8b6cf50cae8ab05eec0d22105feb00133d864ebbd1e9c0c0ca76217c3a8d17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:48:13.075 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:12 smithi196 podman[73430]: 2024-04-16 05:48:12.783649104 +0000 UTC m=+0.113814723 container create eb0cada7984f46a9b6b7eb838ed2820ae114ffe16f04865b1f5c27dae81efd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:48:13.075 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:12 smithi196 podman[73430]: 2024-04-16 05:48:12.68763458 +0000 UTC m=+0.017800215 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:48:13.075 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:13 smithi196 podman[73430]: 2024-04-16 05:48:13.002723281 +0000 UTC m=+0.332888924 container init eb0cada7984f46a9b6b7eb838ed2820ae114ffe16f04865b1f5c27dae81efd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-16T05:48:13.075 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:13 smithi196 podman[73430]: 2024-04-16 05:48:13.006122883 +0000 UTC m=+0.336288511 container start eb0cada7984f46a9b6b7eb838ed2820ae114ffe16f04865b1f5c27dae81efd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:48:13.076 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:13 smithi196 podman[73430]: 2024-04-16 05:48:13.077328524 +0000 UTC m=+0.407494166 container attach eb0cada7984f46a9b6b7eb838ed2820ae114ffe16f04865b1f5c27dae81efd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:48:13.484 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 16 05:48:13 smithi196 podman[73430]: 2024-04-16 05:48:13.2915949 +0000 UTC m=+0.621760544 container died eb0cada7984f46a9b6b7eb838ed2820ae114ffe16f04865b1f5c27dae81efd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, ceph=True) 2024-04-16T05:48:13.896 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@osd.11.service' 2024-04-16T05:48:13.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:13.939 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-16T05:48:13.939 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-16T05:48:13.940 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.a 2024-04-16T05:48:14.235 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:13 smithi086 systemd[1]: Stopping Ceph mds.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:14.500 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:14 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a[71159]: 2024-04-16T05:48:14.236+0000 7fc808404700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:14.501 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:14 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a[71159]: 2024-04-16T05:48:14.236+0000 7fc808404700 -1 mds.a *** got signal Terminated *** 2024-04-16T05:48:18.492 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:18 smithi086 podman[102516]: 2024-04-16 05:48:18.243656687 +0000 UTC m=+4.104803277 container died d1005bb154d0e1aee3b4bc9e8334033be98cc81f4531e4f00ccc599e8160f356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:48:18.752 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:18 smithi086 podman[102516]: 2024-04-16 05:48:18.569057502 +0000 UTC m=+4.430204098 container cleanup d1005bb154d0e1aee3b4bc9e8334033be98cc81f4531e4f00ccc599e8160f356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409) 2024-04-16T05:48:18.752 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:18 smithi086 bash[102516]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a 2024-04-16T05:48:18.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.a.service' 2024-04-16T05:48:19.023 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:18 smithi086 podman[102542]: 2024-04-16 05:48:18.75368396 +0000 UTC m=+0.502688452 container remove d1005bb154d0e1aee3b4bc9e8334033be98cc81f4531e4f00ccc599e8160f356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409) 2024-04-16T05:48:19.023 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:18 smithi086 systemd[1]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.a.service: Deactivated successfully. 2024-04-16T05:48:19.023 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:18 smithi086 systemd[1]: Stopped Ceph mds.a for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:48:19.023 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 16 05:48:18 smithi086 systemd[1]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.a.service: Consumed 2.113s CPU time. 2024-04-16T05:48:19.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:19.038 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-16T05:48:19.038 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-16T05:48:19.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.d 2024-04-16T05:48:19.354 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:48:19 smithi086 systemd[1]: Stopping Ceph mds.d for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:19.717 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:48:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-d[70210]: 2024-04-16T05:48:19.355+0000 7f0c5749e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:19.717 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:48:19 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-d[70210]: 2024-04-16T05:48:19.355+0000 7f0c5749e700 -1 mds.d *** got signal Terminated *** 2024-04-16T05:48:22.737 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:48:22 smithi086 podman[102649]: 2024-04-16 05:48:22.4707479 +0000 UTC m=+3.212914019 container died 695332d77798afec18ae82bfa13e392a2ae975d067e6b0588865b45e8bfb9581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, ceph=True) 2024-04-16T05:48:23.096 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:48:22 smithi086 podman[102649]: 2024-04-16 05:48:22.831007006 +0000 UTC m=+3.573173124 container cleanup 695332d77798afec18ae82bfa13e392a2ae975d067e6b0588865b45e8bfb9581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:48:23.096 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 16 05:48:22 smithi086 bash[102649]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-d 2024-04-16T05:48:23.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.d.service' 2024-04-16T05:48:23.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:23.280 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-16T05:48:23.280 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-16T05:48:23.280 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.g 2024-04-16T05:48:23.599 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:48:23 smithi086 systemd[1]: Stopping Ceph mds.g for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:23.967 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:48:23 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g[70802]: 2024-04-16T05:48:23.601+0000 7f26ed6b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:23.967 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:48:23 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g[70802]: 2024-04-16T05:48:23.601+0000 7f26ed6b3700 -1 mds.g *** got signal Terminated *** 2024-04-16T05:48:26.967 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:48:26 smithi086 podman[102770]: 2024-04-16 05:48:26.710596559 +0000 UTC m=+3.204718148 container died fde136e54058bfe13cdafaea31a315286802037ccd51f302b52de0fcd0efe054 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-16T05:48:27.258 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:48:27 smithi086 podman[102770]: 2024-04-16 05:48:27.051085746 +0000 UTC m=+3.545207344 container cleanup fde136e54058bfe13cdafaea31a315286802037ccd51f302b52de0fcd0efe054 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5) 2024-04-16T05:48:27.258 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 16 05:48:27 smithi086 bash[102770]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-g 2024-04-16T05:48:27.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.g.service' 2024-04-16T05:48:27.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:27.510 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-16T05:48:27.510 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-16T05:48:27.510 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.j 2024-04-16T05:48:27.828 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:48:27 smithi086 systemd[1]: Stopping Ceph mds.j for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:28.217 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:48:27 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-j[69860]: 2024-04-16T05:48:27.829+0000 7f8a5f3e1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:28.218 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:48:27 smithi086 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-j[69860]: 2024-04-16T05:48:27.829+0000 7f8a5f3e1700 -1 mds.j *** got signal Terminated *** 2024-04-16T05:48:31.594 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:48:31 smithi086 podman[102893]: 2024-04-16 05:48:31.332634889 +0000 UTC m=+3.605061633 container died 77388f01fc17018ca44ec6a7120b2711a28ed3c56d6cb1eb52110ed33cd2528b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-j, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-16T05:48:31.953 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:48:31 smithi086 podman[102893]: 2024-04-16 05:48:31.696349052 +0000 UTC m=+3.968775801 container cleanup 77388f01fc17018ca44ec6a7120b2711a28ed3c56d6cb1eb52110ed33cd2528b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-j, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-16T05:48:31.953 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 16 05:48:31 smithi086 bash[102893]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-j 2024-04-16T05:48:32.108 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.j.service' 2024-04-16T05:48:32.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:32.144 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-16T05:48:32.144 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-16T05:48:32.144 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.b 2024-04-16T05:48:32.527 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:48:32 smithi194 systemd[1]: Stopping Ceph mds.b for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:32.905 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:48:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b[66970]: 2024-04-16T05:48:32.528+0000 7f474c596700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:32.906 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:48:32 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b[66970]: 2024-04-16T05:48:32.528+0000 7f474c596700 -1 mds.b *** got signal Terminated *** 2024-04-16T05:48:35.026 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:48:34 smithi194 podman[73641]: 2024-04-16 05:48:34.72196611 +0000 UTC m=+2.381618354 container died 999ed23679377d906c2f72b0590a38f6f1e2bad04836c47e4682835692f05e6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b, org.label-schema.build-date=20240409, ceph=True, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:48:35.027 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:48:35 smithi194 podman[73641]: 2024-04-16 05:48:35.029182754 +0000 UTC m=+2.688834964 container cleanup 999ed23679377d906c2f72b0590a38f6f1e2bad04836c47e4682835692f05e6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-16T05:48:35.298 INFO:journalctl@ceph.mds.b.smithi194.stdout:Apr 16 05:48:35 smithi194 bash[73641]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-b 2024-04-16T05:48:35.449 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.b.service' 2024-04-16T05:48:35.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:35.486 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-16T05:48:35.486 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-16T05:48:35.486 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.e 2024-04-16T05:48:35.799 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:48:35 smithi194 systemd[1]: Stopping Ceph mds.e for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:36.155 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:48:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e[66340]: 2024-04-16T05:48:35.801+0000 7fedebb36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:36.155 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:48:35 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e[66340]: 2024-04-16T05:48:35.801+0000 7fedebb36700 -1 mds.e *** got signal Terminated *** 2024-04-16T05:48:38.976 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:48:38 smithi194 podman[73762]: 2024-04-16 05:48:38.719683624 +0000 UTC m=+3.017575482 container died 8d1011153208e177348e503e966f41c0909e9acce115bd63966690961a66288d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.build-date=20240409, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:48:39.341 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:48:39 smithi194 podman[73762]: 2024-04-16 05:48:39.068164542 +0000 UTC m=+3.366056400 container cleanup 8d1011153208e177348e503e966f41c0909e9acce115bd63966690961a66288d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-16T05:48:39.342 INFO:journalctl@ceph.mds.e.smithi194.stdout:Apr 16 05:48:39 smithi194 bash[73762]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-e 2024-04-16T05:48:39.481 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.e.service' 2024-04-16T05:48:39.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:39.519 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-16T05:48:39.519 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-16T05:48:39.519 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.h 2024-04-16T05:48:39.905 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:48:39 smithi194 systemd[1]: Stopping Ceph mds.h for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:39.905 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:48:39 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h[66663]: 2024-04-16T05:48:39.855+0000 7fec5624c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:39.906 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:48:39 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h[66663]: 2024-04-16T05:48:39.855+0000 7fec5624c700 -1 mds.h *** got signal Terminated *** 2024-04-16T05:48:43.070 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:48:42 smithi194 podman[73896]: 2024-04-16 05:48:42.720264353 +0000 UTC m=+2.973504199 container died 33598d38e68fd354bea5c4ced1d2bbaaad974db4d538d147fff0539202e2c423 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:48:43.320 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:48:43 smithi194 podman[73896]: 2024-04-16 05:48:43.15509118 +0000 UTC m=+3.408331022 container cleanup 33598d38e68fd354bea5c4ced1d2bbaaad974db4d538d147fff0539202e2c423 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True) 2024-04-16T05:48:43.320 INFO:journalctl@ceph.mds.h.smithi194.stdout:Apr 16 05:48:43 smithi194 bash[73896]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-h 2024-04-16T05:48:43.526 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.h.service' 2024-04-16T05:48:43.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:43.561 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-16T05:48:43.562 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-16T05:48:43.562 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.k 2024-04-16T05:48:43.905 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:48:43 smithi194 systemd[1]: Stopping Ceph mds.k for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:43.906 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:48:43 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k[66019]: 2024-04-16T05:48:43.874+0000 7feedafa6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:43.906 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:48:43 smithi194 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k[66019]: 2024-04-16T05:48:43.874+0000 7feedafa6700 -1 mds.k *** got signal Terminated *** 2024-04-16T05:48:47.047 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:48:46 smithi194 podman[74019]: 2024-04-16 05:48:46.718864538 +0000 UTC m=+2.930414679 container died ba3233fc3bde559ddb32356206cd2e6fb8aef82c0bce83a971a7b5a3771200ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240409) 2024-04-16T05:48:47.047 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:48:47 smithi194 podman[74019]: 2024-04-16 05:48:47.049867655 +0000 UTC m=+3.261417791 container cleanup ba3233fc3bde559ddb32356206cd2e6fb8aef82c0bce83a971a7b5a3771200ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:48:47.405 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:48:47 smithi194 bash[74019]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k 2024-04-16T05:48:47.405 INFO:journalctl@ceph.mds.k.smithi194.stdout:Apr 16 05:48:47 smithi194 podman[74033]: 2024-04-16 05:48:47.225601418 +0000 UTC m=+0.506186724 container remove ba3233fc3bde559ddb32356206cd2e6fb8aef82c0bce83a971a7b5a3771200ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-k, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_CLEAN=True) 2024-04-16T05:48:47.463 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.k.service' 2024-04-16T05:48:47.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:47.497 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-16T05:48:47.497 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-16T05:48:47.498 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.c 2024-04-16T05:48:47.851 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:48:47 smithi196 systemd[1]: Stopping Ceph mds.c for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:48.234 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:48:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c[66724]: 2024-04-16T05:48:47.853+0000 7f8c6f02a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:48.235 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:48:47 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c[66724]: 2024-04-16T05:48:47.853+0000 7f8c6f02a700 -1 mds.c *** got signal Terminated *** 2024-04-16T05:48:50.985 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:48:50 smithi196 podman[73536]: 2024-04-16 05:48:50.715790738 +0000 UTC m=+2.972382450 container died b11b26d5c298d8e95906863d2432d12ba6c6fad53c3c21ce5f95d3eae2b0be85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:48:51.435 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:48:51 smithi196 podman[73536]: 2024-04-16 05:48:51.203251479 +0000 UTC m=+3.459843172 container cleanup b11b26d5c298d8e95906863d2432d12ba6c6fad53c3c21ce5f95d3eae2b0be85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-16T05:48:51.435 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 16 05:48:51 smithi196 bash[73536]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-c 2024-04-16T05:48:51.553 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.c.service' 2024-04-16T05:48:51.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:51.590 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-16T05:48:51.590 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-16T05:48:51.590 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.f 2024-04-16T05:48:51.734 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:48:51 smithi196 systemd[1]: Stopping Ceph mds.f for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:52.234 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:48:51 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f[66419]: 2024-04-16T05:48:51.906+0000 7f90233bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:52.235 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:48:51 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f[66419]: 2024-04-16T05:48:51.906+0000 7f90233bb700 -1 mds.f *** got signal Terminated *** 2024-04-16T05:48:54.963 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:48:54 smithi196 podman[73667]: 2024-04-16 05:48:54.713173567 +0000 UTC m=+2.902373510 container died e9abd2da7dad6884dd596da632ca505790801e8c78df0003dd8d73e2d7ca2d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-16T05:48:55.235 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:48:55 smithi196 podman[73667]: 2024-04-16 05:48:55.048071337 +0000 UTC m=+3.237271276 container cleanup e9abd2da7dad6884dd596da632ca505790801e8c78df0003dd8d73e2d7ca2d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-16T05:48:55.235 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 16 05:48:55 smithi196 bash[73667]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-f 2024-04-16T05:48:55.515 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.f.service' 2024-04-16T05:48:55.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:55.549 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-16T05:48:55.550 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-16T05:48:55.550 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.i 2024-04-16T05:48:55.883 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:48:55 smithi196 systemd[1]: Stopping Ceph mds.i for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:56.234 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:48:55 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i[67032]: 2024-04-16T05:48:55.884+0000 7fde566bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:56.235 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:48:55 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i[67032]: 2024-04-16T05:48:55.884+0000 7fde566bf700 -1 mds.i *** got signal Terminated *** 2024-04-16T05:48:58.984 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:48:58 smithi196 podman[73791]: 2024-04-16 05:48:58.702984765 +0000 UTC m=+2.925219550 container died cdc608bfa682c78d1675be11462c34807cc7015448d02d09617f627b67f658c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:48:59.317 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:48:59 smithi196 podman[73791]: 2024-04-16 05:48:59.11028062 +0000 UTC m=+3.332515404 container cleanup cdc608bfa682c78d1675be11462c34807cc7015448d02d09617f627b67f658c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-16T05:48:59.317 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 16 05:48:59 smithi196 bash[73791]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-i 2024-04-16T05:48:59.431 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.i.service' 2024-04-16T05:48:59.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:59.465 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-16T05:48:59.466 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-16T05:48:59.466 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.l 2024-04-16T05:48:59.734 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:48:59 smithi196 systemd[1]: Stopping Ceph mds.l for 6a4a23f8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:49:00.234 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:48:59 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l[66113]: 2024-04-16T05:48:59.780+0000 7feeec908700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:49:00.235 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:48:59 smithi196 ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l[66113]: 2024-04-16T05:48:59.780+0000 7feeec908700 -1 mds.l *** got signal Terminated *** 2024-04-16T05:49:02.984 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:49:02 smithi196 podman[73912]: 2024-04-16 05:49:02.701829588 +0000 UTC m=+3.015383897 container died 0bb4d70276ee4c71a19361b33c24a28f271378b0bbe341577c1727f70b3393db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:49:03.337 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:49:03 smithi196 podman[73912]: 2024-04-16 05:49:03.072235866 +0000 UTC m=+3.385790155 container cleanup 0bb4d70276ee4c71a19361b33c24a28f271378b0bbe341577c1727f70b3393db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-16T05:49:03.337 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 16 05:49:03 smithi196 bash[73912]: ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968-mds-l 2024-04-16T05:49:03.496 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a4a23f8-fbb1-11ee-bc8f-c7b262605968@mds.l.service' 2024-04-16T05:49:03.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:49:03.530 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-16T05:49:03.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 --force --keep-logs 2024-04-16T05:49:03.725 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:49:06.202 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 --force --keep-logs 2024-04-16T05:49:06.395 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:49:08.875 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 --force --keep-logs 2024-04-16T05:49:09.067 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:49:11.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:49:11.508 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:49:11.543 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:49:11.572 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-16T05:49:11.574 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/remote/smithi086/crash 2024-04-16T05:49:11.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/crash -- . 2024-04-16T05:49:11.695 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T05:49:11.695 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-04-16T05:49:11.697 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/remote/smithi194/crash 2024-04-16T05:49:11.698 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/crash -- . 2024-04-16T05:49:11.759 INFO:teuthology.orchestra.run.smithi194.stderr:tar: /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T05:49:11.759 INFO:teuthology.orchestra.run.smithi194.stderr:tar: Error is not recoverable: exiting now 2024-04-16T05:49:11.761 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/remote/smithi196/crash 2024-04-16T05:49:11.762 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/crash -- . 2024-04-16T05:49:11.798 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T05:49:11.798 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-04-16T05:49:11.799 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-16T05:49:11.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-16T05:49:11.840 INFO:tasks.cephadm:Compressing logs... 2024-04-16T05:49:11.840 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T05:49:11.883 DEBUG:teuthology.orchestra.run.smithi194:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T05:49:11.885 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T05:49:11.911 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T05:49:11.911 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-04-16T05:49:11.911 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.l.log 2024-04-16T05:49:11.911 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log 2024-04-16T05:49:11.926 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.i.log 2024-04-16T05:49:11.926 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.11.log 2024-04-16T05:49:11.926 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.z.log 2024-04-16T05:49:11.926 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.i.log: /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.8.log 2024-04-16T05:49:11.926 INFO:teuthology.orchestra.run.smithi196.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T05:49:11.926 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.5.log 2024-04-16T05:49:11.926 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.8.log: --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.f.log 2024-04-16T05:49:11.926 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log 2024-04-16T05:49:11.927 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log.gz 2024-04-16T05:49:11.927 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log 2024-04-16T05:49:11.927 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log.gz 2024-04-16T05:49:11.927 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.2.log 2024-04-16T05:49:11.927 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log: 87.7% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log.gz 2024-04-16T05:49:11.927 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.c.log 2024-04-16T05:49:11.927 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.2.log: 91.9% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.z.log.gz 2024-04-16T05:49:11.927 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log 2024-04-16T05:49:11.929 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-04-16T05:49:11.929 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-16T05:49:11.929 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.6.log 2024-04-16T05:49:11.929 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.a.log 2024-04-16T05:49:11.929 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log 2024-04-16T05:49:11.929 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.g.log 2024-04-16T05:49:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.j.log 2024-04-16T05:49:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.3.log 2024-04-16T05:49:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.x.log 2024-04-16T05:49:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.0.log 2024-04-16T05:49:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.x.log: 90.2% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log.gz 2024-04-16T05:49:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log 2024-04-16T05:49:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.j.log.gz 2024-04-16T05:49:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log 2024-04-16T05:49:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T05:49:11.931 INFO:teuthology.orchestra.run.smithi086.stderr: 84.2% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log.gz 2024-04-16T05:49:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log 2024-04-16T05:49:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.9.log 2024-04-16T05:49:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log: 87.7% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log.gz 2024-04-16T05:49:11.932 INFO:teuthology.orchestra.run.smithi194.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-16T05:49:11.932 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T05:49:11.932 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.10.log 2024-04-16T05:49:11.932 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log 2024-04-16T05:49:11.932 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.b.log 2024-04-16T05:49:11.932 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.10.log: /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.4.log 2024-04-16T05:49:11.932 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.1.log 2024-04-16T05:49:11.933 INFO:teuthology.orchestra.run.smithi194.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T05:49:11.933 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.k.log 2024-04-16T05:49:11.933 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.h.log 2024-04-16T05:49:11.933 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.7.log 2024-04-16T05:49:11.933 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.e.log 2024-04-16T05:49:11.933 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.7.log: 90.3% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log.gz 2024-04-16T05:49:11.933 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.y.log 2024-04-16T05:49:11.934 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.d.log 2024-04-16T05:49:11.935 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.c.log 2024-04-16T05:49:11.935 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log 2024-04-16T05:49:11.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.9.log: 91.0% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log.gz 2024-04-16T05:49:11.937 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.a.log 2024-04-16T05:49:11.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.d.log.gz 2024-04-16T05:49:11.942 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-16T05:49:11.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-16T05:49:11.954 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log 2024-04-16T05:49:11.954 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log: 91.9% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.y.log.gz 2024-04-16T05:49:11.961 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log: /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log.gz 2024-04-16T05:49:11.962 INFO:teuthology.orchestra.run.smithi194.stderr: 83.9%gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log 2024-04-16T05:49:11.967 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log: -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log.gz 2024-04-16T05:49:11.975 INFO:teuthology.orchestra.run.smithi196.stderr: 92.9% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.i.log.gz 2024-04-16T05:49:11.977 INFO:teuthology.orchestra.run.smithi196.stderr: 94.6% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.l.log.gz 2024-04-16T05:49:11.977 INFO:teuthology.orchestra.run.smithi194.stderr: 87.7% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph.log.gz 2024-04-16T05:49:11.978 INFO:teuthology.orchestra.run.smithi086.stderr: 94.5% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.g.log.gz 2024-04-16T05:49:11.979 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.b.log 2024-04-16T05:49:11.982 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log: 92.7% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.h.log.gz 2024-04-16T05:49:11.986 INFO:teuthology.orchestra.run.smithi196.stderr: 92.8% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.c.log.gz 2024-04-16T05:49:11.987 INFO:teuthology.orchestra.run.smithi196.stderr: 93.0% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.f.log.gz 2024-04-16T05:49:11.988 INFO:teuthology.orchestra.run.smithi194.stderr: 94.4% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.k.log.gz 2024-04-16T05:49:11.994 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log.gz 2024-04-16T05:49:12.000 INFO:teuthology.orchestra.run.smithi086.stderr: 94.4% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.a.log.gz 2024-04-16T05:49:12.008 INFO:teuthology.orchestra.run.smithi194.stderr: 94.0% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.e.log.gz 2024-04-16T05:49:12.217 INFO:teuthology.orchestra.run.smithi194.stderr: 93.1% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.b.log.gz 2024-04-16T05:49:12.239 INFO:teuthology.orchestra.run.smithi196.stderr: 92.0% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.c.log.gz 2024-04-16T05:49:12.289 INFO:teuthology.orchestra.run.smithi086.stderr: 90.3% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.x.log.gz 2024-04-16T05:49:12.333 INFO:teuthology.orchestra.run.smithi194.stderr: 92.0% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.b.log.gz 2024-04-16T05:49:12.510 INFO:teuthology.orchestra.run.smithi196.stderr: 92.8% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.11.log.gz 2024-04-16T05:49:12.696 INFO:teuthology.orchestra.run.smithi086.stderr: 92.2% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.6.log.gz 2024-04-16T05:49:12.743 INFO:teuthology.orchestra.run.smithi194.stderr: 92.4% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.7.log.gz 2024-04-16T05:49:12.812 INFO:teuthology.orchestra.run.smithi086.stderr: 92.3% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.3.log.gz 2024-04-16T05:49:13.053 INFO:teuthology.orchestra.run.smithi194.stderr: 92.4% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.10.log.gz 2024-04-16T05:49:13.076 INFO:teuthology.orchestra.run.smithi086.stderr: 90.6% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.a.log.gz 2024-04-16T05:49:13.102 INFO:teuthology.orchestra.run.smithi194.stderr: 92.9% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.1.log.gz 2024-04-16T05:49:13.110 INFO:teuthology.orchestra.run.smithi196.stderr: 92.9% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.8.log.gz 2024-04-16T05:49:13.145 INFO:teuthology.orchestra.run.smithi086.stderr: 93.1% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.0.log.gz 2024-04-16T05:49:13.257 INFO:teuthology.orchestra.run.smithi194.stderr: 93.0% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.4.log.gz 2024-04-16T05:49:13.259 INFO:teuthology.orchestra.run.smithi194.stderr: 2024-04-16T05:49:13.259 INFO:teuthology.orchestra.run.smithi194.stderr:real 0m1.360s 2024-04-16T05:49:13.259 INFO:teuthology.orchestra.run.smithi194.stderr:user 0m5.190s 2024-04-16T05:49:13.259 INFO:teuthology.orchestra.run.smithi194.stderr:sys 0m0.219s 2024-04-16T05:49:13.282 INFO:teuthology.orchestra.run.smithi196.stderr: 92.8% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.2.log.gz 2024-04-16T05:49:13.498 INFO:teuthology.orchestra.run.smithi196.stderr: 92.9% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.5.log.gz 2024-04-16T05:49:13.500 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-04-16T05:49:13.500 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m1.601s 2024-04-16T05:49:13.500 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m5.106s 2024-04-16T05:49:13.500 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.205s 2024-04-16T05:49:13.559 INFO:teuthology.orchestra.run.smithi086.stderr: 92.7% -- replaced with /var/log/ceph/6a4a23f8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.9.log.gz 2024-04-16T05:49:13.561 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-16T05:49:13.561 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m1.664s 2024-04-16T05:49:13.561 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m6.056s 2024-04-16T05:49:13.561 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.207s 2024-04-16T05:49:13.562 INFO:tasks.cephadm:Archiving logs... 2024-04-16T05:49:13.562 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/remote/smithi086/log 2024-04-16T05:49:13.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T05:49:14.445 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/remote/smithi194/log 2024-04-16T05:49:14.446 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T05:49:15.217 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/remote/smithi196/log 2024-04-16T05:49:15.217 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T05:49:15.980 INFO:tasks.cephadm:Removing cluster... 2024-04-16T05:49:15.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 --force 2024-04-16T05:49:16.173 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:49:16.524 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 --force 2024-04-16T05:49:16.716 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:49:16.973 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 --force 2024-04-16T05:49:17.168 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 6a4a23f8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:49:17.577 INFO:tasks.cephadm:Removing cephadm ... 2024-04-16T05:49:17.577 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T05:49:17.597 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T05:49:17.612 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T05:49:17.627 INFO:tasks.cephadm:Teardown complete 2024-04-16T05:49:17.628 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-16T05:49:17.639 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-16T05:49:17.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T05:49:17.642 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T05:49:17.655 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T05:49:17.767 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-16T05:49:17.778 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-16T05:49:17.778 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:49:17.781 DEBUG:teuthology.orchestra.run.smithi194:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:49:17.783 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:49:17.794 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:49:17.795 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:49:17.796 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:49:17.799 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:49:17.799 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-04-16T05:49:17.799 INFO:teuthology.orchestra.run.smithi194.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +1620us[+1616us] +/- 42ms 2024-04-16T05:49:17.799 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 +321us[ +317us] +/- 71ms 2024-04-16T05:49:17.799 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 114 -2981us[-2985us] +/- 69ms 2024-04-16T05:49:17.800 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:49:17.801 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:49:17.801 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-16T05:49:17.801 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 8 377 181 +1592us[+1596us] +/- 40ms 2024-04-16T05:49:17.801 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 179 +303us[ +303us] +/- 69ms 2024-04-16T05:49:17.801 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -3029us[-3029us] +/- 70ms 2024-04-16T05:49:17.802 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:49:17.802 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:49:17.803 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-04-16T05:49:17.803 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +1629us[+1633us] +/- 42ms 2024-04-16T05:49:17.803 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 +313us[ +317us] +/- 71ms 2024-04-16T05:49:17.803 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -3009us[-3009us] +/- 70ms 2024-04-16T05:49:17.803 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:49:17.803 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-16T05:49:17.816 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-16T05:49:17.816 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-16T05:49:17.857 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T05:49:17.947 DEBUG:teuthology.orchestra.run.smithi194:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T05:49:18.017 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T05:49:18.088 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:49:18.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:49:18.121 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:49:18.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:49:18.151 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:49:18.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:49:18.181 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-16T05:49:18.192 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-16T05:49:18.240 INFO:teuthology.task.internal:Duration was 2111.133322 seconds 2024-04-16T05:49:18.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-16T05:49:18.291 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-16T05:49:18.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T05:49:18.293 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T05:49:18.296 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T05:49:18.336 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:49:18.338 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:49:18.339 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:49:18.766 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-16T05:49:18.766 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-04-16T05:49:18.766 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T05:49:18.787 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi194.front.sepia.ceph.com 2024-04-16T05:49:18.787 DEBUG:teuthology.orchestra.run.smithi194:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T05:49:18.806 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-04-16T05:49:18.807 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T05:49:18.853 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-16T05:49:18.853 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T05:49:18.856 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T05:49:18.858 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T05:49:18.885 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-16T05:49:18.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T05:49:18.925 DEBUG:teuthology.orchestra.run.smithi194:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T05:49:18.927 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T05:49:19.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-16T05:49:19.352 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-16T05:49:19.352 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T05:49:19.381 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T05:49:19.408 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T05:49:19.436 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-16T05:49:19.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:49:19.450 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:49:19.452 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:49:19.474 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-04-16T05:49:19.475 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = core 2024-04-16T05:49:19.501 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-04-16T05:49:19.549 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:49:19.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:49:19.564 DEBUG:teuthology.orchestra.run.smithi194:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:49:19.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:49:19.577 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:49:19.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:49:19.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-16T05:49:19.615 INFO:teuthology.task.internal:Transferring archived files... 2024-04-16T05:49:19.616 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/remote/smithi086 2024-04-16T05:49:19.616 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T05:49:19.662 DEBUG:teuthology.misc:Transferring archived files from smithi194:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/remote/smithi194 2024-04-16T05:49:19.662 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T05:49:19.699 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901/remote/smithi196 2024-04-16T05:49:19.699 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T05:49:19.737 INFO:teuthology.task.internal:Removing archive directory... 2024-04-16T05:49:19.737 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T05:49:19.740 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T05:49:19.742 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T05:49:19.785 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-16T05:49:19.799 INFO:teuthology.task.internal:Not uploading archives. 2024-04-16T05:49:19.799 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-16T05:49:19.824 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-16T05:49:19.824 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T05:49:19.827 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T05:49:19.829 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T05:49:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: 266040 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 05:49 /home/ubuntu/cephtest 2024-04-16T05:49:19.841 INFO:teuthology.orchestra.run.smithi194.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 05:49 /home/ubuntu/cephtest 2024-04-16T05:49:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 05:49 /home/ubuntu/cephtest 2024-04-16T05:49:19.844 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-16T05:49:19.875 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-16T05:49:19.998 INFO:teuthology.nuke:Checking targets against current locks 2024-04-16T05:49:20.022 DEBUG:teuthology.nuke:shortname: smithi086 2024-04-16T05:49:20.023 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T05:49:20.047 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:48.255039', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBELp0o3cUKATNtjsL3TDTiIaIDukpq5xX+mE9ecn5vDiuGVcnvKpXedNoveEOYxOZM33r8t26wnPfue9IWwUo9s='} 2024-04-16T05:49:20.054 DEBUG:teuthology.nuke:shortname: smithi194 2024-04-16T05:49:20.054 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T05:49:20.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:48.256297', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZ5w4slM01MMsoRhPYMQmhAnQjuEpb16pq6SzGNOgkPZ7hZZSOHRzGZhijmx9rovC3iPgWBQWNYHsivIjNC82E='} 2024-04-16T05:49:20.086 DEBUG:teuthology.nuke:shortname: smithi196 2024-04-16T05:49:20.087 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T05:49:20.113 INFO:teuthology.orchestra.console.smithi086:Power off 2024-04-16T05:49:20.113 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T05:49:20.124 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657901', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:48.253682', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCxmS4ZcQk2bk5TqIuT7tppLPsGHAV4W2UyGB3dV1fKQzndrg4QxYtmHeAYV4Zyk5/b1tKcx9J9jcWbX+nPHFRQ='} 2024-04-16T05:49:20.136 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-04-16T05:49:20.136 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:20.154 INFO:teuthology.orchestra.console.smithi194:Power off 2024-04-16T05:49:20.155 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T05:49:20.169 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-16T05:49:20.179 DEBUG:teuthology.orchestra.console.smithi194:power off output: Chassis Power Control: Down/Off 2024-04-16T05:49:20.179 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:20.195 INFO:teuthology.orchestra.console.smithi196:Power off 2024-04-16T05:49:20.195 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T05:49:20.205 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-04-16T05:49:20.219 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-04-16T05:49:20.219 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:20.242 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-16T05:49:24.170 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:24.206 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:24.244 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:24.281 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-16T05:49:24.315 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-04-16T05:49:24.355 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-16T05:49:28.286 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:28.316 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:28.356 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:28.397 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-04-16T05:49:28.426 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-04-16T05:49:28.466 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-16T05:49:28.498 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-04-16T05:49:32.429 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:32.467 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:49:32.542 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is off 2024-04-16T05:49:32.577 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-04-16T05:49:32.643 INFO:teuthology.orchestra.console.smithi194:Power off completed 2024-04-16T05:49:32.678 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-04-16T05:49:32.785 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 2111.1333215236664 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi086 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a87fcff6340baeedd43af126e793f3241b6400ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c8dc2f6780524070b550d5c4b0c989e9 status: fail success: false 2024-04-16T05:49:32.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T05:49:32.855 INFO:teuthology.run:FAIL